title-cascade

In one sentence: a small program that looks at a "job title" text field (the kind you'd see in a spreadsheet of contacts) and answers "is this string actually a job title, and does it hold together?" — not "what job is this," and not "does this person really have this job."

It runs on a laptop CPU in well under a millisecond per string. It is not an LLM, not a chatbot, and does not call any external API.

If you're new to this repo, read this file top to bottom once. It's written so you can trace the logic by hand, on paper, without running any code. Everything technical links out to a deeper doc — you don't need to read those to understand what this project is, only to reproduce or extend it.

---

0. Where to go next — the doc index

If you want to...	Read
Understand what this is and how it works (you're reading it)	this file
Clone the repo and retrain everything from scratch, with every data source and license listed	REPRODUCE.md
Run it on your own CSV of contacts, locally, without uploading anything	docs/HOW-TO.md
A from-scratch, file-by-file walkthrough for tracing the logic by hand, plus what your own (bigger) hardware changes about what to build next	docs/MAC-WALKTHROUGH.md
How an embedding model actually classifies a job title (they are not translators), with a ~70-line runnable end-to-end example and measured cross-lingual numbers	docs/EMBEDDINGS-BY-HAND.md + examples/minimal_validator.py
Why a boolean "valid/invalid" is the wrong API, and the scorecard (X-shaped, international)	docs/SCORECARD.md
See exactly where each lexicon file came from	data/lexicons/SOURCES.md
See how the 618 hand-labeled seed examples were labeled and why (with the labeler's own uncertainty notes)	data/generated/TEACHER_NOTES.md
See the exact goals/gates this project is trying to hit, machine-readable	results/GOALS.json
See a real prompt that was drafted for an external "teacher" model (never actually sent — see §9 below)	rounds/round_00/prompt.md
See how much of a real multilingual taxonomy this system correctly recognizes vs. drops, and why company/person-name matching works the way it does	MULTILINGUAL-AUDIT.md
The plan for a genuinely international approach — why every past attempt failed the same way, what from X actually transfers, a menu of scorers, the eval that must exist first, and a held-out-language ablation of the scorecard's neighbor (74% of its gain is same-language fuzzy lookup) plus a script bug that deletes all non-Latin text	PLAN-V2-INTERNATIONAL.md (read alongside docs/SCORECARD.md — same architecture, different engine, see its §11)
What was actually measured in the international build, phase by phase, with the one open exception stated plainly	RESULTS-V2.md
The phased build plan (ACCEPTED; the board in §3 shows what is DONE) and the gate log of what was independently verified per phase	BUILD-PLAN.md, orchestration/ORCHESTRATION.md
Understand the design this repo is implementing, and why it's shaped this way	01_x_teardown_and_approaches.md (a teardown of a real production content-moderation system, in the parent directory)
Understand the original plan for training a small language model this way (not yet built — see §9)	02_distillation_plan.md (parent directory)

---

1. Why does this exist

Imagine a spreadsheet with 100 million contact rows and a "Job Title" column. Real data like this is messy: some rows say "Chief Revenue Officer", some say "Sales", some say the person's own name (typed into the wrong box), some say "n/a", some are a company name, some are typos, some are jokes ("Chief Vibes Officer"), some are in German or Vietnamese.

If you want to use that column — to filter contacts, to build a model, to target outreach — you first need to know: is this string even a usable job title? Not "which occupation is it" (that's a much harder, separate problem this project deliberately does not attempt — see §4). Just: is it a real title, written by someone describing a real role, as opposed to noise?

That's the whole scope of title-cascade.

2. How it works — traced by hand

The program is a pipeline of cheap checks, each one either deciding the answer outright or passing the string to the next, more expensive check. Nothing here is a neural network doing "vibes"-based reasoning — every step is something you could do with a pen, a few lists of words, and a calculator. That's the whole design philosophy: spend the expensive checks (machine learning) only on the strings the cheap checks can't decide.

your string
    |
    v
STAGE 0  clean it up (lowercase, strip weird characters, spot emails/URLs/phones)
    |
    v
STAGE 1  is it even a plausible string at all?      <- pure rules, no lists needed
    |    (empty? "n/a"? an email? mostly digits? keyboard mash?)
    | no  \
    |      -> REJECT, stop here
    | yes
    v
STAGE 2  check it against word lists: is this a company name? a person's
    |    name? a department? does it contradict itself ("Junior Chief
    |    Executive")? does a known job-title word appear in it at all?
    | clearly-not-a-title \
    |                      -> REJECT, stop here
    | clearly-a-known-title \
    |                        -> ACCEPT, stop here
    | genuinely unclear
    v
STAGE 3  (optional) a small statistical model looks at the string's letter
    |    patterns and nudges the confidence down if it looks suspicious.
    |    It can only make the score worse, never rescue a hard reject.
    v
STAGE 6  turn everything gathered so far into one of five final answers:
         ACCEPT · ACCEPT_FLAGGED · DEMOTE · QUARANTINE · REJECT

(There's also a Stage 4 for company-specific context — see §7 — but it's a stub that's skipped entirely unless you supply company data, so it's omitted from the trace above.)

Six real traces, exactly as the code produces them today

Run python -m titlecascade "some string" yourself to reproduce any of these — nothing below is simplified or made up.

"Senior Vice President of Sales" — a normal, valid title. Stage 1 passes it (well-formed). Stage 2 finds "Vice President" (a known role phrase) and "Sales" — a title with a real function, so it exits Stage 2 as v2=YES (a title), v3=YES (internally coherent — no contradictions). Final: ACCEPT, confidence 0.80, reason OK_RARE_BUT_VALID.

"n/a" — someone left the field blank in spirit if not in fact. Stage 1 recognizes it as one of a list of ~30 known "sentinel" values people type when they have nothing to put ("n/a", "-", "tbd", "none", ...). It never reaches Stage 2. Final: REJECT, confidence 0.15, reason V1_SENTINEL.

"Acme Corp Inc" — a company name typed into the job-title box, a very common real-world data-entry mistake. Stage 1 passes it (it's a well-formed string). Stage 2 checks it against a list of legal-entity suffixes ("Inc", "LLC", "GmbH", ...) — finds "Inc" at the end and no role word anywhere in the string — concludes this is a company, not a title. Final: REJECT, confidence 0.15, reason V2_IS_COMPANY.

"Intern VP of Sales" — self-contradictory: "Intern" is one of the most junior possible ranks, "VP" is one of the most senior. Stage 2's grammar check specifically looks for this: it has a fixed list of "junior" words and "executive" words, and if both appear in the same string, that's a coherence violation. v2=YES (it does look like a title), v3=NO (contradicts itself). Final: QUARANTINE, confidence 0.35, reason V3_SENIORITY_CONFLICT — held for a human to look at, not silently thrown away.

"asdkjfh aslkjdf" — random keyboard mashing. This is the interesting one. Stage 1's gibberish detector didn't catch this particular pattern (its detector is tuned to specific letter-adjacency and repetition patterns — see the honest limitations in REPRODUCE.md). Stage 2 doesn't recognize any word in it either. So it falls through both stages undecided — the system explicitly does not know what this is. Final: QUARANTINE, confidence 0.40, reason UNMATCHED_UNKNOWN. This is the most important design decision in the whole project, explained in §5.

"Rainmaker" — a real (if informal) way people describe themselves, but not in any word list this system has. Exact same path as the gibberish example above, for exactly the same reason: the system has never seen this word associated with a role, so it can't confidently say yes or no. Final: QUARANTINE, confidence 0.40, reason UNMATCHED_UNKNOWN.

Notice that the last two examples — one genuine nonsense, one a real (if unusual) word — get the same answer. That's not a bug, it's the point: the system doesn't pretend to know things it doesn't know. More on this in §5.

3. The four questions, in plain language

Every string gets scored on up to four independent yes/no/maybe questions. A title can fail one and pass the others — they're tracked separately, not collapsed into one score, because they fail for completely different reasons and (in a bigger system) you might want to act on them differently.

Code	Plain-language question	A "no" looks like
V1 — well-formed	Is this a plausible string at all?	empty, "n/a", an email address, a phone number, keyboard mash, 500 characters of garbage
V2 — is-a-title	Is this actually describing a job, as opposed to something else that ended up in the job-title box?	a company name, a person's name, a department ("Sales"), a credential ("MBA"), a city, "currently unemployed"
V3 — coherent	Does the title contradict itself?	"Junior Chief Executive Officer", "Head of Head of Sales"
V4 — contextual	Is this title plausible for this specific company? (needs company data — see §7)	"Chief Petroleum Engineer" at a dental clinic

4. What this is not

• Not occupation classification. It never tries to say which job this is (e.g. mapping to a government occupation code). A completely unrecognized-but-real job title and a nonsense string get treated similarly (both quarantined) — the system only ever asks "is this a title," never "which one."
• Not an LLM. No generative model, no chatbot, no API calls. Everything is word lists, simple pattern rules, and one small statistical classifier (§6). This is a deliberate choice, explained there.
• Not a verdict on the person. It never checks whether the named person actually holds that title today. That would require a real employer registry lookup, which is a different, harder, keyless-data problem this project doesn't attempt.

5. Why "I don't know" is a real answer here

Most simple string classifiers only have two outputs: valid or invalid. Early in this project's life, that's effectively what happened by accident: any string that didn't match a known-good pattern and didn't match a known-bad pattern fell through to a default of "valid" (because nothing actively said "no"). That silently let through company names without a legal suffix, non-Western person names, buzzwords, and hand-typed gibberish — anything the word lists hadn't happened to cover — because absence of a match looked identical to a confident yes.

This was found by an independent audit (details in REPRODUCE.md §5) and fixed the same day: unmatched strings now default to QUARANTINE, a fifth answer that means "hold this for a human or a future labeling round, don't guess." The five possible final answers, from most to least trusted:

Action	Meaning	What a downstream system should do
accept	Confident this is a usable title	use it
accept_flagged	Usable, but noted something unusual (e.g. four job titles stuffed into one string)	use it, keep the flag for review
demote	Usable but suspicious (e.g. a senior title at what looks like a tiny company)	use it cautiously, deprioritize
quarantine	We don't have enough evidence either way	hold for human review or a future labeling round
reject	Confident this is not a usable title	drop it

This graded-answer idea (instead of a single true/false) is borrowed directly from how large content-moderation systems work — see 01_x_teardown_and_approaches.md for the source material this design is based on.

6. What "model" is used, and why

There is one small machine-learning model in this system (Stage 3), and it is not a language model. It's a standard, decades-old technique:

• Input: the string is broken into overlapping 3–5 character chunks ("char n-grams" — e.g. "sales" becomes sal, ale, les, sale, ales, ...), hashed into a fixed-size numeric vector, plus about a dozen simple counted features (length, punctuation ratio, digit ratio, etc.).
• Model: a separate SGDClassifier (logistic regression, trained with stochastic gradient descent — a standard scikit-learn algorithm, no GPU needed) for each of the four questions in §3.
• Calibration: when a question has enough labeled examples (1,000+), the raw model score is recalibrated ("isotonic calibration") so that a score of 0.8 actually means "right about 80% of the time" — otherwise raw classifier scores are used uncalibrated.

Why this instead of a language model? Speed and cost. A job title is 1–6 words. Running a full language model on every one of 100 million rows would take hours to days of compute. The shipped v2 path (potion-ml static embedding + scorer bank + policy) was timed on this box in Phase F: 1,000,000 distinct strings in 1538 s on one core (p50 1453 µs/string, 650/s) and 585 s on 4 spawn workers (1709/s). Re-scoring the same 1M from a hash cache takes 1.65 s. Extrapolating from that 1M number, 100 million input rows at an assumed 5% distinct-string ratio is ~131 minutes on one core — an extrapolation, not a 100M measurement. The older “single-digit minutes on 16 cores” figure in 01_x_teardown_and_approaches.md §B.3 was a design estimate; it is not the measured number. Detail: results/phase_F.md.

How was it trained? On 72,101 example strings (§7), most of them synthetically generated by code (not written by hand one at a time) — because there's no ready-made list of "job titles people invalid-ly typed into a spreadsheet," so this project builds that list algorithmically: take a real title and corrupt it (typos, truncation, random casing); take a real company name and use it as-is; take two real English words and stack them into a grammar violation; etc. Each generator is documented in REPRODUCE.md §2.

Important honesty note: the ML model's job is small on purpose. The p_valid confidence score you see in the output above is mostly decided by the plain rules and word lists (Stages 1–2), and the ML model is only allowed to push the score down, never up, if it disagrees. It cannot override a rule-based rejection. This was a deliberate design choice, recorded in the code, because letting the ML model freely blend with the rules made scores less predictable without measurably improving accuracy.

7. What data was used

Source	What it provides	Size used	License
O*NET (US Dept. of Labor)	real job titles	11,560	CC BY 4.0
ESCO (European Commission)	real job titles, 28 languages	15,401	CC BY 4.0
SEC EDGAR company filings	real company names, for the "is this a company?" check	1,868	US public domain
US SSA / Census name lists	common given/surnames, for the "is this a person's name?" check	5,000 + 5,000	US public domain
Hand-labeled seed examples	edge cases the automatic generators can't produce (labeled directly by the project builder — see §9)	618	this repo
Algorithmically generated examples (9 generator functions — typos, company names, grammar violations, gibberish, etc.)	the bulk of training volume	~70,000	derived from the above

Full provenance, exact file paths, and every count: data/lexicons/SOURCES.md and REPRODUCE.md §2. One important caveat stated plainly there: part of the evaluation gold set draws from a private sibling repo, so a from-scratch clone by someone outside this project can't reproduce 100% of it — flagged explicitly, not glossed over.

The company-name and person-name lists are US-only (SEC EDGAR filings; US SSA/Census names) — there is currently no international company or person name coverage at all in those two specific checks. This is measured, not assumed: MULTILINGUAL-AUDIT.md also answers whether the matching against these lists is expensive at scale (short answer: no — it's exact hash-set lookup, which is cheap regardless of list size; the real problem is coverage, not compute cost).

No real vendor/customer contact data was ever used anywhere in this project — not for training, not for evaluation. That's a hard policy for this project line (see the memory notes this session inherited), not an oversight. It means every number in this repo describes performance on public taxonomies and synthetic/hand-written examples, and is silent on how the system performs on your actual data until you run it yourself — which is exactly what docs/HOW-TO.md is for.

8. What evaluation was done, and why

Two separate kinds of check exist, and they answer different questions:

1. Self-reported evaluation (scripts/evaluate.py → results/eval_gold.json) — the model is scored against a 1,158-row "gold" set that is never included in training (this is checked by code every time, not just promised — if a training row's key ever matches a gold row's key, the training script crashes on purpose). The headline number is precision at 90% recall: "if we tune the system to catch 90% of all real titles, what fraction of what it accepts is actually correct?" This number is sliced by where each example came from (a taxonomy? a generator? a hand-labeled edge case?) and by string length, because averaging across very different kinds of input hides where the system is actually weak.

2. Independent audits (scripts/independent_eval.py and ..._v2.py) — built and run by a separate reviewer, using strings the model had never seen and whose expected answers were never shown to the model. This exists because a system that grades its own homework tends to look better than it is — every prior project in this problem's history that skipped this step eventually had to retract its headline number (see MEMORY.md's notes on this project line). The independent sets deliberately target categories the self-reported evaluation is structurally unable to catch, because they're built from a different word list and a different generator, coded independently (gen/independent_negatives.py shares no code or data with gen/negatives.py).

Why precision at fixed recall, not plain accuracy? Because most job titles in a real dataset are valid — if 90% of a test set is valid titles, a system that says "yes" to everything scores 90% accuracy while being useless. Precision-at-recall forces you to pick an operating point (e.g. "catch 90% of the good ones") and then honestly report how much garbage comes along with it at that point.

Current numbers, in full, with the story behind them: REPRODUCE.md §5.

9. What evaluation was deliberately not done, and why

Being upfront about the edges of what's known is as important as reporting what is. In order:

Not done	Why
Testing on real production/vendor contact data	Policy: this project never has access to real vendor contact data, and that's intentional (avoids any risk to real people's data, and keeps every published number reproducible by a stranger with only public data). Practical effect: nobody knows exactly how well this performs on your actual database until you run scripts/score_file.py on it yourself, locally.
Native-speaker review of the 23-language role-word lexicon	Not done — but the scale of the resulting gap is now measured, not just flagged as a risk: scoring all 53,869 real ESCO titles across 26 languages gives an 83.1% accept rate in English vs. 53.3% averaged across every other language, with three languages (Maltese, Bulgarian, Icelandic) under 17%. See MULTILINGUAL-AUDIT.md. Native-speaker review would help close this, but the bigger blocker is raw lexicon volume, not just correctness of what's already there.
A trained language-model version of this system (Stage 5 in the original design)	The original Stage 5 LLM (JobBERT / domain-adaptive MLM) has not been trained. The v2 shipped path is a frozen static multilingual embedding (potion-ml) plus a scorer bank and policy — measured in Phases C–F (RESULTS-V2.md). That is an encoder-as-engine, not the original Stage 5 language model.
The external "AI teacher" labeling round	No response was ever collected from an external frontier model. Phase E ran one internal embed-space round (220 labeled + 82 propagated, gold probes 10/10) with the builder labeling by data/generated/TEACHER_NOTES.md policy — not distilled from a larger model. Scaffolding for an external round still exists (scripts/loop.py, rounds/round_00/prompt.md).
V4 (does this title make sense for this company) fitted on real data	This question needs a real database of companies (size, industry) to compute statistics like "how often does a 5-person company really have a Chief Revenue Officer." No such real database is used in this project (see the vendor-data policy above), so V4 is currently a small illustrative stub, not a fitted model — it correctly reports "not evaluated" rather than guessing when company data is missing, but it hasn't been validated against real company statistics at all.
Throughput/latency benchmarking at scale	Measured on this box for the shipped scorecard_v2 path: 1,000,000 distinct strings in 1538 s (1 core) / 585 s (4 spawn workers). 100 million rows was not timed; the 131-minute figure is an extrapolation from that 1M number at an assumed 5% distinct-string ratio. See results/phase_F.md. The §B.3 “single-digit minutes on 16 cores” number is the old estimate.
A systematic bias/fairness audit	The second independent audit specifically tested name diversity (Vietnamese, Yoruba, Korean, Turkish, Filipino, Bengali, Thai, Ethiopian names) and found and helped close a real gap in the person-name detector. That is a useful spot-check, not a full fairness audit — it covers eight naming traditions among many, done by one reviewer in one session.
A second, fully independent reviewer	Both independent audits so far were built and run within the same review session. There has not yet been a second, separately-briefed person or process checking this system's claims.
Enforcing the 0.85 precision gate	results/GOALS.json defines a minimum acceptable precision, but scripts/evaluate.py explicitly does not fail the build if the number falls short — it records the miss and keeps going. This is deliberate while the evaluation methodology itself is still being refined (see the ongoing back-and-forth of numbers in REPRODUCE.md) — turning it into a hard gate before the measurement itself is trusted would just hide problems instead of fixing them.

10. Try it yourself, right now

cd /home/ubuntu/claude_/title-cascade
export PYTHONPATH=.
PY=/home/ubuntu/venvs/torchcpu/bin/python

$PY -m pytest tests/ -q                          # 160 tests, ~1 minute
$PY -m titlecascade "Chief Executive Officer"     # try any string here
$PY -m titlecascade "Rainmaker"
$PY -m titlecascade "Acme Corp Inc"

To run it on your own file (nothing leaves your machine): docs/HOW-TO.md. To rebuild every dataset and retrain from zero: REPRODUCE.md.

11. Current status, one paragraph

Two stacked systems live in this repo. The original v1 path is rules plus a small hashed-ngram linear model. The v2 international path (Phases 0–F, 2026-09-02) is a frozen static multilingual embedding (potion-ml) plus a scorer bank and policy, gated on languages the fit never saw (mt/bg/is/lt). Held-out-language job_title is 0.885 @ 0.990; person/company/place false-accepts are at the floor; 11/45 held-out placeholders still auto-accept (the one open exception). Throughput of that shipped path is measured on this box (1M distinct strings: 1538 s on 1 core / 585 s on 4 spawn workers) — see RESULTS-V2.md. Native-speaker lexicon review and real customer data still have not happened. Every number is a timestamped snapshot — re-run the scripts before trusting a number that matters to you.