titlecheck

Job-title validation for contact lists. Not classification: this tool never asks "which occupation is this closest to?" — it asks "is this string a job title at all, and is this file internally coherent?", and answers with a decision, not a score:

verdict	meaning	what you do
DROP	not a job title (reason logged: profanity, company name, org unit, placeholder, email, name, gibberish…)	discard the value
USE	a title people verifiably hold — exact or cleaned-up match in a public attested-titles table	keep it
REVIEW	well-formed but unproven — the honest "don't know"	a human clears each distinct string once; frequency does the prioritizing

~590 lines of Python, one dependency (polars). No model, no network, no API key. Runs entirely on your machine; your data never leaves it.

Why this exists

Nine prior projects on this problem (see job-title-validation-context) matched titles against curated taxonomies (ESCO/O*NET) or trained classifiers on them. Retractions followed, always for one of two reasons:

1. Similarity is not validity. Every string has a nearest taxonomy label — including "Regional <slur> Coordinator". Distance to it measures nothing about realness.
2. "Not in the taxonomy" is not "not real". A 251k-label lexicon resolved only 32.9% of real titles. Real titles are decorated, vernacular, and long-tailed.

titlecheck replaces both mistakes:

• Attestation, not similarity. A title is accepted because a public source lists it as a title people hold (89,764 distinct normalized titles from 4 independent sources), after deterministic cleanup of real-world decoration ("Sr. Supplier Development Manager, Operations" → "senior supplier development manager"). Every match is exact and auditable — the reason names the matched form and sources.
• Vetoes are final. Profanity/email/emoji can never be outvoted by a clean-looking role word, and compound-splitting ("CEO | Founder") never runs on a vetoed string. These are pinned in tests/test_invariants.py — each test is a bug class that shipped in a prior project.
• Unknown stays unknown. A plausible novel title is REVIEW, never silently accepted or rejected.

Measured results

600-row eval: 300 real job-board vacancy titles (Malaysian job portal — messy, decorated, some non-English) vs 300 negatives. The rules only ever see the train half of every word list; eval negatives come exclusively from the holdout half (deterministic sha1 split), so a caught negative was generalized to, not memorized. Reproduce with python eval/run_eval.py (needs the contact-title-validity checkout).

slice                                     n    USE  REVIEW   DROP
company names (SEC EDGAR)                60      0       2     58
job-board vacancy titles                300    109     189      2
org units                                60      0      39     21
person names (Census x SSA)              60      0       4     56
place names                              60      0      11     49
placeholders/junk (part SYNTHETIC)       60      0      35     25

false accepts (negative -> USE):  0/300 = 0.0%
false drops   (positive -> DROP): 2/300 = 0.7%   ('Casheir' typo, 'JURUWANG' Malay)

Read the numbers as decisions: no garbage was ever accepted; 2 real titles in 300 were wrongly discarded; 36% of real titles auto-cleared exactly. The 47% REVIEW looks large only because this eval is 50/50 adversarial with every title distinct. Real contact files are frequency-weighted — a few thousand distinct strings cover 95% of rows (see profile below), and each distinct string is reviewed once, ever.

File-level checks (profile)

Vendor data breaks systematically, not one row at a time — and per-row checks can't see it by construction. profile group-bys the file against itself:

$ titlecheck profile contacts.csv --col title --company company --email email --name name

rows: 56
verdicts: DROP 5 (8.9%), USE 28 (50.0%), REVIEW 23 (41.1%)
distinct titles covering 95% of rows: 33  <- a human clears the file by reviewing this many strings, once
rows with a one-off title: 55.4%  <- garbage concentrates here
...
red flags:
  ! 1 rows where title == company (shifted column?) e.g. ['initech llc']
  ! 1 companies with 20+ contacts where >90% of titles are unique (real orgs repeat titles; this looks fabricated or scraped) e.g. ['globex']
  ! 1 emails appearing under more than one company e.g. ['shared@vendor.example']

(Real output on examples/contacts_demo.csv, which has these defects planted.)

Usage

pip install polars
python -m titlecheck check   contacts.csv --col title --out verdicts.csv
python -m titlecheck profile contacts.csv --col title --company company --email email --name name
python -m titlecheck check   titles.txt            # one title per line

Library: from titlecheck import check; check("Sr. Software Engineer") → Verdict(verdict='USE', reason="attested after cleanup as 'senior software engineer' (uk_soc)").

Rebuild the reference table (needs the sibling repo's data): python scripts/build_attested.py.

Scope: validation, not verification

titlecheck decides whether a string is a job title and whether a file is coherent. It cannot decide whether this person holds this title at this company today — that is verification: per-record, network-bound, expensive, and worth doing only for the records you are about to act on. contact-trust does that layer (live registry checks). Run titlecheck on all 50,000 rows at ingest; verify the 50 you're about to call.

Data provenance (all public, all keyless)

source	what	size
ONS UK SOC 2020 coding index	titles as observed on UK forms	23,932
ESCO (EU)	preferred + alternative labels	16,919
O*NET (US)	reported + alternate titles	46,646
gpriday/job-titles	curated real-world list	65,248
SEC EDGAR	company-name negatives	7,994
US Census / SSA	first/surnames for the name rule	40,000

Known limitations (measured, not hidden)

• English-only sources. JURUWANG (Malay: cashier) is wrongly dropped. Fix: add non-English attestation sources (ESCO ships 28 languages; only EN is loaded).
• Official lists lack tech vernacular: technical lead, team lead, fullstack developer are in none of the four sources → REVIEW. Pinned by a test so closing the gap is visible. Fix: a frequency table built from a large public job-postings corpus — this also converts binary attestation into how-common-is-this-title, the single highest-value upgrade.
• Typos land in REVIEW or DROP (Casheir). No fuzzy matching by design: fuzzy matching is how prior projects accepted garbage. A postings-derived frequency table plus edit-distance-1-to-a-frequent-title would handle this auditable-y.
• The eval's positives are job-board postings, not vendor contact records; the file-level checks are demonstrated on synthetic data. Per the data-handling policy, real vendor data never touches this repo — run it locally on yours and judge there.