- Python 100%
Claude Orchestrator
08b94c596d
The repo was documented for code reviewers, not for anyone trying to LEARN it, which made it a black box. Two additions fix that: docs/TUTORIAL.md - plain-English, step-by-step, every command runnable and every number measured live rather than quoted. Covers: why 'is this title real' is three separate questions; rules-before-ML and why (the circular label trap); what an embedding actually IS, hands on; where embeddings measurably FAIL (real output: 'banana bread' scores 0.882 against 'Chief Financial Officer', HIGHER than 'CFO' at 0.874 -- so absolute cosine similarity is not a decision procedure); where they genuinely work (nearest-neighbour with a threshold tuned on held-out data); the six L3 outcomes and why OUT_OF_SCOPE matters; and the three scoring mistakes made in this repo, as lessons. explain.py - traces one record through all five phases showing input, each layer's decision, the reasoning, and the final output. Answers 'how does this actually work' by showing it rather than describing it. Also fixes a real bug found in review: the Companies House adapter read only the first page of officers (CH defaults to 35). Any mid-size company would silently drop officers, and since NOT_LISTED now scores as negative evidence, that meant false accusations against real people. Now paginates properly with a safety valve. Co-Authored-By: Claude Orchestrator <noreply@anthropic.com> |
||
|---|---|---|
| calibration | ||
| data | ||
| docs | ||
| identity | ||
| ingest | ||
| l1_titlegrammar | ||
| l2_consistency | ||
| l3_verify | ||
| model | ||
| results | ||
| scripts | ||
| serve | ||
| tests | ||
| .gitignore | ||
| explain.py | ||
| README.md | ||
contact-trust
Validates third-party-vendor contact/profile data (JSON social profiles and tabular files, public and private companies) for job-title plausibility and identity-level fake-profile risk — tiered confidence output, never a bare true/false.
Working example, run live against the real Brønnøysund registry: a record claiming "Gregory Chandler Whitmore, Chief Executive Officer, Equinor ASA" — a competently invented identity, not on any known-fake list and not gibberish — scores 0.15, because the registry was asked and answered that no such officer exists. Equinor's real, currently-registered CEO scores 0.95 from the same query.
That distinction matters and is the whole point: an earlier version of this
README cited a "Mickey Mouse → 0.05" demo, but an independent review showed
that record was being caught by a hardcoded fake-name list and never
exercised the registry path at all. A competent fake scored 0.28–0.89
until that was fixed. See docs/STATUS.md's "Post-build independent
review" section.
New here? Start with the tutorial, not the code
docs/TUTORIAL.md — a step-by-step guide in plain
English that teaches how this actually works: the three different questions
hiding inside "is this job title real", how to check titles with rules, what
an embedding really is (with runnable code and honest measurements of
where embeddings fail), and how evidence gets combined into a defensible
score. Every command in it runs on your machine and prints real numbers.
explain.py — the de-black-boxer. Traces ONE record
through all five phases and prints what each one decides and why:
python3 explain.py --live # a real registered CEO
python3 explain.py --name "Gregory Chandler Whitmore" \
--title "Chief Executive Officer" --company "Equinor ASA" --country NO --live
Reference docs: docs/ARCHITECTURE.md (the full design — why this exists,
what's reused vs. new, the L0→L1→L2→L3→identity→confidence pipeline, the
VPS/Mac-M5-Pro/VPS-prod split, what's honestly hard or unverifiable and
should never get oversold) and docs/STATUS.md (what's actually built and
measured, step by step — every number sourced, every negative result
reported as plainly as a positive one). docs/EVIDENCE_LADDER.md is the
governing spec this was built against. docs/EVAL.md and
docs/REVIEW_RANKER_DESIGN.md cover two things that were tried, measured,
and honestly not shipped or not built.
Run the QA gate yourself: PYTHONPATH=. python3 scripts/qa_gate.py
(7 checks, all passing). Score your own file locally:
serve/batch_score.py yourfile.csv --vendor "YourVendorName" (add --l3
to enable live registry lookups; off by default so a first run makes no
surprise network calls).
Real vendor data never touches this repo or any AI session that built
it. Everything here is code + public keyless corpora (ESCO/O*NET/SEC/
Brønnøysund/NPI/Wayback/Wikidata, all vendored under data/) + synthetic
test fixtures. serve/batch_score.py, run on your own machine against your
own files, is the one place real data ever gets processed — nothing is
uploaded anywhere.
What's built (10/10 build-order steps — 7 built and measured, 2
correctly rescoped with reasons written down, 1 documented-not-implemented)
| Step | What |
|---|---|
| 1 | L0 ingest (JSON profiles + tabular) + L1 title grammar |
| 2 | L2 file-internal consistency (cardinality, pyramid, C-suite density) |
| 3 | L3 registry/regulated-profession/dated-trace verification (NO, FR, GB, US NPI, Wayback) |
| 4 | Identity/profile plausibility (name checks, employment consistency, tenure priors) |
| 5 | Company-entity resolution (fuzzy matching, measured, one real bug found and fixed) |
| 6 | Calibration harness (stratified vendor-precision transfer) |
| 7-8 | Embedding model (one use proven, one honestly rejected) + the batch-scoring CLI |
| 9 | QA gate + composed-pipeline regression eval |
| 10 | Review-queue ranker — designed, correctly deferred (needs data that doesn't exist yet) |
Read docs/STATUS.md for the real numbers behind every row above.