We identified a documented gap
Research organizations that monitor a fast-moving domain — a trade relationship, a regulatory space, a development-finance corridor — face a choice the market has quietly decided for them.
$50,000–$100,000+ for a small team, and what arrives is a generic firehose analysts must still triage by hand.
Custom assembly from data-extraction APIs — tools that assume in-house engineering most research firms simply don't staff.
No finished, domain-bespoke system a boutique team can own.
Agama Intelligence sat squarely in that gap. The firm tracks projects, announcements, and policy decisions across the China–Africa relationship — dozens of sources, multiple languages, a constant stream of communiqués and coverage. All of it was tracked by hand: analysts reading, judging relevance, and entering records into spreadsheets, with coverage latency measured in weeks and the classification methodology living in one analyst's head.
We investigated why the gap persists
The gap isn't an accident — it's structural, and each existing option misses for a different reason.
Per-seat platforms can't do bespoke
Their economics depend on selling one horizontal product thousands of times. Encoding a single client's taxonomy, country list, and classification rules breaks that model — so they don't. The client adapts to the platform, never the reverse. And their volume is a feature only for large teams: a firm with three analysts drowns in alerts built for a security operations center of twenty.
Extraction APIs sell parts, not systems
The component tools are genuinely good — but they end at structured text. The review workflow, the accuracy measurement, the brief generation, the dashboard an analyst actually uses: that last mile is precisely the part the buyer can't engineer themselves.
Manual domain datasets prove the demand — and the ceiling
Academic projects in Agama's own field maintain rigorous structured databases, updated roughly annually, by hand. They validate that expert-verified structured data is valuable, and demonstrate that manual compilation can't deliver it at operational cadence.
The conclusion of the investigation: Agama didn't need a subscription or a toolkit. It needed the third thing the market doesn't sell — a finished system shaped to its own analytical framework.
We intervened — and deliberately left something out
We built Agama an automated intelligence pipeline: continuous monitoring of a registered source set — never open-web crawling — with a single, schema-validated AI extraction per document that judges relevance and classifies findings against Agama's own controlled taxonomy; an auditable database the client owns; and the deliverables analysts hand to clients. The whole system runs on a daily schedule an analyst can toggle from the browser — no terminal, no technical setup.
Monitor
A registered source set, checked on a daily analyst-controlled schedule.
Extract
One schema-validated call per document — relevance judged, findings classified.
Review
Low-confidence output routes to a queue where analysts confirm, correct, or reject.
Deliver
Spreadsheet exports, date-ranged briefs, and a live dashboard with trends and cost telemetry.
The AI's right to be trusted.
Nothing the model extracts is silently accepted. Every category, country, and entity name it returns is re-checked by the program against the controlled vocabulary; anything low-confidence or unresolvable routes to a review queue inside the dashboard, where Agama's analysts confirm, correct, or reject it. Every correction is logged as an auditable event, immediately fixes the live record, and becomes verified ground truth for measuring the system's accuracy. We also kept spending on a leash: a deterministic filter drops off-topic items from high-volume feeds before a single cent of AI cost is incurred.
The system does the reading; the experts do the judging.
That omission is the design. An intelligence product a research firm will stake its name on cannot contain unverified machine output — so the architecture makes silent acceptance impossible rather than merely discouraged.
For the technically curious (optional depth)
One structured, schema-validated call per document — no chains, no agents, one auditable judgment. Python service layers over PostgreSQL; a client-labeled evaluation set with an automated accuracy harness; 270 automated tests, type-checked and linted on every change; idempotent runs that never duplicate work or spend; per-call token use and cost logged and visible in the dashboard. The stack is deliberately boring: auditability over novelty.
We documented the outcome — and what generalizes
The outcome. Manual collection replaced by an automated pipeline running on an analyst-controlled daily schedule, with expert review built in. Relevance detection measured at precision 1.00 — everything the system flagged as relevant genuinely was, zero false positives — with F1 0.93 against Agama's own hand-labeled ground truth. Processing cost measured at under one cent per article, putting monthly operating cost in the tens of dollars at realistic volumes — against incumbent platforms charging five to six figures per year. All Phase 1 deliverables met or exceeded, the system in active use, and a structured, auditable, growing database that Agama owns — institutional knowledge no longer trapped in spreadsheets.
What generalizes. The architecture separates what is Agama's from what is the system.
The taxonomy
The reference tables
The labelled evaluation set
The branding
Extraction
Review queue
Measurement harness
Deliverable layer
A deployment for a different domain is a re-configuration and a source-feasibility audit, not a rebuild — and every client receives their own independent instance and owns their database, their corrections, and their accumulated evaluation set outright. When the engagement ends, the asset stays.
Sign-off gates
Decision-complete planning documents approved before any code is written.
Cheap probes first
Technical probes disqualified several attractive-looking sources before they could burn a build cycle.
Fixed scope
How a non-technical client stays in control of a technical build — and how this one stayed on scope.