Research agenda / Proposed work
Questions
worth testing.
A plan to turn engineering intuitions into reproducible experiments on data quality and evidence-bounded AI.
Civic data curation & retrieval
How do deduplication and provenance checks change source retrieval and grounded answers?
Experiment direction
Use a rights-audited civic-document corpus. Freeze the retriever and model, hold out jurisdictions, and compare curation interventions through coverage, retrieval errors, answer support, latency, and cost.
Intended publication
A documented corpus, frozen evaluation set, reproducible baseline, and ablation report.
Evidence-bounded answers
Do claim-level citations and explicit unknown states improve the usefulness of an answer?
Experiment direction
Compare ordinary retrieval answers with answers that expose source support, contradictions, and unknowns. Use the same frozen questions and blinded human evidence review for paired comparisons.
Intended publication
An evidence rubric, annotated comparison set, error taxonomy, and paired results.
Temporal evidence & abstention
Can better treatment of dates prevent a system from relying on a superseded rule?
Experiment direction
Separate publication, effective, and retrieval dates. Evaluate current and historical questions, including cases with conflicting or missing current evidence. Measure reliance on superseded rules and appropriate abstention.
Intended publication
A versioned temporal benchmark, source timeline, and evaluation of unsupported answers.
Across every study
Make the result inspectable.
- Document data rights, collection, exclusions, and source dates.
- Keep held-out evaluation material separate from development data.
- Publish baselines and ablations before claiming improvement.
- Report unsupported answers, abstention, latency, and cost.
- Include failure cases and the limits of the conclusion.
These directions grow out of the public Cividian Site Diligence and Phroneme projects. Their purpose is to produce original evidence, not to imply results that already exist.
Research conversations