Footnote
Grounded Q&A over SEC filings: ask a plain-English question about a US public company and get an answer built only from what it has filed.
1. Problem
People researching a company use chatbots to get through its 10-K faster, but they can’t tell whether a conclusion came only from the filing or from somewhere else. So they re-check each answer against the source themselves, which gives back most of the time the chatbot saved.
Evidence: interviews with 6 target users about how they research 10-K filings. The re-checking came up as the common workaround.
2. Target users
People who read a US public company’s filings to understand the business and its risks before making a decision, and who need to be able to defend where a claim came from.
3. Goal
A user can ask a question about a company and trust the answer without opening the filing to verify it.
4. Scope
| v1 | v2 (later) |
|---|---|
| One company per question | Compare companies in one answer |
| Any US company with a ticker | Foreign issuers and funds |
| Latest 10-K, 10-Q, 8-K, DEF 14A | Alerts on new filings |
Out of scope for both: investment advice, and any source other than the company’s own filings.
5. Requirements (v1)
- The user types a ticker; its recent filings are downloaded and indexed while a progress bar shows.
- The user asks a question in plain English and gets an answer scoped to that one company.
- The answer is written only from passages retrieved from the filings.
- Every quote is checked against the source before it is shown, and links back to the filing section it came from.
- Financial tables are read as tables, so a question about a line item is answered from the table’s values.
- If the filings don’t cover the question, the product says “not disclosed” instead of guessing.
6. Success metrics
All four are scored on a reviewed evaluation set built from real filings, and tracked against a saved baseline so a change that makes things worse is visible.
| Metric | What it asks | Bar |
|---|---|---|
| Answer accuracy | Is the answer right, judged against the filing? | Improves on the baseline |
| Citation accuracy | Does the cited passage exist and support the answer? | 100% |
| Abstention | Does it decline the questions the filings can’t answer? | Improves on the baseline |
| Outside content | Does anything in the answer come from outside the filings? | Zero |
7. Risks and open questions
- A wrong answer with a confident citation is worse than no answer, so declining has to be reliable, not just possible.
- Thresholds for when to decline may not carry over from the company they were tuned on to others.
- Checking that a quote is genuine does not prove it supports the claim built on it.
- Table-heavy questions are the hardest to get right and need their own handling.