Introducing Sheldon, Feather's Deterministic Tax Agent

Our Philosophy

When we started working on Sheldon, Feather's Tax Agent for generating tax returns, we internally debated between two competing solutions: one is generating tax rules on the fly, using Feather's dedicated Knowledge Base that contains up-to-date information of all federal tax rules (and is updated every week), and then using the generated rules to prepare the tax returns. The second is to generate the tax rules offline, verify them by running Sheldon on tax preparation test cases, and doing a human review of the generated tax returns to ensure its correctness. The first approach will generate tax returns for any tax case (thus providing 100% coverage), but 100% accuracy is not guaranteed, since it is not possible to manually check all the on-the-fly generated tax rules. The second approach can ensure 100% correctness of the generated rules, but cannot provide a complete coverage of all the tax rules, since it does not have any provision for generating missing tax rules on the fly.

In building Sheldon, we decided to go with the second approach. Our decision was driven by the aim of providing our customers with a tax engine that they can fully trust, and therefore accuracy of the Tax Engine is a non-negotiable requirement for us. If a particular tax rule is not implemented, or a tax form is not supported, Sheldon will upfront tell the user about the missing artifacts, so as to not cause any confusion to the users about the applicability of the tax engine for a given tax preparation case. This behavior is built into the engine’s core semantics: when a document is known to exist but a value from it is missing, every computation that depends on that value resolves as incomplete, rather than silently producing a wrong number. The key differentiator for Sheldon, as compared to other deterministic tax engines, is that new rules can be added and verified quickly using state of the art AI models. Thus, even if there are missing rules in the rule engine, the gaps can be plugged quickly. This leverages one of the biggest strengths of state of the art Large Language Models (LLMs) today: that they are good at code generation when controlled by a well set-up test harness, and which is a well-established practice in industry.

Overview of Sheldon

Sheldon’s deterministic tax preparation agent consists of three parts:

1. A document ingestion engine that ingests information from all the tax documents (PDF/image) and any other relevant information provided by the user.
2. A static rule engine that contains encoded tax rules in JSON format
3. A deterministic interpreter that consumes the information from the ingestion engine, generates a directed acyclic graph for tax computations, performs the required computation, and then prints the output to the relevant tax forms.

Except for the document ingestion step, where we use AI vision models for extracting data from flattened PDF files/images, at no point during the execution of Sheldon are AI models used to generate the final tax returns. AI models are used offline to convert the existing tax rules from Feather’s proprietary Knowledge Base (KB) to the encoded JSON tax rules only. This ensures that the output from Sheldon remains deterministic: once a particular rule has been verified and finalized in the rule engine, it is guaranteed to produce the same, auditable result on every run.

The addition of new rules to the static rule engine is the most critical part of the development process. The current harness we have for adding new rules to the rule engine works as follows:

1. We start with a set of test cases. These are either publicly available test cases (e.g. TaxCalcBench) that provide the expected outputs against which generated tax returns can be cross-checked, or are internally generated test cases. We have developed an agent that can generate artificial test cases (in the form of PDF and plain text files) based on a high level description, but for which we do not have an available tax return output to check against.
2. For a particular test case, we run Sheldon and identify the missing rules in our rule engine. The missing rules are added to the rule engine by looking up the relevant tax document in Feather’s KB, and using an LLM to convert the tax rule written in English to a tax rule in JSON that can be interpreted by Sheldon’s interpreter. Every generated rule records its provenance inside the rule itself - what it implements and the authority it cites, such as the form line, IRC section, or IRS guidance - so each computed value stays audit-ready.
3. Once all the rules have been implemented, we run Sheldon on the test case to get the complete tax return. For the case where the expected output for the test case is already known, the results are checked against this output. If the numbers do not match, we go back to Step 2 to correct the rule implementation and repeat the process in a loop till all the numbers match.
4. Every generated return, whether or not an expected output is known, is also cross-checked by our internal AI-driven Verification agent. For internally generated test cases with no expected output to score against, the Verification agent may still fail to catch every issue, so a member of our CPA team manually reviews Sheldon’s output as the final layer. If the CPA identifies a mistake in the generated tax return, we again identify the relevant erroneous rule and correct its implementation in the rule engine, until the test case passes CPA manual review.

As a concrete example, here are two rules from Sheldon's rule engine, exactly as they are encoded. They implement lines 6 and 7 of Form 8959 (Additional Medicare Tax), whose printed captions read "Subtract line 5 from line 4. If zero or less, enter -0-" and "Multiply line 6 by 0.9% (0.009)":

[
  {
    "factPath": "/federal/additionalMedicare/line6",
    "type": "money",
    "provenance": {
      "status": "mirrors-current-code",
      "mirrors": "Form 8959 line 6 = Medicare wages minus the filing-status threshold, floored at $0",
      "irsRef": "Form 8959 line 6"
    },
    "ast": {
      "op": "max",
      "args": [
        {
          "lit": 0,
          "unit": "money"
        },
        {
          "op": "sub",
          "args": [
            {
              "fact": "/federal/additionalMedicare/wages"
            },
            {
              "fact": "/federal/additionalMedicare/threshold"
            }
          ]
        }
      ]
    }
  },
  {
    "factPath": "/federal/additionalMedicare/line7",
    "type": "money",
    "provenance": {
      "status": "mirrors-current-code",
      "mirrors": "Form 8959 line 7 = 0.9% of line 6, whole-dollar half-up like the printed form",
      "irsRef": "Form 8959 line 7"
    },
    "ast": {
      "op": "scale",
      "arg": {
        "fact": "/federal/additionalMedicare/line6"
      },
      "rate": 0.009,
      "round": "dollarHalfUp"
    }
  }
]

Each rule carries its provenance - the plain-English statement it implements and the authority it cites - alongside the encoded arithmetic, which mirrors the printed form down to its rounding convention (line 7 rounds to whole dollars, half up, exactly as the paper form instructs). The interpreter needs no tax knowledge of its own: it simply resolves each referenced value and applies the encoded operations.

Sheldon on TaxCalcBench

From 50% to 100%, case by case

The first time we pointed Sheldon at the ten TY2025 federal test cases in TaxCalcBench, it scored 50.0%. That's 95 of 190 scored Form 1040 lines exactly right, which sounds respectable until you remember that a tax return with one wrong line is a wrong tax return. Three working days later the engine scored 190 of 190 on all ten cases, under the benchmark's strict scoring. The last five cases went from failing to perfect in a single day.

Here's what those three days actually looked like.

Step 1: Make the benchmark deterministic

Before we fixed anything, we made every run reproducible. We pinned the document-extraction output for each case's input PDFs (the W-2s, 1099s, and 1098s each case starts from), wrote a converter that maps the benchmark's interview-style inputs into the input format of Sheldon's document ingestion engine, and built a scorer that replicates the benchmark's own metric: an exact whole-dollar match on 19 Form 1040 lines per case. A benchmark that gives you a different answer on every run can't tell you whether a fix worked. Ours now gives the same answer every time.

Step 2: Give every wrong line a cause

We didn't chase the score directly. Each wrong line got attributed to a specific cause first: a document type the engine didn't ingest yet, a form module we hadn't built, a constant that didn't match current law, or a rounding convention. Then we worked through the causes in order of how many lines each one would recover.

Step 3: Decode first, then write rules

The most useful habit we developed: for every failing case, write the spec before touching the engine. Each expected line gets traced back to a formula. Each constant in that formula gets traced back to a citation in Feather's tax knowledge base, which holds the Internal Revenue Code, IRS form instructions, and current-year guidance. Only after the spec is written do we express the rule, as declarative JSON the engine executes deterministically. The LLM's job is to construct and cite rules. It never computes a tax return at runtime.

Writing the spec first kept catching things that chasing the score would have missed. Three examples that stand out are:

The IRS Tax Table convention. Our bracket math was persistently a dollar or three off on every return under 100,000 of taxable income. The reason: the IRS Tax Table computes tax at the midpoint of each 50 income band, not at the exact amount. Taxable income of 26,905 is taxed as if it were 26,925. One systemic fix corrected three test cases at once, and the same band-midpoint convention turned out to govern the EIC table too.

A deduction larger than the standard deduction, legally. One case claimed 26,125 against a 23,625 standard deduction without itemizing. The numbers made no sense until the decode surfaced the Form 4684 qualified-disaster-loss mechanic: a qualified disaster loss stacks on top of the standard deduction (23,625 + 2,500 = $26,125). This rule lives in the form instructions rather than the statute, and most tax software don't handle it.

Disability pensions are wages. A 1099-R with distribution code 3, received before minimum retirement age, reports on Form 1040 line 1h as wages. It also counts as earned income for the Earned Income Credit. That single reclassification flipped a case from 9 correct lines to 19, because five downstream credits depended on it.

Step 4: Never move backward

Every fix had to pass three gates before it counted. The full ten-case benchmark re-runs on every change, and no case is allowed to regress. A parity harness proves the declarative rules match an independent reference implementation fact for fact. And unit fixtures pin the intermediate values from the benchmark's ground truth, not just the final lines, so a fix can't quietly break the arithmetic it sits on.

Step 5: Verify the printed return, not just the math

A return can have all 190 scored values right and still not be something a CPA would sign. So our verification agent, an independent reviewer that reads the rendered PDFs against the source documents, audited every return. It caught a class of defects the scored lines can't see: worksheet lines that printed net values where the form wants gross, required attachments (Form 8606, Form 2441, Schedule R) that were computed but never printed, and Schedule E property columns that didn't foot. We fixed every one. The result is ten complete, internally consistent federal returns, assembled in IRS attachment-sequence order, with every scored line exact.

Final Thoughts

These ten cases are public and we developed against them. By benchmark standards they're training cases, not a held-out evaluation. What we'd point to instead is the slope: by the end, each new failing case took hours to close, and the loop never changed. Decode the case, cite the law, write the rule, verify the output. The engine's architecture wasn't touched after day one. That loop is the thing we actually built, and we're happy to run it on any held-out cases the benchmark's maintainers publish next.

We will be rolling out Sheldon for Feather users in the coming days. In the meantime, if you want to try out our existing offering for tax research, you can sign up for free on our website.