# A private demo: the same calculator, inside Claude and ChatGPT.

Source: https://plannerbench.org/chat.html · plain-text twin of that page, made from it at build time · all pages: https://plannerbench.org/llms.txt

Demo walkthrough · private connector, not public

What it is

This is a private demo; you can’t install it. The calculator from [With a calculator](https://plannerbench.org/calculator.html) also works as an add-on (a “connector”) for Claude and ChatGPT. Someone asks a retirement question in everyday words. The calculator, not the chat AI, decides which fact to ask for next, does the calculations, and says what it assumed. The chat AI is instructed to be friendly and quote the calculator’s numbers exactly.

Can I use it?

No. It runs at a private link while I test it, and there’s no public install. This page shows what it does, using its real replies to a made-up couple.

A conversation, from the calculator’s side

## What it sends back, word for word

The chat AI turns these replies into conversation. Its instructions say to quote these numbers and never work out its own; in testing, a checker flags any number in a conversation that no calculator reply contains.

1. 1 · Someone asks, with no details yet
   
   “We’re both retired. Should we be converting some of our IRA to a Roth, and how much?”
   
   It asks one question, says why, and what to have handy
   
   How old are you and your spouse this year?
   
   Why it matters: Age decides when required withdrawals start, Medicare, and how long the plan runs; being married decides the tax brackets, and a widow or widower may have survivor benefits and a spouse's options on inherited accounts.
   
   - Birth dates for both of you
   - Social Security statements from ssa.gov for both of you
   - The latest statement for every account (IRAs, 401(k)s, Roth IRAs, brokerage, savings), with what you paid (cost basis) for brokerage accounts
   - Paychecks or yearly wages, and any pension or annuity amounts
   - Your last two tax returns (they set Medicare premiums, and Form 8606 shows any after-tax IRA money)
   - Roughly what you spend in a year
2. 2 · A few questions later, every fact in
   
   The couple has answered ages, accounts, Social Security, spending and state, one question at a time.
   
   The answer, in sentences
   
   Convert about $56,000 a year from Dana's IRA, 2026 through 2035.
   
   By 2055 that leaves the family about $132,193 better off after taxes, in today's money, than not converting.
   
   - **Best plan**: $56,000 a year for 10 years (2026–2035)
   - **Total converted**: $560,000
   - **Better off by 2055 (today's money)**: +$132,193
   - **Change in lifetime tax (today's money)**: −$113,779
   - **Change in lifetime Medicare premiums (today's money)**: +$0
   
   What it costs now
   
   - Income tax on the extra money taken out or converted **$62,206** IRC §1(j); Rev. Proc. 2025-32
   - Arizona income tax (2.5%) **$11,422** A.R.S. §43-1011; Arizona Form 140
   - Dividends and gains lose their 0% tax rate as income rises **$9,503** IRC §1(h); Qualified Dividends and Capital Gain Tax Worksheet
   - More of their Social Security becomes taxable as other income rises (the "tax torpedo") **$8,246** IRC §86; IRS Pub 915 Worksheet 1
   - Total, 2026–2035 (the biggest reasons are listed; smaller ones make up the rest) **$91,434**
   
   What it saves later
   
   - Less income tax later, because the IRA is smaller and so are the required withdrawals **$247,329** IRC §1(j)
   - Less Arizona income tax **$33,371** A.R.S. §43-1011; Arizona Form 140
   - More dividends and gains stay at the 0% rate **$29,207** IRC §1(h); Qualified Dividends and Capital Gain Tax Worksheet
   - Less of the 3.8% tax on investment income **$21,835** IRC §1411; Form 8960
   - Total, 2036–2055 **$331,742**
   
   How these fit the figures above: the costs and savings are each year’s own dollars, added up as they would appear on the tax returns. The lifetime tax change (−$113,779) and “better off” (+$132,193) are in today’s money, which counts a dollar paid decades from now as worth less. “Better off” also counts what’s left in every account at the end, so it isn’t the tax change alone. The two kinds of figure measure different things and aren’t meant to add up.
   
   Said out loud: what it assumed
   
   - Assumed: Dana on Medicare: Yes, from 65, because nearly everyone enrolls at 65.
   - Assumed: Lee on Medicare: Yes, from 65, because nearly everyone enrolls at 65.
   - Assumed: The plan runs through: 2055, because until the younger one would be 95.
   - Assumed: Investment return: 5% a year on every account, because a planning assumption, steady every year.
   - Assumed: Inflation: 2.5% a year, because tax brackets and Medicare lines rise with it after 2026.
   
   And what could change the answer
   
   With a 7% return, the better choice becomes $100,000 a year for 10 years (about $85,666 better).
   
   24 facts said, 5 assumed, 1 worked out · 1,402 plans run for this one answer, 1.4 s on a laptop
3. 3 · Someone types a Social Security number
   
   “My SSN is 000-12-3456, can you check my Social Security?”
   
   Turned away before it reaches any math
   
   That looks like a Social Security, account or ID number. The engine doesn't need it, so please leave it out and send the rest again. Nothing you sent was kept.

How it behaves

## Most rules live in the calculator, not in a prompt

- **It picks the next question.** It reruns the plan with each unknown fact both ways and asks only for the one that could change the choice or move the answer more than 2%. You can skip any question; it then says what it assumed instead.
- **Numbers come from the calculator.** The chat AI is instructed to quote only the calculator’s numbers. In tests, a checker lists any number in a conversation that no calculator reply contains. That catches a slip; it can’t make one impossible.
- **Close calls get both plans.** When two plans land within $1,000 over 25 years, it says “it’s close” instead of naming a winner.
- **It says what it doesn’t cover.** A question outside its rules gets “that isn’t covered yet”, not a guess.
- **It keeps nothing.** No database, no files, nothing you type in its logs. Anything that looks like a Social Security, account or ID number is refused and never repeated back.
- **Education, not advice.** It prices choices; it never picks investments or products.

How it’s checked

## Tested before anyone used it, then fixed from real use

69 / 69

### Simulated clients

For each household question with a fact left out, a scripted client answers only what the calculator asks. A pass means the same decision as with every fact known, the number within 2%, an answer within 6 questions, the hidden fact asked for, and no wasted questions. No AI model is involved.

6 + 17 notes

### Test drives become tests

I used it in Claude the way a client would, with hard households (a widow with inherited IRAs, someone still working past 62), and wrote down every rough edge. Each was fixed and the session saved as a test so it can’t come back. One changed a headline: the Social Security earnings limit erased a “claim at 62” gain.

210

### Automated tests

Every rule, the everyday-words input, the ID-number refusal, and a full round trip over the network. Given the same households in everyday words, it reproduces the household decisions’ answer key.

**For the technical reader: how it’s built**

- **An MCP server** (the Model Context Protocol, the standard way to plug tools into AI models). One server, two ways in: a local process for Claude Code and the benchmark runners, and a private web address for Claude and ChatGPT. The replies above are from engine commit `3b7d8bc`.
- **Tools.** `plan_check` (the front door, in everyday terms) plus eight calculators for builders: federal income tax, taxable social security, required minimum distribution, social security benefit, medicare part b premium, tax tables, run scenario, solve roth conversion.
- **Replies** are one of `need_more` (one question), `answer`, `not_covered` or `rejected`, each with a `how_to_reply` the chat model follows.
- **Stateless over HTTP** (Streamable HTTP, a fresh server per request): the conversation carries the facts, so there is nothing to store or leak. Requests capped at 64 KB, every list capped, a per-address rate limit.
- **A card inside the chat** (MCP Apps): the answer, the conversion curve and what it costs vs. saves. The text reply is always the whole answer, for apps that don’t draw cards.
- **The benchmark sees the same engine.** PlannerBench calls the builder tools over the local process, so a score on this site and an answer in the chat come from the same code.

[Next **Replays: every calculator call, step by step**](https://plannerbench.org/replay.html)

---

PlannerBench · 2026 rules · [For AI labs](https://plannerbench.org/labs.html) · [For financial firms](https://plannerbench.org/firms.html) · [Report](https://plannerbench.org/report.html) · [Demo walkthrough](https://plannerbench.org/chat.html) · [Replays](https://plannerbench.org/replay.html) · [peytonthalman.com](https://peytonthalman.com)

Human | For AI readers, top right: the same site as plain text for AI agents reading on someone’s behalf, with every page as Markdown.

Decision support, not advice. Answers follow IRS, SSA and CMS publications; nothing here is tax or investment advice.
