01 LDB Finance
Equity research that checks itself before it publishes.
The problem
An analyst report is a chain of numbers: figures copied from filings, a model built on them, a value, a rating. A mistake anywhere in the chain changes the conclusion, and a reader has no way to see where. I wanted reports where every number can be traced and the site refuses to publish when the numbers disagree.
Decisions and trade-offs
| Decision | Why, and what it costs |
|---|---|
| One set of assumptions for every report | Every company is valued with the same Treasury yield and equity risk premium, so ratings can be compared. The cost: a model that sits below Wall Street targets almost everywhere, which each report explains. |
| Ratings relative to the S&P 500 | On the same assumptions the index itself is worth about 68% of its price, so a stock is rated against that, not against 100%. Otherwise almost everything would be a Sell. |
| A simulation instead of three fixed cases | Each model is rerun 4,000 times with inputs drawn from ranges, and the rating uses the average. A 25/50/25 weighting is a convention; a range shows how uncertain the value is. |
| Static pages, data refreshed by scheduled jobs | No server to maintain or secure. The cost is that anything live, such as today’s price against the report, runs in the reader’s browser. |
What went wrong, and what now catches it
| Problem | What happened, and the fix |
|---|---|
| Intraday prices | A manual data run in the afternoon saved intraday prices as closing prices. The price update now refuses to run while the market is open. |
| Wrong rating on the watchlist | A pattern meant to find Visa’s ticker also matched GE Vernova’s. The watchlist now takes its ratings from the rating rule itself, so the two cannot disagree. |
| A dividend copied wrong | Found in a hand audit against the company’s release and corrected in public. |
| A stake counted twice | A $94 billion stake was counted once in cash and once on its own. Found when the balance sheet was checked against the 10-Q; the value moved less than 2% and the rating did not change. The earlier version is kept, unchanged. |
Each build now runs four checks and stops if one fails: every model reruns and must match its report; each report must agree with itself (the scenario weights, the valuation bridge, free cash flow in every year); tests that break real reports on purpose prove the checker notices; and hand-checked formula tests compare the math with textbook answers. Figures copied from filings are audited by hand, and every correction is logged.
What it taught me. Automated checks catch inconsistency, not error: a wrong number copied consistently passes every test. The audit against primary sources is the part that cannot be automated away. See the audit log.
02 ARGUS
Planning software without the price tag.
The problem
The planning software used in wealth management is powerful and expensive, and a student or a first-time investor never sees how it reaches its answers. ARGUS rebuilds those calculations so they can be inspected and used for free.
Decisions and trade-offs
| Decision | Why, and what it costs |
|---|---|
| Everything runs in the browser | Holdings never leave the user’s device, so there is nothing to breach. The cost: no saved accounts; portfolios travel as Excel files instead. |
| Holdings entered lot by lot | Taxes are calculated per lot, so cost basis and gains have to be tracked there. It takes longer to enter than a simple list of tickers. |
| A backtest on returns since 1928 | Damodaran’s long-run data includes the Depression, inflation and every major crash, not just the last decade’s bull market. |
| Only two features go online, and only when pressed | The price refresh and the assistant. Everything else works offline. |
What it taught me. A simulation is only as honest as its assumptions. Most of the work was deciding what to assume and showing it to the user, not the math. Try a piece of it.
03 LDB Lab
Testing AI models with code, not opinions.
The problem
Most people judge a chatbot by whether its answers sound right. That is the one test a fluent model always passes. The Evaluation Arena lets anyone run small open models through the same exam on their own device and see the results graded mechanically.
Decisions and trade-offs
| Decision | Why, and what it costs |
|---|---|
| Grade by code | Every question has a checkable answer (a number, a format, a refusal), so it is scored the same way every time. Open-ended quality is left out on purpose. |
| Sampling turned off | The same model gives the same answer every run, so scores can be compared and reproduced. |
| A fictional company in the passage | The questions and passage were written for the page, so no model can have memorized them. Two questions ask for figures the passage does not give: the right answer is to say so. |
| Hijack attempts included | Two questions hide an instruction to say “HACKED”. Prompt injection is the main risk once a model reads outside text. |
| Models run on the visitor’s device | Nothing is recorded or replayed, and it costs nothing to run. The models must be small, so they make mistakes, which is the point. |
What it taught me. Twenty-two questions is a demonstration, not a ranking: a model can pass a short test by luck. Real evaluations use thousands of items, several graders and repeated runs, and the page says so. Run the arena.