Use Cases

A ladder is the right tool when at least one of the following holds:

• The task rewards search. Math, logic, planning, code analysis, or any domain where multiple solution paths exist and bad paths can be pruned. Tree-of-Thoughts ladders express this directly: draft N solutions, score each, prune low scorers, synthesize survivors.
• The task rewards critique. Drafting, code generation, structured output. A draft-critique-refine ladder produces tighter output than one-shot generation because the critique step catches errors the drafting step missed.
• The task rewards verification. Fact-checking, hypothesis testing, design review. Multiple nodes independently evaluate a claim; a synthesizer merges their verdicts.
• The task rewards iteration. Reflexion-style loops where the model critiques its own output and tries again. Convergence is observed empirically per task; LADR does not guarantee it.
• The task benefits from persistent state. Slates let a ladder remember across calls without re-stuffing context. Workflows that accumulate knowledge (per-repo code review notes, per-customer support history, per-deal investment memos) compose naturally.
• The task benefits from caller-tunable trade-offs. Knobs let the caller pick branch count, iteration depth, or recursion depth per call. One config serves both fast draft and high-stakes final uses.

The common criterion: the cost of running more LLM calls is less than the cost of shipping a worse answer. A ladder that runs 6 LLM calls at 3 seconds each costs ~18 seconds of latency and ~6x the token spend of a one-shot call. Whether that trade is profitable depends on the workload.

Do not use a ladder for:

• Simple Q&A. Most factual questions are answered correctly by a one-shot LLM call. A ladder adds latency and cost without improving the answer.
• Streaming chat. LADR is not designed for token-by-token chatbot UX. Use the direct POST /v1/chat/completions endpoint instead.
• Vector-RAG pipelines. If the workflow is "embed query, search vector store, stuff context, one LLM call", LADR adds little. Slates are a structured document store, not a vector index. A planned update adds an embedding system metatag that enables semantic retrieval via the existing retrieve interface; until then, LADR does not replace vector RAG.
• Real-time request paths. A 5-node ladder with a synthesis step is at least 6 LLM calls deep. At 1-3 seconds per call, the floor latency is 6-18 seconds. Do not place a ladder in a path with a sub-second latency budget.
• Custom tool use. LADR steps cannot call arbitrary Python functions or external APIs. If the workflow requires inline tool calls (database queries, REST calls, file I/O during reasoning), use an imperative framework such as LangChain.
• Model fine-tuning. LADR runs at inference time. It does not modify weights. Use a training pipeline for behavioral changes that need to be baked in.

Research and analysis. A ladder that takes a research question, drafts N independent analyses in parallel, scores each on evidence and rigor, prunes the bottom 60%, and synthesizes the survivors into a briefing. Latency: four LLM rounds (draft, score, prune-synthesize, final). Quality lift: empirically observable on multi-faceted questions where one-shot answers collapse to a single viewpoint.

Code review. A ladder that ingests a unified diff, runs a security-focused reviewer and a style-focused reviewer as parallel nodes, gates each on severity thresholds, and merges surviving findings into a structured report. A slate persists a per-repo lessons-learned log so recurring issue patterns are documented.

Customer support triage. A ladder that classifies an inbound message, generates two independent resolution drafts, evaluates each against policy and tone, refines the winner, and writes the resolution plus a postmortem entry to a slate so future similar tickets surface the prior resolution via retrieve.

Investment memo drafting. A ladder that takes an investment thesis, generates bull and bear cases in parallel, runs a devil's-advocate critique on each, then synthesizes a balanced memo. A slate holds prior memos so the retrieve block can pull analogies by request.

Exam grading. A ladder that takes a student response and a rubric, scores independently across N nodes (self-consistency), drops outliers, returns the median score. Writes the scored response and final score to a slate for later calibration review.

Each of these maps directly onto the grammar primitives documented under Language: parallel nodes for fanout, if: gates for pruning, jump: for backward loops, slates for persistence, retrieve: for on-demand fetch.

A ladder's cost is predictable from its structure:

Ladder shape	LLM calls (typical)	Latency	Token cost vs one-shot
Two-step (draft + refine)	2	2-6s	~2x
ToT with 5 branches, gate, synthesis	7 (5+1+1)	4-10s	~5-7x
ToT + reflexion (1 retry)	14 (7x2)	8-20s	~10-14x
Self-consistency (10 nodes, median)	11 (10+1)	3-8s (parallel)	~8-10x
Crucible (5 branches, verify, reflect, refine)	15-25	15-40s	~15-25x

Latency floor: the sum of sequential step latencies. Parallel nodes within a step all run simultaneously, so a 5-node step takes the latency of one LLM call (plus a small dispatch overhead).

Cost floor: the sum of token costs across all nodes in all steps. Each LLM call is billed at the provider's normal rate.

When the trade-off is worth it: the marginal value of a better answer exceeds the marginal cost of more calls. For a customer-facing feature where wrong answers cost $X in churn, spending $0.05 per call on a ladder is cheap. For internal tooling where a one-shot answer is good enough, the ladder is unnecessary overhead.

Budget enforcement: the executionBudget.maxSpend field caps the worst case. Set it to the maximum you are willing to spend per call. The runtime kills execution if the cap is reached and returns the best-so-far output.

A practical path from "I have a problem" to "I have a working ladder":

1. Identify the bottleneck. What does the one-shot LLM output get wrong? Is it shallow? Confidently wrong? Missing a perspective? Inconsistent across calls?

2. Map the bottleneck to a strategy.

   • Shallow → ToT with synthesis (draft N independent analyses, merge)
   • Confidently wrong → reflexion (draft, verify, critique, retry)
   • Missing perspective → group with multiple reviewers (security, style, correctness)
   • Inconsistent → self-consistency (N nodes, majority vote)

3. Write the smallest version. Start with 3 nodes, no gates, no slates. Get the flow working end-to-end. The goal is to confirm the pipeline runs, not to get the best output.

4. Add the gate. Once the basic flow works, add a scoring step with a gate. Tune the threshold so the gate prunes roughly 30-50% of nodes. Too aggressive (prunes everything) or too lenient (prunes nothing) are both wrong.

5. Add memory (if useful). Does the ladder benefit from remembering past calls? If yes, add a slate and a slateWrite step. If no, skip this — slates add complexity.

6. Tune prompts. The highest-leverage changes are usually:

   • The draft step's system prompt (does it produce diverse outputs?)
   • The scoring step's system prompt (does it produce calibrated scores?)
   • The synthesis step's system prompt (does it extract and preserve the right information?)

7. Measure. Run 10-20 prompts through both the one-shot baseline and the ladder. Score them (programmatically or by human rating). Decide based on data.

See Foundry Workflow for the practical mechanics of authoring, testing, and publishing in Foundry.