Adoption Phases

A phased rollout reduces risk: validate on one workflow before generalizing, and generalize before publishing. Publishing a ladder before it is measured internally risks shipping a strategy that does not hold up; staying internal forever forgoes the distribution a public ladder enables.

A common progression is:

1. Validate on one workflow. Build one ladder. Measure the quality lift versus one-shot. Decide based on data.
2. Generalize internally. Once one ladder works, find the next two or three. Build a small library.
3. Publish. When the library is mature, expose the best strategies to external callers.

Each phase has a different goal and different success criteria. The rest of this page details each phase and how to measure it.

Select a single workflow where one-shot LLM output is visibly underperforming and where a multi-step strategy (ToT, draft-critique-refine, self-consistency) is known to help. Build one ladder in Foundry. Run it internally for at least two weeks.

Selection criteria for the workflow:

• Visibly underperforming one-shot. Don't pick a workflow where the existing one-shot output is already good. The lift won't be measurable.
• Multi-step helps. The task should reward search, critique, verification, or iteration. If a single LLM call is structurally adequate, LADR is the wrong tool.
• Bounded eval surface. You need to be able to score outputs (programmatically or via human review) on a fixed test set. Without scoring, "better" is subjective.
• Real production traffic. Don't pick a hypothetical workflow. Pick one with real users or real downstream consumers.

Measure three quantities:

1. Answer quality via subjective rating or programmatic evals on a fixed test set. Use the same eval set for the baseline (one-shot) and the ladder.
2. Latency at the 50th and 95th percentile. Ladders are slower than one-shot; quantify how much slower.
3. Cost per call (token spend plus provider pass-through). Ladders cost more per call; quantify how much more.

Compare against the existing one-shot baseline.

Decision criteria at end of Phase 1:

• If the lift is real on at least one named workflow, proceed to Phase 2.
• If the lift is not measurable, either:
  • The workflow does not reward multi-step reasoning. A ladder is the wrong tool here. Try a different workflow.
  • The strategy is misconfigured. Iterate on the gate, the synthesis step, or the system prompts before deciding.

Common Phase 1 starting points:

• A customer-support escalation path with shallow one-shot answers.
• An internal analysis pipeline that already uses manual multi-call iteration (an analyst runs a prompt, reads the output, refines, runs again — that's a ladder waiting to be written).
• A drafting task where a refine loop would help.
• A classification task where self-consistency would tighten precision.

Anti-pattern: skipping the measurement. Many teams ship a ladder based on vibes ("the outputs feel better"). Without a fixed eval set and a baseline comparison, you don't actually know. The measurement is the whole point of Phase 1.

Once one ladder is in production with measured lift, look for repeated patterns across workflows. Most teams end up with three to seven internal ladders.

Typical library composition:

• Analysis and synthesis. A multi-perspective ladder that drafts N independent analyses, prunes the weakest, synthesizes the survivors. Used for research, briefings, design review.
• Draft-critique-refine. A two-or-three-step ladder that drafts, critiques against rubric, refines. Used for prose generation, code generation, structured output.
• Self-consistency classifier. A high-node-count ladder with majority-vote synthesis. Used for classification tasks where precision matters.
• Domain specialist. One ladder specialized to the team's domain: legal review, code review, medical summary, financial analysis, etc.

Keep these private or unlisted. Phase 2 ladders are internal tools. Use the local/ namespace or unpublished hosted-private ladders. Don't publish yet; the goal of Phase 2 is to harden the strategies, not to distribute them.

Use knobs to serve multiple use cases from one config. A single draft-critique-refine ladder with knobs for branches and iterations can serve both fast draft (3 branches, 1 iteration) and high-stakes final (8 branches, 4 iterations). One config, two very different behaviors.

Use slates to share context within a workflow. A research ladder that persists findings to a slate composes naturally with a writing ladder that reads them. The slate is the shared state; the ladders are independent programs.

Success criterion: at least three internal teams are calling at least one ladder regularly, and the library appears in the team's default-tools list. When other teams come to you asking "do you have a ladder for X?", Phase 2 has succeeded.

Anti-pattern: too many ladders. A 30-ladder library is hard to maintain. Look for opportunities to consolidate: one ladder with knobs beats three separate configs. Most teams converge on 3-7 ladders total.

Publish a ladder to the public directory at library.redeo.io/@{author}/{name} when all of the following hold:

• The strategy is general-purpose. It works across many callers, not just your application. A ladder tightly coupled to your internal data shapes or your proprietary tools is not ready.
• The execution budget is sustainable for the published ceiling. A caller running 10,000 invocations should not exceed what you can afford. The platform's spend cap protects callers; it does not protect you from your own pricing decisions.
• You want external applications to use the strategy without integrating with you directly. Publishing is the right move when distribution matters more than exclusivity.

A published ladder is callable by anyone via POST /v1/{author}/{name}/chat/completions. The caller swaps one URL in their OpenAI client and immediately gets the strategy behind their existing prompts.

Choose the right visibility tier. See Publishing and Visibility for the full reference. Briefly:

• public-free — AGPL, full config disclosed. Best for research credibility and seeding the directory.
• public-paid — Creator sets price per call / per 1k / subscription. Best for monetizing useful strategies.
• hosted-private — Callable via API, never disclosed. Best for flagship IP you want to expose as a service.
• local — Never uploaded. For everything you don't want to share.

Success criterion: at least one external application is calling your ladder in production. Bonus: someone forks it and improves it.

Anti-pattern: premature publishing. A ladder you would not want to see called 100,000 times is not ready to publish. Iterate privately first. Once published, versions are immutable; you can publish new versions, but you cannot take an existing version back. Pinning callers expect reproducibility.

A typical LADR adoption involves three roles, often overlapping in small teams.

Author. Writes the ladder YAML. Usually a senior engineer or ML practitioner familiar with prompt engineering and reasoning strategies. Owns: the system prompts, the gate conditions, the step composition. Uses Foundry for visual editing and live tracing.

Operator. Deploys and runs the runtime. Configures provider connections, sets engine environment variables (ENGINE_MAX_LOOPS, etc.), monitors spend and traces. Owns: the runtime resource budget, the production provider keys, the observability dashboards.

Caller. Integrates the ladder into applications. Usually a backend or full-stack engineer. Owns: the API integration, the per-call knob values, the session lifecycle, the user-facing error handling.

In a small team, one person plays all three roles. In a larger org, the roles separate. The author publishes; the operator runs the platform; the caller integrates. The handoff between roles happens via the ladder's public page, its documented knobs, and its declared executionBudget.

Three categories of metric, all worth tracking from day one.

Quality metrics. The lift over the one-shot baseline. Measured via:

• Programmatic evals. A fixed test set with scoring rules. Run the eval set against baseline and ladder; compare scores.
• Human ratings. A blind side-by-side: raters see baseline and ladder outputs without knowing which is which, pick the better one. Statistical tie or win for the ladder is the goal.
• Downstream task performance. If the ladder feeds a downstream system (a search index, a classifier, a human reviewer), measure that system's output quality with and without the ladder upstream.

Cost metrics. The trade-off you're making.

• Token cost per call. Ladders cost more than one-shot. Quantify the multiplier.
• Latency per call. Ladders are slower. Quantify the floor (sum of step latencies) and the ceiling (with retries, reflexion, etc.).
• Cost per quality-adjusted call. A ladder that costs 5x more per call but produces 2x better answers may be worth it; one that costs 10x more for 1.2x better usually isn't.

Adoption metrics. Whether anyone uses the ladder.

• Calls per week. Phase 1: yours and a few teammates. Phase 2: dozens of internal callers. Phase 3: external callers.
• Distinct callers. How many different users or applications are invoking the ladder.
• Forks (Phase 3). Whether other authors find the ladder useful enough to remix.

Track all three categories. Quality without cost is unmoored; cost without adoption is irrelevant; adoption without quality is dangerous.

Ladder everything. A one-shot call is the right tool for one-shot questions. LADR pays for itself on hard problems, not on every call. If a workflow doesn't visibly benefit from multi-step reasoning, don't build a ladder for it.

Giant ladders. A 30-step ladder is hard to debug, hard to reason about, and usually indicates the strategy should be decomposed into smaller ladders that call each other via jump: { ladderId: ... }. Aim for ladders with 3-8 steps. Beyond that, decompose.

No execution budget. Shipping a ladder without executionBudget is shipping an open-ended spend commitment. Always declare executionBudget.maxSpend before publishing. The default $5 is generous for most ladders; tighten it for public free-tier listings.

Implicit memory. Do not attempt to "let the model remember" by stuffing context into prompts. Use slates. They are explicit, schema'd, inspectable, and visible in Studio. A prompt with 50K tokens of stuffed context is a bug, not a feature.

Premature publishing. A ladder you would not want to see called 100,000 times is not ready to publish. Iterate privately first. Use the local namespace; let real internal traffic find the failure modes; only publish when you'd stake your reputation on the output.

Dynamic steps without a clear termination story. Dynamic steps can rewrite the reasoning strategy at runtime. They are bounded by the hop and spend ceilings, but a dynamic-step ladder that loops on bad configs will burn its budget before producing useful output. Always pair dynamic: true with a strict if: { jsonMatches: ... } gate on the planner's output, and always declare an onInvalid: fallback action.

Skipping measurement. "The outputs feel better" is not a measurement. Build a fixed eval set, score baseline and ladder, compare. Without measurement, you don't know whether you're helping or hurting.

Treating ladders as prompt templates. A ladder is not a fancy prompt. It's a small program that happens to be written in YAML. The system prompts are inputs to the program, not the program itself. The structure (steps, gates, jumps, slates) is where the leverage is.