Search Strategies

Tree-of-Thoughts

N parallel draft nodes followed by a per-node evaluation gate followed by synthesis of the survivors. The canonical LADR pattern for hard reasoning tasks.

Shape and when to use

Shape: N parallel draft nodes, per-node evaluation gate, synthesis of survivors.

When to use: hard reasoning, multi-angle analysis, any task where "draft many, pick the best, synthesize" beats "draft once." ToT is the default choice when there is no domain-specific pattern that fits better.

Cost: 2N + 1 LLM calls per loop iteration (N drafts + N evaluations + 1 synthesis). Latency is 3 LLM rounds deep (drafts run in parallel, evaluations run in parallel, synthesis is single-node).

Config sketch

yaml

knobs:
  branches:
    name: Branches
    type: nodes
    input: slider
    steps: [{ title: Low, value: 3, default: true }, { title: High, value: 7 }]

steps:
  - id: draft
    type: normal
    nodes: branches
    fields:
      - { name: Context, type: text, from: input.context }
      - { name: Branch Number, type: nodeInfo }
    systemPrompt: "Draft an independent answer. Use the Branch Number field above to vary your angle."

  - id: evaluate
    type: normal
    nodes: branches
    fields:
      - { name: Draft, type: ingest, from: { stepId: draft, loopRef: current, nodeRef: current } }
    systemPrompt: 'Output JSON: {"score": 1-5, "verdict": "strong"|"weak"}.'
    if:
      jsonMatches:
        type: object
        required: [score]
        properties:
          score: { type: integer, minimum: 4 }
        additionalProperties: true
      then: continue
      else: abort

  - id: synthesize
    type: normal
    fields:
      - name: Drafts
        type: multi_ingest
        from:
          - { stepId: draft, loopRef: current, nodeRef: accumulate }
    systemPrompt: "Synthesize the surviving drafts into one final answer."

The evaluate step uses nodeRef: current to enable per-node routing: each evaluation node reads the corresponding draft node. The synthesize step uses nodeRef: accumulate to read all surviving drafts as a joined block.

Variants

Scored ToT. Use jsonMatches gate on JSON-encoded scores (as in the config sketch). Accept only scores at or above a threshold. Default variant.

Verbal ToT. Use if: { equals: "1" } (or any string marker) instead of JSON scoring. Simpler prompt, less nuanced. Useful when the model is bad at structured scoring.

ToT with reflexion. Add else: { jump: { stepId: draft } } to retry on no-survivors instead of aborting. Combines with the Reflexion pattern (see Reflexion).

ToT with self-consistency. Drop the gate. Send all drafts to synthesis regardless of score. The synthesis step acts as a majority-vote aggregator. Useful when scoring is unreliable.

Common pitfalls

Setting branches too high. Cost grows linearly with N, while quality gains often plateau around 5–7 branches. 5 is a reasonable starting point; measure before raising it.
Trusting the evaluator blindly. The gate is only as good as the evaluator's prompt. Bad evaluator equals bad pruning equals bad synthesis. Iterate on the evaluator's system prompt as much as on the drafter's.
Synthesizing low-quality survivors. If the gate threshold is too low, weak drafts survive and pollute the synthesis. Raise minimum on the score field until the survivors are good.
Forgetting per-node routing. If the evaluator's ingest field omits nodeRef: current, every evaluator node reads draft node 1. That defeats the parallel evaluation.