Search Strategies

Self-Consistency

N parallel drafts with no gating, followed by aggregation that extracts the answer from each and takes the majority or median. Reduces variance on tasks with a clear answer type.

Shape and when to use

Shape: N parallel draft nodes (no gating), extract answers, take majority or median.

When to use: tasks with high variance and a clear answer type. Math (numeric answer), multiple choice (letter answer), classification (label answer). Multiple independent attempts plus aggregation reduces variance.

Cost: N + 1 LLM calls (N drafts running in parallel, plus 1 aggregation). Cheaper than ToT (no evaluation round).

Difference from ToT: no gate, no pruning. All N nodes reach the aggregator. The aggregator extracts the answer (not the reasoning) and takes a majority vote.

Config sketch

yaml

knobs:
  samples:
    name: Samples
    type: nodes
    input: slider
    steps: [{ title: Few, value: 3, default: true }, { title: Many, value: 9 }]

steps:
  - id: draft
    type: normal
    nodes: samples
    fields:
      - { name: Context, type: text, from: input.context }
      - { name: Sample Number, type: nodeInfo }
    systemPrompt: |
      Answer the question. Show your reasoning.
      Sample number: {Sample Number}.

  - id: aggregate
    type: normal
    fields:
      - name: Samples
        type: multi_ingest
        from:
          - { stepId: draft, loopRef: current, nodeRef: accumulate }
    systemPrompt: |
      Several independent attempts at the same question follow.
      Identify the answer (not the reasoning) in each, then return
      the majority answer. If no majority, return the most defensible.

Variants and pitfalls

Median for numeric answers. For numeric or categorical answers, instruct the aggregator to emit JSON with the median value directly.

With scoring. Combine with ToT by adding a score-extraction step before aggregation. The aggregator then weights by score rather than counting equally.

Common pitfall: ambiguous answers. Majority vote converges to the most common phrasing, not the best answer. Self-consistency works when the answer is a discrete value the model can re-derive independently. It fails on open-ended generation where "same answer" is ill-defined.