Common Pitfalls and How to Avoid Them

Symptom: integerRange or integerEquals gate always fails, even when the LLM produces the right answer.

Cause: The integer and number predicates parse the entire output string. An output of "Score: 4" or "The answer is 4." fails because the whole string isn't a number.

Fix: Either:

1. Update the system prompt to output the bare number ("4" not "Score: 4").
2. Switch to jsonMatches with a schema like { properties: { score: { type: number, minimum: 4 } } }.
3. Switch to contains if you're looking for a marker in prose.

# Bad: LLM outputs "Score: 4" → parse fails
systemPrompt: "Score the answer."
if: { integerRange: [4, 5] }

# Good: LLM outputs just "4"
systemPrompt: "Score the answer. Output ONLY the number, nothing else."
if: { integerRange: [4, 5] }

# Also good: structured output
systemPrompt: 'Output JSON {"score": N}.'
if:
  jsonMatches: { properties: { score: { type: number, minimum: 4 } } }

Symptom: The ladder errors on the first loop iteration because an ingest field references a step that hasn't run yet.

Cause: Cross-loop references (loopRef: previous) fail on loop 0 because there is no previous loop. Same with self-references in sequential steps where node 1 tries to read a nonexistent previous node.

Fix: Add fallback: "" to fields that might not resolve on the first iteration.

# Bad: fails on loop 0 (no previous loop exists)
- name: PreviousAttempt
  type: ingest
  from: { stepId: draft, loopRef: previous }

# Good: empty string on loop 0, real value on loop 1+
- name: PreviousAttempt
  type: ingest
  from: { stepId: draft, loopRef: previous }
  fallback: ""

For sequential steps, use skipFirstNode: true instead of fallback::

- name: Previous
  type: ingest
  from: { stepId: refine, loopRef: current, nodeRef: previous }
  skipFirstNode: true      # suppresses the error for node 1

Symptom: The slate file stays empty even though the step runs and produces output.

Cause: slateWrite extracts JSON from the output and validates against match. If the LLM produces prose (no JSON) or JSON that doesn't match the schema, the write is a silent no-op — no error, no event, just nothing happens.

Diagnosis: Temporarily change match to { type: string } (accept anything). If the write succeeds, the schema was too strict. Look at what was written to understand what the LLM actually produced.

Fix: Align the match schema with what the LLM produces. Or update the system prompt to produce schema-conforming JSON.

# Debug: accept any output to see what the LLM produces
slateWrite:
  to: { slate: Debug, folder: raw, file: output.md }
  match: { type: string }
  on: overwrite

# Production: validate before persisting
slateWrite:
  to: { slate: Memory, folder: facts, file: core.md }
  field: fact
  on: append
  match:
    type: object
    required: [fact]
    properties:
      fact: { type: string, minLength: 5 }

Symptom: In a ToT pattern, evaluator node 3 seems to evaluate the wrong candidate.

Cause: Missing nodeRef: current on the ingest field. Without it, the field reads the step's combined output (all nodes joined), not the specific node.

Fix: Always declare nodeRef: current when you want parallel alignment.

# Bad: all evaluators see ALL candidates joined
- name: Candidate
  type: ingest
  from: { stepId: generate, loopRef: current }
  # missing nodeRef → reads combined output

# Good: evaluator N reads generator N
- name: Candidate
  type: ingest
  from: { stepId: generate, loopRef: current, nodeRef: current }

Symptom: The API response contains content from the wrong step.

Cause: The exit: field points to the wrong step. The returned output is always the exit step's output from the last completed loop.

Fix: Verify the exit: declaration. It must match the step ID that produces the final user-facing output.

# Bad: exit points to an intermediate step
exit: draft

# Good: exit points to the final formatting step
exit: answer

Also check: If a forward jump skips past the exit step, the output comes from the last step that actually ran. Trace the execution path to verify which step's output is returned.

Symptom: The ladder takes very long and eventually stops with a ceiling-exceeded error.

Cause: Backward jumps create loops. If the loop condition never changes (the critique always says "bad"), the loop runs until the hop ceiling is hit.

This is actually safe — the hop ceiling bounds total iterations. But it wastes budget. The fix is to make the loop condition achievable: eventually the critique should say "good."

Fix:

1. Ensure the critique step can actually produce a "good" verdict. If the gate is too strict, it never passes.
2. Use knobs to let the caller control the threshold.
3. Add a maxLoops parameter to bound the loop externally.

# The critique gate should be achievable
- id: critique
  if:
    jsonMatches:
      type: object
      properties:
        verdict: { type: string, enum: [good] }
    then: continue
    else: { jump: { stepId: draft } }

# The caller can control how many loops to allow
curl ... -d '{
  "maxLoops": 3,
  "knobs": { "strictness": 0.7 }
}'

Symptom: Validation error: groups cannot be nested.

Cause: Group steps cannot contain other group steps as children. This is a structural constraint.

Fix: Flatten the structure or use cross-ladder jumps for deeper composition.

# Bad: nested groups
- id: outer
  type: group
  steps:
    - id: inner          # ← cannot be a group
      type: group
      steps: [...]

# Good: flatten into one group
- id: all_reviewers
  type: group
  steps:
    - id: security
      # ...
    - id: style
      # ...
    - id: correctness
      # ...

# Or: use cross-ladder jumps for deeper composition
- id: delegate
  if: { ... }
  then: { jump: { ladderId: "@alice/specialist-ladder" } }