MESHFLOW_MOCK=1 python3 hands_on/06_guardian_safety.pyThis lesson teaches how to debug AI workflows. Agentic systems fail differently from ordinary code because failures can hide inside context, memory, tool calls, or model output. Traces make those failures visible.
By the end, you should know what to inspect when a workflow gives a bad answer, stops unexpectedly, or silently uses the wrong information.
Traditional code often fails with a clear exception:
TypeError: expected string, got dictAI workflows can fail more quietly:
You need observability for both code behavior and reasoning inputs.
A trace is the execution record of a workflow run.
At minimum, a trace should show:
In a governed AI system, the trace is not optional. It is how you explain what happened.
When something goes wrong, ask these in order:
This order prevents guessing.
| Failure | Symptom | What To Inspect |
|---|---|---|
| Missing artifact | Later node cannot run | Node contracts and dependencies |
| Wrong context | Answer uses wrong facts | Rendered prompt or node input |
| Stale memory | Old decision appears | Memory retrieval and timestamps |
| Tool failure | Empty or malformed output | Tool input, output, error |
| Gate block | Workflow stops early | Gate condition and approval record |
| Hallucination | Unsupported claim | Evidence artifacts and citations |
| Looping | Repeated steps | Stop condition and retry count |
Run:
python3 -m src.mini_meshflow run examples/03_agent_with_gate.jsonThen answer:
This is the simplest debugging exercise in the course.
If a model gives the wrong answer, inspect the exact context it received.
Look for:
Beginner rule: do not debug the model first. Debug the assembled context first.
Memory problems usually come from retrieval quality.
Ask:
Bad memory can be worse than no memory because it can make the model confidently use old or irrelevant facts.
For every tool call, log:
Tool output should be treated as data, not truth. Validate it before relying on it.
When a gate blocks, ask:
A blocked gate is not automatically bad. It may be the safest outcome.
Production systems usually need more than console output:
The goal is not to collect everything forever. The goal is to collect enough to debug, improve, and prove what happened.
Observability explains a single run.
Evaluation measures quality across many runs.
Examples of evaluation:
Good teams use both. Traces explain incidents. Evaluations show trends.
Choose one example workflow and intentionally break it:
Run compile or run commands and inspect the output:
python3 -m src.mini_meshflow compile examples/02_tools_and_memory.json
python3 -m src.mini_meshflow run examples/03_agent_with_gate.jsonAfter each failure, write:
Mistake 1: Blaming the LLM immediately.
Correction: First inspect context, tools, memory, and artifacts.
Mistake 2: Logging only final answers.
Correction: Log node-level inputs, outputs, and decisions.
Mistake 3: Treating blocked gates as crashes.
Correction: Read the gate reason.
Mistake 4: Hiding tool calls.
Correction: Make tool input and output visible in traces.
Mistake 5: Evaluating by vibes.
Correction: Create rubrics and measure repeat runs.
Debugging AI workflows means reconstructing what the system saw, did, produced, and decided. Traces make that possible. Observability turns agentic systems from black boxes into inspectable engineering systems.
Run the gated workflow with the gate set to false:
python3 -m src.mini_meshflow run examples/03_agent_with_gate.jsonAnswer each question:
Create a checklist of fields you would want in a production AI workflow trace. Group your fields into two sections: run-level fields (one per run) and node-level fields (one per node execution).
Use this starter:
Run level:
[ ] run_id
[ ] ...
Per-node:
[ ] node_id
[ ] ...Aim for at least six fields in each section. Compare against answers.md after.
The repository contains audit run files in the root directory:
audit_run_480c.json
audit_run_5e9b.json
audit_run_97da.json
audit_run_e2d6.jsonOpen one of these files and answer:
This exercise builds the habit of reading traces as a primary debugging tool, not as a secondary log you check only when something breaks.
Open ../../examples/02_tools_and_memory.json and add a comment or note to one node describing exactly what you would log for that step. Include:
You do not need to modify the runner — just annotate the JSON with comments and explain your reasoning.
You run a workflow and the final artifact is empty. Plan your debugging session in five steps using only the trace:
Step 1: Look at ___
Step 2: If ___ then investigate ___
Step 3: Trace back to ___
Step 4: Check ___
Step 5: Fix ___ and re-runThis exercise is about building a systematic debugging habit. The fastest debuggers do not guess — they follow the data lineage backwards from the bad output to its root cause.