AIProbe -- Understanding agent failures
A bug in the agent model, or a non-conducive environment configuration? AIProbe uses an independent planner to test whether a task can be completed at all.
Loading experiments…
Accurate agent modelDefective modelsOracle
Agent 1Agent 2Agent 3Oracle
Same starting positions, landmarks, and shared oracle in every panel. New evaluations of the four trained models; each pair pauses when its model finishes.
State defect: The agent cannot observe the top and bottom boundaries of the next two pipe gaps.
Reward defect: The reward for passing a pipe is removed, changing from +1 to 0.