Reviewing agent output

What dg check defs does and doesn't catch

dg check defs validates the definitions layer in the Dagster project: assets are correctly decorated, resources are registered, schedules reference valid jobs. If a definition fails to load, it reports an error with a traceback. Having a quick feedback loop is very handy but doesn't catch everything.

dg check defs doesn't check:

  • Whether the logic inside an asset function is correct
  • Whether the code follows the project's conventions (the factory pattern, the DuckDB path structure)
  • Whether the agent changed files you didn't intend it to change
  • Whether values are hardcoded when they should come from resources or env vars

Having the agent execute and troubleshoot

When using Dagster skills, the agent will use dg check defs quite often. That is because the operation is so quick and can help the agent ensure everything looks good before moving on to more expensive steps.

Agent flow 1

After you have code that passes dg check defs, you can ask the agent to run the newly created code.

/dagster-expert Run the three raw assets and confirm they materialize without errors.

Because the agent will launch everything from dg, it will have access to all of the logs and output of the execution. This way it will be able to tell if code can execute successfully.

Agent flow 2

If there is an error while executing the code, it can take the same iterative approach of editing code, ensuring dg check defs and executing the code until the run is successful.

Agent flow 3

Comparing against the reference implementation

If you want to see one way the lesson code can look, the course repository includes a completed reference implementation for each lesson with code:

dagster_university/ai_driven_data_engineering/src/ai_driven_data_engineering/completed/lesson_4/

Because AI-generated code is non-deterministic, your output will not match the reference exactly. The agent may choose different variable names, structure files differently, or use slightly different patterns. That is expected. The reference is a sanity check, not a specification. What matters is that dg check defs passes, the assets materialize, and the code does what you asked for.