Model debugging overview

MAX includes built-in debugging tools that help you diagnose common issues during model development, such as numerical accuracy problems, correctness bugs, performance bottlenecks, and hard-to-trace runtime errors. This page explains how to configure debugging options.

If you want to enable MAX's debugging defaults right away, you can set the environment variable MODULAR_DEBUG=sensible before running your model:

MODULAR_DEBUG=sensible max serve --model modularai/Llama-3.1-8B-Instruct-GGUF

See the DebugConfig API reference to review all debugging options.

When debugging happens

To configure debugging effectively, you must set debugging options at the correct stage of the model pipeline. Debugging options take effect at one of the following stages:

Graph build time: Set debugging options before you instantiate a Graph object. Options configured after graph creation don't apply.
Model build time: Set debugging options before you compile the graph. Debugging is integrated during compilation, so you must rebuild the model for any changes to take effect.
Run time: Set debugging options before you run the model. You can modify these options between runs without rebuilding an already compiled model.

This matters when using the Python API, since Python execution is order-dependent. Environment variables and config files load before any processes run, so they're always in effect when needed.

For the timing requirements of each debugging option, see the DebugConfig API reference.

Configure debugging options

You can configure debugging options two ways:

Using the Python API
Setting the MODULAR_DEBUG environment variable

If you configure the same debugging option using both strategies, the option set with the Python API takes precedence.

Environment variable and config file options use kebab-case names (nan-check), while the Python API uses the underscored form (nan_check).

Use the Python API

The Python API provides the most direct way to enable debugging options. Most options live on InferenceSession.debug except for source_tracebacks, which lives on Graph.debug because it's consumed during graph construction.

Debug options are process-wide, so set them on the class (rather than an instance) before you create the corresponding object:

from max.engine import InferenceSession
from max.graph import Graph

# Set source_tracebacks before graph construction
Graph.debug.source_tracebacks = True
g = Graph("my_model")


# Set before session.load() compiles the graph
InferenceSession.debug.nan_check = True
InferenceSession.debug.op_log_level = "trace"

session = InferenceSession()
model = session.load(g)

# Set before calling execute()
InferenceSession.debug.device_sync_mode = True
outputs = model.execute(inputs_a)

InferenceSession.debug.device_sync_mode = False
outputs = model.execute(inputs_b)

# Clear every option set through the Python API
# Values from MODULAR_DEBUG remain in effect
InferenceSession.debug.reset()

Set the `MODULAR_DEBUG` environment variable

Another option is to use an environment variable to enable your debugging options. This is the recommended way to set global options.

Set the MODULAR_DEBUG environment variable to a comma-separated list of debug option names. For options that take a value, use =.

export MODULAR_DEBUG=nan-check,device-sync-mode,source-tracebacks,assert-level=safe

See the Debugging section of the environment variables reference for every option you can pass to MODULAR_DEBUG.

Next steps

Now that you know how to set up debugging in MAX, review the full list of debugging options in the DebugConfig class. You'll find detailed specifications for each option (property) including the type and accepted values.

When debugging happens​

Configure debugging options​

Use the Python API​

Set the MODULAR_DEBUG environment variable​

Next steps​

When debugging happens

Configure debugging options

Use the Python API

Set the `MODULAR_DEBUG` environment variable

Next steps