Background
PR #28083 added security allowlists for SessionOptions and RunOptions kwargs in the ONNX backend API (onnxruntime.backend). During review, @tianleiwu noted that the backend API currently has no mechanism for advanced runtime configuration channels.
Problem
The backend's prepare() function creates an InferenceSession but never passes provider_options. Similarly, session_config_entries and run_config_entries use method-based APIs (add_session_config_entry(), add_run_config_entry()) that don't map to Python properties and can't be set via the existing kwargs/setattr pattern.
This means features like CUDA graph (enable_cuda_graph for CUDA EP, trt_cuda_graph_enable for TensorRT EP) and per-run config (gpu_graph_id) are unreachable through the backend API.
Proposed Solution
Add support for structured configuration parameters:
- Provider options: New
provider_options kwarg accepting a dict (e.g., {'CUDAExecutionProvider': {'enable_cuda_graph': True}}), with per-EP allowlists for security
- Session config entries: New
session_config dict kwarg with allowlisted safe keys
- Run config entries: New
run_config dict kwarg with allowlisted safe keys
Scope
- Per-EP option allowlists need security auditing (some EP options can load libraries or write files)
- Config entry key space is large and EP-specific — may need prefix-based allowlisting
- Estimated ~200–400 lines of new code + tests
- Files:
onnxruntime/python/backend/backend.py, backend_rep.py, test file
Context
Background
PR #28083 added security allowlists for
SessionOptionsandRunOptionskwargs in the ONNX backend API (onnxruntime.backend). During review, @tianleiwu noted that the backend API currently has no mechanism for advanced runtime configuration channels.Problem
The backend's
prepare()function creates anInferenceSessionbut never passesprovider_options. Similarly,session_config_entriesandrun_config_entriesuse method-based APIs (add_session_config_entry(),add_run_config_entry()) that don't map to Python properties and can't be set via the existing kwargs/setattr pattern.This means features like CUDA graph (
enable_cuda_graphfor CUDA EP,trt_cuda_graph_enablefor TensorRT EP) and per-run config (gpu_graph_id) are unreachable through the backend API.Proposed Solution
Add support for structured configuration parameters:
provider_optionskwarg accepting a dict (e.g.,{'CUDAExecutionProvider': {'enable_cuda_graph': True}}), with per-EP allowlists for securitysession_configdict kwarg with allowlisted safe keysrun_configdict kwarg with allowlisted safe keysScope
onnxruntime/python/backend/backend.py,backend_rep.py, test fileContext