close
Skip to content

Support advanced EP configuration (provider options, config entries) in ONNX backend API #28209

@titaiwangms

Description

@titaiwangms

Background

PR #28083 added security allowlists for SessionOptions and RunOptions kwargs in the ONNX backend API (onnxruntime.backend). During review, @tianleiwu noted that the backend API currently has no mechanism for advanced runtime configuration channels.

Problem

The backend's prepare() function creates an InferenceSession but never passes provider_options. Similarly, session_config_entries and run_config_entries use method-based APIs (add_session_config_entry(), add_run_config_entry()) that don't map to Python properties and can't be set via the existing kwargs/setattr pattern.

This means features like CUDA graph (enable_cuda_graph for CUDA EP, trt_cuda_graph_enable for TensorRT EP) and per-run config (gpu_graph_id) are unreachable through the backend API.

Proposed Solution

Add support for structured configuration parameters:

  1. Provider options: New provider_options kwarg accepting a dict (e.g., {'CUDAExecutionProvider': {'enable_cuda_graph': True}}), with per-EP allowlists for security
  2. Session config entries: New session_config dict kwarg with allowlisted safe keys
  3. Run config entries: New run_config dict kwarg with allowlisted safe keys

Scope

  • Per-EP option allowlists need security auditing (some EP options can load libraries or write files)
  • Config entry key space is large and EP-specific — may need prefix-based allowlisting
  • Estimated ~200–400 lines of new code + tests
  • Files: onnxruntime/python/backend/backend.py, backend_rep.py, test file

Context

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions