feat: add SHA256 verification for direct downloads by henryiii · Pull Request #2873 · pypa/cibuildwheel

henryiii · 2026-05-27T16:51:23Z

🤖 Human guided, AI assisted PR (using this skill). AI text below. 🤖

Summary

Implements SHA256 checksum verification for direct downloads in cibuildwheel, addressing #908. SHA256 hashes are stored in resource config files when running maintainer update scripts, and verified at download time to detect unexpected changes to artifacts.

What changed

Runtime verification

cibuildwheel/util/file.py: download() now accepts sha256: str | None = None; verifies hash after download and raises FatalError on mismatch

Platform wiring

SHA256 is now stored in PythonConfiguration dataclasses and passed through to download() on:

macOS: CPython, PyPy, GraalPy
iOS: CPython (python.org and BeeWare), including the macOS build Python used for cross-builds
Android: CPython (python.org and Chaquopy/Maven)
Windows: PyPy, GraalPy (CPython uses NuGet, no direct download)
virtualenv: virtualenv.pyz
python-build-standalone: used by the test runner

Update scripts

SHA256 sources vary by provider:

python.org CPython: sha256_sum field from the downloads API (free)
GraalPy: .sha256 sidecar assets from GitHub releases
python-build-standalone: SHA256SUMS file in the release
PyPy, BeeWare iOS, Chaquopy Android: no checksums provided upstream — computed by streaming download+hash on first run, then preserved when URLs are unchanged

Resource files

All URL-bearing entries in the following files now have populated sha256 fields:

cibuildwheel/resources/build-platforms.toml (macOS, iOS, Android, Windows PyPy/GraalPy)
cibuildwheel/resources/virtualenv.toml
cibuildwheel/resources/python-build-standalone-releases.json (179 assets, tag 20260510)

Review follow-up

Require sha256 for URL-backed Python and virtualenv configs instead of allowing empty / missing values
Add the two previously empty GraalPy macOS x86_64 SHA256 entries
Update the GraalPy pin refresh logic to select the newest release that actually contains a matching asset, so macOS x86_64 entries are refreshed instead of skipped when a newer tag omits that artifact
Add unit coverage for required SHA-bearing resource configs and the GraalPy asset-selection fallback

Assisted-by: copilot-cli:gpt-5.4

henryiii · 2026-05-27T17:12:18Z

+
+# Resolve path relative to this script so writes go to the source checkout,
+# not the uv-installed copy of the package.
+DIR: Final[Path] = Path(__file__).parent.parent.resolve()
+PYTHON_BUILD_STANDALONE_RELEASES: Final[Path] = (
+    DIR / "cibuildwheel/resources/python-build-standalone-releases.json"
+)


Unrelated fix.

agriyakhetarpal · 2026-05-27T17:48:26Z

Nice; I had started a local branch to take this up recently, but I did not go as far as Copilot has done here. Could we also add SHA256 checksums for Pyodide cross-build environments?

Copilot

Pull request overview

Adds SHA256 verification for direct downloads in cibuildwheel (addressing #908). Hashes are recorded in resource config files by the maintainer update scripts, then verified at download time; on mismatch, the partial file is removed and a FatalError is raised.

Changes:

download() in cibuildwheel/util/file.py gains a sha256 keyword and verifies the hash post-download.
PythonConfiguration dataclasses on macOS/Windows/iOS/Android/Pyodide and the python-build-standalone/virtualenv helpers carry and forward sha256 to download().
Update scripts (bin/update_pythons.py, bin/update_virtualenv.py, bin/update_python_build_standalone.py) populate sha256 from upstream metadata or by streaming the download when no upstream checksum exists; resource files are repopulated accordingly.

Reviewed changes

Copilot reviewed 12 out of 16 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
cibuildwheel/util/file.py	Adds SHA256 verification to the central `download()` helper.
cibuildwheel/util/python_build_standalone.py	Threads `sha256` through asset lookup and cached download.
cibuildwheel/venv.py	Passes `sha256` for virtualenv.pyz download.
cibuildwheel/platforms/macos.py	Adds `sha256` to config + install_cpython/pypy/graalpy.
cibuildwheel/platforms/windows.py	Adds `sha256` to config + install_pypy/graalpy.
cibuildwheel/platforms/ios.py	Adds `sha256` to config + target CPython install.
cibuildwheel/platforms/android.py	Adds `sha256` to config + target Python install.
cibuildwheel/platforms/pyodide.py	Adds `sha256` field to Pyodide config dataclass.
cibuildwheel/extra.py	Adds `sha256` to `PyodideXBuildEnvRelease` TypedDict.
cibuildwheel/resources/build-platforms.toml	Populates `sha256` for macOS/Windows/iOS/Android/Pyodide URL entries.
cibuildwheel/resources/virtualenv.toml	Adds `sha256` for the default virtualenv.pyz.
cibuildwheel/resources/python-build-standalone-releases.json	Bumps tag to 20260510 and adds `sha256` per asset.
bin/update_pythons.py	Sources SHA256 from CPython API / GraalPy sidecars; falls back to streaming hash.
bin/update_virtualenv.py	Computes/stores SHA256 when missing or version bumped.
bin/update_python_build_standalone.py	Parses `SHA256SUMS` and writes per-asset checksums; resolves output path relative to checkout.
docs/_internal/pyodide-maintenance.md	Updates Pyodide config example to include `sha256`.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

mhsmith · 2026-05-29T12:47:36Z

LGTM as far as Android is concerned.

mhsmith

Although, if the hashes are guaranteed to be present in build-platforms.toml, shouldn't we make them required at runtime? Otherwise, if a change to update_pythons accidentally removes the hashes in the future, we would silently lose the security.

This applies to all platforms.

agriyakhetarpal · 2026-05-29T05:03:44Z

+# Resolve path relative to this script so writes go to the source checkout,
+# not the uv-installed copy of the package.


I was just thinking about this yesterday when I found that [tool.uv.sources] isn't getting recognised in the PEP 723 metadata. Probably a good follow-up for nox's codebase; just noting it here.

I believe this is working as intended, this is a uv-specific field and would any be read by uv run, not nox?

We could require uv then use uv run inside nox.

Oh, what I meant was that nox itself could probably use this and delegate to uv run to run the scripts if nox[uv] is installed. More of a feature request, not a bug. Pretty sure I have a recent local branch lingering around somewhere for this; I'll put together a PR in nox a bit to elaborate.

Store SHA256 hashes when running update scripts and verify them when downloading files at build time. This improves security by detecting unexpected changes to downloaded artifacts. Platforms covered: macOS (CPython, PyPy, GraalPy), iOS, Android, virtualenv, and python-build-standalone. Windows (nuget) and Linux (Docker) are excluded. SHA256 sources per platform: - macOS/iOS/Android CPython (python.org): sha256_sum from API - GraalPy: .sha256 sidecar assets from GitHub releases - python-build-standalone: SHA256SUMS file in release - PyPy, BeeWare iOS, Maven (Chaquopy): stream-download and compute Changes: - cibuildwheel/util/file.py: add sha256 param to download() - cibuildwheel/platforms/{macos,ios,android}.py: add sha256 to PythonConfiguration and pass to download() - cibuildwheel/venv.py: read sha256 from toml and pass to download() - cibuildwheel/util/python_build_standalone.py: add sha256 to PythonBuildStandaloneAsset and pass to download() - cibuildwheel/resources/build-platforms.toml: add sha256 fields - cibuildwheel/resources/virtualenv.toml: add sha256 field - cibuildwheel/resources/python-build-standalone-releases.json: add sha256 - bin/update_pythons.py: compute/store sha256 per source strategy - bin/update_virtualenv.py: compute sha256 by streaming download - bin/update_python_build_standalone.py: parse SHA256SUMS file Closes pypa#908 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Assisted-by: copilot-cli:claude-sonnet-4.6

…ation - Add sha256 field to Windows PythonConfiguration (PyPy/GraalPy have direct download URLs on Windows too) - Pass sha256 to install_pypy() and install_graalpy() in windows.py - Fix update_pythons.py: handle empty sha256 from CPython API (older versions) by streaming download to compute it; fix condition to check 'not sha256' rather than 'not in dict' - Fix update_virtualenv.py: compute sha256 even when version unchanged but sha256 is empty (first-time population) - Fix update_python_build_standalone.py: resolve file path relative to the script itself (not the installed package) so writes go to source checkout, not the uv cache - Populate actual sha256 values by running all three update scripts Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Assisted-by: copilot-cli:claude-sonnet-4.6

Assisted-by: CopilotCLI:gpt-5.3-codex Signed-off-by: Henry Schreiner <henryfs@princeton.edu>

Co-authored-by: henryiii <4616906+henryiii@users.noreply.github.com> Signed-off-by: Henry Schreiner <henryfs@princeton.edu>

Require sha256 for URL-backed Python and virtualenv download configs. Update the GraalPy updater to refresh macOS x86_64 entries by selecting the latest release that still has a matching asset, and fill the two missing GraalPy checksums in build-platforms.toml. Assisted-by: CopilotCLI:gpt-5.4

Signed-off-by: Henry Schreiner <henryfs@princeton.edu>

Merge pyodide_test.py and python_build_standalone_test.py into a single unit_test/test_sha256.py since both test sha256-related behaviour. Assisted-by: opencode:glm-5

henryiii commented May 27, 2026

View reviewed changes

henryiii force-pushed the henryiii/feat/sha branch 3 times, most recently from 9da34d0 to f925e8c Compare May 29, 2026 02:36

henryiii mentioned this pull request May 29, 2026

Plan for cibuildwheel 4.0 #2828

Closed

henryiii marked this pull request as ready for review May 29, 2026 03:20

henryiii requested review from agriyakhetarpal, freakboy3742, hoodmane, mhsmith and ryanking13 as code owners May 29, 2026 03:20

henryiii requested a review from Copilot May 29, 2026 03:20

Copilot started reviewing on behalf of henryiii May 29, 2026 03:20 View session

Copilot AI reviewed May 29, 2026

View reviewed changes

Comment thread docs/_internal/pyodide-maintenance.md Outdated

Comment thread cibuildwheel/util/python_build_standalone.py

Copilot AI mentioned this pull request May 29, 2026

Address PR #2873 review feedback: verify cached PBS checksums and clarify Pyodide docs #2876

Closed

henryiii force-pushed the henryiii/feat/sha branch from f925e8c to 4040326 Compare May 29, 2026 04:36

mhsmith approved these changes May 29, 2026

View reviewed changes

mhsmith reviewed May 29, 2026

View reviewed changes

Comment thread cibuildwheel/platforms/android.py Outdated

Comment thread cibuildwheel/platforms/android.py Outdated

mhsmith reviewed May 29, 2026

View reviewed changes

Comment thread cibuildwheel/util/file.py

henryiii force-pushed the henryiii/feat/sha branch from 148feb7 to a62886d Compare May 29, 2026 14:21

agriyakhetarpal approved these changes May 29, 2026

View reviewed changes

agriyakhetarpal linked an issue May 29, 2026 that may be closed by this pull request

Add hashes for Python downloads #908

Closed

henryiii and others added 6 commits May 29, 2026 13:31

fix: also include pyodide

eb36f09

Assisted-by: CopilotCLI:gpt-5.3-codex Signed-off-by: Henry Schreiner <henryfs@princeton.edu>

fix: PR review comments for cache verification and docs wording

fd69bb4

Co-authored-by: henryiii <4616906+henryiii@users.noreply.github.com> Signed-off-by: Henry Schreiner <henryfs@princeton.edu>

ci: remove unit test for bin item

b79f61e

Signed-off-by: Henry Schreiner <henryfs@princeton.edu>

refactor: combine sha256 unit tests into test_sha256.py

219e188

Merge pyodide_test.py and python_build_standalone_test.py into a single unit_test/test_sha256.py since both test sha256-related behaviour. Assisted-by: opencode:glm-5

henryiii force-pushed the henryiii/feat/sha branch from 4e5804f to 219e188 Compare May 29, 2026 17:39

henryiii merged commit cd38ee1 into pypa:main May 29, 2026
46 checks passed

henryiii deleted the henryiii/feat/sha branch May 29, 2026 19:02

agriyakhetarpal mentioned this pull request May 29, 2026

Add a variant of a nox script runner that entirely delegates to uv run wntrblm/nox#1095

Open

henryiii mentioned this pull request Jun 2, 2026

4.0 pre-release AI review #2885

Closed

Navigation Menu

Uh oh!

feat: add SHA256 verification for direct downloads - #2873

feat: add SHA256 verification for direct downloads#2873
henryiii merged 7 commits into
pypa:mainfrom
henryiii:henryiii/feat/sha

henryiii commented May 27, 2026 •

edited

Loading

Uh oh!

henryiii May 27, 2026 •

edited

Loading

Uh oh!

agriyakhetarpal commented May 27, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

mhsmith commented May 29, 2026

Uh oh!

mhsmith left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

agriyakhetarpal May 29, 2026

Uh oh!

henryiii May 29, 2026

Uh oh!

henryiii May 29, 2026

Uh oh!

agriyakhetarpal May 29, 2026

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

		# Resolve path relative to this script so writes go to the source checkout,
		# not the uv-installed copy of the package.

Uh oh!

Conversation

henryiii commented May 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What changed

Runtime verification

Platform wiring

Update scripts

Resource files

Review follow-up

Uh oh!

henryiii May 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

agriyakhetarpal commented May 27, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

mhsmith commented May 29, 2026

Uh oh!

mhsmith left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

agriyakhetarpal May 29, 2026

Choose a reason for hiding this comment

Uh oh!

henryiii May 29, 2026

Choose a reason for hiding this comment

Uh oh!

henryiii May 29, 2026

Choose a reason for hiding this comment

Uh oh!

agriyakhetarpal May 29, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

henryiii commented May 27, 2026 •

edited

Loading

henryiii May 27, 2026 •

edited

Loading