Gcs csek support#29207
Open
kaveti wants to merge 2 commits intotrinodb:masterfrom
Open
Conversation
Previously, invoking newEncryptedInputFile, newEncryptedOutputFile, or encryptedPreSignedUri on CacheFileSystem threw UnsupportedOperationException from the default TrinoFileSystem interface methods. This prevented backing filesystems that support server-side encryption (e.g. S3 SSE-C, GCS CSEK) from being served through the coordinator file cache. Add overrides that forward the encrypted operations to the underlying filesystem and expire cached entries on writes, mirroring the behavior of the existing unencrypted overrides. Extract the AES-256 construction from randomAes256 into a private ofAes256(byte[]) helper so the "AES256" algorithm string is declared only once.
Google Cloud Storage supports customer-supplied encryption keys (CSEK), which allow users to provide their own AES-256 keys to encrypt objects at rest. This matches the existing S3 SSE-C feature in Trino. Add two new configuration properties to GcsFileSystemConfig: - gcs.encryption-key: Base64-encoded AES-256 key used to encrypt objects written to Google Cloud Storage. - gcs.decryption-key: Base64-encoded AES-256 key used to decrypt objects read from Google Cloud Storage. Typically set to the same value as the encryption key, except during key rotation. Separate encryption and decryption properties allow key rotation without rewriting all existing data: configure a new encryption key while keeping the old decryption key until all objects are re-encrypted. Additionally, expose internal extra-credential property names (internal$gcs_encryption_key / internal$gcs_decryption_key) so callers such as the Iceberg REST catalog can override the configured keys on a per-identity basis. Promote EncryptionKey.ofAes256(byte[]) to public visibility so GcsFileSystemFactory can construct keys from caller-supplied Base64 values without duplicating the algorithm string.
d247abe to
8884ee9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
raised this pr from review comment from
here: #28793 (comment)
Description
Add support for customer-supplied encryption keys (CSEK) to the GCS filesystem, mirroring the existing S3 SSE-C feature in Trino.
Configuration
Two new configuration properties are added to GcsFileSystemConfig:
gcs.encryption-key— Base64-encoded AES-256 key used to encrypt objects written to Google Cloud Storage.gcs.decryption-key— Base64-encoded AES-256 key used to decrypt objects read from Google Cloud Storage. Typically set to the same value asgcs.encryption-key, except during key rotation.Separate encryption and decryption properties allow key rotation without rewriting all existing data: operators can configure a new encryption key while keeping the old key as the decryption key until all objects are re-encrypted.
Additional context and related issues
Release notes
( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text: