Description
Given Configurable encryption has been checked in and being tested via
simulation for more than a month and also to avoid penalty of accessing
KNOBS in inline commit path, patch retires the KNOB and make
ConfigurationEncryption default EaR mode for FDB.
BlobCipher still supports the old format header and encryption semantics,
will remove the dead code as a followup PR.
Testing
devRunCorrectness - 100K
* EaR: reduce metrics logging
BlobCipherMetrics used to break down by usage types (whehter it is for tlog, redwood, backup, etc), and these counters will be printed to trace log even when encryption is not enabled, or the specific usage is not happening on a node (e.g. a node with only stateless roles will also print blob cipher counters for redwood). We are reducing the BlobCipherMetrics loggings by:
1. Default to not breakdown the metrics by usage type, and the behavior is controlled by the knob `ENCRYPT_KEY_CACHE_ENABLE_DETAIL_LOGGING`
2. When the detail breakdown is enabled, the counters are lazily initialize
3. Even if the counters are initialized, they will not be logged if the count is 0 (so like if a node was recruited as tlog but then drops the tlog role later on, the tlog counter inside BlobCipherMetrics will not be logged anymore).
* buggify BlobCipherMetrics detail logging knob
* format
* EaR: Add test case to validate decryption with invalid key
Description
Extend BlobCipher unit test to provide coverage for the scenario
where buffer got encrypted with a EncryptionKey K, however,
decryption for some reason got attempted with K'.
Testing
EncryptionUnit.toml - 100K
* EaR: Add test case to validate decryption with invalid key
Description
Address review comments
Testing
Description
Patch removes an unused CODE_PROBE checking the encryption header
being read flag version is valid, given the flag-version is determined
by peeking into std::variant index and we only have version-1 supported,
for now converted the check to an ASSERT
Testing
EncryptionUnitTests.toml
EncryptionOps.toml
BlobGranuleCorrectness/Clean.toml
* EaR: Implement Key Check Value semantics
Description
Key Check Value (KCV) is a checksum of cryptographic encryption key
used to validate encryption keys's integrity. FDB Encryption at-rest
relies on external KMS to supply encryption keys.
Patch proposes following major changes:
1. Implement Sha256 based KCV implementation to protect against
'baseCipher' corruption in two possible scenarios:
a) potential corruption external to FDB
b) potential corruption within FDB processes.
2. Scheme persists computed KCV token in block encryption header,
which then gets validated as part of header validation during
decryption.
3. FDB Encryption key derivation uses HMAC_SHA256 digest generation
scheme, which allows max 64 bytes of 'cipher buffer', patch add
required check to ensure 'baseCipher' length are within bounds.
OpenSSL HMAC underlying call ignores extra length if supplied, however,
it weakens the security guarantees, hence, disallowed.
Testing
devRunCorrectness - multiple 500K runs
Valgrind & Asan - BlobCipherUnit, RESTKMSUnit, BlobGranuleCorrectness*,
EncryptionOps, EncryptKeyProxyTest
* EaR: Fix BlobCipher cache handling for cipher needs refresh and/or expired
Description
Patch proposes BlobCipher cache bug related to handling of cipherKeys
that either 'needsRefresh' and/or 'expired'
Also, adds a unit-test to cover the following usecase:
1. Test refreshAt and expireAt properties of the cipherKey
2. Validate corresponding Counter value increments
Testing
Extend /blobCipher unitest tests
* EaR: Configurable encryption support for TLog mutations
Description
diff-1 : Address review comments
Major changes includes:
1. Update the code involved in ensuring Tlog mutation encryption to be
compliant with "configurable encryption" feature.
2. Update ENABLE_CONFIGURABLE_ENCRYPTION flag to be 'true' by default
and BUGGIFY it.
Testing
devRunCorrectness - 100K
* EaR: Update encryption methods to make 'cipherHeaderKey' optional
Description
diff-1: Address review comments
Major changes includes:
1. Update BlobCipher Encrypt/Decrypt classes to make 'headerCipher' optional
2. Update GetEncryptionCipherKeys actor methods to make 'headerCipherKey' optional
3. Update the usage across all encryption participant methods
Testing
BlobCipherUnitTest
EnryptedBackupCorrecctness
BlobGranuleCorrectness*
devRunCorrectness - 100K
Changes:
1. Make binary serializer natively support `std::variant`. Serialize size is 1 byte (the type index, i.e. `std::variant::index()`), plus the serialize size of the actual type stored in the `std::variant`. Update `BlobCipherEncryptHeaderRef` to use the `std::variant` binary serializer
3. Remove `flagsVersion` and `algoHeaderVersion` from `BlobCipherEncryptHeaderRef`. The former is replaced by `flags.index() + 1`, and the latter is moved into each of the algorithm-specific sub-headers. Each sub-header types will have nesting version-specific subtypes to handle serialization of that specific version (e.g. for `AesCtrNoAuth` it has a `AesCtrNoAuthV1` subtype).
Commit proxy needs to fetch additional cipher keys post-resolution, since tenant ids for raw access requests and cross-tenant clear ranges are calculated after resolution.
Description
Set `enable_configurable_encryption` knob in the unit test to make
RandomUnitTest runs happy
Testing
BlobCipherUnitTest
EncryptionOps
RandomUnitTest
* EaR: Configurable encryption framework
Description
EaR implementation only supports fixed size on-disk encryption header format.
One drawback of the scheme is, introducing a newer encryption scheme as well
as updating header format in future may incur data migration restrictions.
Major changes proposed in the patch includes:
1. Flexible Encryption header format allowing the following:
1.1. Header flags (metadata) can evolve separately from the encryption algorithm
1.2. Specific encryption algorithm header to allow future extensions.
2. Update the BlobCipher encryption/decryption util classes to work with newer
encryption header format.
3. Continue supporting multiple encryption authentication schemes such as:
HMAC-SHA and AES-CMAC; also, supports no encryption-authentication schemes.
4. Refactor BlobCipher unit test to enable testing of new format.
5. Configuration knobs to control encryption header flags and algorithm
versions.
Note:
The on-disk header storage footprint savings due to the newer scheme is as follows:
1. No encryption authentication: 54% smaller compared to existing implementation.
3. AES-CMAC: 16% smaller compared to existing implementation.
3. HMAC-SHA encryption authentication: almost same size.
Testing
BlobCipherTest
EncryptionOpsTest
Changes:
1. Change `isEncryptionOpSupported` to not check against `clientDBInfo.isEncryptionEnabled`, but instead against ENABLE_ENCRYPTION server knob. The problem with clientDBInfo is before its being broadcast to the workers, its content is uninitialized, during which some data (e.g. item 2) is not getting encrypted when they should.
2. Fix CommitProxy not encrypting metadata mutations which are recovered from txnStateStore
3. Fix KeyValueStoreMemory (thus TxnStateStore) partial transaction coming from recovery is not encrypted
4. new CODE_PROBE for the above fixes
5. Logging changes
Description
The current code supports two modes for authentication encryption:
1. Single auth-token mode
2. Multi auth-token mode
The code currently uses SingleAuthToken mode only, the multi-auth
token mode was added to support large file (such as backup files)
encryption assisting encryption authentication by allowing reading
header independently from encrypted payload itself. However, the
backup files are organized as 'chunks' and every chunk is encrypted,
hence, MultiToken mode isn't used in the code.
Removing the usage saves 32 bytes per encryption header, which might
translate to decent storage saving, further, allows lesser encryption
header overhead when encrypting small Key-Value mutations.
Testing
BlobCipher unittests
EncryptionOps.toml
Description
Major changes include:
1. Eliminate memcpy due to authToken
2. Update TraceEvent message format
3. Remove redundant code.
Testing
flow/BlobCipher unit test
EncryptionOps.toml
devRunCorrectness - 100k
* Introduce "default encryption domain"
Description
In current FDB native encryption data at-rest implementation,
an entity getting encrypted (mutation, KV and/or file) is categorized
into one of following encryption domains:
1. Tenant domain, where, Encryption domain == Tenant boundaries
2. FDB system keyspace - FDB metadata encryption domain
3. FDB Encryption Header domain - used to generate digest for
plaintext EncryptionHeader.
The scheme doesn't support encryption if an entity can't be categorized
into any of above mentioned encryption domains, for instance, non-tenant
mutations are NOT supported.
Patch extend the encryption support for mutations for which corresponding
Tenant information can't be obtained (Key length shorter than TenantPrefix)
and/or mutations do not belong to any valid Tenant
(FDB management cluster data) by mapping such mutations to a
"default encryption domain".
TODO
CommitProxy driven TLog encryption implementation requires every transaction
mutation to contain 1 KV, not crossing Tenant-boundaries. Only exception to
this rule is ClearRange mutations. For now ClearRange mutations are mapped
to 'default encryption domain', in subsequent patch appropriate handling
for ClearRange mutations shall be proposed.
Testing
devRunCorrectness - 100k
Adding the following metrics:
* BlobCipherKeyCache hit/miss
* EKP: KMS requests latencies
* For each component that using encryption, they now need to pass a UsageType enum to the encryption helper methods (GetEncryptCipherKeys/GetLatestEncryptCipherKey/encrypt/decrypt) and those methods will help to log get cipher key latency samples and encryption/decryption cpu times accordingly.
