He Liu 5e591552c4 Cherry-pick physical shard move related changes: #10583, #10962, #11056, #11057 (#11086) ...

* Allow applyUpdates in multiple batches. (#10583)

* Optimize physical shard move (#10962)

* Allow applyUpdates in multiple batches.

* Persist lastAppliedVersion together with a batch of updates.

* Fixed repeatedly applying the same fetched mutation.
Apply updates at different versions.

* Fixed race between removeDataShard and isInVersionedData.

* Ignore mutations earlier than `lastAppliedVersion`.

* Buggify the batch limit for applying updates.

* Implemented async load of updates.

* Fixed out-of-order version issue.

* Cleanup.

* Batch commit MoveInUpdates.

* Avoid popping unpersisted updates.

* Increment MoveInUpdates::lastAppliedVersion in MoveInUpdates::next().
Fixed MoveInUpdates::hasNext().

* Fixed loadUpdates start version.

* cleanup.

* Cleanup.

* fmt

* Commit MoveInUpdates regardless of MoveInShard status.

* Enable move restore.

* Disabled move restore and fallBackToAddingShard from ingestion failure.

* Resolved comments.

* Fixed leaks from other prs.

* Speed up physical shard move (#11056)

* Allow applyUpdates in multiple batches.

* Persist lastAppliedVersion together with a batch of updates.

* Fixed repeatedly applying the same fetched mutation.
Apply updates at different versions.

* Fixed race between removeDataShard and isInVersionedData.

* Ignore mutations earlier than `lastAppliedVersion`.

* Buggify the batch limit for applying updates.

* Implemented async load of updates.

* Fixed out-of-order version issue.

* Cleanup.

* Batch commit MoveInUpdates.

* Avoid popping unpersisted updates.

* Increment MoveInUpdates::lastAppliedVersion in MoveInUpdates::next().
Fixed MoveInUpdates::hasNext().

* Fixed loadUpdates start version.

* cleanup.

* Cleanup.

* fmt

* Commit MoveInUpdates regardless of MoveInShard status.

* Enable move restore.

* Disabled move restore and fallBackToAddingShard from ingestion failure.

* Get rid of persisting MoveInShard states between phases.

* Make updateMoveShardMetadata synchronous.

* Recovered code deleted accidentally.

* Cleanup.

* cleanup.

* Physical shard move tss (#11057)

* Refactored newDataMoveId() and decodeServerKeysValue().

* Enabled physical shard move for tss.

* Added unit test & cleanup.

* clean up test configs.

2023-12-05 12:33:19 -08:00

9.7 KiB

Raw Permalink Blame History

View Raw