mirror of
https://github.com/apple/foundationdb.git
synced 2026-01-25 12:28:19 +00:00
a9cc2dd7dc
* Add multipart retry delay configuration. (#12076) Add documentation to .h file. Co-authored-by: stack <stack@duboce.com> * Add a quickstart section on the bulkdump/bulkload feature to the user doc. (#12090) * Fix assertion failure in fdbcli (#12095) * Add Option to Turn Off Failure Injection to BulkLoad Process (#12096) * add option to turn off failure injection to bulkload process in simulation * nit * DD BulkLoadTask Load Balance (#12103) * bulkload load balance * improve bulkload task submission efficiency * address comments * address comments * cleanup * Take a --proxy argument as backup_agent does. (#12110) * Add "--proxy" to fdbserver (#12109) * add --proxy to fdbserver * fmt * address comments * use g_network to store proxy for s3client_ci * Add 'ls' to s3client for the case where 'aws s3' is not installable (#12118) * Add 'ls' to s3client for the case where 'aws s3' is not installable -- i.e. production -- and we want a tool to test s3 connectivity. Use it like this: ~/build_output/bin/s3client \ --tls-ca-file /etc/ssl/cert.pem \ --blob-credentials /path/to/credentials.json \ ls "blobstore://backup-us-west-2.s3.amazonaws.com/x?bucket=backup-us-west-2®ion=us-west-2" * fdbclient/S3BlobStore.actor.cpp * fdbclient/include/fdbclient/S3BlobStore.h Make constructResourcePath method on endpoint so accessible making ls URLs. Handling for encoded ampersands too. Allow 404 when requesting resource that doesn't exist. * fdbclient/S3Client.actor.cpp * fdbclient/S3Client_cli.actor.cpp Add in ls implementation. * fdbclient/tests/s3client_test.sh Add test for new ls facility. * See if this fixes compile issue --------- Co-authored-by: michael stack <stack@duboce.com> * Ingest sst files rather than their keyvalue content (2nd attempt) (#12108) * Ingest sst files rather than their keyvalue content. * fdbclient/ServerKnobs.cpp * fdbclient/include/fdbclient/ServerKnobs.h Add BULK_LOAD_USE_SST_INGEST knob. * fdbclient/include/fdbclient/IKeyValueStore.actor.h Add ingestSSTFiles and supportSStIngestion. * fdbserver/KeyValueStoreRocksDB.actor.cpp * fdbserver/KeyValueStoreShardedRocksDB.actor.cpp Implement ingestSSTFiles and supportSStIngestion * fdbserver/storageserver.actor.cpp If BULK_LOAD_USE_SST_INGEST and BulkLoadType::SST, ingest sst file rather than read keyvalues. * fdbclient/tests/fdb_cluster_fixture.sh Use rocksdb instead of sqllite in tests. * Revert "Ingest sst files rather than their keyvalue content." This reverts commit 6d50bf15d3566de17fa6febf2afba0f220846ca6. * ctests pass * Restore ingest minus metric * Add back ingestDuration metric * Formatting --------- Co-authored-by: michael stack <stack@duboce.com> * Allow resource to be empty (#12129) * RocksDB Do Compact After BulkLoad (#12130) * rocksdb bulkload compact after load * address comments * address comments * job overwrite task root path (#12138) * BulkLoad Load Balance Based on DataMove Busy Map (#12137) * bulkload load balance in ddqueue * cleanup * address comments * nit * bug fix * nit * address comments * avoid scheduler busy loop * address comments * address comments * avoid busy loop (#12157) * First set of fixes... sizes and implement deleteResource (#12156) Co-authored-by: michael stack <stack@duboce.com> --------- Co-authored-by: Michael Stack <saintstack@users.noreply.github.com> Co-authored-by: stack <stack@duboce.com>