Commit Graph

27529 Commits

Author SHA1 Message Date
Jingyu Zhou
e04c631eca jzhou77
Add log system debug ID to trace event
2024-11-12 08:40:47 -08:00
Johannes M. Scheuermann
786a220e35 Add log system debug ID to trace event 2024-11-12 14:32:24 +01:00
Zhe Wang
ca4ab1eca9 Fix traceTooManyEvents and externalTimeouts in BulkLoad test (#11769) 2024-11-11 11:05:43 -08:00
Jingyu Zhou
3f97fa046b Fix ASAN build errors (#11767)
* Fix ASAN build errors

This fixes https://github.com/apple/foundationdb/issues/10813

There are multiple problems:
- The boost 1.78 doesn't work well with ASAN build, which was fixed in later versions
- The compiling missed "context-impl=ucontext" flag for b2 command
- missing /usr/local/include/c++/v1/__config_site in the developer environment

* Use B2_ADDTTIONAL_BUILD_ARGS instead
2024-11-09 09:38:15 -08:00
Syed Paymaan Raza
aaba814512 Fix two cases of non-determinism in simulation (#11766) 2024-11-08 14:38:15 -08:00
Dan Lambright
31d67404a3 Do not choose two_satellite_fast in simulation tests when version vector unicast enabled. (#11765)
Co-authored-by: Dan Lambright <hlambright@apple.com>
2024-11-08 16:44:09 -05:00
Zhe Wang
85f293fe58 reduce bulkload test size (#11762) 2024-11-07 17:47:45 -08:00
Dan Lambright
4b3525f8a3 Do not test bulk loading and version vector simultaneously (#11759)
Co-authored-by: Dan Lambright <hlambright@apple.com>
2024-11-06 13:39:21 -08:00
Syed Paymaan Raza
7a5f61cc65 Address feedback in PR #11753 (#11757) 2024-11-05 20:30:40 -08:00
Zhe Wang
ab9ce0df15 Cherrypick recent DD changes from release-7.3 (#11754)
* [Release-7.3] TeamRedundant and TeamUnhealthy data moves choose best destination with probability (#11668)

* team redundant and unhealthy data moves can choose best dest with probability

* nits

* nits

* enable wantTrueBestIfMoveout

* fix getteam stuck

* [Release-7.3] Delay team remover when space pivot is low (#11665)

* [Release-7.3] Validate ServerTeam count per server in simulation (#11678)

* validate server team count in simulation

* change naming (not relevant to the PR title)

* address comments and add a new trace event BuildTeamsLastBuildTeamsFailed triggered when buildTeam failed
2024-11-05 18:52:40 -08:00
Syed Paymaan Raza
84fb8f843c Gray failure allows storage servers to complain (#11753) 2024-11-05 16:53:02 -08:00
Dan Lambright
5716e4e7c2 Do not check for PROXY_USE_RESOLVER_PRIVATE_MUTATIONS in rangeLockEnabled (#11752)
* Do not check for PROXY_USE_RESOLVER_PRIVATE_MUTATIONS in rangeLockEnabled

* Dont modify knob proxy_use_resolver_private_mutations in range lock tests

---------

Co-authored-by: Dan Lambright <hlambright@apple.com>
2024-11-05 12:13:18 -05:00
Syed Paymaan Raza
7d529ec724 Invalidate gray failure complaints from excluded processes (#11749) 2024-11-04 13:49:25 -08:00
Zhe Wang
0cf708b174 Fix accumulative checksum (#11751)
* fix accumulative checksum

* ss does not persist acs value per epoch

* nits
2024-11-04 13:15:26 -08:00
Jingyu Zhou
d41eabf7cb Update download page for documentation (#11750) 2024-11-04 10:38:23 -08:00
Dan Lambright
a63631f68c fix vv unicast recovery for streaming peek. (#11745)
Co-authored-by: Dan Lambright <hlambright@apple.com>
2024-11-03 12:43:44 -05:00
Zhe Wang
22249b46d3 make range lock compatibility clear (#11748) 2024-11-01 09:48:25 -07:00
Dan Lambright
317956ee14 disable version vector with range lock tests (#11746)
* disable version vector with range lock tests

* turn off rangeLock if versionvector is on (#11747)

---------

Co-authored-by: Dan Lambright <hlambright@apple.com>
Co-authored-by: Zhe Wang <zhe.wang@wustl.edu>
2024-10-31 21:07:01 -04:00
Zhe Wang
42e17d8bd1 BulkLoading Use RangeLock (#11741)
* use range lock in bulk load

* refactor BulkLoading workload and nits

* add background traffic

* nits

* address comments
2024-10-31 12:58:13 -07:00
Sreenath Bodagala
5bc2d7aeb3 Add 7.3.54 and 7.3.55 release notes (#11740)
* - Add 7.3.54 and 7.3.55 release notes

* - Address a review comment
2024-10-28 16:39:29 -07:00
Yao Xiao
f326c81f85 Upgrade rocksdb to 9.7.3. (#11735) 2024-10-28 12:26:04 -07:00
neethuhaneesha
64030de741 Max range deletions knob update to prevent OOMs. (#11738) 2024-10-28 11:45:44 -07:00
Yao Xiao
afbcf5ef5f Enable backward read in consistency checker. (#11721)
* Do backward reads in consistency checker.

* Add knob for read options in consistency checker.
2024-10-26 09:57:36 -07:00
Jingyu Zhou
aebd907314 Merge pull request #11733 from jzhou77/fix
Fix valgrind errors of uninitialised values
2024-10-26 09:44:44 -07:00
Syed Paymaan Raza
7fd213739b Urgent consistency checker fixes (#11734)
* Drop duplicate or conflicted requests from urgent consistency checker clients

* Fix edge case in urgent consistency check causing infinite loop

* fixup! Fix edge case in urgent consistency check causing infinite loop
2024-10-25 22:37:42 -07:00
Jingyu Zhou
3fc5c1dd03 Fix valgrind errors of uninitialised values 2024-10-25 15:41:56 -07:00
Zhe Wang
ee12b1616b disable range lock injection in range conflict workload (#11731) 2024-10-25 14:45:46 -07:00
Dan Lambright
2095baff55 check correct version in getMappedKeyValuesQ (#11729)
Co-authored-by: Dan Lambright <hlambright@apple.com>
2024-10-24 20:40:24 -04:00
Syed Paymaan Raza
36b113993c [gray_failure] Update CC_ONLY_CONSIDER_INTRA_DC_LATENCY knob documentation (#11728) 2024-10-24 07:48:37 -07:00
Zhe Wang
43446204ed Database Per-Range Lock (#11693)
* range lock framework

* improve the framework

* persist to txnStateStore

* fix bugs

* code clean

* code clean

* bug fix

* address comments

* add complex test workload and fix bugs found by the workload

* add workload correctness check and fix bugs

* code clean up

* add random range lock injection

* fix bugs in RandomRangeLock.actor.cpp

* enable random range lock injection in general workloads

* add rangelockcycle test

* disable random range lock in backup workloads

* nits

* add range lock ownership concept

* enable lock ownership to rangeLock

* api deal with tenant

* fix CI

* add test for multiple rangeLock owners

* nits

* address comments and renaming

* address comments
2024-10-23 16:25:56 -07:00
Syed Paymaan Raza
5f480947ad [fdbserver] Gray failure and simulator improvements related to remote processes (#11717)
* [fdbserver][simulator] Add remoteDesiredTLogCount option

* [fdbserver][simulator] Allow explicitly specifying number of stateless classes in each DC

* [fdbserver][gray_failure] RemoteTLog lagging SS simulation test

* [fdbserver][gray_failure] Consider remote processes + CC inter/intra latency awareness

* [fdbserver][cc] Make processInSameDC O(1)
2024-10-23 13:15:29 -07:00
Dan Lambright
a87e940e05 fix bug TxnStateStoreCycleTest for version vector (#11723)
* fix bug TxnStateStoreCycleTest for version vector

* Respond to review comment

---------

Co-authored-by: Dan Lambright <hlambright@apple.com>
2024-10-23 15:53:15 -04:00
Dan Lambright
9790a53f7c Propagate rv to tLogs on version vector recovery (#11677)
* refactor management of cluster recovery version (crv)

- send the crv to tlogs as they rejoin cluster during recovery, not
just when an rv is computed.
- if a tlog has not yet received the crv, wait for it

* Propagate cluster recovery version to tlog for version vector recovery
Enable RECORD_RECOVER_AT_IN_CSTATE to store recovery version in cstate

* respond to review comments

* - dont sent rv to old tlog generations
- send crv after locking, if rv computed

* Remove CRV RPCs, have cursor pad beyond end tlog to RV

* Send end version in peek request to facility vv recovery

* Fix serialization in peek request interface.

---------

Co-authored-by: Dan Lambright <hlambright@apple.com>
2024-10-22 17:54:26 -04:00
Doğan Çeçen
83d919e9a9 Fix actorcompiler target in CMake add_flow_target
This also fixes #11595 - if Unix Makefiles is chosen for CMake builds,
build was failing with:

```
make[2]: *** No rule to make target 'actorcompiler.exe', needed by 'flow/ActorCollection.actor.g.cpp'.  Stop.
```

I suspect it could have been a problem for Ninja as well since the issue
was due to race condition, but probably it didn't happened so far for
other unknown factors.

See this example in CMake add_custom_command documentation:

https://cmake.org/cmake/help/latest/command/add_custom_command.html#example-generating-files-for-multiple-targets

The correct target to depend on is `actorcompiler` for CMake to generate
the right dependency order, `${actor_exe}` is just a string that points to
the location of the actor compiler. See here:

4260bbb3c2/cmake/CompileActorCompiler.cmake (L26-L27)
2024-10-22 22:54:22 +08:00
Vishesh Yadav
5215eb61bd Merge pull request #11718 from vishesh/dummythreadpool-init-fix 2024-10-21 23:46:53 +05:30
walter
4260bbb3c2 Add missing headers (#11720) 2024-10-16 22:18:21 -07:00
Vishesh Yadav
809c3a86a7 Call IThreadReceiver::init() in DummyThreadPool 2024-10-16 10:26:53 +05:30
He Liu
f35c329f4b Added comments 2024-10-15 19:22:58 -07:00
He Liu
f7fe09c577 Ignore data move conflict on TSS in simulation. 2024-10-15 19:22:58 -07:00
Yao Xiao
7290369aac Use a single iterator pool for all physical shards. (#11699)
* Rewrite iterator pool.

* simulation fix
2024-10-15 17:28:54 -07:00
Syed Paymaan Raza
c146ee0869 [fdbserver] Use STL contains method and std::find for containment checks (#11702) 2024-10-15 11:40:02 -07:00
Vishesh Yadav
b4bad4c1d6 Merge pull request #11704 from vishesh/rdar-137053948
Log all the incoming connections
2024-10-14 10:10:38 +05:30
Jingyu Zhou
4a5924dca1 Merge pull request #11710 from sbodagala/main
Add 7.3.52 and 7.3.53 release notes
2024-10-11 10:35:05 -07:00
Sreenath Bodagala
a303a66693 - Add 7.3.52 and 7.3.53 release notes 2024-10-11 15:52:50 +00:00
Jingyu Zhou
7c6c8ae095 Merge pull request #11709 from yao-xiao-github/knob-main
Update sharded rocksdb knobs.
2024-10-10 15:11:43 -07:00
Jingyu Zhou
a44a30c2df Merge pull request #11708 from kakaiu/main-branch
Improve the probability that ShardedRocksdb is selected in simulation tests
2024-10-10 15:11:15 -07:00
Yao Xiao
6a87e6042f update knobs 2024-10-10 13:51:50 -07:00
Vishesh Yadav
95319e4d76 Format 2024-10-10 13:32:45 -07:00
Zhe Wang
ab93517da4 address comments 2024-10-10 13:11:16 -07:00
Zhe Wang
fcb0030883 add probability that Memory gets selected 2024-10-10 11:24:40 -07:00