Commit Graph

136 Commits

Author SHA1 Message Date
Syed Paymaan Raza
c3e7542cda Update end year in copyright header 2024-08-02 09:40:11 -07:00
Lukas Joswiak
890e7610ad Serve no-op configuration database interface when disabled
By continuing to serve the configuration database interface used by
clients even when disabled, clients no longer need to specify a special
`--no-config-db` option when running commands which talk to the
configuration database.
2024-07-08 13:56:03 -07:00
Dimitris Apostolou
a88114c222 Fix typos 2024-02-07 01:16:00 +02:00
Lukas Joswiak
8b8da372d5 Fix issue with inconsistent coordinator disk queue
An inconsistent coordinator disk queue could cause repeated reboots of
the coordinator process. In simulation, this could cause stuck runs if
another role was running on the coordinator process, as the other role
would also repeatedly reboot (specifically stateful roles like tlogs).

See the comment included in this commit for much more detail.
2023-06-23 15:37:31 -07:00
Evan Tschannen
ef682d304e fix IKeyValueStore include 2023-06-16 13:28:40 -07:00
Evan Tschannen
3dd86d6c22 move IKeyValueStore.h to the client 2023-05-10 15:41:47 -07:00
Sreenath Bodagala
fe4a839600 - Do not add fdbserver processes to the client list. (#9225)
Note: Server processes started getting reported as clients since 7.1.0
(not sure if this change in behavior was intentional or not), and this
breaks the operator upgrade logic.
2023-01-25 14:39:32 -05:00
Nim Wijetunga
10ccaa1ee5 remove client info encryption state (#9096) 2023-01-06 17:14:06 -05:00
FoundationDB CI
86d6106dc1 format source code after switch to clang 15 2022-12-08 17:26:45 +00:00
sfc-gh-tclinkenbeard
ec615181ce Mark several more methods const 2022-11-15 14:57:32 -08:00
Markus Pilman
550488b020 Merge remote-tracking branch 'origin/main' into bugfixes/open-for-ide
# Conflicts:
#	bindings/c/CMakeLists.txt
#	fdbclient/include/fdbclient/GetEncryptCipherKeys.actor.h
#	fdbserver/BackupWorker.actor.cpp
#	fdbserver/BlobWorker.actor.cpp
#	fdbserver/CommitProxyServer.actor.cpp
#	fdbserver/KeyValueStoreMemory.actor.cpp
#	fdbserver/StorageCache.actor.cpp
#	fdbserver/include/fdbserver/GetEncryptCipherKeys.actor.h
#	fdbserver/storageserver.actor.cpp
#	fdbserver/workloads/PhysicalShardMove.actor.cpp
#	flow/CMakeLists.txt
2022-10-04 18:27:48 -06:00
Markus Pilman
97dfc6823f fixed build with OPEN_FOR_IDE 2022-10-04 17:01:02 -06:00
A.J. Beamon
e1fe28b78b Switch some usages of LiteralStringRef to use the _sr suffix 2022-09-30 16:04:16 -07:00
A.J. Beamon
4fd64630e8 Convert literal string ref instances to use _sr suffix 2022-09-19 11:35:58 -07:00
Lukas Joswiak
cd2bbffa4c Add flag to disable the configuration database
The `--no-config-db` flag, passed to `fdbserver`, will disable the
configuration database. When this flag is specified, no `ConfigNode`s
will be started, the `ConfigBroadcaster` will not be started, and on a
coordinator change no attempt will be made to lock `ConfigNode`s.
2022-09-13 16:53:54 -07:00
Lukas Joswiak
74ac617a34 Add support for changing coordinators to the configuration database
Configuration database data lives on the coordinators. When a change
coordinators command is issued, the data must be sent to the new
coordinators to keep the database consistent.
2022-09-13 16:53:54 -07:00
Nim Wijetunga
7d287edc30 initalize values 2022-07-27 17:29:10 -07:00
Chaoguang Lin
5045844ea1 Add -r changeclusterkey tool for snapshot restore (#7687)
* Debug version of fdbserver -r changeclusterkey tool

* Remove debug symbols; move function to Coordinaton.actor.cpp

* Format and add traces

* fix comments
2022-07-25 22:12:28 -07:00
Renxuan Wang
df4e0deb4d coordinatorsKey should not always store IP addresses. (#7204)
* coordinatorsKey should not storing IP addresses.

Currently, when we do a commit of coordinator change, we are always converting hostnames to IP addresses and store the converted results in coordinatorsKey (\xff/coordinators). This result in ForwardRequest also sending IP addresses, and receivers will update their cluster files with IPs, then we lose the dynamic IP feature.

* Remove the legacy coordinators() function.

* Update async_resolve().

ip::basic_resolver::async_resolve(const query & q, ResolveHandler && handler) is deprecated.

* Clean code format.

* Fix typo.

* Remove SpecifiedQuorumChange and NoQuorumChange.
2022-05-23 11:42:56 -07:00
Renxuan Wang
c69a07a858 Check in the new Hostname logic. (#6926)
* Revert #6655.

20220407-031010-renxuan-c101052c21da8346           compressed=True data_size=31004844 duration=4310801 ended=100000 fail_fast=10 max_runs=100000 pass=100000 priority=100 remaining=0 runtime=1:04:15 sanity=False started=100047 stopped=20220407-041425 submitted=20220407-031010 timeout=5400 username=renxuan

* Revert #6271.

20220407-051532-renxuan-470f0fe6aac1c217           compressed=True data_size=30982370 duration=3491067 ended=100002 fail_fast=10 max_runs=100000 pass=100002 priority=100 remaining=0 runtime=0:59:57 sanity=False started=100141 stopped=20220407-061529 submitted=20220407-051532 timeout=5400 username=renxuan

* Revert #6266.

Remove resolving-related functionalities in connection string. Connection string will be used for storing purpose only, and non-mutable.

20220407-175119-renxuan-55d30ee1a4b42c2f           compressed=True data_size=30970443 duration=5437659 ended=100000 fail_fast=10 max_runs=100000 pass=100000 priority=100 remaining=0 runtime=0:59:31 sanity=False started=100154 stopped=20220407-185050 submitted=20220407-175119 timeout=5400 username=renxuan

* Add hostname to coordinator interfaces.

* Turn on the new hostname logic.

* Add the corresponding change in config txns.

The most notable change is before calling basicLoadBalance(), we need to call tryInitializeRequestStream() to initialize request streams first.

Passed correctness tests.

* Return error when hostnames cannot be resolved in coordinators command.

* Minor fixes.
2022-04-27 21:54:13 -07:00
Renxuan Wang
e40cc8722c A few hostname improvements. (#6825)
* Add tryResolveHostnames() in connection string.

* Add missing hostname to related interfaces.

* Do not pass RequestStream into *GetReplyFromHostname() functions.

Because we are using new RequestStream for each request anyways. Also, the passed in pointer could be nullptr, which results in seg faults.

* Add dynamic hostname resolve and reconnect intervals.

* Address comments.
2022-04-20 13:42:46 -07:00
sfc-gh-tclinkenbeard
a71099471b Update copyright header dates 2022-03-21 13:36:23 -07:00
sfc-gh-tclinkenbeard
62f547ff6e Add line before actorcompiler.h include (to prevent IDE from reordering includes) 2022-03-16 14:15:48 -07:00
sfc-gh-tclinkenbeard
66d71e107d Move actorcompiler.h include to the end of includes 2022-03-16 00:09:16 -07:00
A.J. Beamon
250a88e682 Enforce that trace event suppression calls happen first when using trace event call chaining. Fix various instances where we weren't following this requirement. 2022-02-24 12:25:52 -08:00
Renxuan Wang
622d89b552 Rebase on main.
Since we changed ClusterConnectionString's status flag from boolean to enum in #6422, we need to update this PR correspondingly.
2022-02-22 16:29:59 -08:00
Renxuan Wang
8eb7a10404 Address comments. 2022-02-22 16:29:59 -08:00
Renxuan Wang
481587a8c6 Turn on hostname logic. 2022-02-22 16:29:59 -08:00
Lukas Joswiak
d5a562e6b8 Fix dynamic knobs correctness issues 2022-02-09 13:43:32 -08:00
Renxuan Wang
5b079acd66 Coordinator should reply clientInfo when it changes.
This bug is introduced in #5231.
2021-12-10 16:37:48 -08:00
Renxuan Wang
6978486e85 Coordinator should only reply client data if it's valid.
Because when a coordinator restarts or newly joins a cluster, a client trying to connect to it may already have client data, while the coordinator doesn't. In this case, the coordinator should not reply empty client data.
2021-12-10 16:37:48 -08:00
Evan Tschannen
35bce4cd36 added a comment 2021-11-16 13:07:35 -08:00
Evan Tschannen
fd635432c4 fix: coordinators would process forwarding requests before making them durable 2021-11-16 12:21:26 -08:00
A.J. Beamon
e882eb33fc Abstract the cluster file into a cluster connection record that can be backed by something other than the filesystem. 2021-10-22 11:05:18 -07:00
Vaidas Gasiunas
16171d8252 Refactoring well-known endpoint registration
- List all well-known endpoints of FDB in a single enum
- Identify well-known endpoints by plain IDs
2021-09-21 11:05:31 -06:00
Renxuan Wang
96fcde45c2 Minor leader election code improvements.
1. Rename monitorLeaderRemotely* functions to monitorLeaderWithDelayedCandidacy*. "Remote" is not clearly describing what the functions are doing;
2. Rename monitorLeaderForProxies() to monitorLeaderAndGetClientInfo() to better describe the function;
3. Remove monitorLeaderRemotelyInternal() and monitorLeaderRemotely() in MonitorLeader.actor.cpp, to eliminate code duplication. They already exist in worker.actor.cpp;
4. Move the declaration of getLeader() from LeaderElection.actor.cpp to MonitorLeader.h;
5. Update a few comments.
2021-09-16 15:34:45 -07:00
sfc-gh-tclinkenbeard
82546853c0 Rename UseConfigDB to ConfigDBType 2021-08-09 10:04:35 -07:00
sfc-gh-tclinkenbeard
59ddfd78bd Merge remote-tracking branch 'origin/master' into paxos-config-db 2021-07-28 12:54:02 -07:00
Dan Lambright
55094bdd00 Merge branch 'master' into expired-forward 2021-07-26 19:58:29 -04:00
Dan Lambright
9f3e5d9bb5 use a BinaryWriter::toValue and BinaryReader::fromStringRef<double> instead of encoding time using a string 2021-07-26 15:58:41 -04:00
Dan Lambright
34fecb61de Reject connections to clusters forwarded in the (configurable) past 2021-07-26 10:18:10 -04:00
sfc-gh-tclinkenbeard
62026ce42a Remove ReplyPromise::sendErrorOr 2021-07-19 22:51:47 -07:00
sfc-gh-tclinkenbeard
d46feb5468 Add COORDINATOR_LEADER_CONNECTION_TIMEOUT server knob 2021-07-19 20:17:46 -07:00
sfc-gh-tclinkenbeard
b3e2b06553 Remove multiple implementations of ConfigNode 2021-07-18 18:25:06 -07:00
sfc-gh-tclinkenbeard
3c1cabf041 Coordinator lets client know if it cannot communicate with cluster controller 2021-07-13 17:26:28 -07:00
sfc-gh-tclinkenbeard
dee9fec300 Rename coordination files to fix upgrades 2021-06-18 14:16:49 -07:00
sfc-gh-tclinkenbeard
13ee24f464 Add UseConfigDB class 2021-06-10 20:57:50 -07:00
sfc-gh-tclinkenbeard
6fc5bd3480 Remove TestID from stateful configuration database classes, and fix some bugs 2021-06-07 00:29:36 -07:00
sfc-gh-tclinkenbeard
016ef54a73 Clean up unit test data directory between tests 2021-06-06 11:25:58 -07:00
sfc-gh-tclinkenbeard
a775f92fca Merge remote-tracking branch 'origin/master' into config-db 2021-06-01 15:39:34 -07:00