Commit Graph

60 Commits

Author SHA1 Message Date
Syed Paymaan Raza
c3e7542cda Update end year in copyright header 2024-08-02 09:40:11 -07:00
Junhyun Shim
e2df6e3302 Wipe packet buffers that held serialized WipedString (#10018)
* Extend WipedString guarantees to serialized packets

* Apply review suggestions
2023-04-20 16:38:55 +02:00
Junhyun Shim
eb0bc70ea8 Fix flaky ArenaBlock wipe test (#9960) 2023-04-14 20:40:56 +02:00
Yanqin Jin
0faa6538d1 Remove an assertion for UB (#9865)
Original code comment says
"there's no hard guarantee about the above equality and the result could vary by platform, malloc implementation, and tooling instrumentation (e.g. ASAN, valgrind)"

Therefore, we should remove the assertion on UB. Since removal of the UB makes the rest of the test case
less meaningful, we commented out the entire test case.

Furthermore, we have a case in which, even without ASAN or other instrumentation, e.g. Valgrind, etc, the equality does not hold.

Profile: team
Commit hash: 3164cadc6f
Command: devRetryCorrectnessTest bin/fdbserver -r simulation -f tests/fast/RandomUnitTests.toml -s 3939330597 -b on  --crash --trace_format json
2023-04-03 12:01:25 -07:00
A.J. Beamon
64b6a5d257 Allow boolean parameters to be nested inside of namespaces or classes 2023-03-30 15:09:59 -07:00
Junhyun Shim
b6f0d2095a Cases where newBuf == buf assertion fails have been observed in Valgrind (#9528) 2023-03-01 15:49:11 +01:00
A.J. Beamon
4a38bb4c3f Allow performing assert comparisons (e.g. ASSERT_EQ) with any traceable type 2023-02-24 12:53:01 -08:00
Junhyun Shim
06ee5f1e76 Disable client memory wipe test for all XSAN builds (#9409) 2023-02-21 14:10:44 +01:00
Junhyun Shim
d9c126a2d9 Introduce WipedString for Arena block holding AuthZ tokens (#9381)
* Enable secure allocation mode in Arena

This mode allows zeroing out blocks holding sensitive data after use

* Introduce WipedString to all token-holding memory

Also introduce a option flag "sensitive"

* Make pointer equivalency a hard requirement for non-ASAN builds

So that we can detect when Arena/malloc/memory-wipe behavior changes
2023-02-16 10:44:32 +01:00
A.J. Beamon
985bbe66d4 Add support for flatMap in optional and erroror. Allow calling map with a lambda without specifying the return type template parameter. 2022-12-15 15:32:02 -08:00
A.J. Beamon
f1c5eb55ba Add overloads of the Optional::map and ErrorOr::map functions that allow you to more easily convert to a value's members. Add a similar mapRef function if the wrapped type is a pointer-type. 2022-12-14 15:22:54 -08:00
Dan Adkins
fc1d29f429 Arena: avoid null-pointer reference in ASSERT (#8875)
Narrow the self-reference ASSERT in ArenaBlock::dependOn to only
the cases where we're trying to make a reference from self -> other.

It's allowed for self.getPtr() to be null, in which case we create
the ArenaBlock anew and there's no danger of self-reference.

Fixes Issue #8869.
2022-11-18 18:08:58 -05:00
Dan Adkins
a864216d97 Make Arena::dependOn(Arena) a no-op in case they're referring to the same ArenaBlock.
The intent is to have one ArenaBlock live at least at long as another. If they're
the same, that's accomplished without doing any additional bookkeeping.
2022-11-10 19:36:59 -08:00
Dan Adkins
2a8f5c7c3c Assert that we don't try to have an ArenaBlock depend on itself.
It introduces a cycle in the `next` pointers in the ArenaBlockRef's,
which results in infinite recursion as we follow them in `getTotalSize`.

This doesn't fix the underlying problem, but gives a better failure
than a stack overflow.
2022-11-10 19:36:59 -08:00
A.J. Beamon
7324f28441 Add the ability for StringRef.eat to return whether a separator was found. Add a serialize function and equality operators to ClusterConnectionString. Remove unused variable. 2022-07-19 11:55:58 -07:00
Markus Pilman
03d913a1de Flow compiling 2022-06-27 17:05:55 -06:00
Andrew Noyes
38db712e7a Make ASAN arena aware (#7336) 2022-06-13 13:24:02 -07:00
Andrew Noyes
966402a7a2 Allocate at least sizeof(ArenaBlock) for an ArenaBlock (#6770)
* Allocate at least sizeof(ArenaBlock) for an ArenaBlock

* Fix message pack unit test

Previously we were using only the 4 least significan bits as the length
of a message pack string, but it should be 5 according to https://github.com/msgpack/msgpack/blob/master/spec.md#str-format-family
2022-04-05 18:14:10 -07:00
Andrew Noyes
3b7389fbc5 Use malloc for > 256 in FastAllocated, allocateFast, Arena 2022-03-22 11:01:33 -07:00
sfc-gh-tclinkenbeard
a71099471b Update copyright header dates 2022-03-21 13:36:23 -07:00
sfc-gh-tclinkenbeard
44b0e05538 Merge remote-tracking branch 'origin/main' into arena-get-size-boolean-param 2022-03-16 22:53:30 -07:00
Steve Atherton
6cef85e268 Change assertions to not rely on the first argument being evaluated before the second. 2022-03-16 19:00:26 -07:00
sfc-gh-tclinkenbeard
0ae9ba5fd0 Add FastInaccurateEstimate boolean parameter for Arena::getSize 2022-03-16 16:54:27 -07:00
Steve Atherton
6f16702042 Merge branch 'main' of https://github.com/apple/foundationdb into redwood-bug-fixes-and-memory-leak 2022-02-26 18:00:18 -08:00
Andrew Noyes
7a9217a392 Add contrib/debug_determinism (#6389)
* Add contrib/debug_determinism

Add an instrumentation-based technique for debugging unseen mismatches. Also guard a few existing sources of nondeterminism that don't affect unseen with the DEBUG_DETERMINISM macro.

Also change the simulated run loop to not run as the only task inside the real run loop, since that was a source of nondeterminism.

Also fix nondeterminism from calling timer_int

* Add StorageMetadataType::currentTime

Basically a deterministic-in-simulation version of timer_int that we can
use instead of timer_int for StorageMetadataType::createdTime
2022-02-25 12:54:31 -08:00
Steve Atherton
735c5697d0 Added fast size estimates to Arena which are O(1) and are usually accurate. When inaccurate, a full size scan will update all estimates in the tree. 2022-02-18 03:25:28 -08:00
A.J. Beamon
39a7dc7a2d Add two unit tests for the new hashing functionality 2022-01-28 15:56:58 -08:00
Andrew Noyes
eb15746e41 Dereference to get result before marking "no access" 2021-06-02 16:07:04 -07:00
Steve Atherton
a31e4f622f Changed ArenaBlockRef to use 32 bit aligned4kBuffer size. 2021-05-16 03:58:05 -07:00
Steve Atherton
2298567c2b Use of aligned_alloc() for 4k pages causes too much wasted virtual memory. Added new 4k-aligned fast allocator, and changed Arena::allocatedAlignedBuffer() to be 4k-specific, now called Arena::allocate4kAlignedBuffer(). 2021-05-14 23:12:00 -07:00
Steve Atherton
16f9f6e75c Added a test of Arena dependencies and destruction when using the new aligned buffer feature. 2021-05-06 02:05:09 -07:00
Steve Atherton
f8a8bf315b Added Arena::allocateAlignedBuffer() to get an aligned memory block owned by an Arena, and ArenaPage uses this. 2021-05-05 15:00:12 -07:00
Andrew Noyes
a63dc3cfd6 Inform LSAN that we have unaligned pointers 2021-03-13 00:33:47 +00:00
FDB Formatster
df90cc89de apply clang-format to *.c, *.cpp, *.h, *.hpp files 2021-03-10 10:18:07 -08:00
Andrew Noyes
f91079dbe7 Use camelCase 2020-10-22 21:51:00 +00:00
Andrew Noyes
dc5e09ba62 Fix normal build 2020-10-22 16:34:37 +00:00
Andrew Noyes
ab4eb627c2 Replace with environment variable
We need to change the behavior of the allocator before main is called,
so a knob won't work.
2020-10-22 16:31:48 +00:00
Andrew Noyes
79163947a6 Add FLOW_KNOBS->VALGRIND_PRECISE 2020-10-22 16:31:48 +00:00
sfc-gh-tclinkenbeard
0ac08f6a9b Replace NULL with nullptr in flow 2020-09-20 11:31:49 -07:00
Jingyu Zhou
cb582e58c9 Avoid iterator+2 or -2 passes over the boundary 2020-07-30 13:42:36 -07:00
Jingyu Zhou
9a63882cd4 Fix unit test failure 2020-07-30 11:01:52 -07:00
Andrew Noyes
d6d374818c Actually test copy 2020-07-22 19:57:47 +00:00
Andrew Noyes
4195d3ab3b Fix compilation, test copies 2020-07-22 19:57:47 +00:00
Andrew Noyes
a25689e6e5 Add append tests, fix append 2020-07-22 19:57:47 +00:00
Andrew Noyes
82b70a0fa3 Fix test bug 2020-07-22 19:57:47 +00:00
Andrew Noyes
03a45e979d Add unit tests for VectorRef and SmallVectorRef 2020-07-22 19:57:47 +00:00
Andrew Noyes
6caa3e470c Fix valgrind build 2020-07-09 22:05:48 +00:00
Russell Sears
4ce51c4cce Merge pull request #2768 from atn34/atn34/arena-cpp
Check for memory errors in arena-allocated memory
2020-07-06 09:43:28 -07:00
Andrew Noyes
10e094c6d7 Address review comments 2020-06-11 15:43:31 +00:00
Andrew Noyes
e772f3a3f1 Separate knob to control valgrind arena instrumentation 2020-05-07 22:07:33 +00:00