Commit Graph

97 Commits

Author SHA1 Message Date
Syed Paymaan Raza
392bad2bd3 More copyright end year updates (#11556) 2024-08-05 14:00:32 -07:00
Syed Paymaan Raza
c3e7542cda Update end year in copyright header 2024-08-02 09:40:11 -07:00
Vishesh Yadav
6cd5ad2ffe Add code back 2024-07-10 18:52:14 -07:00
Vishesh Yadav
591efa1d1b Remove swift 2024-06-29 11:11:48 -07:00
Dimitris Apostolou
a88114c222 Fix typos 2024-02-07 01:16:00 +02:00
Xiaoge Su
b5e40c5598 fixup! Update per review 2023-09-21 14:55:56 -07:00
Xiaoge Su
91ec1fdf10 Provide actor call backtrace
See design/AcAC.md
2023-09-19 20:58:33 -07:00
Konrad `ktoso` Malawski
c26aa0b2a3 Introduce initial Swift support in fdbserver (#10156)
* [fdbserver] workaround the FRT type layout issue to get Swfit getVersion working

* MasterData.actor.h: fix comment typo

* masterserver.swift: some tweaks

* masterserver.swift: remove getVersion function, use the method

* masterserver.swift: print replied version to output for tracing

* [swift] add radar links for C++ interop issues found in getVersion bringup

* Update fdbserver.actor.cpp

* Migrate MasterData closer to full reference type

This removes the workaround for the FRT type layout issue, and gets us closer to making MasterData a full reference type

* [interop] require a new toolchain (>= Oct 19th) to build

* [Swift] fix computation of toAdd for getVersion Swift implementation

* add Swift to FDBClient and add async `atLeast` to NotifiedVersion

* fix

* use new atLeast API in master server

* =build fixup link dependencies in swift fdbclient

* clocks

* +clock implement Clock using Flow's notion of time

* [interop] workaround the immortal retain/release issue

* [swift] add script to get latest centos toolchain

* always install swift hooks; not only in "test" mode

* simulator - first thing running WIP

* cleanups

* more cleanup

* working snapshot

* remove sim debug printlns

* added convenience for whenAtLeast

* try Alex's workaround

* annotate nonnull

* cleanup clock a little bit

* fix missing impls after rebase

* Undo the swift_lookup_Map_UID_CommitProxyVersionReplies workaround

No longer needed - the issue was retain/release

* [flow][swift] add Swift version of BUGGIFY

* [swiftication] add CounterValue type to provide value semantics for Counter types on the Swift side

* remove extraneous requestingProxyUID local

* masterserver: initial Swift state prototype

* [interop] make the Swiftied getVersion work

* masterserver - remove the C++ implementation (it can't be supported as state is now missing)

* Remove unnecessary SWIFT_CXX_REF_IMMORTAL annotations from Flow types

* Remove C++ implementation of CommitProxyVersionReplies - it's in Swift now

* [swift interop] remove more SWIFT_CXX_REF_IMMORTAL

* [swift interop] add SWIFT_CXX_IMMORTAL_SINGLETON_TYPE annotation for semanticly meaningful immortal uses

* rename SWIFT_CXX_REF_IMMORTAL -> UNSAFE_SWIFT_CXX_IMMORTAL_REF

* Move master server waitForPrev to swift

* =build fix linking swift in all modules

* =build single link option

* =cmake avoid manual math, just get "last" element from list

* implement Streams support (#18)

* [interop] update to new toolchain #6

* [interop] remove C++ vtable linking workarounds

* [interop] make MasterData proper reference counted SWIFT_CXX_REF_MASTERDATA

* [interop] use Swift array to pass UIDs to registerLastCommitProxyVersionReplies

* [interop] expose MasterServer actor to C++ without wrapper struct

* [interop] we no longer need expose on methods 🥳

* [interop] initial prototype of storing CheckedContinuation on the C++ side

* Example of invoking a synchronous swift function from a C++ unit test. (#21)

* move all "tests" we have in Swift, and priority support into real modules (#24)

* Make set continuation functions inline

* Split flow_swift into flow_swift and flow_swift_future to break circular dependency

* rename SwiftContinuationCallbackStruct to FlowCallbackForSwiftContinuation

* Future interop: use a method in a class template for continuation set call

* Revert "Merge pull request #22 from FoundationDB/cpp-continuation" (#30)

* Basic Swift Guide (#29)

Co-authored-by: Alex Lorenz <arphaman@gmail.com>

* Revert "Revert "Merge pull request #22 from FoundationDB/cpp-continuation" (#30)"

This reverts commit c025fe6258.

* Restore the C++ continuation, but it seems waitValue is broken for CInt somehow now

* disable broken tests - waitValue not accessible

* Streams can be async iterated over (#27)

Co-authored-by: Alex Lorenz <arphaman@gmail.com>

* remove work in progress things (#35)

* remove some not used (yet) code

* remove expose func for CInt, it's a primitive so we always have witness info (#37)

* +masterdata implement provideVersions in Swift (#36)

* serveLiveCommittedVersion in Swift (#38)

* Port updateLiveCommittedVersion to swift (#33)

Co-authored-by: Konrad `ktoso` Malawski <konrad_malawski@apple.com>

* Implement updateRecoveryData in Swift (#39)

Co-authored-by: Alex Lorenz <arphaman@gmail.com>

* Simplify flow_swift to avoid multiple targets and generate separate CheckedContinuation header

* Uncomment test which was blocked on extensions not being picked up (#31)

* [interop] Use a separate target for Swift-to-C++ header generation

* reduce boilerplate in future and stream support (#41)

* [interop] require interop v8 - that will fix linker issue (https://github.com/apple/swift/issues/62448)

* [interop] fix swift_stream_support.h Swift include

* [interop] bump up requirement to version 9

* [interop] Generalize the Flow.Optional -> Swift.Optional conversion using generics

* [WIP] masterServer func in Swift (#45)

* [interop] Try conforms_to with a SWIFT_CONFORMS_TO macro for Optional conformance (#49)

* [interop] include FlowOptionalProtocol source file when generating Flow_CheckedContinuation.h

This header generation step depends on the import of the C++ Flow module, which requires the presence of FlowOptionalProtocol

* conform Future to FlowFutureOps

* some notes

* move to value() so we can use discardable result for Flow.Void

* make calling into Swift async funcs nicer by returning Flow Futures

* [interop] hide initial use of FlowCheckedContinuation in flow.h to break dependency cycle

* [fdbserver] fix an EncryptionOpsUtils.h modularization issue (showed up with modularized libc++)

* Pass GCC toolchain using CMAKE_Swift_COMPILE_EXTERNAL_TOOLCHAIN to Swift's clang importer

* [interop] drop the no longer needed libstdc++ include directories

* [cmake] add a configuration check to ensure Swift can import C++ standard library

* [swift] include msgpack from msgpack_DIR

* [interop] make sure the FDB module maps have 'export' directive

* add import 'flow_swift' to swift_fdbserver_cxx_swift_value_conformance.swift

This is needed for CONFORMS_TO to work in imported modules

* make sure the Swift -> C++ manually bridged function signature matches generated signature

* [interop][workaround] force back use of @expose attribute before _Concurrency issue is fixed

* [interop] make getResolutionBalancer return a pointer to allow Swift to use it

We should revert back to a reference once compiler allows references again

* [interop] add a workaround for 'pop' being marked as unsafe in Swift

* masterserver.swift: MasterData returns the Swift actor pointer in an unsafe manner

* Add a 'getCopy' method to AsyncVar to make it more Swift friendly

* [interop] bump up the toolchain requirement

* Revert "[interop][workaround] force back use of @expose attribute before _Concurrency issue is fixed"

This reverts commit b01b271a76.

* [interop] add FIXME comments highlighting new issue workarounds

* [interop] adopt the new C++ interoperability compiler flag

* [interop] generate swift compile commands

* Do not deduplicate Swift compilation commands

* [interop] generate swift compile commands

* Do not deduplicate Swift compilation commands

* flow actorcompiler.h: add a SWIFT_ACTOR empty macro definition

This is needed to make the actor files parsable by clangd

* [cmake] add missing dependencies

* experimental cross compile

* [cmake] fix triple in cross-compiled cmake flags

* [interop] update to interop toolchain version 16

* [x-compile] add flags for cross-compiling boost

* cleanup x-compile cmake changes

* [cmake] fix typo in CMAKE_Swift_COMPILER_EXTERNAL_TOOLCHAIN config variable

* [interop] pass MasterDataActor from Swift to C++ and back to Swift

* [fdbserver] Swift->C++ header generation for FDBServer should use same module cache path

* Update swift_get_latest_toolchain.sh to fetch 5.9 toochains

* set HAVE_FLAG_SEARCH_PATHS_FIRST for cross compilation

* Resolve conflicts in net2/sim2/actors, can't build yet

* undo SWIFT_ACTOR changes, not necessary for merge

* guard c++ compiler flags with is_cxx_compile

* Update flow/actorcompiler/ActorParser.cs

Co-authored-by: Evan Wilde <etceterawilde@gmail.com>

* update the boost dependency

* Include boost directory from the container for Swift

* conform flow's Optional to FlowOptionalProtocol again

* Guard entire RocksDBLogForwarder.h with SSD_ROCKSDB_EXPERIMENTAL to avoid failing on missing rocksdb APIs

* remove extraneous merge marker

* [swift] update swift_test_streams.swifto to use vars in more places

* Add header guard to flow/include/flow/ThreadSafeQueue.h to fix moduralization issue

* Update net and sim impls

* [cmake] use prebuilt libc++ boost only when we're actually using libc++

* [fdbserver] Swift->C++ header generation for FDBServer should use same module cache path

* fixups after merge

* remove CustomStringConvertible conformance that would not be used

* remove self-caused deprecation warnings in future_support

* handle newly added task priority

* reformatting

* future: make value() not mutating

* remove FIXME, not needed anymore

* future: clarify why as functions

* Support TraceEvent in Swift

* Enable TraceEvent using a class wrapper in Swift

* prearing WITH_SWIFT flag

* wip disabled failing Go stuff

* cleanup WITH_SWIFT_FLAG and reenable Go

* wip disabled failing Go stuff

* move setting flag before printing it

* Add SWIFT_IDE_SETUP and cleanup guides and build a bit

* Revert "Wipe packet buffers that held serialized WipedString (#10018)"

This reverts commit e2df6e3302.

* [Swift] Compile workaround in KeyBackedRangeMap; default init is incorrect

* [interop] do not add FlowFutureOps conformance when building flow clang module for Flow checked continuation header pre-generation

* make sure to show  -DUSE_LIBCXX=OFF in readme

* readme updates

* do not print to stderr

* Update Swift and C++ code to build with latest Swift 5.9 toolchain now that we no longer support universal references and bridge the methods that take in a constant reference template parameter correctly

* Fix SERVER_KNOBS and enable use them for masterserver

* Bump to C++20, Swift is now able to handle it as well

* Put waitForPrev behind FLOW_WITH_SWIFT knob

* Forward declare updateLiveCommittedVersion

* Remove unused code

* fix wrong condition set for updateLiveCommittedVersion

* Revert "Revert "Wipe packet buffers that held serialized WipedString (#10018)""

This reverts commit 5ad8dce052.

* Enable go-bindings in cmake

* Revert "Revert "Wipe packet buffers that held serialized WipedString (#10018)""

This reverts commit 5ad8dce052.

* USE_SWIFT flag so we "build without swift" until ready to by default

* uncomment a few tests which were disabled during USE_SWIFT enablement

* the option is WITH_SWIFT, not USE

* formatting

* Fix masterserver compile error

* Fix some build errors.

How did it not merge cleanly? :/

* remove initializer list from constructor

* Expect Swift toolchain only if WITH_SWIFT is enabled

* Don't require Flow_CheckedContinuation when Swift is disabled

* Don't compile FlowCheckedContinuation when WITH_SWIFT=OFF

* No-op Swift macros

* More compile guards

* fix typo

* Run clang-format

* Guard swift/bridging include in fdbrpc

* Remove printf to pass the test

* Remove some more printf to avoid potential issues

TODO: Need to be TraceEvents instead

* Remove __has_feature(nullability) as its only used in Swift

* Don't use __FILENAME__

* Don't call generate_module_map outside WITH_SWIFT

* Add some more cmake stuff under WITH_SWIFT guard

* Some more guards

* Bring back TLSTest.cpp

* clang-format

* fix comment formatting

* Remove unused command line arg

* fix cmake formatting in some files

* Address some review comments

* fix clang-format error

---------

Co-authored-by: Alex Lorenz <arphaman@gmail.com>
Co-authored-by: Russell Sears <russell_sears@apple.com>
Co-authored-by: Evan Wilde <etceterawilde@gmail.com>
Co-authored-by: Alex Lorenz <aleksei_lorenz@apple.com>
Co-authored-by: Vishesh Yadav <vishesh_yadav@apple.com>
Co-authored-by: Vishesh Yadav <vishesh3y@gmail.com>
2023-06-02 16:09:28 -05:00
Steve Atherton
dd334f1b02 Added transaction_option_setter<DB> to determine if a DB-like thing has a ->setOptions(tr) method. This method is called in runTransaction() templates at the top of the retry loop and in the manual retry loops in KeyBackedTypes. Added if constexpr( support to the ActorCompiler to support this. 2023-04-22 11:10:20 -07:00
Andrew Noyes
87b591bc21 Disallow uniform initialization syntax for state variables
The actor compiler does not handle this correctly, so disallow it for
now. We could probably support this someday if someone is sufficiently
motivated.

Previously this would miscompile and hopefully cause some kind of syntax
error in the generated code. Now it issues a helpful diagnostic.

Tested by adding an actor with a state variable that tries to use
uniform initialization syntax and observing the error message looks
good. We don't have any infrastructure for testing that the
ActorCompiler rejects something, so I'm not including a test in this
change. We could probably add that relatively easily though.
2022-11-03 13:47:43 -07:00
A.J. Beamon
e1fe28b78b Switch some usages of LiteralStringRef to use the _sr suffix 2022-09-30 16:04:16 -07:00
Arjun Sharda
e89b077b97 Update ParseTree.cs 2022-08-09 14:27:48 -05:00
Arjun Sharda
dfdfe3f9a1 Update Program.cs 2022-08-09 14:27:37 -05:00
Arjun Sharda
0dce3e5d91 Update ActorCompiler.cs 2022-08-09 14:27:15 -05:00
Andrew Noyes
cbadccaad5 Reject preprocessor directives in ACTORs 2022-01-14 11:37:29 -08:00
Andrew Noyes
fd33d31ff5 Enable -Wdelete-non-virtual-dtor for clang build
We had been disabling -Wdelete-non-virtual-dtor, because this seems to be done intentionally in the generated code of the actor compiler. I spent some time trying to rewrite it in a way that doesn't literally delete/destroy through a pointer to a base class without a virtual destructor, but I was unable to come up with something that passes correctness. My best guess is that we do this so that we can destroy actor state classes, call callbacks registered on the actor SAV, and then destroy the SAV.

Anyway now we'll detect new usages of deleting through a pointer to a base class without a virtual destructor.
2021-12-20 16:19:31 -08:00
A.J. Beamon
aebd12a9aa Fix indentation issue with actor compiler (#5828) 2021-10-25 10:36:37 -07:00
Xiaoxi Wang
09342f43ea fix spelling error 2021-08-19 13:55:30 -07:00
Lukas Joswiak
5dc9a97230 Merge branch 'master' into fixes/alp6 2021-08-01 20:42:52 -07:00
Lukas Joswiak
e9a1679467 Disable sampling everywhere except fdbserver 2021-07-27 09:53:23 -07:00
Lukas Joswiak
dcaf1c1c38 Enable sampling on fdbserver only 2021-07-22 12:47:35 -07:00
sfc-gh-tclinkenbeard
e62e6503ac Fix most delete-non-virtual-dtor clang warnings 2021-07-21 23:32:44 -07:00
Lukas Joswiak
65e3ead37e Cleanup 2021-06-21 10:59:15 -07:00
Lukas Joswiak
805c4200ce Add collector to get running actor name 2021-06-16 18:08:51 -07:00
Lukas Joswiak
6041d7e201 Remove stack lineage 2021-06-15 15:18:52 -07:00
Lukas Joswiak
250d20e7bf Even more cleanup 2021-06-15 15:17:54 -07:00
Lukas Joswiak
eb3fb811d4 Cleanup 2021-06-15 15:03:44 -07:00
Lukas Joswiak
0ff041222a Add removals back 2021-06-13 18:23:59 -07:00
Lukas Joswiak
112be7a763 Remove most work 2021-06-13 16:19:02 -07:00
Lukas Joswiak
23f4aec641 Remove bad line 2021-06-12 22:38:22 -07:00
Lukas Joswiak
cbc42389ab Add stacks back 2021-06-11 14:34:12 -07:00
Lukas Joswiak
0301072690 Refactor 2021-06-11 13:01:32 -07:00
Lukas Joswiak
e9fdbb9c86 Enable sampling 2021-06-07 18:01:14 -07:00
Lukas Joswiak
7d56038e5e Remove restore_lineage 2021-06-04 16:38:12 -07:00
Lukas Joswiak
486a04659f Lazy inititialization 2021-06-04 15:01:18 -07:00
Lukas Joswiak
ca79b8eaab Various ALP fixes 2021-06-04 15:01:18 -07:00
Lukas Joswiak
153de33f57 Revert "Merge pull request #4802 from sfc-gh-ljoswiak/revert/actor-lineage"
This reverts commit 6499fa178e, reversing
changes made to 1512631957.
2021-06-04 13:31:55 -07:00
Lukas Joswiak
4ea760b2a9 Revert "Merge pull request #4136 from sfc-gh-mpilman/features/actor-lineage"
This reverts commit da41534618, reversing
changes made to e6300905d6.
2021-05-10 20:26:12 -07:00
Markus Pilman
7307750e5e Merge remote-tracking branch 'origin/master' into features/actor-lineage 2021-04-19 11:29:52 -06:00
Steve Atherton
b4e42476b7 Unit test parameters are no longer global, they are accessible via a parameter to the unit test and initialized from otherwise unconsumed test options for the UnitTests workload in the test spec or from the fdbserver command line when using the unittests role. 2021-04-06 02:36:10 -07:00
Markus Pilman
eb036b7b02 Merge remote-tracking branch 'origin/master' into features/actor-lineage 2021-03-17 11:59:54 -06:00
Markus Pilman
945d0246cd add actor stacktrace feature 2020-12-09 13:28:15 -07:00
sfc-gh-tclinkenbeard
d15441e85c Replace non-standard sealed with final 2020-12-08 09:09:30 -08:00
Markus Pilman
05f77f905f Added actor lineage 2020-12-07 15:15:25 -07:00
Daniel Smith
a88bbd6405 s/fake/declval/ 2020-07-15 23:33:01 +00:00
A.J. Beamon
b09dddc07e Merge branch 'release-6.2' into merge-release-6.2-into-release-6.3
# Conflicts:
#	cmake/ConfigureCompiler.cmake
#	documentation/sphinx/source/downloads.rst
#	fdbrpc/FlowTransport.actor.cpp
#	fdbrpc/fdbrpc.vcxproj
#	fdbserver/DataDistributionQueue.actor.cpp
#	fdbserver/Knobs.cpp
#	fdbserver/Knobs.h
#	fdbserver/LogSystemPeekCursor.actor.cpp
#	fdbserver/MasterProxyServer.actor.cpp
#	fdbserver/Status.actor.cpp
#	fdbserver/storageserver.actor.cpp
#	flow/flow.vcxproj
2020-07-10 15:06:34 -07:00
Andrew Noyes
42159ccfe0 Add -Wpessimizing-move and -Wredundant-move for clang 2020-06-22 23:45:03 +00:00
tclinken
885a2e020e Generate rvalue reference overloads with actor compiler 2020-04-21 15:18:59 -07:00
Steve Atherton
022b77e288 Actor compiler will std::move() return expressions that exactly match a state variable. 2020-04-20 04:19:33 -07:00
Alex Miller
2e53c8c5e2 Merge pull request #2915 from tclinken/move-optimizations
Avoid unnecessary copies in PromiseStream
2020-04-13 21:15:16 -07:00