For enum values that are constructed, not literal integers, we need to
parse the inner / implicit expression. For example:
```
GIT_DIFF_REVERSE = (1u << 0)
```
Include "minisearch" which is a straightforward client-side search tool;
and a script to generate the search index for minisearch for each
version of libgit2.
The API documentation generation is useful (in `--strict` mode) to
determine whether all our APIs have sufficient documentation. Add a
`--validate-only` mode that will run the validation without emitting the
API JSON. This is useful for ensuring that the documentation is
complete.
libgit2 has a new documentation generator that generates API schema from
our headers, then produces reference documentation that is included into
the website directly.
dlopen sets up some thread-local state that isn't cleaned up by
`dlclose`. Additionally, now that we're linking against different
versions of libssh2 and OpenSSL, we're seeing different leak signatures.
This change fixes a packfile heap corruption that can happen when
interacting with multiple packfiles concurrently across multiple
threads. This is exacerbated by setting a lower mwindow open file limit.
This change:
* Renames most of the internal methods in pack.c to clearly indicate
that they expect to be called with a certain lock held, making
reasoning about the state of locks a bit easier.
* Splits the `git_pack_file` lock in two: the one in `git_pack_file`
only protects the `index_map`. The protection to `git_mwindow_file` is
now in that struct.
* Explicitly checks for freshness of the `git_pack_file` in
`git_packfile_unpack_header`: this allows the mwindow implementation
to close files whenever there is enough cache pressure, and
`git_packfile_unpack_header` will reopen the packfile if needed.
* After a call to `p_munmap()`, the `data` and `len` fields are poisoned
with `NULL` to make use-after-frees more evident and crash rather than
being open to the possibility of heap corruption.
* Adds a test case to prevent this from regressing in the future.
Fixes: #5591
We currently do not set up a project version within CMake, meaning that
it can't be use by other projects including libgit2 as a sub-project and
also not by other tools like IDEs.
This commit changes this to always set up a project version, but instead
of extracting it from the "version.h" header we now set it up directly.
This is mostly to avoid mis-use of the previous `LIBGIT2_VERSION`
variables, as we should now always use the `libgit2_VERSION` ones that
are set up by CMake if one provides the "VERSION" keyword to the
`project()` call. While this is one more moving target we need to adjust
on releases, this commit also adjusts our release script to verify that
the project version was incremented as expected.
This change adds two new build targets: MSan and UBSan. This is because
even though OSS-Fuzz is great and adds a lot of coverage, it only does
that for the fuzz targets, so the rest of the codebase is not
necessarily run with the Sanitizers ever :( So this change makes sure
that MSan/UBSan warnings don't make it into the codebase.
As part of this change, the Ubuntu focal container is introduced. It
builds mbedTLS and libssh2 as debug libraries into /usr/local and as
MSan-enabled libraries into /usr/local/msan. This latter part is needed
because MSan requires the binary and all its dependent libraries to be
built with MSan support so that memory allocations and deallocations are
tracked correctly to avoid false positives.
The current release process is not documented in any way. As a result,
it's not obvious how releases should be done at all, like e.g. which
locations need adjusting.
To fix this, let's introduce a new script that shall from now on be used
to do all releases. As input it gets the tree that shall be released,
the repository in which to do the release, credentials to
authenticate against GitHub and the new version. E.g. executing the
following will create a new release v0.32:
$ ./script/release.py 0.32.0 --user pks-t --password ****
While the password may currently be your usual GitLab password, it's
recommended to use a personal access token intead.
The script will then perform the following steps:
1. Verify that "include/git2/version.h" matches the new version.
2. Verify that "docs/changelog.md" has a section for that new
version.
3. Extract the changelog entries for the current release from
"docs/changelog.md".
4. Generate two archives in "tar.gz" and "zip" format via "git
archive" from the tree passed by the user. If no tree was passed,
we will use "HEAD".
5. Create the GitHub release using the extracted changelog entries
as well as tag and name information derived from the version
passed by the used.
6. Upload both code archives to that release.
This should cover all steps required for a new release and thus ensures
that nothing is missing that shouldn't be.
valgrind will warn that OpenSSL will use undefined data in connect/read
when talking to certain other TLS stacks. Thankfully, this only seems
to occur when gcc is the compiler, so hopefully valgrind is just
misunderstanding an optimization. Regardless, suppress this warning.
On Ubuntu, the combination of libgcrypt and libssh2 is quite old and
known to contain memory leaks. We thus have several functions listed in
our suppressions file that are known to leak. Due to a recent update of
libssh2 or libgcrypt, there now are new memory leaks caused by
libssh2_session_handshake and libssh2_init that cause the CI to fail.
Add a new suppression to fix the issue.
Right now, we have an awful hack in our test CI setup that extracts the
test command from CTest's output and then prepends the leak checker.
This is dependent on non-machine-parseable output from CMake and also
breaks on various ocassions, like for example when we have spaces in the
current path or when the path contains backslashes. Both conditions may
easily be triggered on Win32 systems, and in fact they do break our
Azure Pipelines builds.
Remove the awful hack in favour of a new CMake build option
"USE_LEAK_CHECKER". If specifying e.g. "-DUSE_LEAK_CHECKER=valgrind",
then we will set up all tests to be run under valgrind. Like this, we
can again simply execute ctest without needing to rely on evil sourcery.
Coverity considers that anything that looks like assert() behaves like
it (ie. side-effects would be skipped on a NDEBUG build). As we have a
bunch of those in the test suite (128), this would ensure Coverity isn't
confused.
Simplify the names for the tests, removing the unnecessary
"libgit2-clar" prefix. Make "all" the new default test run, and include
the online tests by default (since HTTPS should always be enabled).
For the CI tests, create an offline-only test, then the various online
tests.
This adds a simple script for backporting pull requests to older
branches. It accepts as parameters a list of pull request numbers which
whose commits are to be cherry-picked. The identification of PRs
currently happens by using the commit message of the merge of the PR,
which should conform to the message "Merge pull request #<PR>".
While the heuristic works in practice, we could instead also use the
direct references from GitHub via "pull/#<PR>/head". This requires the
user to have all these references fetched, though, so we can just use
the current heuristic until we experience any issues with that.
Some tests shall be run against our own SSH server we spin up in Travis.
As those need to be run separate from our previous tests which run
against git-daemon, we have to do this in a separate step. Instead of
bundling all that knowledge in the CI script, move it into the test
build instructions by creating a new test target.
