Commit Graph

70 Commits

Author SHA1 Message Date
Nelson Elhage
1f4f900766 Don't use -fsanitizer for the standalone fuzzer build. 2025-03-03 23:47:41 +00:00
Nelson Elhage
4cd34f1d94 fuzzers: Fix CFLAGS
I'm seeing the current fuzzer build fail (during `cmake`) like so:

```
-- Performing Test IS_FSANITIZE_FUZZER_NO_LINK_SUPPORTED
-- Performing Test IS_FSANITIZE_FUZZER_NO_LINK_SUPPORTED - Failed
CMake Error at cmake/AddCFlagIfSupported.cmake:17 (message):
  Required flag -fsanitize=fuzzer-no-link is not supported
Call Stack (most recent call first):
  fuzzers/CMakeLists.txt:6 (add_c_flag)
```

The cmake log output contains something like so:

```
        /src/aflplusplus/libAFLDriver.a(aflpp_driver.o): in function `main':
        aflpp_driver.c:(.text+0x11b): undefined reference to `LLVMFuzzerTestOneInput'
        clang: error: linker command failed with exit code 1 (use -v to see invocation)
```

I haven't figured out exactly what's happening, but I believe that
once line 5 has added `-fsanitize=fuzzer` to `CFLAGS`, future compile-
tests **also** use it during linking. This in turn pulls in the fuzzer
`main`, which expects an `LLVMFuzzerTestOneInput` symbol, and thus
fails.

Instead, just add `-fsanitize=fuzzer-no-link` to CFLAGS (as suggested
[by the documentation][libfuzzer]), and then use `-fsanitize=fuzzer`
only for linking the fuzzer targets. At least in my environment, this
results in a working fuzzer build.

[libfuzzer]: https://llvm.org/docs/LibFuzzer.html#fuzzer-usage
2025-03-03 22:51:52 +00:00
Edward Thomson
56e2a85643 sha256: simplify API changes for sha256 support
There are several places where users may want to specify the type of
object IDs (sha1 or sha256) that should be used, for example, when
dealing with repositories, indexes, etc.

However, given that sha256 support remains disappointingly uncommon in
the wild, we should avoid hard API breaks when possible. Instead, update
these APIs to have an "extended" format (eg, `git_odb_open_ext`) that
provides an options structure with oid type information.

This allows callers who do care about sha256 to use it, and callers who
do not to avoid gratuitous API breakage.
2025-01-02 13:13:59 +00:00
Edward Thomson
9aa5faa38b indexer: move oid_type into the opts structure
Object ID type should be an option within the options structure; move it
there.
2024-12-18 16:27:46 +00:00
Edward Thomson
d090433ef1 cmake: use CMAKE_C_STANDARD and CMAKE_C_EXTENSIONS
cmake already provides a standard way for callers to override the
C_STANDARD and C_EXTENSIONS properties. Support and document those.
2024-10-19 13:18:53 +01:00
Edward Thomson
41a41b4f4a cmake: make C_STANDARD and C_EXTENSIONS configurable
Users can now set up cmake with -DC_STANDARD=... or -DC_EXTENSIONS=...
We default to C90 with C_EXTENSIONS=OFF, but callers can override if
they desire.
2024-10-18 22:26:57 +01:00
GravisZro
f1cac063ba Mandate C90 conformance
This PR ensures and enforces C90 conformance for all files C, including tests.

* Modify CMakeLists.txt to mandate C90 conformance (for better compiler compatibility)
* Update deps/ntlmclient/utf8.h to latest version
* Modify two tests and one header to use C comments instead of C++ comments
2024-10-18 22:26:57 +01:00
Edward Thomson
e78eec28c3 vector: free is now dispose
In keeping with the libgit2 pattern, we _dispose_ a structure that is
not heap allocated, and _free_ a structure's contents _and the structure
itself_.
2024-09-30 21:28:36 +01:00
Edward Thomson
f015996fe3 fuzzer: add a revparse fuzzer 2024-02-06 16:56:30 +00:00
Edward Thomson
8bd69574bd fuzzers: provide util functions like repo init
Many fuzzers will need to operate with a repository; extract the
repository initialization from downloads_refs_fuzzer.c into its own
utility area.
2024-02-06 16:56:30 +00:00
Edward Thomson
40ce52e51f config: provide two memory-backed config backends
Provide two memory-backed configuration backends -- one that takes a
string in config file format `[section] key=value` and one that takes a
list of strings in `section.key=value` format.
2023-07-21 11:18:44 +01:00
Edward Thomson
fe2ee3a018 object: lookup sha256 objects
This is much of the plumbing for the object database to support SHA256,
and for objects to be able to parse SHA256 versions of themselves.
2023-02-12 22:02:00 +00:00
Edward Thomson
b43567d655 sha256: indirection for experimental functions
The experimental function signature is only available when
`GIT_EXPERIMENTAL_SHA256` is enabled.
2022-07-13 22:50:33 -04:00
Edward Thomson
3eba9181cf odb: add git_odb_options
Users will need to be able to specify the object id type for the given
object database; add a new `git_odb_options` with that option.
2022-06-20 17:05:30 -04:00
Edward Thomson
8444b6dce7 odb_hash*: accept the oid type to hash into
The git_odb_hash helper functions should not assume SHA1, and instead
should be given the oid type that they're producing.
2022-06-20 17:05:29 -04:00
Edward Thomson
3fbf580c91 oid: give oids a type
`git_oid`s now have a type, and we require the oid type when creating
the object id from creation functions.
2022-06-20 17:05:29 -04:00
Edward Thomson
dbc4ac1c76 oid: GIT_OID_*SZ is now GIT_OID_SHA1_*SIZE
In preparation for SHA256 support, `GIT_OID_RAWSZ` and `GIT_OID_HEXSZ`
need to indicate that they're the size of _SHA1_ OIDs.
2022-06-14 22:29:57 -04:00
Edward Thomson
f882140577 fuzzer: use raw oid data
The indexer expects raw oid data, provide it.
2022-04-10 16:14:19 -04:00
lhchavez
33b1d3fd62 [midx] Fix an undefined behavior (left-shift signed overflow)
There was a missing check to ensure that the `off64_t` (which is a
signed value) didn't overflow when parsing it from the midx file. This
shouldn't have huge repercusions since the parsed value is immediately
validated afterwards, but then again, there is no such thing as "benign"
undefined behavior.

This change makes all the bitwise arithmetic happen with unsigned types
and is only casted to `off64_t` until the very end.

Thanks to Taotao Gu for finding and reporting this!
2022-04-05 13:19:26 -07:00
Edward Thomson
5fcfada500 cmake: document CMakeLists.txt hierarchy 2022-02-22 22:07:44 -05:00
Edward Thomson
70d9bfa47c packbuilder: use the packfile name instead of hash
Deprecate the `git_packfile_hash` function.  Callers should use the new
`git_packfile_name` function which provides a unique packfile name.
2022-01-27 20:15:09 -05:00
Edward Thomson
395b3dc403 cmake: refactor global variables
Update the global variables `LIBGIT2_OBJECTS` to
`LIBGIT2_DEPENDENCY_OBJECTS` for clarity and consistency.
2021-11-14 07:25:41 -05:00
Edward Thomson
2374ba8d49 fuzzers: declare static functions as static 2021-11-11 17:21:32 -05:00
Edward Thomson
489aec4447 fuzzers: declare standalone functions 2021-11-11 17:11:25 -05:00
Edward Thomson
4d2a6839dc cmake: move fuzzer args to the fuzzer's cmake 2021-11-11 15:56:10 -05:00
Edward Thomson
95117d4744 path: separate git-specific path functions from util
Introduce `git_fs_path`, which operates on generic filesystem paths.
`git_path` will be kept for only git-specific path functionality (for
example, checking for `.git` in a path).
2021-11-09 15:17:17 +00:00
Martin Kühl
f66e7f36ff libgit2_clar is now libgit2_tests
in #6083 the test runner was renamed to libgit2_tests,
but not all references to the old name were updated.
this change changes all of them to use the new name.
2021-10-28 10:25:09 +02:00
Edward Thomson
52693ab44e cmake: stylistic refactoring
Ensure that we always use lowercase function names, and that we do not
have spaces preceding open parentheses, for consistency.
2021-10-18 08:30:14 -04:00
Edward Thomson
4e14d4c6a6 cmake: BUILD_CLAR is now BUILD_TESTS
Nobody knows what CLAR is.  The test building option should be
`BUILD_TESTS`.
2021-10-17 13:05:32 -04:00
Edward Thomson
f0e693b18a str: introduce git_str for internal, git_buf is external
libgit2 has two distinct requirements that were previously solved by
`git_buf`.  We require:

1. A general purpose string class that provides a number of utility APIs
   for manipulating data (eg, concatenating, truncating, etc).
2. A structure that we can use to return strings to callers that they
   can take ownership of.

By using a single class (`git_buf`) for both of these purposes, we have
confused the API to the point that refactorings are difficult and
reasoning about correctness is also difficult.

Move the utility class `git_buf` to be called `git_str`: this represents
its general purpose, as an internal string buffer class.  The name also
is an homage to Junio Hamano ("gitstr").

The public API remains `git_buf`, and has a much smaller footprint.  It
is generally only used as an "out" param with strict requirements that
follow the documentation.  (Exceptions exist for some legacy APIs to
avoid breaking callers unnecessarily.)

Utility functions exist to convert a user-specified `git_buf` to a
`git_str` so that we can call internal functions, then converting it
back again.
2021-10-17 09:49:01 -04:00
Edward Thomson
f0a0916028 fuzzers: use updated hash functions 2021-10-02 16:34:47 -04:00
lhchavez
25b75cd9bc commit-graph: Create git_commit_graph as an abstraction for the file
This change does a medium-size refactor of the git_commit_graph_file and
the interaction with the ODB. Now instead of the ODB owning a direct
reference to the git_commit_graph_file, there will be an intermediate
git_commit_graph. The main advantage of that is that now end users can
explicitly set a git_commit_graph that is eagerly checked for errors,
while still being able to lazily use the commit-graph in a regular ODB,
if the file is present.
2021-03-10 07:09:47 -08:00
lhchavez
1f32ed25ee commit-graph: Support lookups of entries in a commit-graph
This change introduces `git_commit_graph_entry_find()` and
`git_commit_graph_entry_parent()`. These two functions allow a much
faster lookup of commits by ID, since the ODB does not need to be
consulted, the commit object does not need to be inflated, and the
contents of the commit object do not need to be parsed.

Part of: #5757
2021-01-10 11:18:38 -08:00
lhchavez
3fd57a75e9 commit-graph: Introduce a parser for commit-graph files
This change is the first in a series to add support for git's
commit-graph. This should speed up commit graph traversals by avoiding
object parsing and allowing some operations to terminate earlier.

Part of: #5757
2021-01-10 11:18:38 -08:00
lhchavez
005e77157d multipack: Introduce a parser for multi-pack-index files
This change is the first in a series to add support for git's
multi-pack-index. This should speed up large repositories significantly.

Part of: #5399
2020-10-05 05:08:38 -07:00
Augie Fackler
92e011a715 fuzzers: add a new fuzzer for patch parsing
I was looking at this code anyway because the sr.ht people nerdsniped
me, and it gave me that "I should fuzz this" feeling. So have a fuzzer!
2019-10-17 15:02:36 -04:00
Patrick Steinhardt
82b1d1da2b Merge pull request #5141 from pks-t/pks/azure-drop-powershell
azure: drop powershell
2019-07-21 12:25:10 +02:00
Edward Thomson
ecd4f97b2b fuzzer: use futils instead of fileops 2019-07-20 21:15:47 +01:00
Patrick Steinhardt
e54343a402 fileops: rename to "futils.h" to match function signatures
Our file utils functions all have a "futils" prefix, e.g.
`git_futils_touch`. One would thus naturally guess that their
definitions and implementation would live in files "futils.h" and
"futils.c", respectively, but in fact they live in "fileops.h".

Rename the files to match expectations.
2019-07-20 19:11:20 +02:00
Patrick Steinhardt
86ecd6008d fuzzers: provide test targets
Instead of having to find the fuzzer executables in our Azure test
scripts, provide test targets for each of our fuzzers that will
run them with the correct paths.
2019-07-20 19:10:10 +02:00
Patrick Steinhardt
3c966fb4fb fuzzers: clean up header includes
There's multiple headers included in our fuzzers that aren't required at
all. Furthermore, some of them are not available on Win32, causing
builds to fail. Remove them to fix this.
2019-07-05 11:58:33 +02:00
Patrick Steinhardt
9d43d45b21 fuzzers: use git_buf_printf instead of snprintf
The `snprintf` function does not exist on Win32, it only has
`_snprintf_s` available. Let's just avoid any cross-platform hassle and
use our own `git_buf` functionality instead.
2019-07-05 11:58:33 +02:00
Patrick Steinhardt
a6b2fffd46 fuzzers: use POSIX emulation layer to unlink files
Use `p_unlink` instead of `unlink` to remove the generated packfiles in
our packfile fuzzer. Like this, we do not have to worry about using
proper includes that are known on all platforms, especially Win32.
2019-07-05 11:58:33 +02:00
Patrick Steinhardt
6905581304 fuzzers: make printf formatters cross-platform compatible
The `printf` formatters in our standalone fuzzing driver are currently
using the "%m" specifier, which is a GNU extension that prints the error
message for the error code in `errno`. As we're using libgit2 functions
in both cases anyway, let's just use `git_error_last` instead to make
this valid on all platforms.
2019-07-05 11:58:33 +02:00
Patrick Steinhardt
48d563286c fuzzers: implement mkdtemp alternative for Win32
The `mkdtemp` function is not available on Windows, so our download_refs
fuzzer will fail to compile on Windows. Provide an alternative
implementation to fix it.
2019-07-05 11:58:32 +02:00
Edward Thomson
d3a440ca19 fuzzers: use system includes
Use the system includes (defined by libgit2) as the fuzzer includes.
The fuzzers link against internal libgit2 API and therefore need to have
the full include path that libgit2 uses.
2019-05-19 11:10:09 +01:00
Edward Thomson
a1ef995dc0 indexer: use git_indexer_progress throughout
Update internal usage of `git_transfer_progress` to
`git_indexer_progreses`.
2019-02-22 11:25:14 +00:00
Edward Thomson
1c3daccf1a fuzzers: don't use deprecated types 2019-01-25 09:06:50 +00:00
Edward Thomson
115a6c50c9 errors: remove giterr usage in fuzzers 2019-01-22 22:30:37 +00:00
Edward Thomson
83151018ef object_type: convert final internal users to new names
Update some missed types that were continuing to use the old `GIT_OBJ`
names.
2019-01-17 11:03:19 +00:00