[pull] master from libgit2:master #51
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Instead, globally initialize the system page size.
Improve the support of atomics
clone: update origin's HEAD
Update PCRE to 8.44
Update ntlm to include an htonll that is not dependent on system libraries.
Instead of treating win32 thread initialization specially in the win32 git_libgit2_init function, add a git_global_threads_init function.
Move the MSVC C runtime debugging bits into the allocator's global init function.
Move the settings global data teardown into its own separate function, instead of intermingled with the global state.
Move the mwindow mutex into the mwindow code itself, initializing it in the mwindow global initialization function instead of in the global initializer.
Ensure that we can allocate the error message buffer. In keeping with our typical policiess, we allow (small) memory leaks in the case where we're out of memory.
Our "global initialization" has accumulated some debris over the years. It was previously responsible for both running the various global initializers (that set up various subsystems) _and_ setting up the "global state", which is actually the thread-local state for things like error reporting. Separate the thread local state out into "threadstate". Use the normal subsystem initialization functions that we already have to set it up. This makes both the global initialization system and the threadstate system simpler to reason about.
We were never properly testing git_thread_exit. Do so.
We want to store a pointer to emulate `pthread_exit` on Windows. Do this within the threading infrastructure so that it could potentially be re-used outside of the context of libgit2 itself.
Provide a mechanism for system components to register for initialization and shutdown of the libgit2 runtime.
Now that we've identified that our global settings really aren't global at all, and refactored the library to match that, change global.c to libgit2.c, which is especially nice since the prefix of the functions matches the filename.
This change: * Increases MY_ROW_LIMIT to 2M, since it has been failing in #5595's tests since it's _super_ close to the limit. * Calls `git_repository_free()` on a `git_repository` that was being leaked only in Windows. * Marks the global `git_repository` on `tests/repo/init.c` as `NULL` after being freed to make any accidental access more noisy. * Uses `cl_assert_equal_i()` in `test_trace_windows_stacktrace__leaks` to make the test failures more actionable. * Renames the globals in `tests/repo/init.c` so that they don't start with an underscore.
libgit2 v1.1.0
This should allow folks that build in non-thread-safe environments to still be able to build the library. Fixes: #5663
ntlm: update ntlm dependency for htonll
Define `git___load` when building with `-DTHREADSAFE=OFF`
Make the Windows leak detection more robust
Refactor "global" state
threadstate: rename tlsdata when building w/o threads
Without this, mbedTLS installs in non-default install locations that are otherwise found by the `FindmbedTLS.cmake` module are not found by the C preprocessor at compile time.
Include `${MBEDTLS_INCLUDE_DIR}` when compiling `crypt_mbedtls.c`
repository: use intptr_t's in the config map cache
This change makes it possible to build with newer versions of gcc without warnings. There were two warnings issued: * gcc 8 added [`-Wstringop-truncation`](https://developers.redhat.com/blog/2018/05/24/detecting-string-truncation-with-gcc-8/), which warns if a call to `strncpy(3)` is prone to accidentally truncating the destination string, since `strncpy(3)` does NOT add a terminating `NULL` if the destination buffer is not large enough to hold the input. This change uses the pattern suggested in https://us-cert.cisa.gov/bsi/articles/knowledge/coding-practices/strncpy-and-strncat to fix the locations flagged by gcc. * There was a potentially uninitialized access of `dest` in `fs_copy`.
Specifically: ECDSA_256, ECDSA_384, ECDSA_521 and ED25519.
This allows the library to be built using a pre-1.9.0 version of libssh2.
Fix the `-DENABLE_WERROR=ON` build for gcc 10.2
IPv6 addresses should be used identically internally; we should not denote them with brackets in one operating system and without them in another.
Add support for additional SSH hostkey types.
Handle ipv6 addresses
This change builds libgit2 using Chromium's zlib implementation by
invoking cmake with `-DUSE_BUNDLED_ZLIB=ON -DUSE_CHROMIUM_ZLIB=ON`,
which is ~10% faster than the bundled zlib for the core::zstream suite.
This version of zlib has some optimizations:
a) Decompression (Intel+ARM): inflate_fast, adler32, crc32, etc.
b) Compression (Intel): fill_window, longest_match, hash function, etc.
Due to the introduction of SIMD optimizations, and to get the maximum
performance out of this fork of zlib, this requires an x86_64 processor
with SSE4.2 and CLMUL (anything Westmere or later, ~2010). The Chromium
zlib implementation also supports ARM with NEON, but it has not been
enabled in this patch.
Performance
===========
TL;DR: Running just `./libgit2_clar -score::zstream` 100 times in a loop
took 0:56.30 before and 0:50.67 after (~10% reduction!).
The bundled and system zlib implementations on an Ubuntu Focal system
perform relatively similar (the bundled one is marginally better due to
the compiler being able to inline some functions), so only the bundled
and Chromium zlibs were compared.
For a more balanced comparison (to ensure that nothing regressed
overall), `libgit2_clar` under `perf` was also run, and the zlib-related
functions were compared.
Bundled
-------
```shell
cmake \
-DUSE_BUNDLED_ZLIB=ON \
-DUSE_CHROMIUM_ZLIB=OFF \
-DCMAKE_BUILD_TYPE="RelWithDebInfo" \
-DCMAKE_C_FLAGS="-fPIC -fno-omit-frame-pointer" \
-GNinja \
..
ninja
perf record --call-graph=dwarf ./libgit2_clar
perf report --children
```
```
Samples: 87K of event 'cycles', Event count (approx.): 75923450603
Children Self Command Shared Objec Symbol
+ 4.14% 0.01% libgit2_clar libgit2_clar [.] git_zstream_get_output_chunk
+ 2.91% 0.00% libgit2_clar libgit2_clar [.] git_zstream_get_output
+ 0.69% 0.00% libgit2_clar libgit2_clar [.] git_zstream_get_output (inlined)
0.17% 0.00% libgit2_clar libgit2_clar [.] git_zstream_init
0.02% 0.00% libgit2_clar libgit2_clar [.] git_zstream_reset
0.00% 0.00% libgit2_clar libgit2_clar [.] git_zstream_eos
0.00% 0.00% libgit2_clar libgit2_clar [.] git_zstream_done
0.00% 0.00% libgit2_clar libgit2_clar [.] git_zstream_free (inlined)
Samples: 87K of event 'cycles', Event count (approx.): 75923450603
Children Self Command Shared Objec Symbol
+ 3.12% 0.01% libgit2_clar libgit2_clar [.] deflate
+ 2.65% 1.48% libgit2_clar libgit2_clar [.] deflate_slow
+ 1.60% 0.55% libgit2_clar libgit2_clar [.] inflate
+ 0.53% 0.00% libgit2_clar libgit2_clar [.] write_deflate
0.49% 0.36% libgit2_clar libgit2_clar [.] inflate_fast
0.46% 0.02% libgit2_clar libgit2_clar [.] deflate_fast
0.19% 0.19% libgit2_clar libgit2_clar [.] inflate_table
0.16% 0.01% libgit2_clar libgit2_clar [.] inflateInit_
0.15% 0.00% libgit2_clar libgit2_clar [.] inflateInit2_ (inlined)
0.10% 0.00% libgit2_clar libgit2_clar [.] deflateInit_
0.10% 0.00% libgit2_clar libgit2_clar [.] deflateInit2_
0.03% 0.00% libgit2_clar libgit2_clar [.] deflateReset (inlined)
0.02% 0.00% libgit2_clar libgit2_clar [.] deflateReset
0.02% 0.00% libgit2_clar libgit2_clar [.] inflateEnd
0.02% 0.00% libgit2_clar libgit2_clar [.] deflateEnd
0.01% 0.00% libgit2_clar libgit2_clar [.] deflateResetKeep
0.01% 0.01% libgit2_clar libgit2_clar [.] inflateReset2
0.01% 0.00% libgit2_clar libgit2_clar [.] deflateReset (inlined)
0.00% 0.00% libgit2_clar libgit2_clar [.] inflateStateCheck (inlined)
0.00% 0.00% libgit2_clar libgit2_clar [.] inflateReset (inlined)
0.00% 0.00% libgit2_clar libgit2_clar [.] inflateStateCheck (inlined)
0.00% 0.00% libgit2_clar libgit2_clar [.] deflateStateCheck (inlined)
0.00% 0.00% libgit2_clar libgit2_clar [.] inflateResetKeep (inlined)
```
Chromium
--------
```shell
cmake \
-DUSE_BUNDLED_ZLIB=ON \
-DUSE_CHROMIUM_ZLIB=ON \
-DCMAKE_BUILD_TYPE="RelWithDebInfo" \
-DCMAKE_C_FLAGS="-fPIC -fno-omit-frame-pointer" \
-GNinja \
..
ninja
perf record --call-graph=dwarf ./libgit2_clar
perf report --children
```
```
Samples: 97K of event 'cycles', Event count (approx.): 80862210917
Children Self Command Shared Objec Symbol
+ 3.31% 0.00% libgit2_clar libgit2_clar [.] git_zstream_get_output_chunk
+ 2.27% 0.01% libgit2_clar libgit2_clar [.] git_zstream_get_output
+ 0.55% 0.00% libgit2_clar libgit2_clar [.] git_zstream_get_output (inlined)
0.18% 0.00% libgit2_clar libgit2_clar [.] git_zstream_init
0.02% 0.00% libgit2_clar libgit2_clar [.] git_zstream_reset
0.00% 0.00% libgit2_clar libgit2_clar [.] git_zstream_free (inlined)
0.00% 0.00% libgit2_clar libgit2_clar [.] git_zstream_done
0.00% 0.00% libgit2_clar libgit2_clar [.] git_zstream_free
Samples: 97K of event 'cycles', Event count (approx.): 80862210917
Children Self Command Shared Objec Symbol
+ 2.55% 0.01% libgit2_clar libgit2_clar [.] deflate
+ 2.25% 1.41% libgit2_clar libgit2_clar [.] deflate_slow
+ 1.10% 0.52% libgit2_clar libgit2_clar [.] inflate
0.36% 0.00% libgit2_clar libgit2_clar [.] write_deflate
0.30% 0.03% libgit2_clar libgit2_clar [.] deflate_fast
0.28% 0.15% libgit2_clar libgit2_clar [.] inflate_fast_chunk_
0.19% 0.19% libgit2_clar libgit2_clar [.] inflate_table
0.17% 0.01% libgit2_clar libgit2_clar [.] inflateInit_
0.16% 0.00% libgit2_clar libgit2_clar [.] inflateInit2_ (inlined)
0.15% 0.00% libgit2_clar libgit2_clar [.] deflateInit_
0.15% 0.00% libgit2_clar libgit2_clar [.] deflateInit2_
0.11% 0.01% libgit2_clar libgit2_clar [.] adler32_z
0.09% 0.09% libgit2_clar libgit2_clar [.] adler32_simd_
0.05% 0.00% libgit2_clar libgit2_clar [.] deflateReset (inlined)
0.05% 0.00% libgit2_clar libgit2_clar [.] deflate_read_buf
0.03% 0.00% libgit2_clar libgit2_clar [.] inflateEnd
0.02% 0.00% libgit2_clar libgit2_clar [.] deflateReset
0.01% 0.00% libgit2_clar libgit2_clar [.] deflateEnd
0.01% 0.01% libgit2_clar libgit2_clar [.] inflateReset2
0.01% 0.00% libgit2_clar libgit2_clar [.] inflateReset (inlined)
0.00% 0.00% libgit2_clar libgit2_clar [.] adler32
0.00% 0.00% libgit2_clar libgit2_clar [.] inflateResetKeep (inlined)
0.00% 0.00% libgit2_clar libgit2_clar [.] deflateResetKeep
0.00% 0.00% libgit2_clar libgit2_clar [.] inflateStateCheck (inlined)
0.00% 0.00% libgit2_clar libgit2_clar [.] inflateStateCheck (inlined)
0.00% 0.00% libgit2_clar libgit2_clar [.] inflateStateCheck (inlined)
0.00% 0.00% libgit2_clar libgit2_clar [.] deflateStateCheck (inlined)
```
* Emulated mmap based write without pagefault handling is not
possible since IO happens outside of call to mmap and data is
written to mapped memory
* Potential emulation using userfaultfd() might be possible
* Use pread/pwrite to avoid updating position in file descriptor
* Emulate missing pread/pwrite on win32 using overlapped file IO
Now `USE_BUNDLED_ZLIB` can be set to the string `Chromium` to enable the Chromium implementation of zlib.
Removes doc comment on `git_blob_filter_options.version`, moves information to `git_blob_filter_options` doc comment to remain consistent with other options structures' documentation. `git_blob_filter_options_init` still needed; should be added in another commit/PR (it's out of the scope of this PR, #5759), update this documentation again.
zlib: Add support for building with Chromium's zlib implementation
Build with NO_MMAP
The `git_blob_filter_options_init` function should be included, to allow callers in FFI environments to let us initialize an options structure for them.
Adds info about initializing options with git_blob_filter_options_init
Add documentation for git_blob_filter_options.version
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
13 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot]. Want to support this open source service? Please star it : )