refactor: support parallel execution

* parallel processing of multiple repositories
* parallel parsing of git history
* parallel template output
* configuration option for number of threads
* improved detection of exceeded size limits

Side effects of parallel execution are out-of-order output messages,
and larger potential overshoot in both processing and detecting when
limits are exceeded.  It will stop if it exceeds the limits, but it
can exceed them by more before stopping.

Processing of extremely large repositories is several times faster.

The order of commits in the history output might not match the output
of `git log`, since the order of commits in the log can change when
splitting it up into smaller pieces.
This commit is contained in:
Trevor Bentley
2023-01-19 20:55:26 +01:00
parent 48b6dd056a
commit d83f682aa2
6 changed files with 490 additions and 300 deletions
+9
View File
@@ -159,6 +159,15 @@ syntax_highlight = true
#
syntax_highlight_theme = "base16-ocean.light"
# Number of threads to use for parallel processing
#
# Specify a specify a specific number of threads/cores to split
# parallel processing across, or set to 0 to auto-detect.
#
# The most practical values here are 0 (auto-detect, which typically
# means use all cores), or 1 (to disable parallelism).
threads = 0
# Limits maximum number of history items (i.e. git log) to parse.
#
# After the limit is reached, no more history items will be processed or stored