Each template received `commit_ids`, a list of ALL commit hashes known
to Itsy-Gitsy. For large repos with hundreds of thousands or millions
of commits, this huge list uses lots of memory and takes a long time
to search.
`limit_commit_ids_to_related` reduces the list to just a handful of
relevant commits for certain templates, which can give a massive
performance boost.
`limit_diff` used to apply to each thread, so you would get
NUM_THREADS*DIFF_LIMIT diffs, potentially much higher than expected.
This new implementation can still overshoot the requested limit, but
has a much tighter bound.
* parallel processing of multiple repositories
* parallel parsing of git history
* parallel template output
* configuration option for number of threads
* improved detection of exceeded size limits
Side effects of parallel execution are out-of-order output messages,
and larger potential overshoot in both processing and detecting when
limits are exceeded. It will stop if it exceeds the limits, but it
can exceed them by more before stopping.
Processing of extremely large repositories is several times faster.
The order of commits in the history output might not match the output
of `git log`, since the order of commits in the log can change when
splitting it up into smaller pieces.
Large refactoring that:
* allow any template to be used any number of times
* combine config's gitsy_templates and gitsy_outputs into gitsy_outputs
* improves safety of output file names
* prepares foundation for slugified permalinks
* prepares foundation for parallelizing more execution
Trevor Bentley