Reduce time to apply patches

[PhilippvK]

Most of the flow processing time goes on git operations (times in seconds, from se-henri server at DLR running 4 CPUs, piping demo.py through "ts" util)

| phase | seconds |

| --- | --- |

| loading core_desc | 9 |

| applying seal5 patches | 9 |

| transforming | 14 |

| generating instruction patches | 7 |

| applying instruction patches | 64 |

The git python module used to apply the patches forks a lot of git processes (hundreds), which is where most of the 64 seconds is consumed. Possible remedies:

1. Aggregate the patches into a single commit, rather than the commit-per-instruction approach. Having lots of commits can be good for debugging (bisecting) and cherry-picking but in a real "product" might want to squash them into a single "Add FOO extension" anyway?

2. Write the patches as multiple commits for the instructions into a single mail patch, then apply that with "git am" (whereas current approach is "git apply + add + commit" with each instruction patch

Originally posted by @thomasgoodfellow in #52

[PhilippvK]

For each patch, gitpython spawn those 7 commands:

[seal5.flow::INFO] Applying patch 'riscv_intrinsics_target_Example' on 'llvm'
DEBUG:git.util:sys.platform='linux', git_executable='git' DEBUG:git.util:sys.platform='linux', git_executable='git' 
DEBUG:git.cmd:Popen(['git', 'diff', '--cached', '--abbrev=40', '--full-index', '--raw'], cwd=/tmp/seal5_llvm, stdin=None, shell=False, universal_newlines=False)
DEBUG:git.cmd:Popen(['git', 'diff', '--abbrev=40', '--full-index', '--raw'], cwd=/tmp/seal5_llvm, stdin=None, shell=False, universal_newlines=False)
DEBUG:git.cmd:Popen(['git', 'status', '--porcelain', '--untracked-files'], cwd=/tmp/seal5_llvm, stdin=None, shell=False, universal_newlines=False)
DEBUG:git.cmd:Popen(['git', 'apply', '/tmp/seal5_llvm/.seal5/patches/llvm/riscv_intrinsics_target_Example.patch'], cwd=/tmp/seal5_llvm, stdin=None, shell=False, universal_newlines=False)
DEBUG:git.cmd:Popen(['git', 'add', '-A'], cwd=/tmp/seal5_llvm, stdin=None, shell=False, universal_newlines=False) 
DEBUG:git.cmd:Popen(['git', 'cat-file', '--batch-check'], cwd=/tmp/seal5_llvm, stdin=<valid stream>, shell=False, universal_newlines=False)
DEBUG:git.cmd:Popen(['git', 'cat-file', '--batch'], cwd=/tmp/seal5_llvm, stdin=<valid stream>, shell=False, universal_newlines=False)

6 of of those are just for syncing the state of the repo with the git.Repo object. If we are willing to allow missmatches between the GitPython state and the local repo, we might be better off directly calling git apply via subprocesses.

I will add an optional flag to the flow and compare the proposed approach with the GitPython one.

[PhilippvK]

UPDATE: There seems to be no benefit in calling appy & add directly, because those two commands are the only slow ones. The boilerplate commands do not add any extra time...

Another workaround for this might be applying multiple (merged) patches together...