Draft: Enable a fully parallel build
Rewrite the way rules.gen calls rules.real to avoid recursive make
and enable more parallelism.
Also change the way rules calls rules.gen to enable arch-dependent
and -independent packages to be built in parallel.
I compared the time taken for CI and for local builds with several different build types, before and after these changes. The local builds were all done on a desktop system with a Ryzen 5 3600 (Zen 2 6-core 12-thread) CPU and storage spread between SSD and HD with bcache. It would be interesting to also get results with more CPU threads and/or pure SSD storage.
| Build type | quick Salsa | quick local | indep local | arch local | full local |
|---|---|---|---|---|---|
| Old time | 38:47 | 9:44 | 13:42 | 2:14:23 | 2:23:02 |
| New time | 37:47 | 6:34 | 13:14 | 2:06:25 | 2:09:42 |
| Ratio | 0.97 | 0.67 | 0.97 | 0.94 | 0.91 |
It looks like quick builds and full builds see the most benefit from this. But so far I have only timed one build for each of these, so this is statistically a bit dubious.
Depends on !1951 (merged)