Files
poky/meta/classes
Alexander Kanavin da95ad49d8 package_rpm: restrict rpm to 4 threads
TL;DR version:

with this, and the previous compression level changes
I am seeing drastic speedups in package_write_rpm completion times:

webkitgtk goes from 78 seconds to 37 seconds
glibc-locale goes from 399 seconds to 58 seconds (!)

The long version:

rpm uses multithreading for two purposes:

- spawning compressors (which are nowadays themselves
multi-threaded, so the feature is not as useful as it once
was)
- parallel file classification

While the former behaves well on massively parallel CPUs
(it was written and verified here :), the latter was then added
by upstream and only benchmarked on their very old, slow laptop,
apparently:
41f0e214f2

On anything more capable it starts showing pathologic behavior,
presumably from spawning massive amount of very short-lived threads,
and then having to synchronize them. For example classifying glibc-locale
takes
5m20s with 256 threads (default on my machine!)
1m49s with 64 threads
59s with 16 threads
48s with 8 threads

Even a more typical recipe like webkitgtk is affected:
47s with 256 threads
32s with 64 threads
27s with 16 or 8 threads

I have found that the optimal amount is actually four: this also
means that only four compressors are running at a time, but
as they're themselves using threads, and typical recipes are dominated
by just two or three large packages, this does not affect overall
completion time.

(From OE-Core rev: 896192604d84a6f77095f23cd13232e249b7aac5)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-12-02 06:23:20 -08:00
..
2021-08-02 15:44:10 +01:00
2021-08-02 15:44:10 +01:00
2021-08-02 15:44:10 +01:00
2021-08-02 15:44:10 +01:00
2022-01-07 14:39:17 +00:00
2021-08-02 15:44:10 +01:00
2021-08-02 15:44:10 +01:00
2022-10-20 15:36:02 +01:00
2022-12-01 19:35:05 +00:00
2021-08-02 15:44:10 +01:00
2021-08-02 15:44:10 +01:00