48a6d84de1
Prevent the scheduler from starting new tasks if the current cpu or io
pressure is above a certain threshold and there is at least one active
task. This threshold can be specified through the
"BB_PRESSURE_MAX_{CPU|IO}" variables in conf/local.conf.
The threshold represents the difference in "total" pressure from the
previous second. The pressure data is discussed in this oe-core commit:
061931520b buildstats.py: enable collection of /proc/pressure data
where one can see that the average and "total" values are available.
From tests, it was seen that while using the averaged data was somewhat
useful, the latency in regulating builds was too high. By taking the
difference between the current pressure and the pressure seen in the
previous second, better regulation occurs. Using a shorter time period
is appealing but due to fluctations in pressure, comparing the current
pressure to 1 second ago achieves a reasonable compromise. One can look
at the buildstats logs, that usually sample once per second, to decide a
sensible threshold.
If the thresholds aren't specified, pressure is not monitored and hence
there is no impact on build times. Arbitary lower limit of 1.0 results
in a fatal error to avoid extremely long builds. If the limits are higher
than 1,000,000, then warnings are issued to inform users that the specified
limit is very high and unlikely to result in any regulation.
The current bitbake scheduling algorithm requires that at least one
task be active. This means that if high pressure is seen, then new tasks
will not be started and pressure will be checked only for as long as at
least one task is active. When there are no active tasks, an additional task
will be started and pressure checking resumed. This behaviour means that
if an external source is causing the pressure to exceed the threshold,
bitbake will continue to make some progress towards the requested target.
This violates the intent of limiting pressure but, given the current
scheduling algorithm as described above, there seems to be no other option.
In the case where only one bitbake build is running, the implications of
the scheduler requirement will likely result in pressure being higher
than the threshold. More work would be required to ensure that
the pressure threshold is never exceeded, for example by adding pressure
monitoring to make and ninja.
(Bitbake rev: 502e05cbe67fb7a0e804dcc2cc0764a2e05c014f)
Signed-off-by: Aryaman Gupta <aryaman.gupta@windriver.com>
Signed-off-by: Randy Macleod <randy.macleod@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Poky
Poky is an integration of various components to form a pre-packaged build system and development environment which is used as a development and validation tool by the Yocto Project. It features support for building customised embedded style device images and custom containers. There are reference demo images ranging from X11/GTK+ to Weston, commandline and more. The system supports cross-architecture application development using QEMU emulation and a standalone toolchain and SDK suitable for IDE integration.
Additional information on the specifics of hardware that Poky supports is available in README.hardware. Further hardware support can easily be added in the form of BSP layers which extend the systems capabilities in a modular way. Many layers are available and can be found through the layer index.
As an integration layer Poky consists of several upstream projects such as BitBake, OpenEmbedded-Core, Yocto documentation, the 'meta-yocto' layer which has configuration and hardware support components. These components are all part of the Yocto Project and OpenEmbedded ecosystems.
The Yocto Project has extensive documentation about the system including a reference manual which can be found at https://docs.yoctoproject.org/
OpenEmbedded is the build architecture used by Poky and the Yocto project. For information about OpenEmbedded, see the OpenEmbedded website.
Contribution Guidelines
The project works using a mailing list patch submission process. Patches should be sent to the mailing list for the repository the components originate from (see below). Throughout the Yocto Project, the README files in the component in question should detail where to send patches, who the maintainers are and where bugs should be reported.
A guide to submitting patches to OpenEmbedded is available at:
https://www.openembedded.org/wiki/How_to_submit_a_patch_to_OpenEmbedded
There is good documentation on how to write/format patches at:
https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines
Where to Send Patches
As Poky is an integration repository (built using a tool called combo-layer), patches against the various components should be sent to their respective upstreams:
OpenEmbedded-Core (files in meta/, meta-selftest/, meta-skeleton/, scripts/):
- Git repository: https://git.openembedded.org/openembedded-core/
- Mailing list: openembedded-core@lists.openembedded.org
BitBake (files in bitbake/):
- Git repository: https://git.openembedded.org/bitbake/
- Mailing list: bitbake-devel@lists.openembedded.org
Documentation (files in documentation/):
- Git repository: https://git.yoctoproject.org/cgit/cgit.cgi/yocto-docs/
- Mailing list: docs@lists.yoctoproject.org
meta-yocto (files in meta-poky/, meta-yocto-bsp/):
- Git repository: https://git.yoctoproject.org/cgit/cgit.cgi/meta-yocto
- Mailing list: poky@lists.yoctoproject.org
If in doubt, check the openembedded-core git repository for the content you intend to modify as most files are from there unless clearly one of the above categories. Before sending, be sure the patches apply cleanly to the current git repository branch in question.