The wayland-scanner is missing from SDKs with weston, but the weston build
requires wayland-scanner. Allow the distro feature in order to include
the wayland-scanner packages via nativesdk-packagegroup-sdk-host.bb.
(From OE-Core rev: 858cc6f257e22e39df83f4808ea27c6d12cd1b80)
Signed-off-by: Tom Hochstein <tom.hochstein@nxp.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The url changes from the script accidentally corrupted this mirror
url, fix it.
(From OE-Core rev: a16dd60fb058ec2257eb1c6c0baa86e11e78cb42)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is uncertainty about the default branch name in git going forward.
To try and cover the different possible outcomes, add branch names to all
git:// and gitsm:// SRC_URI entries.
This update was made with the script added to contrib in this patch which
aims to help others convert other layers.
(From OE-Core rev: b51c405faf6f8c0365f7533bfaf470d79152a463)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We want things to be reproduicble and the variable doesn't really change
much any more. Drop the remaining uses and make those code paths always
active.
(From OE-Core rev: d15fb02c7ee7da50e322d74bc6a545234e20c7f3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The reproducibility code was originally developed as separate
standalone class but development is no longer experimental and
the code is widely tested and used by default for poky. Reproducible
builds are the direction we need to take as a project.
Transition the core variable definitions to bitbake.conf as part of
a move to make these part of the default workflow.
This also helps reduce test matrix complexity as there is now one
code path.
(From OE-Core rev: f38a8de19550ae216575b5b39163666f74b07e2d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If gpg is used, it will find the first gpg agent in the path, this
may lead to issues where gpg comes from the host, and the agent
comes from a gnupg-native due to package signing. The versions
being out of sync causes gpg to fail.
(From OE-Core rev: 2b59fb9d52a405a32a1d069d4c5320b72fbd35ce)
Signed-off-by: Daniel McGregor <daniel.mcgregor@vecima.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Tasks shouldn't vary dependning on the value of BB_CURRENTTASK. They
happen not to due to when bitbake sets this but to fix other issues,
bitbake needs to set it earlier. Therefore exclude from hashes
globally.
(From OE-Core rev: 04a4ed1c509b0ef6af6d9ff53901c674ca402252)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, installed packages are listed for images in image-info.txt, but
not for SDKs in sdk-info.txt. Add TOOLCHAIN_HOST_TASK and
TOOLCHAIN_TARGET_TASK to the output variables in sdk-info.txt.
Moreover, package output files for the SDK host are empty because
PKGDATA_DIR defaults to the target directory. Fix this bug and create a new
variable called PKGDATA_DIR_SDK which stores the correct path for the SDK
host package data.
(From OE-Core rev: 82e6172c1df378dff4e503aa878501c08937b5bb)
Signed-off-by: Andres Beltran <abeltran@linux.microsoft.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
zstd has similar time and space performance in compression but is
vastly faster in decompression, which benefits rootfs creation
(especially when installing very large packages) and on-target
package installation.
Also, ensure ZSTD_THREADS doesn't change sstate checksums. The
detailed explanation is in the commit making similar change for
XZ_THREADS.
(From OE-Core rev: e72c0b94554a9bc293844ec2bddb0c04ea19791d)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you leave PARALLEL_MAKEINST at its default from PARALLEL_MAKE,
sstate checksums are fine and don't change as the number of cores do.
If you override it to a specific value, this does the cause the sstate
checksums to change. We don't want the checksums to change if the
value of this variable differs.
Therefore set a vardepvalue so a specific value is used for checksum
purposes.
(From OE-Core rev: 0a20d7f462b42800a8420dbb57e6ac8b84770b39)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
rpm output packages currently depend on the value of XZ_THREADS which
is ok if left to the default value but problematic if system limits
are set such as on the autobuilder.
Force the value to a specific one in the hashes for better sstate reuse
and consistent rpm task checksums.
(From OE-Core rev: 0296dc71c01d1b7953d74ef37d56429e2f4fcfab)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is probably a good idea if the comment that describes how variable
overrides work use the new override syntax...
(From OE-Core rev: 1fd8882db9011a6fe18da7611fba3426fd5cb00d)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add SDKPATHINSTALL which is used as the default install location of the SDK
instead of SDKPATH. This means the default install path isn't encoded into
every SDK binary, meaning if a date is used there the entire SDK doesn't
have to rebuild. Most distros can switch to only customise SDKPATHINSTALL
meaning more sstate reuse too.
[YOCTO #14100]
(From OE-Core rev: bc4ee5453560dcefc4a4ecc5657df5cc1666e153)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is the result of automated script conversion:
scripts/contrib/convert-overrides.py <oe-core directory>
converting the metadata to use ":" as the override character instead of "_".
(From OE-Core rev: 42344347be29f0997cc2f7636d9603b1fe1875ae)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was an oversight when the distro feature was introduced.
(From OE-Core rev: 4a985ff0dd13c455dadf85f4ad35a0a42f4bd9f1)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As is the pattern with all lists, put the list of MIRRORs in
alphabetical order, and add an entry for SAMBA_MIRROR as it appears in
the samba recipe from the meta-networking layer.
(From OE-Core rev: 3301536f821dce86cd3683d9c50abea2d39c6c20)
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Limits the number of OpenMP threads to match BB_NUMBER_THREADS. This
prevents OpenMP (libgomp in particular) from falling back to using all
the available CPUs, which behaves poorly when attempting to limit build
usage, especially when attempting to build in a container.
(From OE-Core rev: fd2b8986aef11609123da917aaf6bcbe41f63112)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a few more tools to the BUILD_* list, to match the target tool list.
(From OE-Core rev: 633393830aea0120c4a2a165917040223630c49d)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
HOSTTOOLS_DIR contains symlinks to host tools, not copies
(From OE-Core rev: fb7692da7faa49b370680decbbaceaeb85b6889d)
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We've had a lot of concern about the readability of diffs of this
path setting, refactor to avoid long line length.
(From OE-Core rev: f3930cba16ccbc4b76ed14187d5586d6479bc59e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bitbake depends on this variable while looking for pseudo.log file in
case of fakeroot task failure.
(From OE-Core rev: e8e9e5f62b144828387e20eec2e66fcd61edd0d3)
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
musl is configured to use /lib and /usr/lib for ppc64 unlike glibc where
it is expected to have libs in /lib64 and /usr/lib64
(From OE-Core rev: 998d15a86282d9cf57fbd0a4eb76a56b9938a348)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than trying to set the umask of every task to the correct
value for determinism, set one value globally. This uses a new
bitbake variable so bump the minimum version to match.
This fixes strange determinism issues in at least quilt-ptest,
valgrind-ptest and kernel-devsrc.
(From OE-Core rev: b07b7deeae3d519d9998d583592f3e4f8f2802b3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We never recommend setting RDEPENDS or RPROVIDES without a package name
against them. The default in bitbake.conf is legacy only, drop it.
The python recipe was trying to add to the empty variable in the native case
fix that too.
(From OE-Core rev: b8bbc1bbe282cce2ea4d0ff293f931f6caf6153b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This reverts commit f5b29367af4d8e5daea5771264774aa49519f9a8.
Will use ccache-native which is more reliable.
(From OE-Core rev: 1b659623430e1a6e6dd266e65bab7ca8155a7138)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Builds were failing on WSL2 which turns out to be due to accesses to
/run/ on those systems. Add this to PSEUDO_IGNORE_PATHS to fix
builds on WSL2.
[YOCTO #14175]
(From OE-Core rev: 1d1bf51217e8b4d54af28739d3271484ee5a7974)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is about to be needed by the psplash recipe and is a standard part
of coreutils so usually present on hosts.
(From OE-Core rev: 99206612a58e6b69718910e0806d9d36997b2fba)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
ccache 4.x has hard dependencies on cmake-native (used as
build system) and zstd, which means inserting ccache-native
as DEPENDS into everything creates circular dependencies which
are impossible to break.
ccache 3.x did not have this problem as it used plain makefiles
and an in-tree copy of zlib.
(From OE-Core rev: f5b29367af4d8e5daea5771264774aa49519f9a8)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently there's a limitation that Extensible SDKs can only be build on a
machine with the same architecture as the SDK.
However, SDKMACHINE defaults to x86-64, which means eSDKs cannot be built
out of the box on arm64 systems.
By defaulting SDKMACHINE to ${BUILD_ARCH} we generate SDKs for the build
host, so they'll always work out of the box.
(From OE-Core rev: 539cd4a5fbb9e4507d90bfb1165fcb06907b46d8)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This also effectively reverts commit b6d30c21b0:
bitbake.conf: Extend PSEUDO_IGNORE_PATHS to ${COREBASE}/meta
The contents of ${COREBASE}/meta were ignored as pyc files could be
generated for the contents of the lib subdirectory if python modules
were imported within a pseudo context. However this doesn't protect us
from pyc files being generated in the lib directories for other layers.
It's far better to tell python not to produce pyc files when running
under pseudo (by setting the PYTHONDONTWRITEBYTECODE variable) as this
will cover any location where pyc files could possibly be created. This
variable is set in FAKEROOTBASEENV so that it applies to the
bitbake-worker instance for fakeroot tasks, preventing pyc files from
being generated for imports in python tasks themselves.
Also add a test case to ensure that pyc files are not created in tasks
which are executed under pseudo.
(From OE-Core rev: 73d538f20743017a44cea4c20dbe09a0327cfc71)
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use oe.path.canonicalize() to canonicalize the paths in
PSEUDO_IGNORE_PATHS before passing them to pseudo. This is needed since
pseudo will compare them to paths that are canonicalized.
(From OE-Core rev: 1f2a497daa09cf3459e27ad6e0e8513938b52c79)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This reverts commit cc30537bb4d0815c4d69f0b1a1a8a200085902d5.
There are issues with this change where the build directory may be included
within the paths included in BBLAYERS, which is particularly problematic
where people are using kas.
An alternative to this to handle the pyc file issue is being worked on but
revert this for now whilst that fix is developed.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of ignoring ${COREBASE}/meta in PSEUDO_IGNORE_PATHS (which may
or may not ignore all layers depending on how they are named and placed
under ${COREBASE}), ignore all layers.
(From OE-Core rev: cc30537bb4d0815c4d69f0b1a1a8a200085902d5)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When ccache enabled as below:
INHERIT += "ccache"
CCACHE_DIR = "/path/build/ccache"
There comes do_install failure for some recipes randomly, take
linux-libc-headerswhen as example.
$ cat /path/build/tmp/work/corei7-64-wrs-linux/linux-libc-headers/5.8-r0/pseudo/pseudo.log
[snip]
path mismatch [1 link]: ino 243004209 db '/path/build/ccache/6/stats' req '/path/build/ccache/7/stats.lock'.
[snip]
Exclude ${CCACHE_DIR} from pseudo database to fix the potential
do_install failure.
(From OE-Core rev: b65e5280bf9e7f1f9c8e6acabab79bcf209e5342)
Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Unfortunately, .pyc files can be generated in meta/lib/oe which corrupt the pseudo
database so we need to extend the ignore list to cover this as well.
(From OE-Core rev: e0cb6dd689a362d8433caa14cc5a9fdd5eb44923)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a pretty big change to the way pseudo operates when used in OpenEmbedded.
Normally, pseudo monitors and logs (adds to its database) any file created or
modified whilst in a fakeroot environment. There are large numbers of files
we simply don't care about the permissions of whilst in that fakeroot context,
for example ${S}, ${B}, ${T}, ${SSTATE_DIR}, the central sstate control
directories,
This change uses new functionality in pseudo to ignore these directory trees,
resulting in a cleaner database with less chance of "stray" mismatches if files
are modified outside pseudo context. It also should reduce some overhead from
pseudo as the interprocess round trip to the server is avoided.
There is a possible complication where some existing recipe may break, for
example, we found a recipe which was writing to "${B}/install" for
"make install" in do_install and since we listed ${B} as not to be tracked,
there were errors trying to chown root for files in this location.
This patch fixes a few corner cases in OE-Core when used with this new
ignore list:
* The archiver directory matched a "${WORKDIR}/deploy*" pattern so was renamed
to something else since that directory does need its root permissions
* The ${S} and ${B} ignoring is conditional on them being different to ${WORKDIR}
* package_write_* task output (the debs/rpms/ipks) are now owned by the build
user so we don't want the file ownership information in the hashequiv outhash
calculation even if they are built under pseudo.
* The fontcache postinstall intercept is run under qemu outside of pseudo context
so delete files it may delete up front where pseudo can see this.
* SSTATE_DIR is in PSEUDO_PATHS_IGNORE, which is in FAKEROOTENV which is cached
by bitbake. We therefore need to trigger reparsing if this changes, which means
SSTATE_DIR can be in BB_HASHBASE_WHITELIST but not BB_HASHCONFIG_WHITELIST.
Rework the variables to handle this. This otherwise breaks some of our sstate
tests in oe-selftest.
* Ignore the temp directory wic uses for rebuilding rootfs.
(From OE-Core rev: ad8f5532ffaead9a5ad13e1034fe9e5e1b7979f4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* move TCMODE and TCLIBC from defaultsetup.conf to bitbake.conf
* set CACHE as it was in defaultsetup.conf and drop it from defaultsetup.conf
* most if not all DISTROs are now including defaultsetup.conf and
TCLIBC is pretty much expected to be always set correctly, e.g.:
meta/recipes-core/systemd/systemd_243.2.bb: if d.getVar('TCLIBC') == "musl":
meta/recipes-devtools/gcc/gcc-runtime.inc: if [ "${TCLIBC}" != "glibc" ]; then
meta/recipes-devtools/gcc/libgcc.inc: if [ "${TCLIBC}" != "glibc" ]; then
meta/recipes-devtools/icecc-toolchain/nativesdk-icecc-toolchain_0.1.bb:ENV_NAME="${DISTRO}-${TCLIBC}-${SDK_ARCH}-@TARGET_PREFIX@${DISTRO_VERSION}.tar.gz"
meta/recipes-devtools/valgrind/valgrind_3.15.0.bb:RRECOMMENDS_${PN} += "${TCLIBC}-dbg"
meta/recipes-kernel/linux/kernel-devsrc.bb:RDEPENDS_${PN} = "bc python3 flex bison ${TCLIBC}-utils"
meta/classes/buildhistory.bbclass:BUILDHISTORY_DIR_IMAGE = "${BUILDHISTORY_DIR}/images/${MACHINE_ARCH}/${TCLIBC}/${IMAGE_BASENAME}"
meta/classes/cross-canadian.bbclass: if d.getVar("TCLIBC") in [ 'baremetal', 'newlib' ]:
meta/classes/kernel.bbclass: tclibc = d.getVar('TCLIBC')
meta/classes/toaster.bbclass: BUILDHISTORY_DIR_IMAGE_BASE = e.data.expand("%s/images/${MACHINE_ARCH}/${TCLIBC}/"% BUILDHISTORY_DIR)
(From OE-Core rev: 57aa60ef6422568b425b6ccc4451567efc578469)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* similar to kernel-artifact-names for other recipes/bbclasses which
need to use some deployed artifacts
* bitbake.conf: move IMAGE_BASENAME, IMAGE_VERSION_SUFFIX, IMAGE_NAME,
IMAGE_LINK_NAME variables
* image_types.bbclass: move IMAGE_NAME_SUFFIX variable
* currently IMAGE_NAME_SUFFIX is used only by image.bbclass,
image_types.bbclass and meta/recipes-core/images/build-appliance-image_15.0.0.bb
but if it's needed by some recipe which isn't itself an image, then
it's useful in bitbake.conf, e.g. we have a recipe for creating
VirtualBox appliances which combines .wic.vmdk with .ovf file to
create .zip with appliance, but for that we need the filename of
.wic.vmdk which now contains IMAGE_NAME_SUFFIX
4980ce52a4/meta-luneos/recipes-core/images/luneos-emulator-appliance.inc (L24)
* we were hardcoding .rootfs suffix where needed, but for quite long
time it's configurable with IMAGE_NAME_SUFFIX since:
commit 380ee36811939d947024bf78de907e3c071b834f
Author: Patrick Ohly <patrick.ohly@intel.com>
Date: Mon Mar 7 18:07:52 2016 +0100
image creation: allow overriding .rootfs suffix
and might not match with hardcoded .rootfs, so make it easier to
use IMAGE_NAME_SUFFIX where needed even without inheritting whole
image_types.bbclass
[YOCTO #12937]
(From OE-Core rev: 456b700d51a5052a285a8477304f902c335223be)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When the PR service is enabled a number of small changes may happen
to variables. In the do_package step a call to package_get_auto_pr
will end up setting PRAUTO and modifying PKGV (if AUTOINC is there).
PRAUTO is then used by EXTENDPRAUTO, which is then used to generate
PKGR.
Since this behavior typically happens BEFORE the BB_UNIHASH is
calculated for do_package, we need a way to defer the expansion
until after we have the unihash value.
Writing out the pkgdata files w/o AUTOPR and PKGV (AUTOINC) expanded
to placeholder values is the easiest way to deal with this. All other
variables are expanded as expected.
In the next task, typically do_packagedata, we will then use the
UNIHASH from the do_package to get the PR (AUTOPR) as well as
generate the AUTOINC replacement value (now PRSERV_PV_AUTOINC).
The do_packagedata then translates the placeholders to the final values
when copying the data from pkgdata to pkgdata-pdata-input.
Also update the prservice test case. With unihash, just changing the
do_package (via a _append) will not change the PR. So write the date
to a specific file that is incorporated into the unihash to ensure it
is always different for the test. Various assert messages were also
updated to make it easier to figure out where/why a problem occured.
(From OE-Core rev: 2e32f37b0e4abc438c8f60e673cd18a5cc110768)
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since BitBake commit 0d5cdd0c0d65f2f81c3af0f3767fee86c4142c3a
TERM is no longer preserved in the environment by BitBake, so there
should be no need to whitelist it by default.
(From OE-Core rev: 72c4222b095a49d5ba8252abbae5025196519cd5)
Signed-off-by: Jacob Kroon <jacob.kroon@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This reverts commit b1f15f651461d07a8cfbd3bdcfea0e89f195212d.
The variable is used by multiple other layers and users are requesting
it be added back.
(From OE-Core rev: 49d0f822618890b61d2498b07dda6418f885321e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Variable DEPLOY_DIR_TOOLS was introduced in 2007:
commit f751a20152
Author: Richard Purdie <richard@openedhand.com>
Date: Wed Aug 8 21:02:39 2007 +0000
bitbake.conf: Sync with OE.dev
apparently never used so get rid of it.
(From OE-Core rev: b1f15f651461d07a8cfbd3bdcfea0e89f195212d)
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
