The current test code contains a subtle race. It is building an ext4,
then trying to reuse that image in a wic build for the same recipe.
Whilst this happened to work, the code is within its rights to clean
up the output before the wic code runs.
Avoid this by using separate image targets instead.
(From OE-Core rev: 590398080e81fb5e2f81e12b8900858837dfe25f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
CVE_VERSION_SUFFIX in "patch" to treat version string with suffix "pX"
or "patchX" as patched release.
also update testcases to cover this changes and set CVE_VERSION_SUFFIX
for sudo.
(From OE-Core rev: 8076815fc2ffc8f632e73527ce2b7d158a29e9ea)
Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The hardcoded path issue was fixed, drop the exclusion.
(From OE-Core rev: 58e02c83dcf4d6c40c206a0371ef5d7f8db2b272)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The meson determinism issue is the same frozenset issue we encountered with
python itself. Remove the problematic pyc file until upstream work out the
best way to address the issue (as with core python).
(From OE-Core rev: ad00107dc02b3211f5d1a7fe889f538ee2bc064d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This test is to keep track of recipes that do not have a HOMEPAGE or DESCRIPTION,
with exceptions to recipes whose SRC_URI are not linked to external sources.
Results of this test: it take about 2 to 3 minutes for this test to complete execution
depending on the laptop used, Recipes that didn't have a homepage or description were found,
and using list data structure instead of set() returned an ordered list of recipes that
didn't have a homepage. Hence it's easier to view recipes with missing homepage and description.
[YOCTO #13471]
(From OE-Core rev: 5e0cfce7a126c194bfb7855952355c62eae3831a)
Signed-off-by: Dorinda Bassey <dorindabassey@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
go-dep was an effort for dependency management before go modules, which
since 2020 has been deprecated in favor of go modules. Since its not
developed any longer and go mdules is officially supported, this should
be retired from OE-core as well.
(From OE-Core rev: 1e7ed44d87034446f1d07692c9378c3b0a8a9dd3)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Cc: Otavio Salvador <otavio.salvador@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We have fixes for perf reproducibility, so we can drop it from the
exclusion list.
(From OE-Core rev: 2989779fa244ec2c2c714c27f67753eefeeaf4af)
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some platforms do not use ttyS* for their serial consoles (e.g., qemuarm
and qemuarm64). The hardcoding of this can cause issues. Modify
runqemu to use the serial consoles defined in SERIAL_CONSOLES instead of
hardcoding.
(From OE-Core rev: 9dea4cd2f9f46ab3a75562639a22d8f56b4d26af)
Signed-off-by: Jon Mason <jon.mason@arm.com>
Change-Id: I746d56de5669c955c5e29d3ded70c0a4d3171f17
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a recipe that explicitly searches /usr/include, and use that in
oe-selftest to verify that host include paths are correctly causing
build failures.
(From OE-Core rev: b3e3eba796b843021b264f0e98dc30f983775d58)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit adds an optional layerid argument which can
be passed to the bitbake-layers create-layer command.
This allows for creation of a layer with a layer id
different to that of the layer's name. The default
behaviour of the command where the layer's id is set
to the layer's name is still retained.
(From OE-Core rev: 8f896bd9b34f19535838757c8f3049bae438e4fe)
Signed-off-by: Charlie Davies <charles.davies@whitetree.xyz>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The help message for the optional argument of setting
the priority of the new layer was incorrect.
(From OE-Core rev: d10d928c6030951994bc2404cde78137c86917a4)
Signed-off-by: Charlie Davies <charles.davies@whitetree.xyz>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Under systemd weston's socket is in /run, not in /run/user/0.
(From OE-Core rev: ea7dfd857ce3779745dccda0cbfd93b5e5de1a36)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are build paths which are in generated sources injected
into ${PN}-src. These are in generated files and don't influence the
binary output. As such we can correct them at do_package time by
tweaking in an appropriately injected function. This fixes
the reproducbility of the recipe.
(From OE-Core rev: cc1de818247c9ae329fdc0536658b3968a4caea2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a patch adding sorting to a couple of points in the Makefiles
which removes most of the determinism issues in ltp.
Build swapon before the main build to ensure libswapon.o is built
deterministically as it races with swapoff.
All issues reported on the upstream mailing list.
(From OE-Core rev: 0f51f9a37e5d058bce28cfe7b9a32a895f83c091)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This change adds core-image-weston to the reproducible build test.
For this we also need to update the static-group and static-passwd.
Besides the world package build which includes the wayland package,
this adds a wayland/weston-based image to the reproducible build test.
(From OE-Core rev: ffc4de47988ccf7568eecc8a27e8964beeaaacfb)
Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a hack to hardcode in specific rpaths which we then remove,
allowing the build to be reproducible.
Strip build patches out of one of the test scripts too.
(From OE-Core rev: 18299a114c66280ba2d00becc3fae235d3a6cbd7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a patch to address two determinism issues and allow reproducible
builds.
Also strip full paths we don't need out of some ptest files,
we can use the installed binaries.
(From OE-Core rev: 14e7cc6c6040b983024bec56fbd32d9101507ff0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When debugging why a single recipe doesn't reproduce, its a pain
to wait for the world to rebuild from scratch. Update the selftest
to allow this to be configured, for example you could set
targets as ['perf'] and sstate_targets as ['virtual/kernel']
and then it should only be rebuilding perf in the test rather than
things like the toolchain (parts of the kernel may be unavoiable
as they're not in sstate).
Can be run as:
OEQA_DEBUGGING_SAVED_OUTPUT=/tmp/perf-diffoscope oe-selftest -r reproducible.ReproducibleTests.test_reproducible_builds
to save diffoscope output.
(From OE-Core rev: 132a17d02f29711572e14a2f38a841323fbb6df6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
LTO likely doesn't buy us much here, disable it to allow the binaries
to be reproducible.
(From OE-Core rev: b4b5e52b5bd98deb0855700e4aae6228d52e5a83)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add sorting to the globbing within the Makefile to make the output
reproducible.
(From OE-Core rev: c43ae151f572786818fe048233b4bbfd6b0ba2cf)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove hardcoding the build configuration into the help/version output
from swig to make the binaries reproducible.
(From OE-Core rev: 7ed28ae9717ea9dad4e131012186d5f08e8f0bec)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add sorting to wildcard expansion in the makefile to make builds
reproducible.
(From OE-Core rev: 5541ba76ccc0c416f315bc0dc14a20a33059bd5f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The library hardcodes paths to the python library internally and currently
these are build paths. Fix this to use the correct target path and
fix reproducibility in the process.
(From OE-Core rev: 4c190e8c9a2b53de3def1065fb6e479b72d040dc)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need to use CFLAGS with the correct WORKDIR in them, replace those
in the sysroot file with the ones appropriate to the current recipe.
(From OE-Core rev: 45edf189961aff1858be9bb7b63116073c0a0c10)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Configure was swallowing our cflags meaning the resulting binaries were
not reproducible. Tweak configure not to do that and fix reproducibility.
(From OE-Core rev: 1395269a6fae5cec01d721e2ad52bd7cb8fc38ce)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a configuration option to pass in srcdir, removing hard coded
build paths from the binaries.
(From OE-Core rev: d7e92e3a22f0c87aff4f452b51f50ec417d2949b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The compressed docs contained a timestamp of the original file which
meant the SDE clamping during package creation didn't work. The
benefits of compression are minor, decompress the files to avoid
the reproducibility issues.
(From OE-Core rev: a3b59a67bea16899b57a0d187120c988495da4d5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Distros may not use shadow for login manager
secondly login --help returns 1 only with shadow provided login
utility, with busybox it returns normal 0, the testcase clearly
assumes shadow being part of image, therefore add needed
checks to ensure the pre-requisites are met
(From OE-Core rev: 0820aeba74869bf6e85b200f39bc610a43ab9a7a)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Calling sync between each file compare is horrible performance wise
as we compare thousands of files. We don't care about IO latency here
so disable.
(From OE-Core rev: 25f78abd8bbeb201fd9452e7983e015027954948)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't use tbe BUILDINFO line of host information in the Makefile
so remove it for reproducibility.
(From OE-Core rev: a9742595fa90d4977fdd8129a4fe4932ddb96a18)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't want to encide BUILD_ROOT into target packages. This is used
for build time tests but in our case those would be on target anyway
do use the target paths.
(From OE-Core rev: 2818dc90bd62487879e7da2c68357b39282415b0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
config_data was including a build system path to pkg-config, fix
this.
(From OE-Core rev: 7b8f56224c12a40f4f0aca3f612198deb933af4d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
configure inspects the host's /etc/group for these configuration
options, fix this to the correct values by using configure options.
(From OE-Core rev: f16f9c727569414cd52862dcba18d8e423f4e961)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* not sure if there are some valid use-cases for missing manifest, but
recently I'm seeing increasing number of build failures where something
from native recipe is missing (seen it with pseudo, autoconf, nodejs
recently) and the only indication that something is wrong (before showing
sometimes misleading error like:
recipe-sysroot-native/usr/bin/node: No such file or directory
is this warning:
NOTE: Running task 7844 of 12431 (/meta-oe/meta-oe/recipes-devtools/nodejs/nodejs_12.20.2.bb:do_prepare_recipe_sysroot)
NOTE: recipe nodejs-12.20.2-r0: task do_prepare_recipe_sysroot: Started
WARNING: nodejs-12.20.2-r0 do_prepare_recipe_sysroot: Manifest /BUILD/sstate-control/manifest-x86_64_ubuntu-18.04-nodejs-native.populate_sysroot not found in x86_64 x86_64_ubuntu-18.04 (variant '')?
NOTE: Running task 7845 of 12431 (/meta-oe/meta-oe/recipes-devtools/nodejs/nodejs_12.20.2.bb:do_unpack)
NOTE: recipe nodejs-12.20.2-r0: task do_unpack: Started
WARNING: nodejs-12.20.2-r0 do_prepare_recipe_sysroot: Manifest /BUILD/sstate-control/manifest-x86_64_ubuntu-18.04-nodejs-native.populate_sysroot not found in x86_64 x86_64_ubuntu-18.04 (variant '')?
NOTE: recipe nodejs-12.20.2-r0: task do_prepare_recipe_sysroot: Succeeded
if I rebuild that native dependency, then it gets fixed and I don't
see these failures in clean builds (as without sstate and with empty
TMPDIR), only in incremental builds
* but if there isn't valid reason for missing manifest file, then I think
it would be better to error early (or even bb.fatal())
(From OE-Core rev: 95607a26854d873399d2b9d7e5fcbffc0cbdba4c)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
rpm packaging has shown good numbers wrt reproducible builds as well
when testing core-image-minimal locally:
Reproducibility summary for rpm: same=4671 different=0 missing=0 total=4671
Thus enable it in the autobuilder to be more widely tested.
(From OE-Core rev: 84d3a90557444d8cd83d780b1c3f2b278ab07af2)
(From OE-Core rev: 85b9dd01f2e1d5429f8da0c48a25ec034acebf67)
Signed-off-by: Jan-Simon Möller <jsmoeller@linuxfoundation.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
rpm uses "XXX-VERSION" compared to deb/ipk which use "XXX_VERSION"
which breaks the go package exclusion. Work around with both go_
and go- for now.
(From OE-Core rev: 2af535282833dddb58f24ec5b360230c47e6f6b1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
dep tool is now deprecated and its replaced with go modules so using
this as sample for testing is also getting arcane. Replace it with
another project direnv[1] which is quite active and uptodate
[1] https://github.com/direnv/direnv
(From OE-Core rev: bb510037beaaaa723b9158d1f4e11a3547f6fb73)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
git, libproxy, libcap-ng(-python), babeltrace2, rsync and groff
have their reproducibility issues fixed, remove from the exclusion
list. Also fix whitespace.
(From OE-Core rev: 044eda9c70ddda195b1c0ce1d228904392431582)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ensure the fallback value if used is written to the SDE file
and hence stored in sstate, reducing any confusion within the
code over '0' values.
Bump the HASHEQUIV_VERSION since we've had a ton of trouble
with ensuring this rolls out correctly on the autobuilder so
others may too, take a clean slate for it.
(From OE-Core rev: 53ffffa32b76330835287dfc05d4e4796841af08)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Das U-Boot 2021.4-rc1 has the following commit:
commit 3f04db891a353f4b127ed57279279f851c6b4917
Author: Simon Glass <sjg@chromium.org>
Date: Mon Feb 15 17:08:12 2021 -0700
image: Check for unit addresses in FITs
Using unit addresses in a FIT is a security risk. Add a check for
this and disallow it.
CVE-2021-27138
Adjust the kernel-fitimage.bbclass accordingly to not use unit
addresses. This changte is required before we can bump U-Boot to 2021.4.
(From OE-Core rev: 6047be9f8f0f5d616fda11d83b682c1b8aeaa0ae)
Signed-off-by: Klaus Heinrich Kiwi <klaus@linux.vnet.ibm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the separate of the "-only" and "-or-later" licenses, we need to
update the tests to match the messages now given in the output.
Also use a mix of canonicalised and non-canonlised names in the
reference recipes to help test those cases and ensure coverage.
(From OE-Core rev: b7f38af7ac9449178c603c5349808b8c0dd84d35)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
GPLv2 and GPLv2+ are two difference licenses with different meanings
and we can't just pretend they're the same thing. Change the code
to treat them separately.
(From OE-Core rev: d1baf74ac92fe0c8c32dff101fd77d77f70fd583)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
`devtool` uses `copy_recipe_files` for the upgrade operation when
creating the new, workspace recipe. Before handling the copy operations,
the function checks the entry in `SRC_URI` against `FILE` while in turn
uses absolute paths. When BBLAYERS contains entries that are not
normalised, this check will fail resulting in having the recipe in the
workspace without the initial patches.
Signef-off-by: Robert Drab <robert.drab@huawei.com>
(From OE-Core rev: 777fc24ab2c332954b56dac28cd9b3032808828c)
Signed-off-by: Andrei Gherzan <andrei.gherzan@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the target is using a read-only rootfs, the available space on '/'
will be zero. This will cause the test to incorrectly fail and skipping
seems appropriate in this case.
(From OE-Core rev: af1dbea3c9b9b42a3e6803b231e425423d70e210)
Signed-off-by: Wes Lindauer <wesley.lindauer@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
