configure inspects the host's /etc/group for these configuration
options, fix this to the correct values by using configure options.
(From OE-Core rev: f16f9c727569414cd52862dcba18d8e423f4e961)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* not sure if there are some valid use-cases for missing manifest, but
recently I'm seeing increasing number of build failures where something
from native recipe is missing (seen it with pseudo, autoconf, nodejs
recently) and the only indication that something is wrong (before showing
sometimes misleading error like:
recipe-sysroot-native/usr/bin/node: No such file or directory
is this warning:
NOTE: Running task 7844 of 12431 (/meta-oe/meta-oe/recipes-devtools/nodejs/nodejs_12.20.2.bb:do_prepare_recipe_sysroot)
NOTE: recipe nodejs-12.20.2-r0: task do_prepare_recipe_sysroot: Started
WARNING: nodejs-12.20.2-r0 do_prepare_recipe_sysroot: Manifest /BUILD/sstate-control/manifest-x86_64_ubuntu-18.04-nodejs-native.populate_sysroot not found in x86_64 x86_64_ubuntu-18.04 (variant '')?
NOTE: Running task 7845 of 12431 (/meta-oe/meta-oe/recipes-devtools/nodejs/nodejs_12.20.2.bb:do_unpack)
NOTE: recipe nodejs-12.20.2-r0: task do_unpack: Started
WARNING: nodejs-12.20.2-r0 do_prepare_recipe_sysroot: Manifest /BUILD/sstate-control/manifest-x86_64_ubuntu-18.04-nodejs-native.populate_sysroot not found in x86_64 x86_64_ubuntu-18.04 (variant '')?
NOTE: recipe nodejs-12.20.2-r0: task do_prepare_recipe_sysroot: Succeeded
if I rebuild that native dependency, then it gets fixed and I don't
see these failures in clean builds (as without sstate and with empty
TMPDIR), only in incremental builds
* but if there isn't valid reason for missing manifest file, then I think
it would be better to error early (or even bb.fatal())
(From OE-Core rev: 95607a26854d873399d2b9d7e5fcbffc0cbdba4c)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
rpm packaging has shown good numbers wrt reproducible builds as well
when testing core-image-minimal locally:
Reproducibility summary for rpm: same=4671 different=0 missing=0 total=4671
Thus enable it in the autobuilder to be more widely tested.
(From OE-Core rev: 84d3a90557444d8cd83d780b1c3f2b278ab07af2)
(From OE-Core rev: 85b9dd01f2e1d5429f8da0c48a25ec034acebf67)
Signed-off-by: Jan-Simon Möller <jsmoeller@linuxfoundation.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
rpm uses "XXX-VERSION" compared to deb/ipk which use "XXX_VERSION"
which breaks the go package exclusion. Work around with both go_
and go- for now.
(From OE-Core rev: 2af535282833dddb58f24ec5b360230c47e6f6b1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
dep tool is now deprecated and its replaced with go modules so using
this as sample for testing is also getting arcane. Replace it with
another project direnv[1] which is quite active and uptodate
[1] https://github.com/direnv/direnv
(From OE-Core rev: bb510037beaaaa723b9158d1f4e11a3547f6fb73)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
git, libproxy, libcap-ng(-python), babeltrace2, rsync and groff
have their reproducibility issues fixed, remove from the exclusion
list. Also fix whitespace.
(From OE-Core rev: 044eda9c70ddda195b1c0ce1d228904392431582)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ensure the fallback value if used is written to the SDE file
and hence stored in sstate, reducing any confusion within the
code over '0' values.
Bump the HASHEQUIV_VERSION since we've had a ton of trouble
with ensuring this rolls out correctly on the autobuilder so
others may too, take a clean slate for it.
(From OE-Core rev: 53ffffa32b76330835287dfc05d4e4796841af08)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Das U-Boot 2021.4-rc1 has the following commit:
commit 3f04db891a353f4b127ed57279279f851c6b4917
Author: Simon Glass <sjg@chromium.org>
Date: Mon Feb 15 17:08:12 2021 -0700
image: Check for unit addresses in FITs
Using unit addresses in a FIT is a security risk. Add a check for
this and disallow it.
CVE-2021-27138
Adjust the kernel-fitimage.bbclass accordingly to not use unit
addresses. This changte is required before we can bump U-Boot to 2021.4.
(From OE-Core rev: 6047be9f8f0f5d616fda11d83b682c1b8aeaa0ae)
Signed-off-by: Klaus Heinrich Kiwi <klaus@linux.vnet.ibm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the separate of the "-only" and "-or-later" licenses, we need to
update the tests to match the messages now given in the output.
Also use a mix of canonicalised and non-canonlised names in the
reference recipes to help test those cases and ensure coverage.
(From OE-Core rev: b7f38af7ac9449178c603c5349808b8c0dd84d35)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
GPLv2 and GPLv2+ are two difference licenses with different meanings
and we can't just pretend they're the same thing. Change the code
to treat them separately.
(From OE-Core rev: d1baf74ac92fe0c8c32dff101fd77d77f70fd583)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
`devtool` uses `copy_recipe_files` for the upgrade operation when
creating the new, workspace recipe. Before handling the copy operations,
the function checks the entry in `SRC_URI` against `FILE` while in turn
uses absolute paths. When BBLAYERS contains entries that are not
normalised, this check will fail resulting in having the recipe in the
workspace without the initial patches.
Signef-off-by: Robert Drab <robert.drab@huawei.com>
(From OE-Core rev: 777fc24ab2c332954b56dac28cd9b3032808828c)
Signed-off-by: Andrei Gherzan <andrei.gherzan@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the target is using a read-only rootfs, the available space on '/'
will be zero. This will cause the test to incorrectly fail and skipping
seems appropriate in this case.
(From OE-Core rev: af1dbea3c9b9b42a3e6803b231e425423d70e210)
Signed-off-by: Wes Lindauer <wesley.lindauer@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This test mimic the boot_from_runlevel_3 and boot_from_runlevel_5 test cases from oeqa/manual/bsp-hw.json.
The boot_from_runlevel_3 and boot_from_runlevel_5 manual test cases should be remove from oeqa/manual/bsp-hw.json if this patch get merged.
(From OE-Core rev: 6effd30f0e3726bc1f2eb7768c57c6b95eddb079)
Signed-off-by: Teoh Jay Shen <jay.shen.teoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This reverts commit 84d3a90557444d8cd83d780b1c3f2b278ab07af2.
This shouldn't have been merged, its not ready yet.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
rpm packaging has shown good numbers wrt reproducible builds as well
when testing core-image-minimal locally:
Reproducibility summary for rpm: same=4671 different=0 missing=0 total=4671
Thus enable it in the autobuilder to be more widely tested.
(From OE-Core rev: 84d3a90557444d8cd83d780b1c3f2b278ab07af2)
Signed-off-by: Jan-Simon Möller <jsmoeller@linuxfoundation.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes reproducibility when build acpica yacc sources by instructing
bison on how to do the file prefix mapping
(From OE-Core rev: d13a1df7bd173fca3fc2cdf889d5eb487e35d6fd)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add ruby-ri-docs and meson back as not reproducible, remove watchdog,
xorg-minimal-fonts and xmlto as issues fixed.
(From OE-Core rev: 5e209a7455a231342cded99985b4a6caff424e4f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The reproducible build tests can take a long time, so having more
logging messages at various points in the build can help debug where the
build is taking a long time.
(From OE-Core rev: 6b792afe8759d62af8e713b86dad8f6721961a05)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The SSTATE_MIRRORS variable was misspelled, which allowed the "clean"
test build to pull from the mirror.
(From OE-Core rev: e42497bd84d0bb370a9f7b0448bff29f01fd1b0c)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When building SDKs with the deb backend you could see errors like:
Setting up nativesdk-python3-ndg-httpsclient (0.5.1-r0) ...
mkdir: cannot create directory ‘/usr/lib/opkg’: Permission denied
dpkg: error processing package nativesdk-python3-ndg-httpsclient (--configure):
which is due to environment misconfiguration when removing packages.
Fix this by setting the same environment variables as used for installation.
(From OE-Core rev: f23c7e319a192ada14bb9a82822ef2967309aaea)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It appears these exclusions are no longer needed with master, drop
them and improve our reproducibilty metrics.
(From OE-Core rev: c71c984a9cd6d130ece08153d7d92bb33c7ec444)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This makes comparisions between lists easier.
(From OE-Core rev: d2c52125d1cdc06c7e08d507ca68f3e4612a4314)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This helps with trimming down the list, and towards 100% reproducibility :)
(From OE-Core rev: da7a173d7a01524229c8515326465968a845e96f)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bitbake changed the debug() logging call to make it compatible with
standard python logging by no longer including a debug level as the
first argument. Fix up the few places this was being used.
Tweaked version of a patch from Joshua Watt.
(From OE-Core rev: 5aecb6df67b876aa12eec54998f209d084579599)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
files in wic tmp directory can be usefull for debugging, so do not remove
tmp directory when wic create run with debugging mode (-D or --debug).
also update wic.Wic.test_debug_short and wic.Wic.test_debug_long to
check for tmp directory.
[YOCTO#14216]
(From OE-Core rev: a122e2418b67d38f691edcf8dd846c167d6b4fa9)
Signed-off-by: Lee Chee Yang <Chee.Yang.Lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
existing test case test_permissions use Wic command as standalone
tools to create wic image and check that wic image for permissions.
add extra steps to the test case to also check against image build
using bitbake do_image_wic.
(From OE-Core rev: 551ce73a90757ba43501fe5cf9ac84a7b77de549)
Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This test mimic the Test_if_usb_hid_device_works_well_after_resume_from_suspend_state manual test case from oeqa/manual/bsp-hw.json.
(From OE-Core rev: 23a3dc370a52907ee3261746405fb9b2af9e9a11)
Signed-off-by: Teoh Jay Shen <jay.shen.teoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If path is not owned by any user installed on target it gives
insufficient error "getpwuid(): uid not found" which may be misleading.
This exception occurs if uid/gid of path was not found in PSEUDO_PASSWD
files, which simply means the path is owned by host user and there is
host user contamination.
Add more information to the exception message to make it easier for user
to debug.
[YOCTO #14031]
(From OE-Core rev: 38540b59ed4ec8632e30a5fd6364b010d9da8470)
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The "devtool modify" could remove message body before scissors line, so
patches re-generated from git tree were incorrectly modified.
Adding --no-scissors to "git am" invocation to prevent this behaviour.
[YOCTO #12674]
(From OE-Core rev: 13ea33fbd197b9ee3cf913d9995617115f22798f)
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a function returns any stderr it will be passed to extractPatches and
used as path to patch.
For example subprocess command output can be:
| sh: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
| /tmp/oepatchhuqle8fj/0001-foo.patch
| /tmp/oepatchhuqle8fj/0002-bar.patch
that will result in:
| FileNotFoundError: [Errno 2] No such file or directory: 'sh:'
To fix this I separated output, made the function return stdout and
print stderr only in case of command error.
(From OE-Core rev: 482589e2cc7c3ddeefb0a0fb98d97a9cbb18c9ec)
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
add CVE_VERSION_SUFFIX to indicate the version suffix type, currently
works in two value, "alphabetical" if the version string uses single
alphabetical character suffix as incremental release, blank to not
consider the unidentified suffixes. This can be expand when more suffix
pattern identified.
refactor cve_check.Version class to use functools and add parameter to
handle suffix condition.
Also update testcases to cover new changes.
(From OE-Core rev: 5dfd5ad5144708b474ef31eaa89a846c57be8ac0)
Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This test mimic the ethernet_static_ip_set_in_connman and ethernet_get_IP_in_connman_via_DHCP test case from oeqa/manual/bsp-hw.json.
The ethernet_static_ip_set_in_connman and ethernet_get_IP_in_connman_via_DHCP manual test case should be remove from oeqa/manual/bsp-hw.json if this patch get merged.
(From OE-Core rev: aaabc94dbe353b12297ba4a237f6817b2c6d4a31)
Signed-off-by: Teoh Jay Shen <jay.shen.teoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The cmd redirected stderr to stdout that was then assigned to variable
with pkgs to install. Then this variable was passed to package manager
that then tried to install it and generated confusing warnings.
For example this variable could contain:
| ['(en_US.UTF-8)', 'LC_ALL:', 'bash:', 'cannot', 'change', 'locale', 'setlocale:', 'warning:']
and the warning was:
| WARNING: addon-bci-1.0-r0 do_populate_sdk: Unable to install packages.
| Command 'PATH/usr/bin/opkg ... install (en_US.UTF-8) LC_ALL: bash:
| cannot change locale setlocale: warning:' returned 255:
| Collected errors:
| * opkg_prepare_url_for_install: Couldn't find anything to satisfy '(en_US.UTF-8)'.
In this change I remove stderr redirection to stdout and pass it to
bb.note instead.
(From OE-Core rev: 70d8ced3d7f53f988c02ff03d8dfa448f088fdf1)
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If recipe uses only file:// fetcher devtool unpacks sources under
oe-local-files/ and adds symlink in source root directory. We need to
verify if the symlink in subdirectory has correct path.
See [YOCTO #13738] for details.
(From OE-Core rev: 044c90d152bfd1b2a23272df39327ba4cd862dff)
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When building package indexes for RPM feeds, delete repodata first to
be absolutely sure that the indexes are up to date.
We've seen some mysterious failures where the repodata doesn't match
the repository, so hopefully this will stop that happening.
[ YOCTO #14190 ]
(From OE-Core rev: 88cd40c08109bf732934fa4c8d602c701c1eb2a0)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The way distutils.version.LooseVersion compare version are tricky, it treat
all these ( "1.0-beta2", "1.0-rc1", "1.0A", "1.0p2" and "1.0pre1") as greater
version than "1.0". This might be right for "1.0A" and "1.0p1" but not for
the rest, also these version could be confusing, the "p" in "1.0p1" can be
"pre" or "patched" version or even other meaning.
Replace Looseversion with custom class, it uses regex to capture common
version format like "1.1.1" or tag format using date like "2020-12-12" as
release section, check for following known string/tags ( beta, rc, pre, dev,
alpha, preview) as pre-release section, any other trailing characters
are difficult to understand/define so ignore them. Compare release
section and pre-release section saperately.
included selftest for the version class.
[YOCTO#14127]
(From OE-Core rev: 6ced85e9ddd3569240f1e8b82130d1ac0fffbc40)
Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The test would timeout on autobuilders. This patch increases the
timeout to 60s
The test will now also exit as soon as we receive the 2 expected events
Expected runtime is around 1s if successful
Bug 14158
(From OE-Core rev: be02aa9283f805de718badd5ea12c4968da8774f)
Signed-off-by: Yi Fan Yu <yifan.yu@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The binutils-cross-x86_64 is not avaliable for non x86-64 machines.
(From OE-Core rev: 02385b180bc4a1af7120e85a9998566bfcb85977)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bash 5.1 inserts escape sequences into its output (specifically
disabling bracketed paste mode via \x1b[?2004l). I am not sure
if somehow terminal detection isn't working correctly there,
but in any case the marker is still in the output, but needs to
be checked by 'in' rather than exact equivalence.
(From OE-Core rev: c25a1b16cc50e51ecf040c2c1db128e1dfceddec)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit provides a testcase for the initramfs bundle support implemented
in kernel-fitimage.bbclass
The testcase verifies the content of the initramfs bundle node in the FIT
Image Tree Source (its).
The testcase is self-contained and the configurations are set by the test case itself.
To verify the initramfs bundle support, the testcase uses beaglebone-yocto machine.
This testcase can be run through the following command:
oe-selftest -r fitimage.FitImageTests.test_initramfs_bundle
Change-Id: I8ab8abf2c150ea515fd439784eb20c6b092bfbc5
(From OE-Core rev: 1119d577756b386507f33669fe29dafb5579a1a7)
Signed-off-by: Abdellatif El Khlifi <abdellatif.elkhlifi@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since host's ccache is not reliable, so disable ccache for native recipes and
use ccache-native for other types of recipes. We need disable ccache for native
recipes is because ccache-native now depends on cmake-native which causes
circular dependencies, and it's not easy to break the circular.
(From OE-Core rev: 631bbd4896882ba2acbe5bc85bc90ab7abc794ef)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This reverts commit f5b29367af4d8e5daea5771264774aa49519f9a8.
Will use ccache-native which is more reliable.
(From OE-Core rev: 1b659623430e1a6e6dd266e65bab7ca8155a7138)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows spotting ptest regressions without having hard ptest failures
(for that full ptest stability should be achieved).
(From OE-Core rev: 79b54d902df2788b0822b6c9cc14705ab00e6a5b)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
- The dir /etc/apt was created in package apt, if package apt was not
installed, there is no need to insert package feed. Otherwise, it
will fail with no such dir
- Output the result of apt install
- Explicitly trust the deb package repository from build
This could avoid apt install warning:
...
WARNING: The following packages cannot be authenticated!
...
- Also trust the inserted deb package repository from PACKAGE_FEED_URIS
(From OE-Core rev: 9ec65b77c9a4a0ba240117edee0e84208c58328e)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Example use case in OpenBMC: rootfs is squashfs and the system has either
overlayfs for whole rootfs or for some parts (e.g. /etc).
This option will allow to create migration one-shot postinsts using
"pkg_postinst_ontarget_${PN}" routines defined in recipes to fix
files under upper workdir in overlayfs.
(From OE-Core rev: 0977204e16279b117811b5d5cdac5918287e95ac)
Signed-off-by: Anton D. Kachalov <rnouse@google.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In cmake.bbclass we set CMAKE_BUILD_PARALLEL_LEVEL using parallel_make
function and if PARALLEL_MAKE is set to empty string then this variable
is exported as "None" causing cmake to fail with:
"'CMAKE_BUILD_PARALLEL_LEVEL' environment variable
invalid number 'None' given."
(From OE-Core rev: 2f790ded554a52ac18d1c28002142f9c62abec8b)
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Switch do_populate_sdk for the ipk package manager to use a separate target
opkg config file and separate the lockfiles restricting do_rootfs and
do_populate_sdk from running in parallel.
This way if an image recipe includes a dependency to do_populate_sdk by
default then it will run in parallel to do_rootfs saving time compared to the
sequential execution.
(From OE-Core rev: 8c210407d07483075a70c8b97ad52b5eae062c9c)
Signed-off-by: Michael Ho <Michael.Ho@bmw.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The oe-selftest code already keeps the selftest build directory in place
if any tests failed. By default the build directory is deleted if all
tests pass but there may be cases where it's desirable to keep this
directory around, for example to compare intermediate files between
passing and failing test runs.
(From OE-Core rev: 67aa7069dbe8f5f5f186eb67708ece5c4bd42976)
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
