If getpwduid fails, we don't see which file it failed on which is key information
to aid debugging. Print this information when exceptions are raised.
(From OE-Core rev: 2bba0052597020ea887c84419440df11f9859283)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 9d7a5219713af8117eda145052c6d9abdf577d8f)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We can have one taskhash which represents multiple native/cross sstate objects
since they're stored by BUILD_ARCH or possibly host distro (or host gcc version).
We need to put these into separate namespaces on hashserv since their outhashes
will never match and we need deterministic lookups for the different namespaces.
Use this extramethod option to handle this. This fixes several problematic
failures on the autobuilder.
(From OE-Core rev: 766b57a1f9030429b546276a537fbce3a175cc25)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 2a76082363d189880613765ad339718e3614049d)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ensure that RECIPE_SYSROOT is the same for -native recipes whether
multilib.conf is included or not.
Without this change task signatures for -native recipes change when
switching between MACHINEs that require multilib.conf and those that
don't.
This fix was one of the ones suggested by Khem Raj in
http://lists.openembedded.org/pipermail/openembedded-core/2019-December/290303.html
Add test_sstate_multilib_or_not_native_samesigs test case to
sstatetests.py to ensure that this stays fixed.
(From OE-Core rev: c3e957409bf68e7b310e27781ac91d24cf4452bf)
Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit aa05f1ded71366b86eda7fce24d8b5395e85ada2)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously, there was SRC_DISTRIBUTE_LICENSES, an undocumented
variable that contained a static list of licenses. It was used by
expand_wildcard_licenses() to expand any wildcards used in, e.g.,
INCOMPATIBLE_LICENSE. However, since this static list of licenses has
not been kept up-to-date, many licenses were missing, with the result
that if one tried to use any of those licenses with a wildcard, no
licenses would be found, effectively ignoring that they should be
marked as incompatible.
This introduces a new (documented) variable, AVAILABLE_LICENSES, that
is automatically updated to contain all licenses found in any
directories specified by ${COMMON_LICENSE_DIR} and ${LICENSE_PATH},
and uses it instead of SRC_DISTRIBUTE_LICENSES when expanding
wildcards.
(From OE-Core rev: 5ed714139f91eb03871e01b68a4370784071234d)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 8c9ef587fe499c612a878a1ab42092eb79b334ef)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since this file is written during recipe parsing, having it in the
${BUILDDIR}/conf directory, which is covered by an inotify watcher,
will trigger a re-parse the next time bitbake is run and the resident
bitbake server is enabled. This causes the sanity_info file to be
updated again, which triggers a new parse the next time bitbake is run
ad infinitum. Moving it to ${BUILDDIR}/cache should avoid this.
(From OE-Core rev: f1a609e7176a37e7e5f86f0b786f7da915085f83)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit f98103b548aa7dba6b1be6c8e02ef41858a8e85c)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds an option that can be used to copy the offending packages to a temp
directory for later evaluation. This is useful on the Autobuilder to
investigate failures.
(From OE-Core rev: 91d657a0c4cbb273e1e74d38bfd6b4b05d9b372e)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If this test fails then the output doesn't help in any meaningful way, so
improve the test to output the unparsable JSON and display unexpected output.
(From OE-Core rev: 6a710ad0b445295991b17545f634684f4f317099)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Check package busybox which is required by cases test_dnf_installroot
and test_dnf_installroot_usrmerge.
(From OE-Core rev: 0f09c5b7a9a66af825951354b436b69dd9a596a4)
Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Verify that the build inside the SDK is detected as a cross compilation.
(From OE-Core rev: 3dc75ab44c23c4ff26502b96abded3c1c0b94e38)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[Bugzilla Bug 13201] -- https://bugzilla.yoctoproject.org/show_bug.cgi?id=13201
Newer versions of gnome-terminal (3.32.0 and up) are not starting
as expected for commands e.g. "bitbake -c devshell zlib". This
manifests as the instance appearing as a new tab rather than a
new window. Fix this (and maintain new window preferred behavior)
by changing the "-x" option to "--" as per the warning message,
avoiding deprecated options:
# Option “--command” is deprecated and might be removed in a later version of gnome-terminal.
# Use “-- ” to terminate the options and put the command line to execute after it.
(From OE-Core rev: d5fd205239c8e3a1d68649562a7e91c8fbbc805e)
Signed-off-by: Trevor Gamblin <trevor.gamblin@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In commit 8689e561 (lib/oe/lsb: attempt to ensure consistent distro id
regardless of source), the distro ID returned by
oe.lsb.distro_identifier() was lowercased, but only if a release
version is also present.
This changes the code to always lowercase the distro ID, including the
default distro ID "unknown", which is used if no other ID can be
identified.
(From OE-Core rev: 4ba7ef79d23a4cf688d7a794064893fe5f2f473b)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Do not decode the log content into a string only to re-encode it as
binary data again. Some logs might un-intentionally contain bytes that
do not decode as utf-8, as such preserve the log file content as it was
on disk.
Handle the decoding on the resulttool side, but also handle the failure
to decode the data.
(From OE-Core rev: 20531dc0b8f76a6e37cc856f36cd94077b6aba50)
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Handle the streaming of exception content with details data. The
testtools package allows both 'err' and 'details' kwargs but can only
pass one of them to the parent.
To handle the passing of exception traceback and details data at the
same time, encode the traceback into the details object and remove the
'err' arg from the add* result call. This encodes the traceback similar
to how 'err' is handled without any details object. Decoding is already
done by testtools when the traceback is encoded in the details object.
(From OE-Core rev: 3613451825b251784b7673d89db465b9782c3a31)
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The locked sigs class needs to be inherited after the hashequiv mixin so
that get_unihash can correctly wrap the underlying hashequiv function.
To do this turn the locked sigs class into a second mixin, then the order
can be correctly handled. Tweak the get/set_taskdata to match.
(From OE-Core rev: 73d3414ecf594e426e6024b71931038a201eca52)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There were two issues with the test one is that an equivalent hash
could come from the server meaning the signature didn't change when it
should. A uuid string is injected to ensure this does not happen.
If there were multiple warnings the test would also fail as only the
first is prefixed with WARNING. Tweak the string to avoid that failure
mode.
(From OE-Core rev: e58eaf5b8c93521dc311b77593e0dd7debca602d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Using locked signatures with the hash equivalency server ran into
problems. We need to:
a) Ensure the lockedhashes data object is passed from the core to
any individual tasks using the get/set_taskdata methods
b) Return a locked singature instead of a unihash
c) Write the unihash being used to locked signature lists rather than
the calculated taskhash
d) Skip warnings of hash mismatch if the hash is a unihash
These changes fix esdk builds (which use locked sigs) when a hash equivalence
server is in use.
(From OE-Core rev: 25dc3d78de01dffa77a3a2452d6a97d741b446d9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a test to verify that when multiconfig conf files changed, recipes
are correctly reparsed.
[YOCTO #13541]
(From OE-Core rev: a424ef0a0c49123f4518e8fef993fd10f6fc5f4e)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
install_locales() here is actually operating on nativesdk and only glibc
is the default library for nativesdk, since thats what most of
desktop/server distros use, therefore bailing out based on TCLIBC is not
needed here, since nativesdk-glibc would be required for all non-glibc
targetting SDKs as well.
Fixes SDK install time error
ERROR: OE-core's config sanity checker detected a potential misconfiguration.
Either fix the cause of this error or at your own risk disable the checker (see sanity.conf).
Following is the list of potential problems / advisories:
Your system needs to support the en_US.UTF-8 locale.
ERROR: SDK preparation failed
(From OE-Core rev: ba2df1fa43b79c4959911c1b0bcad996f255b7a8)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It checked whether busybox is in task-depends.dot after run "bitbake -g
core-image-minimal", but busybox is not a must for core-image-minimal since it
is configurable:
VIRTUAL-RUNTIME_base-utils = ""
VIRTUAL-RUNTIME_base-utils-syslog = ""
VIRTUAL-RUNTIME_base-utils-hwclock = ""
VIRTUAL-RUNTIME_login_manager = "shadow"
VIRTUAL-RUNTIME_syslog = ""
So the case may fail when busybox is not present, check
base-files which is more reliable.
(From OE-Core rev: f73676b07f6d8a4573a5f46c3e082831b34b7d18)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
when build with EXTERNALSRC, checksum of EXTERNALSRC is one of inputs
for generate sigdata of task do_compile. without change any source, the
sigdata will not changed, and buildclean task only do make clean, and
not clean stampfiles like clean task, so the stampfile keep there, then
do_compile after do_buildclean will not rerun, and targetfile
tempdir_mdadm/mdadm will not be generated, report below error:
AssertionError: '/tmp/devtoolqarxv8ct6u/mdadm' does not exist
Fix by add a comments in Makefile to trigger rebuild of compile task
(From OE-Core rev: f4621cf4516f9de9980f3864a193f3f61c4abedc)
Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
if do clean after remove-layer, it will make stamp file not cleaned
since variable STAMP has changed, this will cause error like:
if testcase buildclean failed, when run next testcase modify, since
do_prepare_recipe_sysroot's stamp file exists, sysroot not populated,
then do_compile will fail with error:
/bin/sh: x86_64-poky-linux-gcc: command not found
(From OE-Core rev: 046d7e1de094105db0949c2cf5333ff55d9ad9c8)
Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that opkg-build can build reproducible ipkgs, we can also add those to the
test case.
[ YOCTO #13513 ]
(From OE-Core rev: 3806d38e6194a42534381993fcfcc3ff39550a13)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Autobuilder type infrastructure can benefit from deletion of certain files as
background IO due to the way Linux filesystem priority works.
We have problems where build directories as part of oe-selftest being
delete starves the running tasks of IO to the point builds take much
longer to compelte.
Having this option of running the deletion at "idle" helps a lot with
that. Use the new option added to bb.utils.prunedir().
(From OE-Core rev: d41e7018be56902b7a1be4590e468cd15e02a3b5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When doing a devtool build-sdk from within an esdk all nativesdk
components would be rebuilt. This patch introduces SDK_INCLUDE_NATIVESDK
flag to toggle the inclusion of nativesdk packages when creating the
esdk sstate
Currently locked-sigs.inc is generated during do_sdk_depends which
doesn't pull in nativesdk packages. Generating another locked-sigs.inc
in do_populate_sdk_ext and pruning it to only nativesdk* packages by
using a modified version of the already existing function
prune_locked_sigs and merging it with the current locked-sigs.inc
Also adding SDK_INCLUDE_NATIVESDK tasklistfn to the logic surrounding
setting tasklist file to not prune esdk sstate during creation
[YOCTO #13261]
(From OE-Core rev: d046afd12e1c209b29dca6ba402b9aa14680c5ce)
Signed-off-by: Jaewon Lee <jaewon.lee@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removes all references to the SSTATE_HASHEQUIV_SERVER variable. This
variable is redundant now that BB_HASHSERVE is present.
(From OE-Core rev: 54b3adb6bc90a8e4b9e92952688772ee074d36e5)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The server no longer uses a "http://" URI, since it has been updated to
use a different protocol.
(From OE-Core rev: 519561172c48bc7f7a61a3d02edd418fc0895b7b)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix error like:
lib32-e2fsprogs-1.45.3-r0 do_package_qa: QA Issue: /usr/sbin/e2scrub_all
contained in package lib32-e2fsprogs-e2scrub requires /bin/bash,
but no providers found in RDEPENDS_lib32-e2fsprogs-e2scrub
For some lib32 packages(eg: lib32-bash, lib32-sed) which probvides files,
extend is not needed
Eg: RPROVIDES of lib32-bash expects to have /bin/bash, with original
extend, it will become lib32-/bin/bash, then will cause above error
Fix by don't extend file dependency, and skip multilib check for file dependency
in do_package_qa to avoid error like:
WARNING: lib32-bash-5.0-r0 do_package: QA Issue: lib32-bash package lib32-bash
- suspicious values '/bin/bash /bin/sh' in RPROVIDES [multilib]
(From OE-Core rev: a9163120ed52534e7dbf4db50dc2b03bbf69f06b)
Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This will allow not having to multiply these options for the sdl
frontend, instead combining them as needed.
(From OE-Core rev: 922eb5012364b1603338cfa617712b941e892bbf)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The escaping, splitting and matching of NO_PROXY in oe-git-proxy
deserves its own testcase, add it.
(From OE-Core rev: c07134711f97c966d70aaf2798800214d5426005)
Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use OEPTestResultTestCase to collect results and add logfile collection
and compression.
(From OE-Core rev: b75aced558013a459f29617b53cf477cb9b387ac)
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add the OEPTestResultTestCase class as a mix-in class to provide helper
functions for interacting with ptestresults within the extraresults
object generated by the test case.
This class also provides default compression of log text and log files.
Also add support to resulttool for decoding/decompressing log files
embedded in the test results.
(From OE-Core rev: 06cba9883a5964320969301fd05eeb6bec3e786d)
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use the 'append' action of argparse instead of nargs. This changes the
behaviour of the option from "-t foo bar -r" to "-t foo -t bar -r".
Additionally rename the long form options to be consistent with
behaviour, such that they specifying a single tag at a time.
(From OE-Core rev: ffe9e4303fa9799d2e8af9188853a262e15af226)
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This matches the other toolchain tests.
(From OE-Core rev: d41606244c170fd547496e5df9e3d28ce2d2af68)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rework the glibc execution into a common base class. Additionally tag
the tests with "toolchain-user" and "toolchain-system".
(From OE-Core rev: 94bf24268108774e022ad247c647e48a781debbb)
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Split the gcc selftest cases into multiple classes one for each test.
This is done in order to make it easy to execute multiple gcc tests in
parallel when using oe-selftest with the '-j' arg.
Additionally tag the user tests with "toolchain-user" and the system
emulation (qemu system) tests with "toolchain-system".
(From OE-Core rev: 7b2f03eff9fc9b4ce48d5ea7e54faa114a6cdcae)
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Change the -t/-T args to be optional so that they can be used together
with the existing -r/-a/... args to run a more flexible filtering of
test tags.
(From OE-Core rev: 55ee27bb07113a45da18711b5509764f62be4d75)
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to take advantage of multiprocess execution of tests the
extraresults must be passed through the TestResult. With changes to how
oeqa/core handles test cases the extraresults attribute of the testcase
is passed to the TestResult, with passing across process boundaries
handled automatically.
(From OE-Core rev: 6a1b0c2003a0b4a1983f9494440e6ea02dc25585)
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Suffix the ptestresults suite with "-user" for tests that are executing
against usermode qemu.
(From OE-Core rev: 0becf9c1fabb080a2481ebdacef6221f52301621)
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Do not fail the running selftest test case if the test suite has a
failed test case. Currently toolchain tests suites (binutils, gcc,
glibc) fail but this does not indicate failure to execute the tests.
Also remove the logging of each test that failed.
(From OE-Core rev: 073575ff9c06b2791cc2bd88063d815d2220f038)
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When running the system emulation test case, do not include python3 or
pexpect in the image. The test cases that use these also need gdb (with
python configured).
(From OE-Core rev: 7e5be0803ea0cbfd8e5b052e43b54e16ab3230ed)
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Create a oeqa selftest test case to execute the glibc test suite and
report the results. The results are populated into the extraresults
variable of the test case which are written to testresults.json for
resulttool to analyse.
An additional subclass is created to separate the execution with qemu
linux-user and qemu system. The GlibcSelfTestSystemEmulated test case
handles setup of the target image, setup of and NFS server as well as
execution with runqemu.
(From OE-Core rev: 730832ebcca305477e1c13248cd35eea095b35c6)
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Create a oeqa selftest test case to execute the gcc test suites and
report the results. The results are populated into the extraresults
variable of the test case which are written to testresults.json for
resulttool to analyse.
An additional subclass is created to separate the execution with qemu
linux-user and qemu system. The GccSelfTestSystemEmulated test case
handles setup of the target image as well as execution with runqemu.
(From OE-Core rev: 2c86a25f8992243311e7fa1a8654b41f12b749de)
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Create a oeqa selftest test case to execute the binutils test suites and
report the results. The results are populated into the extraresults
variable of the test case which are written to testresults.json for
resulttool to analyse.
(From OE-Core rev: e5629aa4bd939072208f6eb5b30a98e17eb6a8ae)
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These test cases are run by the autobuilder on a machine specific basis.
Add tags to these classes so they can be controlled by the metadata rather
than hardcoded in the autobuilder config.
(From OE-Core rev: de0b761b550d591f301ee5e9c232e0d5bd1342f2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
