Heartbeat events default to once a second and we need to ensure we have
enough time in the task to see them.
Add a nostamp delay task 5s long so we can have a consistently timed
task which doesn't need cleanup or have unneeded dependencies. This
ensures we should deterministically see the disk moinitor events
regardless of the state of the build. This is done in a way which
doesn't corrupt build state or need cleanup and is efficient.
(From OE-Core rev: ecc49ee8986929e2429d948000a0ca588fe63959)
(From OE-Core rev: a883aa053ddeb4591109c7c1374525e63a59bd80)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This test occisionally fails as m4 doesn't recompile, meaning the logfile test
then doesn't find mention of ccache.
To ensure m4 does recompile, clean m4 before force compiling it.
(Reading the test is confusing due to the test cleanup also involving a clean)
(From OE-Core rev: 6e0b9214a0d57ed45a5df0ba5c9887a9045b89b1)
(From OE-Core rev: bddb09a411ce69f49a37260e2188bbd9b02f5902)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If runqemu fails it would leak an unclosed socket and file. Ensure we
close these in all cases to remove the resource warning.
(From OE-Core rev: ed80e46ccbc8fe8e9148d80723152066fa00ba28)
(From OE-Core rev: 81ce67fb1274faafb9e7386c5982aeaf4c74294d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Each time a runqemu() fails, the log handler would be left behind meaning
messages from any subsequent run would be duplicated (or worse/more).
This ensures we remove the handler regardless and means we no longer
have the duplication.
(From OE-Core rev: 532984708436bdfa3a8cac2c684a425eb249bad0)
(From OE-Core rev: 018549bf94d1e5d33344691fde0fb27ca50f675f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than just referring the user to the logs containing the failure, print
them on the console. This aids debugging with oe-selftest with parallelisation
as the logs may otherwise be lost.
(From OE-Core rev: 36a018e245a232f520ff946f152cc875927a6fb4)
(From OE-Core rev: 85b373f571cf2076d93e96db2aca295c53d3c16a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The current failure mode doesn't show us what the logs actually looked like
and later cleans can lose them. Show the whole log in case of failure
to aid debugging intermittent problems on the autobuilder.
(From OE-Core rev: 7c3a0dc5978cea898b1ca51decf4d6e7cf9d519f)
(From OE-Core rev: 8a9fa1597245d13db89361c40db7867786f137ff)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This avoids problems where shutil.remove will error with:
File "/usr/lib/python3.5/shutil.py", line 436, in _rmtree_safe_fd
os.unlink(name, dir_fd=topfd)
FileNotFoundError: [Errno 2] No such file or directory: 'S.gpg-agent.extra'
when there are races over file deletion (gpg agent may be slow to exit).
We already worked around speed and race issues in bb.utils.
(From OE-Core rev: 00a8fd5b93a5c19ce0b7498e2bc653ce8ad58aaf)
(From OE-Core rev: ee8255128e11d69d82e6093b58b761dfe773e71d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This should speed the test up signficiantly without any loss of functionality
for the purposes of the test.
(From OE-Core rev: 3dde0b749643575878bfbca2f8d2d9ec30bad166)
(From OE-Core rev: 9fcfc43e1a51520a86761cf5534cff9e70167b77)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We ideally don't want to use gpg from the host. This is straightforward for package
management but not for sstate.
For sstate, create a second build directory to run the test in using gnupg-native
from the original build directory.
(From OE-Core rev: 10afa94c3f0d7eb7524a26deda86949073d55fde)
(From OE-Core rev: 8c91e74ebcd4b1b851026c7ed5e984aa6a39cf90)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Raising an assertionError in the class setup isn't a particuarly good way to
indicate gpg isn't installed. Instead skip the tests if the required binary
isn't present. For the signing tests we do require it to be present and can't
use a prebuilt one.
(From OE-Core rev: 2d486af97e51b9daa9c40482c31d637c9ab4ae79)
(From OE-Core rev: 9bb190d5bee70d2d3d5ec2eeee3439cd24548cb8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
On python versions with close_fds=True (python 3.2 onwards), the tap
device lockfile isn't passed to the child process.
Since this guards against use of an active interface, we really want this
here, so pass it in pass_fds. This means if the parent exits early, the child
still holds the lock, avoiding messages like:
runqemu - ERROR - Failed to run qemu: qemu-system-x86_64: could not configure /dev/net/tun (tap0): Device or resource busy
(From OE-Core rev: 17a0a067d597c445c5892ff9914e91a2187f7e09)
(From OE-Core rev: 1d03e0d54863124c5a65b59ecdc76dbac13d312a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Various tweaks:
- Balance up the aquire/release functions
- Use debug messge for both acquiring and release message for consistency in logs
- Use None instead of an empty string
- Reset the value of the field if we don't have the lock any more
(From OE-Core rev: 2728394ed75526f338cc9cb6bc62fb0ed6dc605f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
subprocess.run() was introduced in Python 3.5. We currently support down to
Python 3.4 so I've replaced it with subprocess.check_call() which is available
in that version.
(From OE-Core rev: 4ba803d9834565cbe9a89838eb2fcf0328c44bef)
Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Small tweak to ensure these items are printed into the log since there
is other logging code which looks for the header and this makes it clearer
the server did start but is slow somewhere in startup.
(Bitbake rev: 9b15f265643af8bd85af87164ee5cab0bf58ccbb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The ${DL_DIR}/svn directory is used by BitBake to keep checked-out SVN
repositories from which tarballs are generated. These repositories were
protected from concurrent update with a lock on the tarballs. However,
the tarballs are specific to the SRCREV and module checked out (many
tarballs can come from the same repository), meaning a repository could
be modified concurrently if two recipes checked out two different
SRCREVs or modules from it in parallel. This caused errors like the
following:
ERROR: Fetcher failure: Fetch command failed with exit code 1, output:
svn: E155004: Run 'svn cleanup' to remove locks (type 'svn help cleanup' for details)
svn: E155004: Working copy '/home/foo/downloads/svn/repo/trunk' locked.
svn: E155004: '/home/foo/downloads/svn/repo/trunk' is already locked.
Fix it by adding a per-repository lock that's independent of the module
and SRCREV.
(Bitbake rev: cddd27fb66bf82465cf8eb5012cb8e3c088625a5)
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@bmw.de>
Signed-off-by: Michael Ho <Michael.Ho@bmw.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently if there are no setscene tasks, the disk monitor isn't started.
Move the startup code to somewhere to ensure it always is started. This
issue would partially explain occasional selftest failures.
(Bitbake rev: 7e76237c930d354ee7eb37d247d71b4f4fbf7993)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Using vaapisink (which doesn't supports DRI3 [1] and uses DRI2) with
default poky configuration currently results in an unresponsive display
because DRI2 rendering doesn't work (as of xserver 1.20.3) in non-composited
environments [2].
Downgrade vaapisink to marginal for now so playbin (and in turn gst-play
and gtk-play examples) uses next best sink element and works out of box.
[1] https://github.com/intel/libva/issues/122
[2] https://gitlab.freedesktop.org/xorg/xserver/issues/13
(From OE-Core rev: 0ac1b496823a57ac6a234a14c60318bf1ffa19b0)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Also use assetIn instead of assertTrue which aids debugging failures.
(Bitbake rev: 7e9c0bffb95ca92cab61deeeef18f45f1b4edb6f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When multiconfig is enabled the cooker adds providers
for all the targets to be built on all the multiconfig
variables that were set, regardless if there is a dependency
to it or not.
This causes an issue when a certain target is incompatible
with one or more of the multiconfigs, e.g. the target is not
in COMPATIBLE_MACHINE for one of the MACHINEs being built,
causing the cooker to error out since no providers can be
found for that certain target on that multiconfig.
This patch modifies the behavior to only look for PROVIDERS
for a target on the multiconfig that was selected to be built,
PROVIDERS are then looked for in other multiconfigs only when
there is a defined dependency to them.
[YOCTO #12985]
(Bitbake rev: 8bd519b04c7d6ee4cf626d5e04110c3759488d28)
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This code checks to see if shallow is either disabled or the tarball is
missing, but the else block tries to print the tarball filename, and
this attribute doesn't exist at all when shallow is disabled. Handle the
two cases separately to give sane errors for both cases without the
exception:
Exception: AttributeError: 'FetchData' object has no attribute 'fullshallow'
(Bitbake rev: b0ad961064ef6ecd77fab3435e5652c4ae284426)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Users are starting to expect OE to work under WSL which it doesn't. Add a warning to
tell them about this up front and manage expectations.
(From OE-Core rev: 4f22710f9a310412f1de0b4e6905c058ec416f25)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
testimage.bbclass installs a SIGTERM handler which conflicts with the
use of multiprocessing here. This is paritcularly problematic if the http
service is terminated before its started and hence before its had a chance
to reset the default signal handler (as the code was written).
Instead, temporarily remove testimage's handler whilst forking the http process
which means the correct handler is installed and won't deadlock.
Also take the opportunity to add in some log messages about the server start
and shutdown so that future debugging is easier and its clearer what the code
is doing.
(From OE-Core rev: cc0471439aa0085ca87deccf061c5b676ef12388)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The openssl build system generates buildinf.h containing the full
compiler command line used to compile objects. This breaks
reproducibility, as the compile command is baked into libcrypto, where
it is used when running `openssl version -f`.
Add stripped build variables for the compiler and cflags lines, and use
those when generating buildinfo.h.
This is based on a similar patch for older openssl versions:
https://patchwork.openembedded.org/patch/147229/
(From OE-Core rev: 8f5b6a3789a7fcbac0a384b84b4c7ef5994023b6)
Signed-off-by: Martin Hundebøll <martin@geanix.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The busybox.link.* files are generated from autoconf.h and applets.h,
which are both auto-generated by the build system. The contents of the
two files might be in different order, and so the link files are not
reproducble as is.
Fix this by sorting the lists using `sort`.
(From OE-Core rev: bade7cc344c2f0e9316f973c34e9c9dfcbdbe32d)
Signed-off-by: Martin Hundebøll <martin@geanix.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The shadow configure script tries really hard to detect the running
shell to make sure it doesn't do unsupported calls.
On my system the shell is detected as /bin/sh, while a build in an
ubuntu docker it resolves to /bin/bash. And since the shell path is
baked into the target binaries through config.h, the build becomes
inreproducible.
Fix reproducibility by hard-coding the shell to be /bin/sh
(From OE-Core rev: 5f4fe91cb6c21cd3ecd0b68d1c6b46a9530c7570)
Signed-off-by: Martin Hundebøll <martin@geanix.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The upstream source tarball checksums changed. Use the copy from our source
mirror to avoid failures.
[YOCTO #12979]
(From OE-Core rev: e97a31e6bbaec5cb56d4750bf5171dbba510ee33)
Signed-off-by: Mohamad Noor Alim Hussin <mohamad.noor.alim.hussin@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Having image-buildinfo enabled causes containerimage.ContainerImageTests.test_expected_files
to fail due to the presence of an unexpected file:
['./',
'./etc/',
- './etc/build',
'./etc/default/',
'./etc/default/postinst',
Tweak the class to allow it to be disabled and disable it from the test just in
case it was enabled.
(From OE-Core rev: af67bf422a4df5b7e07894512ff73a5f493682ab)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
valgrind for qemumips64 multilib builds fails to configure
for libn32 with the error:
configure:6190: checking for 32 bit build support
...
fatal error: bits/long-double-32.h: No such file or directory
It seems that the toolchain is producing:
tmp-glibc/sysroots-components/mips64-n32/libn32-glibc/usr/include/bits/long-double-n32.h
Until the toolchain problem is resolved, skip valgrind for libn32.
(From OE-Core rev: 17d5574f05384edeb5c80ada2724fff4a1c3c94b)
Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently sdk & sdkext will output json file to LOG_DIR, while
selftest will output json file to TOPDIR/log.
Standardize selftest json output file to LOG_DIR.
(From OE-Core rev: 2012d6b076fc833864a0254d56d78536314a6799)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Play the whack-a-mole game and add the .file directive to another
assembly file that otherwise shows itself in ld-2.28.so debug file,
which in turns alters the build-id of ld-2.28.so on target.
(From OE-Core rev: 4cbf901b3a127ed039371e614946002d26d56997)
Signed-off-by: Martin Hundebøll <martin@geanix.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Due to a bug in find [1], -ignore_readdir_race does not work correctly with
-delete. This can lead to spurious build failures when files disappear
while such a command is running; specifically this was seen in the case of
do_configure and do_populate_lic running concurrently for packages
with ${B} == ${WORKDIR}:
find: '.../sstate-build-populate_lic': No such file or directory
While the issue is fixed in the findutils git master, the find command of
the host system is called here, so we can't ensure that the used version
contains the fix. Many common distros have not updated to a recent enough
findutils version yet (Ubuntu 18.10 contains the fix, while 18.04 is still
affected).
Work around the issue by passing the output of find to 'rm -f' instead of
using -delete.
[1] https://savannah.gnu.org/bugs/?52981
(From OE-Core rev: 8079e2d62e23f7c274f46185e6dad64fa95394c1)
Signed-off-by: Matthias Schiffer <matthias.schiffer@ew.tq-group.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use % operator to ensure that the dependency is substituted into the error
message correctly.
(Bitbake rev: 8e587386b0b3a0ed11edf71dfbe9fb508f60d0b3)
Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Support users who want to specify a custom ssh to Git using an environment
variable. (This lets build systems work without requiring users to
configure Git explicitly.)
(Bitbake rev: db46fb2755a4b1033ab60051ce511cc9dd7e34fb)
Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Updates for YOCTO #12891 allowed a user to have a directory
structure different to that of yocto (bitbake isn't inside
oe-core) whereas the definition of OE_ROOT in the main toaster
binary still assumes the same while checking for .templateconf
and hence we see an error on the cmdline in such cases:
bash: <repo-path>/bitbake/bin/../../.templateconf: No such file or directory
The change here now allows the user to provide OE_ROOT through
the environment in such cases and otherwise defaults to the older
mechanism to fix this issue.
(Bitbake rev: 9cc6fe4be797e106899d2448797a4b3af8eace4e)
Signed-off-by: Awais Belal <awais_belal@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Git does not require the module and target path to be the same in the
.gitmodules file. This incorrect assumption was being made previously
causing various unpack failures.
An example .gitmodule showing this issue:
[submodule "plugins/WaveShaper/Libs/inih"]
path = plugins/wolf-shaper/Libs/inih
url = https://github.com/pdesaulniers/inih.git
The unpack function also needed to work in a loop on the overall
submodules_queue. Before it could have missed items that were not in the
primary repository.
(Bitbake rev: 5a7009c204f4d2254e3b2d83ad9319ac23f1cf4d)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Submodules by definition refer to a specific commit, not branch. If we don't
ignore the branch, then any commits on a submodule on a branch different then
the original module will trigger a failure that the commit is not on the
branch.
(Bitbake rev: fdc1dbf96f153b496de52acd8263366a1ff303ad)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
libunwind is not yet ported for ARC & RISCV64 and on attempt
to build it for those arches we just get an error message.
If we explicitly disable libunwind it is gracefully handled by
perf build system and it just gets configured to not use it
so perf is still usable even on those arches.
(From OE-Core rev: bb06900a1f29fdd4066f6d7e7a961d230c2e9438)
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
Cc: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In sources directory we can find patches/ and temp/.
The first one is filled with symbolic link unusable on another
machines.
The second contains yocto logs to create this archives and are
typically copied when 'S = "${WORKDIR}"'
(From OE-Core rev: 3904f98851c6a63dd9377e38f1432be6b1c0a94d)
Signed-off-by: Fabien Lahoudere <fabien.lahoudere@collabora.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Temporarily modify path to run as non-privileged user on more distros.
Change the recommended value to match what we use on the autobuilder.
(From OE-Core rev: 2f86c3f1180ba8bf77301a0bc29c3b290bc28d91)
Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rationale - excerp from `info chown`
====================================
OWNER‘:’GROUP
If the OWNER is followed by a colon and a GROUP (a group name or
numeric group ID), with no spaces between them, the group ownership
of the files is changed as well (to GROUP).
Some older scripts may still use ‘.’ in place of the ‘:’ separator.
POSIX 1003.1-2001 (*note Standards conformance::) does not require
support for that, but for backward compatibility GNU ‘chown’ supports
‘.’ so long as no ambiguity results. New scripts should avoid the use
of ‘.’ because it is not portable, and because it has undesirable
results if the entire OWNER‘.’GROUP happens to identify a user whose
name contains ‘.’.
(From OE-Core rev: 185918234a07cb506d7d7464a49ac33972c7d963)
Signed-off-by: Kosta Zertsekel <zertsekel@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
