The existing logfile is simply placed in the current directory. Since the test
changes cwd to BUILDDIR, the symlink to the log can be placed in an invalid
directory. We also see trackbacks if the symlink is invalid.
Improve things by:
* Placing logs in LOG_DIR (or BUILDDIR if unset).
* Using a full path to the log meaning the log and link are placed in the same directory.
* Using lexists instead of exists so invalid symlinks are handled correctly.
(From OE-Core rev: 750ece11bed0e62a11e0003d1d16a81f7c219761)
(From OE-Core rev: 1ec53b8d82491aeb9f49e7a78f531e98b5608f0f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Heartbeat events default to once a second and we need to ensure we have
enough time in the task to see them.
Add a nostamp delay task 5s long so we can have a consistently timed
task which doesn't need cleanup or have unneeded dependencies. This
ensures we should deterministically see the disk moinitor events
regardless of the state of the build. This is done in a way which
doesn't corrupt build state or need cleanup and is efficient.
(From OE-Core rev: ecc49ee8986929e2429d948000a0ca588fe63959)
(From OE-Core rev: a883aa053ddeb4591109c7c1374525e63a59bd80)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This test occisionally fails as m4 doesn't recompile, meaning the logfile test
then doesn't find mention of ccache.
To ensure m4 does recompile, clean m4 before force compiling it.
(Reading the test is confusing due to the test cleanup also involving a clean)
(From OE-Core rev: 6e0b9214a0d57ed45a5df0ba5c9887a9045b89b1)
(From OE-Core rev: bddb09a411ce69f49a37260e2188bbd9b02f5902)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If runqemu fails it would leak an unclosed socket and file. Ensure we
close these in all cases to remove the resource warning.
(From OE-Core rev: ed80e46ccbc8fe8e9148d80723152066fa00ba28)
(From OE-Core rev: 81ce67fb1274faafb9e7386c5982aeaf4c74294d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Each time a runqemu() fails, the log handler would be left behind meaning
messages from any subsequent run would be duplicated (or worse/more).
This ensures we remove the handler regardless and means we no longer
have the duplication.
(From OE-Core rev: 532984708436bdfa3a8cac2c684a425eb249bad0)
(From OE-Core rev: 018549bf94d1e5d33344691fde0fb27ca50f675f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than just referring the user to the logs containing the failure, print
them on the console. This aids debugging with oe-selftest with parallelisation
as the logs may otherwise be lost.
(From OE-Core rev: 36a018e245a232f520ff946f152cc875927a6fb4)
(From OE-Core rev: 85b373f571cf2076d93e96db2aca295c53d3c16a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The current failure mode doesn't show us what the logs actually looked like
and later cleans can lose them. Show the whole log in case of failure
to aid debugging intermittent problems on the autobuilder.
(From OE-Core rev: 7c3a0dc5978cea898b1ca51decf4d6e7cf9d519f)
(From OE-Core rev: 8a9fa1597245d13db89361c40db7867786f137ff)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This avoids problems where shutil.remove will error with:
File "/usr/lib/python3.5/shutil.py", line 436, in _rmtree_safe_fd
os.unlink(name, dir_fd=topfd)
FileNotFoundError: [Errno 2] No such file or directory: 'S.gpg-agent.extra'
when there are races over file deletion (gpg agent may be slow to exit).
We already worked around speed and race issues in bb.utils.
(From OE-Core rev: 00a8fd5b93a5c19ce0b7498e2bc653ce8ad58aaf)
(From OE-Core rev: ee8255128e11d69d82e6093b58b761dfe773e71d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This should speed the test up signficiantly without any loss of functionality
for the purposes of the test.
(From OE-Core rev: 3dde0b749643575878bfbca2f8d2d9ec30bad166)
(From OE-Core rev: 9fcfc43e1a51520a86761cf5534cff9e70167b77)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We ideally don't want to use gpg from the host. This is straightforward for package
management but not for sstate.
For sstate, create a second build directory to run the test in using gnupg-native
from the original build directory.
(From OE-Core rev: 10afa94c3f0d7eb7524a26deda86949073d55fde)
(From OE-Core rev: 8c91e74ebcd4b1b851026c7ed5e984aa6a39cf90)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Raising an assertionError in the class setup isn't a particuarly good way to
indicate gpg isn't installed. Instead skip the tests if the required binary
isn't present. For the signing tests we do require it to be present and can't
use a prebuilt one.
(From OE-Core rev: 2d486af97e51b9daa9c40482c31d637c9ab4ae79)
(From OE-Core rev: 9bb190d5bee70d2d3d5ec2eeee3439cd24548cb8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Using vaapisink (which doesn't supports DRI3 [1] and uses DRI2) with
default poky configuration currently results in an unresponsive display
because DRI2 rendering doesn't work (as of xserver 1.20.3) in non-composited
environments [2].
Downgrade vaapisink to marginal for now so playbin (and in turn gst-play
and gtk-play examples) uses next best sink element and works out of box.
[1] https://github.com/intel/libva/issues/122
[2] https://gitlab.freedesktop.org/xorg/xserver/issues/13
(From OE-Core rev: 0ac1b496823a57ac6a234a14c60318bf1ffa19b0)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Users are starting to expect OE to work under WSL which it doesn't. Add a warning to
tell them about this up front and manage expectations.
(From OE-Core rev: 4f22710f9a310412f1de0b4e6905c058ec416f25)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
testimage.bbclass installs a SIGTERM handler which conflicts with the
use of multiprocessing here. This is paritcularly problematic if the http
service is terminated before its started and hence before its had a chance
to reset the default signal handler (as the code was written).
Instead, temporarily remove testimage's handler whilst forking the http process
which means the correct handler is installed and won't deadlock.
Also take the opportunity to add in some log messages about the server start
and shutdown so that future debugging is easier and its clearer what the code
is doing.
(From OE-Core rev: cc0471439aa0085ca87deccf061c5b676ef12388)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The openssl build system generates buildinf.h containing the full
compiler command line used to compile objects. This breaks
reproducibility, as the compile command is baked into libcrypto, where
it is used when running `openssl version -f`.
Add stripped build variables for the compiler and cflags lines, and use
those when generating buildinfo.h.
This is based on a similar patch for older openssl versions:
https://patchwork.openembedded.org/patch/147229/
(From OE-Core rev: 8f5b6a3789a7fcbac0a384b84b4c7ef5994023b6)
Signed-off-by: Martin Hundebøll <martin@geanix.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The busybox.link.* files are generated from autoconf.h and applets.h,
which are both auto-generated by the build system. The contents of the
two files might be in different order, and so the link files are not
reproducble as is.
Fix this by sorting the lists using `sort`.
(From OE-Core rev: bade7cc344c2f0e9316f973c34e9c9dfcbdbe32d)
Signed-off-by: Martin Hundebøll <martin@geanix.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The shadow configure script tries really hard to detect the running
shell to make sure it doesn't do unsupported calls.
On my system the shell is detected as /bin/sh, while a build in an
ubuntu docker it resolves to /bin/bash. And since the shell path is
baked into the target binaries through config.h, the build becomes
inreproducible.
Fix reproducibility by hard-coding the shell to be /bin/sh
(From OE-Core rev: 5f4fe91cb6c21cd3ecd0b68d1c6b46a9530c7570)
Signed-off-by: Martin Hundebøll <martin@geanix.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The upstream source tarball checksums changed. Use the copy from our source
mirror to avoid failures.
[YOCTO #12979]
(From OE-Core rev: e97a31e6bbaec5cb56d4750bf5171dbba510ee33)
Signed-off-by: Mohamad Noor Alim Hussin <mohamad.noor.alim.hussin@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Having image-buildinfo enabled causes containerimage.ContainerImageTests.test_expected_files
to fail due to the presence of an unexpected file:
['./',
'./etc/',
- './etc/build',
'./etc/default/',
'./etc/default/postinst',
Tweak the class to allow it to be disabled and disable it from the test just in
case it was enabled.
(From OE-Core rev: af67bf422a4df5b7e07894512ff73a5f493682ab)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
valgrind for qemumips64 multilib builds fails to configure
for libn32 with the error:
configure:6190: checking for 32 bit build support
...
fatal error: bits/long-double-32.h: No such file or directory
It seems that the toolchain is producing:
tmp-glibc/sysroots-components/mips64-n32/libn32-glibc/usr/include/bits/long-double-n32.h
Until the toolchain problem is resolved, skip valgrind for libn32.
(From OE-Core rev: 17d5574f05384edeb5c80ada2724fff4a1c3c94b)
Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently sdk & sdkext will output json file to LOG_DIR, while
selftest will output json file to TOPDIR/log.
Standardize selftest json output file to LOG_DIR.
(From OE-Core rev: 2012d6b076fc833864a0254d56d78536314a6799)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Play the whack-a-mole game and add the .file directive to another
assembly file that otherwise shows itself in ld-2.28.so debug file,
which in turns alters the build-id of ld-2.28.so on target.
(From OE-Core rev: 4cbf901b3a127ed039371e614946002d26d56997)
Signed-off-by: Martin Hundebøll <martin@geanix.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Due to a bug in find [1], -ignore_readdir_race does not work correctly with
-delete. This can lead to spurious build failures when files disappear
while such a command is running; specifically this was seen in the case of
do_configure and do_populate_lic running concurrently for packages
with ${B} == ${WORKDIR}:
find: '.../sstate-build-populate_lic': No such file or directory
While the issue is fixed in the findutils git master, the find command of
the host system is called here, so we can't ensure that the used version
contains the fix. Many common distros have not updated to a recent enough
findutils version yet (Ubuntu 18.10 contains the fix, while 18.04 is still
affected).
Work around the issue by passing the output of find to 'rm -f' instead of
using -delete.
[1] https://savannah.gnu.org/bugs/?52981
(From OE-Core rev: 8079e2d62e23f7c274f46185e6dad64fa95394c1)
Signed-off-by: Matthias Schiffer <matthias.schiffer@ew.tq-group.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
libunwind is not yet ported for ARC & RISCV64 and on attempt
to build it for those arches we just get an error message.
If we explicitly disable libunwind it is gracefully handled by
perf build system and it just gets configured to not use it
so perf is still usable even on those arches.
(From OE-Core rev: bb06900a1f29fdd4066f6d7e7a961d230c2e9438)
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
Cc: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In sources directory we can find patches/ and temp/.
The first one is filled with symbolic link unusable on another
machines.
The second contains yocto logs to create this archives and are
typically copied when 'S = "${WORKDIR}"'
(From OE-Core rev: 3904f98851c6a63dd9377e38f1432be6b1c0a94d)
Signed-off-by: Fabien Lahoudere <fabien.lahoudere@collabora.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rationale - excerp from `info chown`
====================================
OWNER‘:’GROUP
If the OWNER is followed by a colon and a GROUP (a group name or
numeric group ID), with no spaces between them, the group ownership
of the files is changed as well (to GROUP).
Some older scripts may still use ‘.’ in place of the ‘:’ separator.
POSIX 1003.1-2001 (*note Standards conformance::) does not require
support for that, but for backward compatibility GNU ‘chown’ supports
‘.’ so long as no ambiguity results. New scripts should avoid the use
of ‘.’ because it is not portable, and because it has undesirable
results if the entire OWNER‘.’GROUP happens to identify a user whose
name contains ‘.’.
(From OE-Core rev: 185918234a07cb506d7d7464a49ac33972c7d963)
Signed-off-by: Kosta Zertsekel <zertsekel@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We've had a number of occasions where the Yocto Project source mirrors have not
been complete or functioning correctly. This adds a test so that if this happens
we find out out it sooner.
It also only works over http meaning we should be able to test that anyone behind
an http only proxy (no git protocol) also has functional fetches for OE-Core and
layers built by the core of the project.
(From OE-Core rev: 9a25b2eb9cc13dd77a774dad83067ea5f9bf9716)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Both RHEL and SLES uses ppc64/ppc64le for powerpc 64 bit big/little
endian targets instead of powerpc64/powerpc64le in libgpg-error.
Also libgpg-error provides common target system names in form like
<arch>-unknown-linux-gnu.
Add mapping for ppc64/ppc64le targets to their libgpg-error equivalents
to fix native builds.
Cross build for arm64 tested on IBM Power 8 machine with RHEL7 for
ppc64le variant only, but should work for ppc64 as well.
(From OE-Core rev: f1af780769477f06eb925fd87c844baba04ada2d)
Signed-off-by: Serhey Popovych <serhe.popovych@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since commit 709c603dec19 ("v86d: Accept aarch64 as build host") we
support cross compilation on aarch64 host in addition to x86 host.
However building on hosts different than two above will fail.
Make cross compilation support more generic by checking for TARGET_ARCH
in v86d configure script with fallback to `uname -m` when not present in
environment and pass TARGET_ARCH explicitly in do_configure().
Cross build for x86 tested on IBM Power 8 machine with RHEL7. Should
work on aarch64 and rest too.
(From OE-Core rev: 69f7579eb36c00b557225377e1a3cc61b103be23)
Signed-off-by: Serhey Popovych <serhe.popovych@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Several distros are now shipping "python" as python v3 contra to the original
python guidelines. This causes users confusion/pain in trying to use our tools.
We can just force "python" to "python2" within HOSTTOOLS to avoid this issue
and hide the complexity from the user.
(From OE-Core rev: b06a6cde5c5503f456f260c773cf126085e18c8d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows the ptest results from ptest-runner, run in an image to be
transferred over to the resulting json results output.
Each test is given a pass/skip/fail so individual results can be monitored
and the raw log output from the ptest-runner is also dumped into the
results json file as this means after the fact debugging becomes much easier.
Currently the log output is not split up per test but that would make a good
future enhancement.
I attempted to implement this as python subTests however it failed as the
output was too confusing, subTests don't support any kind of log
output handling, subTest successes aren't logged and it was making things
far more complex than they needed to be.
We mark ptest-runner as "EXPECTEDFAILURE" since its unlikely every ptest
will pass currently and we don't want that to fail the whole image test run.
Its assumed there would be later analysis of the json output to determine
regressions. We do have to change the test runner code so that
'unexpectedsuccess' is not a failure.
Also, the test names are manipuated to remove spaces and brackets with
"_" used as a replacement and any duplicate occurrences truncated.
(From OE-Core rev: a13e088942e2a3c3521e98954a394e61a15234e8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Similarlly to OE-Core rev 4b936cde58ca0a6f34092ce82640a02859110411 for
cross.sdk, BUILD_* flags can't be used as TARGET_* flags
gcc-crosssdk buils leaks config.log's through "gcc-stashed-builddir" and
TARGET_* flags to libgcc cross-build through "gcc/libgcc.mvars" file
on "gcc-stashed-builddir". This means that if BUILD_CFLAGS contains
host-specific flags like "-isystem/usr/include" libgcc build will
fail "do_qa_configure" and "do_package_qa" checks.
Remove host-related flags from TARGET_* flags for gcc-crosssdk builds.
[YOCTO #11874]
(From OE-Core rev: 6e162e619b6f5173c073cd9bedbcadf205017e30)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Right now go-cross is changing signatures when you change TUNE for a given
architecture. In particular this breaks layer tests like:
yocto-check-layer ../meta-yocto-bsp/ --machines qemuarm beaglebone-yocto
This changes the PN addtion to something containing the tune rather than
the arch which avoids these kinds of errors. If go-cross can be tune
independent that would be nice but currently that isn't the case.
[YOCTO #12586]
(From OE-Core rev: e3c7e1703499e6a5332d9ab8a941671ec8235c4f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The current go-dep does not compile ptest successfully on mips
and mips64. So as a workaround, disable PTEST_ENABLED explicitly
to avoid error like below.
| vet config not found
(From OE-Core rev: 786322ec408e2ef5cd6fb809456e0453e5f5e162)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Pyvenv is just a small script that uses venv to create virtual
environments.
https://www.python.org/dev/peps/pep-0405/#creating-virtual-environments
This patch adds the python3-venv module as a self-contained package which
python3-pyvenv must depend on at run-time.
The patch also provides the package python3-pyvenv from the pyhton3-venv
package.This is good for future-proofing since python3-pyvenv has been
deprecated and only python3-venv is now available in Python 3.6.
https://docs.python.org/3/library/venv.html.
Without this patch python3-pyvenv is broken because it is missing the
venv module at run-time. This patch specifies the newly created
python3-venv as a run-time dependency of python3-pyvenv.
(From OE-Core rev: effa141bfce55aab25142ee578c95383c755ad73)
Signed-off-by: Hugues Kamba <hugues.kamba@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some of gdbm's ptest cases require gdbmtool, which is packaged into
${PN}-bin. So extend the RDEPENDS_${PN}-ptest to include the package.
(From OE-Core rev: e188a75aa882efc98b8390f43f18279c3707314a)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Four unrelated lines were extracted from pixman-matrix.c for the
license information.
License-Update: Only extract the relevant part from pixman-matrix.c
(From OE-Core rev: d0a209e8cf29d982567e3978e1dcbb3871505a39)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
License-Update: Only extract relevant parts from init.c & gpg-error.h.in
(From OE-Core rev: a5c1ff6deb6393666745889eee8297112848ba28)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Two unrelated lines were extracted from apu_version.h for the license
information.
License-Update: Only extract the relevant part from apu_version.h
(From OE-Core rev: 2edb0f24a13f27b2fae94fb447221ad2ddb924a0)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Two unrelated lines were extracted from apr_lib.h for the license
information.
License-Update: Only extract the relevant part from apr_lib.h
(From OE-Core rev: 90ab83ecc509c2fdc1f6083d771031decdcaad63)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Once the value of SOURCE_DATE_EPOCH is determined, it is stored in the recipe's SDE_FILE.
If none of the existing mechanisms are suitable, replace the do_deploy_source_date_epoch task
with recipe-specific functionality to write the appropriate SOURCE_DATE_EPOCH into the SDE_FILE.
(From OE-Core rev: e7b891b76954c784f5a93bd0a1c91315673ce40d)
Signed-off-by: Douglas Royds <douglas.royds@taitradio.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Backport patch to fix CVE-2018-0735 for openssl 1.1.1.
(From OE-Core rev: 78e751e33d3ec4394d96391e737cc39cad960ebe)
Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
