Commit Graph

2557 Commits

Author SHA1 Message Date
Alexander Kanavin
ff952280c8 oeqa/logparser: correctly set test result markers
I spotted this by noticing ptests started to unexpectedly pass,
which shouldn't happen yet.

(From OE-Core rev: 1cd6e9a8fdfef927916c6cea2371fc0430b1ffdf)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-22 23:57:27 +00:00
Richard Purdie
325bc7438e oeqa/logparser: Fix performance issues with ptest log parsing
On the autobuilder a ptest log with 2.1 million lines took around 18 hours to
process. This is clearly crazy.

We can tweak the processing code to:
a) Stop repeatedly joining large strings together (append to a list instead)
b) Use one startswith expression instead of multiple re.search() operations

With this change it takes 18 hours down to around 12s.

[YOCTO #13696]

(From OE-Core rev: 01c9f40b979e36a53e789a6bedd89b6d9557dce3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-21 09:39:00 +00:00
Richard Purdie
bf745476ea oeqa/selftest: Drop 'backup' code and SIGTERM handler
Now selftest is using its own copied build directory, we can stop worrying about
copying files around as backup, and drop the SIGTERM handler to try and restore
them, simplifying the code.

(From OE-Core rev: b8ea8a910267fee4bb9e57f24ba829064e22d016)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-21 09:39:00 +00:00
Richard Purdie
c569157d9d oeqa/selftest: Standardise seperate builddir for concurrent and non-concurrent selftest
Currently oe-selftest reuses the current build directory and the concurrent
version run with -j does not.

Standardise and use a separate new build directory in both cases. This will lead
to simpler code and more reliable user run tests.

(From OE-Core rev: 50238b0717b04e0a1fa69d618e8c8aa8445a80b0)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-21 09:39:00 +00:00
Richard Purdie
dba8c1d5ef selftest/context: Avoid tracebacks from tests using multiprocessing
We can see tracebacks where the SIGTERM handler catches things
it shouldn't. Avoid exit(1) unless we're the process that
it was intended for.

[YOCTO #13664]

(From OE-Core rev: d9c62ffac611310efd47ed6397d31dccb72fe868)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-21 09:39:00 +00:00
Trevor Gamblin
0f54a2fd69 oescripts.py: use unittest.SkipUnless for class skip
Yocto Bug: https://bugzilla.yoctoproject.org/show_bug.cgi?id=13663

Prior to this fix, the try/except block in OEScriptTests' setUpClass
did both the import and the raise unittest.SkipTest for the
subsequent OEPybootchartguyTests. The consequence of the this meant
that, when cairo was not available on the test host, the raise was
counted only once as a class setup issue, instead of once for each
of the tests to be run, resulting in incorrect skip counts in the
test results.

Instead of raising unittest.SkipTest in setUpClass, the more
proper method is to use the unittest.skipUnless method along with
importlib.util.find_spec to check for the presence of the
python3-cairo module. This allows the entire class to be skipped
if cairo is not present, while also correctly passing along the
skip state for each of the class's tests, and avoiding the UNKNOWN
results to appear in cases where the -j option was provided to
oe-selftest, but the tests were still skipped.

(From OE-Core rev: 10c2216cb6fb61c74babd8bd17cb3481754b7475)

Signed-off-by: Trevor Gamblin <trevor.gamblin@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-20 13:00:28 +00:00
Joshua Watt
95d695b237 oeqa: reproducible: Include jquery in results
Includes jquery in the output directory to make the diffoscope output
easier to navigate

(From OE-Core rev: ff7c386455848e497ef40f9fee2be87d5171e203)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-19 21:28:06 +00:00
Trevor Gamblin
450688b010 concurrencytest.py: add outSideTestaddSkip for subunit
see: https://bugzilla.yoctoproject.org/show_bug.cgi?id=13663

When running oe-selftest in concurrency mode (e.g. with oe-selftest
--run-tests oescripts.OEPybootchartguyTests -j 4), if a skip occurred
during setUpClass() rather than within individual tests, the entire
suite would show "UNKNOWN" as each test's result. This is because
subunit doesn't know how to handle skips outside of individual tests.
An example of where this occurs is when running the above call to
oe-selftest in concurrency mode on a host machine that does not have
python3-cairo installed.

Patch subunit inside concurrencytest.py to provide a method called
outSideTestaddSkip, which will allow subunit to correctly detect the
skip in setUpClass().

(From OE-Core rev: 9b8734b584d6e8d9c32ff2a721b29f3f3e61cca7)

Signed-off-by: Trevor Gamblin <trevor.gamblin@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-18 23:53:55 +00:00
Robert P. J. Day
e1a2a5a206 fix various, innocuous typos
Corrections:

  - environment
  - accommodate
  - conversion
  - compatible

(From OE-Core rev: 9797d3b45b4e1b9d77f0f2ee299c17b48d8d3cf6)

Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-18 23:53:54 +00:00
Alexander Kanavin
ae6ec0b809 weston: add a basic runtime test
The test is checking that weston is able to start.

(From OE-Core rev: 57700767f90eea8f2b78187c42581aca42d50bbf)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-18 23:53:54 +00:00
Alexander Kanavin
ec7beb650f qemux86: drop resolution setting via uvesafb
I am not sure if this has ever worked, but uvesafb is a really
outdated (VBE from the 1990s), awkward (needs v86d) and limited
(no support for high resolutions) way to do it.

The specific reason 640x480-32 was introduced (ages ago) was
to force 32 bit mode with vmware driver, as 16bit had rendering issues.

The modern, supported option is video=... kernel parameter documented here:
https://wiki.archlinux.org/index.php/kernel_mode_setting#Forcing_modes_and_EDID
https://github.com/torvalds/linux/blob/master/Documentation/fb/modedb.rst
which can be passed directly to runqemu and doesn't require special
kernel modules.

Sato under X will continue to use 640x480 as that is hardcoded into
xorg.conf under qemu.

(From OE-Core rev: 1cf26f69fd89b43be24cd1232c43e5050b9d718a)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-18 23:53:54 +00:00
Kevin Hao
623326ac5a selftest: wic: Add APPEND to the optional variables list
The APPEND is an optional variable, so add it to the optional variables
list to make the wic selftest happy.

(From OE-Core rev: 7c8b3a1b920fd4cc1598357985eef0e5e0e0ac79)

Signed-off-by: Kevin Hao <kexin.hao@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-15 22:42:14 +00:00
Joshua Watt
d59bfa1334 oeqa: reproducible: Run diffoscope on saved output
If there are differing packages and they are being saved for review,
automatically run diffoscope on them and include the output in the saved
output. The output is currently done in HTML format since these are
typically published on a webpage by the autobuilder.

(From OE-Core rev: 6e1d5e8b58f0940ba6dfd99536159dd974e6f24c)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-14 13:07:23 +00:00
Khem Raj
21d97d00d5 oeqa: Use --disable-maintainer-mode configure option
since the versions of autotools might differ on target and build host, plus difference in
timestamps for configure and system can result in reconfigure lets avoid
that by disabling maintainer mode

Avoids
error: newly created file is older than distributed files!

[YOCTO #13779]

Suggested-by: Andre McCurdy <armccurdy@gmail.com>
(From OE-Core rev: 3dc4e27e7633ce3ca6b9647810d0996bdee48771)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-14 13:07:23 +00:00
Alejandro Hernandez Samaniego
a3416a5933 testimage: Extend runtime testing infrastructure to allow unconventional booting processes to be tested
The current runtime infrastructure contains hardcoded values which Ill refer to
as patterns, these patterns are either searched through or sent via the serial
terminal to communicate between HOST and TARGET.

These patterns are required since they allow us to check when a device has
finished booting, to log in, and to check whether a command sent from our tests
has returned, this way we are able to check both the status of the commands that
were sent along with its output.

The testing process goes somewhat as follows:
1. Launch QEMU and start booting.
2. Check when the device has booted by looking for the pattern login:.
3. Log in as the root user (default for our images).
4. Check that we were able to log in succesfully.
5. Start running the runtime test cases defined by TEST_SUITES.
6. One of such test cases could send a command to the QEMU target.
7. Check whether that command returned.
8. Check its output and status, return whether the test case passed or failed.

This patch allows this set of patterns to be defined instead of being hardcoded,
but it also automatically sets the defaults that we have been using in the past
if they have not been manually defined, for this reason, the patch is less
invasive and should not affect in any way how tests are currently being run.

Cases that can be enabled with this patch:
- A customized image that does not use the root user (or maybe we want to check
what happens if we dont use the root user).
- An image where the PS1 env variable has been modified, and the prompt pattern
wouldnt match the default.
- Baremetal applications, which do not follow the conventional way of booting
Linux and would probably not show a prompt for a user to log in, same applies
for testing bootloaders.
- poky-tiny: Using DISTRO=poky-tiny and an image such as the core-image-tiny
from meta-intel, which boots directly to RAM, and does not show a log in prompt
since it does not contain a conventional init process.

The code itself contains comments that should be self explanatory but here is an
example on how these patterns can be defined in a hypothetical case where we
want to run test cases as the webserver user instead:

TESTIMAGE_BOOT_PATTERNS = "send_login_user search_login_succeeded"
TESTIMAGE_BOOT_PATTERNS[send_login_user] = "webserver\n"
TESTIMAGE_BOOT_PATTERNS[search_login_succeeded] = "webserver@[a-zA-Z0-9\-]+:~#"

The variable TESTIMAGE_BOOT_PATTERNS defines which patterns to override when
used to communicate with the target when booting, anyone familiar with the
PACKAGECONFIG syntax should have no trouble setting these.

Other patterns would still be set up as default, e.g.
search_reached_prompt would still be login:

The accepted flags for TESTIMAGE_BOOT_PATTERNS are the following:
search_reached_prompt, send_login_user, search_login_succeeded,
search_cmd_finished.

They are prefixed with either search/send, to differentiate if the pattern is
meant to be sent or searched to/from the target terminal.

A working example of this code that falls under the baremetal case mentioned
above along with a test case is present on the meta-freertos layer, which tests
an RTOS image built with OpenEmbedded and automatically runs a test case on it
after booting such image:

As usual, INHERIT += "testimage" needs to be present on local.conf
$ bitbake freertos-demo -c testimage

RESULTS:
RESULTS - freertos_echo.FreeRTOSTest.test_freertos_echo: PASSED (2.00s)
SUMMARY:
freertos-demo () - Ran 1 test in 2.006s
freertos-demo - OK - All required tests passed (successes=1, skipped=0,
failures=0, errors=0)

(From OE-Core rev: 3ab2cbfeff371e8791b031a2852eeef80101a831)

Signed-off-by: Alejandro Hernandez Samaniego <aehs29@gmail.com>
Signed-off-by: Alejandro Hernandez Samaniego <alejandro@enedino.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-13 12:19:14 +00:00
Alejandro Hernandez Samaniego
e597809307 testimage: Allow testing on QEMU machines with a single serial port
commmit 6cde67d0a8 enables the use of qemurunner on machines that
only have a single serial port, but still sets the default value
as serial_ports=2 if not provided.

The testimage class does not call qemurunner with a serial_ports
argument, hence always defaulting to two.

Pass the serial_ports argument from the testimage class to allow
tests to run on QEMU machines with a single serial port.

(From OE-Core rev: a50d0163770f0b405a8de8a8a9cccd48c1de4112)

Signed-off-by: Alejandro Hernandez Samaniego <alejandro@enedino.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-13 12:19:14 +00:00
Trevor Gamblin
9afefb02b0 qemurunner.py: add try/except for pid handling race
In some instances, attempts to remove the qemu pidfile within the
stop() method fail despite the os.path.exists() call immediately
before implying that the file is present. Add a try/except block
to log a warning if this occurs, rather than failing outright,
since the process simply appears to be exiting at an inconvenient
time.

[YOCTO #13675]

(From OE-Core rev: eadb899e23b18eb9eaff145c3bf5b20fb417c3e8)

Signed-off-by: Trevor Gamblin <trevor.gamblin@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:02 +00:00
Khem Raj
504b206e37 oeqa: Use cpio 2.13 as testcase
cpio 2.12 was released in 2015 and might have used older autotools
which could result in errors like

https://bugzilla.yoctoproject.org/show_bug.cgi?id=13779

Bumping to 2.13 will help in matching the tool versions
A good change on top would be to run

aclocal -I .; autoheader; autoconf; automake --add-missing -c

before running configure step perhaps

[YOCTO #13779]

(From OE-Core rev: 84eb1dc4fe8a11cd2d05b703070a6fb6de05b873)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:02 +00:00
Mark Hatle
cdcacf26ec microblaze: Adjust Linux items from microblazeeb to microblaze
Due to recent changes to the tune, in order to match config.guess, the name
of the big-endian microblaze architecture was changes to 'microblaze'.

(From OE-Core rev: 6f6a6bbac684ead3fe6d070d61f17c2f611a2c87)

Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-06 12:16:34 +00:00
Richard Purdie
7c201279cb oeqa/reproducible: Improve test output and ensure deb+ipk compared
Adding newline characters between the packages in the failure output
massively improves readability.

Also ensure to output ipk failures when there are deb failures by
calling self.fail() at the end, else sometimes only partial differences
are returned.

(From OE-Core rev: 6e2e0480852177db75a6108d77c99c92c4e9950f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-06 12:16:34 +00:00
Trevor Gamblin
b486903ccc logrotate.py: improve oeqa test implementation
See bug https://bugzilla.yoctoproject.org/show_bug.cgi?id=13632

Autobuilder tests occasionally fail, reporting that a new logfile
could not be created. While this failure did occur multiple times, it
could not be manually reproduced. However, there are issues with the
implementation of the logrotate.py script that can be fixed. These
changes will help make the failures clearer, should they continue to
occur.

Previously, the test_2_logrotate test would, after running the
logrotate tool, use "ls -al $HOME/logrotate_dir | wc -l" to count
the number of files in the rotation directory and determine if the
rotation was successful. The test to see if there are at least three
files is problematic, because depending on the version of ls used, it
may report the target value of 3 even when there are only hidden files
in the directory, potentially reporting a pass for the test when it
should actually fail. An example with coreutils:

root@qemux86-64:~# ls -al emptydir/
total 2
drwxr-xr-x 2 root root 1024 Jan 14 19:50 .
drwx------ 3 root root 1024 Jan 14 19:50 ..
root@qemux86-64:~#

Where "total" is the number of blocks used. Compare with busybox ls:

root@qemux86-64:~# ls -al emptydir/
drwxr-xr-x    2 root     root          1024 Jan 14 19:54 .
drwx------    3 root     root          1024 Jan 14 19:54 ..
root@qemux86-64:~#

Instead of using ls to verify that a certain number of files exists
in $HOME/logrotate_dir, the tests have been changed to rotate two
specific logs: the log for wtmp and a new logrotate_testfile created
during the second test. Both tests check that the logs are correctly
rotated into $HOME/logrotate_dir by using find and grep on the
expected filename (e.g. "wtmp" when rotated becomes "wtmp.1", so we
check to see that wtmp.1 is present in $HOME/logrotate_dir). In
addition, should the test fail, the output from logrotate -vf is
included in the test log to aid debugging. It has also been seen that
in some cases, the logrotate test fails because the /var/log/wtmp file
is not yet present. Since the objective of the test is to check the
logrotate functionality and not the presence of certain log files,
test_logrotate_wtmp uses the touch command to help ensure that the
file is present before the call to logrotate is issued.

Finally, note that while the autobuilder failures that this patch
addresses were only seen during core-image-full-cmdline tests, these
changes were successfully tested on core-image-minimal and
core-image-sato with the manual addition of logrotate and openssh-sshd
to the images.

(From OE-Core rev: b9392403e96c6d880de844b44da2c7300fcf411c)

Signed-off-by: Trevor Gamblin <trevor.gamblin@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-05 12:00:34 +00:00
André Draszik
e4893d4830 oeqa: modernise subprocess: preexec_fn=os.setsid -> start_new_session=True
start_new_session was added to python3 subprocess in v3.2 and
is meant to take the place of the common use of preexec_fn to
call os.setsid() in the child - as done here.

Update to use the new equivalent.

(From OE-Core rev: c3bd502f31a621394a2f6ea3ae90443ff4ac98a4)

Signed-off-by: André Draszik <git@andred.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-04 15:56:29 +00:00
Henning Schild
d02d938cc6 lib/oe/path: try hardlinking instead of guessing when it might fail
The comparison of the stat st_dev is not enough to judge whether
hardlinking will work. One example would be where you try and hardlink
across two bind-mounts of a directory. The st_dev will be the same and
the operation will still fail.

Instead of implementing a check to try and figure out hardlink support
just try hardlinking and fall back to a copy when running into an
exception.

(From OE-Core rev: f5571bda8327f927feb23b167ab4594b7d0c95bc)

Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-04 15:56:29 +00:00
Daniel McGregor
a26efa2875 cmake: prefer CMAKE_BUILD_PARALLEL_LEVEL
cmake 3.12 introduced this environment variable. Prefer it to passing
PARALLEL_MAKE and PARALLEL_MAKEINST on the cmake command line, because
it gets passed to second stage cmake invocations while command-line
arguments do not (for example, multi-stage clang builds)

(From OE-Core rev: cdd44c93f02bb8cc2fa773e13c8ce36e3da23921)

Signed-off-by: Daniel McGregor <daniel.mcgregor@vecima.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-04 15:56:29 +00:00
Joshua Watt
fdce09a022 oeqa: reproducible: Do not strip packages
Do not strip packages when testing reproducible builds. In some cases,
stripped data differs between builds, but then gets removed. However,
the contents affect the generation of the GCC build-id, which then
differs in the resulting ELF files, even though the data that caused
this is no longer there. Inhibit stripping so that their causes can be
evaluated.

(From OE-Core rev: 0e23c6faf8169a333ae29064553705b5d7d6b589)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-03 00:10:11 +00:00
Richard Purdie
c404b88959 reproducible: Allow configuration of saved debug output
If OEQA_DEBUGGING_SAVED_OUTPUT is set in the environment, use this location
to store reproducibile build failure output. This aids debugging on the
YP autobuilder in particular.

Use a date in the directory name to make it easier to find failure output.

Also clean up empty directories as they're unnecessary distracting noise.

(From OE-Core rev: 138ed4aa96ec5069f9f7fd02994a42452dbccc2d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-02 16:53:02 +00:00
Armin Kuster
55bd462940 parselogs: remove mpc8315e-rdb from errors
(From OE-Core rev: 0de55be071efff60944c89582ae0e736f3bdc828)

Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-02 10:39:52 +00:00
Armin Kuster
c932bf9e86 bsp-hw.json: remove mpc8315e-rdb manual tests
(From OE-Core rev: 72af20f445dfd8ac0b472f80ed84e1b1c556e8ef)

Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-02 10:39:52 +00:00
Richard Purdie
b1fe4aab8b distrodata/maintainers: Add PARSE_ALL_RECIPES flag to fix test issues
Currently the test depends on which DISTRO_FEATURES and targets are selected.
Similar to SOURCE_MIRROR_FETCH, add PARSE_ALL_RECIPES to allow the test to see
a much wider range of them. This avoids issues added by the new ucontext musl
specific recipe and allows the current whitelist to be cleared out a bit (more
cleanup can follow).

(From OE-Core rev: c139e058bb2df48c79784cf3fd701e51bbc68427)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-27 16:48:10 +00:00
Jean-Marie LEMETAYER
22dd46cc34 oeqa/selftest/devtool: add npm recipe build test
This commit adds a devtool build test for npm recipe:

 - devtool.DevtoolAddTests.test_devtool_add_npm

(From OE-Core rev: af299817c2fe9fa504762c54bd0593f3163399fb)

Signed-off-by: Jean-Marie LEMETAYER <jean-marie.lemetayer@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-27 16:48:09 +00:00
Jean-Marie LEMETAYER
44b2ab8d5e oeqa/selftest/recipetool: add npm recipe creation test
This commit adds a recipetool creation test for npm recipe:

 - recipetool.RecipetoolTests.test_recipetool_create_npm

(From OE-Core rev: 61cabc2ccaa6a972b799d19fda8964926cc7ecec)

Signed-off-by: Jean-Marie LEMETAYER <jean-marie.lemetayer@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-27 16:48:09 +00:00
Jean-Marie LEMETAYER
3b0640c993 lib/oe/package: remove unneeded npm_split_package_dirs function
The npm_split_package_dirs function was used by the recipetool when
creating npm recipes. This is not the case anymore.

(From OE-Core rev: 6cd834737eaa42592e83300099c152e2cfef568c)

Signed-off-by: Jean-Marie LEMETAYER <jean-marie.lemetayer@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-27 16:48:09 +00:00
Alex Kiernan
fd983f2e9b lib/oe/package_manager: Improve locale-archive reproducibility
The generation of locale-archive depends on the order of the input
files. Fix the order by sorting the file list.

(From OE-Core rev: 3f781ebd9dfbee143907190c6db5b86d501c3e63)

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-22 15:56:14 +00:00
Richard Purdie
9f225cf966 oeqa/reproducible: Fix unset save_dir variable
Previous refactoring broke the case where save_dir was set. Fix this.

(From OE-Core rev: 9bfa04aa1f1a0b38ffa51f0211f3a46a90e17b0a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-22 15:56:14 +00:00
Joshua Watt
1d65844ab0 oeqa/reproducible: Fix extra data reporting
A typo was preventing the extra data about the reproducible build from
being reported in the test results

(From OE-Core rev: 7c653337360dd5e49809b930f22a999c1d6e09ee)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-22 15:56:14 +00:00
Joshua Watt
5af3b55369 oeqa/reproducible: Add flag for building from sstate
Adds a flag to control if the reproducible QA test should allow building
from sstate or not. Building from sstate may not be reproducible
depending on how the sstate is populated.

(From OE-Core rev: c24b71e9ec2e518bd24da20ddbb1406910b1b92e)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-22 15:56:14 +00:00
Alexander Kanavin
9896be8f75 selftest: check maintainers.inc for entries without recipes
Also remove a couple of entries found by the test :)

(From OE-Core rev: 749f44b3735e4ae3657255b373fa55c357501cc5)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-21 12:52:53 +00:00
Alejandro Enedino Hernandez Samaniego
0f9ba4d7a0 machine_dict: Add i686 to the ELF machine dictionary
An error like the following is thrown when building
baremetal applications on some x86 architectures:

  (machine, osabi, abiversion, littleendian, bits) \
    = oe.elf.machine_dict(d)[target_os][target_arch]

Exception: KeyError: i686

Since the i686 (target_arch) key does not exist in the dictionary.

Add the key to fix the error.

(From OE-Core rev: e7862d2cdd478556ec0310d2b3c140da9cb2ff0b)

Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandro@enedino.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-19 23:49:39 +00:00
Khem Raj
bf2b73e1f3 classes, conf, lib: Add support for powerpc64le
LE is default for modern powerpc64, power8+

PowerPC64 Little Endian Linux ABI specifies Power8 as the minimum ISA.
The basic ABI can run on earlier versions of the 64 bit PowerPC ISA,
but it was helpful to define a new, minimum instruction set for Linux
distribution releases during the switch to Little Endian.

(From OE-Core rev: b9c73fb6d1afb3367d871a3d6bf7d0d6a53968a9)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-19 23:49:39 +00:00
Richard Purdie
ecbf203e34 sstatesig: Improve debug output if getpwuid() fails
If getpwduid fails, we don't see which file it failed on which is key information
to aid debugging. Print this information when exceptions are raised.

(From OE-Core rev: 9d7a5219713af8117eda145052c6d9abdf577d8f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-16 22:33:09 +00:00
Alexander Kanavin
f0028da1e6 ltp: update to 20190930
Drop a big pile of backports.

Add a patch to address absent string formatting error.

Additional fixing of builds with musl.

timers group has been merged upstream into syscalls.

(From OE-Core rev: 7126431cc3512e3d7f55300c561339c87283b605)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-16 22:33:09 +00:00
Peter Kjellerstedt
2f90c165aa sanity.bbclass: Move sanity_info from conf to cache
Since this file is written during recipe parsing, having it in the
${BUILDDIR}/conf directory, which is covered by an inotify watcher,
will trigger a re-parse the next time bitbake is run and the resident
bitbake server is enabled. This causes the sanity_info file to be
updated again, which triggers a new parse the next time bitbake is run
ad infinitum. Moving it to ${BUILDDIR}/cache should avoid this.

(From OE-Core rev: f98103b548aa7dba6b1be6c8e02ef41858a8e85c)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-10 21:18:22 +00:00
Paul Barker
26cc774954 oeqa: archiver: Tidy up test case descriptions
(From OE-Core rev: 8f9119cb1451ef6337b98e4bb566069aa8d66a35)

Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-10 21:18:22 +00:00
Paul Barker
61e1b4136f archiver.bbclass: Add new mirror archiver mode
We define a new method of populating a source mirror using the archiver
bbclass instead of simply copying the contents of the downloads
directory. This allows the archiver features such as copyleft license
filtering and recipe type filtering to be used when preparing a source
mirror.

This new archiver mode is selected by setting `ARCHIVE_MODE[src]` to
'mirror'.

The source mirror mode can either be 'split' (default) or 'combined',
controlled by `ARCHIVER_MODE[mirror]`. Additionally, sources can be
excluded as needed by setting `ARCHIVER_MIRROR_EXCLUDE` to a list of
URI prefixes. These options are described in more detail in the new
entries in the header of archiver.bbclass.

New oeqa selftest cases are added to cover the mirror archiver mode.

(From OE-Core rev: 2c8b31ae0ab95a8b100e8bade23f51574e273c9a)

Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-10 21:18:22 +00:00
Paul Barker
2c81446248 wic: Add --include-path argument
This option adds the contents of the given path to a partition built
with the rootfs source plugin. The path is relative to the directory in
which wic is running not the rootfs itself so use of an absolute path
is recommended. This option is most useful when multiple copies of the
rootfs are added to an image and it is required to add extra content to
only one of these copies. This option only has an effect with the
rootfs source plugin.

(From OE-Core rev: d4cd27a9837426e809190548a83c6c7c76505114)

Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-10 21:18:22 +00:00
Alexander Kanavin
544cad3791 ptest: report ptests that couldn't be run at all
Currently if a ptest does not produce PASS or FAIL, but simply
errors out, this is not caught or reported; I think some ptests
may have silently regressed due to this.

(From OE-Core rev: 4b78b6571a3653d7d8e9dab9512bad5d39483988)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-10 21:18:21 +00:00
Alexander Kanavin
25fd2bf815 lib/oe/package_manager.py: put the sdk_provides_dummy_target upfront
Otherwise, the package utilizing this architecure does not get
properly prioritized by rpm/dnf. This isn't seen single-lib builds
(as the architecture ends up upfront anyway), but is triggered in
multulib where the architecture may end up in the middle of the list
due to multilib expansion/trimming duplicates.

(From OE-Core rev: a807d54655d7377ef8ac49abd5ab428303831663)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-10 21:18:21 +00:00
Richard Purdie
4cb1b4b409 sstate: Add extra directory level
We're having speed issues on the autobuilder due to the numbers of files in sstate
directories. We previously split these by the first two characters of the hash.
This change extends this to split by the next two characters as well, creating
more layers of directories.

This should signifiantly speed up eSDK builds on the autobuilder as the current
sstate layout simply isn't scaling there but addresses a general complaint.

gen-lockedsig-cache needed to be updated for the new split level sstate.

Also update tests for new layout.

(From OE-Core rev: d05bde16bdad761ed8f4c0a48de60c649aa33e85)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-06 23:54:39 +00:00
Richard Purdie
76af397213 sstatesig: Avoid resetting taskhash within siggen for locked sigs
Since get_unihash uses taskhash as a key internally, changing it means
different bebahour when locked sigs are active verses not active. Under
corner cases this leads to a signature mismatch.

Avoid this by by adding a wrapper for the place its externally exposed
and then not changing the internals.

(From OE-Core rev: 23b7484d326cf7fac013384598d7745a042eaa75)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-03 22:35:48 +00:00
Richard Purdie
737c5b27bb sstatesig: Fix locked signature handling with unihashes
get_taskhash will call get_unihash internally in the parent class. We
need to disable our filter of it whilst this runs else incorrect hashes
can be calculated.

This is believed to be causing the locked signatures test to fail under
some circumstances (depending on whether earlier hashes are being
remapped).

[YOCTO #13605]

(From OE-Core rev: 523c093a882f6831ba75b5c4513837554d7e2414)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-03 22:35:48 +00:00