Commit Graph

2545 Commits

Author SHA1 Message Date
Joshua Watt
d59bfa1334 oeqa: reproducible: Run diffoscope on saved output
If there are differing packages and they are being saved for review,
automatically run diffoscope on them and include the output in the saved
output. The output is currently done in HTML format since these are
typically published on a webpage by the autobuilder.

(From OE-Core rev: 6e1d5e8b58f0940ba6dfd99536159dd974e6f24c)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-14 13:07:23 +00:00
Khem Raj
21d97d00d5 oeqa: Use --disable-maintainer-mode configure option
since the versions of autotools might differ on target and build host, plus difference in
timestamps for configure and system can result in reconfigure lets avoid
that by disabling maintainer mode

Avoids
error: newly created file is older than distributed files!

[YOCTO #13779]

Suggested-by: Andre McCurdy <armccurdy@gmail.com>
(From OE-Core rev: 3dc4e27e7633ce3ca6b9647810d0996bdee48771)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-14 13:07:23 +00:00
Alejandro Hernandez Samaniego
a3416a5933 testimage: Extend runtime testing infrastructure to allow unconventional booting processes to be tested
The current runtime infrastructure contains hardcoded values which Ill refer to
as patterns, these patterns are either searched through or sent via the serial
terminal to communicate between HOST and TARGET.

These patterns are required since they allow us to check when a device has
finished booting, to log in, and to check whether a command sent from our tests
has returned, this way we are able to check both the status of the commands that
were sent along with its output.

The testing process goes somewhat as follows:
1. Launch QEMU and start booting.
2. Check when the device has booted by looking for the pattern login:.
3. Log in as the root user (default for our images).
4. Check that we were able to log in succesfully.
5. Start running the runtime test cases defined by TEST_SUITES.
6. One of such test cases could send a command to the QEMU target.
7. Check whether that command returned.
8. Check its output and status, return whether the test case passed or failed.

This patch allows this set of patterns to be defined instead of being hardcoded,
but it also automatically sets the defaults that we have been using in the past
if they have not been manually defined, for this reason, the patch is less
invasive and should not affect in any way how tests are currently being run.

Cases that can be enabled with this patch:
- A customized image that does not use the root user (or maybe we want to check
what happens if we dont use the root user).
- An image where the PS1 env variable has been modified, and the prompt pattern
wouldnt match the default.
- Baremetal applications, which do not follow the conventional way of booting
Linux and would probably not show a prompt for a user to log in, same applies
for testing bootloaders.
- poky-tiny: Using DISTRO=poky-tiny and an image such as the core-image-tiny
from meta-intel, which boots directly to RAM, and does not show a log in prompt
since it does not contain a conventional init process.

The code itself contains comments that should be self explanatory but here is an
example on how these patterns can be defined in a hypothetical case where we
want to run test cases as the webserver user instead:

TESTIMAGE_BOOT_PATTERNS = "send_login_user search_login_succeeded"
TESTIMAGE_BOOT_PATTERNS[send_login_user] = "webserver\n"
TESTIMAGE_BOOT_PATTERNS[search_login_succeeded] = "webserver@[a-zA-Z0-9\-]+:~#"

The variable TESTIMAGE_BOOT_PATTERNS defines which patterns to override when
used to communicate with the target when booting, anyone familiar with the
PACKAGECONFIG syntax should have no trouble setting these.

Other patterns would still be set up as default, e.g.
search_reached_prompt would still be login:

The accepted flags for TESTIMAGE_BOOT_PATTERNS are the following:
search_reached_prompt, send_login_user, search_login_succeeded,
search_cmd_finished.

They are prefixed with either search/send, to differentiate if the pattern is
meant to be sent or searched to/from the target terminal.

A working example of this code that falls under the baremetal case mentioned
above along with a test case is present on the meta-freertos layer, which tests
an RTOS image built with OpenEmbedded and automatically runs a test case on it
after booting such image:

As usual, INHERIT += "testimage" needs to be present on local.conf
$ bitbake freertos-demo -c testimage

RESULTS:
RESULTS - freertos_echo.FreeRTOSTest.test_freertos_echo: PASSED (2.00s)
SUMMARY:
freertos-demo () - Ran 1 test in 2.006s
freertos-demo - OK - All required tests passed (successes=1, skipped=0,
failures=0, errors=0)

(From OE-Core rev: 3ab2cbfeff371e8791b031a2852eeef80101a831)

Signed-off-by: Alejandro Hernandez Samaniego <aehs29@gmail.com>
Signed-off-by: Alejandro Hernandez Samaniego <alejandro@enedino.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-13 12:19:14 +00:00
Alejandro Hernandez Samaniego
e597809307 testimage: Allow testing on QEMU machines with a single serial port
commmit 6cde67d0a8 enables the use of qemurunner on machines that
only have a single serial port, but still sets the default value
as serial_ports=2 if not provided.

The testimage class does not call qemurunner with a serial_ports
argument, hence always defaulting to two.

Pass the serial_ports argument from the testimage class to allow
tests to run on QEMU machines with a single serial port.

(From OE-Core rev: a50d0163770f0b405a8de8a8a9cccd48c1de4112)

Signed-off-by: Alejandro Hernandez Samaniego <alejandro@enedino.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-13 12:19:14 +00:00
Trevor Gamblin
9afefb02b0 qemurunner.py: add try/except for pid handling race
In some instances, attempts to remove the qemu pidfile within the
stop() method fail despite the os.path.exists() call immediately
before implying that the file is present. Add a try/except block
to log a warning if this occurs, rather than failing outright,
since the process simply appears to be exiting at an inconvenient
time.

[YOCTO #13675]

(From OE-Core rev: eadb899e23b18eb9eaff145c3bf5b20fb417c3e8)

Signed-off-by: Trevor Gamblin <trevor.gamblin@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:02 +00:00
Khem Raj
504b206e37 oeqa: Use cpio 2.13 as testcase
cpio 2.12 was released in 2015 and might have used older autotools
which could result in errors like

https://bugzilla.yoctoproject.org/show_bug.cgi?id=13779

Bumping to 2.13 will help in matching the tool versions
A good change on top would be to run

aclocal -I .; autoheader; autoconf; automake --add-missing -c

before running configure step perhaps

[YOCTO #13779]

(From OE-Core rev: 84eb1dc4fe8a11cd2d05b703070a6fb6de05b873)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:02 +00:00
Mark Hatle
cdcacf26ec microblaze: Adjust Linux items from microblazeeb to microblaze
Due to recent changes to the tune, in order to match config.guess, the name
of the big-endian microblaze architecture was changes to 'microblaze'.

(From OE-Core rev: 6f6a6bbac684ead3fe6d070d61f17c2f611a2c87)

Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-06 12:16:34 +00:00
Richard Purdie
7c201279cb oeqa/reproducible: Improve test output and ensure deb+ipk compared
Adding newline characters between the packages in the failure output
massively improves readability.

Also ensure to output ipk failures when there are deb failures by
calling self.fail() at the end, else sometimes only partial differences
are returned.

(From OE-Core rev: 6e2e0480852177db75a6108d77c99c92c4e9950f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-06 12:16:34 +00:00
Trevor Gamblin
b486903ccc logrotate.py: improve oeqa test implementation
See bug https://bugzilla.yoctoproject.org/show_bug.cgi?id=13632

Autobuilder tests occasionally fail, reporting that a new logfile
could not be created. While this failure did occur multiple times, it
could not be manually reproduced. However, there are issues with the
implementation of the logrotate.py script that can be fixed. These
changes will help make the failures clearer, should they continue to
occur.

Previously, the test_2_logrotate test would, after running the
logrotate tool, use "ls -al $HOME/logrotate_dir | wc -l" to count
the number of files in the rotation directory and determine if the
rotation was successful. The test to see if there are at least three
files is problematic, because depending on the version of ls used, it
may report the target value of 3 even when there are only hidden files
in the directory, potentially reporting a pass for the test when it
should actually fail. An example with coreutils:

root@qemux86-64:~# ls -al emptydir/
total 2
drwxr-xr-x 2 root root 1024 Jan 14 19:50 .
drwx------ 3 root root 1024 Jan 14 19:50 ..
root@qemux86-64:~#

Where "total" is the number of blocks used. Compare with busybox ls:

root@qemux86-64:~# ls -al emptydir/
drwxr-xr-x    2 root     root          1024 Jan 14 19:54 .
drwx------    3 root     root          1024 Jan 14 19:54 ..
root@qemux86-64:~#

Instead of using ls to verify that a certain number of files exists
in $HOME/logrotate_dir, the tests have been changed to rotate two
specific logs: the log for wtmp and a new logrotate_testfile created
during the second test. Both tests check that the logs are correctly
rotated into $HOME/logrotate_dir by using find and grep on the
expected filename (e.g. "wtmp" when rotated becomes "wtmp.1", so we
check to see that wtmp.1 is present in $HOME/logrotate_dir). In
addition, should the test fail, the output from logrotate -vf is
included in the test log to aid debugging. It has also been seen that
in some cases, the logrotate test fails because the /var/log/wtmp file
is not yet present. Since the objective of the test is to check the
logrotate functionality and not the presence of certain log files,
test_logrotate_wtmp uses the touch command to help ensure that the
file is present before the call to logrotate is issued.

Finally, note that while the autobuilder failures that this patch
addresses were only seen during core-image-full-cmdline tests, these
changes were successfully tested on core-image-minimal and
core-image-sato with the manual addition of logrotate and openssh-sshd
to the images.

(From OE-Core rev: b9392403e96c6d880de844b44da2c7300fcf411c)

Signed-off-by: Trevor Gamblin <trevor.gamblin@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-05 12:00:34 +00:00
André Draszik
e4893d4830 oeqa: modernise subprocess: preexec_fn=os.setsid -> start_new_session=True
start_new_session was added to python3 subprocess in v3.2 and
is meant to take the place of the common use of preexec_fn to
call os.setsid() in the child - as done here.

Update to use the new equivalent.

(From OE-Core rev: c3bd502f31a621394a2f6ea3ae90443ff4ac98a4)

Signed-off-by: André Draszik <git@andred.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-04 15:56:29 +00:00
Henning Schild
d02d938cc6 lib/oe/path: try hardlinking instead of guessing when it might fail
The comparison of the stat st_dev is not enough to judge whether
hardlinking will work. One example would be where you try and hardlink
across two bind-mounts of a directory. The st_dev will be the same and
the operation will still fail.

Instead of implementing a check to try and figure out hardlink support
just try hardlinking and fall back to a copy when running into an
exception.

(From OE-Core rev: f5571bda8327f927feb23b167ab4594b7d0c95bc)

Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-04 15:56:29 +00:00
Daniel McGregor
a26efa2875 cmake: prefer CMAKE_BUILD_PARALLEL_LEVEL
cmake 3.12 introduced this environment variable. Prefer it to passing
PARALLEL_MAKE and PARALLEL_MAKEINST on the cmake command line, because
it gets passed to second stage cmake invocations while command-line
arguments do not (for example, multi-stage clang builds)

(From OE-Core rev: cdd44c93f02bb8cc2fa773e13c8ce36e3da23921)

Signed-off-by: Daniel McGregor <daniel.mcgregor@vecima.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-04 15:56:29 +00:00
Joshua Watt
fdce09a022 oeqa: reproducible: Do not strip packages
Do not strip packages when testing reproducible builds. In some cases,
stripped data differs between builds, but then gets removed. However,
the contents affect the generation of the GCC build-id, which then
differs in the resulting ELF files, even though the data that caused
this is no longer there. Inhibit stripping so that their causes can be
evaluated.

(From OE-Core rev: 0e23c6faf8169a333ae29064553705b5d7d6b589)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-03 00:10:11 +00:00
Richard Purdie
c404b88959 reproducible: Allow configuration of saved debug output
If OEQA_DEBUGGING_SAVED_OUTPUT is set in the environment, use this location
to store reproducibile build failure output. This aids debugging on the
YP autobuilder in particular.

Use a date in the directory name to make it easier to find failure output.

Also clean up empty directories as they're unnecessary distracting noise.

(From OE-Core rev: 138ed4aa96ec5069f9f7fd02994a42452dbccc2d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-02 16:53:02 +00:00
Armin Kuster
55bd462940 parselogs: remove mpc8315e-rdb from errors
(From OE-Core rev: 0de55be071efff60944c89582ae0e736f3bdc828)

Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-02 10:39:52 +00:00
Armin Kuster
c932bf9e86 bsp-hw.json: remove mpc8315e-rdb manual tests
(From OE-Core rev: 72af20f445dfd8ac0b472f80ed84e1b1c556e8ef)

Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-02 10:39:52 +00:00
Richard Purdie
b1fe4aab8b distrodata/maintainers: Add PARSE_ALL_RECIPES flag to fix test issues
Currently the test depends on which DISTRO_FEATURES and targets are selected.
Similar to SOURCE_MIRROR_FETCH, add PARSE_ALL_RECIPES to allow the test to see
a much wider range of them. This avoids issues added by the new ucontext musl
specific recipe and allows the current whitelist to be cleared out a bit (more
cleanup can follow).

(From OE-Core rev: c139e058bb2df48c79784cf3fd701e51bbc68427)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-27 16:48:10 +00:00
Jean-Marie LEMETAYER
22dd46cc34 oeqa/selftest/devtool: add npm recipe build test
This commit adds a devtool build test for npm recipe:

 - devtool.DevtoolAddTests.test_devtool_add_npm

(From OE-Core rev: af299817c2fe9fa504762c54bd0593f3163399fb)

Signed-off-by: Jean-Marie LEMETAYER <jean-marie.lemetayer@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-27 16:48:09 +00:00
Jean-Marie LEMETAYER
44b2ab8d5e oeqa/selftest/recipetool: add npm recipe creation test
This commit adds a recipetool creation test for npm recipe:

 - recipetool.RecipetoolTests.test_recipetool_create_npm

(From OE-Core rev: 61cabc2ccaa6a972b799d19fda8964926cc7ecec)

Signed-off-by: Jean-Marie LEMETAYER <jean-marie.lemetayer@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-27 16:48:09 +00:00
Jean-Marie LEMETAYER
3b0640c993 lib/oe/package: remove unneeded npm_split_package_dirs function
The npm_split_package_dirs function was used by the recipetool when
creating npm recipes. This is not the case anymore.

(From OE-Core rev: 6cd834737eaa42592e83300099c152e2cfef568c)

Signed-off-by: Jean-Marie LEMETAYER <jean-marie.lemetayer@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-27 16:48:09 +00:00
Alex Kiernan
fd983f2e9b lib/oe/package_manager: Improve locale-archive reproducibility
The generation of locale-archive depends on the order of the input
files. Fix the order by sorting the file list.

(From OE-Core rev: 3f781ebd9dfbee143907190c6db5b86d501c3e63)

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-22 15:56:14 +00:00
Richard Purdie
9f225cf966 oeqa/reproducible: Fix unset save_dir variable
Previous refactoring broke the case where save_dir was set. Fix this.

(From OE-Core rev: 9bfa04aa1f1a0b38ffa51f0211f3a46a90e17b0a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-22 15:56:14 +00:00
Joshua Watt
1d65844ab0 oeqa/reproducible: Fix extra data reporting
A typo was preventing the extra data about the reproducible build from
being reported in the test results

(From OE-Core rev: 7c653337360dd5e49809b930f22a999c1d6e09ee)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-22 15:56:14 +00:00
Joshua Watt
5af3b55369 oeqa/reproducible: Add flag for building from sstate
Adds a flag to control if the reproducible QA test should allow building
from sstate or not. Building from sstate may not be reproducible
depending on how the sstate is populated.

(From OE-Core rev: c24b71e9ec2e518bd24da20ddbb1406910b1b92e)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-22 15:56:14 +00:00
Alexander Kanavin
9896be8f75 selftest: check maintainers.inc for entries without recipes
Also remove a couple of entries found by the test :)

(From OE-Core rev: 749f44b3735e4ae3657255b373fa55c357501cc5)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-21 12:52:53 +00:00
Alejandro Enedino Hernandez Samaniego
0f9ba4d7a0 machine_dict: Add i686 to the ELF machine dictionary
An error like the following is thrown when building
baremetal applications on some x86 architectures:

  (machine, osabi, abiversion, littleendian, bits) \
    = oe.elf.machine_dict(d)[target_os][target_arch]

Exception: KeyError: i686

Since the i686 (target_arch) key does not exist in the dictionary.

Add the key to fix the error.

(From OE-Core rev: e7862d2cdd478556ec0310d2b3c140da9cb2ff0b)

Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandro@enedino.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-19 23:49:39 +00:00
Khem Raj
bf2b73e1f3 classes, conf, lib: Add support for powerpc64le
LE is default for modern powerpc64, power8+

PowerPC64 Little Endian Linux ABI specifies Power8 as the minimum ISA.
The basic ABI can run on earlier versions of the 64 bit PowerPC ISA,
but it was helpful to define a new, minimum instruction set for Linux
distribution releases during the switch to Little Endian.

(From OE-Core rev: b9c73fb6d1afb3367d871a3d6bf7d0d6a53968a9)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-19 23:49:39 +00:00
Richard Purdie
ecbf203e34 sstatesig: Improve debug output if getpwuid() fails
If getpwduid fails, we don't see which file it failed on which is key information
to aid debugging. Print this information when exceptions are raised.

(From OE-Core rev: 9d7a5219713af8117eda145052c6d9abdf577d8f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-16 22:33:09 +00:00
Alexander Kanavin
f0028da1e6 ltp: update to 20190930
Drop a big pile of backports.

Add a patch to address absent string formatting error.

Additional fixing of builds with musl.

timers group has been merged upstream into syscalls.

(From OE-Core rev: 7126431cc3512e3d7f55300c561339c87283b605)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-16 22:33:09 +00:00
Peter Kjellerstedt
2f90c165aa sanity.bbclass: Move sanity_info from conf to cache
Since this file is written during recipe parsing, having it in the
${BUILDDIR}/conf directory, which is covered by an inotify watcher,
will trigger a re-parse the next time bitbake is run and the resident
bitbake server is enabled. This causes the sanity_info file to be
updated again, which triggers a new parse the next time bitbake is run
ad infinitum. Moving it to ${BUILDDIR}/cache should avoid this.

(From OE-Core rev: f98103b548aa7dba6b1be6c8e02ef41858a8e85c)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-10 21:18:22 +00:00
Paul Barker
26cc774954 oeqa: archiver: Tidy up test case descriptions
(From OE-Core rev: 8f9119cb1451ef6337b98e4bb566069aa8d66a35)

Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-10 21:18:22 +00:00
Paul Barker
61e1b4136f archiver.bbclass: Add new mirror archiver mode
We define a new method of populating a source mirror using the archiver
bbclass instead of simply copying the contents of the downloads
directory. This allows the archiver features such as copyleft license
filtering and recipe type filtering to be used when preparing a source
mirror.

This new archiver mode is selected by setting `ARCHIVE_MODE[src]` to
'mirror'.

The source mirror mode can either be 'split' (default) or 'combined',
controlled by `ARCHIVER_MODE[mirror]`. Additionally, sources can be
excluded as needed by setting `ARCHIVER_MIRROR_EXCLUDE` to a list of
URI prefixes. These options are described in more detail in the new
entries in the header of archiver.bbclass.

New oeqa selftest cases are added to cover the mirror archiver mode.

(From OE-Core rev: 2c8b31ae0ab95a8b100e8bade23f51574e273c9a)

Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-10 21:18:22 +00:00
Paul Barker
2c81446248 wic: Add --include-path argument
This option adds the contents of the given path to a partition built
with the rootfs source plugin. The path is relative to the directory in
which wic is running not the rootfs itself so use of an absolute path
is recommended. This option is most useful when multiple copies of the
rootfs are added to an image and it is required to add extra content to
only one of these copies. This option only has an effect with the
rootfs source plugin.

(From OE-Core rev: d4cd27a9837426e809190548a83c6c7c76505114)

Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-10 21:18:22 +00:00
Alexander Kanavin
544cad3791 ptest: report ptests that couldn't be run at all
Currently if a ptest does not produce PASS or FAIL, but simply
errors out, this is not caught or reported; I think some ptests
may have silently regressed due to this.

(From OE-Core rev: 4b78b6571a3653d7d8e9dab9512bad5d39483988)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-10 21:18:21 +00:00
Alexander Kanavin
25fd2bf815 lib/oe/package_manager.py: put the sdk_provides_dummy_target upfront
Otherwise, the package utilizing this architecure does not get
properly prioritized by rpm/dnf. This isn't seen single-lib builds
(as the architecture ends up upfront anyway), but is triggered in
multulib where the architecture may end up in the middle of the list
due to multilib expansion/trimming duplicates.

(From OE-Core rev: a807d54655d7377ef8ac49abd5ab428303831663)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-10 21:18:21 +00:00
Richard Purdie
4cb1b4b409 sstate: Add extra directory level
We're having speed issues on the autobuilder due to the numbers of files in sstate
directories. We previously split these by the first two characters of the hash.
This change extends this to split by the next two characters as well, creating
more layers of directories.

This should signifiantly speed up eSDK builds on the autobuilder as the current
sstate layout simply isn't scaling there but addresses a general complaint.

gen-lockedsig-cache needed to be updated for the new split level sstate.

Also update tests for new layout.

(From OE-Core rev: d05bde16bdad761ed8f4c0a48de60c649aa33e85)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-06 23:54:39 +00:00
Richard Purdie
76af397213 sstatesig: Avoid resetting taskhash within siggen for locked sigs
Since get_unihash uses taskhash as a key internally, changing it means
different bebahour when locked sigs are active verses not active. Under
corner cases this leads to a signature mismatch.

Avoid this by by adding a wrapper for the place its externally exposed
and then not changing the internals.

(From OE-Core rev: 23b7484d326cf7fac013384598d7745a042eaa75)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-03 22:35:48 +00:00
Richard Purdie
737c5b27bb sstatesig: Fix locked signature handling with unihashes
get_taskhash will call get_unihash internally in the parent class. We
need to disable our filter of it whilst this runs else incorrect hashes
can be calculated.

This is believed to be causing the locked signatures test to fail under
some circumstances (depending on whether earlier hashes are being
remapped).

[YOCTO #13605]

(From OE-Core rev: 523c093a882f6831ba75b5c4513837554d7e2414)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-03 22:35:48 +00:00
Richard Purdie
7538be0a50 sstatesig: Test cross/native hashserv method extension
We can have one taskhash which represents multiple native/cross sstate objects
since they're stored by BUILD_ARCH or possibly host distro (or host gcc version).
We need to put these into separate namespaces on hashserv since their outhashes
will never match and we need deterministic lookups for the different namespaces.

Use this extramethod option to handle this. This fixes several problematic
failures on the autobuilder.

(From OE-Core rev: 2a76082363d189880613765ad339718e3614049d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-02 16:43:01 +00:00
Mike Crowe
c1ad30fd80 multilib.conf: Ensure that RECIPE_SYSROOT is unchanged for native
Ensure that RECIPE_SYSROOT is the same for -native recipes whether
multilib.conf is included or not.

Without this change task signatures for -native recipes change when
switching between MACHINEs that require multilib.conf and those that
don't.

This fix was one of the ones suggested by Khem Raj in
http://lists.openembedded.org/pipermail/openembedded-core/2019-December/290303.html

Add test_sstate_multilib_or_not_native_samesigs test case to
sstatetests.py to ensure that this stays fixed.

(From OE-Core rev: aa05f1ded71366b86eda7fce24d8b5395e85ada2)

Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-30 23:39:10 +00:00
André Draszik
c6743c05a3 oeqa/runtime/ptest: fix detection of failed tests
Since commit d6065f136f6d ("oeqa/logparser: Various misc cleanups"),
7b17274c30 in poky, the ptest OEQA is unable to detect failures
in any of the test results.

The reason is that the test result string changed from 'fail' to
'FAILED', because the original mapping has been removed as part of
that commit, but the code in here is still trying to match against
the old string, resulting in no matches, i.e. everything is treated
as successful, even if it shouldn't be.

Update the OEQA ptest test to actually work again and report
failure if there was a failure.

Note that the ptest test is marked as @expectedfail, so even though
this test now again starts to fail, the overall OEQA test result is
not affected - but at least the overall OEQA test summary reflects
the correct status again.

In other words:
    RESULTS:
    RESULTS - ping.PingTest.test_ping: PASSED (0.26s)
    RESULTS - ptest.PtestRunnerTest.test_ptestrunner: PASSED (4.05s)
    RESULTS - ssh.SSHTest.test_ssh: PASSED (0.60s)
    SUMMARY:
    image-debug () - Ran 3 tests in 4.937s
correctly changes to:
    AssertionError: Failed ptests:
    {'dummytest': ['check_True_is_True', 'test_basic']}

    RESULTS:
    RESULTS - ping.PingTest.test_ping: PASSED (0.24s)
    RESULTS - ssh.SSHTest.test_ssh: PASSED (0.56s)
    RESULTS - ptest.PtestRunnerTest.test_ptestrunner: EXPECTEDFAIL (4.13s)
    SUMMARY:
    image-debug () - Ran 3 tests in 4.937s
instead and we see a summary of the ptest subtests that failed.

(From OE-Core rev: 08ff4b9195a9de91b8090173c5bd03a5ff263616)

Signed-off-by: André Draszik <git@andred.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-30 08:47:13 +00:00
Ross Burton
3780744968 chrpath: Cleanup and fix previous patch
Ensure self.data isn't accessed without assignment. Also clean up old style
popen use and replace with modern/simpler subprocess.

(From OE-Core rev: 39825cba4761a6b4b2473825705975f9f421ec8b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-29 09:27:16 +00:00
Alexander Kanavin
40f28d15c5 recipeutils: set SRCPV for devupstream variant
Otherwise bitbake is unable to expand it if the main
variant is not using a SCM fetcher.

(From OE-Core rev: 24e59094de42b4b0bed3584ca8571c4c19080428)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-28 23:25:42 +00:00
Ross Burton
1ac2b6c82f rootfs: don't use oe.cachedpath
Unless cachedpath is used correctly then it's just a glorified clone of
os.walk, but without any of the recent optimisations in os.walk.  In
this codepath there is no point to using cachedpath.

(From OE-Core rev: 765e0c3410d6bf3b4a50c4a036f555eae760acd2)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-28 23:25:41 +00:00
Ross Burton
42fe4c9068 oeqa/selftest/package: mark string as raw
This regex uses \ so is actually parsed incorrectly, mark it up as a raw
string.

(From OE-Core rev: 120e687bad03ad2cf4df4092e8a3f35a569e09f4)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-28 23:25:41 +00:00
Joshua Watt
6bb4a25219 oeqa: reproducible: Test core-image-sato and core-image-full-cmdline
Adds core-image-sato and core-image-full-cmdline to the list of images
that the reproducible build test builds.

(From OE-Core rev: d6c62cc0305ab082e990041da104fedb5798e0e4)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-18 10:18:06 +00:00
Alexander Kanavin
06a0adbb25 sysklogd: update to 2.0.3
Drop all patches as they are no longer necessary with autotools,
fix problems that are addressed upstream, or (in case of no-vectorization.patch)
don't apply to the old version either, which means no one is using that hw target
anymore.

Remove custom systemd configs (provided by upstream), and custom
syslog config (also provided by upstream but not installed by default).

Adjust parselogs to not error out on not being able to find
System.map file during boot (that is packaged into kernel-dev
and is not normally installed).

License-Update: various tweaks; the license terms are the same.
(From OE-Core rev: dcf0bd14d78b0848f206eb35355f05bfbd51c02d)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-18 10:18:06 +00:00
Peter Kjellerstedt
86f318e524 incompatible_lic.py: Add tests for incompatible licenses with wildcards
Suggested-by: Quentin Schulz <quentin.schulz@streamunlimited.com>
(From OE-Core rev: ef7c44c9bdf30a02ccc71f26c27aab45d6adf1fb)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-16 23:25:50 +00:00
Peter Kjellerstedt
29b3cc9655 license_image.bbclass: Report only the licenses that are incompatible
Instead of reporting ${LICENSE} when a package cannot be installed
into an image because it is using an incompatible license, report the
license(s) that are actually incompatible.

(From OE-Core rev: b1863e570d4b169cd2f0ea7b4fe7c2348943cb2c)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-16 23:25:50 +00:00
Peter Kjellerstedt
5560c50316 base.bbclass: Report only the licenses that are incompatible for a package
Instead of reporting ${LICENSE} when a package is identified as using
an incompatible license, report the license(s) that are actually
incompatible.

(From OE-Core rev: fb3405fac7b933a3f9d23e5faf6a3cf2d1990982)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-16 23:25:50 +00:00