Commit Graph

1769 Commits

Author SHA1 Message Date
Richard Purdie
10763296f8 oeqa/selftest/archiver: Allow tests to ignore empty directories
If we tweak sstate to not remove empty directories under conditions
where a race could occur, we see failures from:

"oe-selftest -r archiver.Archiver.test_archiver_filters_by_type archiver.Archiver.test_archiver_filters_by_type_and_name"

since an empty directory is left behind. Update the tests to ignore
empty directories.

(From OE-Core rev: c96bcf97272f243df14598c84a41097746884b65)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 10cda713faea9a348fd278137ac75e4a6d76a71c)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-10 22:46:20 +01:00
Richard Purdie
522be6c4bf oeqa/selftest/runcmd: Tweal test timeouts
Load on the autobuilder meant we see occasionaly timeout issues with these tests.
Slightly increase the test timeouts to better reflect the real world timings we
see.

[YOCTO #14262]

(From OE-Core rev: 102ac28cf41e36c5d619be87ebb33b4af32ec817)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit fccd2ade0e345625ed9a4b74a7431b000ce2214f)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-10 22:46:19 +01:00
Alexander Kanavin
de1578c825 selftest: do not hardcode /tmp/sdk
This races if there are several copies of the test running at the same
time.

[YOCTO #14438]

(From OE-Core rev: deab11848036941771f2b3dc5cdaee83395280b5)

(From OE-Core rev: 969c29c39b0ceb73ace615c478ca4544be803c9a)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit ea707e8726cd7012d101d02e69503b7c98bdaf3e)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-10 22:46:19 +01:00
Richard Purdie
94ecd96f90 oeqa/runtime/rpm: Drop log message counting test component
This test is flawed since multiple parts of the system can write to the log
and we obtain different numbers of log messages depending on factors we
can't control.

Drop the log testing component of the test.

[YOCTO #12465]

(From OE-Core rev: fb22cf1891fa8d057348c5270715c45ac3ab9e25)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 2ad815dbafda0b90f5164f05d22dbbc26cb53f13)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-03 16:30:56 +01:00
Richard Purdie
76c2624a1f oeqa/qemurunner: Improve handling of run_serial for shutdown commands
When running a shutdown command, the serial port can close without the
command returning. This is seen as the socket being readable but having
no data. Change the way this case is handled in the code to avoid
tracebacks.

(From OE-Core rev: 9c0b242856de519c58be179f82441a35fc635ad9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 396a3ba884820d040c91f7592daf20ac28c49b5d)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-20 12:36:42 +01:00
Richard Purdie
4f428303e0 oeqa/qemurunner: Fix binary vs str issue
The recent logging changes for qemurunner showed up as errors on the
autobuilder where decode couldn't be called on the returned string.
Since the code returns binary data, return b'' instead of '' to match
to avoid tracebacks.

One of these cases was newly added, copied from the other which has
been there for a long time, always broken.

(From OE-Core rev: 8f24a7b35861b6aec39bc8d589e090ea9816732c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit b8995b27db265b0a0b2d2ca595915f70f9f96e07)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-20 12:36:42 +01:00
Richard Purdie
0cbb701834 oeqa/qemurunner: Improve logging thread exit handling for qemu shutdown test
Rather than totally disabling the logging, inform it we're about to exit
so we can log messages over the exit cleanly too. This aids debugging. It
also avoids a race where the logging handler could still error whilst
shutting down.

Also remove a race window by notificing the handler of the shutdown
first, before triggering it. This removes a race window I watched in
local testing.

(From OE-Core rev: 57249316b6c66c5e17804e1b04f2d5cf0db92683)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 0e19f31a1005f94105e1cef252abfffcef2aafad)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-20 12:36:42 +01:00
Steve Sakoman
1e4d928f31 reproducible.py: add quilt-ptest and valgrind-ptest
Both don't seem to be reproducible with fedora 33

(From OE-Core rev: 55dc503f4ab33e2aa51a3a6e4003131e0b9355ff)

Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-13 22:10:06 +01:00
Alexander Kanavin
a51996d684 Revert "oeqa: Set LD_LIBRARY_PATH when executing native commands"
LD_LIBRARY_PATH leaks into host executables too, and breaks them
as they are not uninative-enabled. E.g. on ubuntu 18.04 trying
to run host bash with a sysroot that was built on Fedora 33:

akanavin@ubuntu1804-ty-3:/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/build-st-24341/tmp/work/x86_64-linux/gnupg-native/2.3.1-r0/recipe-sysroot-native$ LD_LIBRARY_PATH=./usr/lib /bin/bash
/bin/bash: ./usr/lib/libtinfo.so.5: no version information available (required by /bin/bash)
/bin/bash: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by ./usr/lib/libtinfo.so.5)

This was seen e.g. here:
https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/2090/steps/14/logs/stdio

(From OE-Core rev: a150f6fba94518509ee80869ffbfb973c1283e3b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 0e9850486b74a3de934527ca1077df001d3a8d22)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-13 22:10:06 +01:00
Alexander Kanavin
d2691d772d meta/lib/oeqa/core/tests/cases/timeout.py: add a testcase for the previous fix
This is the sequence that didn't properly operate:

- a test case that skips and isn't executed
- a second test case that is skipped via a dependency decorator, and sets a timeout
- a third test case that takes longer than the timeout from the second
test case

Without the fix, the timeout is not cleared, and the third test case is
erroneously aborted. With the fix, the timeout is cleared and the third
test case is able to complete.

(From OE-Core rev: 79d216817c16237a9d660fc7956bf782ff60c65a)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 54ef07a9aa1af8f41cfb9a4802929c918efc43c8)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-04 22:57:55 +01:00
Alexander Kanavin
7d86fc1d9f oeqa: tear down oeqa decorators if one of them raises an exception in setup
Some of the decorators need proper cleanup, such as OETimeout
which sets a signal handler that needs to be cleared via teardown.
If this is not done then the signal gets called later with unpredictable effects.

This can be seen if there's a test that is skipped via a decorator and sets a timeout
at the same time: the timeout isn't cleared, and is invoked later in a
completely unrelated context. The test case for this is added in the
next commit.

(From OE-Core rev: d004c39ae70077c2c6f59afcfdecb6e9378cf692)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit f42a08e1aabf1ca57e0c09d69fb69cc717c7f156)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-04 22:57:55 +01:00
Diego Sueiro
9e68bd2c94 oeqa/selftest/bblayers: Add test case for bitbake-layers layerindex-show-depends
(From OE-Core rev: 1a1bce6d9a84d268fd2c0e87a33dc4591d792dd0)

Signed-off-by: Diego Sueiro <diego.sueiro@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 80090c31164d62a169431ab71c4aaee5475b6f40)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-04 22:57:55 +01:00
Yanfei Xu
71818465cb parselogs: ignore floppy error on qemu-system-x86 at boot stage
We can disable floppy drive by BIOS on a hardware, but an empty floppy
drive is connected by default on qemu-system-x86. Linux usually detect
the device and modprode the matched floppy.ko at the boot stage. Due to
we don't specify a floppy deivce in qemu boot arguments, then the errors
about floppy reading comes out.

It is harmless and normal, so we could ignore this error message on
qemux86.

Seen if kernel-modules is included in the image which pulls in the
relavent kernel module.

https://lists.gnu.org/archive/html/qemu-devel/2021-04/msg01402.html

(From OE-Core rev: 982236ed40f7e1fb912d52155495286fa1ca4757)

Signed-off-by: Yanfei Xu <yanfei.xu@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 3359f23ee9351c70997d5e0a17d17d1e47d59623)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-30 14:37:37 +01:00
Richard Purdie
18421b909b oeqa/selftest: Ensure packages classes are set correctly for maintainers test
The dnf packages aren't parsed if rpm isn't in PACKAGE_CLASSES which means
the aintainers test failes for OE-Core (where ipk is the default) but not
for poky (where the default is rpm).

Ensure PACKAGE_CLASSES is set so it works in all cases.

[YOCTO #14277]

(From OE-Core rev: 8717bf4f3c875b094ee29b4f3420ff5ad9cad1f3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 842b11107363357ed933cfcf619f1cf23f0d841e)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-30 14:37:37 +01:00
Richard Purdie
871de5c14d oeqa/selftest: Hardcode test assumptions about heartbeat event timings
Setting a value of 10 for heartbeat events causes the test to fail. Hardcode
a value to ensure it works correctly even if the default is changed.

(From OE-Core rev: cad75a7aa940efaf6674bfbbeb38d3fc11fab143)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 08b2c9a23ce43ed65a16f5f0714b19a571e1b54a)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-30 14:37:37 +01:00
Richard Purdie
723e15ccd1 selftest/reproducible: Sort the unused exclusion list
This makes comparisions between lists easier.

(From OE-Core rev: 950e4c5405dfafb8ec4ea1a66b462977be550509)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit d2c52125d1cdc06c7e08d507ca68f3e4612a4314)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-23 10:41:14 +01:00
Alexander Kanavin
5ec80538bd selftest/reproducible: track unusued entries in the exclusion list
This helps with trimming down the list, and towards 100% reproducibility :)

(From OE-Core rev: 857a8d8c619277ca0e40aa6c5c99793e6a019d84)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit da7a173d7a01524229c8515326465968a845e96f)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-23 10:41:14 +01:00
Steve Sakoman
548397c156 selftest/reproducible: adjust exclusion list for dunfell
Signed-off-be: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-23 10:41:14 +01:00
Alexander Kanavin
a225f167d0 selftest/reproducible: add an exclusion list for items that are not yet reproducible
Hopefully over time this list will be reduced to an empty one.

Non-reproducible excluded packages are not given to diffoscope and do not cause a
failure, but still saved side-by-side with non-reproducible failing ones to make
investigation easier.

(From OE-Core rev: 7cd8b42f11e39b473851b6603a5709f95b4dbf74)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 406bd0d48d8f90e2c836f7d3e204f21d5f13c833)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-23 10:41:14 +01:00
Alexander Kanavin
d685a71c23 selftest/reproducible: enable world reproducibility test
Add systemd and pam distro features, and commercial license flag
to include more recipes into the world set.

(From OE-Core rev: 3e87a2bb2ee4db33e80b93a864d1153353fc12ad)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit ed8f94942c07784f9f6eef5c00b75aa9b398ea5e)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-23 10:41:14 +01:00
Richard Purdie
eac29d491a selftest/wic: Fix dependency issue in rawcopy test
The current test code contains a subtle race. It is building an ext4,
then trying to reuse that image in a wic build for the same recipe.
Whilst this happened to work, the code is within its rights to clean
up the output before the wic code runs.

Avoid this by using separate image targets instead.

(From OE-Core rev: d30a90062e29a3069f6e8d9f6d4eadd7a48adf4f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 590398080e81fb5e2f81e12b8900858837dfe25f)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-28 22:31:47 +01:00
Lee Chee Yang
b9739ad63a cve-check: CVE_VERSION_SUFFIX to work with patched release
CVE_VERSION_SUFFIX in "patch" to treat version string with suffix "pX"
or "patchX" as patched release.

also update testcases to cover this changes and set CVE_VERSION_SUFFIX
for sudo.

(From OE-Core rev: d75f95a09e5e85eb759e748f9e0fee1c5fa1b318)

Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 8076815fc2ffc8f632e73527ce2b7d158a29e9ea)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-28 22:31:47 +01:00
Khem Raj
11089dbefb oeqa/pam: Need shadow installed for the tests
Distros may not use shadow for login manager
secondly login --help returns 1 only with shadow provided login
utility, with busybox it returns normal 0, the testcase clearly
assumes shadow being part of image, therefore add needed
checks to ensure the pre-requisites are met

(From OE-Core rev: 0ac2dee8756573a793af56582565f221ff6b93c5)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 0820aeba74869bf6e85b200f39bc610a43ab9a7a)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-18 21:20:25 +00:00
Richard Purdie
97b0e371fa selftest/reproducible: Add ability to pull some objects from sstate
When debugging why a single recipe doesn't reproduce, its a pain
to wait for the world to rebuild from scratch. Update the selftest
to allow this to be configured, for example you could set
targets as ['perf'] and sstate_targets as ['virtual/kernel']
and then it should only be rebuilding perf in the test rather than
things like the toolchain (parts of the kernel may be unavoiable
as they're not in sstate).

Can be run as:

OEQA_DEBUGGING_SAVED_OUTPUT=/tmp/perf-diffoscope oe-selftest -r reproducible.ReproducibleTests.test_reproducible_builds

to save diffoscope output.

(From OE-Core rev: 8b38cd1b36f3e227a63c0aa3955f3f5ab172c509)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 132a17d02f29711572e14a2f38a841323fbb6df6)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-18 21:20:24 +00:00
Richard Purdie
9ee329c18f selftest/reproducible: Don't call sync between each file compare
Calling sync between each file compare is horrible performance wise
as we compare thousands of files. We don't care about IO latency here
so disable.

(From OE-Core rev: dd893279f352e290c23577a0f91df1b8c6232f10)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 25f78abd8bbeb201fd9452e7983e015027954948)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-04 17:39:10 +00:00
Richard Purdie
8db35527ff oeqa/commands: Fix compatibility with python 3.9
Python 3.9 dropped isAlive() so use the preferred is_alive().

(From OE-Core rev: 48acaa7d53b677f55c96289750aa0dd26345af7f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 9bb06428cbb2ac0f3d98a1696f050d3393385503)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-04 17:39:10 +00:00
Wes Lindauer
c6227ebb9a df.py: Add feature check for read-only-rootfs
If the target is using a read-only rootfs, the available space on '/'
will be zero. This will cause the test to incorrectly fail and skipping
seems appropriate in this case.

(From OE-Core rev: 69c8987da199933554a9dd0eda0a045a4534bfab)

Signed-off-by: Wes Lindauer <wesley.lindauer@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit af1dbea3c9b9b42a3e6803b231e425423d70e210)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-04 17:39:10 +00:00
Teoh Jay Shen
d1000e553d oeqa/runlevel : add test for runlevels
This test mimic the boot_from_runlevel_3 and boot_from_runlevel_5 test cases from oeqa/manual/bsp-hw.json.
The boot_from_runlevel_3 and boot_from_runlevel_5 manual test cases should be remove from oeqa/manual/bsp-hw.json if this patch get merged.

(From OE-Core rev: 56d4c2867743b4b6bf79a4a9472799038522a331)

Signed-off-by: Teoh Jay Shen <jay.shen.teoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 6effd30f0e3726bc1f2eb7768c57c6b95eddb079)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-04 17:39:08 +00:00
Joshua Watt
76a3b07f90 oeqa: reproducible: Add more logging
The reproducible build tests can take a long time, so having more
logging messages at various points in the build can help debug where the
build is taking a long time.

(From OE-Core rev: 60a9e095297b756175e4a414bbee297b74547a7a)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 6b792afe8759d62af8e713b86dad8f6721961a05)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-04 17:39:08 +00:00
Joshua Watt
790949cec9 oeqa: reproducible: Fix SSTATE_MIRRORS variable
The SSTATE_MIRRORS variable was misspelled, which allowed the "clean"
test build to pull from the mirror.

(From OE-Core rev: f81fa4cf7ff1e48f7499f2cb5e2c7d7f9ea438be)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit e42497bd84d0bb370a9f7b0448bff29f01fd1b0c)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-04 17:39:08 +00:00
Lee Chee Yang
dc38d5e494 wic/selftest: test_permissions also test bitbake image
existing test case test_permissions use Wic command as standalone
tools to create wic image and check that wic image for permissions.

add extra steps to the test case to also check against image build
using bitbake do_image_wic.

(From OE-Core rev: 9efabaff73090b08233b4fcef22142b9ac0c11aa)

Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 551ce73a90757ba43501fe5cf9ac84a7b77de549)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-10 23:55:53 +00:00
Ricardo Ribalda Delgado
6c7a7d371f oeqa: wic: Add tests for permissions and change-directory
Make sure that the permissions and username are respected when using all
the rootfs modifiers.

Add tests for change-directory command

Cc: Paul Barker <pbarker@konsulko.com>
(From OE-Core rev: 5a3ce870ff54e73543caba5899aabd6c872691b0)

Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 4aad9531df44d1b0637bd559161702ad86861b46)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-10 23:55:53 +00:00
Mark Hatle
a90e96977e package.bbclass: hash equivalency and pr service
When the PR service is enabled a number of small changes may happen
to variables.  In the do_package step a call to package_get_auto_pr
will end up setting PRAUTO and modifying PKGV (if AUTOINC is there).

PRAUTO is then used by EXTENDPRAUTO, which is then used to generate
PKGR.

Since this behavior typically happens BEFORE the BB_UNIHASH is
calculated for do_package, we need a way to defer the expansion
until after we have the unihash value.

Writing out the pkgdata files w/o AUTOPR and PKGV (AUTOINC) expanded
to placeholder values is the easiest way to deal with this.  All other
variables are expanded as expected.

In the next task, typically do_packagedata, we will then use the
UNIHASH from the do_package to get the PR (AUTOPR) as well as
generate the AUTOINC replacement value (now PRSERV_PV_AUTOINC).

The do_packagedata then translates the placeholders to the final values
when copying the data from pkgdata to pkgdata-pdata-input.

Also update the prservice test case.  With unihash, just changing the
do_package (via a _append) will not change the PR.  So write the date
to a specific file that is incorporated into the unihash to ensure it
is always different for the test.  Various assert messages were also
updated to make it easier to figure out where/why a problem occured.

(From OE-Core rev: 0c28edf4bf0d2f92bf3a47406041c63acd90bacf)

Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 2e32f37b0e4abc438c8f60e673cd18a5cc110768)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-10 23:55:53 +00:00
Lee Chee Yang
2b526d9079 cve_check: add CVE_VERSION_SUFFIX to indicate suffix in versioning
add CVE_VERSION_SUFFIX to indicate the version suffix type, currently
works in two value, "alphabetical" if the version string uses single
alphabetical character suffix as incremental release, blank to not
consider the unidentified suffixes. This can be expand when more suffix
pattern identified.

refactor cve_check.Version class to use functools and add parameter to
handle suffix condition.

Also update testcases to cover new changes.

(From OE-Core rev: 37a40c30709bf80c74948f47361b2be2c646c9d8)

Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 5dfd5ad5144708b474ef31eaa89a846c57be8ac0)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-10 23:55:53 +00:00
Lee Chee Yang
f829419105 cve-check: replace Looseversion with custom version class
The way distutils.version.LooseVersion compare version are tricky, it treat
all these ( "1.0-beta2", "1.0-rc1", "1.0A", "1.0p2" and "1.0pre1") as greater
version than "1.0". This might be right for "1.0A" and "1.0p1" but not for
the rest, also these version could be confusing, the "p" in "1.0p1" can be
"pre" or "patched" version or even other meaning.

Replace Looseversion with custom class, it uses regex to capture common
version format like "1.1.1" or tag format using date like "2020-12-12" as
release section, check for following known string/tags ( beta, rc, pre, dev,
alpha, preview) as pre-release section, any other trailing characters
are difficult to understand/define so ignore them. Compare release
section and pre-release section saperately.

included selftest for the version class.

[YOCTO#14127]

(From OE-Core rev: 294baea424472341d2ec880f13699076315d8274)

Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 6ced85e9ddd3569240f1e8b82130d1ac0fffbc40)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-10 23:55:53 +00:00
Tomasz Dziendzielski
48097ee02f selftest/devtool: Add modify_localfiles_only test checking symlink path
If recipe uses only file:// fetcher devtool unpacks sources under
oe-local-files/ and adds symlink in source root directory. We need to
verify if the symlink in subdirectory has correct path.

See [YOCTO #13738] for details.

(From OE-Core rev: d9d8a04c9564b2801a1f613ffb6471de3cabdb03)

Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 044c90d152bfd1b2a23272df39327ba4cd862dff)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-05 23:37:04 +00:00
Yi Fan Yu
c604b510f4 oeqa/selftest/cases/tinfoil.py: increase timeout 10->60s test_wait_event
The test would timeout on autobuilders. This patch increases the
timeout to 60s

The test will now also exit as soon as we receive the 2 expected events
Expected runtime is around 1s if successful

Bug 14158

(From OE-Core rev: cbc72203c82e56ebf4a6a685fbeb27679ed9a2e0)

Signed-off-by: Yi Fan Yu <yifan.yu@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit be02aa9283f805de718badd5ea12c4968da8774f)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-05 23:37:03 +00:00
Alexander Kanavin
e89fbddd48 oeqa/ptest: print a warning if ptests failed
This allows spotting ptest regressions without having hard ptest failures
(for that full ptest stability should be achieved).

(From OE-Core rev: 6097b1cbffc048185c1ed658f16f0f90a9d19054)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 79b54d902df2788b0822b6c9cc14705ab00e6a5b)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
(cherry picked from commit 5232c0b480eba4537c8db71c95189950011fb64d)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-05 23:37:03 +00:00
Paul Barker
1c68a71486 bitbake.conf: Prevent pyc file generation in pseudo context
This also effectively reverts commit b6d30c21b0:
    bitbake.conf: Extend PSEUDO_IGNORE_PATHS to ${COREBASE}/meta

The contents of ${COREBASE}/meta were ignored as pyc files could be
generated for the contents of the lib subdirectory if python modules
were imported within a pseudo context. However this doesn't protect us
from pyc files being generated in the lib directories for other layers.
It's far better to tell python not to produce pyc files when running
under pseudo (by setting the PYTHONDONTWRITEBYTECODE variable) as this
will cover any location where pyc files could possibly be created. This
variable is set in FAKEROOTBASEENV so that it applies to the
bitbake-worker instance for fakeroot tasks, preventing pyc files from
being generated for imports in python tasks themselves.

Also add a test case to ensure that pyc files are not created in tasks
which are executed under pseudo.

(From OE-Core rev: 109406724d29b289d19f87dfdf6604767157277a)

Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 73d538f20743017a44cea4c20dbe09a0327cfc71)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-03 14:13:53 +00:00
Richard Purdie
f54fb698e2 oeqa/selftest/runtime_test: Exclude gpg directory from pseudo database
Avoid pseudo abort()s like:
path mismatch [1 link]: ino 6295376 db '/tmp/oeqa-feed-sign-2mw7z81v/S.gpg-agent.yocto-native' req '/tmp/jwkivmu6'.

(From OE-Core rev: 4909fa31a4faabb1add10ded69f29cb27d5e1fb7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 189630ca6cdf7ceb6cf9b8f9d86c58997f505efc)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-03 14:13:53 +00:00
Paul Barker
ecd4fbc0ca selftest: Add argument to keep build dir
The oe-selftest code already keeps the selftest build directory in place
if any tests failed. By default the build directory is deleted if all
tests pass but there may be cases where it's desirable to keep this
directory around, for example to compare intermediate files between
passing and failing test runs.

(From OE-Core rev: 51fa76c699517ad63f8d6c4db4b6938d894f5c6c)

Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 67aa7069dbe8f5f5f186eb67708ece5c4bd42976)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-13 08:57:57 +00:00
Milan Shah
f9a95834a7 oe-pkgdata-util: Added a test to verify oe-pkgdata-util without parameters
A test is implemented on poky/meta/lib/oeqa/selftest/pkgdata.py to test
the scenario when oe-pkgdata-util is executed without parameters and
help is displayed.

See [YOCTO #10726] for detailed bug information.

(From OE-Core rev: 59272d46d61b3328ebd5c72d702d7e5b368afeb4)

Signed-off-by: Milan Shah <mshah@mvista.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 6b4e2eafa6eb71ca94ccc8a18d05b473b352367c)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-13 08:57:57 +00:00
Richard Purdie
85964fcb8e oeqa/commands: Ensure sync can be found regardless of PATH
Avoid command not found errors shown in selftest logs due to changes to PATH
settings which also risks intermittent problems due to IO load.

(From OE-Core rev: e2814ca1105e6ed9b688f2e8e6a48dff0c7d402f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 40bcae01b0be2f293dea9ab42c6b7f8f47827cf5)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-09 09:17:16 +00:00
Ross Burton
a297097cff lib/oe/qa: handle the 'no specific instruction set' ELF e_machine value
[RP: Update OEQA selftest to match change]
(From OE-Core rev: a4339b1c958b736be951ae3142e6be72702ecad2)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit b7cfc0f51cc0b4866f913f6eae4fcc6f72d2578c)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-09 09:17:16 +00:00
Steve Sakoman
c0cf641dc8 oeqa/selftest/cases/devtool.py: fix typo in ignore_patterns call
Causes intermittent autobuilder errors:

2020-12-21 19:34:23,035 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/meta/lib/oeqa/selftest/cases/devtool.py", line 68, in setUpModule
    bb.utils.edit_bblayers_conf(bblayers_conf, None, None, bblayers_edit_cb)
  File "/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/bitbake/lib/bb/utils.py", line 1460, in edit_bblayers_conf
    (updated, newlines) = edit_metadata(newlines, ['BBLAYERS'], handle_bblayers)
  File "/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/bitbake/lib/bb/utils.py", line 1290, in edit_metadata
    if handle_var_end():
  File "/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/bitbake/lib/bb/utils.py", line 1215, in handle_var_end
    (newvalue, newop, indent, minbreak) = varfunc(in_var, full_value, op, newlines)
  File "/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/bitbake/lib/bb/utils.py", line 1425, in handle_bblayers
    res = edit_cb(layer, canonicalise_path(layer))
  File "/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/meta/lib/oeqa/selftest/cases/devtool.py", line 60, in bblayers_edit_cb
    shutil.copytree(pth, destdir, ignore=ignore_patterns('*.pyc', '__pycache__'))
NameError: name 'ignore_patterns' is not defined

(From OE-Core rev: 8bfc666b7cddf37b7681e45dd0f87365014ac5a3)

Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit c2d9612279fce9cbcb738913b2042949f692c4a5)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-01 11:55:35 +00:00
sangeeta jain
a71a0ed31a meta/lib/oeqa/manual/oe-core.json: Update test_bitbake_devshell
Update command used for cross compilation to include ${CONFIGUREOPTS}
to ensure right arguments are passed for cross compiling on any host.

(From OE-Core rev: e902026496ea026c4316d1a74265c633db9431b8)

Signed-off-by: sangeeta jain <sangeeta.jain@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit e995e67bc7a3d30adf9f8d0d2f4df92a941baeae)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-12-18 14:20:19 +00:00
Ross Burton
b7fdda2d03 oeqa/devtool: use Yocto mirror for pv-1.5.3 tarball
Some of the selftests failed over the weekend with "access denied" errors
fetching this tarball.  Instead of relying on upstream when fetching the
tarball, use the Yocto source mirrors instead.

(From OE-Core rev: ab3a9bd56d86ed22515906b2d23e6a83053fcd37)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 988e0ff4131f46dfed14516ff5f61d72b9fb6941)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-12-18 14:20:19 +00:00
Konrad Weihmann
199d43ac1e oeqa/core/context: initialize _run_end_time
with _run_start_time as value. For partial results of interrupted runs,
this info might be otherwise missing for at least one testcase

(From OE-Core rev: 4ebf97e36d9def38fb869cbbbd5ce10f5d2669a3)

Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 1c5e8baf57fa2a33b9ef507b11d9ea9acaa77238)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-20 10:44:34 +00:00
Konrad Weihmann
51e39a58b4 oeqa/core/context: expose results as variable
register an unittest handler for testresults and expose it as
variable result.
With this even partial results from an interrupted test suite run
can be made available

(From OE-Core rev: 07a3ea1aece5c67d75b60417ee0978cc0e15719a)

Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit a97ae47525157871b6c098ffc352293e365a4335)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-20 10:44:34 +00:00
Richard Purdie
13936f5f35 oeqa: Add sync call to command execution
We previously put a sync call into devtool to try and combat the bitbake
timeout issues on the autobuilder. It isn't enough as the timeouts occur
mid test. They are also occurring on non-devtool tests.

Add in sync calls around command execution instead.

(From OE-Core rev: ed912771ea98c42f61bf927b1ca708650b0bed4c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit ceca5ed121e2b54415a7ab3a217882e4ea86923a)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-30 17:23:19 +00:00