For the same reasons as the runtime Python test, clean up the SDK test.
Also port from Python 2 to Python 3, as that's what is supported now.
(From OE-Core rev: bead742a3ffc0a53162fb0c36610d74a1422e7b3)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently this is three test cases:
1) test_python_exists. Fail if python3 isn't in PATH.
2) test_python_stdout. Run a Python script and check the output is as expected
3) test_python_testfile. Check that a file test_python_stdout wrote to exists.
(1) should be a setup and skip the test module if it isn't present.
(2) and (3) should be merged, there's no point copying over a two line Python
file, and the test doesn't verify that the file doesn't exist in the first
place.
Rewrite the test to check that Python is present in a class setup so the entire
test is skipped if it isn't and do some simple rot13 to verify that bytecode is
being executed correctly.
(From OE-Core rev: a35be5f32b4fe70b18ac1e2eccfd94558cecfbba)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These platform specifc BSPs were removed from meta-intel and superseded by
intel-core* BSPs.
(From OE-Core rev: a6fd20fc57da3e1910e775f5994bd71bf79934ea)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Checking if the dependency had any failure is unreliable, for example
if the underlying data doesn't get transferred and the list is empty,
success of the dependency is assumed.
Since we now have success data available, change the code to use it.
(From OE-Core rev: 4abba4c30d5a6163a968a119395a679e5e281ab4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This uses the new concurrenttest code to enable parallel test execution
if specified.
(From OE-Core rev: 07d19fb3adab7a8d83ba83d9a16395f70d7b7a47)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This isn't present on modern distros by default and doesn't work with
testtools, needing multiple code paths in the code. Remove it in favour
of finding a better replacement for results collection/analysis.
(From OE-Core rev: 8001d933a8dc86004db014777f094d718086687d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There doesn't appear to be any reason we need this _results indirection
any more so remove it.
(From OE-Core rev: b618261811c48ff3b98eab1b340a8cd09ef183c6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We have several options for parallel processing in oeqa, parallel
execution of modules, threading and mulitple processes for the runners.
After much experimentation is appears the most scalable and least
invasive approach is multiple processes using concurrenttestsuite
from testtools. This means we can drop the current threading code
which is only used by the sdk test execution.
oeqa/decorator/depends: Remove threading code
Revert "oeqa/sdk: Enable usage of OEQA thread mode"
This reverts commit adc434c063.
Revert "oeqa/core/tests: Add tests of OEQA Threaded mode"
This reverts commit a4eef558c9.
Revert "oeqa/core/decorator/oetimeout: Add support for OEQA threaded mode"
This reverts commit d3d4ba902d.
(From OE-Core rev: a98ab5e560e73b6988512fbae5cefe9e42ceed53)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows oe-selftest to take a -j option which specifies how much test
parallelisation to use. Currently this is "module" based with each module
being split and run in a separate build directory. Further splitting could
be done but this seems a good compromise between test setup and parallelism.
You need python-testtools and python-subunit installed to use this but only
when the -j option is specified.
See notes posted to the openedmbedded-architecture list for more details
about the design choices here.
Some of this functionality may make more sense in the oeqa core ultimately.
(From OE-Core rev: 326ababfd620ae5ea29bf486b9d68ba3d60cad30)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This improves test parallelism.
(From OE-Core rev: 26e04b23ba1b6942aa7c7df478d41dfe7b73e6e0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Only show the output in the failure case.
(From OE-Core rev: 324785a99e3f2449cb443233e7ce224598c02a3b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows better parallelism between the different tests as currently
this block takes the longest time to execute. devtool tests are still
all grouped into the "devtool" module for ease of exection.
This also makes it easier to execute some subset of devtool tests for
testing devtool changes.
(From OE-Core rev: 75148c190dd4823947557e9a07f1722e817c1fea)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently some tests run in buffer mode and some don't. Those that don't can
corrupt stdout/stderr. Switch to using buffer mode everywhere so we're consistent.
If there is useful output on stdout/stderr, it will be displayed if the test
fails.
(From OE-Core rev: 978548c0abde2cb94c2782538552f39bdf2bf630)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
AssertionError: Command 'wic create mkhybridiso --image-name core-image-minimal -o /var/tmp/wic.oe-selftest/' returned non-zero exit status 1:
ERROR: _exec_cmd: gzip -f -9 -c /var/tmp/wic.oe-selftest/tmp.wic.drhn6edm/initrd.cpio > /var/tmp/wic.oe-selftest/tmp.wic.drhn6edm/initrd.cpio.gz returned '1' instead of 0
output: gzip: /var/tmp/wic.oe-selftest/tmp.wic.drhn6edm/initrd.cpio: No such file or directory
This is because in a clean build directory, the initramfs may not be rebuilt.
Add a call to ensure it is built to avoid the error.
(From OE-Core rev: 2a80fa234d31992691a157425e8990db30158fd1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Using /var/ leave wic open to races with other processes on the system, use
a subdir of builddir instead to avoid this.
(From OE-Core rev: e07ec908ce7f26143a7bdf0a07a1230c0fd6ac87)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If dbus isn't build first the dbus dependency of dbus-wait can't be detected
through pkgconfig and the test fails:
AssertionError: {'DEPENDS': {'dbus'}} != {}
- {'DEPENDS': {'dbus'}}
+ {} : Some expected variables not found in recipe: {'DEPENDS': {'dbus'}}
Ensure dbus is built and present in the sysroot.
(From OE-Core rev: 28699b4257436fb6079eafe50ca8cab09a2fdd90)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The main setUp function needs to be called and tests are meant to
define setUpLocal. For some reason this one didn't leading to errors
with recent code changes. Fix it.
(From OE-Core rev: aef07f09e224485539d8bc66ddac8bf394e4092a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The current code assumes if something isn't a failure of some
kind, it was a pass. When test case IDs weren't matching, this lead
to very confusing output where things would fail, then be listed as
passing.
This adds code to track successes, ensuring we don't end up in this
position again with unmatched entries being listed as UNKNOWN.
(From OE-Core rev: 4374c296d8963e4f6a1aa7bef7983ad0a1c2fcff)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By having a consistent config for the test, a second reparse is avoided
and the test runs slightly faster.
(From OE-Core rev: a910d90dc18f9bc63142ccae2eeadc1feefc756b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If something goes wrong with the test, dep can be undefined. Avoid a
traceback for this and handle it more gracefully.
(From OE-Core rev: 0b74cb42aefe5c1b812fd155784e4c6581e26ce2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We can directly compare the test case IDs rather than representations,
then if we're using subunit to split the tests, the comparisions still
work as intended.
(From OE-Core rev: 72e5f46f75454ba4c445c65c1cbc616a9e72fc6e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
"allow root login" should not be bundled in ssh_allow_empty_password,
because some distro may want only one of "allow root login" and "allow
empty password", so split it out into ssh_allow_root_login and add new
imagefeature allow-root-login so they can be controlled separately,
debug-tweaks will still include both of them.
(From OE-Core rev: 1ab494f06a12548a902298afabd0a842161ef10d)
Signed-off-by: Jackie Huang <jackie.huang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The following error message when starting core-image-weston is not
critical as long as the image could start up correctly. So extend
the common_errors list for parselog.py test case to ignore this
message.
logind: cannot setup systemd-logind helper (-61), using legacy fallback
[YOCTO #12835]
(From OE-Core rev: edc244b2f346ad2eb3c90664ec17c904b356b992)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds a test that creates a layer, adds it to bblayers.conf, then ensure
that it appears properly in bitbake-layers show-layers.
(From OE-Core rev: 828462d2c39fe6f4d188b5eb129f7b2969e1ee18)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We are going to let runtime test support kernel tests. Now we just add
kernel self-contained sample tests. And we plan to add overall kernel
tests in the future.
This patch is just add kernel samples test which contains about 13 tests
enabled by kernel-sample.scc. So it needs statement,
KERNEL_FEATURES_append += " features/kernel-sample/kernel-sample.scc" in
local.conf.
(From OE-Core rev: be6ef728fd36049ea3822a54909a995bdecf6dd0)
Signed-off-by: Hongzhi.Song <hongzhi.song@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add an automated test to exercise that fetching a git: recipe...
1) works over git without mirrors
2) fails without git connectivity or mirrors
3) works without git connectivity but with a mirror
(2) is done by setting GIT_PROXY_COMMAND to 'false' which should break any git
network operations.
[ YOCTO #12805 ]
(From OE-Core rev: 13269dfbbc62faef32595343dc78250fdb2a2946)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The test cases assume that rpm-doc is built out, as it says it its
comment. This is not always true. And it sometimes results in
following error.
| cls.tc.target.copyTo(test_file, dst)
| UnboundLocalError: local variable 'test_file' referenced before assignment
Change to use base-passwd-doc, as this package is more likely to be
built out than rpm-doc.
(From OE-Core rev: 2e23543b48921182307065c1fa9e8b9d7fbb3cdc)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The following message from watchdog is not an error.
"error retry time-out = 60 seconds"
Checking watchdog's source codes, we have:
src/watchdog.c: log_message(LOG_INFO, "error retry time-out = %d seconds", retry_timeout);
It's clear this is an info message, so parselogs should ignore it.
(From OE-Core rev: c90dff78310ab1517555c4df39b3e2d2f59ff06d)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add the following line to common_errors list.
Failed to read /var/lib/nfs/statd/state: Success
This message is not harmful, it does not result in rpc.statd starting
failure.
(From OE-Core rev: 0f2c39149941d95fa979ec675f9dc25655886e2c)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The test works by excluding curl-dev which curl-staticdev depends upon.
When static libraries aren't disabled, this leads to an odd looking test
failure.
Simply exclude curl-staticdev as well in case its enabled to make sure
the test always works.
(From OE-Core rev: 0a77f4b680332a692abf0231e397fab44c59bde3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As I will be leaving Intel, this address will no longer be valid,
so swap it for my personal one for now.
(From OE-Core rev: ea58ff101e28dfda3410de66d775df3d8a1e5a96)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When running testimage task for core-image-sato-sdk, the following
error appeared.
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x82 at position 0: invalid start byte
Checking the codes, I found it's caused by setting a 1024 limit for the
read method of the StreamReader object.
Comments from the manual:
"""
The chars argument indicates the number of decoded code points or bytes to
return. The read() method will never return more data than requested, but
it might return less, if there is not enough available.
"""
When running `systemctl status --full' on target, this error occurs.
This patch increase the bytes limit to 4096 to fix the error.
(From OE-Core rev: f1fad60ae3be4450aca6058d5665fb10a9148b44)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The test_file_connman should be executed only when 'lib32-connman' is
installed and 'connman' is not installed.
When lib32-connman and connman are both installed, the /usr/sbin/connmand
could be from connman or lib32-connman, depending on the installation
order. What we want to check is the connmand command from lib32-connman,
so we need to make sure that connman is not there to cause chaos.
(From OE-Core rev: bc6839394c06bb695b92b2183337e7381da1e86c)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
1) The test cases use 'readelf' command to do the check. This command
is from binutils. So skip the test if the needed binutils package is
not installed.
The related error message in log.do_testimage is like below.
Output: sh: readelf: not found
2) The test case tests /lib/libc.so.6 from lib32-libc6. So skip the test
if lib32-libc6 is not installed.
The related error message in log.do_testimage is like below.
Output: readelf: Error: 'lib/libc.so.6': No such file
(From OE-Core rev: eae929a5c224f5c3468d6a0466d1bbb3f678a5a1)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This test case should only run when rpm package is installed.
So skip it if rpm package is not installed. This fixes:
RESULTS - rpm.RpmBasicTest.test_rpm_help - Testcase 1059: FAILED
(From OE-Core rev: bb909a60c04248d015d988e4454f0a11b1c287da)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The var might not be set, resulting in unexpected error.
RESULTS - multilib.MultilibTest.test_check_multilib_libc - Testcase 1593: ERROR
The above error is due to MULTILIBS being not set, which is the default
for OE. This patch fixes this problem.
Also, the debugging message in skipIfNotInDataVar is currently confusing.
Instead of
DEBUG: Checking if 'MULTILIBS' value is in 'multilib:lib32' to run the test
it should be
DEBUG: Checking if 'MULTILIBS' value contains 'multilib:lib32' to run the test
This patch also fixes it.
(From OE-Core rev: 3f5c678664a2bba43d99508779dc2ce227cf52a2)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use 'cls' instead of 'obj' to better reflect that registerDecorator
actually serves as a class decorator.
(From OE-Core rev: e06e4c859e8be5225d80806a2ebe175f0b152fe1)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
OETestFilter is a subclass of OETestDecorator. It wants to make
use of @abstractmethod decorator. But such decorator requires
metaclass to be ABCMeta to have effect. So add it now to achieve
the designed behaviour.
Comments from python's manual:
"""
Using this decorator requires that the class's metaclass is ABCMeta
or is derived from it.
"""
(From OE-Core rev: 28c4fafb2322ea8c37bcd7710f22f46ef552a902)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In previous OEQA, having 'auto' in TEST_SUITES results in executing
as many test cases as possible.
This behaviour is broken for now. From the codes in core/loader.py,
I can see that it tries to use another keyword 'all'. But in fact,
it does not work.
I've checked the current manual. The manual says using 'auto'.
Below is the current information in manual.
"""
Alternatively, you can provide the "auto" option to have all applicable
tests run against the image.
TEST_SUITES_append = " auto"
"""
So we should restore this behaviour. This patch does so.
Also, output warning message is some module is named as 'auto', as this
is a reserved keyword.
(From OE-Core rev: a65460a063a958cc887c756db5f7ab18e3f5a8c1)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need to skip this testcase when gettext is not available. Otherwise,
we will have the following error at configure.
error: possibly undefined macro: AM_NLS
(From OE-Core rev: ccc10e95c9fcdabcf4ae5e6f9cc34c1d632b4654)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add test cases to test “exclude” and “installroot“ options, also modify
the logic of filtering packages on the feed to have all the packages
needed by the tests.
[YOCTO #10744]
(From OE-Core rev: 1121806603c6f621d084b692216f3f616a0768dc)
Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For example, core-image-sato skipped the test alltogether, as it
no longer pulls in Python 2.x at all.
(From OE-Core rev: 5ad0fe9ac6b6362011a17afaa7bee8e788093915)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Drop upstreamed 0001-Fix-build-with-musl-missing-header-include-for-dev_t.patch
Add ftw-subdir-walk.patch as it resolves the RECIPE_NO_UPDATE_REASON.
Add --disable-zstd as libzstd isn't provided in oe-core.
Fix wic testcase, as the minimal fs size is now bigger.
(From OE-Core rev: 94b645aa77a4193371e8c77ddc477ec00d858961)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since the wtmp and btmp definitions had been moved from logrotate.conf
to logrotate.d in this release, we also need to install them to
/etc/logrotate.d/.
Also update oeqa runtime logrotate test case.
(From OE-Core rev: 5b4aedd6b18b6ba6ca1bcd460a0b51ced41656cd)
Signed-off-by: Yi Zhao <yi.zhao@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This test shouldn't have merged yet since we don't run portmap/rpcbind
on the autobuilder infrastructure and the test therefore cannot succeed.
We need to document this, set it up, then enable the test. The test itself
is fine and good to have so its left in the code but disabled for now.
(From OE-Core rev: 9640af873d490c5d22b70e32d918c2db37371d21)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
QA team were testing qemu boot image and shutdown on each
qemu architecture manually. Add automated test to test qemu boot on
ext4 and nfs, finally check that it can shutdown properly.
(From OE-Core rev: 1df5f2dff832528905ff6fcf1d324619fb3d307f)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add runtime test for stap to test basic SystemTap
operations: can compile very basic module and run on
target device.
Note we disable (-DSTP_NO_VERREL_CHECK) SystemTap
additional kernel release check since during OE testing
mismatching kernel-devsrc and kernels are used.
(From OE-Core rev: 659d19fcddb7edaca8f5221148d479e73304b430)
Signed-off-by: Victor Kamensky <kamensky@cisco.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that we see warnings if LAYERSERIES_COMPAT is unset, the auto generated
code from devtool needs to set this to avoid warnings which break
various tests.
(From OE-Core rev: f65ebfeda0bfbac78e4a2a6609ba654ca38a8b0e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that we see warnings if LAYERSERIES_COMPAT is unset, the auto generated
code from devtool/oeqa needs to set this to avoid warnings which break
various tests.
(From OE-Core rev: 27568410ebb0d40db3428550704f35199df0e034)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
