Commit Graph

318 Commits

Author SHA1 Message Date
Changqing Li
7643168778 logrotate.py: fix testimage occasionally failure
testcase test_systemd_failed occasionally failed with below error:
Failed to start Rotate log files.
logrotate.service: Failed with result 'exit-code'.
logrotate.service: Main process exited, code=exited, status=1/FAILURE
error: stat of /var/log/logrotate_test failed: No such file or directory
error: logrotate_test:1 lines must begin with a keyword or a filename (possibly in double quotes)

above failure caused since testcase test_logrotate_wtmp
add /etc/logrotate.d/logrotate_test, which need /var/log/logrotate_test,
but there is no such file. so when logrotate.service is triggerd
by logrotate.timer after testcase test_logrotate_wtmp is runned,
the testcase test_systemd_failed will fail.

these 3 lines are useless, so remove them to fix above problem.

(From OE-Core rev: 198fe0d64e4bdeda84c0f3e44afcc1d99ca9dbbe)

Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 22f5f7f86a4d47624c476be00e5121009c48cb7b)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-07 23:15:10 +01:00
Andrej Valek
c330f61704 oeqa/runtime/cases/ptest: Make output content path absolute
The output content is created in current directory, because json content
has no defined absolute path to WORKDIR as in bitbake.

(From OE-Core rev: 394d90efdc8d06dc61114bf6279ba6d094667289)

Signed-off-by: Andrej Valek <andrej.valek@siemens.com>
Signed-off-by: Peter Marko <peter.marko@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 14203b2cb9aa62f55cb12230ac8012b3cd995be7)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-02 16:12:36 +01:00
Changqing Li
3fc7afb010 parselogs.py: ignore pulseaudio startup warning messages
If set default syslog to rsyslog, we can see below messages
in user.log,

[pulseaudio] authkey.c: Failed to open cookie file
[pulseaudio] authkey.c: Failed to load authentication key

They are only warnings when cookie file is not found. And
PulseAudio will create it if it doesn't exist.

refer:
https://wiki.archlinux.org/index.php/PulseAudio/Configuration
https://lists.freedesktop.org/archives/pulseaudio-discuss/2014-December/022719.html

(From OE-Core rev: 2cc3fac9cd1a0d77931c9e49dbe2941fa8619c51)

Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-05 08:45:31 +01:00
Changqing Li
9c760ffd3d parselogs.py: update network interface related messages
along with systemd upgrade, error message related change
network interface have changed, update it.

(From OE-Core rev: a8b2cd7470bcc25527577b95a26a0a528949232d)

Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-07 21:57:16 +01:00
Stefan Kral
80c1002c2e oeqa: default runtime json results for testexport
Set the json result output dir in the oeqe runtime context to create
testresults.json file by default for exported runtime test runs.

Use current datetime for the json result property name (not DATETIME
from build) to allow multiple result entries.

(From OE-Core rev: e6c73ed8d3c5d45f387cab619ca73c21e850582f)

Signed-off-by: Stefan Kral <sk@typedivision.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-29 12:05:31 +01:00
Alex Kiernan
ef0c086844 oeqa/runtime/cases: Disable and stop systemd-timesyncd
Stopping systemd-timesyncd doesn't prevent it being restarted by a
different transaction within systemd. Disable the service instead during
the date test to ensure it can't be restarted.

(From OE-Core rev: 3dd4e637c11abdd6341a3e0c6b67639d3d703862)

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Acked-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-29 12:05:31 +01:00
Khem Raj
350979b278 oeqa/qemuarm64: Ignore logind: failed to get session seat
When booting weston images this error is seen commonly, but Qemu boots
the image fine, session seat error is thrown by libweston perhaps using
--seat option or setting XDG_SEAT variable in weston.ini could fix it

[YOCTO #13828]

(From OE-Core rev: 1f9d36722132c93c19651ea59062b3d5be01bb72)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-13 11:32:10 +00:00
Stefan Kral
5c8a1312fa oeqa/runtime/context.py: fix typo
(From OE-Core rev: bf959e6fd175d0841a1c042bf925c54200bea3d4)

Signed-off-by: Stefan Kral <sk@typedivision.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-12 12:46:09 +00:00
Mingli Yu
d03c12bb8f parselogs.py: ignore rdrand initialization failure
On the system whose cpu doesn't support rdrand,
there comes below message when start rngd service
 #systemctl status rngd
 [snip]
 Feb 25 05:08:14 qemux86-64 rngd[133]: [rdrand]: Initialization Failed
 [snip]

Actually the failed message doesn't matter as it
only indicates one entropy source as rdrand fails
to initialize and won't affect rngd function.

So add to ignore the failure message to fix below
error during do_testimage:
NOTE: ======================================================================
NOTE: FAIL: test_parselogs (parselogs.ParseLogsTest)
NOTE: ----------------------------------------------------------------------
NOTE: Traceback (most recent call last):
  File "/buildarea/layers/oe-core/meta/lib/oeqa/core/decorator/__init__.py", line 36, in wrapped_f
    return func(*args, **kwargs)
  File "/buildarea/layers/oe-core/meta/lib/oeqa/runtime/cases/parselogs.py", line 370, in test_parselogs
    self.assertEqual(errcount, 0, msg=self.msg)
AssertionError: 1 != 0 : Log: /buildarea/tmp/work/qemux86-64-wrs-linux/wrlinux-image-std/1.0-r5/target_logs/daemon.log
Central error: 2020-03-06T09:45:12.774286+00:00 qemux86-64 rngd[134]: [rdrand]: Initialization Failed

Reference: https://github.com/nhorman/rng-tools/pull/84

(From OE-Core rev: 09c4f9068858bf59eb60584a497f987bdd1e196e)

Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-07 10:06:16 +00:00
Stefan Kral
56bbdd33d8 oeqa-runtime: add missing import os to ptest case
Add missing import os statement to the oeqa runtime ptest.py

(From OE-Core rev: e41c92c60694052c8292c1ea6b3d0b427aafafd7)

Signed-off-by: Stefan Kral <sk@typedivision.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-06 08:19:18 +00:00
Petr Vorel
a37dba3063 ltp: Update to 20200120
Removed ffsb copying file and associated runtest file fs_ext4
(both removed in this release).

Added LIBS=-lfts for MUSL.

Removed patches (accepted upstream)
* 0001-mkswap01.sh-Add-udevadm-trigger-before-swap-verifica.patch
* 0001-testcases-fix-an-absent-format-string-issue.patch
* 0001-nm01-Remove-prefix-zeros-of-the-addresses-output-by-.patch

Merged patches
* 0006-rt_tgsigqueueinfo-disable-test-on-musl.patch
(into 0001-build-Add-option-to-select-libc-implementation.patch as they
touch the same files)

Refreshed patches (some tests have been fixed for musl)
* 0001-Add-more-musl-exclusions.patch
* 0001-build-Add-option-to-select-libc-implementation.patch
* 0002-kernel-controllers-Link-with-libfts-explicitly-on-mu.patch
IMHO this one shouldn't be needed not, but CI still fails without it

Updated status
* 0008-testcases-network-nfsv4-acl-acl1.c-Security-fix-on-s.patch
* 0001-syscalls-Check-for-time64-unsafe-syscalls-before-usi.patch

(From OE-Core rev: 62e2063698477a13c55ae370451e52a2f1e18e45)

Signed-off-by: Petr Vorel <petr.vorel@gmail.com>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-02 16:16:27 +00:00
Alexander Kanavin
c43423cd28 ptest: add an option to enforce a failure (e.g. ptests are required to pass)
This can be useful with a more specific, targeted and robust set of ptest
packages; the benefit is that ptest regressions are caught as they happen
and can be more easily traced to changes that caused them.

The existing AB ptest image continues to be expected to fail, my observation
of the AB runs is that the full set of ptests is not robust enough
(particularly around socket/network related tests) and sporadically fails
in random places. This can probably be addressed by making ptests exclusive
to a worker (e.g. there is no other workload happening at the same time as
ptests).

(From OE-Core rev: 102afcc7ea7d7673183f272a29ab233084ad168b)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-25 10:41:22 +00:00
Alexander Kanavin
ae6ec0b809 weston: add a basic runtime test
The test is checking that weston is able to start.

(From OE-Core rev: 57700767f90eea8f2b78187c42581aca42d50bbf)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-18 23:53:54 +00:00
Alexander Kanavin
ec7beb650f qemux86: drop resolution setting via uvesafb
I am not sure if this has ever worked, but uvesafb is a really
outdated (VBE from the 1990s), awkward (needs v86d) and limited
(no support for high resolutions) way to do it.

The specific reason 640x480-32 was introduced (ages ago) was
to force 32 bit mode with vmware driver, as 16bit had rendering issues.

The modern, supported option is video=... kernel parameter documented here:
https://wiki.archlinux.org/index.php/kernel_mode_setting#Forcing_modes_and_EDID
https://github.com/torvalds/linux/blob/master/Documentation/fb/modedb.rst
which can be passed directly to runqemu and doesn't require special
kernel modules.

Sato under X will continue to use 640x480 as that is hardcoded into
xorg.conf under qemu.

(From OE-Core rev: 1cf26f69fd89b43be24cd1232c43e5050b9d718a)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-18 23:53:54 +00:00
Khem Raj
21d97d00d5 oeqa: Use --disable-maintainer-mode configure option
since the versions of autotools might differ on target and build host, plus difference in
timestamps for configure and system can result in reconfigure lets avoid
that by disabling maintainer mode

Avoids
error: newly created file is older than distributed files!

[YOCTO #13779]

Suggested-by: Andre McCurdy <armccurdy@gmail.com>
(From OE-Core rev: 3dc4e27e7633ce3ca6b9647810d0996bdee48771)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-14 13:07:23 +00:00
Khem Raj
504b206e37 oeqa: Use cpio 2.13 as testcase
cpio 2.12 was released in 2015 and might have used older autotools
which could result in errors like

https://bugzilla.yoctoproject.org/show_bug.cgi?id=13779

Bumping to 2.13 will help in matching the tool versions
A good change on top would be to run

aclocal -I .; autoheader; autoconf; automake --add-missing -c

before running configure step perhaps

[YOCTO #13779]

(From OE-Core rev: 84eb1dc4fe8a11cd2d05b703070a6fb6de05b873)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:02 +00:00
Trevor Gamblin
b486903ccc logrotate.py: improve oeqa test implementation
See bug https://bugzilla.yoctoproject.org/show_bug.cgi?id=13632

Autobuilder tests occasionally fail, reporting that a new logfile
could not be created. While this failure did occur multiple times, it
could not be manually reproduced. However, there are issues with the
implementation of the logrotate.py script that can be fixed. These
changes will help make the failures clearer, should they continue to
occur.

Previously, the test_2_logrotate test would, after running the
logrotate tool, use "ls -al $HOME/logrotate_dir | wc -l" to count
the number of files in the rotation directory and determine if the
rotation was successful. The test to see if there are at least three
files is problematic, because depending on the version of ls used, it
may report the target value of 3 even when there are only hidden files
in the directory, potentially reporting a pass for the test when it
should actually fail. An example with coreutils:

root@qemux86-64:~# ls -al emptydir/
total 2
drwxr-xr-x 2 root root 1024 Jan 14 19:50 .
drwx------ 3 root root 1024 Jan 14 19:50 ..
root@qemux86-64:~#

Where "total" is the number of blocks used. Compare with busybox ls:

root@qemux86-64:~# ls -al emptydir/
drwxr-xr-x    2 root     root          1024 Jan 14 19:54 .
drwx------    3 root     root          1024 Jan 14 19:54 ..
root@qemux86-64:~#

Instead of using ls to verify that a certain number of files exists
in $HOME/logrotate_dir, the tests have been changed to rotate two
specific logs: the log for wtmp and a new logrotate_testfile created
during the second test. Both tests check that the logs are correctly
rotated into $HOME/logrotate_dir by using find and grep on the
expected filename (e.g. "wtmp" when rotated becomes "wtmp.1", so we
check to see that wtmp.1 is present in $HOME/logrotate_dir). In
addition, should the test fail, the output from logrotate -vf is
included in the test log to aid debugging. It has also been seen that
in some cases, the logrotate test fails because the /var/log/wtmp file
is not yet present. Since the objective of the test is to check the
logrotate functionality and not the presence of certain log files,
test_logrotate_wtmp uses the touch command to help ensure that the
file is present before the call to logrotate is issued.

Finally, note that while the autobuilder failures that this patch
addresses were only seen during core-image-full-cmdline tests, these
changes were successfully tested on core-image-minimal and
core-image-sato with the manual addition of logrotate and openssh-sshd
to the images.

(From OE-Core rev: b9392403e96c6d880de844b44da2c7300fcf411c)

Signed-off-by: Trevor Gamblin <trevor.gamblin@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-05 12:00:34 +00:00
Armin Kuster
55bd462940 parselogs: remove mpc8315e-rdb from errors
(From OE-Core rev: 0de55be071efff60944c89582ae0e736f3bdc828)

Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-02 10:39:52 +00:00
Alexander Kanavin
f0028da1e6 ltp: update to 20190930
Drop a big pile of backports.

Add a patch to address absent string formatting error.

Additional fixing of builds with musl.

timers group has been merged upstream into syscalls.

(From OE-Core rev: 7126431cc3512e3d7f55300c561339c87283b605)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-16 22:33:09 +00:00
Alexander Kanavin
544cad3791 ptest: report ptests that couldn't be run at all
Currently if a ptest does not produce PASS or FAIL, but simply
errors out, this is not caught or reported; I think some ptests
may have silently regressed due to this.

(From OE-Core rev: 4b78b6571a3653d7d8e9dab9512bad5d39483988)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-10 21:18:21 +00:00
André Draszik
c6743c05a3 oeqa/runtime/ptest: fix detection of failed tests
Since commit d6065f136f6d ("oeqa/logparser: Various misc cleanups"),
7b17274c30 in poky, the ptest OEQA is unable to detect failures
in any of the test results.

The reason is that the test result string changed from 'fail' to
'FAILED', because the original mapping has been removed as part of
that commit, but the code in here is still trying to match against
the old string, resulting in no matches, i.e. everything is treated
as successful, even if it shouldn't be.

Update the OEQA ptest test to actually work again and report
failure if there was a failure.

Note that the ptest test is marked as @expectedfail, so even though
this test now again starts to fail, the overall OEQA test result is
not affected - but at least the overall OEQA test summary reflects
the correct status again.

In other words:
    RESULTS:
    RESULTS - ping.PingTest.test_ping: PASSED (0.26s)
    RESULTS - ptest.PtestRunnerTest.test_ptestrunner: PASSED (4.05s)
    RESULTS - ssh.SSHTest.test_ssh: PASSED (0.60s)
    SUMMARY:
    image-debug () - Ran 3 tests in 4.937s
correctly changes to:
    AssertionError: Failed ptests:
    {'dummytest': ['check_True_is_True', 'test_basic']}

    RESULTS:
    RESULTS - ping.PingTest.test_ping: PASSED (0.24s)
    RESULTS - ssh.SSHTest.test_ssh: PASSED (0.56s)
    RESULTS - ptest.PtestRunnerTest.test_ptestrunner: EXPECTEDFAIL (4.13s)
    SUMMARY:
    image-debug () - Ran 3 tests in 4.937s
instead and we see a summary of the ptest subtests that failed.

(From OE-Core rev: 08ff4b9195a9de91b8090173c5bd03a5ff263616)

Signed-off-by: André Draszik <git@andred.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-30 08:47:13 +00:00
Alexander Kanavin
06a0adbb25 sysklogd: update to 2.0.3
Drop all patches as they are no longer necessary with autotools,
fix problems that are addressed upstream, or (in case of no-vectorization.patch)
don't apply to the old version either, which means no one is using that hw target
anymore.

Remove custom systemd configs (provided by upstream), and custom
syslog config (also provided by upstream but not installed by default).

Adjust parselogs to not error out on not being able to find
System.map file during boot (that is packaged into kernel-dev
and is not normally installed).

License-Update: various tweaks; the license terms are the same.
(From OE-Core rev: dcf0bd14d78b0848f206eb35355f05bfbd51c02d)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-18 10:18:06 +00:00
André Draszik
969b9974c6 oeqa/runtime/apt dnf opkg: support running from within docker container
If the user specified an http port to use for serving
files, use that instead of a random one. At the same
time, have the http server bind to all interfaces.

Binding to the server_ip might not always be possible,
e.g. in the case of running bitbake / oeqa from within
a docker container. In this case, the ip address is valid
outside the container, but not inside, and hence can't
be bound to. So switch to simply binding to all interfaces.

(From OE-Core rev: da529e6522959c2024023e90de52b5e42f160596)

Signed-off-by: André Draszik <git@andred.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-16 23:25:49 +00:00
André Draszik
ae6c729a53 oeqa/runtime/context.py: support listening port in TEST_SERVER_IP
Similar to the existing possibility to specify a port in
TEST_TARGET_IP, allow TEST_SERVER_IP to also contain a
port.

The intention is for this port to be passed into e.g.
the http server from the apt / dnf / opkg tests, or
any other (custom) tests that might need the target to
connect to a service spawned by bitbake / oeqa, where
bitbake is actually running inside a docker container.

(From OE-Core rev: 9f401cd8b010ca613a151d0323b251f2243f399f)

Signed-off-by: André Draszik <git@andred.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-16 23:25:49 +00:00
Armin Kuster
5834f2ff44 oeqa/manual/compliance-test: move crashme to runtime
[v3]
remove fork12 from crashme. runs forever
missed in v2 resend

(From OE-Core rev: 5d32ff52a36bdcc6abe55b89bf81c3312a03160a)

Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-21 23:08:19 +00:00
Armin Kuster
5b5b75a5bf oeqa/manual/bsp-hw: move storage tests to runtime
move usb mount, r/w and umount tests to runtime
move SD card mount, r/w and umount tests to runtim

(From OE-Core rev: 246d73b35f6be7880d98274e87033a14cdced9ba)

Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-21 23:08:19 +00:00
Armin Kuster
7ad0876086 oeqa/runtime/boot: add reboot test
(From OE-Core rev: c8ddebee04a7c752986b407b98f4d0175f447d01)

Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-21 23:08:19 +00:00
André Draszik
f0f2a492f3 oeqa/runtime/context.py: ignore more files when loading controllers
When loading controllers as (external) modules, the code currently
tries to load all files ending with .py. This is a problem when
during development using an editor that creates a lock-file
in the same directory as the .py file, as the lock file is
typically called '.#xxxx.py'.
Python will try to load the lock file and fail miserably with
an exception:

    The stack trace of python calls that resulted in this exception/failure was:
    File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
         0001:
     *** 0002:do_testimage(d)
         0003:
    File: 'poky/meta/classes/testimage.bbclass', lineno: 114, function: do_testimage
         0110:    netstat -an
         0111:}
         0112:
         0113:python do_testimage() {
     *** 0114:    testimage_main(d)
         0115:}
         0116:
         0117:addtask testimage
         0118:do_testimage[nostamp] = "1"
    File: 'poky/meta/classes/testimage.bbclass', lineno: 294, function: testimage_main
         0290:
         0291:    # the robot dance
         0292:    target = OERuntimeTestContextExecutor.getTarget(
         0293:        d.getVar("TEST_TARGET"), logger, d.getVar("TEST_TARGET_IP"),
     *** 0294:        d.getVar("TEST_SERVER_IP"), **target_kwargs)
         0295:
         0296:    # test context
         0297:    tc = OERuntimeTestContext(td, logger, target, host_dumper,
         0298:                              image_packages, extract_dir)
    File: 'poky/meta/lib/oeqa/runtime/context.py', lineno: 116, function: getTarget
         0112:            # XXX: Don't base your targets on this code it will be refactored
         0113:            # in the near future.
         0114:            # Custom target module loading
         0115:            target_modules_path = kwargs.get('target_modules_path', '')
     *** 0116:            controller = OERuntimeTestContextExecutor.getControllerModule(target_type, target_modules_path)
         0117:            target = controller(logger, target_ip, server_ip, **kwargs)
         0118:
         0119:        return target
         0120:
    File: 'poky/meta/lib/oeqa/runtime/context.py', lineno: 128, function: getControllerModule
         0124:    # ImportError raised if a provided module can not be imported.
         0125:    @staticmethod
         0126:    def getControllerModule(target, target_modules_path):
         0127:        controllerslist = OERuntimeTestContextExecutor._getControllerModulenames(target_modules_path)
     *** 0128:        controller = OERuntimeTestContextExecutor._loadControllerFromName(target, controllerslist)
         0129:        return controller
         0130:
         0131:    # Return a list of all python modules in lib/oeqa/controllers for each
         0132:    # layer in bbpath
    File: 'poky/meta/lib/oeqa/runtime/context.py', lineno: 163, function: _loadControllerFromName
         0159:    # Raise ImportError if a provided module can not be imported
         0160:    @staticmethod
         0161:    def _loadControllerFromName(target, modulenames):
         0162:        for name in modulenames:
     *** 0163:            obj = OERuntimeTestContextExecutor._loadControllerFromModule(target, name)
         0164:            if obj:
         0165:                return obj
         0166:        raise AttributeError("Unable to load {0} from available modules: {1}".format(target, str(modulenames)))
         0167:
    File: 'poky/meta/lib/oeqa/runtime/context.py', lineno: 173, function: _loadControllerFromModule
         0169:    @staticmethod
         0170:    def _loadControllerFromModule(target, modulename):
         0171:        obj = None
         0172:        # import module, allowing it to raise import exception
     *** 0173:        module = __import__(modulename, globals(), locals(), [target])
         0174:        # look for target class in the module, catching any exceptions as it
         0175:        # is valid that a module may not have the target class.
         0176:        try:
         0177:            obj = getattr(module, target)
    Exception: ImportError: No module named 'oeqa.controllers.'

Simply ignore those when collecting the list of files to try
to load.

(From OE-Core rev: 682f223cf2e2dabe8cf60634b6779bb2d5e359bd)

Signed-off-by: André Draszik <git@andred.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-23 16:30:36 +01:00
André Draszik
ee9d2f4fcf oeqa/runtime/systemd: skip unit enable/disable on read-only-rootfs
This doesn't work on read-only-rootfs:
    AssertionError: 1 != 0 : SYSTEMD_BUS_TIMEOUT=240s systemctl disable avahi-daemon.service
    Failed to disable unit: File /etc/systemd/system/multi-user.target.wants/avahi-daemon.service: Read-only file system

This patch does two things:
1) Decorate the existing test to be skipped if the rootfs is
   read-only
2) add a new test to be executed only if the rootfs is
   read-only. This new test remounts the rootfs read-write
   before continuing to execute the existing test, making
   sure to clean up correctly after itself (remount r/o
   again).

(From OE-Core rev: ec856301887b1139f93aa54d04bed9b842357b4f)

Signed-off-by: André Draszik <git@andred.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-19 23:18:33 +01:00
André Draszik
568fd6ea47 oeqa/runtime/opkg: skip install on read-only-rootfs
Images can have package management enabled, but be
generally running as read-only. In this case, the
test fails at the moment with various errors due to
that.

Use the new @skipIfFeature decorator to also skip
this test in that case.

(From OE-Core rev: 84640e2b4daf4cf22c5b0324c22332f59e4d51e3)

Signed-off-by: André Draszik <git@andred.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-19 23:18:33 +01:00
André Draszik
077c3bbf2f oeqa/runtime/df: don't fail on long device names
When device names are long (more than 20 characters), the
df test will fail with an exception:
    self.assertTrue(int(output)>5120, msg=msg)
    ValueError: invalid literal for int() with base 10: ''
at least when busybox is in use.

The reason is that busybox breaks the line in that case:
    Filesystem           1K-blocks      Used Available Use% Mounted on
    /dev/disk/by-partuuid/8e991e5a-cebd-4f88-9494-c9db4f30cb02
                           1998672     87024   1790408   5% /
and the code tries to extract the fourth field from the
second line, which is empty of course.

df can be told not to break lines, though, using the -P
flag, which turns on the POSIX output format, and is
supported by busybox df and coreutils df:
    Filesystem           1024-blocks    Used Available Capacity Mounted on
    /dev/disk/by-partuuid/8e991e5a-cebd-4f88-9494-c9db4f30cb02   1998672     87024   1790408   5% /

(From OE-Core rev: 8c23c1476d0c64b9bc8806db03414fa914c1e658)

Signed-off-by: André Draszik <git@andred.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-19 23:18:33 +01:00
Kai Kang
ff4b1d7ba7 dnf.py: check busybox for case test_dnf_installroot
Check package busybox which is required by cases test_dnf_installroot
and test_dnf_installroot_usrmerge.

(From OE-Core rev: 0f09c5b7a9a66af825951354b436b69dd9a596a4)

Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-09 14:07:43 +01:00
Khem Raj
8753c00f7a oeqa: Recognise svlogd as another logger
This is provided by runit which another init system like systemd,
sysvinit, this lets oeqa run on systems which are using runit

(From OE-Core rev: e4eb3ca113985ab2be123eb5b5ea76764761df88)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-07 13:08:34 +01:00
Changqing Li
46448785a4 dnf.py: installroot support usrmerge
(From OE-Core rev: 65f8dfd81abcf48d472ee28c2cdae819fd87ef32)

Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-07 07:47:24 +01:00
Alexander Kanavin
58e85c60cd qemu: switch to '-vga std' emulated hardware from vmware/cirrus for x86/mips
This is the qemu default since qemu 2.2, is generally supported better,
and is recommended by upstream. It also has already been in use for arm/risc
and ovmf.

Additional information:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=13466
https://www.kraxel.org/blog/2014/10/qemu-using-cirrus-considered-harmful/

'-vga virtio' emulated hardware remains in use when virgl is enabled via a runqemu override.

Also, adjust the error whitelist, as there is a number of new messages
coming from the drivers that are not actual errors.

(From OE-Core rev: 73cb104f3307736f4922f2e0c9648f9b2d3b3b6b)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-01 22:33:07 +01:00
Khem Raj
e64e8aa725 oeqa/parselogs: grep for exact errors list keywords
Given grep expression looks for error strings regardless of the
semantics and this ends up it flagging things like

***********************
Log: /mnt/a/yoe/build/tmp/work/raspberrypi3_64-yoe-linux-musl/yoe-qt5-wayland-image/1.0-r0/target_logs/weston.log
-----------------------
Central error:                EGL_KHR_create_context_no_error EGL_KHR_fence_sync
***********************

here EGL_KHR_create_context_no_error is an API, therefore the check
should be tightened a bit so explicitly look for error words so such
false positives don't show up

This helps in fixing wrong matches e.g. on rpi3 dmesg have strings like
above

(From OE-Core rev: e54ecea0a7d309533973b31bdbda178d9af92632)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-14 17:33:49 +01:00
Khem Raj
6456bec6ce oeqa/buildgalculator.py: Add dependency on gtk+3
This helps in skipping this test on images which are not using gtk and
as per the gcalculator website it needs gtk+3 or gtk+2 and libglade as
the build first looks for gtk+3

(From OE-Core rev: 2cdb4b3697e02947352803a9db013d61b812c4fd)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-14 17:33:49 +01:00
Chen Qi
41bac92fdb oeqa/runtime/rpm: Move test_rpm_query_nonroot test case to RpmBasicTest
The test_rpm_query_nonroot test case was in RpmInstallRemoveTest.
But it should logically belong to RpmBasicTest. So move it there.

(From OE-Core rev: 506388a10a26613524602dcb4e630f216c6fee60)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-15 09:29:59 +01:00
Chen Qi
d0d4c079d1 oeqa/runtime/rpm: ensure no user process running before deleting user
In case of systemd, `su -c 'xxx' test1' via ssh will create
several processes owned by test1, e.g. /lib/system/systemd --user.

These processes are actually managed by user@UID.service
(e.g. user@1000.service). And such service is managed
automatically by systemd. In other words, it will be cleaned
up by systemd automatically.

So we need to wait for systemd to clean it up before trying to
use `userdel' to delete the user.

(From OE-Core rev: 9d398be42a69d25277b929d760aaed1679f3cd54)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-15 09:29:59 +01:00
Richard Purdie
897639d7dc oeqa/runtime/syslog: Add delay to test to avoid failures
On a loaded builder we've seen the log message not make it to the log file
before the ssh command completes. Add a short delay to try and ensure
this does happen. There is unforunately no way to flush syslog in all
cases we test.

(From OE-Core rev: 66322b689e46520647e2d94d5e3f3ce282a41247)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-27 12:20:36 +01:00
Richard Purdie
2e91163d59 oeqa/runtime/oesyslog: systemd syslog restart doesn't change pid
The systemd-journald process doesn't restart/change the way syslog
does, don't test/error in this case.

(From OE-Core rev: be48190dca0643df4881624d29c2eae453395919)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-27 12:20:36 +01:00
Richard Purdie
abfe4b3d5c oeqa/runtime/syslog: Improve test debug messages
Its useful to test whether the restart command returned an error code and
exit early from the test if so.

Also add different messages to tell if the syslog processes didn't
die or didn't restart.

(From OE-Core rev: f19e95b8571a0d8213c4dec0da056e3d243fbbd1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-27 12:20:36 +01:00
Jon Mason
3e2ab10159 oe_syslog.py: Handle syslogd/klogd restart race
syslogd and klogd can occasionally take too long to restart, which
causes tests to fail by starting before the log daemons are ready.  To
work around this problem, poll for up to 30 seconds on the processes to
verify the old ones are killed and the new ones are up and running.
Similarly, add checks for rsyslogd and systemd-journald to possibly
catch issues with those daemons.

[YOCTO #13379]

(From OE-Core rev: dc73872b828ea271678fa624c15199364a5cba9e)

Signed-off-by: Jon Mason <jdmason@kudzu.us>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-27 12:20:36 +01:00
Anuj Mittal
4782ccbbde runtime/cases/logrotate: make test more reliable
By default logrotate uses 'dateext' in logrotate.conf which results in a
date string being appended at the end of log name. In cases when a test
that installs configuration in logrotate.d is executed first (dnf for
example), it might result in errors when logrotate test is executed:

error: destination /var/log/dnf.librepo.log-20190528 already exists, skipping rotation
error: destination /var/log/dnf.rpm.log-20190528 already exists, skipping rotation
error: destination /var/log/hawkey.log-20190528 already exists, skipping rotation

Tweak the test to use it's own temporary configuration that doesn't
enable dateext.

Fixes [YOCTO #13363]

(From OE-Core rev: d0a30c1c57edc9a724658e2ed312b34eed3af5a9)

Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-19 22:13:39 +01:00
Tim Orling
716dff0b2f oeqa/runtime: add simple test for scons
This test simply compiles a hello world program using scons.

(From OE-Core rev: bf6e3f0f3a7a134e8e3cb16366ef01b8c956e4c8)

Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-08 16:01:41 +01:00
Richard Purdie
ffae400179 meta/lib+scripts: Convert to SPDX license headers
This adds SPDX license headers in place of the wide assortment of things
currently in our script headers. We default to GPL-2.0-only except for the
oeqa code where it was clearly submitted and marked as MIT on the most part
or some scripts which had the "or later" GPL versioning.

The patch also drops other obsolete bits of file headers where they were
encoountered such as editor modelines, obsolete maintainer information or
the phrase "All rights reserved" which is now obsolete and not required in
copyright headers (in this case its actually confusing for licensing as all
rights were not reserved).

More work is needed for OE-Core but this takes care of the bulk of the scripts
and meta/lib directories.

The top level LICENSE files are tweaked to match the new structure and the
SPDX naming.

(From OE-Core rev: f8c9c511b5f1b7dbd45b77f345cb6c048ae6763e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-09 16:31:55 +01:00
Richard Purdie
c7592b0147 oeqa: Drop OETestID
These IDs refer to testopia which we're no longer using. We would now use the test
names to definitively reference tests and the IDs can be dropped, along with their
supporting code.

(From OE-Core rev: 8e2d0575e4e7036b5f60e632f377a8ab2b96ead8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-09 16:31:55 +01:00
Armin Kuster
b49916c252 ltp_compliance: add new runtime
test runtimes in sec.
AIO: 14
MEM: 94
MSG: 89
SEM: 30
SIG: 194
THR: 399
TMR: 867
TPS: 23

(From OE-Core rev: e099204c34d1cf90ae4bb44fb0f56e72904c664a)

Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-29 14:16:30 +01:00
Armin Kuster
f45d63b484 ltp: add runtime test
This adds the framework for running ltp tests.

Here are some times:
math: 61
syscalls: 3957
dio: 18472
io: 29
mm: 551
ipc: 48
sched: 165
nptl: 46
pty: 37
containers: 52
controllers: 9625
filecaps: 27
cap_bounds: 27
fcntl-locktests: 29
connectors: 27
timers: 37
commands: 165
net.ipv6_lib: 30
input: 29
fs_perms_simple: 31
fs: 3476
fsx: 30
fs_bind: 28
fs_ext4: 28
cve: 675

(From OE-Core rev: 1ec677549d490a29a7c713610db4a56be7b55e26)

Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-29 14:16:30 +01:00
Naveen Saini
7519b0fc32 parselogs.py: ignore regulatory.db load failure messages for x86*
[YOCTO #13098]

(From OE-Core rev: 9c20f8c6e4bb5157550e347ffd6ef790eb6a1200)

Signed-off-by: Naveen Saini <naveen.kumar.saini@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-03 15:38:13 +00:00