Commit Graph

1799 Commits

Author SHA1 Message Date
Chen Qi
20cba32f2d sdk/buldgalculator.py: check against multilib for gtk+3
When determining whether to skip the test case, the check should be
done with consideration of multilib. Otherwise, we will meet the
following error when testing against lib32 environment.

  No package 'gtk+-3.0' found

(From OE-Core rev: ce82ee46f4a7beb5663238b276e779e5c9657777)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-04 11:03:55 +01:00
Chen Qi
a7e6de1716 sdk/context.py: add ability to check for multilib version of target package
Add a named argument 'multilib' for the hasTargetPackage function. Its default
value is False. When setting to True, it will try to get the correct multilib
prefix from the sdk_env, the environment setup script.

We need this because we don't want unexpected run of some sdk test cases.
The following steps will generate error.

1. Enable multilib for qemux86-64
   require conf/multilib.conf
   MULTILIBS ?= "multilib:lib32"
   DEFAULTTUNE_virtclass-multilib-lib32 ?= "core2-32"
2. bitbake core-image-sato -c populate_sdk
3. bitbake core-image-sato -c testsdk

The error message is like below.

  No package 'gtk+-3.0' found
  RESULTS - buildgalculator.GalculatorTest.test_galculator - Testcase -1: FAILED

As we don't have lib32-gtk+3 installed, the test case should be skipped when
testing against the lib32 environment setup script.

(From OE-Core rev: 163764ad4760a5fabf65640df5f968be98ad13d2)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-04 11:03:55 +01:00
Alexander Kanavin
5e3a07180b lib/oe/package_manager: turn postinst failure warnings into bitbake failures
Sumo release provides a transition period so that deferrals to first boot
via 'exit 1' can be converted to pkg_postinst_ontarget(). For the next release
however, postinst script failures should be treated as such.

[YOCTO #12607]

(From OE-Core rev: 42acb0ebde4e88bcdf34a541b700f19d8607abb0)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-04 11:03:55 +01:00
Hongxu Jia
1c048479af selftest/package: Add package separated debug symbols hardlink test
Tweak recipe selftest-hardlink
- addition of libexecdir to simulate multiple directories
- add gdb.sh to run gdb from script which is invoked at test time.
- rename `hello' -> `hello1' to workaround name confliction with the one in lmbench

Add test_gdb_hardlink_debug to selftest/package
- run a qemu and invoke gdb.sh to gdb binaries of selftest-hardlink
- check gdb to read symbols from separated debug hardlink file
- check debug symbols works correctly

[Test without commit `package.bbclass: only one hardlink of separated debug info file in each directory']
2018-08-26 01:27:30,195 - oe-selftest - INFO - test_gdb_hardlink_debug (package.PackageTests)
2018-08-26 01:30:29,005 - oe-selftest - INFO - gdbtest /usr/bin/hello1
2018-08-26 01:30:36,539 - oe-selftest - INFO - gdbtest /usr/bin/hello2
2018-08-26 01:30:43,568 - oe-selftest - INFO - gdbtest /usr/libexec/hello3
2018-08-26 01:30:50,157 - oe-selftest - ERROR - No debugging symbols found. GDB result:
Reading symbols from /usr/libexec/hello3...(no debugging symbols found)...done.^M
(gdb) Function "main" not defined.^M
Make breakpoint pending on future shared library load? (y or [n]) [answered N; input not from terminal]^M
(gdb) Starting program: /usr/libexec/hello3 ^M
Hello World!^M
[Inferior 1 (process 320) exited normally]^M
(gdb) The program is not being run.^M
(gdb)
2018-08-26 01:30:51,180 - oe-selftest - INFO -  ... FAIL
2018-08-26 01:30:51,181 - oe-selftest - INFO - Traceback (most recent call last):
  File "oe-core/meta/lib/oeqa/selftest/cases/package.py", line 148, in test_gdb_hardlink_debug
    self.fail('GDB %s failed' % binary)
AssertionError: GDB /usr/libexec/hello3 failed
[Test without commit `package.bbclass: only one hardlink of separated debug info file in each directory']

[Test with commit `package.bbclass: only one hardlink of separated debug info file in each directory']
2018-08-26 12:40:30,976 - oe-selftest - INFO - test_gdb_hardlink_debug (package.PackageTests)
2018-08-26 12:42:15,149 - oe-selftest - INFO - gdbtest /usr/bin/hello1
2018-08-26 12:42:24,064 - oe-selftest - INFO - gdbtest /usr/bin/hello2
2018-08-26 12:42:31,078 - oe-selftest - INFO - gdbtest /usr/libexec/hello3
2018-08-26 12:42:38,646 - oe-selftest - INFO - gdbtest /usr/libexec/hello4
2018-08-26 12:42:46,824 - oe-selftest - INFO -  ... ok
[Test with commit `package.bbclass: only one hardlink of separated debug info file in each directory']

(From OE-Core rev: 104d07e57488f4a414fb5e1f60d0c8b0c02d6b4d)

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-28 10:30:28 +01:00
Richard Leitner
a0de8a6fb8 oeqa/core/decorator: add skipIfInDataVar
skipIfInDataVar will skip a test if a value is in a certain variable.

(From OE-Core rev: 10b935c713748346aea6c36c2f41e0ae6c320821)

Signed-off-by: Richard Leitner <richard.leitner@skidata.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-23 18:02:23 +01:00
Richard Purdie
652d25c09d oeqa/runtime/ldd: Clean up test
* Merge the two tests together as having them separate is pointless
* Test that ldd runs correctly
* Add in a dependency on the "ldd" package being installed instead of
  the sdk tools feature

(From OE-Core rev: 80db456387fb63ee74d53a9719ab3997432f4c80)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-23 07:50:00 +01:00
Richard Purdie
b45f3ecc00 oeqa/runner: Print any errors/failures early
Its a pain to have to wait until oe-selftest finishes to see the
failures for example.

(From OE-Core rev: 4c499a1b10a0c2647b6a753b8f9cd934ae4ad0da)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-23 07:50:00 +01:00
Richard Purdie
af87cc83cf oeqa/runner: Use the proper logger functions instead of print()
(From OE-Core rev: 5ccd2284e5dd994230e9e229b7931d049c9f46c0)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-23 07:50:00 +01:00
Richard Purdie
f2288adb57 oeqa/context: Only set buffer mode for non-concurrent tests
Periodically we'd see:

NOTE: core-image-sato-1.0-r0 do_testsdk: ======================================================================
NOTE: core-image-sato-1.0-r0 do_testsdk: ERROR: broken-runner
NOTE: core-image-sato-1.0-r0 do_testsdk: ----------------------------------------------------------------------
NOTE: core-image-sato-1.0-r0 do_testsdk: testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-mips/build/meta/lib/oeqa/core/utils/concurrencytest.py", line 122, in _run_test
    test.run(process_result)
  File "/usr/lib/python3.6/site-packages/subunit/__init__.py", line 1194, in run
    protocol = TestProtocolServer(result, self._passthrough, self._forward)
  File "/usr/lib/python3.6/site-packages/subunit/__init__.py", line 514, in __init__
    stream = stream.buffer
AttributeError: '_io.StringIO' object has no attribute 'buffer'

which seems to occur if a result arrives before all the runner threads
have started. The runner's result handling changes sys.stdout to a buffer
temporarily which can be seen in other threads and it can sometimes fail.

Since the tests are running in a separate process we don't need this buffer
handling in the concurrent case so only set when not parallelising. The
concurrent class handle setting buffer mode internally.

(From OE-Core rev: e2c66aa0f74d19fbc7e56af1fb440ad6473d3a8e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-23 07:50:00 +01:00
Richard Purdie
c2e589ee0d oeqa/concurrencytest: Ensure subunit streams are flushed at exit
Without this, error output such as that in the teardown can be lost
and processes may recieve signals they're not expecting causing other
strange errors.

(From OE-Core rev: 1e3f44737a15feb3128ba7fc0dbe896dd8782e07)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-23 07:50:00 +01:00
Chen Qi
920e9a4ac4 assimp.py: fix AttributeError in tearDownClass
When running this test case, we will see the following error.

  AttributeError: type object 'BuildAssimp' has no attribute 'project'

assimp.py test case does not make use of SDKBuildProject, so remove
the import statement and the tearDownClass.

(From OE-Core rev: ca0a40a852abed981d54503ef2d86708471c821e)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-21 14:49:42 +01:00
Bruce Ashfield
9af0f1a46b kernel-devsrc: restructure for out of tree (and on target) module builds
The existing kernel-devsrc package starts with a full copy of the kernel
source and then starts to strip out elements that are not required.

This results in extra time (I/O) and extra space being taken up in the
final package. The main purpose of the kernel-devsrc package has been to
build modules against the running kernel, not to include a full copy of
the source code for re-building the kernel. The end result was a
600M kernel-devsrc package.

This restructuring of the package uses an approach similar to other
distros, where the kernel-devsrc package is for building against the
running kernel and uses a curated set of copied infrastructure, versus
a mass copy of the entire kernel.

The differences in this approach versus other is largely due to the
architecture support and the split build/source directory of the
kernel.

The result is a kernel-devsrc package of about 10M, which is capable
of running "make scripts" and compiling kernel modules against the
running kernel.

Along with the changes to the copying of the infrascture, we also
have the following changes:

 - a better/more explicit listing of dependencies for on-target
   builds of "make scripts" or "make modules_prepare"

 - The kernel source is installed into /lib/modules/<version>/build
   and a symlink created from /usr/src/kernel to the new location.
   This aligns with the standard location for module support
   code

 - There is also a symlink from /lib/modules/<version>/source -> build
   to reserve a spot for a new package that is simply the kernel
   source. That package is not part of this update.

(From OE-Core rev: 007ef1f468110b2698a27ea9a6d43fed5a0a9fc2)

Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-19 16:35:22 +01:00
Hongzhi.Song
704e725bba runtime selftest: limit kernel hw bp arches
1. So far, only qemux86[-64] support hw breakpoint, no matter whether or
not with kvm.

qemuppc: The oe-core configuration uses a PPC G4 system as the
default cpu but qemu doesn't simulate the hw breakpoint register for G4.

qemuarm: The arch more than v7 supports hw breakpoint, however arm use
v5 as default.

qemuarm64: We temporarily drop qemuarm64 for the moment. Normally it
will print debug info once, but endlessly when we trigger the break
point. Now it is hard to located the issue, but we will confirm it
later.

qemumips*: Kernel dosen't support hw bp for mips.

2. Syslog maybe not started, so we use dmesg to confirm.

3. Running 'ls' to trigger the hardware breakpoint test.

(From OE-Core rev: 176e50fb177dc40e439700d5f3f838dd7eaaa427)

Signed-off-by: Hongzhi.Song <hongzhi.song@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-16 22:40:28 +01:00
Hongzhi.Song
742a801536 runtime selftest: test_trace_events_sample of ksample.py
The debug info can not be written to trace buffer immediately
by thread. So we should sleep some seconds.

(From OE-Core rev: 5ccf8150b4c65f2feac16cc2881188f14d9e5390)

Signed-off-by: Hongzhi.Song <hongzhi.song@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-16 22:40:28 +01:00
Richard Purdie
79d72d9045 selftest/package: Improve test to cover sparseness and hardlinking from sstate
The sparseness test was sometimes working and sometimes failing depending
on whether sstate was valid. This adds an explict test of sstate
to the test for both hardlinking and sparseness. Tweak the test name to
cover the fact its tests sparseness too.

(From OE-Core rev: fe5b37c07b6d07c350516ab6bf849d6d86a84004)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-15 09:44:33 +01:00
Richard Purdie
899a3694de selftest: Replace bitbake -p with bitbake -e
Parsing all the recipes is annoying when trying to re-execute oe-selftest
and also unnecessary as its really just a sanity check. When the tests were
originally being developed the guard was useful but less so now.

Replace it with bitbake -e which is fast and checks the basic configuration
is valid.

(From OE-Core rev: acec5180b8d2731002979179e08439b615631e70)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-15 09:44:33 +01:00
Richard Purdie
26a786f869 package_manager/sdk: Use filtered copies of the deploy ipk/deb directories
Similar to rpm, use copies of the ipk/deb directories for rootfs construction.
This means the image creation code can no longer "see" recipes wich aren't in its
dependency chain which is good for a variety of reasons including determinism,
incompatible recipe (e.g. systemd/sysvinit) package conflicts and locking
performance.

(From OE-Core rev: c7c5f4065c102fde4e11d138fb0b6e25bffe0379)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-15 09:44:33 +01:00
Richard Purdie
20c8be711f selftest/package: Add test to ensure sparse files are preserved
Add a new element to the hardlink test to check we also preseve file
sparseness during the packing process. This should ensure we don't regress this
issue again.

(From OE-Core rev: 0a4e6974b49bf68c4a4098d339b5d655e202a3fd)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-15 09:44:33 +01:00
Chen Qi
fd85436b04 parselogs.py: output correct log location
The log entry in results is altered to remove 'target_logs'. This
causes wrong log location in output.

e.g.
   AssertionError: 1 != 0 : Log: /path/to/image/1.0-r0/postinstall.log

But when user wants to check the log, the user will find the log is
not present. The actual log file is /path/to/image/1.0-r0/target_logs/postinstall.log.

So fix to use the correct log location.

(From OE-Core rev: df3a2e21648d6b649ebda7e6032afbd63c939f2b)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-14 11:36:31 +01:00
Richard Purdie
b5970ab651 oeqa/esdk/devtool: Drop OETestDepends usage
OETestDepends doesn't work with parallelism and in this case we don't
really need this dependency, it would just short out some tests quickly
in the rare case the esdk environment was broken.

Currently this is masking tests which is a much worse problem and we
can't make OETestDepends work reliably with parallelism so drop the
dependencies.

(From OE-Core rev: b3b1e1881240b8e2a32dd5c1dc3b7387f0819576)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-14 11:36:31 +01:00
Ross Burton
37c4f89d71 oeqa/sdk/buildgalculator: check for nativesdk-gettext-dev
We don't need target gettext to build, but nativesdk-gettext-dev (for nls.m4).

(From OE-Core rev: 0474326d79b7675dabe63f691733e8c6b24b2fb0)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-14 11:36:31 +01:00
Paulo Neves
4a127a70a2 context.py: Do not mask exceptions needlessly.
There were a lot of assumptions in the controller import
code of context.py which were not true anymore.  These
assumptions reflected themselves by catching exceptions and
re-raising them with unhelpful error messages in other parts
of the code.

This commit does not fix the classes controller classes that
became broken after the refactor but at least it allows for
the exceptions to be thrown where the imports fail, thus
actually showing what exactly went wrong with the import.

An example of such an improvement is that before if the
controller class failed during it's init contructor the
controller would just be skipped and the task would just
complain it could not find the controller.  Now for example,
if there is a NamerError due to a variable not being
declared, the user will get that report.

(From OE-Core rev: 719b0e88b875b067551067874778061f99b8c4b4)

Signed-off-by: Paulo Neves <ptsneves@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-08 10:52:00 +01:00
Chen Qi
e77d45609a oe_syslog.py: fix for syslog-ng
When using syslog-ng as the syslog provider, oe_syslog test case fails
because it cannot find the syslog daemon. This is because it greps for
'syslogd' but syslog-ng's daemon is 'syslog-ng'. So fix it to check both
'syslogd' and 'syslog-ng'.

Also, when the test case fails, what I get is:
| AssertionError: 1 != 0 : No syslogd process; ps output:
<empty here>

This does not help user. The output is actually from the 'PS | GREP' command.
And when the 'PS | GREP' command fails, the output is always empty. So also fix
this problem. After the change, it looks like:
| AssertionError: False is not true : No syslog daemon process; ps output:
|   PID USER       VSZ STAT COMMAND
|     1 root     16476 S    {systemd} /sbin/init
|     2 root         0 SW   [kthreadd]
|     3 root         0 IW   [kworker/0:0]
...

(From OE-Core rev: b180fbd9d93db1c6351e183fbc08fc81cdc240c0)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-08 10:51:59 +01:00
Richard Purdie
fc77dc63fe selftest/package: Fix hardlink test when using sstate
The build target won't cause the package task to run if things were already
built and in sstate. Ensure we run the package task explicitly to ensure
the test works as intended.

(From OE-Core rev: be20eb4f4ad77d9444028ca9632b74d5866d96b8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-07 12:13:03 +01:00
Ross Burton
b152f84a73 oeqa/sdk: add test that CMake works
Add a new SDK testcase that builds assimp, a project that uses cmake.

Using TARGET_ARCH and TARGET_OS which is now exported into the environment,
check that the generated binaries match the ELF headers we expect.

(From OE-Core rev: b4acfa11b35b47c86d2d83d7b0693284a8dc7495)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-07 12:13:03 +01:00
Paul Eggleton
d9a9a2d80e oe-selftest: devtool: avoid parallel races by using temporary copy of core
Some of the devtool tests make changes to files under meta/ -
legitimately since we want these tests to be working with real recipes
and associated files. Unfortunately with the new oe-selftest
parallelisation this can break other tests if files go missing at the
wrong time (among other scenarios). To avoid this issue, simply take a
copy of the core repository and use that for these tests. (We copy the
entire repository since changing the path of meta/ influences COREBASE
and thus we need to have things like scripts/ alongside as well).

(From OE-Core rev: 2457cd57b4195924ef127f497efa2f34f411e660)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-31 22:47:36 +01:00
Chen Qi
e8143566cc systemd.py: increase default bus timeout
Use SYSTEMD_BUS_TIMEOUT to set default timeout to 240s to avoid
timeout problem on slow qemu machines.

(From OE-Core rev: a5b1dcabdd84915eb0527be4e7bd64ab46860d9d)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-31 22:47:36 +01:00
Chen Qi
fc2eea3d28 parselogs.py: ignore network interface name changing failure
The following error from systemd-udevd is not harmful. It's just
because our qemu targets are using eth0.

  Error changing net interface name 'eth0' to 'enp0s3': Device or resource busy

Note that systemd is using a different network interface naming scheme
from traditional ethX naming scheme. To make this error message go away,
we could symlink /etc/udev/rules.d/80-net-setup-link.rules to /dev/null
to recover traditional naming scheme. But I'm not sure if this will
cause regression in user experience for systemd users of OE.

So just ignore this error message so that parselogs.py test case does not
fail.

(From OE-Core rev: f1735fed088ddda6517fa4ff6fbd6ef683b14878)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-31 22:47:36 +01:00
Chen Qi
b195bff204 oe_syslog.py: fix settings in OEHasPackage decorator
1. For test_syslog_running, we should not restrict it to run for
   only busybox-syslog and sysklogd. So extend it to all syslog
   providers in oe-core and meta-openembedded.

2. For test_syslog_startup_config, fix to make it depend on the
   existence of busybox-syslog. The previous condition "!sysklogd
   && busybox" is incorrect, because busybox may be compiled without
   syslog support, and sysklogd and busybox-syslog conflict with
   each other.

(From OE-Core rev: 119c22342d446e4f638b8d4c81480ebc7b444245)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-31 22:47:35 +01:00
Richard Purdie
318595754b oeqa/selftest: Add package hardlink test
We keep breaking the preservation of hardlinks during the packaging process.
Add a selftest which tests this to try and prevent this breaking again.

(From OE-Core rev: 751fc7802f57a836a0be5fc6a8d5fdf8b1769c39)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-31 22:47:35 +01:00
Anuj Mittal
9f7923c7e0 oeqa/selftest: check if rm_work is enabled
rm_work if enabled leads to some tests failing that rely on artifacts
being present. Check if rm_work.bbclass is included and show an error
and exit if it is.

Fixes [YOCTO #12694]

(From OE-Core rev: dde7e2f590834aa8034f1371954f9b4fbc7a60b7)

Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-30 12:44:35 +01:00
Ross Burton
9de268dd60 oeqa/runtime/scanelf: remove
These tests are intended to search for bad RPATHs and text relocations, but we
do these tests at buildtime and as pax-utils is never installed in any default
images the tests are never executed.

(From OE-Core rev: ff81b58d5f322ea4a24c1b9ed40377c742459149)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-30 12:44:35 +01:00
Richard Purdie
d9087c7c47 selftest/wic: Fix/cleanup class setup code
The code was using Wic instead of WicTestCase for image_is_ready and
native_sysroot which was confusing some tests. Fix image_is_ready.

For native_sysroot, convert the only user to just run the call itself
as there is no point in executing this for all the other tests which
don't use the value.

Whilst here, move _get_image_env_path to the base class alongside
its parent variabe and fix it to use WicTestCase.

(From OE-Core rev: 3e83364ed7031d53f281216214b04c7acace844a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-30 12:44:34 +01:00
Richard Purdie
1d4760a47f oeqa/loader: Ensure invalid test names don't trigger a traceback
oe-selftest -r <invalid test name>

currently triggers a traceback. Ensure this doesn't happen and the user gets a
sensible error message.

(From OE-Core rev: f2042bf3638ed4edfb167e7f7d4be6da60997ead)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-26 17:14:42 +01:00
Robert Yang
c467bc8569 qemurunner.py: fix is_alive() to avoid confusing with recycled pid
[YOCTO #12493]

Fixed:
- qemu started with pid 10000
- qemu exited unexpectedly
- The pid 10000 is re-used by another different process.

The is_alive() returned True in such a case because both qemu_pidfile and
/proc/10000 exist, but it's another process, this patch fixed the problem.

(From OE-Core rev: 78033af85698b4e112ae78e3140b241b1b3c06a3)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-26 13:16:41 +01:00
Ross Burton
5b6fbaf727 oeqa/sdk/sdkbuildproject: use os.path.join
Instead of mushing two paths together and hoping that the slashes line up
correctly, use os.path.join.

(From OE-Core rev: ed0413babaf10ccf729472f603bbe32d84ba1931)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-26 13:16:40 +01:00
Ross Burton
19905c78e7 oeqa: rationalise Perl tests
As with the Python test, this can be both better and faster.  No need to copy a
file, just run a one-liner.

(From OE-Core rev: c6eef46747fe58bb2310be4f06d2fa9b67901d72)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-26 13:16:40 +01:00
Chen Qi
869e501544 logging: use warning instead warn
The warn method is deprecated. We should use the documented warning instead.

Quoting from the python's official doc:
"""
Note: There is an obsolete method warn which is functionally identical to warning.
      As warn is deprecated, please do not use it - use warning instead.
"""

(From OE-Core rev: f467fd277eb77336097cfc0f5f329bdc8d0f70cb)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-26 13:16:40 +01:00
Richard Purdie
1b7eccff64 selftest/bbtests: Add test for bitbake execution outside the build directory
Also fix a related test's error message to match what it does.

(From OE-Core rev: 662be652ed39f32c3c7bee6539a9e91af191557e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-25 16:48:27 +01:00
Richard Purdie
ce76b71666 selftest/bblayers: Place the test layer directory in builddir
Placing the layer in meta means the directory is in an unclean state
which may influence other tests. Use our build directory instead
since we 'own' that. This helps keep oe-selftest parallelisation
clean.

(From OE-Core rev: f2d32bdf3f27a9ef62fe3abb7fdd2e04ab84b5f3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-25 16:48:27 +01:00
Richard Purdie
b84b073396 selftest/wic: clean up runCmd error code handling
runCmd handles checking exit 0 internally unless specified otherwise and its
error messages are much more useful including the failed command output.

Clean up the unneeded 0 exit code checks in the wic tests.

(From OE-Core rev: 394f7d4efe80e390e2c9b0be5e6d10954395a82b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-25 16:48:27 +01:00
Richard Purdie
e3dcd61337 selftest/wic: Improve error message for test_fixed_size
Currently this can fail with a message like 127 != 0 which is unhelpful.

If we remove the ignore_status=False, the debugging from runCmd is much
more helpful printing status.output.

Also remove the now unneeded exit code check.

(From OE-Core rev: 1aa7471b11aedc68de5116c461fe73152e3985fd)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-25 16:48:27 +01:00
Paul Eggleton
5aef7cd919 oe-selftest: fix for changes to buildhistory-diff
Now we're not reporting the related fields (as of openembedded-core
8658b3677b9f7cb70806061c41570c709086ef05) we shouldn't expect to see
PR reported here since it's not monitored by buildhistory-diff. However,
with a bit of messing about we can check for the exact output that we
should now see as a result of the test changing PR to go backwards.

(From OE-Core rev: f9c233908e6ada208d7abfdfe3ff98c6b278f405)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-24 11:52:27 +01:00
Richard Purdie
6d66b57409 utils: Add multiprocess_launch API and testcase
The current methods of spawning processes for parallel execution have
issues around collection of results or exceptions.

Take the code from package_ipk/deb, make it generic, add a results
collection mechanism, fix the exception handling and for it into a
standard library function.

Also add a test case which tests both the success and failure modes
of operation to stop this functionality regressiing again.

In particular, compared to multiprocess_exec, this fork off the parent
approach means we can pass in the datastore and functions work in the
same scope as the parent. This removes some of the complexities
found trying to scale multiprocess_exec to wider use.

(From OE-Core rev: 88f0c214e593a45566df5131bda4c946f5ccc8c2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-24 11:52:27 +01:00
Ross Burton
ebad780985 oeqa/sdk/python: clean up Python test
For the same reasons as the runtime Python test, clean up the SDK test.

Also port from Python 2 to Python 3, as that's what is supported now.

(From OE-Core rev: bead742a3ffc0a53162fb0c36610d74a1422e7b3)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-18 10:18:42 +01:00
Ross Burton
731ce8c760 oeqa/runtime/python: clean up Python test
Currently this is three test cases:

1) test_python_exists.  Fail if python3 isn't in PATH.
2) test_python_stdout.  Run a Python script and check the output is as expected
3) test_python_testfile.  Check that a file test_python_stdout wrote to exists.

(1) should be a setup and skip the test module if it isn't present.
(2) and (3) should be merged, there's no point copying over a two line Python
file, and the test doesn't verify that the file doesn't exist in the first
place.

Rewrite the test to check that Python is present in a class setup so the entire
test is skipped if it isn't and do some simple rot13 to verify that bytecode is
being executed correctly.

(From OE-Core rev: a35be5f32b4fe70b18ac1e2eccfd94558cecfbba)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-18 10:18:42 +01:00
Anuj Mittal
ba598c7db3 oeqa/parselogs: remove references to BSPs that no longer exist
These platform specifc BSPs were removed from meta-intel and superseded by
intel-core* BSPs.

(From OE-Core rev: a6fd20fc57da3e1910e775f5994bd71bf79934ea)

Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-18 10:18:41 +01:00
Richard Purdie
928d6159ed oeqa/decorator: Improve reliability
Checking if the dependency had any failure is unreliable, for example
if the underlying data doesn't get transferred and the list is empty,
success of the dependency is assumed.

Since we now have success data available, change the code to use it.

(From OE-Core rev: 4abba4c30d5a6163a968a119395a679e5e281ab4)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-18 10:18:41 +01:00
Richard Purdie
da596a2ad6 testsdk: Enable multiprocess execution
This uses the new concurrenttest code to enable parallel test execution
if specified.

(From OE-Core rev: 07d19fb3adab7a8d83ba83d9a16395f70d7b7a47)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-18 10:18:41 +01:00
Richard Purdie
f5975278e3 oeqa: Remove xmlrunner
This isn't present on modern distros by default and doesn't work with
testtools, needing multiple code paths in the code. Remove it in favour
of finding a better replacement for results collection/analysis.

(From OE-Core rev: 8001d933a8dc86004db014777f094d718086687d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-18 10:18:41 +01:00