Parsing all the recipes is annoying when trying to re-execute oe-selftest
and also unnecessary as its really just a sanity check. When the tests were
originally being developed the guard was useful but less so now.
Replace it with bitbake -e which is fast and checks the basic configuration
is valid.
(From OE-Core rev: acec5180b8d2731002979179e08439b615631e70)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Similar to rpm, use copies of the ipk/deb directories for rootfs construction.
This means the image creation code can no longer "see" recipes wich aren't in its
dependency chain which is good for a variety of reasons including determinism,
incompatible recipe (e.g. systemd/sysvinit) package conflicts and locking
performance.
(From OE-Core rev: c7c5f4065c102fde4e11d138fb0b6e25bffe0379)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This function is generic, tweak the variable names and move out the rpm specific
directory name to make it truly generic and reusable for deb/ipk.
(From OE-Core rev: dba876639b1fb8ea3ccb182c91e19966c4052115)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Refactor the locale archive function from the SDK to also make it work during
general image creation. This reduces the size of the locales from 900MB to 220MB
in core-image-lsb-sdk.
The exception handling around subprocess was dropped as the standard subprocess
exception printing is better handled than the catchall exception.
(From OE-Core rev: 8ffd93bdb09b0a4a84b27dafcd684c6abba392ed)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a new element to the hardlink test to check we also preseve file
sparseness during the packing process. This should ensure we don't regress this
issue again.
(From OE-Core rev: 0a4e6974b49bf68c4a4098d339b5d655e202a3fd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The log entry in results is altered to remove 'target_logs'. This
causes wrong log location in output.
e.g.
AssertionError: 1 != 0 : Log: /path/to/image/1.0-r0/postinstall.log
But when user wants to check the log, the user will find the log is
not present. The actual log file is /path/to/image/1.0-r0/target_logs/postinstall.log.
So fix to use the correct log location.
(From OE-Core rev: df3a2e21648d6b649ebda7e6032afbd63c939f2b)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
OETestDepends doesn't work with parallelism and in this case we don't
really need this dependency, it would just short out some tests quickly
in the rare case the esdk environment was broken.
Currently this is masking tests which is a much worse problem and we
can't make OETestDepends work reliably with parallelism so drop the
dependencies.
(From OE-Core rev: b3b1e1881240b8e2a32dd5c1dc3b7387f0819576)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
- Prefer collections.abc (new in Python 3.3) over collections for abstract base classes
- In Python 3.8, the abstract base classes in collections.abc will no longer be exposed in
the regular collections module. This will help create a clearer distinction between
the concrete classes and the abstract base classes."
- https://docs.python.org/3.7/whatsnew/3.7.html#deprecated
- see c66f9f8d39
(From OE-Core rev: e763151e1f7cfe9ea56de06f41769f8a3d74d219)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't need target gettext to build, but nativesdk-gettext-dev (for nls.m4).
(From OE-Core rev: 0474326d79b7675dabe63f691733e8c6b24b2fb0)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Executing strip action on kernel modules removes the signature.
Is not possible to strip and keep the signature, therefore avoid
strip signed kernel modules.
(From OE-Core rev: 4c47e5f171fa2603355e2f9183065ce8137a18c7)
Signed-off-by: Omar Ocampo <omar.ocampo.coronado@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There were a lot of assumptions in the controller import
code of context.py which were not true anymore. These
assumptions reflected themselves by catching exceptions and
re-raising them with unhelpful error messages in other parts
of the code.
This commit does not fix the classes controller classes that
became broken after the refactor but at least it allows for
the exceptions to be thrown where the imports fail, thus
actually showing what exactly went wrong with the import.
An example of such an improvement is that before if the
controller class failed during it's init contructor the
controller would just be skipped and the task would just
complain it could not find the controller. Now for example,
if there is a NamerError due to a variable not being
declared, the user will get that report.
(From OE-Core rev: 719b0e88b875b067551067874778061f99b8c4b4)
Signed-off-by: Paulo Neves <ptsneves@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using syslog-ng as the syslog provider, oe_syslog test case fails
because it cannot find the syslog daemon. This is because it greps for
'syslogd' but syslog-ng's daemon is 'syslog-ng'. So fix it to check both
'syslogd' and 'syslog-ng'.
Also, when the test case fails, what I get is:
| AssertionError: 1 != 0 : No syslogd process; ps output:
<empty here>
This does not help user. The output is actually from the 'PS | GREP' command.
And when the 'PS | GREP' command fails, the output is always empty. So also fix
this problem. After the change, it looks like:
| AssertionError: False is not true : No syslog daemon process; ps output:
| PID USER VSZ STAT COMMAND
| 1 root 16476 S {systemd} /sbin/init
| 2 root 0 SW [kthreadd]
| 3 root 0 IW [kworker/0:0]
...
(From OE-Core rev: b180fbd9d93db1c6351e183fbc08fc81cdc240c0)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The build target won't cause the package task to run if things were already
built and in sstate. Ensure we run the package task explicitly to ensure
the test works as intended.
(From OE-Core rev: be20eb4f4ad77d9444028ca9632b74d5866d96b8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a new SDK testcase that builds assimp, a project that uses cmake.
Using TARGET_ARCH and TARGET_OS which is now exported into the environment,
check that the generated binaries match the ELF headers we expect.
(From OE-Core rev: b4acfa11b35b47c86d2d83d7b0693284a8dc7495)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch enables multiconfig dependencies (mcdepends) to be used on
recipes using the following format:
task[mcdepends] = "multiconfig:FROM-MC:TO-MC:PN:task-to-depend-on"
For the sake of simplicity consider the following example:
Assuming we have set up multiconfig builds, one for qemux86 and one for
qemuarm, named x86 and arm respectively.
Adding the following line to an image recipe (core-image-sato):
do_image[mcdepends] = "multiconfig:x86:arm:core-image-minimal:do_rootfs"
Would state that core-image-sato:do_image from x86 will depend on
core-image-minimal:do_rootfs from arm so it can be executed.
This patch makes modifications to bitbake.conf to enable mcdepends, and
to sstatesig and staging.bbclass to avoid conflicts between packages from
different multiconfigs.
[YOCTO #10681]
(From OE-Core rev: f71bfe833c657244d2fd07b3b71e86081d7d1c04)
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some of the devtool tests make changes to files under meta/ -
legitimately since we want these tests to be working with real recipes
and associated files. Unfortunately with the new oe-selftest
parallelisation this can break other tests if files go missing at the
wrong time (among other scenarios). To avoid this issue, simply take a
copy of the core repository and use that for these tests. (We copy the
entire repository since changing the path of meta/ influences COREBASE
and thus we need to have things like scripts/ alongside as well).
(From OE-Core rev: 2457cd57b4195924ef127f497efa2f34f411e660)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use SYSTEMD_BUS_TIMEOUT to set default timeout to 240s to avoid
timeout problem on slow qemu machines.
(From OE-Core rev: a5b1dcabdd84915eb0527be4e7bd64ab46860d9d)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The following error from systemd-udevd is not harmful. It's just
because our qemu targets are using eth0.
Error changing net interface name 'eth0' to 'enp0s3': Device or resource busy
Note that systemd is using a different network interface naming scheme
from traditional ethX naming scheme. To make this error message go away,
we could symlink /etc/udev/rules.d/80-net-setup-link.rules to /dev/null
to recover traditional naming scheme. But I'm not sure if this will
cause regression in user experience for systemd users of OE.
So just ignore this error message so that parselogs.py test case does not
fail.
(From OE-Core rev: f1735fed088ddda6517fa4ff6fbd6ef683b14878)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
1. For test_syslog_running, we should not restrict it to run for
only busybox-syslog and sysklogd. So extend it to all syslog
providers in oe-core and meta-openembedded.
2. For test_syslog_startup_config, fix to make it depend on the
existence of busybox-syslog. The previous condition "!sysklogd
&& busybox" is incorrect, because busybox may be compiled without
syslog support, and sysklogd and busybox-syslog conflict with
each other.
(From OE-Core rev: 119c22342d446e4f638b8d4c81480ebc7b444245)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We keep breaking the preservation of hardlinks during the packaging process.
Add a selftest which tests this to try and prevent this breaking again.
(From OE-Core rev: 751fc7802f57a836a0be5fc6a8d5fdf8b1769c39)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
rm_work if enabled leads to some tests failing that rely on artifacts
being present. Check if rm_work.bbclass is included and show an error
and exit if it is.
Fixes [YOCTO #12694]
(From OE-Core rev: dde7e2f590834aa8034f1371954f9b4fbc7a60b7)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These tests are intended to search for bad RPATHs and text relocations, but we
do these tests at buildtime and as pax-utils is never installed in any default
images the tests are never executed.
(From OE-Core rev: ff81b58d5f322ea4a24c1b9ed40377c742459149)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code was using Wic instead of WicTestCase for image_is_ready and
native_sysroot which was confusing some tests. Fix image_is_ready.
For native_sysroot, convert the only user to just run the call itself
as there is no point in executing this for all the other tests which
don't use the value.
Whilst here, move _get_image_env_path to the base class alongside
its parent variabe and fix it to use WicTestCase.
(From OE-Core rev: 3e83364ed7031d53f281216214b04c7acace844a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
oe-selftest -r <invalid test name>
currently triggers a traceback. Ensure this doesn't happen and the user gets a
sensible error message.
(From OE-Core rev: f2042bf3638ed4edfb167e7f7d4be6da60997ead)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[YOCTO #12493]
Fixed:
- qemu started with pid 10000
- qemu exited unexpectedly
- The pid 10000 is re-used by another different process.
The is_alive() returned True in such a case because both qemu_pidfile and
/proc/10000 exist, but it's another process, this patch fixed the problem.
(From OE-Core rev: 78033af85698b4e112ae78e3140b241b1b3c06a3)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of mushing two paths together and hoping that the slashes line up
correctly, use os.path.join.
(From OE-Core rev: ed0413babaf10ccf729472f603bbe32d84ba1931)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As with the Python test, this can be both better and faster. No need to copy a
file, just run a one-liner.
(From OE-Core rev: c6eef46747fe58bb2310be4f06d2fa9b67901d72)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The warn method is deprecated. We should use the documented warning instead.
Quoting from the python's official doc:
"""
Note: There is an obsolete method warn which is functionally identical to warning.
As warn is deprecated, please do not use it - use warning instead.
"""
(From OE-Core rev: f467fd277eb77336097cfc0f5f329bdc8d0f70cb)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Also fix a related test's error message to match what it does.
(From OE-Core rev: 662be652ed39f32c3c7bee6539a9e91af191557e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Placing the layer in meta means the directory is in an unclean state
which may influence other tests. Use our build directory instead
since we 'own' that. This helps keep oe-selftest parallelisation
clean.
(From OE-Core rev: f2d32bdf3f27a9ef62fe3abb7fdd2e04ab84b5f3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
runCmd handles checking exit 0 internally unless specified otherwise and its
error messages are much more useful including the failed command output.
Clean up the unneeded 0 exit code checks in the wic tests.
(From OE-Core rev: 394f7d4efe80e390e2c9b0be5e6d10954395a82b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently this can fail with a message like 127 != 0 which is unhelpful.
If we remove the ignore_status=False, the debugging from runCmd is much
more helpful printing status.output.
Also remove the now unneeded exit code check.
(From OE-Core rev: 1aa7471b11aedc68de5116c461fe73152e3985fd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now we're not reporting the related fields (as of openembedded-core
8658b3677b9f7cb70806061c41570c709086ef05) we shouldn't expect to see
PR reported here since it's not monitored by buildhistory-diff. However,
with a bit of messing about we can check for the exact output that we
should now see as a result of the test changing PR to go backwards.
(From OE-Core rev: f9c233908e6ada208d7abfdfe3ff98c6b278f405)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After this replacement, the parent exception handling works so we don't
need subprocess wrapping with bb.error in the underlying functions.
The underlying contexts also have better module handling so the imports
can be cleaned up.
(From OE-Core rev: aa8260adf53139d776a2affe6118d28b295c1fab)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows the calls to is_elf (which calls file) to happen in parallel
allowing a speedup of do_package and do_populate_sysroot for native
recipes.
(From OE-Core rev: bbe0d3e26484f3f347262d40a8a9d415ce21fb43)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are probably further cleanups needed here but this at least removes
the major code duplication between these two similar funcitons, keeping the
kernel module ".ko" extension check for efficiency to avoid opening and
reading file contents in the general case.
(From OE-Core rev: 7ad0c0d6ab12bebeac097fc0f5210c876dcfe9be)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't need any functionality from the shell here, its just extra fork
overhead. Therefore remove it and use subprocess directly.
(From OE-Core rev: bcc03ea19e103f6aa93bada2f49fcc5cc7bc0790)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The current methods of spawning processes for parallel execution have
issues around collection of results or exceptions.
Take the code from package_ipk/deb, make it generic, add a results
collection mechanism, fix the exception handling and for it into a
standard library function.
Also add a test case which tests both the success and failure modes
of operation to stop this functionality regressiing again.
In particular, compared to multiprocess_exec, this fork off the parent
approach means we can pass in the datastore and functions work in the
same scope as the parent. This removes some of the complexities
found trying to scale multiprocess_exec to wider use.
(From OE-Core rev: 88f0c214e593a45566df5131bda4c946f5ccc8c2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For the same reasons as the runtime Python test, clean up the SDK test.
Also port from Python 2 to Python 3, as that's what is supported now.
(From OE-Core rev: bead742a3ffc0a53162fb0c36610d74a1422e7b3)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently this is three test cases:
1) test_python_exists. Fail if python3 isn't in PATH.
2) test_python_stdout. Run a Python script and check the output is as expected
3) test_python_testfile. Check that a file test_python_stdout wrote to exists.
(1) should be a setup and skip the test module if it isn't present.
(2) and (3) should be merged, there's no point copying over a two line Python
file, and the test doesn't verify that the file doesn't exist in the first
place.
Rewrite the test to check that Python is present in a class setup so the entire
test is skipped if it isn't and do some simple rot13 to verify that bytecode is
being executed correctly.
(From OE-Core rev: a35be5f32b4fe70b18ac1e2eccfd94558cecfbba)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The FILELIST field of the package info file in the buildhistory
repository is a space-separated list of all of the files in the package.
If a name of a file packaged by a recipe contains a space character then
of course the result was that we didn't handle its name properly. To fix
that, use quotes around any filename containing spaces and at the other
end use these quotes to extract the proper entries.
Fixes [YOCTO #12742].
(From OE-Core rev: 801b705957dc683030d11393f43407d0b3506b6a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The original idea here was that changes to certain fields might be able
to be explained if there was a change to another field, for example if
RDEPENDS changed it might be because DEPENDS changed. Thus we were
printing this kind of thing out with each change. Unfortunately in
practice this turned out to be noisy and not particularly useful, so we
might as well remove it.
Fixes [YOCTO #7336].
(From OE-Core rev: 8658b3677b9f7cb70806061c41570c709086ef05)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These platform specifc BSPs were removed from meta-intel and superseded by
intel-core* BSPs.
(From OE-Core rev: a6fd20fc57da3e1910e775f5994bd71bf79934ea)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Checking if the dependency had any failure is unreliable, for example
if the underlying data doesn't get transferred and the list is empty,
success of the dependency is assumed.
Since we now have success data available, change the code to use it.
(From OE-Core rev: 4abba4c30d5a6163a968a119395a679e5e281ab4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This uses the new concurrenttest code to enable parallel test execution
if specified.
(From OE-Core rev: 07d19fb3adab7a8d83ba83d9a16395f70d7b7a47)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This isn't present on modern distros by default and doesn't work with
testtools, needing multiple code paths in the code. Remove it in favour
of finding a better replacement for results collection/analysis.
(From OE-Core rev: 8001d933a8dc86004db014777f094d718086687d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There doesn't appear to be any reason we need this _results indirection
any more so remove it.
(From OE-Core rev: b618261811c48ff3b98eab1b340a8cd09ef183c6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We have several options for parallel processing in oeqa, parallel
execution of modules, threading and mulitple processes for the runners.
After much experimentation is appears the most scalable and least
invasive approach is multiple processes using concurrenttestsuite
from testtools. This means we can drop the current threading code
which is only used by the sdk test execution.
oeqa/decorator/depends: Remove threading code
Revert "oeqa/sdk: Enable usage of OEQA thread mode"
This reverts commit adc434c063.
Revert "oeqa/core/tests: Add tests of OEQA Threaded mode"
This reverts commit a4eef558c9.
Revert "oeqa/core/decorator/oetimeout: Add support for OEQA threaded mode"
This reverts commit d3d4ba902d.
(From OE-Core rev: a98ab5e560e73b6988512fbae5cefe9e42ceed53)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
