Currently processed being killed by the OOM killer may not be spotted by
ptest-runner. After we complete the tests, check the logs and report if there
were any. This ensures the user is aware of OOM conditions affecting the
ptest results.
(From OE-Core rev: 20a441d53817f80e0ce1597e77f6e794422ac49a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sometimes we meet the following failure for the test_lookup_recipe
test case.
AssertionError: 'zlib\nbusybox is in the RPROVIDES of target-sdk-provid[32 chars]ummy' != 'zlib\nbusybox'
zlib
+ busybox- busybox is in the RPROVIDES of target-sdk-provides-dummy:
- target-sdk-provides-dummy
This is because target-sdk-provides-dummy rprovides busybox.
So clean things up to avoid failure.
(From OE-Core rev: b3001770df6640549270361bfaa449cb3e79a0b7)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Get rid of further unneeded code complications:
* value mappings we could just direct use
* ftools when we can write files easily ourself
* test result status filtering we don't use
* variable overwriting module imports
(From OE-Core rev: d6065f136f6d353c3054cc3f440a4e259509f876)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Merge the results handling into the ptest log parser as a seperate
method.
Drop the weird "pass.skip.fail." prefix to the results filename, its
just bizarre.
Drop the code turning a list into a regex then searching the regex for
an item, "x in y" is perfectly capable.
Use a dict, sort the keys as needed and drop the list sorting code.
(From OE-Core rev: f317800e950b4a37b4034133bc52e0c47f04dc29)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow parsing of the ptest duration, exit code and timeout keywords
from the logs, returning data on each section.
Also include the logs broken out per section.
(From OE-Core rev: a9a67dccaa5be0f06eedcab46dcff7cbf9202850)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now we have a dedicated ptest parser, merge in the remaining ptest
specific pieces to further clarify and simplify the code, moving to
a point where we can consider extending/enhancing it.
(From OE-Core rev: 05991bb5bc8018275d03fdeecee3d5a757840c7c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rename the paster to be ptest specific and apply some further cleanups
to the code to simplify and clarify what its doing.
(From OE-Core rev: 45a5886f1ec458d4c306b8d68fd31d568bc36b47)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
logparser is only used by ptest. Its slightly overcomplicated as it was
intended to be reusable but wasn't. Simplify it as a dedicated parser is
likely to me more readable and maintainable.
(From OE-Core rev: c7478345b2b4a85cb1fec40e762633871f0e94cb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some tests end up without a section, avoid tracebacks trying to use
None as a string in that case.
(From OE-Core rev: 86fb5d898a29761f120c2eaa538a32cf2e078487)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Manual test step for SDK is updated. Previously toolchain was "poky-glibc-x86_64-core-image-sato-sdk<type-arch>-toolchain-<release-version>.sh"
But toochain for core-image-sato-sdk is not available in releases after 2.1, hence changed it to "poky-glibc-x86_64-core-image-sato-<type-arch>-toolchain-<release-version>.sh
Other less significant update is to add exepcted result in intermediate test steps.
(From OE-Core rev: f5083ba27e667789c32dcd00bc1b883f4b76bd4f)
Signed-off-by: sangeeta jain <sangeeta.jain@intel.com>
"poky-glibc-x86_64-core-image-sato-<type-arch>-toolchain-<release-version>.sh"
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added json file for compliance test cases to the manual runtime tests
(From OE-Core rev: 946f37d57f7b23c7875089ed98be9148ccc05890)
Signed-off-by: Sudhir Sharma <sudhirx.sharma@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Changes the default PACKAGE_DEBUG_SPLIT_STYLE to generate separate
source and debug packages. SDKIMAGE_FEATURES is updated to include the
source packages so that there is not change for the SDK contents.
[YOCTO #12931]
(From OE-Core rev: 8df14b678e44cc749b361224af05ccbcfa9ae9b5)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is particularly useful when setting up GL tests.
(From OE-Core rev: 167a46775059b782c6f82ce8c5a47b27262e95d4)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When executing eSDK test case, the following error appears.
WARNING: attempting to use the extensible SDK in an environment
set up to run bitbake - this may lead to unexpected
results. Please source this script in a new shell session
instead.
FileExistsError: [Errno 17] File exists: '/.../tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/selftest-esdk-fcuyzsqu/tmp/sysroots/x86_64/bin/pigz' -> '/.../tmp/hosttools/pigz'
So unset these two vars to avoid messing things up.
(From OE-Core rev: bc07d825ce8bb3b337623c232fef61f2781c82af)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When removing the temporary directory, it's possible that bitbake.lock
file is removed by bitbake during the cleanup. And this leads to the
following error.
FileNotFoundError: [Errno 2] No such file or directory: 'bitbake.lock'
So add a check to remove this file before cleaning up the temporary
directory.
(From OE-Core rev: 984f56b37bd0014e5bf9509fc8ed181973e61773)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A new commit was introduced to kernel, which caused kprobe sample case
failed on arm.
[kernel commit: e46daee53bb50b, ARM: 8806/1: kprobes: Fix false positive
with FORTIFY_SOURCE]
And according to author, the case just works on x86 and powerpc.
So we just promise that the case can pass on x86 and powerpc.
(From OE-Core rev: 1a5b14bc3fe29618beea016ab2b32f55c19dd6ad)
Signed-off-by: Hongzhi.Song <hongzhi.song@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This test case boots the image in qemu and checks for mounted partitions.
But the outputs of mount are different between sysvinit and systemd:
sysvinit:
/dev/root /\r\n/dev/sda1 /boot\r\n/dev/sda3 /media\r\n/dev/sda4 /mnt
systemd:
/dev/sda1 /boot\r\n/dev/sda2 /\r\n/dev/sda3 /media\r\n/dev/sda4 /mnt
So check mounted partitions by egrep rather than check output of runqemu.
(From OE-Core rev: 8e3311d45ec68d74e934e966f67cf5102634981c)
Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using linux-yocto-dev, we will have one more amba error message
for qemuarm, so ignore it too.
(From OE-Core rev: 37cb293cf0f4194ec923d3bc4ebb9645f07536c0)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now we can compile m4-native rather than target m4, this can save a lot of
build time.
(From OE-Core rev: 8e33cef647f4e8a104da61aecab4ac6c6b9bfd3b)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are failures when multiple users run oe-selftest on the same
host:
PermissionError: [Errno 13] Permission denied: '/tmp/oe-saved-
tests/201812250324_qemu'
This is because /tmp/oe-saved-tests was created by user A, while user B tries
to write data in it, then the error will happen. This patch can fix the
problem.
Move the dumped data to ${LOG_DIR}/runtime-hostdump/ rather than
/tmp/oe-saved-tests/ to fix the problem.
(From OE-Core rev: e219fe5329599cd6c3682f521eaee3852a2c8980)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
And fix the reported upstream check failures.
(From OE-Core rev: 63af0edcd954fac530ba17b04e5df5837cddc0a4)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use a different STAMPS_DIR and SSTATE_DIR in test_yocto_source_mirror.
Otherwise, when executing `oe-selftest -a', we will get a lot of failures
due to do_unpack failure.
(From OE-Core rev: f8502c68163ec288cf6fb6754829b455501b4a05)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now QEMU_USE_KVM can only be boolean, can not contain MACHINE any more.
(From OE-Core rev: 37e5035a2ba27595eeef22062580a5b04a73ba15)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixed:
MACHINE = "qemux86"
QEMU_USE_KVM = "qemux86"
IMAGE_CLASSES += "testimage"
$ oe-selftest -r runqemu.RunqemuTests.test_boot_rootfs
[snip]
File "/buildarea1/lyang1/poky/meta/lib/oe/types.py", line 122, in boolean
raise ValueError("Invalid boolean value '%s'" % value)
ValueError: Invalid boolean value 'qemux86'
Now QEMU_USE_KVM can only be boolean, can not contain MACHINE any more, kvm
will be enabled if target_arch == build_arch or both of them are x86 archs.
(From OE-Core rev: 7c1a8a624cad8d967635c6cb5f99cf655bde3d44)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The 5s timeout for non-kvm is too short, especially when the load is high,
which leads to unexpected errors, so set timeout to 60s by default.
(From OE-Core rev: 8197d0f638a760fc03062c7a9009117d083d7ead)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The previous code assumed builddir and meta-selftest are in the same dir, but
this isn't always true, builddir can be anywhere, use get_test_layer() to
locate meta-selftest can fix the problem.
(From OE-Core rev: 56d2493a9adfcc47ae7e265439e05ff42cdbbbbf)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than use the obsolete do_checkpkg function, use the new recipeutils
function which uses tinfoil to get the data rather than needing csv file
manipulation.
(From OE-Core rev: 3f3f80b00cd999f1b2aef8f5c0ce0900aa4dcbcb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use tinfoil to enumerate recipes and get the value of RECIPE_MAINTAINER
to make it a bit more reliable in the face of do_checkpkg issues we are
currently seeing on the Yocto Project autobuilder. This also makes it a
little less painful to re-execute test_maintainers() since you don't
have to wait for bitbake -c checkpkg to complete every time.
Note that the new test has been written in such a way that it will still
function if RECIPE_MAINTAINER values are ever moved to the recipes.
Also, the test still currently fails as there are recipes that don't
have an assigned maintainer.
(From OE-Core rev: 47282a2f6f12acebf58961ea9410cfbc335d560b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Don't use the helper class as it gets in the way more than it helps, exercise
the out-of-tree paths, and verify the installed files match the expected
architecture.
(From OE-Core rev: 6d666b0413336de2e556b2722c5be97ae5cd40ad)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Don't use the helper class as it gets in the way more than it helps, exercise
the out-of-tree paths, and verify the installed files match the expected
architecture.
(From OE-Core rev: 920ae8c6537c2469f21ab9439587fd094ecc40f6)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Drop redundant imports and variables, and use os.makedirs() instead of
bb.utils.mkdirhier().
(From OE-Core rev: 2de9b1e611e5047afb540f98756994925c22e446)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use oeqa.utils.subprocesstweak to monkey-patch the subprocess exception so that
any output is shown, and remove any explicit try/catch handling that would have
hidden this.
(From OE-Core rev: 55964b33b561397287779ee474170790dfd03e85)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For example, instead of saying "3 != 62", say "Binary was x86-64 but expected
i586".
(From OE-Core rev: 9ab94cea589fca4394ec1fd8dc06b23fd8e990b9)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Expecting 1s accuracy on a 2s timeout on a heavily loaded system has proven to be
unreliable. Update this to a 5s timeout with a 3s delta which should be achievable.
(From OE-Core rev: 5feecb639d49d72d8a6abc589b937e07ee72f252)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ensure log directory exists to avoid the following error.
FileNotFoundError: [Errno 2] No such file or directory: '/.../build-selftest/tmp/log/oe-selftest-results-20181207043431.log'
(From OE-Core rev: 6c41de48db76087fee596d9440d8f05346ab1094)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
INFO - ======================================================================
INFO - ERROR: setUpClass (eSDK.oeSDKExtSelfTest)
INFO - ----------------------------------------------------------------------
INFO - Traceback (most recent call last):
File "/home/pokybuild/yocto-worker/oe-selftest-debian/build/meta/lib/oeqa/core/case.py", line 32, in _oeSetUpClass
clss.setUpClassMethod()
File "/home/pokybuild/yocto-worker/oe-selftest-debian/build/meta/lib/oeqa/selftest/cases/eSDK.py", line 76, in setUpClass
cls.tmpdirobj = tempfile.TemporaryDirectory(prefix="selftest-esdk-", dir=bb_vars["WORKDIR"])
File "/usr/lib/python3.5/tempfile.py", line 929, in __init__
self.name = mkdtemp(suffix, prefix, dir)
File "/usr/lib/python3.5/tempfile.py", line 507, in mkdtemp
_os.mkdir(file, 0o700)
FileNotFoundError: [Errno 2] No such file or directory: '/home/pokybuild/yocto-worker/oe-selftest-debian/build/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/selftest-esdk-q7ln84gc'
(From OE-Core rev: 48719bc4d108df7e357e7f22f9f356cd72b3ebbd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
