This covers the traditional API as well as the new output_log feature.
While testing, it was noticed that killing hanging commands does not
work when a shell is used to run the command(s). This might be worth
fixing.
(From OE-Core rev: 62489e58ca9975f58b48fc2bd8cf27fd22e25564)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There should be no reason to execute a full build, as we're
just interested in the deployment of the archives.
The newly added tests already do the same.
(From OE-Core rev: 78c2897dff7cd9fe2cab511549cb146d5231e573)
Signed-off-by: André Draszik <adraszik@tycoint.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
remake was removed from oe-core: use another recipe in the devtool
extract test.
(From OE-Core rev: dbf680ce09ce54f97c781fc7a8e5e05ed4706073)
Signed-off-by: Jussi Kukkonen <jussi.kukkonen@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The test runs an upstream version check, and then compares the
list of recipes that failed the check (i.e. those where latest
upstream version could not be established) against the list of
known-broken upstreams. Mismatches either way (upstream check failed,
recipe not in the exception list, or upstream check worked,
recipe in exception list) fail the test.
[YOCTO #11031]
(From OE-Core rev: 544e04a11ab6c26a367ac0b67774bf7ffdf43c88)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When add meta-selftest by the script the testlayer_path needs to be
reloaded to avoid None value.
(From OE-Core rev: d1120849e12c46aa46b02905a3f43c0853bd11a0)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Current osselftest print confusing assertion message when using
self.assertTrue(os.path.exists(filepath)) to test file path,
example of confusing assertion message:
AssertionError: False is not true
Replce assertTrue/assertFalse with assertExists/assertNotExists to test
file path, this will improve assertion message and simplify coding,
self.assertExists(filepath) will print below
AssertionError: <filepath> does not exist
[YOCTO #11356]
(From OE-Core rev: 08b0702492536d41d3cf2c9c05d1fab36d32a566)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It was pointed out +1 is safer than -1 for systems with one processor.
(From OE-Core rev: 78041e20e43d9583448ff31f8b9b1c6157da8625)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added test case for "wic rm" functionality.
- remove file from vfat partition
- remove directory from vfat partition
(From OE-Core rev: fc42914d426ad2dad8c3026668726c003b3ab10b)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added test case for "wic cp" functionality.
- copy file to vfat partition
- copy directory to vfat partition
(From OE-Core rev: cb416d1feea042bcdedc9f522d588fef2c4929bc)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Tested 'wic ls' functionality:
- list of image partitions
- list of directory content of vfat partition
(From OE-Core rev: 3b271f6a40d7ef93fff9700dd8ac98bddd92d581)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since the processing code for signature generation is now threaded,
use higher thread values as examples in this code for better performance.
(From OE-Core rev: f68ec7191546474f0bd688e57d2381a8e92be617)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Modify “test_devtool_virtual_kernel_modify” to be executed with default
configuration defined on oe-selftest test suite. A shorter string was
added to the the header file to avoid overlapped sections when building
kernel for qemux86-64.
[Yocto #11300]
(From OE-Core rev: a8bda790fded43eac72b2ad76d47c27363f02070)
Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There were two completely different tests inside selftest.py,
and the only reason for them to be together was that both needed
the same mechanism to execute (run testimage from within selftest)
This fixes the design issue and error-prone situation by separating
these tests. In add a new module for dnf-runtime and also has an
extra time added to running the tests, but it is minimal since
the builds reuse data from previuous build.
[YOCTO #11436]
(From OE-Core rev: 2f6a506018973f0eaf1306555df3e74a3ab25680)
Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Catching a possible exception when importing the git module,
allows us to skip test if the latter is missing on the host.
Also, import oe.buildhistory_analysis inside fuctions
because this module also needs git to work correctly.
[YOCTO #11620]
(From OE-Core rev: 186882ca62bf683b93cd7a250963921b89ba071f)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
New framework has different classes/decorators so adapt current test cases to
support these. Changes include changes on base classes and decorators.
Also include paths in selftest/__init__.py isn't needed because the
loader is the standard unittest one.
(From OE-Core rev: ddbbefdd124604d10bd47dd0266b55a764fcc0ab)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The new OEQA framework aims to re-use code into the different Test
components.
The previous oe-selftest implements it-self loading, run, and list test
cases in a non-standard way (unittest base) and other functionalities
like logging that is now on oeqa core. This ends on a compact oe-selftest
script.
All needed command line options was migrated but there are some of them
pending of implementation and others deprecated.
Deprecated options:
list-tags: The tag functionality into the old oeqa framework isn't
work, the selftest doesn't has tag decorators.
{run, list}-tests-by: Ambiguos options it accepts all the posibilites module,
class, name, id or tag.
Remaining to implement:
coverage: It enables covrage reports over a test run, currently isn't on
on use and some bugs [1], i filed a bug to add support to OEQA core module in
this way other Test components could enable it.
repository: It push XML results into a git repository and isn't in use,
i filed a bug to implement this into OEQA core module. [2]
[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11582#c0
[2] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11583#c0
(From OE-Core rev: 3b2a20eee4a39f40287bf67545839eaa09fc892d)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The test layer is added at init of selftest and is the same
across test runs so pass it as a attr in the context.
(From OE-Core rev: 8441da8fbc7ff2237d9ccd3a5c5117345f5c87b2)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The machine selection is an operation that needs to be made
in every test run, the best place to it is on the context
module.
Use self.tc.custommachine variable instead of use environment.
SIgned-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
(From OE-Core rev: f295b70ace0ffc28256140a21af5c3a8903297cb)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The oe-selftest creates include files to store custom configuration to
make specific tests, every class executes a different test and may be
uses custom configuration.
So move to case class in order to simplify oe-selftest script and later
implement later a build folder per class.
(From OE-Core rev: 1130b40c3dfa65e7ece08a95b3941e4d1d20bcf0)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Summary of the changes:
- Use OETestCase as base class instead of unittest.TestCase
- Remove LogResults decorator the new framework provides logging into
the core functionality.
- Logger is now self.logger instead of self.log
- Move comments into docstrings in several help methods
- Use get_test_layer() method instead of access monkey patched variable
in old oeSelfTest case class.
(From OE-Core rev: c38cab77893f9d8fd505f050cc880a15677b73db)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The test_recipetool_create_github test fetches HEAD of the repository so
upstream changes can (and do) break the test. Avoid these problems by passing
the rev= argument in the URL to lock the checkout to the same version that is
fetched in the github_tarball test.
Also pass the commands to runCmd() as a list instead of a string, the semicolon
in the URL needs more quotes if the shell is involved and passing a list
bypasses the shell entirely.
(From OE-Core rev: b35bedd209092432c560e998043b6a8c5c2e4d34)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Current osselftest print confusing assertion message when using
self.assertTrue(os.path.exists(filepath)) to test file path,
example of confusing assertion message:
AssertionError: False is not true
Add the assertExists and assertNotExists to improve assertion
message and simplify coding, using selft.assertExists(filepath),
will print meaningful assertion message:
AssertionError: <filepath> does not exist
[YOCTO #11356]
(From OE-Core rev: b0a74554ef926ce05078494ca8e67178b56232f5)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixed failure of test_unsupported_subcommand caused by
switching to argparser by checking that parser returns
non-zero return code.
[YOCTO #9636]
(From OE-Core rev: 581db38a280bf4cb03faf568cefd8d97383b5417)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When uses write it creates a full new configuration in selftest.inc
causing to fail populate_sdk_ext linux-yocto because the signature
of the sstate changes.
[YOCTO #11300]
(From OE-Core rev: 65bab9210be51aeb431ea85c90e31ad9f0d2340c)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This test case verifies that opkg, dpkg, and rpm all have the same behaviour for
version ordering, specifically the behaviour of ~ which should be sorting
*before* nothing:
1.0 < 2.0~pre < 2.0 < 2.0-fix
(From OE-Core rev: 0bf875ea234bb9ff50d347345782e14d6b7d3ff9)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Latest DNF has changed the amount of whitespace it outputs so use matches
instead of hardcoding the exact number of spaces.
(From OE-Core rev: 9ebeb374e49ecedecba8fe16fff3717edbc41994)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Split the configuration values (common and specific) so it is easier to read
what goes into the config file. Also the specific configurations are
set in every loop so these do not append on each iteration.
(From OE-Core rev: 58d43b470ffa9b498234b6845cacfd867218ca03)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The path to where to install and find the sysroot components is used
in many places. This warrants it to get its own variable.
(From OE-Core rev: 70a84b525470f72339568409daf84845904e4cab)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
sstates are cleaned when ruining test_devtool_virtual_kernel_modify to
have a clean environment but this is affecting eSDK test that are
dependent of those sstates, hence “cleansstate” is replaced for
“clean”.
[YOCTO #11300]
(From OE-Core rev: 182fdb97ed6c11c7b55fdb99773da6f72c9828b3)
Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This test was assuming the format of SRC_URI so broke when SRC_URI was changed.
Fix the test by hardcoding a complete SRC_URI instead of appending and hoping
for the best.
(From OE-Core rev: afe1d3073a435ef6c838cc676748842af4c4af83)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously, runqemu grep root=/dev/sd or root=/dev/hd on the image, and
would use vda if no grep result, now we have set QB_DRIVE_TYPE to
"/dev/sd" by default, and the device will be /dev/sda, so use sda to
replace vda in the test case.
(From OE-Core rev: a722016df3f452f7a870157a99a1abb7d97d8280)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added new parameter 'api' to sparse_copy function to specify
underlying filemap API to use. By default sparse_copy will
try both available APIs.
Added test case for sparse_copy to wic test suite.
(From OE-Core rev: 88701cef6ba399e82f96ed1b0eef9a44ed8c1687)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This script appears broken and is actively breaking build directories.
For example, binutils-cross-i586 gets run do_fetch to do_populate_sysroot by:
$ oe-selftest -r devtool.DevtoolTests.test_devtool_virtual_kernel_modify
then:
$ oe-selftest -r oescripts.BuildhistoryDiffTests.test_buildhistory_diff
wipes out the contents of tmp/work/x86_64-linux/binutils-cross-i586/2.28-r0/
but does not wipe out the corresponding stamps, then:
$ oe-selftest -r runtime-test.Postinst.test_postinst_rootfs_and_boot
needs binutils-cross-i586:do_populate_lic and if and only if this is
unavailable from sstate, it fails since it thinks the source is already
unpacked when it isn't resulting in:
WARNING: binutils-cross-i586-2.28-r0 do_populate_lic: Could not copy license file /home/pokybuild/yocto-autobuilder/yocto-worker/nigh
tly-oe-selftest/build/build/tmp/work/x86_64-linux/binutils-cross-i586/2.28-r0/git/COPYING3.LIB to /home/pokybuild/yocto-autobuilder/yocto
-worker/nightly-oe-selftest/build/build/tmp/work/x86_64-linux/binutils-cross-i586/2.28-r0/license-destdir/binutils-cross-i586/COPYING3.LI
B: [Errno 2] No such file or directory: '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/build/tmp/work/x86_64-l
inux/binutils-cross-i586/2.28-r0/git/COPYING3.LIB'
ERROR: binutils-cross-i586-2.28-r0 do_populate_lic: QA Issue: binutils-cross-i586: LIC_FILES_CHKSUM points to an invalid file: /home/
pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/build/tmp/work/x86_64-linux/binutils-cross-i586/2.28-r0/git/COPYING3 [license-checksum]
and similar errors.
Its safer for users to wipe tmp than try and maintain scripts which try
to remove pieces of tmp and get it wrong so remove the script.
(From OE-Core rev: 809b23c829f352c0eae455ea89f53e2a9ee87f06)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
cleanup-workdir isn't a partcularly good way to attempt to cleanup after
tests and in some cases is actively breaking the workdir. Whilst this is
a bug in cleanup-workdir, I'd prefer it didn't break tests, particularly
when as far as I can see, these calls are "belt and braces" and don't appear
to serve a specific purpose. If this introduces bugs, we should fix this
bugs.
For the purposes of history, binutils-cross-i586 gets run do_fetch to
do_populate_sysroot by:
$ oe-selftest -r devtool.DevtoolTests.test_devtool_virtual_kernel_modify
then:
$ oe-selftest -r oescripts.BuildhistoryDiffTests.test_buildhistory_diff
wipes out the contents of tmp/work/x86_64-linux/binutils-cross-i586/2.28-r0/
but does not wipe out the corresponding stamps, then:
$ oe-selftest -r runtime-test.Postinst.test_postinst_rootfs_and_boot
needs binutils-cross-i586:do_populate_lic and if and only if this is
unavailable from sstate, it fails since it thinks the source is already
unpacked when it isn't resulting in:
WARNING: binutils-cross-i586-2.28-r0 do_populate_lic: Could not copy license file /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/build/tmp/work/x86_64-linux/binutils-cross-i586/2.28-r0/git/COPYING3.LIB to /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/build/tmp/work/x86_64-linux/binutils-cross-i586/2.28-r0/license-destdir/binutils-cross-i586/COPYING3.LIB: [Errno 2] No such file or directory: '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/build/tmp/work/x86_64-linux/binutils-cross-i586/2.28-r0/git/COPYING3.LIB'
ERROR: binutils-cross-i586-2.28-r0 do_populate_lic: QA Issue: binutils-cross-i586: LIC_FILES_CHKSUM points to an invalid file: /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/build/tmp/work/x86_64-linux/binutils-cross-i586/2.28-r0/git/COPYING3 [license-checksum]
and similar errors.
(From OE-Core rev: a66857096df3f68d38271b3a6792150f009a54b1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The intent in these tests was to find something early in the bootstrap
process to run tests against which didn't require long build times.
This breaks with the removal of the glibc-initial do_build target.
Replacing it with linux-libc-headers seems like a good choice
and simplifies the conditionals too.
(From OE-Core rev: c37d610272f9d0a506334ff9f724c025acace137)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
WIC test case requires wic-tools which in turn depends on intltool-native and
gettext-native. However, if NLS is disable, the gettext-minimal-native is used instead
of gettext-native causing a failure on intltool-native as seen below:
checking for msgfmt... no
checking for gmsgfmt... no
configure: error: GNU gettext tools not found; required for intltool
ERROR: Function failed: do_configure (log file is located at /home/lsandov1/poky/build/tmp/work/i586-poky-linux-musl/systemd-boot/232-r0/temp/log.do_configure.7518)
ERROR: Task (/home/lsandov1/poky/meta/recipes-bsp/systemd-boot/systemd-boot_232.bb:do_configure) failed with exit code '1'
NOTE: Tasks Summary: Attempted 609 tasks of which 604 didn't need to be rerun and 1 failed.
[YOCTO #10902]
(From OE-Core rev: ccd2faca99331e010badbb78d42b0ee644ca1a0a)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The recipe being tested (devtoo-test-patch-gz) by devtool has dependencies (at
least libxres and virtual/libx11) that cannot be built with poky-tiny distro so
skip the test for this particular policy.
[YOCTO #10891]
(From OE-Core rev: 03751783cb063bc6a57cd19357bc1016bfa2a814)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
fedorahosted.org was retired on March 1st, 2017. This is to
update the SRC_URI to point to github.com.
[YOCTO #11226]
(From OE-Core rev: 9b3c9193d4a04061e64e43d55782a4d73ee166ad)
Signed-off-by: Choong YinThong <yin.thong.choong@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To generalize the unit test for other kernel providers, use virtual/kernel instead
of linux-yocto, allowing to run the selftest on non-poky distros, i.e. poky-tiny.
(From OE-Core rev: 087e1204fb27e2952f16c2e4761f5993b7722732)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is not much sense in creation of empty squashfs
partition. It's also not possible to create empty squashfs
partition of specified size.
Even more, prepare_empty_partition_squashfs method is
absolutely broken. It raises exception when called and
even its signature differs from the rest of of similar
methods. It means that nobody uses it and it's safe
to remove it.
Removed prepare_empty_partition_squashfs method and
testing of empty squashfs partition.
(From OE-Core rev: 9152960f250cb4df1e559d747fb09005675a0d75)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added msdos partition to the .wks file in test_fs_types
wic test case.
[YOCTO #11137]
(From OE-Core rev: 044dd146d39542d0e6b598ee1dcadc49e0db3de9)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
