Update older exception syntax to modern one required by python 3.
Compatible with python 2.7.
(From OE-Core rev: d13f0ac614f1d1e2ef2c8ddc71cbfcf76a8dc3f2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
file() API doesn't exist in python 3, convert to open(). Also handle
some cases where files aren't closed. Compatible with python 2.7.
[Contributions from Ed and Richard]
(From OE-Core rev: 0f4ec13e11bb8abe21aba2a28547dfb9372bc377)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used print function instead of print statement to make
the code work in python 3.
[Changes from both Ed and Richard]
(From OE-Core rev: ced1995694c394d92cb82fb9c25a33dc027a3b69)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Having two code paths here makes maintenance difficult, and it doesn't
seem likely that you would use the local case in real usage anyway, so
drop the local support entirely.
This should allow us to resolve [YOCTO #9301].
(From OE-Core rev: 7a4c9c96fee4fb514c2b69b52e811c4f896a16f1)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In commit 20ea6d274bb0a9a5addb111f32793de49b907865, debugfs generation
for images using opkg, which included openssl was fixed.
However, that broke the generation of the opkg-based images, that lacks
openssl. The error is a python stack trace, showing that shutil.copytree
tries to copy a non-existing directory.
This relates to [YOCTO #9040].
(From OE-Core rev: f6b0b260ce18a30d04edfb0afb7942b9f9a5480b)
Signed-off-by: Anders Darander <anders@chargestorm.se>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use a common _log_check_common() function (based on the old
_log_check_warn() function) to implement the logic for both
_log_check_warn() and _log_check_error().
The main benefit of this is that now all error messages will be
reported again, not just the first one found. Additionally the output
will now look the same for both error and warning messages.
This removes the context for the error messages. However, since there
was no indication in the output that some of the lines were context,
they were more confusing than helping.
(From OE-Core rev: dd73dcac36b80b3b886a2e9bf575b91c4f60d039)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For each warning found in the log, _log_check_warn() would output a
line stating that it had found a warning, then the actual warning and
finally an empty line. This is quite excessive when there are many
warnings in the log.
With this change the output is instead a line stating how many
warnings were found, followed by the warnings. This makes the output
much more compact and actually much more readable.
(From OE-Core rev: d6e3477749b1d09d40a773e0ac857a24d5851984)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This will make _log_check_warn() exclude the same lines as
_log_check_error() does.
(From OE-Core rev: 85f64c68278f797c6f73f002f63d7f46fe80aef4)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Before there were three different ways to exclude a line from being
searched for error messages in _log_check_error(). Now there is only
one: an array of regular expressions. This should make it easy to add
more excludes if nedded.
(From OE-Core rev: 321df88088fbfa657b61b2bae32751f03daec46f)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In commit 0387d095 lines with "NOTE:" in them were excluded from the
log check for warnings. However, those lines were only there in the
first place since the regular expression that is used to find warning
messages explicitly included those lines...
(From OE-Core rev: 865ab39f18a52ed84217df56d0e65113e2894d02)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The fact that this function was overridden in the RpmRootfs class
seems to have led to a number of misstakes when changes have been made
to the base function in the Rootfs class. E.g., this change will
properly solve ticket 7789, which was supposedly solved in 38871dc0,
but that change had no effect in practice as the log_check_regex that
was modified for RpmRootfs class was not used by the RpmRootfs version
of _log_check_error()...
The only thing _log_check_error() in RpmRootfs did that the base
function in Rootfs did not do was to skip lines in the log that begin
with a + sign. This has now been moved to the base function instead.
[YOCTO #7789]
(From OE-Core rev: 1eb0a46502fca4b2ee30ccd2508f4e21a40c25ca)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The reverted commit added a warning with the log line that triggered
the log check for error messages before the warning that states that
an error has been found in the log. However, the error line is output
by the call to bb.fatal() that follows immediately after the original
warning, which makes it redundant. Additionaly, having two warnings
contradicts the intent of commit 8dfdd329 where the log warnings were
tidied up.
This reverts commit f9cf31525fc885e1a0f65bd55654631257f87078.
(From OE-Core rev: e509a83c8eb5a572aad7e7451d019fa2469d70c3)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Tested generation of .bmap file using wic --bmap command
line option.
(From OE-Core rev: 1f21c46e517685ff1363d3b6f4fd840300b1930b)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are parts of the fuctionallity missing when using the
dummy ExportTestContext class in runexported.py.
This changes the use of ExportTestContext dummy class from
runexported.py to the real class in oetest.py.
(From OE-Core rev: b0f105eecf25d334dd5fd0d936ec98cb8430ed23)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
pscmd is used by some tests to get the process
running on the target. If the test are exported
there won't be any pscmd attibute in the oeTest.
This adds "ps" as default pscmd.
(From OE-Core rev: 44aa8c3b6747179a0c4c156fc4922d838cdc19a4)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The host dumper is used to get information about the host
running the test when a test fails. This is used for the
autobuilders of Yocto Project.
Now that exported tests have thier own class the host dumper
is not necessary anymore.
(From OE-Core rev: fab3e4865a356ceb73d7deab5b4fd38ed795ca72)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adding the class is needed to separate the exported test
from the test image; both test run under different conditions,
i.e. an exported test doesn't require to change the signal
handling.
This change adds clasess ExportTestContext and ImageTestContext,
both of them inherits from RuntimeTestContext. Also refactors
RuntimeTestContext class, to keep the code common in this class.
(From OE-Core rev: ef9618a6e7f99e485cc6b59484d11807475c7823)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Found some more syntax cleanups needed for python3.
(From OE-Core rev: 1181d86e8707c5b8e8d43d5e785d7d9cf01fa491)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use print functions for comptibility with python3.
(From OE-Core rev: 0c89a8a276b67a9292ee3100003c789126bd9ea9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We want the decorator to leave the function names of the test unchanged. Some
decorators are already using wraps for this but not all. Fix this to be consistent
allowing inspection of the test to give the wanted values.
(From OE-Core rev: 9e4d60b29ff5667d23a89953ce7139b34c11d40b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added test_bmap to imagefeatures tests.
It tests if bmap file is generated for the images and
if the image is sparse.
[YOCTO #9414]
(From OE-Core rev: db27d8fbb44d2cdd524ac992630c781fd0c45b1b)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This new method extract the content of RPM file to a tmpdir,
without actually installing the package.
[YOCTO #9569]
(From OE-Core rev: 5f5c2a0fac5ad2baca162902410064375e8c610c)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sometimes it is needed to have the content of a package outside
the recipe context. This new method extract the content of an
IPK/DEB file to a tmpdir, without actually installing the package.
A new OpkgDpkgPM class was added to share the code for opkg and dpkg.
There were need some changes to opkg_query() in order to use it
with apt-cache output. Also set default values to avoid UnboundLocalError
[YOCTO #9569]
(From OE-Core rev: 7d214b34e11dc57316ed5c1c7747c4601286f6d2)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using the opkg and apt-get package managers the function
opkg_query() can be useful when query for package information.
This change moves the function outside the Indexer class so
the Indexer, OpkgPM, DpkgPM can benefit from it.
[YOCTO #9569]
(From OE-Core rev: 799bc1d1c747aad02b6d844bf55abfbd3ecc034c)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The check now runs at populate_lic time so change this from configure. We
also need to set a SRC_URI so that the check triggers.
(From OE-Core rev: a5970809a2f01dbd152684266e2a3d946d896620)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The ordering of dependency variables needs to be deterministic to avoid task checksums
changing. Use an OrderedDict to achieve this.
(From OE-Core rev: 855a2d21503856af392ab2d54ccfa270505ba142)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch add new methods to SSHControl class. These methods
include:
- Copy a dir to DUT
- Delete a file in the DUT
- Delete a directory in the DUT (if empty)
- Delete a directory structure in the DUT
[YOCTO #9565]
(From OE-Core rev: f22afb09fefdcb568d79ccd81277a43dacee2a82)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When enabling extra DEBUGFS image generation with opkg, the following error is
seen when openssl-cnf is included in the image.
Collected errors:
* file_md5sum_alloc: Failed to open file /mnt/cs-builds/anders/oe-build/build-ccu/tmp/work/ccu-oe-linux-gnueabi/ccu-image/1.0-r0/rootfs/usr/lib/ssl/openssl.cnf: No such file or directory.
Lots of similar issues was fixed by an earlier commit in oe-core,
5084ed9401250ed269a49d27b303806ab173c5d5, but openssl-cnf is outside of that fix.
Followup to [YOCTO #9490]
(From OE-Core rev: 20ea6d274bb0a9a5addb111f32793de49b907865)
Signed-off-by: Anders Darander <anders@chargestorm.se>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This test is a subset of the new sstate_noop_samesigs test, and less helpful
when it breaks, so remove it.
(From OE-Core rev: 7157261014e1dcbe9a57e7504dbb0ab2a53aa4d8)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is possible for a Node.js module to have node_modules subdirectories
that contain no package.json file (e.g. iotivity-node has such a
directory). It appears these should simply be ignored, or else with the
way the current code works we will get errors later.
(From OE-Core rev: 8c522f1f536270e195c8c73f5c72801495e7b33b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add two values for http_proxy to verify that changing it doesn't change any
unexpected tasks.
As this causes uninative to fail to fetch, ensure that uninative is always
disabled.
(From OE-Core rev: 7d8ffd22303a5b89cb129e804c124a2d1dedf9ab)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is not enabled by default, as there are still limitations and possible
issues with opkg (and rpm?) packaging data containing broken symlinks for
local indexes:
http://cgit.openembedded.org/openembedded-core/commit/?id=c8e0ec2da9ad4ce1c103966906a85f68c15400dd
There are other use cases for the packaging data to be available in SDK,
since it provides comprehensive info about SDK's contents and in the case of
opkg and dpkg is all text-based and can be easily parsed by simple scripts.
Introduce new "package-management" flag for SDKIMAGE_FEATURES list (similar
to the one already used for IMAGE_FEATURES) that controls presence of the
packaging data in resulting SDK, while unifying this behavior across the
board for supported pkg managers - rpm, opkg, dpkg.
(From OE-Core rev: 9ab934e4aecb759c922049245888dcd2a8c55477)
Signed-off-by: Denys Dmytriyenko <denys@ti.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When enabling extra DEBUGFS image generation with opkg, errors are seen like:
ERROR: core-image-minimal-1.0-r0 do_rootfs: Cannot get the installed packages list. Command '/media/build1/poky/build/tmp/sysroots/x86_64-linux/usr/bin/opkg -f /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/opkg.conf -o /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs --force_postinstall --prefer-arch-to-version status' returned 0 and stderr:
Collected errors:
* file_md5sum_alloc: Failed to open file /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs/etc/syslog-startup.conf.busybox: No such file or directory.
* file_md5sum_alloc: Failed to open file /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs/etc/fstab: No such file or directory.
basically for all CONFFILES in the image. This is due to the file rearranging
the rootfs generation code does. If we preserve the /etc directory,
the avoids the problem.
We need to tell copyfile to preserve symlinks since some are present in /etc.
[YOCTO #9490]
(From OE-Core rev: 5084ed9401250ed269a49d27b303806ab173c5d5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added output of test runner to bitbake console output.
bitbake <image> -c testimage now reports test progress
to the console:
NOTE: Executing RunQueue Tasks
...
NOTE: recipe core-image-lsb-1.0-r0: task do_testimage: Started
core-image-lsb-1.0-r0 do_testimage: test_ping (oeqa.runtime.ping.PingTest) ... ok
core-image-lsb-1.0-r0 do_testimage: test_ssh (oeqa.runtime.ssh.SshTest) ... ok
...
----------------------------------------------------------------------
core-image-lsb-1.0-r0 do_testimage: Ran 38 tests in 785.100s
core-image-lsb-1.0-r0 do_testimage:
core-image-lsb-1.0-r0 do_testimage: OK (skipped=1)
Adding more output to the console should also prevent autobuilder to kill
long running tests as ab assumes that test is stuck if no console output
produced by it for a long time.
[YOCTO #8238]
(From OE-Core rev: 06aa68138ec67623892221fbe9516bbb3795b48e)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When installing the esdk with INHERIT += "rm_work", the script complains
about do_rm_work as unexpected output from the bitbake run. This patch
ignores any output lines with do_rm_work and further refactors the
output comparison into its own function creates a new unit test to
verify the fix. The unit test can be run direct from the command line or
via oe-selftest.
[YOCTO #9019]
(From OE-Core rev: d41930e1daa933cf4bf063fa79a2e8fc9129e1b1)
Signed-off-by: Bill Randle <william.c.randle@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The test to verify that certain classes when inherited were no-ops only passed
if the build already inherited buildhistory.
To ensure that the test works as expected, use INHERITS_remove to ensure that
the classes are not inherited in the base run.
(From OE-Core rev: fc90c5db45933369b8fc0a33d6238876f5e5a087)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a recipe sets SRC_URI and then appends more items to it with +=
(such as the current rpm recipe in OE-Core), the code in
patch_recipe_file() was failing with a traceback. Work around the
problem for now by dropping the existing lines if we understand the
operation, else just set the value outright at the end. This leaves
something to be desired as it either doesn't respect the existing
structure or leaves a mess but it's better than the current
breakage.
We'll need to come up with a better solution later. Part of the problem
is the existing code structure doesn't allow for patch_recipe_file() to
know what's being added or removed - it only knows the final value that
the caller wants set.
Fixes [YOCTO #9458].
(From OE-Core rev: da5cd84794e1ab5c4efcd6250d3b93a46f610744)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When creating an eSDK ensure that any .git directories are not included.
[ YOCTO #9426 ]
(From OE-Core rev: 6a5e2b2196e5654fc54ba5b2e51a390c966fd1b7)
Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Package names cannot contain underscores yet some npm modules use them as part
of the name, replace them with hyphens in the package name.
(From OE-Core rev: fea932c79c8201e3e7649f4443874ea540e33461)
Signed-off-by: Brendan Le Foll <brendan.le.foll@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously, ROOTFS_POSTINSTALL_COMMAND was run only after the opkg rootfs
install post processing phase. This patch makes it generic so it is run fter
any package manager specific rootfs post processing, but before _run_intercepts().
[YOCTO #9248]
(From OE-Core rev: 8f2571881cd01ace88d282a63ad802cab4ab6940)
Signed-off-by: Bill Randle <william.c.randle@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The distribution identifier is often used to create filenames, so it needs to be
safe to use as a filename. Whilst most distributions have e.g. Fedora or Debian
as their name, it is possible that the name contains special characters.
To ensure this doesn't cause a problem strip out any non-alphanumerics from the
distribution name before returning it.
[ YOCTO #9443 ]
(From OE-Core rev: 8a96a7207561e00eb92e4fb69e7340f20bfa2053)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the changes introduced to test the eSDK
the runexported test failed during the execution.
This change fix runexported test in the least invasive
way, because of the release cycle.
(From OE-Core rev: da0abb9679cb1fd639859a2fdbd82101d0a81259)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As more variables were being added it made sense to split this test into two:
one quick test that verifies that the host being 32-bit or 64-bit doesn't impact
the hashes, and another which compares stamps for two builds with several
variables changed and classes inherited.
(From OE-Core rev: d29ede773fc94da5ff932e0cd306295a8d3871a3)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a testcase to build a directdisk image and check that the
used disk size is less than the apparent size, as wic now
assembles images as sparse files.
(From OE-Core rev: 2b8299eb6b911e64d9a05b86c07b489d15eeaa5e)
Signed-off-by: Joshua Lock <joshua.lock@collabora.co.uk>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to support workflows using devtool where a user might want to
modify tasks that exist in locked-sigs.inc, there must be a way to unlock
recipes.
This patch adds that support by allowing the user to add recipes to
SIGGEN_UNLOCKED_RECIPES. Recipes that exist in that variable will have
all their tasks unlocked, as well as any tasks that depend on that
recipe.
For example if foo->bar->baz, if you unlock baz, it will also unlock bar
so that foo can be rebuilt without explicitly specifying bar as being
unlocked.
[YOCTO #9195]
(From OE-Core rev: 8a8fc54d824767a6a94d12a4ace98b0bdbb1aa25)
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The previous message when signatures didn't match between the metadata
and the locked signatures file, the message output was a bit confusing.
Now the message should be of the form:
The zlib-native:do_install sig is computed to be
53531910a2a7848432da89def942a91a, but the sig is locked to
d25ba9035f7ccb308e51bbe1066e8d27 in SIGGEN_LOCKEDSIGS_t-x86-64
which will hopefully be more useful in understanding the problem.
[YOCTO #9195]
(From OE-Core rev: 49eeabfff8bbea69401db41f7219e29acf47af73)
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
