This class was added by me back in 2007 and has changed one for a whitespace issue
since. It only has two lines and neither are particularly useful, one was replaced
by the nopackages class, the other adding recursive dependencies also is now
mainly problematic adding tons of unneeded dependencies. The name is hard to
understand and the class doesn't have a clear purpose. Drop it.
Remove the references in devtool (which may be the one reason to keep it around
but in my view still not worth it).
(From OE-Core rev: 6e2cbfc561dac89bf9183d24d90e52f7d9117826)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is the result of automated script conversion:
scripts/contrib/convert-overrides.py <oe-core directory>
converting the metadata to use ":" as the override character instead of "_".
(From OE-Core rev: 42344347be29f0997cc2f7636d9603b1fe1875ae)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If recipe uses only file:// fetcher devtool unpacks sources under
oe-local-files/ and adds symlink in source root directory. We need to
verify if the symlink in subdirectory has correct path.
See [YOCTO #13738] for details.
(From OE-Core rev: 044c90d152bfd1b2a23272df39327ba4cd862dff)
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Causes intermittent autobuilder errors:
2020-12-21 19:34:23,035 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
File "/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/meta/lib/oeqa/selftest/cases/devtool.py", line 68, in setUpModule
bb.utils.edit_bblayers_conf(bblayers_conf, None, None, bblayers_edit_cb)
File "/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/bitbake/lib/bb/utils.py", line 1460, in edit_bblayers_conf
(updated, newlines) = edit_metadata(newlines, ['BBLAYERS'], handle_bblayers)
File "/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/bitbake/lib/bb/utils.py", line 1290, in edit_metadata
if handle_var_end():
File "/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/bitbake/lib/bb/utils.py", line 1215, in handle_var_end
(newvalue, newop, indent, minbreak) = varfunc(in_var, full_value, op, newlines)
File "/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/bitbake/lib/bb/utils.py", line 1425, in handle_bblayers
res = edit_cb(layer, canonicalise_path(layer))
File "/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/meta/lib/oeqa/selftest/cases/devtool.py", line 60, in bblayers_edit_cb
shutil.copytree(pth, destdir, ignore=ignore_patterns('*.pyc', '__pycache__'))
NameError: name 'ignore_patterns' is not defined
(From OE-Core rev: c2d9612279fce9cbcb738913b2042949f692c4a5)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some of the selftests failed over the weekend with "access denied" errors
fetching this tarball. Instead of relying on upstream when fetching the
tarball, use the Yocto source mirrors instead.
(From OE-Core rev: 988e0ff4131f46dfed14516ff5f61d72b9fb6941)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We previously put a sync call into devtool to try and combat the bitbake
timeout issues on the autobuilder. It isn't enough as the timeouts occur
mid test. They are also occurring on non-devtool tests.
Add in sync calls around command execution instead.
(From OE-Core rev: ceca5ed121e2b54415a7ab3a217882e4ea86923a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
devtool doesn't quite behave right when a recipe has patches applied in overrides, so
add a test case to exercise that behaviour.
(From OE-Core rev: ad89c3254cc6c06026f1bebe23ec3717c1546633)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The tests are actually skipped if meta-oe isn't present which isn't
quite the same thing, but hopefully close enough.
(From OE-Core rev: d22ed015d8f38241acb783e1a468fb15d4317670)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Devtool tests are heavy on IO and if bitbake can't write out its caches, we see
timeouts. Call "sync" around the tests to ensure the IO queue doesn't get too
large, taking any IO hit here rather than in bitbake shutdown.
(From OE-Core rev: ce265cf467f1c3e5ba2edbfbef2170df1a727a52)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In certain conditions, most likely under heavy load on the
AutoBuilder, the prebuilt .pyc files are attempting to be
executed before they have been completely copied. Avoid
this by not copying the .pyc files (nor the __pycache__
directory). The impact of python3-native recreating the .pyc
files should hopefully be negligible.
YOCTO#13421
YOCTO#13803
(From OE-Core rev: 4df098aeecd589ddd11a883e281285588a6966ca)
Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This bug has plagued the autobuilder for a couple of years and we've
struggled to reproduce/debug it.
The problem is the "lib" directory in meta-poky used during the load_plugin
tests for recipetool and devtool can race and one can delete the files
from the other leading to test failures.
Deleting the lib directory only if empty will avoid this.
[YOCTO #13070]
(From OE-Core rev: d603e2fd589e1edbc67c8c2fefb6337e728b9d01)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I took the opportunity to rewrite the recipe from scratch; there was just too much baggage in it.
(From OE-Core rev: f058272de9cba188d96940c8c921cf31727fe4d1)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Corrections:
- environment
- accommodate
- conversion
- compatible
(From OE-Core rev: 9797d3b45b4e1b9d77f0f2ee299c17b48d8d3cf6)
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit adds a devtool build test for npm recipe:
- devtool.DevtoolAddTests.test_devtool_add_npm
(From OE-Core rev: af299817c2fe9fa504762c54bd0593f3163399fb)
Signed-off-by: Jean-Marie LEMETAYER <jean-marie.lemetayer@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There was a regression in this functionality that went unnoticed
due to lack of tests.
(From OE-Core rev: a75e4eceb703b2b13ddd4ba3dea3a86af0b0a9eb)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
inherits can be unset resulting in:
UnboundLocalError: local variable 'inherits' referenced before assignment
which can mask real errors. Avoid this.
(From OE-Core rev: 29a0502e101ed0667e960f9f8591b0a2d60a4bcb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
when build with EXTERNALSRC, checksum of EXTERNALSRC is one of inputs
for generate sigdata of task do_compile. without change any source, the
sigdata will not changed, and buildclean task only do make clean, and
not clean stampfiles like clean task, so the stampfile keep there, then
do_compile after do_buildclean will not rerun, and targetfile
tempdir_mdadm/mdadm will not be generated, report below error:
AssertionError: '/tmp/devtoolqarxv8ct6u/mdadm' does not exist
Fix by add a comments in Makefile to trigger rebuild of compile task
(From OE-Core rev: f4621cf4516f9de9980f3864a193f3f61c4abedc)
Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
if do clean after remove-layer, it will make stamp file not cleaned
since variable STAMP has changed, this will cause error like:
if testcase buildclean failed, when run next testcase modify, since
do_prepare_recipe_sysroot's stamp file exists, sysroot not populated,
then do_compile will fail with error:
/bin/sh: x86_64-poky-linux-gcc: command not found
(From OE-Core rev: 046d7e1de094105db0949c2cf5333ff55d9ad9c8)
Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
`devtool create-workspace' would create devtool.conf, so track to
clean it up. Otherwise, this devtool.conf file would mess things up.
e.g.
oe-selftest -r devtool && oe-selftest -r devtool -j 2
AssertionError: '/PROJ_DIR/build-selftest-st-15753/workspace/conf/layer.conf' does not exist : Workspace directory not created
This is because the devtool.conf is also copied to build-selftest-st-xxxx
directory, resulting in devtool to create and use workspace specified
in this file.
(From OE-Core rev: a0a96db23686a38235fb0373e75e8b2951216c18)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This adds SPDX license headers in place of the wide assortment of things
currently in our script headers. We default to GPL-2.0-only except for the
oeqa code where it was clearly submitted and marked as MIT on the most part
or some scripts which had the "or later" GPL versioning.
The patch also drops other obsolete bits of file headers where they were
encoountered such as editor modelines, obsolete maintainer information or
the phrase "All rights reserved" which is now obsolete and not required in
copyright headers (in this case its actually confusing for licensing as all
rights were not reserved).
More work is needed for OE-Core but this takes care of the bulk of the scripts
and meta/lib directories.
The top level LICENSE files are tweaked to match the new structure and the
SPDX naming.
(From OE-Core rev: f8c9c511b5f1b7dbd45b77f345cb6c048ae6763e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These IDs refer to testopia which we're no longer using. We would now use the test
names to definitively reference tests and the IDs can be dropped, along with their
supporting code.
(From OE-Core rev: 8e2d0575e4e7036b5f60e632f377a8ab2b96ead8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix another load of regex escape sequence warnings for newer
python versions.
(From OE-Core rev: bd2c125bb9c362b6122e99dfdf4e1cfe12c26a90)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
oe-selftest's devtool tests have been broken since commit 2457cd57
(oe-selftest: devtool: avoid parallel races by using temporary copy of
core) if meta is a symbolic link.
(From OE-Core rev: daba6c5a991b370709d17e51305334f55a3858ec)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some of the devtool tests make changes to files under meta/ -
legitimately since we want these tests to be working with real recipes
and associated files. Unfortunately with the new oe-selftest
parallelisation this can break other tests if files go missing at the
wrong time (among other scenarios). To avoid this issue, simply take a
copy of the core repository and use that for these tests. (We copy the
entire repository since changing the path of meta/ influences COREBASE
and thus we need to have things like scripts/ alongside as well).
(From OE-Core rev: 2457cd57b4195924ef127f497efa2f34f411e660)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows better parallelism between the different tests as currently
this block takes the longest time to execute. devtool tests are still
all grouped into the "devtool" module for ease of exection.
This also makes it easier to execute some subset of devtool tests for
testing devtool changes.
(From OE-Core rev: 75148c190dd4823947557e9a07f1722e817c1fea)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently some tests run in buffer mode and some don't. Those that don't can
corrupt stdout/stderr. Switch to using buffer mode everywhere so we're consistent.
If there is useful output on stdout/stderr, it will be displayed if the test
fails.
(From OE-Core rev: 978548c0abde2cb94c2782538552f39bdf2bf630)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If dbus isn't build first the dbus dependency of dbus-wait can't be detected
through pkgconfig and the test fails:
AssertionError: {'DEPENDS': {'dbus'}} != {}
- {'DEPENDS': {'dbus'}}
+ {} : Some expected variables not found in recipe: {'DEPENDS': {'dbus'}}
Ensure dbus is built and present in the sysroot.
(From OE-Core rev: 28699b4257436fb6079eafe50ca8cab09a2fdd90)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that we see warnings if LAYERSERIES_COMPAT is unset, the auto generated
code from devtool needs to set this to avoid warnings which break
various tests.
(From OE-Core rev: f65ebfeda0bfbac78e4a2a6609ba654ca38a8b0e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixed:
$ oe-selftest -r devtool.DevtoolTests.test_create_workspace
[snip]
2017-12-20 18:28:59,404 - oe-selftest - INFO - Traceback (most recent call last):
File "/workspace2/lyang1/poky/meta/lib/oeqa/core/decorator/__init__.py", line 32, in wrapped_f
return func(*args, **kwargs)
File "/workspace2/lyang1/poky/meta/lib/oeqa/selftest/cases/devtool.py", line 177, in test_create_workspace
self.assertTrue('/workspace' not in result.output, 'This test cannot be run with a workspace layer in bblayers.conf')
AssertionError: False is not true : This test cannot be run with a workspace layer in bblayers.conf
[snip]
$ bitbake-layers show-layers
NOTE: Starting bitbake server...
layer path priority
==========================================================================
meta /workspace2/lyang1/poky/meta 5
meta-poky /workspace2/lyang1/poky/meta-poky 5
meta-yocto-bsp /workspace2/lyang1/poky/meta-yocto-bsp 5
meta-selftest /workspace2/lyang1/poky/meta-selftest 5
There is no workspace layer, but I'm in /workspace2, this patch can fix the problem.
[YOCTO #12442]
(From OE-Core rev: 695b234ea4f034d428f8cffacceabc2b8f00bc74)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
mkelfimage is about to be removed, so test_devtool_modify_git needs to
use a different recipe. psplash is a reasonable choice given it uses a
git repository (and probably will forever), and doesn't have too many
dependencies, so change the test to use that recipe instead.
(From OE-Core rev: daf583efc87faa058684dfe34df596d088caa8ef)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
acinclude.m4 is about to be removed from the lzo recipe which breaks
test_devtool_update_recipe_local_files_2. Create a synthetic recipe in
meta-selftest with some local files and use that instead.
(From OE-Core rev: ed27470fa6a9f3cc1a0eb884474fe7985babde7a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This further cleans up the output of oe-selftest so that runqemu output
is hidden unless tests fail.
(From OE-Core rev: efa064d5026538ab513edc11869364ce2f14b977)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since it was first implemented, devtool's source extraction (as used by
the devtool modify, extract and upgrade subcommands) ignored other recipe
dependencies - so for example if you ran devtool modify on a recipe that
fetches from svn or is compressed using xz then it would fail if those
dependencies hadn't been built first. Now that we can execute tasks in
the normal way (i.e. tinfoil.build_targets()) then we can rework it to
use that. This is slightly tricky in that the source extraction needs to
insert some logic in between tasks; luckily we can use a helper class
that conditionally adds prefuncs to make that possible.
Some side-effects / aspects of this change worth noting:
* Operations are a little slower because we have to go through the task
dependency graph generation and other startup processing. There's not
really any way to avoid this though.
* devtool extract didn't used to require a workspace, now it does
because it needs to create a temporary bbappend for the recipe. (As
with other commands the workspace be created on the fly if it doesn't
already exist.)
* I want any existing sysroot files and stamps to be left alone during
extraction since we are running the tasks off to the side, and
especially devtool extract should be able to be used without touching
these. However, this was hampered by the automatic removal process in
sstate.bbclass triggered by bb.event.ReachableStamps when the task
signatures change, thus I had to introduce a way to disable this
removal on a per-recipe basis (we still want it to function for any
dependencies that we aren't working on). To implement this I elected
to use a file written to tmp/sstate-control which gets deleted
automatically after reading so that there's less chance of stale files
affecting future sessions. I could have used a variable but this would
have needed to be whitelisted and I'd have to have poked its value in
using the setVariable command.
Fixes [YOCTO #11198].
(From OE-Core rev: 830dbd66992cbb9e731b48d56fddf8f220349666)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code in scriptutils which implements the logic for running the
editor used by devtool edit-recipe looks at the VISUAL environment
variable before EDITOR, and thus if VISUAL is set in the environment it
will override the EDITOR value we are setting here, the editor (usually
vim) launches and there's nothing to stop it running forever short of
manually killing it. Set VISUAL instead to fix this.
Apparently VISUAL is in fact the variable we should really be preferring
here - I don't think I knew that but somehow I got it right in the code,
just not in the test. Here are the details for the curious:
https://unix.stackexchange.com/questions/4859/visual-vs-editor-whats-the-difference
Fixes [YOCTO #12074].
(From OE-Core rev: 6a7c50def569b5e86aa17bd9b287e8c63781dcb0)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use a tempdir to copy the .config file from the kernel instead of being
copied to build directory.
(From OE-Core rev: ccccf975c75055639c0ed5052ac0ad53ebbf53ca)
Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We weren't testing the devtool find-recipe and edit-recipe subcommands,
with the result that when they regressed recently we didn't notice. Add
some code into the test_devtool_add to test this (since we need a
recipe in the workspace, and adding a new test with all that preamble
would seem a bit excessive for these simple checks). Also take the
opportunity to refactor the test a little bit so that the recipe name
and version are variables rather than hardcoding them everywhere.
(From OE-Core rev: 355d8f42679e37610c2947dece597ed7db774bee)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
remake was removed from oe-core: use another recipe in the devtool
extract test.
(From OE-Core rev: dbf680ce09ce54f97c781fc7a8e5e05ed4706073)
Signed-off-by: Jussi Kukkonen <jussi.kukkonen@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Current osselftest print confusing assertion message when using
self.assertTrue(os.path.exists(filepath)) to test file path,
example of confusing assertion message:
AssertionError: False is not true
Replce assertTrue/assertFalse with assertExists/assertNotExists to test
file path, this will improve assertion message and simplify coding,
self.assertExists(filepath) will print below
AssertionError: <filepath> does not exist
[YOCTO #11356]
(From OE-Core rev: 08b0702492536d41d3cf2c9c05d1fab36d32a566)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Modify “test_devtool_virtual_kernel_modify” to be executed with default
configuration defined on oe-selftest test suite. A shorter string was
added to the the header file to avoid overlapped sections when building
kernel for qemux86-64.
[Yocto #11300]
(From OE-Core rev: a8bda790fded43eac72b2ad76d47c27363f02070)
Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
New framework has different classes/decorators so adapt current test cases to
support these. Changes include changes on base classes and decorators.
Also include paths in selftest/__init__.py isn't needed because the
loader is the standard unittest one.
(From OE-Core rev: ddbbefdd124604d10bd47dd0266b55a764fcc0ab)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
