The way this class uses overrides to support generation of multiple
sections is subject to two different issues: 1) labels that conflict
with existing override names causing the value for the conflicting label
to be set for all labels, and 2) reusing the override list through each
iteration, prepending each new label to the list of overrides makes
earlier labels' value take precedence over later labels, making later
labels virtually impossible to customize.
The first issue is resolved by removing all label names from overrides
before iterating over labels. The second issue is resolved by
generating a fresh list of overrides with only the current label added.
The current label is also appended to the list of overrides instead of
prepended, which makes it the highest priority override. This is
matches the behavior of devtool-source.bbclass, which similarly
monkey-patches overrides.
Closes https://bugzilla.yoctoproject.org/show_bug.cgi?id=13469 .
(From OE-Core rev: 42b44fdd9e63b8d39cf83baecec08328275558ff)
Signed-off-by: Will Page <wpage@polysync.io>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The old URI returns 404, and has an invalid TLS certificate.
(From OE-Core rev: 73ff6aba0a53ffc3ee0a5859a3ad4c8021be4de0)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Recursive RDEPENDS resolution requires that all of the dependent
recipes' packaging has completed. There is no mechanism to ensure that
and therefore races were observed.
This change effectively requires recipes to list their runtime file
dependencies explicitly rather than have them pulled indirectly.
This may require a bit of fixing in layers, but should result
in a better definition of runtime file dependencies.
(From OE-Core rev: 08e2b5a31b23b98459479caf6ec5f5c6fc19d57a)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a very old change as and be inferred from the name in the comment.
We've since had many changes to pkgdata including separating it
to its own sysroot now so the reasons for this blanket exclusion are
likely long gone.
If the shlib provides were really the problem I'd much rather have
a dedicated variable for that too.
Removing this fixes missing dependencies on nativesdk-libc and other
libs which would then happen automatically.
(From OE-Core rev: 096fa15efbcb704451b2f38ceab36508ef64f07e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
OE does not use the traditional /usr/lib/gcc prefix to store gcc-runtime
it basically is moved into libdir, however some newer files were
installed by newer versions of gcc especially libgomp ( omp.h openacc.h )
into gcclibdir, so we have content in both directories, this confuses
other tools which are trying to guess the gcc installation and its
runtime location, since now we have two directories, the tools either
choose one or other and we get inconsistent behavior, e.g. clang for
aarch64 uses /usr/lib but same clang for riscv64 chose /usr/lib/gcc
This change ensures that OE ends up with single valid location for gcc
runtime files
Move more common bits into common inc file
(From OE-Core rev: e9e5744ba8b0d43c8b874d365f83071ce20bf0a1)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Given grep expression looks for error strings regardless of the
semantics and this ends up it flagging things like
***********************
Log: /mnt/a/yoe/build/tmp/work/raspberrypi3_64-yoe-linux-musl/yoe-qt5-wayland-image/1.0-r0/target_logs/weston.log
-----------------------
Central error: EGL_KHR_create_context_no_error EGL_KHR_fence_sync
***********************
here EGL_KHR_create_context_no_error is an API, therefore the check
should be tightened a bit so explicitly look for error words so such
false positives don't show up
This helps in fixing wrong matches e.g. on rpi3 dmesg have strings like
above
(From OE-Core rev: e54ecea0a7d309533973b31bdbda178d9af92632)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This helps in skipping this test on images which are not using gtk and
as per the gcalculator website it needs gtk+3 or gtk+2 and libglade as
the build first looks for gtk+3
(From OE-Core rev: 2cdb4b3697e02947352803a9db013d61b812c4fd)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
fix below error:
file /usr/lib64/go/src/cmd/cgo/zdefaultcc.go conflicts between attempted installs of go-dev-1.12.6-r0.core2_64 and go-runtime-dev-1.12.6-r0.core2_64
file /usr/lib64/go/src/cmd/go/internal/cfg/zdefaultcc.go conflicts between attempted installs of go-dev-1.12.6-r0.core2_64 and go-runtime-dev-1.12.6-r0.core2_64
these 2 files existed in both go-dev and go-runtime-dev
remove it from go-runtime-dev to fix the problem
(From OE-Core rev: 60dcae509a210f2cae8eda525a1b006663aef9bf)
Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Its now past EOL so retire it.
(From meta-yocto rev: 03046c2fe69c399173cf6303abb55264adaee2b9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When the task hashes change we need to ensure the stampcache is cleared out
else tasks don't rerun when they should as we're basing decisions on stale
cache data.
(Bitbake rev: 08962092d3bb7887d82f97d442a6103c0677eae7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We weren't marking this special case of setscene task as buildable leading to
runqueue task failures.
(Bitbake rev: 930efbc563443d82df8d692bb8ff172ca2bae192)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Further testing shows we should test some extra datastructures to help pinpoint logic
errors more precisely. This adds some further data structure sanity checks.
(Bitbake rev: 83c4370b25c3a14cc946965c5c5f83ea28f488a1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This debug statement could result in a long list of tasks which when
repeatedly sent over our IPC, slowed down the builds immensely. Remove
it in favour of other more targeted debugging added recently, bringing
back some lost performance, particularly on builds with large numbers
of tasks.
(Bitbake rev: 85fe627fdb6510f0942917964386fad9d8c479c8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The event from the task notifiing of hash equivalency should only be processed
when the task completes. This can otherwise result in a race where a dependent
task may run before the original task completes causing various failures.
To make this work reliably, the code had to be restructured quite a bit.
(Bitbake rev: 1bf5be46f92f125193638cf41ff207d68f592259)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The changed_setscene variable here is just odd and not needed. Worse,
it could prevent some tasks from being removed from the holdoff tasks
list. The list is being rebuilt and should work as intended just from
the other data, this is a leftover from previous versions of the code
as far as I can tell.
(Bitbake rev: 030b9f2b3ce6ed40e79304eb0ffee6c6613f43be)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Whilst we had good runqueue failure mode debug, it hadn't adapted to the
scenequeue changes. Run the scenequeue sanity tests at the end of
a build and output the results regardless of whether all setscene tasks
completed or not. This *massively* improves the ability to debug runqueue
problems.
(Bitbake rev: b9b2177473c0b95a23bd519a201e1d2ba101c6c1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Looking at the profile information, a lot of time is being spent in
next_buildable_task. This is probably due to the generator expressions
not working well with the empty test.
The easiest way to improve things is to switch to using set manipulations.
We also don't need to update self.buildable the way the original code did
as we don't rely on that anywhere.
(Bitbake rev: 3bcf9ad4964b7e42d1a02ce231e9db42a81ead2a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If tasks are in the covered list of tasks for a given setscene task,
it needs to wait for those to complete before we can start.
(Bitbake rev: fdee640c26750b852eb68f5c80437377aa300ed8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
At exit the hashserv code was causing tracebacks as join() wasn't
being called from the thread that started the process. Ensure that
the hashserver is started from the pre_serve hook which is the
final thread the cooker runs in. This avoids the traceback at the
expense of some horrific poking into data stores which will ultimately
need improving through a proper API.
(Bitbake rev: 05888700e5f6cba48a26c8a4c447634a28e3baa6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add some extra hash equivalence runqueue tests based on recent scenarios
that caused problems during testing.
(Bitbake rev: 373b085ead992a725b2230ededd992b4c61a1a05)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The previous tasks_covered and tasks_notcovered were basically unstable
data structures. We couldn't always tell whether tasks should be covered
or not when trying to repair the sturcture if sstate tasks reran.
In the end its simpler to throw the lists away and rebuild them based upon
current data rather than trying to patch it adhoc. This turns out to be
simpler and much more reliable and I've much more confidence in this code.
(Bitbake rev: 52ee2ba2c617d928569f5afa404925c8b6f317bc)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need to copy this set, not modify the original else all kinds
of weird and bad things break, mostly from circular references.
We'll not go into how much sleep I lost tracking down the fallout
from this.
(Bitbake rev: 49927546d2b306830c98f6f9da4a6ad828f6a3a6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[YOCTO #13447]
When running bitbake-layers layerindex-fetch from 'master', there is a
circular dependency between meta-oe and meta-python. This triggered a maximum
recursion depth exception.
To fix the exception, as we walk down a branch (depth first search), we track
the layers we've already seen. If we are about to recurse into a layer we've
already seen we report a warning and then stop recursion.
(Bitbake rev: d6155d095513be3f500d089c4ed4c4b89949d560)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I rebuilt all cmake recipes from poky and meta-oe without issue.
(From OE-Core rev: b1ebf9122a5670044998adb83ca7a5a6c5454eaa)
Signed-off-by: Pascal Bach <pascal.bach@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add switch for building database verification, enable
this, it will solve the following issue:
root@qemux86-64:~# db_verify /var/lib/rpm/Packages
db_verify: BDB0571 library build did not include support for database verification
BDB5105 Verification of /var/lib/rpm/Packages failed.
(From OE-Core rev: a785157c7d78bd4becdc03c93a21a5dba37015b9)
Signed-off-by: Wenlin Kang <wenlin.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
-Refresh the following patch:
files/pie-flags.patch
-files/fix_SHAREDDIR.patch
Removed since this is included in 0.52.21.
(From OE-Core rev: 5b7fd7a945f8eeece51f01ad20a6de12fea89433)
Signed-off-by: Zang Ruochen <zangrc.fnst@cn.fujitsu.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Before meson conversion this was always enabled; now it needs to
be explicitly switched on.
This un-breaks the virgl headless test in particular, which regressed
with the meson conversion (and the test is not currently run on AB
due to oudated graphics hardware in it).
(From OE-Core rev: 22d3f959e44b77cb6d7375a77cf028ceeb9d6cb4)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously, 'nographic' was set first, and was overriding anything set via runqemuparams.
Particularly, egl-headless from the virgl headless selftest was overriden,
which broke the test in non-X environments.
(From OE-Core rev: 344c55a58042ecd6ffed8b20add41b6b335c51bf)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Like for ARM bjam need some hints about the ABI to properly build on
aarch64. While at it also enable context and coroutine as these are
supported on aarch64.
(From OE-Core rev: 5140e0a64aac8c621fe0d839dea41b7b43a96b4d)
Signed-off-by: Alban Bedel <alban.bedel@aerq.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* this might be related to:
commit d2fcaeb153fdc3f8d7143ea823139f1537055ff1
Author: Douglas Royds <douglas.royds@taitradio.com>
Date: Thu Dec 20 11:59:47 2018 +1300
icecc: Don't generate recipe-sysroot symlinks at recipe-parsing time
* it's still a bit unclear when and why this happends, but I'm seeing
random tasks sometimes failing with:
WARNING: Exception during build_dependencies for set_icecc_env
WARNING: Error during finalise of /build/meta-oe/meta-python/recipes-devtools/python/python-markupsafe_1.0.bb
ERROR: Traceback (most recent call last):
File "/build/bitbake/lib/bb/data_smart.py", line 411, in expandWithRefs
s = __expand_python_regexp__.sub(varparse.python_sub, s)
File "/build/bitbake/lib/bb/data_smart.py", line 136, in python_sub
value = utils.better_eval(codeobj, DataContext(self.d), {'d' : self.d})
File "/build/bitbake/lib/bb/utils.py", line 421, in better_eval
return eval(source, ctx, locals)
File "Var <set_icecc_env>", line 1, in <module>
File "/build/oe-core/meta/classes/icecc.bbclass", line 287, in icecc_get_and_check_tool
link_path = icecc_get_tool_link(t, d)
File "/build/oe-core/meta/classes/icecc.bbclass", line 246, in icecc_get_tool_link
return subprocess.check_output("readlink -f %s" % tool, shell=True).decode("utf-8")[:-1]
File "/usr/lib/python3.6/subprocess.py", line 336, in check_output
**kwargs).stdout
File "/usr/lib/python3.6/subprocess.py", line 418, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command 'readlink -f /build/BUILD/work/qemux86-oe-linux/python-markupsafe/1.0-r0/recipe-sysroot-native/usr/bin/x86_64-oe-linux/x86_64-oe-linux-g++' returned non-zero exit status 1.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/build/bitbake/bin/bitbake-worker", line 239, in child
the_data = bb_cache.loadDataFull(fn, appends)
File "/build/bitbake/lib/bb/cache.py", line 327, in loadDataFull
bb_data = self.load_bbfile(virtualfn, appends, virtonly=True)
File "/build/bitbake/lib/bb/cache.py", line 340, in load_bbfile
datastores = parse_recipe(bb_data, bbfile, appends, mc)
File "/build/bitbake/lib/bb/cache.py", line 303, in parse_recipe
bb_data = bb.parse.handle(bbfile, bb_data)
File "/build/bitbake/lib/bb/parse/__init__.py", line 107, in handle
return h['handle'](fn, data, include)
File "/build/bitbake/lib/bb/parse/parse_py/BBHandler.py", line 142, in handle
return ast.multi_finalize(fn, d)
File "/build/bitbake/lib/bb/parse/ast.py", line 386, in multi_finalize
finalize(fn, d)
File "/build/bitbake/lib/bb/parse/ast.py", line 351, in finalize
bb.parse.siggen.finalise(fn, d, variant)
File "/build/bitbake/lib/bb/siggen.py", line 147, in finalise
taskdeps = self._build_data(fn, d)
File "/build/bitbake/lib/bb/siggen.py", line 118, in _build_data
tasklist, gendeps, lookupcache = bb.data.generate_dependencies(d)
File "/build/bitbake/lib/bb/data.py", line 388, in generate_dependencies
deps[dep], values[dep] = build_dependencies(dep, keys, shelldeps, varflagsexcl, d)
File "/build/bitbake/lib/bb/data.py", line 317, in build_dependencies
value, parsedvar = d.getVarFlag(key, "_content", False, retparser=True)
File "/build/bitbake/lib/bb/data_smart.py", line 802, in getVarFlag
parser = self.expandWithRefs(value, cachename)
File "/build/bitbake/lib/bb/data_smart.py", line 424, in expandWithRefs
raise ExpansionError(varname, s, exc).with_traceback(tb) from exc
File "/build/bitbake/lib/bb/data_smart.py", line 411, in expandWithRefs
s = __expand_python_regexp__.sub(varparse.python_sub, s)
File "/build/bitbake/lib/bb/data_smart.py", line 136, in python_sub
value = utils.better_eval(codeobj, DataContext(self.d), {'d' : self.d})
File "/build/bitbake/lib/bb/utils.py", line 421, in better_eval
return eval(source, ctx, locals)
File "Var <set_icecc_env>", line 1, in <module>
File "/build/oe-core/meta/classes/icecc.bbclass", line 287, in icecc_get_and_check_tool
link_path = icecc_get_tool_link(t, d)
File "/build/oe-core/meta/classes/icecc.bbclass", line 246, in icecc_get_tool_link
return subprocess.check_output("readlink -f %s" % tool, shell=True).decode("utf-8")[:-1]
File "/usr/lib/python3.6/subprocess.py", line 336, in check_output
**kwargs).stdout
File "/usr/lib/python3.6/subprocess.py", line 418, in run
output=stdout, stderr=stderr)
bb.data_smart.ExpansionError: Failure expanding variable set_icecc_env, expression was if [ "${@use_icecc(bb, d)}" = "no" ]
then
return
fi
ICECC_VERSION="${@icecc_version(bb, d)}"
if [ "x${ICECC_VERSION}" = "x" ]
then
bbwarn "Cannot use icecc: could not get ICECC_VERSION"
return
fi
ICE_PATH="${@icecc_path(bb, d)}"
if [ "x${ICE_PATH}" = "x" ]
then
bbwarn "Cannot use icecc: could not get ICE_PATH"
return
fi
ICECC_BIN="${@get_icecc(d)}"
if [ -z "${ICECC_BIN}" ]; then
bbwarn "Cannot use icecc: icecc binary not found"
return
fi
if [ -z "$(which patchelf patchelf-uninative)" ]; then
bbwarn "Cannot use icecc: patchelf not found"
return
fi
# Create symlinks to icecc in the recipe-sysroot directory
mkdir -p ${ICE_PATH}
if [ -n "${KERNEL_CC}" ]; then
compilers="${@get_cross_kernel_cc(bb,d)}"
else
compilers="x86_64-oe-linux-gcc x86_64-oe-linux-g++"
fi
for compiler in $compilers; do
ln -sf ${ICECC_BIN} ${ICE_PATH}/$compiler
done
ICECC_CC="${@icecc_get_and_check_tool(bb, d, "gcc")}"
ICECC_CXX="${@icecc_get_and_check_tool(bb, d, "g++")}"
# cannot use icecc_get_and_check_tool here because it assumes as without target_sys prefix
ICECC_WHICH_AS="${@bb.utils.which(os.getenv('PATH'), 'as')}"
if [ ! -x "${ICECC_CC}" -o ! -x "${ICECC_CXX}" ]
then
bbwarn "Cannot use icecc: could not get ICECC_CC or ICECC_CXX"
return
fi
ICE_VERSION=`$ICECC_CC -dumpversion`
ICECC_VERSION=`echo ${ICECC_VERSION} | sed -e "s/@VERSION@/$ICE_VERSION/g"`
if [ ! -x "/build/BUILD/work/qemux86-oe-linux/python-markupsafe/1.0-r0/recipe-sysroot-native/usr/bin/icecc-create-env" ]
then
bbwarn "Cannot use icecc: invalid ICECC_ENV_EXEC"
return
fi
ICECC_AS="`${ICECC_CC} -print-prog-name=as`"
# for target recipes should return something like:
# /OE/tmp-eglibc/sysroots/x86_64-linux/usr/libexec/arm920tt-oe-linux-gnueabi/gcc/arm-oe-linux-gnueabi/4.8.2/as
# and just "as" for native, if it returns "as" in current directory (for whatever reason) use "as" from PATH
if [ "`dirname "${ICECC_AS}"`" = "." ]
then
ICECC_AS="${ICECC_WHICH_AS}"
fi
if [ ! -f "${ICECC_VERSION}.done" ]
then
mkdir -p "`dirname "${ICECC_VERSION}"`"
# the ICECC_VERSION generation step must be locked by a mutex
# in order to prevent race conditions
if flock -n "${ICECC_VERSION}.lock" \
/build/BUILD/work/qemux86-oe-linux/python-markupsafe/1.0-r0/recipe-sysroot-native/usr/bin/icecc-create-env "${ICECC_CC}" "${ICECC_CXX}" "${ICECC_AS}" "${ICECC_VERSION}"
then
touch "${ICECC_VERSION}.done"
elif ! wait_for_file "${ICECC_VERSION}.done" 30
then
# locking failed so wait for ${ICECC_VERSION}.done to appear
bbwarn "Timeout waiting for ${ICECC_VERSION}.done"
return
fi
fi
# Don't let ccache find the icecream compiler links that have been created, otherwise
# it can end up invoking icecream recursively.
export CCACHE_PATH="$PATH"
export CCACHE_DISABLE="1"
export ICECC_VERSION ICECC_CC ICECC_CXX
export PATH="$ICE_PATH:$PATH"
bbnote "Using icecc path: $ICE_PATH"
bbnote "Using icecc tarball: $ICECC_VERSION"
which triggered exception CalledProcessError: Command 'readlink -f /build/BUILD/work/qemux86-oe-linux/python-markupsafe/1.0-r0/recipe-sysroot-native/usr/bin/x86_64-oe-linux/x86_64-oe-linux-g++' returned non-zero exit status 1.
ERROR: Task (virtual:multilib:lib32:/build/meta-oe/meta-python/recipes-devtools/python/python-markupsafe_1.0.bb:do_patch) failed with exit code '1'
(From OE-Core rev: 577919d71f9be3de851c653e6981d715b37042ea)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update e2fsprogs from 1.44.5 to 1.45.3:
* rebase ptest.patch
* remove 0001-create_inode-fix-copying-large-files.patch which merged by upstream
* add new sub-package e2fsprogs-e2scrub and disable cron support
(From OE-Core rev: e5f8c1facfc3507710d7111951fe075e01c58e4e)
Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the removal of setuputils, there was a missing runtime dependency on pkgutil
which was breaking the selftests for gobject-introspection at runtime. Fix this.
(From OE-Core rev: 50048ff2da2cc209fe6bf1452dc2cf2a545fe886)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
python3-setuptools is an optional build-dependency of pygobject but
it is not necessary to run it, as also documented by the upstream
packaging guide:
https://gitlab.gnome.org/GNOME/pygobject/blob/master/docs/packagingguide.rst
Remove it from RDEPENDS to shrink the runtime dependency chain, as
setuptools includes scripts to download other packages, which might not
be desired on a minimal image.
(From OE-Core rev: 3054791e62c6ac1b03b7658f9c59761b254fa05c)
Signed-off-by: Luca Boccassi <luca.boccassi@microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With removal of python 2.x numpy, nothing needs it anymore.
(From OE-Core rev: 9f07b2836405d86e869780f8f1ae00843eadc409)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
