cf9df9e8d89f ("ipk: Decode byte data to string in manifest handling")
did a bit of least-effort fix to a string vs byte sequence issue in the
manifest handling. The approach was chosen as it localised the fix,
rather than having to analyse further call sites.
However since then f2167ae80258 ("package_manager/ipk: do not pipe
stderr to stdout") was applied, reworking the output handling from the
subcommand. dummy_bytes() now returns a string, so stop trying to decode
it.
Fixes: f2167ae80258 ("package_manager/ipk: do not pipe stderr to stdout")
Cc: Curtis Meier <cmeier@us.ibm.com>
Cc: Pam Eggler <eggler@us.ibm.com>
(From OE-Core rev: b61739554780d70307d2b6b37d2b3b1c7df93c77)
(From OE-Core rev: 29bd0b6272680769b438f39fa96b47d21c846f37)
Signed-off-by: Andrew Jeffery <andrew@aj.id.au>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 07e5a6331be60d5e35d7336a6215a972ced6eb57)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
"sysconfdir" contains "/" by definition and thus using os.path.join()
leads to self.target_rootfs being always ignored (and thus attempting to
generate paths in host's /etc).
Use oe.path.join() instead which was made for this purpose.
(From OE-Core rev: 116e2b1d0c75c6be333339812468550145915774)
Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 8414c504138f6de663f5130c6b4a6ede5605d88b)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
This is for a specific case where:
- A recipe use a subpath on a git repo (e.g. git://repo.git/projects;subpath=subproject)
- The recipe contains a patch to apply
- a devtool modify is used on this recipe
With these conditions, the patch cannot be applied at all.
GitApplyTree class is used for handling patch under devtool, but
when subpath is present in SRC_URI, the resulting git tree
is dirty (every files and directories which was not in subpath are suppressed)
and so "git am" refuse to apply patches.
That would not be an issue since the GitApplyTree have a fallback
to PatchTree in case of error, but during this error management,
there is a "git reset --hard HEAD" call which suppress the subpath
operation and finally prevents the patch to be applied even with PatchTree.
When devtool is not involved, only PatchTree class is used and the
above problem is irrelevant.
To support git patching during devtool, the presence of subpath and
the dirtyness of the repo are checked. If both conditions are
met, we directly call PatchTree like it was already done
in case of error during git apply.
(From OE-Core rev: d86cac2759cf7e91f4ff240833385e28e729ab79)
Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Move the signature file into place only after it is successfully signed.
This to avoid race and corrupted .sig files in cases multiple onging
builds write to a shared sstate-cache dir.
(From OE-Core rev: b4ec08ea9efebac262d43f47d95a356fe2829de9)
Signed-off-by: Tobias Hagelborn <tobiasha@axis.com>
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
NVD DB store version and update in the same value, separated by '_'.
The proposed patch check if the version from NVD DB contains a "_",
ie 9.2.0_p1 is convert to 9.2.0p1 before version comparison.
[YOCTO #14127]
Reviewed-by: Yoann CONGAL <yoann.congal@smile.fr>
(From OE-Core rev: 7d00f6ec578084a0a0e5caf36241d53036d996c4)
Signed-off-by: Geoffrey GIRY <geoffrey.giry@smile.fr>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, the latest version abseil-cpp contains a new library named "absl_log_internal_format", it's
basic package config(.pc file) as below:
prefix=/usr
exec_prefix=${prefix}
......
Requires: absl_config = 20230125, absl_core_headers = 20230125, absl_log_internal_append_truncated = 20230125,
absl_log_internal_config = 20230125, absl_log_internal_globals = 20230125, absl_log_severity = 20230125,
absl_strings = 20230125, absl_str_format = 20230125, absl_time = 20230125, absl_span = 20230125
......
Normally, the process_pkgconfig() would process variable data before field data in a .pc file, but in the
absl_log_internal_format, the field data in "Requires" section contains "xxxx = xxxx" format, the
process_pkgconfig() treats them as normal variable and using the setVar() in bitbake's data_smart.py
try to process. The absl_log_internal_format field data contains "_append_", this hit the setVar() checking
and finally bitbake stop building and reporting an error as below:
"Variable xxx contains an operation using the old override syntax. Please convert this layer/metadata before attempting to use with a newer bitbake."
This patch move the field data process before variable process to avoid the process_pkgconfig() treat the field
data as variable.
(From OE-Core rev: a73e269d3e591a10bb397b94b82e3fb960112d33)
Signed-off-by: Xiangyu Chen <xiangyu.chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* to be used by standalone script scripts/contrib/patchreview.py
as well
(From OE-Core rev: c326efeec8f576200728a44c694becdeab4fe2db)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Moves the downloaded items from SRC_URI into separate packages in the
recipe document. This is much better than the previous implementation
because:
1) It can report multiple download locations in SRC_URI, instead of
just the first one reported.
2) It prevents the assumption that the source files listed in the
recipe are the exact file from the source URL; in particular, files
that come from file:// SRC_URI entries, and source files that have
been patched were problematic, since these aren't from the upstream
source.
3) It allows the checksums to be specified
(From OE-Core rev: 1dd4369b3638637a2cbba2a3c37c6b6f4df335cd)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Symbolic links to the files are included during the output hash
calculation but symlinks to the directories are missed.
So if the new symlink to a directory was the only change made,
then the output hash won't change,
and the Hash Equivalence server may change unihash.
In the next run bitbake may use an older package from sstate-cache.
To fix this followlinks=True flag could be set for os.walk
but it can lead to infinite recursion if link points
to a parent directory of itself.
Also, all files from a directory to which symlink points
would be included in depsig file.
Therefore another solution was applied, I added code that will loop
through directories and process those that are symlinks.
(From OE-Core rev: ee729163f31f26b1462a47e1e53f7a0f9de9b464)
Signed-off-by: Mateusz Marciniec <mateuszmar2@gmail.com>
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously, if "showSignature" is present in user gitconfig, parsing
of the timestamp will fail.
Ideally we should replace this command with a git plumbing command.
(From OE-Core rev: 3bd6f78f79b3d3e87d8db1e11f58d8021f929843)
Signed-off-by: Benoît Mauduit <bmauduit@beneth.fr>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This function is used by the packaging tasks/classes and makes much
more sense in the packagedata function library.
(From OE-Core rev: 7a512a8803101310772d83836e6b78ebaf8121de)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Move the bulk of the remaining package "processing" functions over
to the package function library for parsing efficiency.
(From OE-Core rev: f8785117074908330faca0b99afa7f60ed6ad952)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Move one of the PACKAGEFUNCS from the package bbclass to packagedata
library code for parsing efficiency.
(From OE-Core rev: ceba33bf2897f7dd5b1ffe6b742c47bf616243c8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To avoid reparsing the bbclass code all the time, move the functions
to the packagedata python function library code which is more efficient.
(From OE-Core rev: f520a3039540b1183b1b2bdaaf8b9195995c0187)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To avoid reparsing the bbclass code all the time, move the functions
to the python function library code which is more efficient.
(From OE-Core rev: 424e65627c018b3119050f515b0c7cfb43be5573)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To avoid reparsing the bbclass code all the time, move the function
to the python function library code which is more efficient.
(From OE-Core rev: 7c8c4cfd9355a9cee1144e0444e1b54402f1951c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We're now used to using hashes as part of the task hashes and the sstate code
relies on this. The older OEBasic hash approach therefore wouldn't work and
can be removed.
(From OE-Core rev: 3667e589ba16eb261cfd72c2b11429f482c239f6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since bitbake now supports an official method to inject python modules,
switch to it.
Anyone using OE_EXTRA_IMPORTS will need to adjust their code accordingly,
probably switching to their own module namespace.
Also switch to using BB_GLOBAL_PYMODULES to list the global modules
to import.
(From OE-Core rev: 1f56155e91da2030ee0a5e93037c62e1349ba89f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since oe-core commit 64b89f3c8fc31842256c482a3039d90d3f12c1cc
("sstatesig.py: make it fatal error when sstate manifest isn't found")
errors like:
| Manifest [..]/tmp/sstate-control/manifest-x86_64_x86_64-nativesdk-dbus.populate_sysroot not found in imx8mm_dummy cortexa53-mx8mm cortexa53 armv8a-crc armv8a aarch64 allarch x86_64_x86_64-nativesdk (variant '')?
are fatal now and cannot be ignored but must be debugged.
Unfortunately, the currently emitted error message is a bit imprecise
with telling the reader what has actually gone wrong.
This commit:
* adds the word 'sstate' to the error message to clarify the scope we
are dealing with ('sstate manifests', since there are other manifests,
too)
* does not randomly print the last manifest file searched for as THE
manifest file that could not be found
Instead, we print the name of the task the sstate was searched for
* adds the word 'multilib' to variant to make clear which variant we are
talking about
* adds a separate line noting the searched pkgarchs and adds explicitly
mentions this word ('pkgarchs')
* prints a list of ALL manifest file locations attempted
* removes the '?' at the end of the message since such errors indeed
leave the question of what is the cause but the error message itself
is more like a statement.
The result for the exact same issue as noted above then looks as
follows:
| The sstate manifest for task 'dbus:populate_sysroot' (multilib variant '') could not be found.
| The pkgarchs considered were: imx8mm_dummy, cortexa53-mx8mm, cortexa53, armv8a-crc, armv8a, aarch64, allarch, x86_64_x86_64-nativesdk.
| But none of these manifests exists:
| [..]/tmp/sstate-control/manifest-imx8mm_dummy-dbus.populate_sysroot
| [..]/tmp/sstate-control/manifest-cortexa53-mx8mm-dbus.populate_sysroot
| [..]/tmp/sstate-control/manifest-cortexa53-dbus.populate_sysroot
| [..]/tmp/sstate-control/manifest-armv8a-crc-dbus.populate_sysroot
| [..]/tmp/sstate-control/manifest-armv8a-dbus.populate_sysroot
| [..]/tmp/sstate-control/manifest-aarch64-dbus.populate_sysroot
| [..]/tmp/sstate-control/manifest-allarch-dbus.populate_sysroot
| [..]/tmp/sstate-control/manifest-x86_64_x86_64-nativesdk-dbus.populate_sysroot
(From OE-Core rev: 735ec126ec219c7cb89cb05b0e433201bb7f59eb)
Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We can skip the rm_work task signature to avoid running the task
when we remove some tasks from the dependencie chain.
The inject_rm_work handler on the rm_work bbclass triggers the
rm_work task running for any signature change in the dependencie
chain of the task do_build of each recipe.
i.e INHERIT:remove = "create-spdx" will trigger the do_rm_work
when we collect the sstate cache with INHERIT = "create-spdx"
(From OE-Core rev: 292305700e39d0ebd64763f5032c39ace5005fad)
Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When machine configuration defines a mount point, which is not used in
any recipe, allow to fall through and only report a note in the logs.
This can be expected behavior, when a mount point is defined for several
machines, but not used in all of them
(From OE-Core rev: a9c604b5e0d943b5b5f7c8bdd5be730c2abcf866)
Signed-off-by: Vyacheslav Yurkov <Vyacheslav.Yurkov@bruker.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The _exec_shell_cmd function is used by a couple of other functions that
are ultimatelly called from the create_rootfs function. The latter is
used in image bbclass' do_rootfs which is running using the fakeroot
support in bitbake. This makes the fakeroot support in _exec_shell_cmd
redundant and never actually used.
(From OE-Core rev: d1439c41049e3cf201939bbc2b1fee58a46137a1)
Signed-off-by: Andrei Gherzan <andrei.gherzan@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We run depmod (through depmodwrapper) at the end of the rootfs
generation process. This part of the process assumes in its current
implementation that the kernel package name is always 'kernel' and that
there is only one set of kernel modules for which we need to generate
the modules.dep and map files.
The kernel package name can be configured via a variable
(KERNEL_PACKAGE_NAME) and becomes a namespace that enables the build
system to deal with multiple compiled kernel recipes. This patch checks
for all the depmod pkgdata and runs depmod for each of the detected
kernel versions/kernel package name.
(From OE-Core rev: efa88e1c227d695319197f511701e0230d301f39)
Signed-off-by: Andrei Gherzan <andrei.gherzan@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Theres not a rootfs for baremetal images, hence we should avoid
trying to list rootfs packages for them.
This fixes an issue where some classes (e.g. license_image) rely on
rootfs functionality when included for baremetal images even if its
nonexistent
(From OE-Core rev: 5a7e13f46083ce3b08aa762238c1e93b7626dda4)
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandro@enedino.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since we're keeping cve-check aligned between the active branches,
and dunfell is supported on Python 3.5, we can't use f-strings.
(From OE-Core rev: 1821cf7464cbba521b55a9c128fe8812c0cc5eca)
Signed-off-by: Ernst Sjöstrand <ernstp@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Where there isn't a copyright statement, add one to make it explicit.
Also add license identifiers as MIT if there isn't one.
(From OE-Core rev: bb731d1f3d2a1d50ec0aed864dbca54cf795b040)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The correct field name is "supplier" according to SPDX schema.
The "supplier" field translates to "PackageSupplier", but that's for
tag-value format.
(From OE-Core rev: ca8db0e0a2860ac1e3f537471fa71b43c3be0a58)
Signed-off-by: Mihai Lindner <mihai.lindner@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some opkg commands print an error during cleanup when the tmp_dir
does not exist and an attempt is made to delete it. The error messages
are harmless and the opkg commands eventually succeed.
When these commands are run and stderr is piped to stdout, the error
messages may clobber the stdout and cause unexpected results while
parsing the output of the command. Therefore, when parsing the output
of a command, do not pipe stderr to stdout. Instead, capture stderr
and stdout separately, and upon success, send stderr to bb.note().
(From OE-Core rev: f2167ae80258253eb47a5b148546b265320284cc)
Signed-off-by: Shruthi Ravichandran <shruthi.ravichandran@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In bc, we found that there are files newer than Changelog (e.g. scan.l)
which means after install runs, the timestamp is clamped to SDE which means
"bitbake bc -C compile" would then try and regenerate generated files and
complain flex was missing.
Rather than assuming Changelog/NEWS are magic for dates, drop that scan
method and scan all unpacked files. This shouldn't be that slow as they'd
be in the disk cache already after an unpack.
For bc, this changes SDE by about two minutes and avoids the problems
rebuilding.
Add a version comment to the task definition as changes in python library
code won't trigger a rebuild and we need one here.
(From OE-Core rev: 32dda0ad91a9e7946351c897578b4c97ae142341)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
SDPX generation involves looking through BB_TASKDEPDATA for
dependencies, then linking to the generated documents for those
dependencies. These document links use a checksum to validate the
document, which means that if a upstream document changes, all
downstream documents must be regenerated to get the new checksum,
otherwise the compendium of documents produced by the build will have
broken links; therefore all dependent task should be included in the
signature (even from "ABI safe" recipes).
(From OE-Core rev: 5fe543b9ceec971cf0297ff0ae3b0ccc4703cece)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The ThreadedPool in OE-core is mainly because python2
doesn't have threaded pools but python2 is dead for
some time now and python3 have a ThreadPoolExecutor.
The only local in OE-core where this ThreadedPool
is in use is on the sstate.bbclass that is ported
to the python3 ThreadPoolExecutor.
(From OE-Core rev: aa8fd5e7c2a1125895accfd55ce9320819a10959)
Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If devtool finish needs to create a patch and have it applied to the
sources for a recipe where S points to a subdirectory of the sources,
then the patch needs to be applied at the root of the repo i.e. we need
to add a patchdir= parameter to the SRC_URI entry.
(From OE-Core rev: ad3736d9ca14cac14a7da22c1cfdeda219665e6f)
Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If PATCHTOOL = "git", SRC_URI fetches from a git repo and S points to
a subdirectory of the checked out sources, then we were erroneously
initialising the subdirectory as its own git repo. Check if the returned
top-level repo directory is a subdirectory of WORKDIR and do not
run initialise the source directory if that is the case.
(This was a regression introduced with OE-Core revision
6184b56a7a0fc6f5d19fdfb81e7453667f7da940, however we didn't have a test
that verified the behaviour.)
(From OE-Core rev: 9cca53a2bcbf6809615ce5626c86c6ee481a7a76)
Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We've some long standing bugs where the RDEPENDS from -dev packages causes
problems, e.g. dropbear and openssh components on an image working fine together
but then the SDK failing to build as the main openssh and dropbear packages
conflict with each other (pulled in by openssh-dev and dropbear-dev).
We propose changing the behavour of complementary package installation to
ignore RRECOMMENDS. If we then change the ${PN}-dev dependency on ${PN}
to a RRECOMMENDS, we can avoid many of the issues people run into yet still
have the desired behaviour of ${PN}-dev pulling in ${PN}.
This therefore changes the package manager code so that it doesn't follow
RRECOMMENDS for completementary package globs.
[RP: Added deb support]
(From OE-Core rev: b44b0b9294675f89aa51ff84f532664f4c479677)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Helper module to:
- generate meta information from package.json content. This data has
a format as provided by https://registry.npmjs.org
- put this meta information and the corresponding tarball in the
nodejs cache. This uses an external, nodejs version specific helper
script (oe-npm-cache) shipped in oe-meta
To avoid further nodejs version dependencies, future versions of this
module might omit the caching completely and serve meta information
and tarball by an http server.
(From OE-Core rev: 6cd5886ad05fee704e8a5892bd370c360c8c3b54)
Signed-off-by: Enrico Scholz <enrico.scholz@sigma-chemnitz.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Reduce code duplication to a common function in oe.buildcfg.
(From OE-Core rev: 0b191a1df4c3722defb09dde0c16b1d9e7fe7ef6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This isn't used anywhere and everyone is using git now, the functions
are now hardcoded as such too. Drop this function.
(From OE-Core rev: 09ba96a2d7fa02c7a82758bbf4109b04ffca2c55)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The comments even say this was copy/paste code. Move to a
shared library function.
(From OE-Core rev: ac3de2f850a418673b87e1c454970cb099e191b0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is a load of duplicated git/layer/revision code which makes
most sesne as a python library, not bbclass code. Start to refactor as such.
(From OE-Core rev: 439cdf8a1e52fd2c4dc81dc40ce7e6af282ce7ac)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With CONFIG_MODULE_COMPRESS_ZSTD enabled, kernel modules will have a
.ko.zst extension. This fixes depmod not being run.
Fixes: 1b696a45ddb ("rootfs.py: Add check for kernel modules before running depmod")
(From OE-Core rev: 425efac7110f0f42d70643e0a448e834d0f01a7a)
Signed-off-by: Sean Anderson <sean.anderson@seco.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Move the function to a library, it could be useful in other places.
(From OE-Core rev: debd37abcdde8788761ebdb4a05bc61f7394cbb8)
Signed-off-by: Marta Rybczynska <marta.rybczynska@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
