The test_lic_files_chksum_modified_not_mentioned test in TestMetadata is
outputting very long lines that fail the maximum length check when
sending email results, preventing the actual errors from being
displayed. Reduce the length of the failure message by rewording and
removing redundant information.
(From OE-Core rev: 2e2625735181160e9760a6f3af4955bda2ea6d4d)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Move most imports to the top of the file.
(From OE-Core rev: d2c287db0739b249604cd1beaa03ec38512ba718)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The autobuilder scripts post-process the generated JSON to inject recipe
and commit counts into the data. We can do this easily in patchreview
instead.
(From OE-Core rev: 77c96e43090cbf485aec612cc2315b85e5635dda)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A previous patch[1] added the ability to allow the search pattern for
patches to be changed, so that patchreview can be used across the entire
meta-oe repository by changing the patterns.
However, this means the caller needs to write long patterns when calling
patchreview.
Instead, we can see if the specified directory contains a layer by
checking if conf/layer.conf exists. If it does, then search for patches
inside this directory. If it doesn't, assume that the specified
directory is a repository that contains sublayers (such as
meta-openembedded) and look through each of the directories that match
the pattern meta-*.
This means patchreview can both scan either a single layer (eg
.../poky/meta) or a repository of sublayers (eg .../meta-openembedded).
[1] oe-core 599046ea9302af0cf856d3fcd827f6a2be75b7e1
(From OE-Core rev: a3a868519beab1b9cac94fefd7dbeffb09d047e9)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you try and fetch using mercurial-native, you see certificate errors since
it is configured to find ones in the sysroot, not the system. Add the missing
dependency so that mercurial recipes using the native tool work.
Found trying to make mirroring for old meta-oe stable branches work.
(From OE-Core rev: fc567e35b374f8b08975602609ee71e64357fb3d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was writing out locked-sigs.inc into cwd with every
'bitbake -S' invocation. When the intent is only to to get task
stamps (-S none), or print the difference between them (-S printdiff),
the file is unnecessary clutter.
A couple of selftests/scripts were however relying on this, so they're
adjusted to explicitly request the file.
eSDK code calls dump_lockedsigs() separately via
oe.copy_buildsystem.generate_locked_sigs() and so isn't affected.
(From OE-Core rev: ad57c3cac2a8d3e60222e3cca0685f582dcea135)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
'bitbake -S printdiff' is a useful diagnostic facility for finding out
why sstate is not being reused, but until now it had no tests that would
ensure it works. This commit adds three basic scenarios:
1. make a change in a really basic, common recipe that is at the very root
of dependency trees (quilt-native), and ensure that change is correctly discovered when
building an image.
2. make a change in gcc-source recipe, which is somewhat special
(operates in work-shared), and ensure that gcc-runtime builds track
that down as well.
3. make a change in base_do_configure() definition from base.bbclass,
which is not recipe-specific, but affects many basic recipes, and ensure that
is correctly reported as well.
The test itself actually runs twice:
- first against a fully populated build directory, where
the printdiff code is guaranteed to find the correct previous
stamp that can be compared with in a predictable manner.
- then in an empty build directory where the printdiff code
goes to look in the sstate cache, and so the existence of the
previous signature can be tested, but not the difference with it
(what the exact difference would be is unpredictable as the
sstate cache is indeed shared between many builds).
(From OE-Core rev: 7a7d76aa8a8d590ebc99156f9f4b9535cdf868c7)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes "2efb14648 toaster: Monitoring - implement Django logging system" when
running in a container.
When running in a container, the previous approach of using BASE_DIR
is not a writable path. Also, we really do not want to be writing logs into
the source tree, as the BASE_DIR was resolving to bitbake/lib/toaster/logs
Since Toaster is only ever running in an environment where oe-init-buildenv
or similar has been sourced, we should instead write the logs to BUILDDIR.
Using BUILDDIR to logs make path writable but django-log-viewer does'nt manage
to write logs using an absolute path as BUILDDIR, where the existing toaster_ui.log
was already being written.
Also drop the /logs/ directory, as it has not been created which also breaks
in a container environment
To handle the constraints linked to django-log-viewer and /logs/, we've updated
bitbake/bin/toaster to create a toaster_logs/ directory in BUILDDIR if it doesn't exist,
when toaster starts up.
Also manage to set BUILDDIR/toaster_logs/ as default location for toaster logs.
(Bitbake rev: efbd9d54f57be7a7a10f0b56e7e62c25974e99e6)
Signed-off-by: Alassane Yattara <alassane.yattara@savoirfairelinux.com>
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Tested-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using debug-btf.scc in a clean workspace, the CONFIG_MODULE_ALLOW_BTF_MISMATCH cannot
apply to kernel until clean the kernel code(bitbake linux-yocto -c cleanall) and rebuild.
After tracking the code, some options depend on CONFIG_PAHOLE_VERSION, it was generated by
scripts/pahole-version.sh in kernel, but during do_kernel_configme step, the pahole-native
is not available in sysroot-native, so need to wait pahole-native install to sysroot-native
before do_kernel_configme.
(From OE-Core rev: 217a4db53edbd88001f6390bbff39e5dd3d137af)
Signed-off-by: Xiangyu Chen <xiangyu.chen@windriver.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add 3 tests to check the creation of PEP-517 project using the 3
backends supported by bitbake:
- setuptools.build_meta
- poetry.core.masonry.api
- flit_core.buildapi
Theses tests requires the tomllib python module, so skip theses tests
if module is not present. tomllib module is part of python starting from 3.11
(From OE-Core rev: 54356c6f1290d0d4170ed52f7bb358bb9efc1aec)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add basic support for PEP517 [1] for the 3 following backends that are
supported by bitbake:
- setuptools.build_meta
- poetry.core.masonry.api
- flit_core.buildapi
If a pyproject.toml file is found, use it to create the recipe,
otherwise fallback to the old setup.py method.
Some projects can declare a minimal pyproject.toml file, and put all
the metadata in setup.py/setup.cfg/requirements.txt .. theses cases
are not handled. If a pyproject.toml file is found, assumes it has all
necessary metadata.
As for the old setup.py method, version numbers for dependencies are not
handled.
Some features may be missing, such as the extra dependencies.
[YOCTO #14737]
[1]: https://peps.python.org/pep-0517/
(From OE-Core rev: c7d8d15b2d0a9ecd210bd247fa0df31d9f458873)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to prepare the support for pyproject.toml (PEP517 [1]) enabled
projects, refactor the code and move setup.py specific code into a
specific class in order to allow sharing the PythonRecipeHandler class
No functionnal changes expected
[1]: https://peps.python.org/pep-0517/#source-tree
(From OE-Core rev: 2281e93347da4129062cfb40710df03c87c63168)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By convention, all python recipes start with "python3-" so update
create_buildsys_python to do this
This rule doesn't apply for packages already starting with "python"
Update recipetool's selftest accordingly
(From OE-Core rev: b0d87440e610b80f763d09784d4a90a148bb3e7b)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
License field of setup is not always standardized, so we usually use the
classifier to determine the correct license format to use in the recipe.
A warning note is added above the LICENSE field of the create recipe
in case a license is provided in setup. But when the plugin is called,
"LICENSE =" is not yet present so we can never display this note.
Replace the "LICENSE =" condition with "##LICENSE_PLACEHOLDER##"
to actually be able to display the note message
(From OE-Core rev: b7c26ca2028aa60f740464de85a11a01a531f32e)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
if user run devtool selftests with a local workspacelayer
the tests fail with various error such as:
- devtool.DevtoolAddTests.test_devtool_add just hangs
- devtool.DevtoolModifyTests.* fail with the following error:
ERROR: Found duplicated BBFILE_COLLECTIONS 'workspacelayer', check bblayers.conf or layer.conf to fix it.
Found duplicated BBFILE_COLLECTIONS 'workspacelayer', check bblayers.conf or layer.conf to fix it.
Check if a workspacelayer exists, warn the user and abort the tests
(From OE-Core rev: a74962cfb0485f6f2b9e2b751c33c8eafca8705a)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
os.environ does not pass this down to runstrip() function and in
strip_execs() its using STRIP bitbake variable to find the strip utility
to use. Since there might be a trailing whitespace in KERNEL_STRIP
remove that otherwise python is not able to launch it.
e.g.
FileNotFoundError: [Errno 2] No such file or directory: 'riscv64-yoe-linux-strip '
This is more evident when STRIP and KERNEL_STRIP are different utilities
e.g. when using clang as default toolchain but using gcc+binutils only for
kernel build.
(From OE-Core rev: 77497dbdca92ab4d6386a071bc281c42a7e8a14b)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Cc: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Commit 4a4d5f78a6 ("package_rpm: use zstd
instead of xz") changed the rpm package compressor from 'xz' to 'zstd'
which results in decompression failure with BusyBox-provided 'rpm2cpio'
applet and 'rpm' applet when given the '-i' (Install package) option:
rpm2cpio: no gzip/bzip2/xz magic
Introduce a variable which makes it possible to use a different
compression mode, making it possible to override the default value for
example like
RPMBUILD_COMPMODE = "${@'w6T%d.xzdio' % int(d.getVar('XZ_THREADS'))}"
to enable rpm decompression without including the full rpm package in
the resulting root filesystem.
(From OE-Core rev: a40d9258148e28cbee2168c93179cd4c1232fb62)
Signed-off-by: Niko Mauno <niko.mauno@vaisala.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some local variables defined in do_package_rpm() are not referenced, so
remove such dead code lines.
(From OE-Core rev: 5d387bc9001726937ffa7d3cfc333cfa31b681fb)
Signed-off-by: Niko Mauno <niko.mauno@vaisala.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add the missing conventional space characters around bitbake variable
assignment operators. Also fix a typo on a comment line.
(From OE-Core rev: 3dea51ce6c91dc2b12a5520dede51ec6357e87d5)
Signed-off-by: Niko Mauno <niko.mauno@vaisala.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix following subset of observations reported by version 2.10.0 of
pycodestyle utility:
meta/classes-global/package_rpm.bbclass:65:46: E231 missing whitespace after ','
meta/classes-global/package_rpm.bbclass:66:46: E231 missing whitespace after ','
meta/classes-global/package_rpm.bbclass:107:19: E231 missing whitespace after ','
meta/classes-global/package_rpm.bbclass:109:69: E202 whitespace before ')'
meta/classes-global/package_rpm.bbclass:122:103: W291 trailing whitespace
meta/classes-global/package_rpm.bbclass:194:74: W291 trailing whitespace
meta/classes-global/package_rpm.bbclass:448:16: E713 test for membership should be 'not in'
meta/classes-global/package_rpm.bbclass:450:16: E713 test for membership should be 'not in'
meta/classes-global/package_rpm.bbclass:520:1: W293 blank line contains whitespace
meta/classes-global/package_rpm.bbclass:521:15: E231 missing whitespace after ','
meta/classes-global/package_rpm.bbclass:542:12: E713 test for membership should be 'not in'
meta/classes-global/package_rpm.bbclass:544:12: E713 test for membership should be 'not in'
meta/classes-global/package_rpm.bbclass:647:67: W291 trailing whitespace
(From OE-Core rev: 7920599c05c066767025063b277df73c6560753d)
Signed-off-by: Niko Mauno <niko.mauno@vaisala.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This issue relates to a floating point exception in stress-test, which
is an unlikely security exploit at the best of times, but the test is
not installed so isn't relevant.
(From OE-Core rev: 72f2d4cf44b795f766ecdee0b8362c7e162c5efc)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Occasionally the cve-check tool will warn that it is adding the same
package twice. Knowing what this package is might be the first step
towards understanding where this message comes from.
(From OE-Core rev: c1179faec8583a8b7df192cf1cbf221f0e3001fc)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The JSON report generated by the cve-check class is basically a huge
list of packages. This list of packages is, however, unsorted.
To make things easier for people comparing the JSON, or more
specifically for git when archiving the JSON over time in a git
repository, we can sort the list by package name.
(From OE-Core rev: e9861be0e5020830c2ecc24fd091f4f5b05da036)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This CVE relates to a bug in the minizip tool, but we don't build that.
(From OE-Core rev: a32f285501b459cfe18e3135a3c531b63f58034c)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This CVE is a use-after-free which theoretically can be an exploit
vector, but this UAF only occurs when malloc() fails. As it's
unlikely that the user can orchestrate malloc() failures at just the
place to break on _this_ malloc and not others it is disputed that this
is actually a security issue.
The underlying bug has been fixed, and will be incorporated into the
next release.
(From OE-Core rev: b93dd888b861aa6df97cd78b70fa9f757cfcdf61)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since we are skipping the merge test, two of the selftests now report
SKIP instead of XPASS/XFAIL as expected. Adjust the two files to have
the right endings for XSKIP, and add the category so that it can be used
for more extensive testing in the future.
(From OE-Core rev: 3331f53c0be2575784a042bb2401eeba4f2a5a3e)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Check that the maximum line length of the testresult file is less than
220 characters, to help guard against malicious changes being sent in
email responses. If any line exceeds this length, replace the normal
testresults used in the response with a line stating that tests failed,
but the results could not be processed. Also clean up the respone
substrings slightly to go along with the change.
(From OE-Core rev: b0d53cf587dc9afb97f00c1089e45b758e96dd7c)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Building perf without security_flags.inc being included in one's
distro results in the buildpaths warning
WARNING: perf-1.0-r9 do_package_qa: QA Issue: File /usr/bin/trace in
package perf contains reference to TMPDIR
because the ${DEBUG_PREFIX_MAP} does not get used. Most recipes get
that from CFLAGS, but the perf recipe explicitly unsets that.
Now ${SELECTED_OPTIMIZATION} of course contains more than just
${DEBUG_FLAGS}/${DEBUG_PREFIX_MAP}. For most TUs, perf's build system
adds its own optimization flags (-O6 for odd reasons), so for those
including the -O2 or -Og doesn't change anything. But looking at the
.o.cmd files show that there are some TUs which currently get built
without any -O flag. So for those adding the distro's
SELECTED_OPTIMIZATION seem to be the right thing to do.
(From OE-Core rev: aa01c9122ef4a2159df503ef6ed25e802277f13a)
Signed-off-by: Rasmus Villemoes <rasmus.villemoes@prevas.dk>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Disable the merge-on-head test until patchtest properly handles merging
of series subsets and accounts for patches that are rapidly merged (i.e.
before patchtest is run).
(From OE-Core rev: e561c614dc72b7f8bf5e09a09bbe6ebc3cf500bb)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Pull the actual email's subject line from the .mbox file and use that in
patchtest's test results response, so that it's clearer which patch it
is replying to.
(From OE-Core rev: 98ca0b151517b3544454fd5c1656a2de631c4897)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a SECURITY.md file with hints for security researchers and other
parties who might report potential security vulnerabilities.
(Bitbake rev: baeaa73df2e2f2edc98f8779d57f3841d382d8fc)
Signed-off-by: Marta Rybczynska <marta.rybczynska@syslinbit.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Pre-fetching of LFS blobs now also works if a destsuffix parameter is
supplied.
This also fixes issues with Yocto Kirkstone (and newer) builds, where
fetching the LFS blobs is retried unsuccessfully during do_unpack, as
network access is blocked for any task other than do_fetch.
(Bitbake rev: e411dc07d032be6811d0393c50a06fc28e669b24)
Signed-off-by: Ragesh Nair <ragesh.nair@duagon.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To test custom image creation, a file for base_recipe should exists otherwise test fail,
User we need to build a base image first, that will create a file for base_recipe, which take a while.
To avoid test dependencies between test and run build, i create a tmp file for base recipe.
Signed-off-by Alassane Yattara <alassane.yattara@savoirfairelinux.com>
(Bitbake rev: 99c24df73831a72f311090b3aebcf3cc4e86851a)
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is printed by 'bitbake -S printdiff' and more accurately reflects what
the code does.
Most of the time the most recent item should be what the user wants to see when debugging sstate
misses, but there could also be printdiff-all (print differences with all
matching sstate objects), or printdiff-N (N most recent, and not just the latest).
(Bitbake rev: 315f48f965a54da5cec92908d91aa61c2d450add)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Create repository with few branches and test if fetcher can work
with such repository as PREMIRROR
(Bitbake rev: a1737610e5d5b61e126ec3632d7f27b337a87818)
Signed-off-by: Pavel Zhukov <pavel@zhukoff.net>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a "-l"/"--limit" option to allow changing the display limit in
resulttool.
- If no value is passed, resulttool uses its default value.
- If 0 is passed, the display limit is removed and every regression will be
displayed
- If a custom value is passed, this value overrides the vlaue configured in
resulttool
(From OE-Core rev: d3f536b3fc3f7027f6f5cf8bdaf5d7c050c7974b)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
