Changelog:
added optional encoding parameter for parsing files
fixed handling of the FilesAnalyzed field in Tag-Value format
fixed the validation of the DownloadLocation field
fixed the error handling while parsing license expressions
fixed output of timezone-sensitive datetimes
added code architecture documentation
(From OE-Core rev: 70cfcb9ffadf463c3f96b97925db958e8c65af9d)
Signed-off-by: Marta Rybczynska <marta.rybczynska@syslinbit.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The DynamicUser needs libnss-systemd to be installed to function
well. The tweaks to nsswitch.conf should be conditional on PACKAGECONFIG
values.
(From OE-Core rev: ba3a78c08cb0ce08afde049610d3172b9e3b0695)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #15225]
Yocto Bug #15255 is not reproducible. To obtain more useful information
for debugging, the OpenSSL test code is improved so that more detailed
state information in the handshake loop is printed when an error occurs.
(From OE-Core rev: 5bf9a70f580357badd01f39822998985654b0bfc)
Signed-off-by: William Lyu <William.Lyu@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update the COPYING checksum; only the copyright date changed.
Noteworthy changes in strace 6.6 (2023-10-31)
=============================================
* Improvements
* Implemented --kill-on-exit option that instructs the tracer to set
PTRACE_O_EXITKILL option to all tracee processes and not to detach them
on cleanup so they will not be left running after the tracer exit.
* Implemented automatic activation of --kill-on-exit option when
--seccomp-bpf is enabled and -p/--attach option is not used.
* Implemented decoding of map_shadow_stack syscall.
* Implemented decoding of FSCONFIG_CMD_CREATE_EXCL fsconfig command.
* Implemented decoding of IFLA_BRPORT_BACKUP_NHID netlink attribute.
* Implemented decoding of SECCOMP_IOCTL_NOTIF_SET_FLAGS ioctl.
* Implemented decoding of UFFDIO_CONTINUE, UFFDIO_POISON, and
UFFDIO_WRITEPROTECT ioctls.
* Updated lists of ARCH_*, BPF_*, DEVCONF_*, IORING_*, KEXEC_*, MAP_*, NT_*,
PTRACE_*, QFMT_*, SEGV_*, UFFD_*, V4L2_*, and XDP_* constants.
* Updated lists of ioctl commands from Linux 6.6.
ptest-runner results on qemux86-64/kvm with qemuparms="-m 1024 -smp 4":
============================================================================
Testsuite summary for strace 6.6
============================================================================
============================================================================
(From OE-Core rev: 12ae00d4cc4ed78643cafce97cd02e396b3348f4)
Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Backport the fix for the so_peerpidfd-test:
44cf51a38 tests: fix so_peerpidfd test
and drop the patch that skipped that test.
(From OE-Core rev: 0a52da6530ebba08f9a3b30d5098b0c3ef1730d7)
Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since ccache version 4.0, according to
https://github.com/ccache/ccache/blob/master/doc/NEWS.adoc#ccache-40
* An appropriate cache directory level structure is now chosen automatically. The cache_dir_levels (CCACHE_NLEVELS) configuration option has therefore been removed.
Therefore remove the option which has not been supported by ccache
recipe version since Yocto Hardknott.
(From OE-Core rev: 1d7c1b71e2639b8d069403920368292bd214347d)
Signed-off-by: Niko Mauno <niko.mauno@vaisala.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
machine features like vfat are needed for ptests to pass ( e..g. parted)
This brings it closer to what x86 qemu config looks like as well.
(From OE-Core rev: a6d97ecb25567ec7e56c061be9ab281b54b4d524)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Commit ef0654f145
("Set XZ_COMPRESSION_LEVEL to -9") changed the xz compression preset
level from previous value of -3 to -9. The commit message explains that
the change was made in order to be consistent with other compressors
that also use their best compression. However looking at xz man page,
under the compression preset level selection chapter there is mentioned
that
The differences between the presets are more significant than with gzip(1) and
bzip2(1). The selected compression settings determine the memory requirements of
the decompressor, thus using a too high preset level might make it painful to
decompress the file on an old system with little RAM. Specifically, it's not a
good idea to blindly use -9 for everything like it often is with gzip(1) and
bzip2(1).
which is then followed by a table, which mentions that the decompressor
memory requirement for preset -9 is 65 MiB, whereas for xz default
preset -6 it is just 9 MiB. Given that the use case where a device
running a Yocto generated Linux OS decompresses an ext4 root filesystem
image to non-volatile memory as part of firmware upgrade process is not
far-fetched, and considering that a range of these devices can run low
on available RAM when there are other applications running at the same
time, the lower decompressor memory requirement of the default preset
level makes sense in order to prevent an OOM situation from occurring.
This change was tested on a 32 CPU core build host with 128 GB RAM by
issuing
$ bitbake -c cleansstate core-image-minimal core-image-sato
$ time bitbake core-image-minimal
$ time bitbake core-image-sato
With MACHINE="qemux86-64" and IMAGE_FSTYPES="ext4 ext4.xz" using
XZ_COMPRESSION_LEVEL values "-6" and "-9". In both cases the resulting
'ext4' image size remained same, 38141952 bytes for core-image-minimal,
and 565043200 bytes for core-image-sato.
The observation was that with this change there is a small increase in
the resulting 'ext4.xz' file size, and a build speed improvement that
was significant for larger rootfs image.
core-image XZ real time time delta ext4.xz size size delta
-----------------------------------------------------------------------
minimal -9 0m44.992s 15932508
minimal -6 0m42.445s -5.66% 16243484 +1.95%
sato -9 2m40.828s 85080416
sato -6 1m38.891s -38.51% 87447456 +2.78%
Regarding decompression speed, issuing following command in qemux86-64
target OS
$ time xz -dkc --memlimit=MEMLIMIT core-image-sato-qemux86-64.rootfs.ext4.xz > /dev/null
using the lowest accepted value for MEMLIMIT for each case (providing a
lower value caused xz to exit with 'Memory usage limit reached' error)
showed that decompression time saw a minuscule improvement with the -6
compression preset level:
XZ MEMLIMIT real time
-------------------------
-9 65M 0m43.83s
-6 9M 0m43.28s
(In the above tables, XZ refers to XZ_COMPRESSION_LEVEL value used when
images were generated with Yocto).
(From OE-Core rev: 9ca62e24a6a0f5d2778b2b587646df7447e3c65f)
Signed-off-by: Niko Mauno <niko.mauno@vaisala.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The machines option to oe-selftest isn't used in our CI and is never likely to
be, we focus and execute testing explictly. The YOCTO #15247 is about how
this code doesn't interact well with build directory cleanup and at this point
I think we should just remove the option/code.
(From OE-Core rev: 815d04a2007e1154b69f1a027c8677ea86935354)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The cachegrind scripts have been rewritten in python3, so the RDEPENDS
on perl is no longer sufficient. This is unfortunately not caught by
QA checks since the scripts use
#! /usr/bin/env python3
as shebang line.
Since the valgrind binary by itself can be quite useful on target,
allow including that in the rootfs without also having to pull in the
whole perl and python interpreters and stdlibs.
(From OE-Core rev: 9797adebd1296f0cf08b1f6f6322fbc804aee14a)
Signed-off-by: Rasmus Villemoes <rasmus.villemoes@prevas.dk>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Commit c304fcbe05 introduced a grouping when
listing regressions. This grouping has been added only for ptests. It has
been observed that any other kind of tests could benefit from it. For
example, current regression reports can show the following:
1 regression(s) for oescripts.OEGitproxyTests.test_oegitproxy_proxy_dash
oescripts.OEGitproxyTests.test_oegitproxy_proxy_dash: PASSED -> SKIPPED
1 regression(s) for oescripts.OEPybootchartguyTests.test_pybootchartguy_help
oescripts.OEPybootchartguyTests.test_pybootchartguy_help: PASSED -> SKIPPED
1 regression(s) for oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_pdf_output
oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_pdf_output: PASSED -> SKIPPED
1 regression(s) for oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_png_output
oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_png_output: PASSED -> SKIPPED
1 regression(s) for oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_svg_output
oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_svg_output: PASSED -> SKIPPED
[...]
This output is not so useful in its current state and should be grouped per
test type too.
Enable grouping for all kind of tests, to make it llok like the following
in reports:
5 regression(s) for oescripts
oescripts.OEGitproxyTests.test_oegitproxy_proxy_dash: PASSED -> SKIPPED
oescripts.OEPybootchartguyTests.test_pybootchartguy_help: PASSED -> SKIPPED
oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_pdf_output: PASSED -> SKIPPED
oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_png_output: PASSED -> SKIPPED
oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_svg_output: PASSED -> SKIPPED
(From OE-Core rev: 982798ef96e3a32bf15341bdd3bb7c4356709412)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It was previously discovered that there was a race condition during the Makefile
execution between the assemble and compile targets, the previous fix attempted
to serialize the build targets, but the fix was missing for x86-64.
Pull in latest commit from upstream to fix this issue on x86-64.
[YOCTO #15146]
(From OE-Core rev: e7e1631a1efbcf421de801e94734f67f25668540)
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandro@enedino.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
FILES:${PN}-utils is += extended and than replaced completely later,
remove the first extension.
(From OE-Core rev: d9d61c5217938749e3edc5f8a5c987f46bbab3d7)
Signed-off-by: Rouven Czerwinski <r.czerwinski@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By removing libnss_mymachines.so.2 from FILES:${PN}-container it
correctly gets packaged into libnss-mymachines, just like the other
libnss-* components, due to PACKAGES_DYNAMIC.
To make up for the missing lib in the container package, I added
libnss-mymachines to RDEPENDS:${PN}-container. As I understand it,
the lib should be installed even without this if anything from the
container package links against it. But since I don't know if there
was a special reason to explicitly add it to the container files list,
like something using dlopen to load the lib, it seemd to be the
safest option.
(From OE-Core rev: 90f143984732ba0453c2d3a734eeb4d1a289269b)
Signed-off-by: Florian Wickert <fw@ferncast.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows easier replication of esdk environment (which provides
a curated, limited set of tools that for example does not include bitbake)
in a standard yocto build. Switchover between various sets can be achieved
via PATH manipulation.
(From OE-Core rev: 20c548f2edca3888152adb63de7b23d84e3848e7)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was added (I think) for the purpose of supporting layers that refer
to items outside of the layer via relative symlinks:
https://git.yoctoproject.org/poky-contrib/commit/?id=d31d1ad4e566e42d0bbcf1f41ac25e33181fb517
I do not think copying the link target into the layer that references it is the
correct solution: rather the original target should be included
into the SDK with the same relative path.
This change is done for the sake of preserving symlinks
that are referencing things inside the layer as they are;
particularly the content of scripts/esdk-tools/.
(From OE-Core rev: 52a7bbd5c4875c5f61ea65dda38e495a2925a20d)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was done in 2016 to support Eclipse plugin (long dead),
it's currently broken as image sysroot is not in the SDK
until the image is built in that context, and current tools
all rely on runqemu-export-rootfs which does not rely on PATH
and runs unfsd with full path to recipe-specific sysroots.
(From OE-Core rev: 0f1361061c8c0b16ea2b50349b08a3b03140c45c)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't make do_cve_check depend on do_unpack because that would be a
waste of time 99% of the time. The compromise here is that we can't
scan remote patches for issues, but this isn't a problem so downgrade
the warning to a note.
Also move the check for CVEs in the filename before the local file check
so that even with remote patches, we still check for CVE references in
the name.
(From OE-Core rev: 0251cad677579f5b4dcc25fa2f8552c6040ac2cf)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Not all mbox 'from' fields will contain angle brackets, so the
re.findall invocation used for getting a reply_address may fail. Use a
simpler reference to the field to get the sender's email address.
(From OE-Core rev: 86e9afe09a346586114133f5a7470304d2ed733f)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After patchtest went live it was determined that testing for a CVE tag
in the mbox commit message is unnecessary, since it will already be in
the shortlog and in any carried patches. Remove the test and the
associated selftest files so that its absence isn't flagged in future
test results.
(From OE-Core rev: 54690f18f04a2ab993a85d551ce4f8d0fa56618a)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
pylint 3.x has removed epylint, which is now a separate module. To avoid
adding another recipe or using outdated modules, modify the
test_python_pylint tests so that they use the standard pylint API.
(From OE-Core rev: 72be3d6a116febf46130cccbe12afe5ad93779b5)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rework the script for sending results to use send_raw_email and specify
the 'In-Reply-To' field so that patchtest replies to the emails, rather
than sending them standalone to the submitter and mailing list.
(From OE-Core rev: 0c45c92e7f26aea4edf2cfa577b7ba51384e59d3)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Specifically, the test checks that everything needed for building
standard oe-core images for x86_64 and arm64 is available from
the cache (with minor exceptions). Going forward, a complete
world check could be enabled and additional configurations,
but that requires improvements to performance of hash equivalence
server in particular.
RP: I've disabled the tests by default so we can merge them. We will
make them live once we get to the bottom of the failures.
(From OE-Core rev: 5f3aeadb65d3b7216db783b2c500ac241b03deb8)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Updated tox file to fix enviroments variables.
to run tox use the following command:
tox -e env_name -c /path/to/file/tox.ini
See tox cli https://tox.wiki/en/latest/cli_interface.html
The variable env_name is linked to the python versions (py38, py39, py310).
If env_name option is not set, the test suite will run for all indicated versions.
It was also modified the webdriver options for chrome to run selenium test in console because of the error detected in the previous test run
selenium.common.exceptions.SessionNotCreatedException: Message: session not created: Chrome failed to start: exited normally.
(session not created: DevToolsActivePort file doesn't exist)
(The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
(Bitbake rev: 11dfd7c05642269b7a8bff2918667348d041ec79)
Signed-off-by: Marlon Rodriguez Garcia <marlon.rodriguez-garcia@savoirfairelinux.com>
Changes in v2
- Updated variable path BUILDDIR and EVENTREPLAY_DIR to use enviroment value
- Updated toaster-requirements.txt file path
- Removed flake8
- Added webdriver options to chrome
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Drop three backport patches as they're applied upstream.
(From OE-Core rev: 7480ff9064a74858e83e0ae275dbdb78dd6fc2a4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The appendVar setting should have a preceding space, otherwise, when
KERNEL_MODULE_SPLIT is set to "0", we'll sometimes get dependency error
due to lacking of space.
(From OE-Core rev: 266cd948d4aa68de34075e8ed6299f7d80d19346)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Modify patchtest-send-results so that it extracts the submitter's email
address and responds to them with the patch testresults. Also make a
minor adjustment to the suggestions provided with each email and include
a link to the Patchtest wiki page for additional clarification on
specific failures.
(From OE-Core rev: 64ed88e32cf9e04772319ff6e66c602d1cff4fd7)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
git show-ref looks at the _remote_ ref called HEAD, which is fine when it
matches the local HEAD but problematic when you're iterating a series of
commits.
Use rev-parse to resolve the local name to a proper hash.
(From OE-Core rev: 3c04747b681cf6090ba9c77752f6c2f304dbbe17)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
- Mention CVE_STATUS_GROUPS in the development manual
(otherwise only present in the reference manual, but with
no reference to it)
- In the reference manual description of CVE_STATUS,
link back to the development manual, to provide context.
(From yocto-docs rev: cfef5fe41b6c819e783c88829448ae38141650a5)
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
ccache used to support FORTRAN (versions 3.3-3.6) but no longer does
(From OE-Core rev: 0cc2c0a9ec16fceeb19e01cd47af8c0743168030)
Signed-off-by: david d zuhn <david.zuhn@sonos.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The hooks are pulled from the impure environment and are often broken in
our environments. There is no reason to add extra metadata or verify the
commit message as its arbitrary to turn the tarball into a git repo.
(From OE-Core rev: dd52102a6c3f69d156bfcf85827123c96f18d5a1)
Signed-off-by: William A. Kennington III <wak@google.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By default, /var/lib is bind mounted on /var/volatile/lib. If this is
the case, the recipe adds conditions on systemd-random-seed in the
service file mounting it. But as the VOLATILE_BINDS may be modified,
/var/lib may be mounted elsewhere, for example in /persistent/var/lib.
In this case, the conditions are not set because the service file name
does not match expected one.
This patch automatically records the name of the service mounting
/var/lib, if any, in order to set the condition in the appropriate file.
(From OE-Core rev: 66f0c2a1678cb69cf8d50372b0592c55e2dc3e3c)
Signed-off-by: Stéphane Veyret <sveyret@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The mount-copybind script will create the parent directory of the bind
mount if it does not exist. But actually, if this is the case, the
service will not even start because of the ConditionPathIsReadWrite.
This patch adds a "or" condition to allow the service to start also if
the parent directory of the bind mount does not exist.
(From OE-Core rev: 1ca031b77546056ca1994469b0f2e93ea2018edf)
Signed-off-by: Stéphane Veyret <sveyret@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
