create_npm.py duplicated the logic for matching licenses from files and
also finding them. This patch refactors the code to reuse the license
utils. This will make the code more maintainable and also align both
behaviors. For instance, some licenses weren't matched properly because
the duplicate logic did not support the difference in format in the md5
tables for COMMON_LICENSE_DIR and licenses.csv.
This is also faster since the license files were being read twice.
The result is slightly more accurate since the utils have better
implementations, and I was able to reuse the logic for the root PN
package, as well as the base LICENSE variable.
I chose to extract generate_common_licenses_chksums into create.py
since it can be considered a general utility function to allow
other recipetool creators to refer to COMMON_LICENSE_DIR files.
I updated the wording in the code when appropriate.
v3:
- added commit
- this replaces the commit that added all the COMMON_LICENSE_DIR md5
to licenses.csv
(From OE-Core rev: 7bc18bed63b94689890bcde63402d7cc1cedffa9)
Signed-off-by: Enguerrand de Ribaucourt <enguerrand.de-ribaucourt@savoirfairelinux.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The npm recipetool handler redefines the license code the could be
unified. In order to do this refactoring, extract the bits we'll
need into separate functions.
guess_license() is renamed to find_licenses() and is split into
find_license_files() and match_licenses().
(From OE-Core rev: f1ec28feaea8ea6a2df894dd4ddba561c8a04ed2)
Signed-off-by: Enguerrand de Ribaucourt <enguerrand.de-ribaucourt@savoirfairelinux.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some npm packages do not copy the LICENSE or COPY file into their
git repository. They'll instead simply use SPDX identifiers in their
package.json. A fallback for those repositories attempted to match
the README file to a license file instead, which had a very low
probability of success.
This commit replaces this fallback with parsing the package.json and
looking for the license in COMMON_LICENSE_DIR. If the license is not
found, "Unknown" will still be produced.
This also generates "Unknown" for packages which had no README file,
which could silently not appear in the generated recipe. The user was
more likely to miss them.
Co-authored-by: Tanguy Raufflet <tanguy.raufflet@savoirfairelinux.com>
(From OE-Core rev: 445604cfc4a5813ea635f18053cd1f673bf0b830)
Signed-off-by: Tanguy Raufflet <tanguy.raufflet@savoirfairelinux.com>
Signed-off-by: Enguerrand de Ribaucourt <enguerrand.de-ribaucourt@savoirfairelinux.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Our current emulation mandates that the package.json contains a version
field. Some packages may not provide it when they are not published to
the registry. The actual `npm pack` would allow such packages, so
should we.
This patch adds a default value to allow building such packages.
This applies for instance to this package which doesn't declare a
version:
- 23701a555a/package.json (L2)
v3:
- Split bitbake npmsw.py modification in another commit
Co-authored-by: Tanguy Raufflet <tanguy.raufflet@savoirfairelinux.com>
(From OE-Core rev: 470c4c027c2b8bbecf23aa63650a22a312de9aa6)
Signed-off-by: Tanguy Raufflet <tanguy.raufflet@savoirfairelinux.com>
Signed-off-by: Enguerrand de Ribaucourt <enguerrand.de-ribaucourt@savoirfairelinux.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We fetch npm dependencies from the npm-shrinkwrap.json file. They can
point to a package on the NPM registry with a version field, or to a
git/http/file URL with the resolved field. Such packages are allowed not
to declare a registry version field because they may not have been
published to the NPM registry. The previous implementation refuses to
fetch such packages and throws an error.
The resolved field contains the exact source, including the revision,
wich we can use to pass as SRC_URI to the git/http/file fetcher. The
integrity field is also mandatory for HTTP tarballs which will ensure
reproducibility. So even if the version field is not present, we are
still fetching a precise revision of the package.
Another commit published along this stack is also required in the npm
class to support these packages.
v5:
- improve commit message
v3:
- Split bitbake npmsw.py modification in another commit
Co-authored-by: Tanguy Raufflet <tanguy.raufflet@savoirfairelinux.com>
(Bitbake rev: 209982b5a3efc8081e65b4326bf9b64eef7f0ba0)
Signed-off-by: Tanguy Raufflet <tanguy.raufflet@savoirfairelinux.com>
Signed-off-by: Enguerrand de Ribaucourt <enguerrand.de-ribaucourt@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The NPM package.json documentation[1] states that git URLs may contain
a commit-ish suffix to specify a specific revision. When running
`npm install`, this revision will be looked for on any branch of the
repository.
The bitbake implementation however translates the URL stored in
package.json into a git URL to be fetch by the bitbake git fetcher. The
bitbake fetcher git.py, enforces the branch to be master by default. If
the revision specified in the package.json is not on the master branch,
the fetch will fail while the package.json is valid.
To fix this, append the ";nobranch=1" suffix to the revision in the git
URL to be fetched. This will make the bitbake git fetcher ignore the
branch and respect the behavior of `npm install``.
This can be tested with the following command:
$ devtool add --npm-dev https://github.com/seapath/cockpit-cluster-dashboard.git -B version
Which points to a project which has a package.json with a git URL:
```json
"devDependencies": {
"cockpit-repo": "git+https://github.com/cockpit-project/cockpit.git#d34cabacb8e5e1e028c7eea3d6e3b606d862b8ac"
}
```
In this repo, the specified revision is on the "main" branch, which
would fail without this fix.
[1] https://docs.npmjs.com/cli/v10/configuring-npm/package-json#git-urls-as-dependencies
Co-authored-by: Tanguy Raufflet <tanguy.raufflet@savoirfairelinux.com>
(Bitbake rev: 37a35adf7882f231c13643dbf9168497c6a242a1)
Signed-off-by: Tanguy Raufflet <tanguy.raufflet@savoirfairelinux.com>
Signed-off-by: Enguerrand de Ribaucourt <enguerrand.de-ribaucourt@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When tasks fail, it's very frustrating to have to scroll up to find the
log path(s). Many of us have the muscle memory to navigate to the 'temp'
directories under tmp/work/, but new users do not.
This change enhances the final summary to include log paths (reported
via bb.build.TaskFailed events). Here's an example:
NOTE: Tasks Summary: Attempted 856 tasks of which 853 didn't need to be rerun and 3 failed.
Summary: 3 tasks failed:
virtual:native:/home/chris/repos/poky/meta/recipes-core/ncurses/ncurses_6.5.bb:do_fetch
log: /home/chris/repos/poky/build/tmp/work/x86_64-linux/ncurses-native/6.5/temp/log.do_fetch.1253462
/home/chris/repos/poky/meta/recipes-core/ncurses/ncurses_6.5.bb:do_fetch
log: /home/chris/repos/poky/build/tmp/work/core2-64-poky-linux/ncurses/6.5/temp/log.do_fetch.1253466
virtual:nativesdk:/home/chris/repos/poky/meta/recipes-core/ncurses/ncurses_6.5.bb:do_fetch
log: /home/chris/repos/poky/build/tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-ncurses/6.5/temp/log.do_fetch.1253467
Summary: There were 3 WARNING messages.
Summary: There were 6 ERROR messages, returning a non-zero exit code.
Each log is rendered as a clickable hyperlink in the terminal. See
https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda
(Bitbake rev: 2852a478ab03a482989c3a7e247860ab4f0e9f3e)
Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Upgrade util-linux from 2.40.1 to 2.40.2.
0001-Revert-autotools-make-pam-install-path-configurable.patch
is added to solve a problem of lastlog2 pam module not installed
in the expected location.
(From OE-Core rev: 2303f28de507ceb88a012647f70b74e0fad6ec4b)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
While do_compile and do_install already have the scons_ prefix,
do_configure did not. This was most likely an oversight from when it
was converted from do_configure[noexec] = "1".
(From OE-Core rev: c2213e0393fc589ac0ee1dd6e67ad624eb0a2706)
Signed-off-by: Ola x Nilsson <olani@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows automated updates of the recipe together with mesa-gl
variant, and avoids having to check that update submissions
do include mesa-gl file rename.
(From OE-Core rev: f5cfb3e23603cefb2f3f6bfe776afaedefd10808)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is the first post-backdoor release.
These are the release notes:
https://github.com/tukaani-project/xz/releases/
There are also backdoor notes:
https://tukaani.org/xz-backdoor/
"I plan to write an article how the backdoor got into the releases and
what can be learned from this." - that'd be most welcome, as it would
be first hand information that sets the record straight.
And there's a commit by commit review of Jia Tan's contributions:
https://tukaani.org/xz-backdoor/review.html
Add an option for landlock sandbox (off by default as it clashes with running under pseudo).
License-Update: public domain bits were relicensed under 0BSD license
(From OE-Core rev: 6446d82a533da091ec2acc613b4cf06951d78ff3)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
License-Update: copyright years
(From OE-Core rev: 4e96b886e0e84ecb5f19e73dea444883257da25f)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
License-Update: copyright years
(From OE-Core rev: 807babe5392d3db02b237b17ad5ae773265c26de)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a patch to disable a failing test that is proving difficult
to investigate.
(From OE-Core rev: 0dc2dfcacaa99bdb306215d2a5e135038f72d895)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
history.3 manpage is no longer installed.
(From OE-Core rev: 93a2a1bc48562645ede4ec61f5fd2eb05f54e648)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Disable manpages, as they require scdoc, which is not currently
available in core (and adjust a related selftest).
Drop 0001-Use-portable-implementation-for-basename-API.patch
as upstream fixed the issue differently.
(From OE-Core rev: f868b75ab22cd528d9add744042f13d475715ef4)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When xwayland PACKAGECONFIG option is set, xwayland is enabled in
weston.ini. However, if the xwayland module isn't installed, weston will
refuse to start with the following error message:
Failed to load module: /usr/lib/libweston-13/xwayland.so: cannot open shared object file: No such file or directory
Therefore, whenever the xwayland PACKAGECONFIG is set, weston-init
should depend on weston-xwayland to bring this module in.
Fixes: fdbe559c66c9 ("weston.init: enabled xwayland")
(From OE-Core rev: fa2314125318634108452af4e40c9eeee260767c)
Signed-off-by: Quentin Schulz <quentin.schulz@cherry.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As of mesa version 24.1.0, the etnaviv gallium driver requires the Python
pycparser module when building.
Without it, build fails with something like:
../mesa-24.1.4/src/etnaviv/hwdb/meson.build:17:2: ERROR: Problem encountered: Python (3.x) pycparser module >= 2.20 required to build mesa.
(From OE-Core rev: bfc6ccd171b1cf5544ef79839fbc94e2c682bfda)
Signed-off-by: Esben Haabendal <esben@geanix.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Changelog:
aea9f42 ptest_list_remove: Fix pointer adjustment of prev and next
(From OE-Core rev: f70ec9bcd379b5fc4c85d7479d42789c2e22f4a9)
Signed-off-by: Jörg Sommer <joerg.sommer@navimatix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The 2.39 version of util-linux took new file descriptors based mount
kernel API into use. In relation to this change, the upstream release
notes in
https://github.com/util-linux/util-linux/blob/v2.39/Documentation/releases/v2.39-ReleaseNotes#L14-L21
mention that
This change is very aggressive to libmount code, but hopefully, it does not introduce regressions in traditional mount(8) behavior.
After observing following failure when booting a board using a bit
older 6.1 series kernel together with initramfs rootfs based boot flow
[FAILED] Failed to start Remount Root and Kernel File Systems.
See 'systemctl status systemd-remount-fs.service' for details.
closer inspection revealed:
demoboard ~ # systemctl status -l systemd-remount-fs.service
x systemd-remount-fs.service - Remount Root and Kernel File Systems
Loaded: loaded (/usr/lib/systemd/system/systemd-remount-fs.service; enabled-runtime; preset: disabled)
Active: failed (Result: exit-code) since Wed 2024-08-14 14:53:48 UTC; 1min 22s ago
Docs: man:systemd-remount-fs.service(8)
https://www.freedesktop.org/wiki/Software/systemd/APIFileSystems
Process: 76 ExecStart=/usr/lib/systemd/systemd-remount-fs (code=exited, status=1/FAILURE)
Main PID: 76 (code=exited, status=1/FAILURE)
Aug 14 14:53:48 demoboard systemd-remount-fs[76]: /usr/bin/mount for / exited with exit status 32.
Aug 14 14:53:48 demoboard systemd-remount-fs[81]: mount: /: mount point not mounted or bad option.
Aug 14 14:53:48 demoboard systemd-remount-fs[81]: dmesg(1) may have more information after failed mount system call.
Aug 14 14:53:48 demoboard systemd[1]: systemd-remount-fs.service: Main process exited, code=exited, status=1/FAILURE
Aug 14 14:53:48 demoboard systemd[1]: systemd-remount-fs.service: Failed with result 'exit-code'.
Aug 14 14:53:48 demoboard systemd[1]: Failed to start Remount Root and Kernel File Systems.
also consequentially, 'systemctl status' reported:
State: degraded
When issuing 'strace -ff mount -o remount /' the failure occurred at
mount_setattr(3, "", AT_EMPTY_PATH, {attr_set=MOUNT_ATTR_RDONLY|MOUNT_ATTR_NOATIME|MOUNT_ATTR_NODIRATIME, attr_clr=MOUNT_ATTR_NOSUID|MOUNT_ATTR_NODEV|MOUNT_ATTR_NOEXEC|MOUNT_ATTR_NOATIME|MOUNT_ATTR_STRICTATIME|MOUNT_ATTR_NOSYMFOLLOW|0x40, propagation=0 /* MS_??? */, userns_fd=0}, 32) = -1 EINVAL (Invalid argument)
After further investigation, The issue was pinpointed to lack of Linux
kernel commit
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=95de4ad173ca0e61034f3145d66917970961c210
("fs: relax mount_setattr() permission checks") in the kernel version
that was being used. Above mitigation was discussed in email related to
then-rejected CVE-2024-26821:
https://lore.kernel.org/linux-cve-announce/2024051606-imaging-entrench-b327@gregkh/T/
After testing with qemuarm64 machine different linux-yocto versions,
it was observed that the issue impacts following versions of currently
supported LTS kernels:
- 6.6.17 (fixed since 6.6.18 i.e. mount_setattr() returns 0)
- 6.1.78 (fixed since 6.1.79 i.e. mount_setattr() returns 0)
- 5.15.164 which is currently the newest of 5.15.y series (i.e. no
known working version)
Taking the above findings into consideration, add a new PACKAGECONFIG
option removing which enables users to opt-out from using the feature
which can cause issues with a bit older kernels. The option is enabled
only for class-target here, since it otherwise causes following error
during util-linux-native's do_configure task on Debian 11 build host
(mountfd_api requirement fails):
| configure: error: libmount_mountfd_support selected, but required mount FDs based API not available
Versions 5.10.223, 5.4.279 and 4.10.317 were also tested with qemuarm64
but the issue was not reproduced with those versions - using strace
showed that the mount_setattr call associated with the new mount API
problem was not issued with these LTS kernel versions, which seemed to
be confirmed also by following libmount debug message in these cases:
415: libmount: HOOK: [0x7fa115e818]: failed to init new API
Note: In addition to the aforementioned, this change was tested also
briefly using the current latest kernel versions 6.1.104, 6.6.45 and
6.10.3 that using the old mount API with newest kernels did not
introduce any observable regression to the boot flow.
(From OE-Core rev: dc086d9a8613143607af3583c72ed892e20b4d66)
Signed-off-by: Niko Mauno <niko.mauno@vaisala.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Under high load, the ssh test is hitting the amount of retries.
Increase it to 20 to avoid this issue. This would increase the maximum
failure time from 50 seconds (5 * 10) to 100 seconds.
(From OE-Core rev: c796438eec5dd6b4671b798f85506bc89ff402ab)
Signed-off-by: Jon Mason <jdmason@kudzu.us>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
