Commit Graph

1118 Commits

Author SHA1 Message Date
BELOUARGA Mohamed
983548705a meta: lib: oe: npm_registry: Add more safe caracters
NPM registry cache should support caracaters like '(' and ')'
Explanation: NPM packages can contains these caracters like : @(._.)/execute

(From OE-Core rev: d3c1638077d4acbd61e7770c8e1d299ea33df638)

Signed-off-by: BELOUARGA Mohamed <m.belouarga@technologyandstrategy.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 6110d9e24e43e286781afd1b3634a4ad1a2050d0)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-07-12 05:11:38 -10:00
Pavel Zhukov
18469232f6 lib/terminal.py: Add urxvt terminal
This fixes failure [1] of menuconfig task in rxvt-unicode terminal in case if
xterm/Konsole/Gnome is not installed. Tested with rxvt-unicode-256color

[1]
WARNING: Terminal screen is supported but did not start
ERROR: No valid terminal found, unable to open devshell.
Tried the following commands:
	tmux split-window -c "{cwd}" "do_terminal"
	tmux new-window -c "{cwd}" -n "zephyr-helloworld Configuration" "do_terminal"
	xfce4-terminal -T "zephyr-helloworld Configuration" -e "do_terminal"
	terminology -T="zephyr-helloworld Configuration" -e do_terminal
	mate-terminal --disable-factory -t "zephyr-helloworld Configuration" -x do_terminal
	konsole --separate --workdir . -p tabtitle="zephyr-helloworld Configuration" -e do_terminal
	gnome-terminal -t "zephyr-helloworld Configuration" -- do_terminal
	xterm -T "zephyr-helloworld Configuration" -e do_terminal
	rxvt -T "zephyr-helloworld Configuration" -e do_terminal
	tmux new -c "{cwd}" -d -s devshell -n devshell "do_terminal"
	screen -D -m -t "zephyr-helloworld Configuration" -S devshell do_terminal
DEBUG: Python function do_menuconfig finished

(From OE-Core rev: 59f1a1b2326e2191aa882cfc73a851ea79672a6f)

Signed-off-by: Pavel Zhukov <pazhukov@suse.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 8d2fe4df8ae33e033caf4119a76715f085be1d15)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-06-23 04:16:41 -10:00
Andrew Jeffery
5e0dd7be76 Revert "ipk: Decode byte data to string in manifest handling"
cf9df9e8d89f ("ipk: Decode byte data to string in manifest handling")
did a bit of least-effort fix to a string vs byte sequence issue in the
manifest handling. The approach was chosen as it localised the fix,
rather than having to analyse further call sites.

However since then f2167ae80258 ("package_manager/ipk: do not pipe
stderr to stdout") was applied, reworking the output handling from the
subcommand. dummy_bytes() now returns a string, so stop trying to decode
it.

Fixes: f2167ae80258 ("package_manager/ipk: do not pipe stderr to stdout")
Cc: Curtis Meier <cmeier@us.ibm.com>
Cc: Pam Eggler <eggler@us.ibm.com>
(From OE-Core rev: b61739554780d70307d2b6b37d2b3b1c7df93c77)

(From OE-Core rev: 0e17a5a4f0e3301bf78f77bb5ca4aaf3e4dbc7af)

Signed-off-by: Andrew Jeffery <andrew@aj.id.au>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 07e5a6331be60d5e35d7336a6215a972ced6eb57)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-06-21 04:00:58 -10:00
Enrico Jörns
1ba75c259c package_manager/ipk: fix config path generation in _create_custom_config()
"sysconfdir" contains "/" by definition and thus using os.path.join()
leads to self.target_rootfs being always ignored (and thus attempting to
generate paths in host's /etc).

Use oe.path.join() instead which was made for this purpose.

(From OE-Core rev: 4766ba017b7562e42b33fde7f2e84c2a339e3f4c)

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 8414c504138f6de663f5130c6b4a6ede5605d88b)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-05-30 04:11:15 -10:00
Geoffrey GIRY
8064466b45 cve-check: Fix false negative version issue
NVD DB store version and update in the same value, separated by '_'.
The proposed patch check if the version from NVD DB contains a "_",
ie 9.2.0_p1 is convert to 9.2.0p1 before version comparison.

[YOCTO #14127]

Reviewed-by: Yoann CONGAL <yoann.congal@smile.fr>
(From OE-Core rev: f331c80df6c447d3073ebe3f00102c78ced242f3)

Signed-off-by: Geoffrey GIRY <geoffrey.giry@smile.fr>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 7d00f6ec578084a0a0e5caf36241d53036d996c4)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-04-11 11:31:52 +01:00
Mateusz Marciniec
88ddcabcca sstatesig: Improve output hash calculation
Symbolic links to the files are included during the output hash
calculation but symlinks to the directories are missed.
So if the new symlink to a directory was the only change made,
then the output hash won't change,
and the Hash Equivalence server may change unihash.
In the next run bitbake may use an older package from sstate-cache.

To fix this followlinks=True flag could be set for os.walk
but it can lead to infinite recursion if link points
to a parent directory of itself.
Also, all files from a directory to which symlink points
would be included in depsig file.
Therefore another solution was applied, I added code that will loop
through directories and process those that are symlinks.

(From OE-Core rev: 477a4e816494e256b309fd7e84b2c3796708e6e8)

Signed-off-by: Mateusz Marciniec <mateuszmar2@gmail.com>
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit ee729163f31f26b1462a47e1e53f7a0f9de9b464)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-09 13:19:03 +00:00
Martin Jansa
54d0147307 meta: remove True option to getVar and getVarFlag calls (again)
* True is default since 2016 and most layers were already updated
  not to pass this parameter where not necessary, e.g. oe-core was
  updated couple times, first in:
  https://git.openembedded.org/openembedded-core/commit/?id=7c552996597faaee2fbee185b250c0ee30ea3b5f

  Updated with the same regexp as later oe-core update:
  https://git.openembedded.org/openembedded-core/commit/?id=9f551d588693328e4d99d33be94f26684eafcaba

  with small modification to replace not only d.getVar, but also data.getVar as in e.g.:
  e.data.getVar('ERR_REPORT_USERNAME', True)

  and for getVarFlag:
  sed -e 's|\(d\.getVarFlag \?\)( \?\([^,()]*, \?[^,()]*\), \?True)|\1(\2)|g' \
      -i $(git grep -E 'getVarFlag ?\( ?([^,()]*), ?([^,()]*), ?True\)' \
          | cut -d':' -f1 \
          | sort -u)

(From OE-Core rev: de7bf6689a19dc614ce4b39c84ffd825bee1b962)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 26c74fd10614582e177437608908eb43688ab510)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
(cherry picked from commit 24a86d0c55ee89ae0dc77975e1d0ee02898d2289)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-15 21:46:56 +00:00
Benoît Mauduit
93b4705b76 lib/oe/reproducible: Use git log without gpg signature
Previously, if "showSignature" is present in user gitconfig, parsing
of the timestamp will fail.

Ideally we should replace this command with a git plumbing command.

(From OE-Core rev: 3db6db22770515fe52d847bdafd6ec9809ab50e0)

Signed-off-by: Benoît Mauduit <bmauduit@beneth.fr>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 3bd6f78f79b3d3e87d8db1e11f58d8021f929843)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-01-26 23:37:05 +00:00
Joshua Watt
99483cff5c classes/create-spdx: Add SPDX_PRETTY option
Adds an option to make the SPDX more human-readable (at the expense of a
larger files)

(From OE-Core rev: e680a7402edec2803b03c56590c9d08d07497c73)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 4799594b26f77ed259dc661bf077519b338390c8)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-01-06 17:33:23 +00:00
Enrico Jörns
f8880f548e sstatesig: emit more helpful error message when not finding sstate manifest
Since oe-core commit 64b89f3c8fc31842256c482a3039d90d3f12c1cc
("sstatesig.py: make it fatal error when sstate manifest isn't found")
errors like:

| Manifest [..]/tmp/sstate-control/manifest-x86_64_x86_64-nativesdk-dbus.populate_sysroot not found in imx8mm_dummy cortexa53-mx8mm cortexa53 armv8a-crc armv8a aarch64 allarch x86_64_x86_64-nativesdk (variant '')?

are fatal now and cannot be ignored but must be debugged.

Unfortunately, the currently emitted error message is a bit imprecise
with telling the reader what has actually gone wrong.

This commit:

* adds the word 'sstate' to the error message to clarify the scope we
  are dealing with ('sstate manifests', since there are other manifests,
  too)
* does not randomly print the last manifest file searched for as THE
  manifest file that could not be found
  Instead, we print the name of the task the sstate was searched for
* adds the word 'multilib' to variant to make clear which variant we are
  talking about
* adds a separate line noting the searched pkgarchs and adds explicitly
  mentions this word ('pkgarchs')
* prints a list of ALL manifest file locations attempted
* removes the '?' at the end of the message since such errors indeed
  leave the question of what is the cause but the error message itself
  is more like a statement.

The result for the exact same issue as noted above then looks as
follows:

| The sstate manifest for task 'dbus:populate_sysroot' (multilib variant '') could not be found.
| The pkgarchs considered were: imx8mm_dummy, cortexa53-mx8mm, cortexa53, armv8a-crc, armv8a, aarch64, allarch, x86_64_x86_64-nativesdk.
| But none of these manifests exists:
|     [..]/tmp/sstate-control/manifest-imx8mm_dummy-dbus.populate_sysroot
|     [..]/tmp/sstate-control/manifest-cortexa53-mx8mm-dbus.populate_sysroot
|     [..]/tmp/sstate-control/manifest-cortexa53-dbus.populate_sysroot
|     [..]/tmp/sstate-control/manifest-armv8a-crc-dbus.populate_sysroot
|     [..]/tmp/sstate-control/manifest-armv8a-dbus.populate_sysroot
|     [..]/tmp/sstate-control/manifest-aarch64-dbus.populate_sysroot
|     [..]/tmp/sstate-control/manifest-allarch-dbus.populate_sysroot
|     [..]/tmp/sstate-control/manifest-x86_64_x86_64-nativesdk-dbus.populate_sysroot

(From OE-Core rev: cb4ad96a5d71c0a7e7a24fe12f12c2fbe06ae119)

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 735ec126ec219c7cb89cb05b0e433201bb7f59eb)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-12-07 15:02:45 +00:00
Jose Quaresma
39f3e183c1 sstatesig: skip the rm_work task signature
We can skip the rm_work task signature to avoid running the task
when we remove some tasks from the dependencie chain.

The inject_rm_work handler on the rm_work bbclass triggers the
rm_work task running for any signature change in the dependencie
chain of the task do_build of each recipe.

i.e INHERIT:remove = "create-spdx" will trigger the do_rm_work
when we collect the sstate cache with INHERIT = "create-spdx"

(From OE-Core rev: ad52bb4643eb8063ff4db6bcb86b9a593ad6ce7d)

Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 292305700e39d0ebd64763f5032c39ace5005fad)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-12-01 19:35:05 +00:00
Vyacheslav Yurkov
75be860f34 overlayfs: Allow not used mount points
When machine configuration defines a mount point, which is not used in
any recipe, allow to fall through and only report a note in the logs.
This can be expected behavior, when a mount point is defined for several
machines, but not used in all of them

(From OE-Core rev: 89d83920dc7f80abb20fadde97b47aba9cd992cc)

Signed-off-by: Vyacheslav Yurkov <Vyacheslav.Yurkov@bruker.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
(cherry picked from commit a9c604b5e0d943b5b5f7c8bdd5be730c2abcf866)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
(cherry picked from commit c7c6b273656a3e2b8b959004b996e56d4086ce5e)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-11-09 17:42:08 +00:00
Ross Burton
36bc011714 oe/packagemanager/rpm: don't leak file objects
(From OE-Core rev: 406abef218a7e743f7892eff4c13fa0f889aa7f2)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 28706c27680745c9f8df27713ce63ef5d611138c)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-11-09 17:42:08 +00:00
Alejandro Hernandez Samaniego
26dbc1f641 rootfs.py: dont try to list installed packages for baremetal images
Theres not a rootfs for baremetal images, hence we should avoid
trying to list rootfs packages for them.

This fixes an issue where some classes (e.g. license_image) rely on
rootfs functionality when included for baremetal images even if its
nonexistent

(From OE-Core rev: e713e118b46aa0c8a5015b915a93718b7aa20a74)

Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandro@enedino.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 5a7e13f46083ce3b08aa762238c1e93b7626dda4)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-09-16 17:53:23 +01:00
Enrico Scholz
bf1987dbe5 lib:npm_registry: initial checkin
Helper module to:

- generate meta information from package.json content.  This data has
  a format as provided by https://registry.npmjs.org

- put this meta information and the corresponding tarball in the
  nodejs cache.  This uses an external, nodejs version specific helper
  script (oe-npm-cache) shipped in oe-meta

To avoid further nodejs version dependencies, future versions of this
module might omit the caching completely and serve meta information
and tarball by an http server.

(From OE-Core rev: 17132402031f4659db5cc1f84263278b82b27ffa)

Signed-off-by: Enrico Scholz <enrico.scholz@sigma-chemnitz.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 6cd5886ad05fee704e8a5892bd370c360c8c3b54)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-09-12 08:41:48 +01:00
Ernst Sjöstrand
6413ed11f8 cve-check: Don't use f-strings
Since we're keeping cve-check aligned between the active branches,
and dunfell is supported on Python 3.5, we can't use f-strings.

(From OE-Core rev: 057a6f826bd45ebf240df47f0baf7c8027f5711c)

Signed-off-by: Ernst Sjöstrand <ernstp@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 1821cf7464cbba521b55a9c128fe8812c0cc5eca)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-28 07:51:29 +01:00
Mihai Lindner
d8c792f12c create-spdx: Fix supplier field
The correct field name is "supplier" according to SPDX schema.
The "supplier" field translates to "PackageSupplier", but that's for
tag-value format.

(From OE-Core rev: 504b50aec662f177fea452e05e29af8b36ca69fc)

Signed-off-by: Mihai Lindner <mihai.lindner@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit ca8db0e0a2860ac1e3f537471fa71b43c3be0a58)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-23 15:22:52 +01:00
Shruthi Ravichandran
f6bc8dfadb package_manager/ipk: do not pipe stderr to stdout
Some opkg commands print an error during cleanup when the tmp_dir
does not exist and an attempt is made to delete it. The error messages
are harmless and the opkg commands eventually succeed.
When these commands are run and stderr is piped to stdout, the error
messages may clobber the stdout and cause unexpected results while
parsing the output of the command. Therefore, when parsing the output
of a command, do not pipe stderr to stdout. Instead, capture stderr
and stdout separately, and upon success, send stderr to bb.note().

(From OE-Core rev: fd5689696731fefa0d035fde86f27a0135dc31f1)

Signed-off-by: Shruthi Ravichandran <shruthi.ravichandran@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit f2167ae80258253eb47a5b148546b265320284cc)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-08 16:23:37 +01:00
Richard Purdie
a78211a182 base/reproducible: Change Source Date Epoch generation methods
In bc, we found that there are files newer than Changelog (e.g. scan.l)
which means after install runs, the timestamp is clamped to SDE which means
"bitbake bc -C compile" would then try and regenerate generated files and
complain flex was missing.

Rather than assuming Changelog/NEWS are magic for dates, drop that scan
method and scan all unpacked files. This shouldn't be that slow as they'd
be in the disk cache already after an unpack.

For bc, this changes SDE by about two minutes and avoids the problems
rebuilding.

Add a version comment to the task definition as changes in python library
code won't trigger a rebuild and we need one here.

(From OE-Core rev: 400ed3bc77232b9d67594fa012b1278a6c8be3c6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 32dda0ad91a9e7946351c897578b4c97ae142341)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-04 16:29:15 +01:00
Joshua Watt
ca3a1f17cc sstatesig: Include all dependencies in SPDX task signatures
SDPX generation involves looking through BB_TASKDEPDATA for
dependencies, then linking to the generated documents for those
dependencies. These document links use a checksum to validate the
document, which means that if a upstream document changes, all
downstream documents must be regenerated to get the new checksum,
otherwise the compendium of documents produced by the build will have
broken links; therefore all dependent task should be included in the
signature (even from "ABI safe" recipes).

(From OE-Core rev: 7549429fc93218dee33b216010b2c36a9f814091)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 5fe543b9ceec971cf0297ff0ae3b0ccc4703cece)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-01 16:27:30 +01:00
Paul Eggleton
ee60960da3 devtool: finish: handle patching when S points to subdir of a git repo
If devtool finish needs to create a patch and have it applied to the
sources for a recipe where S points to a subdirectory of the sources,
then the patch needs to be applied at the root of the repo i.e. we need
to add a patchdir= parameter to the SRC_URI entry.

(From OE-Core rev: 7cf53810b1dc3d14c4838a610b3d53170f552c19)

Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit ad3736d9ca14cac14a7da22c1cfdeda219665e6f)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-25 15:11:46 +01:00
Paul Eggleton
9e7a56bef7 patch: handle if S points to a subdirectory of a git repo
If PATCHTOOL = "git", SRC_URI fetches from a git repo and S points to
a subdirectory of the checked out sources, then we were erroneously
initialising the subdirectory as its own git repo. Check if the returned
top-level repo directory is a subdirectory of WORKDIR and do not
run initialise the source directory if that is the case.

(This was a regression introduced with OE-Core revision
6184b56a7a0fc6f5d19fdfb81e7453667f7da940, however we didn't have a test
that verified the behaviour.)

(From OE-Core rev: 577a69137eac6a44869d384b9027fbfdfea5740e)

Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 9cca53a2bcbf6809615ce5626c86c6ee481a7a76)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-25 15:11:46 +01:00
Martin Jansa
4aeda14352 rootfs.py: close kernel_abi_ver_file
* fixes:
  oe-core/meta/lib/oe/rootfs.py:331: ResourceWarning: unclosed file <_io.TextIOWrapper name='pkgdata/mach/kernel-depmod/kernel-abiversion' mode='r' encoding='UTF-8'>
  kernel_ver = open(kernel_abi_ver_file).read().strip(' \n')

(From OE-Core rev: 0f7a8359ba370c7f5d5153453ed699e9566f5b1d)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit f9dd8ee063c1132265248457fcd628e1e93727be)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-18 16:34:41 +01:00
Sean Anderson
6c51379206 rootfs.py: find .ko.zst kernel modules
With CONFIG_MODULE_COMPRESS_ZSTD enabled, kernel modules will have a
.ko.zst extension. This fixes depmod not being run.

Fixes: 1b696a45ddb ("rootfs.py: Add check for kernel modules before running depmod")
(From OE-Core rev: 6ade2bfdd9297896b9f251dd62f55e461fef1a3e)

Signed-off-by: Sean Anderson <sean.anderson@seco.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 425efac7110f0f42d70643e0a448e834d0f01a7a)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-11 10:06:13 +01:00
Martin Jansa
411fbde5ac patch.py: make sure that patches/series file exists before quilt pop
* Since quilt upgrade to 0.67 some recipes sometimes fail in do_patch with
  errors like:

  ERROR: Applying patch 'GPLv2.patch' on target directory '/OE/build/oe-core/tmp-glibc/work/qemux86_64-oe-linux/keymaps/1.0-r31'
  CmdError('quilt --quiltrc /OE/build/oe-core/tmp-glibc/work/qemux86_64-oe-linux/keymaps/1.0-r31/recipe-sysroot-native/etc/quiltrc push', 0, 'stdout:
  stderr: File series fully applied, ends at patch GPLv2.patch
  ')

* It affects only recipes with S = "${WORKDIR}", which wipe only
  ${S}/patches, because in other cases whole ${S} is wiped when
  do_unpack is re-executed.

* It was originally added in:
  https://git.openembedded.org/openembedded-core/commit/?id=5fe5e6a15f26f23f0c5b863fafad7a0d382a55e2

  since then it was extended to wipe whole ${S} when ${S} != ${WORKDIR} in:
  https://git.openembedded.org/openembedded-core/commit/?id=5fe5e6a15f26f23f0c5b863fafad7a0d382a55e2
  https://git.openembedded.org/openembedded-core/commit/?id=eccae514b71394ffaed8fc45dea7942152a334a1

  this is now causing issues to quilt-0.67 because it checks that
  ${S}/patches/series exists during 'quilt pop -a -f' which we call
  from QuiltTree.Clean to undo patches possibly already applied
  in ${S} in previous do_patch execution.

* There are couple recipes affected by this e.g. keymaps (.patch already
  removed in oe-core), makedevs (.patch removal sent to ML yesterday
  https://lists.openembedded.org/g/openembedded-core/message/166172),
  devmem2
  (https://lists.openembedded.org/g/openembedded-devel/message/97270), but
  there are other recipes with S = "${WORKDIR}" where you can trigger this
  e.g. by having a .patch file in DISTRO layer .bbappend (e.g. tzdata with
  webOS
  06e5298d9f/meta-webos/recipes-extended/tzdata/tzdata.bbappend (L10))

  This do_patch issue is caused by:
  https://git.savannah.nongnu.org/cgit/quilt.git/commit/?id=8b39a960afcf45cd4f5804ae62b6b0656bdb191d
  introduced in kirkstone with:
  https://git.openembedded.org/openembedded-core/commit/?h=kirkstone&id=fa71afcee9ab42198c619333b77a15bd2ae02b20

  The shortest sequence to reproduce this is just
  bitbake keymaps -c patch
  bitbake keymaps -c unpack -f
  bitbake keymaps -c patch
  with
  https://git.openembedded.org/openembedded-core/commit/?id=17d981005a0c0c97702ad88602b7181b69bcc9eb
  reverted.

  And the change in quilt behavior is causing QuiltTree.Clean (quilt pop -a -f) in:
  https://git.openembedded.org/openembedded-core/tree/meta/lib/oe/patch.py?id=17d981005a0c0c97702ad88602b7181b69bcc9eb#n601

  to silently fail with "No series file found" before undoing the
  patches in ${S} and then quilt push failing, because all the
  patches are _still_ applied in ${S}.

  Removing ".pc" doesn't help, because we really
  need quilt's help to undo the patches (in this case to delete COPYING
  file from WORKDIR before applying the .patch which tries to add it
  again), because do_unpack cannot just wipe S and start over (because S
  == WORKDIR) - nor selectively removing the files listed in SRC_URI,
  because COPYING file isn't listed there.

  Using skip_series_check in 'quilt pop' (partially reverting the change
  from upstream) does fix this as well and it's simple one line patch
  (just adding skip_series_check=1 in pop.in), but might be difficult
  to upstream, because it's this strange OE specific behavior that we
  remove 'patches' directory and then still need quilt pop to work.

(From OE-Core rev: 5323f0951923f201b0ad8f78aa3f42b8c3c97a17)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit c9d36882044b1c633d8611a77df54cd68c9bee25)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-11 10:06:13 +01:00
Marta Rybczynska
7f22fb48eb cve-check: move update_symlinks to a library
Move the function to a library, it could be useful in other places.

(From OE-Core rev: 66bf08f2df478a909a12756c650dcc2ae09a0a47)

Signed-off-by: Marta Rybczynska <marta.rybczynska@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit debd37abcdde8788761ebdb4a05bc61f7394cbb8)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-11 10:06:13 +01:00
Richard Purdie
983c257517 lib/sstatesig: Fix find_siginfo to match sstate filename generation
sstate filename generation was changed a while ago and taskname has to be
passed into core functions for the correct filename to be generated. Update
find_siginfo to match those changes and pass in taskname via SSTATE_CURRTASK.

Thanks to Gregory Lumen <gregorylumen@microsoft.com> for spotting.

[YOCTO #14774]

(From OE-Core rev: 3230134c0e5654afccf4f8831ab1101615502162)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 51d7310b6cf8ef9033d461868c07f686656070ba)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-12 16:44:05 +01:00
Peter Kjellerstedt
f8a450a39d terminal.py: Restore error output from Terminal
In bitbake commit 1ecc1d94 (process: Do not mix stderr with stdout),
bb.process.Popen() was changed to no longer combine stdout and stderr by
default. However, the Terminal class was not updated to reflect this and
subsequently only output stdout in case of failures.

(From OE-Core rev: f8f8e2e159a5ac03f619e6d0882011445e6a2545)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 116d0bb07ba044cf8847bf3d5c3996ad7e58b7ae)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-04 13:07:33 +01:00
Ross Burton
498bbee789 cve_check: skip remote patches that haven't been fetched when searching for CVE tags
If a remote patch is compressed we need to have run the unpack task for
the file to exist locally.  Currently cve_check only depends on fetch so
instead of erroring out, emit a warning that this file won't be scanned
for CVE references.

Typically, remote compressed patches won't contain our custom tags, so
this is unlikely to be an issue.

(From OE-Core rev: b52a508bd7693c40e6416db9c9076a8789499501)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit cefc8741438c91f74264da6b59dece2e31f9e5a5)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-04 13:07:31 +01:00
Ferry Toth
0b4231b597 package_manager: sign DEB package feeds
Implement debian package repository signature.
For each Release file created in repository subdirectory, a signature
Release.gpg is created.

Signature is performed using gpg backend when the following variables
are set in local.conf:
PACKAGE_CLASSES += "sign_package_feed"
PACKAGE_FEED_GPG_NAME = "<Id of GPG key>"
PACKAGE_FEED_GPG_PASSPHRASE_FILE="<path to password file>"

(From OE-Core rev: fcc3cee276999efe6402959eb295e7a0e1e96f96)

Signed-off-by: Xavier Berger <xavier.berger@bio-logic.net>
Signed-off-by: Ferry Toth <ftoth@exalondelft.nl>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-04-05 22:25:13 +01:00
Xavier Berger
bd8f1f7787 gpg-sign: Add parameters to gpg signature function
output_suffix: If defined, add output_suffix as file name extension.
use_sha256: If True, use sha256 for gpg as digest algorithm

(From OE-Core rev: cfcaa54dc73925df448099fb60f75b18350b2a3b)

Signed-off-by: Xavier Berger <xavier.berger@bio-logic.net>
Signed-off-by: Ferry Toth <ftoth@exalondelft.nl>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-04-05 22:25:13 +01:00
Marta Rybczynska
777f1d42b6 cve-check: add json format
Add an option to output the CVE check in a JSON-based format.
This format is easier to parse in software than the original
text-based one and allows post-processing by other tools.

Output formats are now handed by CVE_CHECK_FORMAT_TEXT and
CVE_CHECK_FORMAT_JSON. Both of them are enabled by default.

The JSON output format gets generated in a similar way to the
text format with the exception of the manifest: appending to
JSON arrays requires parsing the file. Because of that we
first write JSON fragments and then assemble them in one pass
at the end.

(From OE-Core rev: df567de36ae5964bee433ebb97e8bf702034994a)

Signed-off-by: Marta Rybczynska <marta.rybczynska@huawei.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-30 13:07:41 +01:00
Ming Liu
bbc60671ad image.bbclass: make sure do_rootfs run from a clean workspace
Add ${IMAGE_ROOTFS} and ${IMGDEPLOYDIR} to do_rootfs[dirs] and
do_rootfs[cleandirs], this ensures do_rootfs run from a clean
workspace, with this change, we can now remove two bb.utils.mkdirhier
lines from meta/lib/oe/rootfs.py.

(From OE-Core rev: cb8b6f7eee4e059bb311330c57068e11bc477366)

Signed-off-by: Ming Liu <liu.ming50@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-13 12:23:48 +00:00
Saul Wold
d9e500f83d meta/scripts: Improve internal variable naming
Update internal variable names to improve the terms used.

(From OE-Core rev: f408068e5d7998ae165f3002e51bc54b380b8099)

Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-10 08:00:28 +00:00
Andrew Jeffery
ef518a327d ipk: Decode byte data to string in manifest handling
```
File: '/home/andrew/src/openbmc/openbmc/meta/lib/oe/package_manager/ipk/manifest.py', lineno: 69, function: create_full
     0065:        output = pm.dummy_install(pkgs_to_install)
     0066:
     0067:        with open(self.full_manifest, 'w+') as manifest:
     0068:            pkg_re = re.compile('^Installing ([^ ]+) [^ ].*')
 *** 0069:            for line in set(output.split('\n')):
     0070:                m = pkg_re.match(line)
     0071:                if m:
     0072:                    manifest.write(m.group(1) + '\n')
     0073:
Exception: TypeError: a bytes-like object is required, not 'str'
```

Change-Id: Ifefb13bfa22c766d20ab9f73f7abe5163b3df86f
(From OE-Core rev: cf9df9e8d89fee9cea4785c94a1e3004a5f3469d)

Signed-off-by: Andrew Jeffery <andrew@aj.id.au>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-04 17:14:15 +00:00
Andrew Jeffery
1366335749 ipk: Import re in manifest module
```
File: '/home/andrew/src/openbmc/openbmc/meta/lib/oe/package_manager/ipk/manifest.py', lineno: 67, function: create_full
     0063:
     0064:        output = pm.dummy_install(pkgs_to_install)
     0065:
     0066:        with open(self.full_manifest, 'w+') as manifest:
 *** 0067:            pkg_re = re.compile('^Installing ([^ ]+) [^ ].*')
     0068:            for line in set(output.split('\n')):
     0069:                m = pkg_re.match(line)
     0070:                if m:
     0071:                    manifest.write(m.group(1) + '\n')
Exception: NameError: name 're' is not defined
```

Change-Id: I769a2ab5e57c7b60598ea0390b576d707356db9d
(From OE-Core rev: e240a8ede690e56bcf53a97b3be5592e6e4a5a15)

Signed-off-by: Andrew Jeffery <andrew@aj.id.au>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-04 17:14:15 +00:00
Daniel Wagenknecht
4c298459ab copy_buildsystem: allow more layer paths
Layers could be located anywhere. The eSDK should work with them even if
they are not located in TOPDIR or in the same parent directory as
COREBASE.

For layers located in the same parent directory as COREBASE this preserves
the intent from the previous
  copy_buildsystem: include layer tree during build structure creation
commit.

Related OE-Core rev: 5a59a6997f41e606d088e3e86812de56f72f543b

(From OE-Core rev: 16d330d42e03085769eddb1b60ba1df7228baf36)

Signed-off-by: Daniel Wagenknecht <dwagenknecht@emlix.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-04 17:14:14 +00:00
Andrew Jeffery
102e4c029e rust: Introduce arch_to_rust_arch()
On modern Power systems `uname -m` yields 'ppc64le' while the toolchain
knows the architecture as 'powerpc64le'. Provide a mapping from one to
the other to integrate with the existing architecture configuration
flags.

arch_to_rust_arch() only exists to map the OE *_ARCH variables before
any further processing, unlike arch_to_rust_target_arch() which is
specific to the internal triple handling of rust.

On Linux ppc64le systems the changes give the following config:

```
$ cat ./tmp/work/ppc64le-linux/rust-native/1.58.0-r0/targets/ppc64le-linux.json
{
    "llvm-target": "powerpc64le-unknown-linux-gnu",
    "data-layout": "e-m:e-i64:64-n32:64-v256:256:256-v512:512:512",
    "max-atomic-width": 64,
    "target-pointer-width": "64",
    "target-c-int-width": "64",
    "target-endian": "little",
    "arch": "powerpc64",
    "os": "linux",
    "env": "gnu",
    "vendor": "unknown",
    "target-family": "unix",
    "linker": "gcc",
    "cpu": "generic",
    "dynamic-linking": true,
    "executables": true,
    "linker-is-gnu": true,
    "linker-flavor": "gcc",
    "has-rpath": true,
    "has-elf-tls": true,
    "position-independent-executables": true,
    "panic-strategy": "unwind"
}
```

Change-Id: Ief0c01189185d7d4da31d307270bec4e1de674ca
(From OE-Core rev: 9ab61e3cfef0157393cb870d606c2f362e190889)

Signed-off-by: Andrew Jeffery <andrew@aj.id.au>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-02 18:44:17 +00:00
Richard Purdie
321cf8962e license/insane: Show warning for obsolete license usage
We want to use SPDX identifiers in LICENSE variables. There is now a
conversion script to make most of the translations. Add a list of
strings which have been replaced so we can show warnings to users
if they're still used anywhere.

Add checks to the package as insane check. This is currently a warning
by default but can be turned off or made an error as per the other standard
checks.

(From OE-Core rev: 9379f80f484f94686a4d494e9e237fadfb72a938)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-02 18:43:25 +00:00
Saul Wold
d6449581c9 base/license: Rework INCOMPATIBLE_LICENSE variable handling
This re-writes the INCOMPATIBLE_LICENSE checking code to replace
the WHITELIST_<lic> with
INCOMPATIBLE_LICENSE_EXCEPTIONS = '<pkg>:<lic> <pkg>:<lic> ...'

This initial change leaves most of the code structure in place,
but the code in base.bbclass needs to be re-written to make
the check more consistent around packages (PKGS) and not recipe
names (PN). This also is taking into account the changes for SPDX
licenses.

The aim is to provide a mode consistent variable where the variable
name is known and can easily be queried.

(From OE-Core rev: 0d19c45ba6cf43518f380ca5afe9753a2eda0691)

Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-02 18:43:25 +00:00
Alexander Kanavin
c486c6609a sstate: do not add TARGET_ARCH to pkgarch for cross recipes.
This is redundant (target arch is already in PN), and breaks
compiling a cross-canadian toolchain, as that needs populating the
sysroot with two different native-hosted toolchains built from
cross recipes. Inserting TARGET_ARCH allows only one or the other.

(From OE-Core rev: 33fc1792cd782feb8dbb4285e3006bb588f7978f)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-02 18:43:24 +00:00
Peter Kjellerstedt
9da87f1119 license.py: Correct a comment
(From OE-Core rev: 1f7a34c6d246c6f42ab823ffd0bd0306705ad88d)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-27 12:34:40 +00:00
Richard Purdie
2a06ab3eb8 licenses: Fix logic error introduced in rename
The previous commit introduced a small logic error. Fix the renaming issue.

(From OE-Core rev: b01e10b27d23ea1b4ac58376c2423505a70832d6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-25 12:45:41 +00:00
Saul Wold
e629653463 license.py: rename variables
Update the comment to reflect new variable names

(From OE-Core rev: 7fbab1aefc127f0e1834f51a8a793b0d7e7b4f07)

Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-25 12:41:24 +00:00
Pavel Zhukov
da344db43c patch.py: Prevent git repo reinitialization
There were few bugs in the _isInitialized() function which might trigger
git repo to be reinitialized and patches failing to apply.

(From OE-Core rev: 80500ecda4c1bc8812e6e078b6b0db5ec46624de)

Signed-off-by: Pavel Zhukov <pavel.zhukov@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-25 12:41:23 +00:00
Andres Beltran
e43a9d15ea create-spdx: add support for SDKs
Currently, SPDX SBOMs are only created for images. Add support for
SDKs.

(From OE-Core rev: c3acbb936a339636153903daf127eec9f36de79b)

Signed-off-by: Andres Beltran <abeltran@linux.microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-25 12:41:23 +00:00
Richard Purdie
eb72e3c381 recipeutils: Add missing get_srcrev() call
Updates within bitbake now require this call to be triggered somewhere with
floating git source revs. Add the missing call. Issue can be reproduced with:

devtool check-upgrade-status dbus-wait

(From OE-Core rev: d7088b2b5012f9e386914a6837685f5869379cc5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-18 11:37:34 +00:00
Daiane Angolini
76b3d141c0 classes/lib/useradd: The option -P is deprecated
This is also covered in documentation since:
https://git.yoctoproject.org/yocto-docs/commit/?id=3f3e5574ac9801ad92940168b61b532e0bd53a80

[YOCTO 14605]

(From OE-Core rev: 3f9153986e4e6f667b4bbe97613ec0b279665a97)

Signed-off-by: Daiane Angolini <daiane.angolini@foundries.io>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-12 17:05:35 +00:00
Alexander Kanavin
18f165c9e8 insane.bbclass: use multiprocessing for collecting 'objdump -p' output
This was prompted by ltp's unreasonably long package_qa times; it has
a massive amount of executables and insane runs objdump for all of
them, serially.

This reduces the time from 4 minutes to 1m20s on my machine.

(From OE-Core rev: fac984b99fdb46949879516cb87153860f402c75)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-08 14:20:18 +00:00
Joshua Watt
d366a1a223 spdx: Add set helper for list properties
This adds the set helper for list properties to allow list members to be
replaced with a new list. This is necessary as it is (currently) the
only way to replace the default value for a list member.

(From OE-Core rev: 8280b857c510646cd10167dd2ea9a0c2d5f1e800)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-07 10:08:59 +00:00