Commit Graph

79 Commits

Author SHA1 Message Date
Richard Purdie
b3573d38ef package: Move emit_pkgdata to packagedata.py
Move one of the PACKAGEFUNCS from the package bbclass to packagedata
library code for parsing efficiency.

(From OE-Core rev: ceba33bf2897f7dd5b1ffe6b742c47bf616243c8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-01-05 11:52:50 +00:00
Richard Purdie
a099ed2125 package: Move pkgdata handling functions to oe.packagedata
To avoid reparsing the bbclass code all the time, move the functions
to the packagedata python function library code which is more efficient.

(From OE-Core rev: f520a3039540b1183b1b2bdaaf8b9195995c0187)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-01-05 11:52:50 +00:00
Richard Purdie
93be2cdf49 package: Move get_conffiles/files_from_filevars functions to lib
To avoid reparsing the bbclass code all the time, move the functions
to the python function library code which is more efficient.

(From OE-Core rev: 424e65627c018b3119050f515b0c7cfb43be5573)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-01-05 11:52:50 +00:00
Richard Purdie
ed07d52b47 package: Move fixup_perms function to bb function library
To avoid reparsing the bbclass code all the time, move the function
to the python function library code which is more efficient.

(From OE-Core rev: 7c8c4cfd9355a9cee1144e0444e1b54402f1951c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-01-05 11:52:50 +00:00
Changqing Li
06763de40a base.bbclass: Fix way to check ccache path
The previous code had 2 issues:
1. make hosttools/ccache always link to host's ccache (/usr/bin/ccache)
even we have one buildtools
2. make hosttools/gcc etc, link to host's gcc event we have one
buildtools when keyword ccache in buildtools's path, eg:
/mnt/ccache/bin/buildtools

This patch is for fix above issues.

Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-12-23 12:05:52 +00:00
Marek Vasut
728b86575f package_rpm: Fix Linux 6.1.0 perf 1.0 version mistranslation
With Linux 6.1.0 and perf 1.0-r9, a build which includes perf-dev fails due
to perf-dev depending on perf 6.6.1.0-r9 . This is because translate_vers()
operates on perf-dev and mangles its version. The following scenario occurs:

  ver=6.1.0-r9
  pv=1.0
  pkgv=6.1.0
  reppv=6.1.0

With Linux 6.1.0, a corner case is hit where pv is a substring of ver, which
yields this corrupted version 6.6.1.0-r9 . Example in python3:

  >>> "6.1.0-r9".replace("1.0", "6.1.0")
  '6.6.1.0-r9'
  >>> "6.0.13-r9".replace("1.0", "6.0.13")
  '6.0.13-r9'

The fix is to only replace pv with reppv in case pv is at the beginning
of ver , instead of replacing all occurences.

(From OE-Core rev: bf2096a43d56c2d633a4b6e3db9e4390da4be6e2)

Signed-off-by: Marek Vasut <marex@denx.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-12-17 23:49:56 +00:00
Ola x Nilsson
c7b64446cb insane: Add QA check for 32 bit time and file offset functions
Check for known symbols that should have been redirected to 64bit
variants when -D_FILE_OFFSET_BITS=64 and -D_TIME_BITS=64 are set.

(From OE-Core rev: 8e2af04f24443fad2040bb32e6033d49e3120517)

Signed-off-by: Ola x Nilsson <olani@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-12-17 11:10:04 +00:00
Richard Purdie
9c36aef610 sanity: Update minimum python version to 3.8
Bitbake is moving to a minimum python version of 3.8, update OE-Core
to match to make things consistent.

(From OE-Core rev: cc05bc04156122fd6f918191a9cec7bc0392415e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-12-12 16:16:56 +00:00
Richard Purdie
4901c9d471 base: Switch to use addpylib directive and BB_GLOBAL_PYMODULES
Since bitbake now supports an official method to inject python modules,
switch to it.

Anyone using OE_EXTRA_IMPORTS will need to adjust their code accordingly,
probably switching to their own module namespace.

Also switch to using BB_GLOBAL_PYMODULES to list the global modules
to import.

(From OE-Core rev: 1f56155e91da2030ee0a5e93037c62e1349ba89f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-12-08 11:15:47 +00:00
Richard Purdie
9596082b50 base: Drop do_package base definition
I can't see how anyone would be using this very old package function definition
since package.bbclass is always inherited in modern OE. All it seems to do
is waste CPU cycles. Drop it and it's associated EXPORT_FUNCTIONS entry.

(From OE-Core rev: 56f7aa4cc0256aff96a1d720bd1931ea9a9bac8a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-11-27 23:54:51 +00:00
Enrico Scholz
276e3af23a sstate: show progress bar again
Transition to ThreadPoolExecutor (eb6a6820928472ef194b963b606454e731f9486f)
broke the

| Checking sstate mirror object availability: ...

progress bar because the removed 'thread_worker' was still referenced
in an asynchronous function.  As the result of the future is never
read, the resulting backtrace is silently discarded.

Replace the information given to 'ProcessProgress' by a counter.

(From OE-Core rev: eacd52216815e0c75e5f257b6db792afd8b1b6df)

Signed-off-by: Enrico Scholz <enrico.scholz@sigma-chemnitz.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-11-22 12:26:46 +00:00
Tim Orling
82a6e73384 mirrors.bbclass: update CPAN_MIRROR
Both of these redirect to https://cpan.metacpan.org/:
http://cpan.metacpan.org/
http://search.cpan.org/CPAN/

(From OE-Core rev: f1b74fc09f70d52d9ac629b04d81aa94fd97ff40)

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-11-22 12:26:45 +00:00
Richard Purdie
b955eea197 sanity: Drop data finalize call
This call was effectively like update_data and no longer did anything
in bitbake. Drop it as it is obsolete.

(From OE-Core rev: d3eb4531aae28a07cb7e52ed5fe1102445d2effd)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-11-14 16:19:43 +00:00
Jose Quaresma
7c1e70efcc sstate: Allow optimisation of do_deploy_archives task dependencies
do_deploy_archives tasks don't need their dependencies so we can optimistion
this as we do for some other tasks.

(From OE-Core rev: 3dd9f6e398844380d3765c54d35afe0d2ccf82e7)

Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-11-14 16:19:43 +00:00
Alexander Kanavin
48abeedd63 sanity.bbclass: do not check for presence of distutils
This will be removed in 3.12 and has been deprecated for a while.

If anything breaks because of its absence on the host, this would
expose the breakage so it can be fixed.

icu source specifically does not refer to distutils anywhere.

(From OE-Core rev: 8e3a5b0709384f2b455a82ac1e8e212686fe4456)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-11-11 13:43:40 +00:00
Nathan Rossi
0f21df06a7 package: Fix handling of minidebuginfo with newer binutils
Newer versions of binutils (2.38+) have changed how the
"--only-keep-debug" of objcopy behaves when stripping non-debug sections
from an ELF.

  https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=68f543154e92ab0f5d6c569e0fa143f5e8bd2d80

This change causes associated sections to be correctly marked as NOBITS
with the section contents removed from the output. The side effect is
that this causes issues with objcopy's ability to perform symbol and
relocation stripping (-S/--strip-all) on the debug split ELF, such that
with some object files (e.g. kernel modules) objcopy fails to strip
symbols/relocations with an error like the following:

  .../.debug/nls_cp950.ko[.rodata]: file truncated

Because of this it is now problematic to generate minidebuginfo for
these types of ELF objects. However it is not typically useful to inject
minidebuginfo into these types of ELFs, and other distributions (e.g.
Fedora, referring to find-debuginfo.sh of debugedit) only insert
minidebuginfo into executables and shared libraries.

This change causes the minidebuginfo injection to only apply to EXEC/DYN
type ELFs, which limits the injection to executables and shared
libraires.

Additionally this change fixes the parsing of the sections from the
"readelf -W -S" output which was not accounting for the section index
column having leading spaces for single digit index values e.g. "[ 1]".

(From OE-Core rev: 2084cfcb3d15db3e02637f1cd63ab9c997f38a65)

Signed-off-by: Nathan Rossi <nathan.rossi@digi.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-11-11 13:43:40 +00:00
Etienne Cordonnier
e1b27258d5 mirrors.bbclass: use shallow tarball for nativesdk-binutils
This is useful e.g. when using meta-clang, which introduces a dependency to nativesdk-binutils when
populating the SDK, and then a full tarball of binutils is fetched additionally to a shallow tarball.

(From OE-Core rev: 18624a89cb09bab78f9200a7c63406c75845e3e3)

Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-11-07 14:32:42 +00:00
Ross Burton
2a9bfc7dc8 insane: add codeload.github.com to src-uri-bad check
GitHub redirects /archive/ URLs to codeload.github.com, a dedicated
service for caching git archives:

$ wget -v https://github.com/unicode-org/icu/archive/refs/tags/release-72-1.zip
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/unicode-org/icu/zip/refs/tags/release-72-1 [following]

This is not the case for uploaded artifacts:

$ wget -v https://github.com/unicode-org/icu/releases/download/release-72-1/icu4c-72_1-data-bin-l.zip.asc
HTTP request sent, awaiting response... 302 Found
Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/49244766/... [following]

Check for codeload.github.com URLs in the src-uri-bad check in case
the SRC_URI contains this final URL, and not the public URI.

(From OE-Core rev: a3b4575259fa304c596ed227ed60769b5f72f0a8)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-11-07 14:32:42 +00:00
Ross Burton
05de18eae7 sanity: check for GNU tar specifically
We need the system tar to be GNU tar, as we reply on --xattrs.  Some
distributions may be using libarchive's tar binary, which is definitely
not as featureful, so check for this and abort early with a clear
message instead of later with mysterious errors.

(From OE-Core rev: 7dd2b1cd1bb10e67485dab8600c0787df6c2eee7)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-10-29 16:28:36 +01:00
Etienne Cordonnier
bb715127cb mirrors.bbclass: use shallow tarball for binutils-native
This is useful e.g. when using meta-clang, which introduces a dependency to binutils-native, and then a
full tarball of binutils is fetched additionally to a shallow tarball.

The original BB_GIT_SHALLOW lines were added because of https://www.mail-archive.com/yocto@lists.yoctoproject.org/msg08752.html

(From OE-Core rev: bd83b8b502ae935c75b59aaf71bbb531c9771dcc)

Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-10-28 09:44:52 +01:00
Adrian Freihofer
988a27974f own-mirrors: add crate
Support downloading crate files from a mirror at SOURCE_MIRROR_URL.

(From OE-Core rev: aebf4f183267a1e2f073078ade0ddc916ceed53f)

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-10-25 13:42:03 +01:00
Mark Hatle
5f60768030 insane.bbclass: Allow hashlib version that only accepts on parameter
Some versions of hashlib don't appear to implement the second FIPS
related argument.  Detect this and support both versions.

(From OE-Core rev: 2bbabed51e3aca138486d3feef640f5d3249be40)

Signed-off-by: Mark Hatle <mark.hatle@amd.com>
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-10-25 13:42:03 +01:00
Richard Purdie
fc32855512 sstate: Fix sstate file mirroring checksum issue
After a recent fix in bitbake, the checksums may be checked for local file
fetches at an earlier code point. The underlying recipe checksums won't
match sstate values so ensure these aren't set via SRC_URI flags when
reconfiguring the fetch for sstate.

(From OE-Core rev: 315cfebe77a46c868f307919d3540046a6cb8587)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-09-07 21:28:41 +01:00
Yang Xu
b915b5c282 insane.bbclass: Skip patches not in oe-core by full path
The full path of patch may contain '/meta/' but not in oe-core, skip
patches by checking it starts with oe-core full path or not.

(From OE-Core rev: d8a525afdfb5d371e76b09301c8b2741d23d1d10)

Signed-off-by: Yang Xu <yang.xu@mediatek.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-30 10:33:35 +01:00
Shubham Kulkarni
359ca09e7d sanity: add a comment to ensure CONNECTIVITY_CHECK_URIS is correct
In sanity.bbclass, in function check_connectivity :
Connectivity is tested by fetching URIs in variable CONNECTIVITY_CHECK_URIS.
If none is accessible, the status error is ambiguous.

It says to ensure the host's network is correctly configured but
never if the remote is available.

(From OE-Core rev: 9865176df6b6362a410b0fc3c115d942462dc338)

Signed-off-by: Shubham Kulkarni <skulkarni@mvista.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-21 22:51:43 +01:00
Pavel Zhukov
48124db7ca package_rpm: Do not replace square brackets in %files
This reverts commit f95adb749619e70920c6cc6cd01c6d02cd348fd8.
At some point RPM stopped treating "[]?" characters in quoted text as a glob
This causes rpmbuild error [1] in oe-core if package contains filenames
with such characters [Yocto #13746]. Reverting commit which replaces "[]" with "?"
fixes the issue.
It should be safe now becuase rpm tries to use filename directly if not
found by glob [2].

[1]
| error: File not found: /mnt/builds/yocto/build/build/tmp/work/core2-64-poky-linux/testrpm/0.0.1-r0/package/42 All-Time Classics (E) ?v1.1?.cht
|     File not found: /mnt/builds/yocto/build/build/tmp/work/core2-64-poky-linux/testrpm/0.0.1-r0/package/42 All-Time Classics (E) ?v1.1?.cht
|

[2] c16c70cbd6

(From OE-Core rev: 9916212b4810280108107d94cd2c0cd3e07ae6ed)

Signed-off-by: Pavel Zhukov <pavel.zhukov@huawei.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-21 22:51:42 +01:00
Richard Purdie
1d6c7af0e3 package: Switch debug source handling to use prefix map
Reproducible builds are no longer a configuration option but are required.
We also rely on the prefix mapping capability of the compilers now.

As such, rewrite the source locating code to use the prefix maps instead
of taking a guess about WORKDIR which isn't correct for kernels, gcc,
externalsrc and probably more.

Instead, iterate the maps to locate any matching source code, keeping
in mind that multiple maps may map to one target location.

(From OE-Core rev: cbd6144a9769d21371ae0fe04db2adc05f6eed02)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-17 19:57:32 +01:00
Alexander Kanavin
fd65379613 patchelf: replace a rejected patch with an equivalent uninative.bbclass tweak
This was the original reason to add the patch:
https://git.openembedded.org/openembedded-core/commit/?id=18efcbcb896239c64fedd009ce57f3f0c668cbc0

and this is the upstream discussion which suggests handling
read-only files explicitly outside of patchelf:
https://github.com/NixOS/patchelf/pull/89

(From OE-Core rev: 728775c87a7e35a03408675ce1c7a8cd061b2fda)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-14 08:13:32 +01:00
Richard Purdie
fd1517e2b5 classes: Update classes to match new bitbake class scope functionality
Move classes to classes-global or classes-recipe as appropriate to take
advantage of new bitbake functionality to check class scope/usage.

(From OE-Core rev: f5c128008365e141082c129417eb72d2751e8045)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-12 15:27:17 +01:00