We want to use SPDX identifiers in LICENSE variables. There is now a
conversion script to make most of the translations. Add a list of
strings which have been replaced so we can show warnings to users
if they're still used anywhere.
Add checks to the package as insane check. This is currently a warning
by default but can be turned off or made an error as per the other standard
checks.
(From OE-Core rev: 9379f80f484f94686a4d494e9e237fadfb72a938)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This re-writes the INCOMPATIBLE_LICENSE checking code to replace
the WHITELIST_<lic> with
INCOMPATIBLE_LICENSE_EXCEPTIONS = '<pkg>:<lic> <pkg>:<lic> ...'
This initial change leaves most of the code structure in place,
but the code in base.bbclass needs to be re-written to make
the check more consistent around packages (PKGS) and not recipe
names (PN). This also is taking into account the changes for SPDX
licenses.
The aim is to provide a mode consistent variable where the variable
name is known and can easily be queried.
(From OE-Core rev: 0d19c45ba6cf43518f380ca5afe9753a2eda0691)
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
- replace python3 prefix when guessing the wheel name
as there are still plenty of recipes out there that do use
python3 prefixes
- remove all previously generated wheels matching the glob
to avoid installing any outdated blob via cleandirs
in setuptools3 class.
Unfortunetaly proposed dist-dir or bdist-dir are not
respected by setuptools, likely due because they
are overridable by the setup script
- don't use PV in glob, as PV doesn't necessarily align with the
version used inside of the setuptools configuration.
this will avoid having the user set PYPA_WHEEL in a lot
of recipes
- respect SETUPTOOLS_SETUP_PATH in PIP_INSTALL_DIST_PATH
and use B as a fallback only (in case this class is inherited
without setuptools3 class being there as well).
recipes like python3-smbus run in a subfolder of the
workspace and were failing in before this adjustment
(From OE-Core rev: 6f2d85a7b7d94101f2ce67115166fa86c185650f)
Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The class should export the tasks so they can be extended or overriden
elsewhere (like what setuptools3.bbclass has).
(From OE-Core rev: 14c2dcbebccf072225a089518ffeb010401f479c)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is redundant (target arch is already in PN), and breaks
compiling a cross-canadian toolchain, as that needs populating the
sysroot with two different native-hosted toolchains built from
cross recipes. Inserting TARGET_ARCH allows only one or the other.
(From OE-Core rev: 33fc1792cd782feb8dbb4285e3006bb588f7978f)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The icecc.bbclass needs network access to work properly.
Currently I build with icecc inside a container with network isolation and
my icecc daemon runs outside of the build container in my host.
The only thing I need to do for using the icecc inside my build container is
mounting the unix socket /var/run/icecc/iceccd.socket inside the container.
I think we need something like this mount functionality to have access to
some sockets connections inside the tasks that runs on the new namespace
created with unshare system call.
This patch is not a the real solution for the problem and is more like
an hack so we can use the icecc.bbclass again.
(From OE-Core rev: 25ea276a13a6ac2342c2b0945c8fafe878d56095)
Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
hashlib.md5() is not permitted on a FIPS enabled host system. This is due
to md5 not being an approved hash algorithm.
Instead use:
hashlib.new('MD5', usedforsecurity=False)
This is allowed, as it's clear the hash is used for a non-security purpose.
Using an md5 to identify when a license has changed is permitted, as we're
not using it for file integrity.
(From OE-Core rev: 7f7eb82ce47ede6ec6f50cbcb56cbfbe10e8d20c)
Signed-off-by: Mark Hatle <mark.hatle@xilinx.com>
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In commit ceda3238 (meta/meta-selftest/meta-skeleton: Update LICENSE
variable to use SPDX license identifiers) all LICENSE variables were
updated to only use SPDX license identifiers.
This does the same for comments and other variables where it is
appropriate to use the official SPDX license identifiers. There are
still references to, e.g., "GPLv3", but they are then typically in
descriptive text where they refer to the license in a generic sense.
(From OE-Core rev: 165759dced7fbe73b1db2ede67047896071dc6d0)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code doesn't work if the variable is unset, fix that.
(From OE-Core rev: b5248cc232629b021d8934899326468c3ef46351)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This helps bridge the old setuptools3 behavior.
FILES:${PN} has sane defaults in setuptools3-base
(From OE-Core rev: b1bb4e2d73985c6e8cf03b0fea94e8b739648cf7)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that all of the functions in cve-check open the database read-only,
we can remove this lockfile.
This means cve-check can run in parallal again, improving runtimes
massively.
This reverts commit d55fbf4779.
(From OE-Core rev: d3d3e7b324698ec3e6fce1951aba389805f3edd3)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This fixes the name for native and nativesdk recipes.
(From OE-Core rev: 498342f483118d22f529c4e255cc50455d51e9ed)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than only use PYPI_PACKAGE as a guess, fall back on PN for cases
where a recipe does not inherit pypi.
Wheels can only have alphanumeric characters in the 'distribution'
name [1]. Any other characters are replaced with an underscore. Provide a
function to replace dash with underscore.
[1] https://www.python.org/dev/peps/pep-0491/#escaping-and-unicode
(From OE-Core rev: 9fc8e55892374f1e63b4c995fe1a5539c42d24e0)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Almost everywhere those are the same, except when making a cross toolchain
where HOST_ is where it's going to run, and TARGET_ is what it's going to
produce.
(From OE-Core rev: cd25e5544ca3f48cc4c32001e917aa3dc20dd79a)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This matters when cross-compiling a cross-toolchain:
we need to specify the system where the toolchain
will run, not the system it will produce output for.
For everything else, HOST and TARGET are the same.
(From OE-Core rev: 03679622c5d1088e96d3d2a444c99021e5ae6ee6)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
distutils has been deprecated in Python 3.10 and will be removed in
Python 3.12 (predicted release date October 2023). For now, move these
classes from oe-core to meta-python to allow users to migrate.
[YOCTO #14610]
(From OE-Core rev: 782ce913b3611da8571a758c821b1491493eabec)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Depend on python3-wheel-native so that we can build with 'setup.py
bdist_wheel'.
Use pip_install_wheel class to install the built wheels with pip, as
intended by upstream Python.
[YOCTO #14638]
(From OE-Core rev: 8b39c0bc535814e04d01d50a4891cb31b6bf84bd)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This class uses the PEP 517 compliant setuptools.buil_meta to build
wheels.
[YOCTO #14638]
(From OE-Core rev: b8f25c75f152683de4fa6c176118583d3dea894d)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Replace the use of --force-reinstall with --ignore-installed when running pip
install. It can detect currently installed modules in the wrong environment and
try to remove them currently which is not what is desired. Ignoring them is the
correct thing to do.
(From OE-Core rev: 39b0d36b00d98a848297d0667c6cffc049d215e3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we modify the file, we need to recompile the pyc files since the file
hash has changed. This would otherwise result in reproducibility failrues.
(From OE-Core rev: 9573034eb8cdc3b9c2add67ac0a92277dc26389f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Provide a helper class to use pip to install wheels built by either
bdist_wheel or a PEP-517 backend.
Set pip install arguments via PIP_INSTALL_ARGS, which can be overriden
by recipes.
Pass --root and --prefix to ensure that pip installs things into the
proper place in sysroot.
By passing --no-deps and --no-index we avoid finicky dependency
checking (pip expects wheels in its cache) and avoid trying to fetch
wheels from pypi.org. This is basically the same behavior we have now,
the dependencies should be declared in the recipe.
Also pass --force-reinstall to make sure built wheels are always installed
so that FILES gets properly populated.
Pass --no-cache to avoid a (harmless) warning about the pip cache in
$HOME be avoiding use of cache. We do not likely want wheels cached
anyway,
pip install changes the python interpreter in scripts installed in
${bindir}, e.g. to #!/usr/bin/nativepython3, correct the behavior after
install to #!/usr/bin/env python3.
[YOCTO #14638]
(From OE-Core rev: 32a61afde0e7d8df6634b88525d8c3e8c6c3516e)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some python packages now use pyproject.toml and declare
flit_core.buildapi as the build engine
Use pip_install_wheel class to install.
[YOCTO #14638]
(From OE-Core rev: f0292098dc5afc62e0cd91d475edeb1784700795)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The overlayfs_qa_check in the rootfs-postcommands class throws an
exception if the overlayfs distro feature is enabled but no global
OVERLAYFS_MOUNT_POINT is specifived.
File: '/home/stefan/dalos-linutronix/poky/meta/classes/rootfs-postcommands.bbclass', lineno: 389, function: overlayfs_qa_check
0385: searchpaths = [oe.path.join(imagepath, d.getVar("sysconfdir"), "systemd", "system"),
0386: oe.path.join(imagepath, d.getVar("systemd_system_unitdir"))]
0387:
0388: allUnitExist = True;
*** 0389: for mountPoint in overlayMountPoints:
0390: path = d.getVarFlag('OVERLAYFS_MOUNT_POINT', mountPoint)
0391: unit = mountUnitName(path)
0392:
0393: if not any(os.path.isfile(oe.path.join(dirpath, unit))
Exception: TypeError: 'NoneType' object is not iterable
(From OE-Core rev: 4396348c24efb1185d68876d3cf6ade90324dee5)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By renaming LICENSE_EXCLUSION to _exclude_incompatible, it makes it
clear that this is an internal variable.
(From OE-Core rev: 20a4cc2c2dcf345ef898abfe7735b7bc75ac0059)
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
All of the function in cve-check should open the database read-only, as
the only writer is the fetch task in cve-update-db. However,
get_cve_info() was failing to do this, which might be causing locking
issues with sqlite.
(From OE-Core rev: 8de517238f1f418d9af1ce312d99de04ce2e26fc)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
nativesdk is a cross compiled target and therefore should use the target
config, not the native one. Copy the target entries accordingly.
(From OE-Core rev: b1b5fec350b390fa7f2d26966df1411b032faf87)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
meta-openembedded has gtk4 which has its own version of
gtk-update-icon-cache. Allow programs that want to use gtk4
use the proper version of the gtk-update-icon-cache utility
and the proper build and runtime dependencies.
Also use a more readable syntax for conditional values in DEPENDS.
(From OE-Core rev: 2a927a581e599af8e544e17517cdda1cfaf63e16)
Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, SPDX SBOMs are only created for images. Add support for
SDKs.
(From OE-Core rev: c3acbb936a339636153903daf127eec9f36de79b)
Signed-off-by: Andres Beltran <abeltran@linux.microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
'class' is already taken as keyword and python doesn't like this:
ERROR: Error in compiling python function in oe-core/meta/classes/icecc.bbclass, line 151:
The code lines resulting in this error were:
0020: check_pn = set([pn, bpn])
0021:
0022: class_disable = (d.getVar('ICECC_CLASS_DISABLE') or "").split()
0023:
*** 0024: for class in class_disable:
0025: if bb.data.inherits_class(class, d):
0026: bb.debug(1, "%s: class %s found in disable, disable icecc" % (pn, class))
0027: return "no"
0028:
SyntaxError: invalid syntax (icecc.bbclass, line 151)
(From OE-Core rev: 31be8f47502681d5621af032bca216c22f78fb84)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add further tweaks to comments and variable names for license variable change.
(From OE-Core rev: 4125d86f1d3adc53230c02bce4bab46b8c21116f)
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The SYSTEM and USER seperation between variables seems odd and not necessary,
drop it. Avoid the use of whitelist/blacklist and also change "packages" to
"recipes" since that misuse causes confusion.
(From OE-Core rev: 0df0eb6401a02139b9110bc95e21d97a67125ec5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a more descriptive variable name updated in base.bbclass
(From OE-Core rev: d28227ff665f4dcc4e7522829e531cdc1fbb1da4)
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove the old class and rename VarFlag to SKIP_RECIPE, handling this
in base.bbclass for efficiency. This means a separate inherit is no longer
needed.
This change better describes what the VarFlag is doing since it
is implemeted with the SkipRecipe() function.
By moving this into base.bbclass we simplify the distro inherit.
(From OE-Core rev: a5f735746cba6af41a25aa2aa121453a8bc363b4)
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After other variables were renamed in bitbake, update OE-Core to match.
(From OE-Core rev: 91812ba5a34598e03ad860745707c7cba1ae5d91)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After the change to bitbake, update the references in OE-Core to match the updates.
(From OE-Core rev: 193affb9f28b0116c3fd619834f145326fee08c5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is useful when needing to test layer's recipes, where this special
bbclass can define a global python function that gets called on each
recipe parsing during "bitbake -S none world" signature dump and be able
to fail layer's check accordingly.
First test being added is to detect recipes skipping "installed-vs-shipped"
QA check. As "installed-vs-shipped" is a packaging QA check, it happens very
late in the build process and failing it could mean some potential issues
with packaging, especially when recipe uses BBCLASSEXTEND="nativesdk" and
resulting package is used in an SDK.
In OE-Core failing this QA check leads to an error, but other layers can
suppress it or change it to a warning. Detecting weird packaging problems
with SDKs is quite difficult and time consuming. Also, waiting for the
actual "installed-vs-shipped" packaging QA check to fail means that all
recipes in the layer under test have to run through all standard tasks in
the build chain, equivalent to a multi-hour world-build.
Hence yocto-check-layer takes a shortcut and only detects a mere attempt
at skipping "installed-vs-shipped" QA check in the INSANE_SKIP list during
initial parsing when dumping the signature information for the layer.
(From OE-Core rev: e8baa75535fc888f1d768b23a0140475e832c910)
Signed-off-by: Denys Dmytriyenko <denis@denix.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As kernel modules depend on the base kernel package, and the base kernel
recipe depends on the kernel image, it's impossible to build file system
images which contain kernel modules but not the kernel itself, such as
an initramfs.
Change the RDEPENDS to RRECOMMENDS so that the disk images can set
PACKAGE_EXCLUDE = "kernel-image-*" to remove the kernel.
(From OE-Core rev: 1c90b27d2c65cfb4f9debf0272820b6a95942f76)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The threading code here can race as the fetcher changes the environment which is
shared between the threads. By setting it up in advance, it isn't changed and
therefore no longer races.
(From OE-Core rev: cabc3cc2eac5916e63340c18d1074411b377ced4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
An automated conversion using scripts/contrib/convert-spdx-licenses.py to
convert to use the standard SPDX license identifiers. Two recipes in meta-selftest
were not converted as they're that way specifically for testing. A change in
linux-firmware was also skipped and may need a more manual tweak.
(From OE-Core rev: ceda3238cdbf1beb216ae9ddb242470d5dfc25e0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
example.com is proving unreliable at present so switch to our own connectivity
page instead. That page is very simple avoiding app overhead on our web server
which was an original reason for switching to example.com.
(From OE-Core rev: dc6b043cb75c5751b5a98afd2201aa31f9b4b9f6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For $UBOOT_ENV file to appear in sysroot, virtual/bootloader
must populate sysroot first. Add the missing dependency.
(From OE-Core rev: 77d96b70f37b70dde65cf31f917c9f524fa7016a)
Signed-off-by: Marek Vasut <marex@denx.de>
Cc: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I got this error on current master:
File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:do_write_qemuboot_conf(d)
0003:
File: '.../layers/openembedded-core/meta/classes/qemuboot.bbclass', lineno: 141, function: do_write_qemuboot_conf
0137: else:
0138: val = d.getVar(k)
0139: # we only want to write out relative paths so that we can relocate images
0140: # and still run them
*** 0141: if val.startswith(topdir):
0142: val = os.path.relpath(val, finalpath)
0143: cf.set('config_bsp', k, '%s' % val)
0144:
0145: # QB_DEFAULT_KERNEL's value of KERNEL_IMAGETYPE is the name of a symlink
Exception: AttributeError: 'NoneType' object has no attribute 'startswith'
Do nothing if "val" is None, which may happen for k = "UNINATIVE_LOADER".
(From OE-Core rev: 020043c16ce25238313cbf0d10e40e9f6551bf14)
Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
CMake includes a FetchContent module, which will download further
source code at configure time. With the network isolation this will now
fail, but as not all environments support network isolation we can tell
cmake to not download either for extra safety.
(From OE-Core rev: 138df1940fff303de48c98a95ac1bbaef4f120e2)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch will read the begining of source files and try to find
the SPDX-License-Identifier to populate the licenseInfoInFiles
field for each source file. This does not populate licenseConcluded
at this time, nor rolls it up to package level.
We read as binary file since some source code seem to have some
binary characters, the license is then converted to ascii strings.
(From OE-Core rev: 51e5f328635eb022143178c3169bae719509697a)
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
