diff --git a/documentation/migration-guides/migration-4.0.rst b/documentation/migration-guides/migration-4.0.rst index 202999ece8..097eeafdb4 100644 --- a/documentation/migration-guides/migration-4.0.rst +++ b/documentation/migration-guides/migration-4.0.rst @@ -1,61 +1,74 @@ Release 4.0 (kirkstone) ======================= +Migration notes for 4.0 (kirkstone) +----------------------------------- + This section provides migration information for moving to the Yocto Project 4.0 Release (codename "kirkstone") from the prior release. -Recipe changes --------------- +.. _migration-4.0-inclusive-language: -- To use more `inclusive language `__ - in the code and documentation, some variables have been renamed or even - deleted. BitBake will stop with an error when renamed or removed variables - still exist in your recipes or configuration. +Inclusive language improvements +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Please note that the change applies also to environmental variables, so - make sure you use a fresh environment for your build. +To use more `inclusive language `__ +in the code and documentation, some variables have been renamed, and +some have been deleted where they are no longer needed. In many cases the +new names are also easier to understand. BitBake will stop with an error when +renamed or removed variables still exist in your recipes or configuration. - The following variables have changed their names: +Please note that the change applies also to environmental variables, so +make sure you use a fresh environment for your build. - - ``BB_ENV_WHITELIST`` became :term:`BB_ENV_PASSTHROUGH` - - ``BB_ENV_EXTRAWHITE`` became :term:`BB_ENV_PASSTHROUGH_ADDITIONS` - - ``BB_HASHBASE_WHITELIST`` became :term:`BB_BASEHASH_IGNORE_VARS` - - ``BB_HASHCONFIG_WHITELIST`` became :term:`BB_HASHCONFIG_IGNORE_VARS` - - ``BB_HASHTASK_WHITELIST`` became ``BB_TASKHASH_IGNORE_TASKS`` - - ``BB_SETSCENE_ENFORCE_WHITELIST`` became ``BB_SETSCENE_ENFORCE_IGNORE_TASKS`` - - ``CVE_CHECK_PN_WHITELIST`` became :term:`CVE_CHECK_SKIP_RECIPE` - - ``CVE_CHECK_WHITELIST`` became :term:`CVE_CHECK_IGNORE` - - ``ICECC_USER_CLASS_BL`` became :term:`ICECC_CLASS_DISABLE` - - ``ICECC_SYSTEM_CLASS_BL`` became :term:`ICECC_CLASS_DISABLE` - - ``ICECC_USER_PACKAGE_WL`` became :term:`ICECC_RECIPE_ENABLE` - - ``ICECC_USER_PACKAGE_BL`` became :term:`ICECC_RECIPE_DISABLE` - - ``ICECC_SYSTEM_PACKAGE_BL`` became :term:`ICECC_RECIPE_DISABLE` - - ``LICENSE_FLAGS_WHITELIST`` became :term:`LICENSE_FLAGS_ACCEPTED` - - ``MULTI_PROVIDER_WHITELIST`` became :term:`BB_MULTI_PROVIDER_ALLOWED` - - ``PNBLACKLIST`` became :term:`SKIP_RECIPE` - - ``SDK_LOCAL_CONF_BLACKLIST`` became :term:`ESDK_LOCALCONF_REMOVE` - - ``SDK_LOCAL_CONF_WHITELIST`` became :term:`ESDK_LOCALCONF_ALLOW` - - ``SDK_INHERIT_BLACKLIST`` became :term:`ESDK_CLASS_INHERIT_DISABLE` - - ``SSTATE_DUPWHITELIST`` became ``SSTATE_ALLOW_OVERLAP_FILES`` - - ``SYSROOT_DIRS_BLACKLIST`` became :term:`SYSROOT_DIRS_IGNORE` - - ``UNKNOWN_CONFIGURE_WHITELIST`` became :term:`UNKNOWN_CONFIGURE_OPT_IGNORE` +The following variables have changed their names: - In addition, ``BB_STAMP_WHITELIST``, ``BB_STAMP_POLICY``, ``INHERIT_BLACKLIST`` - and ``TUNEABI_WHITELIST`` have been removed. +- ``BB_ENV_WHITELIST`` became :term:`BB_ENV_PASSTHROUGH` +- ``BB_ENV_EXTRAWHITE`` became :term:`BB_ENV_PASSTHROUGH_ADDITIONS` +- ``BB_HASHBASE_WHITELIST`` became :term:`BB_BASEHASH_IGNORE_VARS` +- ``BB_HASHCONFIG_WHITELIST`` became :term:`BB_HASHCONFIG_IGNORE_VARS` +- ``BB_HASHTASK_WHITELIST`` became ``BB_TASKHASH_IGNORE_TASKS`` +- ``BB_SETSCENE_ENFORCE_WHITELIST`` became ``BB_SETSCENE_ENFORCE_IGNORE_TASKS`` +- ``CVE_CHECK_PN_WHITELIST`` became :term:`CVE_CHECK_SKIP_RECIPE` +- ``CVE_CHECK_WHITELIST`` became :term:`CVE_CHECK_IGNORE` +- ``ICECC_USER_CLASS_BL`` became :term:`ICECC_CLASS_DISABLE` +- ``ICECC_SYSTEM_CLASS_BL`` became :term:`ICECC_CLASS_DISABLE` +- ``ICECC_USER_PACKAGE_WL`` became :term:`ICECC_RECIPE_ENABLE` +- ``ICECC_USER_PACKAGE_BL`` became :term:`ICECC_RECIPE_DISABLE` +- ``ICECC_SYSTEM_PACKAGE_BL`` became :term:`ICECC_RECIPE_DISABLE` +- ``LICENSE_FLAGS_WHITELIST`` became :term:`LICENSE_FLAGS_ACCEPTED` +- ``MULTI_PROVIDER_WHITELIST`` became :term:`BB_MULTI_PROVIDER_ALLOWED` +- ``PNBLACKLIST`` became :term:`SKIP_RECIPE` +- ``SDK_LOCAL_CONF_BLACKLIST`` became :term:`ESDK_LOCALCONF_REMOVE` +- ``SDK_LOCAL_CONF_WHITELIST`` became :term:`ESDK_LOCALCONF_ALLOW` +- ``SDK_INHERIT_BLACKLIST`` became :term:`ESDK_CLASS_INHERIT_DISABLE` +- ``SSTATE_DUPWHITELIST`` became ``SSTATE_ALLOW_OVERLAP_FILES`` +- ``SYSROOT_DIRS_BLACKLIST`` became :term:`SYSROOT_DIRS_IGNORE` +- ``UNKNOWN_CONFIGURE_WHITELIST`` became :term:`UNKNOWN_CONFIGURE_OPT_IGNORE` +- ``WHITELIST_`` became ``INCOMPATIBLE_LICENSE_EXCEPTIONS`` - Many internal variable names have been also renamed accordingly. +In addition, ``BB_STAMP_WHITELIST``, ``BB_STAMP_POLICY``, ``INHERIT_BLACKLIST``, +``TUNEABI``, ``TUNEABI_WHITELIST``, and ``TUNEABI_OVERRIDE`` have been removed. - In addition, in the ``cve-check`` output, the CVE issue status ``Whitelisted`` - has been renamed to ``Ignored``. +Many internal variable names have been also renamed accordingly. - A :oe_git:`convert-variable-renames.py - ` - script is provided to convert your recipes and configuration, - and also warns you about the use of problematic words. The script performs - changes and you need to review them before committing. An example warning - looks like:: +In addition, in the ``cve-check`` output, the CVE issue status ``Whitelisted`` +has been renamed to ``Ignored``. - poky/scripts/lib/devtool/upgrade.py needs further work at line 275 since it contains abort +The :term:`BB_DISKMON_DIRS` variable value now uses the term ``HALT`` +instead of ``ABORT``. + +A :oe_git:`convert-variable-renames.py +` +script is provided to convert your recipes and configuration, +and also warns you about the use of problematic words. The script performs +changes and you need to review them before committing. An example warning +looks like:: + + poky/scripts/lib/devtool/upgrade.py needs further work at line 275 since it contains abort + +Fetching changes +~~~~~~~~~~~~~~~~ - Because of the uncertainty in future default branch names in git repositories, it is now required to add a branch name to all URLs described @@ -70,7 +83,8 @@ Recipe changes - Because of `GitHub dropping support for the git: protocol `__, recipes now need to use ``;protocol=https`` at the end of GitHub - URLs. The same script as above can be used to convert the recipes. + URLs. The same ``convert-srcuri`` script mentioned above can be used to convert + your recipes. - Network access from tasks is now disabled by default on kernels which support this feature (on most recent distros such as CentOS 8 and Debian 11 onwards). @@ -84,87 +98,169 @@ Recipe changes usually undermines fetcher source mirroring, image and licence manifests, software auditing and supply chain security. -- The :term:`TOPDIR` variable and the current working directory are no longer modified - when parsing recipes. Any code depending on that behaviour will no longer work. +License changes +~~~~~~~~~~~~~~~ -- The ``append``, ``prepend`` and ``remove`` operators can now only be combined with - ``=`` and ``:=`` operators. To the exception of the ``append`` plus ``+=`` and - ``prepend`` plus ``=+`` combinations, all combinations could be factored up to the - ``append``, ``prepend`` or ``remove`` in the combination. This brought a lot of - confusion on how the override style syntax operators work and should be used. - Therefore, those combinations can simply be replaced by a single ``append``, - ``prepend`` or ``remove`` operator without any additional change. - For the ``append`` plus ``+=`` (and ``prepend`` plus ``=+``) combinations, - the content should be prefixed (respectively suffixed) by a space to maintain - the same behavior. You can learn more about override style syntax operators - (``append``, ``prepend`` and ``remove``) in the BitBake documentation: - :ref:`bitbake:bitbake-user-manual/bitbake-user-manual-metadata:appending and prepending (override style syntax)` - and :ref:`bitbake:bitbake-user-manual/bitbake-user-manual-metadata:removal (override style syntax)`. +- The ambiguous "BSD" license has been removed from the ``common-licenses`` directory. + Each recipe that fetches or builds BSD-licensed code should specify the proper + version of the BSD license in its :term:`LICENSE` value. -- :ref:`allarch ` packagegroups can no longer depend on packages - which use :term:`PKG` renaming such as :ref:`ref-classes-debian`. - -- :term:`LICENSE` definitions now have to use `SPDX identifiers `__. - A :oe_git:`convert-spdx-licenses.py ` +- :term:`LICENSE` variable values should now use `SPDX identifiers `__. + If they do not, by default a warning will be shown. A + :oe_git:`convert-spdx-licenses.py ` script can be used to update your recipes. + +- :term:`INCOMPATIBLE_LICENSE` should now use `SPDX identifiers `__. + Additionally, wildcarding is now limited to specifically supported values - + see the :term:`INCOMPATIBLE_LICENSE` documentation for further information. -- :term:`SRC_URI`: a new :ref:`bitbake:bitbake-user-manual/bitbake-user-manual-fetching:crate fetcher (\`\`crate://\`\`)` - is available for Rust packages. +- The ``AVAILABLE_LICENSES`` variable has been removed. This variable was a performance + liability and is highly dependent on which layers are added to the configuration, + which can cause signature issues for users. In addition the ``available_licenses()`` + function has been removed from the :ref:`license ` class as + it is no longer needed. + +Removed recipes +~~~~~~~~~~~~~~~ -Class changes -------------- +The following recipes have been removed in this release: -- The ``distutils*.bbclasses`` have been moved to ``meta-python``. The classes and - `DISTUTILS*` variables have been removed from the documentation. +- ``dbus-test``: merged into main dbus recipe +- ``libid3tag``: moved to meta-oe - no longer needed by anything in OE-Core +- ``libportal``: moved to meta-gnome - no longer needed by anything in OE-Core +- ``linux-yocto``: removed version 5.14 recipes (5.15 and 5.10 still provided) +- ``python3-nose``: has not changed since 2016 upstream, and no longer needed by anything in OE-Core +- ``rustfmt``: not especially useful as a standalone recipe -- ``blacklist.bbclass`` is removed and the functionality moved to the - :ref:`base ` class with a more descriptive - ``varflag`` named :term:`SKIP_RECIPE` which will use the `SkipRecipe()` - function. The usage will remain the same:: - - SKIP_RECIPE[my-recipe] = "Reason for skipping recipe" - -- The Python package build process based on `wheels `__. +Python changes +~~~~~~~~~~~~~~ + +- ``distutils`` has been deprecated upstream in Python 3.10 and thus the ``distutils*`` + classes have been moved to ``meta-python``. Recipes that inherit the ``distutils*`` + classes should be updated to inherit ``setuptools*`` equivalents instead. + +- The Python package build process is now based on `wheels `__. Here are the new Python packaging classes that should be used: - :ref:`python-flit_core `, - :ref:`setuptools_python-build_meta ` - and :ref:`python_poetry_core `. + :ref:`python_flit_core `, + :ref:`python_setuptools_build_meta ` + and :ref:`python_poetry_core `. -- ``image-prelink.bbclass`` class is removed. +- The :ref:`setuptools3 ` class ``do_install()`` task now + installs the ``wheel`` binary archive. In current versions of ``setuptools`` the + legacy ``setup.py install`` method is deprecated. If the ``setup.py`` cannot be used + with wheels, for example it creates files outside of the Python module or standard + entry points, then :ref:`setuptools3_legacy ` should + be used instead. -- New :ref:`overlayfs ` and - :ref:`overlayfs-etc ` classes are available - to make it easier to overlay read-only filesystems (for example) - with `OverlayFS `__. +Prelink removed +~~~~~~~~~~~~~~~ -Configuration changes ---------------------- +Prelink has been dropped by ``glibc`` upstream in 2.36. It already caused issues with +binary corruption, has a number of open bugs and is of questionable benefit +without disabling load address randomization and PIE executables. + +We disabled prelinking by default in the honister (3.4) release, but left it able +to be enabled if desired. However, without glibc support it cannot be maintained +any further, so all of the prelinking functionality has been removed in this release. +If you were enabling the ``image-prelink`` class in :term:`INHERIT`, :term:`IMAGE_CLASSES`, +:term:`USER_CLASSES` etc in your configuration, then you will need to remove the +reference(s). -- The Yocto Project now allows to reuse Shared State from its autobuilder. - If the network connection between our server and your machine is faster - than you would build recipes, you can try to speed up your builds - by using such Share State and Hash Equivalence by setting:: +Reproducible as standard +~~~~~~~~~~~~~~~~~~~~~~~~ - BB_SIGNATURE_HANDLER = "OEEquivHash" - BB_HASHSERVE = "auto" - BB_HASHSERVE_UPSTREAM = "typhoon.yocto.io:8687" - SSTATE_MIRRORS ?= "file://.* https://sstate.yoctoproject.org/&YOCTO_DOC_VERSION;/PATH;downloadfilename=PATH" +Reproducibility is now considered as standard functionality, thus the +``reproducible`` class has been removed and its previous contents merged into the +:ref:`base ` class. If you have references in your configuration to +``reproducible`` in :term:`INHERIT`, :term:`USER_CLASSES` etc. then they should be +removed. + +Additionally, the ``BUILD_REPRODUCIBLE_BINARIES`` variable is no longer used. +Specifically for the kernel, if you wish to enable build timestamping functionality +that is normally disabled for reproducibility reasons, you can do so by setting +a new :term:`KERNEL_DEBUG_TIMESTAMPS` variable to "1". Supported host distribution changes ------------------------------------ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- New support for `AlmaLinux `__ +- Support for `AlmaLinux `__ hosts replacing `CentOS `__. The following distribution versions were dropped: CentOS 8, Ubuntu 16.04 and Fedora 30, 31 and 32. -Changes for release notes -------------------------- +- ``gcc`` version 7.5 is now required at minimum on the build host. For older + host distributions where this is not available, you can use the + ``buildtools-extended-tarball`` (easily installable using + ``scripts/install-buildtools``). -- Share State cache: now using `ZStandard (zstd) `__ - instead of Gzip compression, for better performance. +:append/:prepend in combination with other operators +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- BitBake has an improved ``setscene`` task display. +The ``append``, ``prepend`` and ``remove`` operators can now only be combined with +``=`` and ``:=`` operators. To the exception of the ``append`` plus ``+=`` and +``prepend`` plus ``=+`` combinations, all combinations could be factored up to the +``append``, ``prepend`` or ``remove`` in the combination. This brought a lot of +confusion on how the override style syntax operators work and should be used. +Therefore, those combinations should be replaced by a single ``append``, +``prepend`` or ``remove`` operator without any additional change. +For the ``append`` plus ``+=`` (and ``prepend`` plus ``=+``) combinations, +the content should be prefixed (respectively suffixed) by a space to maintain +the same behavior. You can learn more about override style syntax operators +(``append``, ``prepend`` and ``remove``) in the BitBake documentation: +:ref:`bitbake:bitbake-user-manual/bitbake-user-manual-metadata:appending and prepending (override style syntax)` +and :ref:`bitbake:bitbake-user-manual/bitbake-user-manual-metadata:removal (override style syntax)`. -- This release fixes the reproducibility issues with ``rust-llvm`` and ``golang``. - Recipes in OpenEmbedded-Core are now fully reproducible. +Miscellaneous changes +~~~~~~~~~~~~~~~~~~~~~ + +- ``blacklist.bbclass`` is removed and the functionality moved to the + :ref:`base ` class with a more descriptive + ``varflag`` variable named :term:`SKIP_RECIPE` which will use the `bb.parse.SkipRecipe()` + function. The usage remains the same, for example:: + SKIP_RECIPE[my-recipe] = "Reason for skipping recipe" + +- :ref:`allarch ` packagegroups can no longer depend on packages + which use :term:`PKG` renaming such as :ref:`ref-classes-debian`. Such packagegroups + recipes should be changed to avoid inheriting :ref:`allarch `. + +- The ``lnr`` script has been removed. ``lnr`` implemented the same behaviour as `ln --relative --symbolic`, + since at the time of creation `--relative` was only available in coreutils 8.16 + onwards which was too new for the older supported distros. Current supported host + distros have a new enough version of coreutils, so it is no longer needed. If you have + any calls to ``lnr`` in your recipes or classes, they should be replaced with + `ln --relative --symbolic` or `ln -rs` if you prefer the short version. + +- The ``package_qa_handle_error()`` function formerly in the :ref:`insane ` + class has been moved and renamed - if you have any references in your own custom + classes they should be changed to ``oe.qa.handle_error()``. + +- When building ``perl``, Berkeley db support is no longer enabled by default, since + Berkeley db is largely obsolete. If you wish to reenable it, you can append ``bdb`` + to :term:`PACKAGECONFIG` in a ``perl`` bbappend or ``PACKAGECONFIG:pn-perl`` at + the configuration level. + +- For the ``xserver-xorg`` recipe, the ``xshmfence``, ``xmlto`` and ``systemd`` options + previously supported in :term:`PACKAGECONFIG` have been removed, as they are no + longer supported since the move from building it with autotools to meson in this release. + +- For the ``libsdl2`` recipe, various X11 features are now disabled by default (primarily + for reproducibility purposes in the native case) with options in :term:`EXTRA_OECMAKE` + within the recipe. These can be changed within a bbappend if desired. See the + ``libsdl2`` recipe for more details. + +- The ``cortexa72-crc`` and ``cortexa72-crc-crypto`` tunes have been removed since + the crc extension is now enabled by default for cortexa72. Replace any references to + these with ``cortexa72`` and ``cortexa72-crypto`` respectively. + +- The Python development shell (previously known as ``devpyshell``) feature has been + renamed to ``pydevshell``. To start it you should now run:: + + bitbake -c pydevshell + +- The ``packagegroups-core-full-cmdline-libs`` packagegroup is no longer produced, as + libraries should normally be brought in via dependencies. If you have any references + to this then remove them. + +- The :term:`TOPDIR` variable and the current working directory are no longer modified + when parsing recipes. Any code depending on the previous behaviour will no longer + work - change any such code to explicitly use appropriate path variables instead.