This issue describes expected behaviour, do not use tarfile with
untrusted data.
(From OE-Core rev: 391ed53928db0df325798a0bce18ec6947e09ddd)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit f4c22e83f2e68ff157da5ea1303acc2931d63f5f)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is used by some of the results handling code and needed as part of
buildtools tarball on various autobuilder worker for testing.
ptest is disabled for OE-Core, at least for now since it depends on
python3-pytest which in turn has may other dependencies.
Acked-by: Tim Orling <ticotimo@gmail.com>
(From OE-Core rev: 9d30a3b95ef4dce8925f88c8d8d854ed48c5f367)
Signed-off-by: Tim Orling <ticotimo@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Needed as a dependency of python3-jinja2. ptest is disabled for OE-Core, at
least for now since it depends on python3-pytest which in turn has may other
dependencies.
Acked-by: Tim Orling <ticotimo@gmail.com>
(From OE-Core rev: 94a04a42826ee95a27abe52d43b31b0f053260d9)
Signed-off-by: Tim Orling <ticotimo@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
- Regenerate using create_manifest
- Fix up some indentation compared to generated
- Add "secrets" module into netclient package (introduced in 3.6)
- Move "urllib" python package into core package, it is used more
commonly than just netclient (e.g. email, xml, mimetypes, pydoc)
- Update compression package dependencies due to some modules moving
into core package
- Update dependencies due to improvements to get_module_deps handling
modules of a python package (e.g. ctypes.utils)
(From OE-Core rev: 892a8fb85a986a65e9ebef368ddab15068fb6d92)
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Specifically cover detection of modules within a python package that do
not import anything within their __init__.py. This is at least the case
with the xmlrpc package which is only used via its modules xmlrpc.server
and xmlrpc.client. Other important corner cases include ctypes.utils
which depends on some modules not used by ctypes.
This is implemented by generally assuming that importing all the modules
of a package (aka *.py within a package, excluding _*.py) will provide
enough information.
Also due to this change some modules import sysconfig, resulting in
sysconfigdata being imported. Handle the conversion of its path to a
wildcard based on the platform dependent name being replaced.
(From OE-Core rev: bef4be54e02df5f230d250487f85994a3b7bbd77)
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rebased patch and got rid of the .inc file as Python 2 is not supported
anymore.
(From OE-Core rev: ff3d9a2966e6593be9ac048d930fc841c24eae7d)
Signed-off-by: Oleksandr Kravchuk <open.source@oleksandr-kravchuk.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After ccache upgrade to 3.7.11, there is below logic introduced.
58895dc Add knowledge about -fprofile-{correction,reorder-functions,values}
And the python3 ccache build issue [1] is fixed in ccache 3.7.11, so there
is no need to define a profile directory path for python3 any more.
[1] https://github.com/ccache/ccache/issues/615
(From OE-Core rev: 6cd08bdf72c05982ba000b1186cf139382679e04)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After ccache upgrade to 3.7.10, there comes below error when
enable ccache to build python3 and check [1] and [2] for more
details.
| Python-3.8.3/Modules/_contextvarsmodule.c:43:1: error: source locations for function 'PyInit__contextvars' have changed, the profile data may be out of date [-Werror=coverage-mismatch]
That's because the logic for profile directory changes a little in
[3] after ccache upgrades to 3.7.10.
So define a profile directory path accordingly to fix the above error.
[1] https://github.com/ccache/ccache/issues/615
[2] https://lists.openembedded.org/g/openembedded-core/topic/75170448#140065
[3] 91a2954eb4
(From OE-Core rev: 0000d30fb82aafa5064051502a927c5fc447b63b)
Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The ctypes module needs to use "ldconfig -p" to find the
library path and it simply has below logic if no ldconfig
installed.
except OSError:
pass
Before the patch:
>>> from ctypes.util import find_library
>>> lib_path = find_library('archive')
>>> print(lib_path)
None
After the patch:
>>> from ctypes.util import find_library
>>> lib_path = find_library('archive')
>>> print(lib_path)
libarchive.so.13
(From OE-Core rev: ddb96902a124a6e1f035f0fd868b0139989bc1bc)
Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Apologies, but the patch is not possible to rebase onto the new setuptools,
as the code has changed too much. Please get it accepted upstream first.
(From OE-Core rev: f2feb53c967256431f03a07c1b4b9a0d8568d9b5)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
setuptools' pkg_resources module has major performance issues with how
it loads entry points (e.g. the console_script entry point, which sets
up a module as a command-line executable), leading even the simplest
"hello world" scripts to take on the order of 150ms to run if
pkg_resources is incorporated. This is prohibitive for code that needs
to run quickly, and so we patch setuptools to reduce this time. As of
Python 3.7, importlib.resources is available and intended to replace
much of the functionality that causes this sluggishness, but since
many projects still utilize the legacy setuptools modules, a patch is
still required. Note that python3-fastentrypoints (which is available
in the meta-virtualization layer) is also intended to help alleviate
the problem, but since it must be added to existing projects it has
the same disadvantage as resorting to importlib.resources, requiring
manual additions to existing code to see the performance gains.
The intent here is to patch easy_install to load module entry points
directly with the installed setuptools, rather than importing
pkg_resources and having it search out the entry points itself. This
leads to a drastic performance improvement - the changes in this patch
have been shown to result in load time ~6-8x lower, depending on the
complexity of the code it is tested with. A simple "hello world"
example on core-image-full-cmdline gave these results with and without
the patch:
core-image-full-cmdline, without setuptools ScriptWriter patch:
root@qemux86-64:~# time /usr/bin/minimal
hello world
real 0m0.198s
user 0m0.174s
sys 0m0.023s
core-image-full-cmdline, with setuptools ScriptWriter patch:
root@qemux86-64:~# time /usr/bin/minimal
hello world
real 0m0.034s
user 0m0.024s
sys 0m0.010s
More details on the pkg_resources issue are available at:
https://github.com/pypa/setuptools/issues/510
(From OE-Core rev: 9ff7c2f4a43e28ac6a89045c38effe03063f2061)
Signed-off-by: Trevor Gamblin <trevor.gamblin@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As the URL pypi.python.org simply redirects to pypi.org, simplify a
number of Python recipes by using that URL explicitly.
(From OE-Core rev: 934c8302e4ead9b09d6e7396ff11b6a4c1be7b21)
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When PGO was disabled by default by e53ebf29, a bug was introduced that
prevented PGO from ever being enabled. At the time at which extra config
is appended to PACKAGECONFIG_CONFARGS, PACKAGECONFIG_PGO remains
unevaluated in PACKAGECONFIG_class-target, due to setting its value in
an anonymous Python function. As a result, the PGO options options will
never be included.
(From OE-Core rev: 21446d4b6c5f59b6acb66133a9675ec3d3dbabe2)
Signed-off-by: Ryan Rowe <rrowe@xevo.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are issues with multilib due to the ordering of events where some
functions see the remapped multilib dependencies and some do not. A significant
problem is that the multilib class needs to make some changes before key expansion
and some afterwards but by using existing event handlers, some code sees things
in a partially translated state, leading to bugs.
This patch changes things to use a new event handler from bitbake which makes the
ordering of the changes explcit.
The challenge in doing this is that it breaks some existing anonymous python and
dyanmic assignments. In some cases these used to be translated and no longer are,
meaning MLPREFIX has to be added. In some cases these are now translated and the
MLPREFIX can be removed.
This change does now make it very clear when MLPREFIX is required and when it is
not, its just the migration path which is harder. The patch changes the small number
of cases where fixes are needed.
In particular, where a variable like RDEPENDS is conditionally extended (e.g.
with an override), MLPREFIX is now required.
This patch also reverts:
base: Revert 'base.bbclass: considering multilib when setting LICENSE_EXCLUSION'
This reverts 6597130256a1609c3e05ec5891aceaf549c37985 as the changes
to multilib datastore handling mean its no longer necessary.
(From OE-Core rev: b3fda056a674889cd9697e779de023d4f993d3ce)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Also splits apart the SRC_URI checksums to make automatic upgrades
easier
(From OE-Core rev: ae1f210546396b761ea86d9e32bf90c0867ff845)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
