Make sure the python interpreter is "#!/usr/bin/env python3" for
nativesdk to avoid the shebang path exceeding the limit when install
the sdk under the directory with long path.
Before:
$ cd ${target_sdk_dir}
$ vi ./sysroots/x86_64-oesdk-linux/usr/bin/2to3
#!${target_sdk_dir}/sysroots/${SDK_SYS}/usr/bin/python3.9
[snip]
After:
$ cd ${target_sdk_dir}
$ vi ./sysroots/x86_64-oesdk-linux/usr/bin/2to3
#!/usr/bin/env python3
[snip]
(From OE-Core rev: 57af8ed4d3f1645d9a648ae1e569c792d9bcb361)
Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit a748e6098fa7d2ff594319937c7e0cf97700e83a)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are several thousand files in the test directory which we don't need.
Adding these for the native and target sysroots is a crazy amount of files
to be throwing around needlessly. Delete the files from the sysroot side
of things to tidy up the sysroots and improve performance.
(From OE-Core rev: 653f8b1a8adf59d924028bfc69ead1c7437de11e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit f6bced03011ad1663d68b0322a2f8aeb4d836646)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The externalsrc class deletes do_patch task which results with:
| ERROR: Task do_create_manifest in <PATH>/python3_3.8.2.bb depends upon
| non-existent task do_patch in <PATH>/python3_3.8.2.bb
Use addtask to define correct order to prevent this error, since addtask
mechanism accepts deleted tasks.
[YOCTO #14151]
(From OE-Core rev: a746d034fa7eaad4f4876fa61c5a8c3c15e211c8)
(From OE-Core rev: 782f7f4f73d7b1e1bd649db1984a63d94d2410fe)
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 8ed8b81af6)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This issue describes expected behaviour, do not use tarfile with
untrusted data.
(From OE-Core rev: 391ed53928db0df325798a0bce18ec6947e09ddd)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit f4c22e83f2e68ff157da5ea1303acc2931d63f5f)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is used by some of the results handling code and needed as part of
buildtools tarball on various autobuilder worker for testing.
ptest is disabled for OE-Core, at least for now since it depends on
python3-pytest which in turn has may other dependencies.
Acked-by: Tim Orling <ticotimo@gmail.com>
(From OE-Core rev: 9d30a3b95ef4dce8925f88c8d8d854ed48c5f367)
Signed-off-by: Tim Orling <ticotimo@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Needed as a dependency of python3-jinja2. ptest is disabled for OE-Core, at
least for now since it depends on python3-pytest which in turn has may other
dependencies.
Acked-by: Tim Orling <ticotimo@gmail.com>
(From OE-Core rev: 94a04a42826ee95a27abe52d43b31b0f053260d9)
Signed-off-by: Tim Orling <ticotimo@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
- Regenerate using create_manifest
- Fix up some indentation compared to generated
- Add "secrets" module into netclient package (introduced in 3.6)
- Move "urllib" python package into core package, it is used more
commonly than just netclient (e.g. email, xml, mimetypes, pydoc)
- Update compression package dependencies due to some modules moving
into core package
- Update dependencies due to improvements to get_module_deps handling
modules of a python package (e.g. ctypes.utils)
(From OE-Core rev: 892a8fb85a986a65e9ebef368ddab15068fb6d92)
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Specifically cover detection of modules within a python package that do
not import anything within their __init__.py. This is at least the case
with the xmlrpc package which is only used via its modules xmlrpc.server
and xmlrpc.client. Other important corner cases include ctypes.utils
which depends on some modules not used by ctypes.
This is implemented by generally assuming that importing all the modules
of a package (aka *.py within a package, excluding _*.py) will provide
enough information.
Also due to this change some modules import sysconfig, resulting in
sysconfigdata being imported. Handle the conversion of its path to a
wildcard based on the platform dependent name being replaced.
(From OE-Core rev: bef4be54e02df5f230d250487f85994a3b7bbd77)
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rebased patch and got rid of the .inc file as Python 2 is not supported
anymore.
(From OE-Core rev: ff3d9a2966e6593be9ac048d930fc841c24eae7d)
Signed-off-by: Oleksandr Kravchuk <open.source@oleksandr-kravchuk.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After ccache upgrade to 3.7.11, there is below logic introduced.
58895dc Add knowledge about -fprofile-{correction,reorder-functions,values}
And the python3 ccache build issue [1] is fixed in ccache 3.7.11, so there
is no need to define a profile directory path for python3 any more.
[1] https://github.com/ccache/ccache/issues/615
(From OE-Core rev: 6cd08bdf72c05982ba000b1186cf139382679e04)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After ccache upgrade to 3.7.10, there comes below error when
enable ccache to build python3 and check [1] and [2] for more
details.
| Python-3.8.3/Modules/_contextvarsmodule.c:43:1: error: source locations for function 'PyInit__contextvars' have changed, the profile data may be out of date [-Werror=coverage-mismatch]
That's because the logic for profile directory changes a little in
[3] after ccache upgrades to 3.7.10.
So define a profile directory path accordingly to fix the above error.
[1] https://github.com/ccache/ccache/issues/615
[2] https://lists.openembedded.org/g/openembedded-core/topic/75170448#140065
[3] 91a2954eb4
(From OE-Core rev: 0000d30fb82aafa5064051502a927c5fc447b63b)
Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The ctypes module needs to use "ldconfig -p" to find the
library path and it simply has below logic if no ldconfig
installed.
except OSError:
pass
Before the patch:
>>> from ctypes.util import find_library
>>> lib_path = find_library('archive')
>>> print(lib_path)
None
After the patch:
>>> from ctypes.util import find_library
>>> lib_path = find_library('archive')
>>> print(lib_path)
libarchive.so.13
(From OE-Core rev: ddb96902a124a6e1f035f0fd868b0139989bc1bc)
Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Apologies, but the patch is not possible to rebase onto the new setuptools,
as the code has changed too much. Please get it accepted upstream first.
(From OE-Core rev: f2feb53c967256431f03a07c1b4b9a0d8568d9b5)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
setuptools' pkg_resources module has major performance issues with how
it loads entry points (e.g. the console_script entry point, which sets
up a module as a command-line executable), leading even the simplest
"hello world" scripts to take on the order of 150ms to run if
pkg_resources is incorporated. This is prohibitive for code that needs
to run quickly, and so we patch setuptools to reduce this time. As of
Python 3.7, importlib.resources is available and intended to replace
much of the functionality that causes this sluggishness, but since
many projects still utilize the legacy setuptools modules, a patch is
still required. Note that python3-fastentrypoints (which is available
in the meta-virtualization layer) is also intended to help alleviate
the problem, but since it must be added to existing projects it has
the same disadvantage as resorting to importlib.resources, requiring
manual additions to existing code to see the performance gains.
The intent here is to patch easy_install to load module entry points
directly with the installed setuptools, rather than importing
pkg_resources and having it search out the entry points itself. This
leads to a drastic performance improvement - the changes in this patch
have been shown to result in load time ~6-8x lower, depending on the
complexity of the code it is tested with. A simple "hello world"
example on core-image-full-cmdline gave these results with and without
the patch:
core-image-full-cmdline, without setuptools ScriptWriter patch:
root@qemux86-64:~# time /usr/bin/minimal
hello world
real 0m0.198s
user 0m0.174s
sys 0m0.023s
core-image-full-cmdline, with setuptools ScriptWriter patch:
root@qemux86-64:~# time /usr/bin/minimal
hello world
real 0m0.034s
user 0m0.024s
sys 0m0.010s
More details on the pkg_resources issue are available at:
https://github.com/pypa/setuptools/issues/510
(From OE-Core rev: 9ff7c2f4a43e28ac6a89045c38effe03063f2061)
Signed-off-by: Trevor Gamblin <trevor.gamblin@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As the URL pypi.python.org simply redirects to pypi.org, simplify a
number of Python recipes by using that URL explicitly.
(From OE-Core rev: 934c8302e4ead9b09d6e7396ff11b6a4c1be7b21)
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When PGO was disabled by default by e53ebf29, a bug was introduced that
prevented PGO from ever being enabled. At the time at which extra config
is appended to PACKAGECONFIG_CONFARGS, PACKAGECONFIG_PGO remains
unevaluated in PACKAGECONFIG_class-target, due to setting its value in
an anonymous Python function. As a result, the PGO options options will
never be included.
(From OE-Core rev: 21446d4b6c5f59b6acb66133a9675ec3d3dbabe2)
Signed-off-by: Ryan Rowe <rrowe@xevo.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
