glibc 2.28 slightly changed the behaviour of sigaddset() which broke
multiprocessing. Backport a patch from Python 3.6 to solve this.
(From OE-Core rev: df1874eda517070dab86270ffafd5fe9a307e76b)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch dates back to the addition of the Python 3 recipe to oe-core, and as
listxattr is never added to supports_follow_symlinks the extended attribute
support will never be enabled.
(From OE-Core rev: 9806e141e85b4e4c38eb7b45e6f2fbc2d2aed29d)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
None of the Python invocations that this changes are actually called, and
there's no need to provide a HOSTPGEN variable when the recipe can just override
PGEN directly.
(From OE-Core rev: 47a8602171428b7ce5d897f7e2c2f26b203b8b63)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With OpenSSL 1.1.x TLS 1.3 can be used, so backport a patch from Python 3.6 to
fix the ftplib unit test.
(From OE-Core rev: a31047bec6b7c368674d4620e70e526ac211b936)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
PyBench takes a long time to run, also upstream have removed it from Python and
instead use test.regrtest —pgo to profile the interpreter.
The results are good: not only does Python compile faster (~300s vs ~600s on my
machine) but Phoronix’s PyBench test runs in 2130ms compared to 2229ms when
using PyBench to train (and 2345ms with PGO disabled).
(From OE-Core rev: 98b2b6c3eae15d9b84c97c6a1495040f6b7e389a)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Python uses AC_RUN_IFELSE to determine the byte order for floats and doubles,
and falls back onto "I don't know" if it can't run code. This results in
crippled floating point numbers in Python, and the regression tests fail.
Instead of running code, take a macro from autoconf-archive which compiles C
with a special double in which has an ASCII representation, and then greps the
binary to identify the format.
(From OE-Core rev: 1781b87ae0765c1867da2fa8c56bf988b4e84405)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There's no need to delete the line that removes the profile data, as we're not
using it after the build. This reduces the size of the patch, making it easier
to maintain.
(From OE-Core rev: 561577add4b935d9bfb7b7dd6abdc3e26c1d9143)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a PACKAGECONFIG for profile-guided-optimisation, and default to whether
qemu-usermode is available.
Move --enable-optimizations to the pgo optimisation as all this currently does
is use the PGO rules, causing a performance hit if PGO isn't actually used.
(From OE-Core rev: 231e13c76d3ca7beebfbc8d8a856f72fa3780601)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We're repeating the same make invocation over and over, twice without setting
OPT=${CFLAGS} which doesn't seem right.
Centralise the make invocation to clean up the tasks.
(From OE-Core rev: d2b5bf97eedc67f77d87a44bab86ab086176b8db)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There was a bug where modules were being added twice to the
core package, this patches the manifest to reflect the fix on
the create_manifest script.
(From OE-Core rev: 9ddbff824052751b4d0bb30c453ecb5098d728c9)
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch intends to clean up the whole create_manifest script/task
for python3.
This is an effort to make the code more human friendly and facilitate
adoption, it not only cleans up the code but it also improves comments,
it should also be easier to upgrade the manifest after each python3
upgrade now, with these fixes the transition to python 3.7 should be
seamless.
It fixes a rather harmless bug where module dependencies were being
added twice to the core package and adds tests and sqlite3-tests
as special packages since we want specific dependencies on those.
It also fixes a bug that happened on a few packages that
contained a directory with the same name as the module itself
e.g. asyncio, where the script avoided checking that module for
dependencies.
Lastly, it improves the output, it errors out if a module is found
on more than one package, which is what usually happens when
python upstream introduces a new file, hence why the current
manifest is unaware of, it is better to exit with an error because
otherwise the user wouldnt know if anything went wrong unless the output
of the script was checked manually.
(From OE-Core rev: 658042073cb58c58ac4db4ff13689d1ffd89b72e)
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch improves the create_manifest script by making it
use PYTHON_MAJMIN instead of hard coded paths containing the
version number when looking at the necessary modules for
every package, the script should now be independent of the
python(3) version on which were working
(From OE-Core rev: b94af33b5ffdd62617cf69fca4d99e927447740a)
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently the manifest contains hard coded paths with the version number,
e.g. python3.5, this patch changes the paths to use the variable
PYTHON_MAJMIN instead, this should make the python upgrades easier
(From OE-Core rev: a6fab3ca81e3dc6ee226840ed7e9f1cd21ecbc2a)
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is no good reason to loop 10 times on the test since the profile
optimisation won't change between runs, we don't need/want a statistical
average. This is just burning cpu cycles, just make 1 run.
(From OE-Core rev: 78af3a3e6865cac8bc9fd881af6158ce5e21d4e5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since `6146b8c glibc: Disable crypt support in glibc' in oe-core,
python2/3 could not find symbol crypt which caused import crypt failed.
[snip]
>>> import crypt
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib64/python3.5/crypt.py", line 3, in <module>
import _crypt
ImportError: /usr/lib64/python3.5/lib-dynload/_crypt.cpython-35m-x86_64-linux-gnu.so: undefined symbol: crypt
[snip]
Add virtual/crypt to DEPENDS, and python's build system (setup.py)
will search libcrypt.so in recipe-sysroot and add `-lcrypt' if it
exists.
(From OE-Core rev: d2a929d932900f753af1a06a8cb98d953369e3b4)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
License-Update: Update checksum for copyright year changes
Update to the latest stable version
(From OE-Core rev: 30d0e8e895813bc422d60bd00076a7f16035feeb)
Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Enable profile guided optimization (pgo) for python3. Enabling pgo in
python is generally as simple as invoking the target profile-opt which:
- builds python binaries with profile instrumentation enabled,
- runs a specific profile task using that python to get the profile
data and,
- feeds the compiler with this profile data and rebuilds python.
This change invokes qemu-user for the second step of running a profile
task using target python. Depending on how long profile task takes to
run, this might add a significant time to compilation (which would be
true for native builds too). The default profile task can be changed by
the users depending on what makes sense for their use case (or can be
left empty). In case qemu-user isn't supported, profile task won't be run.
(From OE-Core rev: 05a2a53f9cc7e75b4a3838ab9368cadf0f15ba1b)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The license file changes from README to README.rst, and the lines which
contains license info is from 1 to 20. The license is still Apache-2.0.
(From OE-Core rev: ef60510204013957d3aea3108ccd5fee72825293)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
- It is required by oeqa, add it so that we can add nativesdk-python3-subunit
to buildtools-tarball.
- The original one is python-subunit_1.1.0.bb which is from meta-openstack
layer, I divided it into python-subunit.inc and python3-subunit_1.1.0.bb
and extended to nativesdk.
(From OE-Core rev: f55d5a8022dda18f3e9aa4138dc1961d9ef979c4)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
- It is required by oeqa, add it so that we can add nativesdk-python3-testtools
to buildtools-tarball.
- The original one is python-testtools_2.3.0.bb which is from meta-openstack
layer, I divided it into python-testtools.inc and python3-testtools_2.2.0.bb,
and extended to nativesdk.
(From OE-Core rev: 50864df5b0eb3d1704e8fe75c06e957af0b41c38)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
- The python3-testtools RDEPENDS on it.
- It is from meta-python, I extended it to nativesdk.
(From OE-Core rev: 4cc30de57dd70571005a1f31f9465d745c92aad2)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
- The python3-testtools RDEPENDS on it.
- The original one is python-extras_1.0.0.bb which is from meta-openstack
layer, I divided it into python-extras.inc and python3-extras_1.0.0.bb, and
extended to nativesdk.
(From OE-Core rev: 0176e798364689774bed4ac4bd0e345c32ea3078)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since unify var-DISTUTILS_INSTALL_ARGS in distutils and setuptools,
- The local DISTUTILS_INSTALL_ARGS assignment is obsolete;
- The site.py is not be generated any more;
- The layout is in a standard pip dir (such as /usr/lib/python3.5/site-
packages/pip rather than /lib/python3.5/site-packages/pip-10.0.1-
py3.5.egg/pip), the pth file is not required;
`#!/usr/bin/env python3' is already used, do not manually sed.
[YOCTO #8446]
(From OE-Core rev: 5797631dc7a68e10c848ef06dd9eed011bc06e34)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a python recipe is using setuptools and the setup_requires argument, where
setuptools will use easy_install to fetch the module if it isn't
present.
The build failed on a machine where a proxy was required, but succeeded on a
machine which had direct access to the internet
Add var-NO_FETCH_BUILD, and set it in distutils_do_compile which does not
allow to fetch code from internet during do_compile.
Example result:
...
ERROR: Do not try to fetch `pytest-runner1' for building. Please add its native recipe to DEPENDS.
Traceback (most recent call last):
File "setup.py", line 56, in <module>
...
The improvement is flexible for test_requirements argument (used at
`setup.py test'), where use easy_install also.
[YOCTO #12084]
(From OE-Core rev: 57414f4a1c549e6faaa110e6e95ff601b05b7361)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There's no need to do native- and nativesdk-specific DEPENDS as the
BBCLASSEXTENDs handles those renames for us.
There's no need to have a subset of RDEPENDS for class-native as all with the
new manifest the python-native PROVIDES are complete.
(From OE-Core rev: 3e3c5cc579482041f0233e3e03ace736b62fb364)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
An over-ride replaces the original value regardless of whether or
not it's set up with +=. As replacing the original value seems to be
the intention here, drop the += to make it more explicit. Also some
minor recipe formatting tweaks.
(From OE-Core rev: bda531e65bb6cc9f654c333e2f71283eccd17f27)
Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update the python3-pygobject package to the latest stable version
Tested on the qemu with core-image-minimal
(From OE-Core rev: 3c22f2080ece178b0cc68ae0b74310f81945ea52)
Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update the python{3}-setuptools to the latest stable version
Tested on the qemu with core-image-minimal
(From OE-Core rev: 96971531e0d17a05bfd2b14c95d366f9c499edf9)
Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove the PACKAGECONFIG for documentation as it now requires python3-sphinx which is not
provided in any existing recipe.
(From OE-Core rev: 32dded1bd2f6c5cf6437330830399b72f15096c4)
Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update to the latest stable version
License-Update: Copyright year updated to include 2018
Remove the alignment patch that is included upstream
(From OE-Core rev: 855020053906478cea164ed254c08bedce48479d)
Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
License-Update: Update checksum for whitespace (CRLF) changes
Update to the latest stable version
(From OE-Core rev: 467321703315f6e50aa8bc0e28cfe56f86c5fab7)
Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
python3-git was updated to the latest stable version
Tested on qemux86-64 using core-image-minimal
(From OE-Core rev: 6aa6f57c4d42d45ce1d3dbdca0e8f38ae31c0fbe)
Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update the python{3}-setuptools to the latest stable version
Tested on the qemu with core-image-minimal
(From OE-Core rev: 920dc165abe2484c240b76f95123f5a28f48f9ce)
Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Config file python3.5m-config conflicts between 32 and 64 bit packages.
Use update-alternatives to add base_libdir as suffix to avoid it.
[YOCTO #12511]
(From OE-Core rev: bbb5e55abd0e628a5b0dae90d5eace0a2483f1fc)
Signed-off-by: Zhang Xiao <xiao.zhang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, $HOME/.local is being added into sys.path in the Python SDK
causing subtle host contamination. Suppress this by exporting
PYTHONNOUSERSITE = "1" as documented in PEP 370.
This issue occurred in the past for python*-native and was fixed
similarly in OE-core commit 8fe9fb4d5a61dcbcb3fc5b9ee0234cc135af873f
("python*native.bbclass: suppress user site dirs").
(From OE-Core rev: 0dc36439cb9fe1cea50bed59da6302f78372a30b)
Signed-off-by: Martin Kelly <mkelly@xevo.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Redefiine regen-all in Makefile to invoke regen-importlib after
building other regen- targets. Change the recipe to not build it
before regen-all. This avoids trying to build it multiple times,
which can occasionally fail.
(From OE-Core rev: 72d62c9af07bf34bb8fbb3958742eb592985acc2)
Signed-off-by: Joe Slater <joe.slater@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Using 'basename' to strip the prefix fails when using multiarch style paths.
(From OE-Core rev: c61c416a6504f7e8885df3c94c839d1031920a1c)
Signed-off-by: Koen Kooi <koen.kooi@linaro.org>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update the python{3}-setuptools to the latest stable version
Tested on the qemu with core-image-minimal
(From OE-Core rev: 43b3a293c34e8bfc047bd61a2b4ce3b3586f0d71)
Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We have a problem when for example, a glibc 2.27 based system builds some
library like libpopt-native and puts it into sstate then it is reused
on a pre glibc-2.27 system to build something which depends on popt like
rpm-native. This results in an error like:
recipe-sysroot-native/usr/lib/libpopt.so: undefined reference to `glob@GLIBC_2.27'
In the past we've had this problem with new symbols like getrandom and
getentropy, here its with a more complex symbol where there is an old
version and a newer version.
We've looked into various options, basically we cannot link against our
uninative libc/ld.so since we don't have the right headers or compiler
link libraries. The compiler doesn't allow you to switch in a new set
either, even if we did want to ship them. Shipping a complete compiler,
dev headers and libs also isn't an option.
On the other hand if we follow the ld man page, it does say:
"""
The reasons for allowing undefined symbol references in shared libraries
specified at link time are that:
- A shared library specified at link time may not be the same as the one
that is available at load time, so the symbol might actually be
resolvable at load time.
"""
which is exactly this case. By the time the binary runs, it will use
our uninative loader and libc and the symbol will be available.
Therefore we basically have a choice, we get weird intermittent bugs,
we drop uninative entirely, or we pass this option.
If we pass the option, we can drop the other workarounds too.
(From OE-Core rev: 75a62ede393bf6b4972390ef5290d50add19341a)
(From OE-Core rev: d18bf7fa8e80d6cfaf3fdbe1ab06eec84b954432)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
