Python uses AC_RUN_IFELSE to determine the byte order for floats and doubles,
and falls back onto "I don't know" if it can't run code. This results in
crippled floating point numbers in Python, and the regression tests fail.
Instead of running code, take a macro from autoconf-archive which compiles C
with a special double in which has an ASCII representation, and then greps the
binary to identify the format.
This is essentially a backport of the Python 3 patch in oe-core 1781b87.
(From OE-Core rev: 94cea72a23a374eb616d5642977b45172537beac)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As the manifest handling is done differently now, just inherit ptest with the
other inherits.
test_shutil needs unzip so add to RDEPENDS.
Instead of using a patched Makefile, call test.regrtest directly.
(From OE-Core rev: 84f34ad223b1e3f36cab2ac12246eb90efc919bc)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It's likely that the certificate root chain is needed if python3-crypt is
installed, so recommend it.
Also remove the redundant nativesdk-python3-crypt assignment, as the general
form is sufficient when class-extended.
(From OE-Core rev: dd644ee9991e953474d41dbbf713a7e33b4c9141)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Backport two and pick some other in-review patches from Ubuntu to fix
ssl test failures because of OpenSSL 1.1.x changes.
Fixes [YOCTO #12788]
(From OE-Core rev: 4f4bbb936231dd30c3745ef573993f1062937ffd)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The module was accidentally included in python-misc.
Also re-run create_manifest to update the dependencies of python-sqlite3,
causing some whitespace changes.
[ YOCTO #12933 ]
(From OE-Core rev: 9c2b1a0bfc8783052b5dda344cc334b9c13736f5)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We manually maintain the tests package's content and dependencies, so mark is as
special (matching create_manifest3.py).
(From OE-Core rev: b4bd4e7e26bb79223129abb9fb6e22e9045e3707)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
sqlite3/__init__.py was accidentally included in python3-misc.
(From OE-Core rev: 4e397e16996662bce51608cf5e20448b9803c476)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This package doesn't exist anymore so the manifest tool doesn't need to handle
it specially.
(From OE-Core rev: 30eadb119a64d8561b946d5b8ee30244caeaf134)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently the bulk of the tests in python3-tests, some more in
python3-sqlite3-tests, and others in their parent module (such as
python3-ctypes). This is pointless space usage if we're not planning on running
the tests, so consolidate all the tests into python3-tests.
(From OE-Core rev: 554d3690dab49fee3c93b6b677eb3ef58cff64c1)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Don't sort the manifest when using it to generate packaging rules, so ordering
can be used to have complex packaging rules.
(From OE-Core rev: 80dae6218efd25c92b9c43360e0846bb0af56c7e)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of sorting the entire manifest when it is updated, use OrderedDict to
preserve the order of fields. This means that packages can be ordered in the
manifest to allow non-trivial FILES assignments (such as a package that picks up
pieces of other packages)
The manifest has been regenerated with the new stable ordering, and
distutils-staticdev moved above distutils so the packaging rules work as
expected.
(From OE-Core rev: 1c67c2146e3644a26367a32885d27a4378f17ac6)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently the bulk of the tests in python-tests, some more in
python-sqlite3-tests, and others in their parent module (such as
python-ctypes). This is pointless space usage if we're not planning on
running the tests, so consolidate all the tests into python-tests.
This is a backport of the same changes done by Ross Burton for python3
Changes since v1:
- Rebase
(From OE-Core rev: 0e94737e7124f689c3697d4227bfcd228cc04295)
Signed-off-by: Andrew Geissler <geissonator@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Don't sort the manifest when using it to generate packaging rules, so
ordering can be used to have complex packaging rules.
This is a backport of the same changes done by Ross Burton for python3
Changes since v1:
- Rebase
(From OE-Core rev: 5a5a97b1c3914ad4a3e94981bf53897d0a3f4fb3)
Signed-off-by: Andrew Geissler <geissonator@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of sorting the entire manifest when it is updated, use
OrderedDict to preserve the order of fields.This means that
packages can be ordered in the manifest to allow non-trivial
FILES assignments (such as a package that picks up pieces of
other packages)
The manifest has been regenerated with the new stable ordering, and
distutils-staticdev moved above distutils so the packaging rules work as
expected.
This is a backport of the same changes done by Ross Burton for python3
Changes since v1:
- Moved distutils-staticdev above distutils so packaging rules work
as expected.
Changes since v2:
- Rebase
(From OE-Core rev: 3c62c42ebde9dd4acdc74c56160d6ce8639b497c)
Signed-off-by: Andrew Geissler <geissonator@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Due to human error an older revision of the SSL patch was merged.
(From OE-Core rev: 325af0f4a821971a7aeeca35b10e3558f86029e0)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use variables instead of hard-coding to remove another variation between
releases.
(From OE-Core rev: 6f6b384799bf093fabac90230dcdef1541ea9c75)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of forcibly disabling Bluetooth (support for Bluetooth sockets in the
socket module), add a PACKAGECONFIG. The default remains disabled for
consistency.
(From OE-Core rev: fd5b497cc8a2f9f93f732070123b073bfb6d2eca)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For target Python we use the libffi that we build, so we don't need to
autoreconf the in-tree libffi.
(From OE-Core rev: 12626b26aca281d0d5ee90dc15627083a517fa3b)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
_PYTHON_PROJECT_BASE and _PYTHON_PROJECT_SRC appear to be set in setup.py from
the source directory and build directory correctly, so this is redundant.
(From OE-Core rev: 983206d4ccab2b27adba2776f73c0c711d3ec98e)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of including the Py2 python.inc and having to undo some of the
assignments, create a new python3.inc to hold the common configuration.
Over time we can move more from the recipes into this file to unify the target
and native recipes.
(From OE-Core rev: fc4767113adbdfbf4aeaaf6dd8605e7fd4bbaa46)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
glibc 2.28 slightly changed the behaviour of sigaddset() which broke
multiprocessing. Backport a patch from Python 3.6 to solve this.
(From OE-Core rev: df1874eda517070dab86270ffafd5fe9a307e76b)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch dates back to the addition of the Python 3 recipe to oe-core, and as
listxattr is never added to supports_follow_symlinks the extended attribute
support will never be enabled.
(From OE-Core rev: 9806e141e85b4e4c38eb7b45e6f2fbc2d2aed29d)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
None of the Python invocations that this changes are actually called, and
there's no need to provide a HOSTPGEN variable when the recipe can just override
PGEN directly.
(From OE-Core rev: 47a8602171428b7ce5d897f7e2c2f26b203b8b63)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With OpenSSL 1.1.x TLS 1.3 can be used, so backport a patch from Python 3.6 to
fix the ftplib unit test.
(From OE-Core rev: a31047bec6b7c368674d4620e70e526ac211b936)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
PyBench takes a long time to run, also upstream have removed it from Python and
instead use test.regrtest —pgo to profile the interpreter.
The results are good: not only does Python compile faster (~300s vs ~600s on my
machine) but Phoronix’s PyBench test runs in 2130ms compared to 2229ms when
using PyBench to train (and 2345ms with PGO disabled).
(From OE-Core rev: 98b2b6c3eae15d9b84c97c6a1495040f6b7e389a)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Python uses AC_RUN_IFELSE to determine the byte order for floats and doubles,
and falls back onto "I don't know" if it can't run code. This results in
crippled floating point numbers in Python, and the regression tests fail.
Instead of running code, take a macro from autoconf-archive which compiles C
with a special double in which has an ASCII representation, and then greps the
binary to identify the format.
(From OE-Core rev: 1781b87ae0765c1867da2fa8c56bf988b4e84405)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There's no need to delete the line that removes the profile data, as we're not
using it after the build. This reduces the size of the patch, making it easier
to maintain.
(From OE-Core rev: 561577add4b935d9bfb7b7dd6abdc3e26c1d9143)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a PACKAGECONFIG for profile-guided-optimisation, and default to whether
qemu-usermode is available.
Move --enable-optimizations to the pgo optimisation as all this currently does
is use the PGO rules, causing a performance hit if PGO isn't actually used.
(From OE-Core rev: 231e13c76d3ca7beebfbc8d8a856f72fa3780601)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We're repeating the same make invocation over and over, twice without setting
OPT=${CFLAGS} which doesn't seem right.
Centralise the make invocation to clean up the tasks.
(From OE-Core rev: d2b5bf97eedc67f77d87a44bab86ab086176b8db)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There was a bug where modules were being added twice to the
core package, this patches the manifest to reflect the fix on
the create_manifest script.
(From OE-Core rev: 9ddbff824052751b4d0bb30c453ecb5098d728c9)
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch intends to clean up the whole create_manifest script/task
for python3.
This is an effort to make the code more human friendly and facilitate
adoption, it not only cleans up the code but it also improves comments,
it should also be easier to upgrade the manifest after each python3
upgrade now, with these fixes the transition to python 3.7 should be
seamless.
It fixes a rather harmless bug where module dependencies were being
added twice to the core package and adds tests and sqlite3-tests
as special packages since we want specific dependencies on those.
It also fixes a bug that happened on a few packages that
contained a directory with the same name as the module itself
e.g. asyncio, where the script avoided checking that module for
dependencies.
Lastly, it improves the output, it errors out if a module is found
on more than one package, which is what usually happens when
python upstream introduces a new file, hence why the current
manifest is unaware of, it is better to exit with an error because
otherwise the user wouldnt know if anything went wrong unless the output
of the script was checked manually.
(From OE-Core rev: 658042073cb58c58ac4db4ff13689d1ffd89b72e)
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch improves the create_manifest script by making it
use PYTHON_MAJMIN instead of hard coded paths containing the
version number when looking at the necessary modules for
every package, the script should now be independent of the
python(3) version on which were working
(From OE-Core rev: b94af33b5ffdd62617cf69fca4d99e927447740a)
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently the manifest contains hard coded paths with the version number,
e.g. python3.5, this patch changes the paths to use the variable
PYTHON_MAJMIN instead, this should make the python upgrades easier
(From OE-Core rev: a6fab3ca81e3dc6ee226840ed7e9f1cd21ecbc2a)
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is no good reason to loop 10 times on the test since the profile
optimisation won't change between runs, we don't need/want a statistical
average. This is just burning cpu cycles, just make 1 run.
(From OE-Core rev: 78af3a3e6865cac8bc9fd881af6158ce5e21d4e5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since `6146b8c glibc: Disable crypt support in glibc' in oe-core,
python2/3 could not find symbol crypt which caused import crypt failed.
[snip]
>>> import crypt
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib64/python3.5/crypt.py", line 3, in <module>
import _crypt
ImportError: /usr/lib64/python3.5/lib-dynload/_crypt.cpython-35m-x86_64-linux-gnu.so: undefined symbol: crypt
[snip]
Add virtual/crypt to DEPENDS, and python's build system (setup.py)
will search libcrypt.so in recipe-sysroot and add `-lcrypt' if it
exists.
(From OE-Core rev: d2a929d932900f753af1a06a8cb98d953369e3b4)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
License-Update: Update checksum for copyright year changes
Update to the latest stable version
(From OE-Core rev: 30d0e8e895813bc422d60bd00076a7f16035feeb)
Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Enable profile guided optimization (pgo) for python3. Enabling pgo in
python is generally as simple as invoking the target profile-opt which:
- builds python binaries with profile instrumentation enabled,
- runs a specific profile task using that python to get the profile
data and,
- feeds the compiler with this profile data and rebuilds python.
This change invokes qemu-user for the second step of running a profile
task using target python. Depending on how long profile task takes to
run, this might add a significant time to compilation (which would be
true for native builds too). The default profile task can be changed by
the users depending on what makes sense for their use case (or can be
left empty). In case qemu-user isn't supported, profile task won't be run.
(From OE-Core rev: 05a2a53f9cc7e75b4a3838ab9368cadf0f15ba1b)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The license file changes from README to README.rst, and the lines which
contains license info is from 1 to 20. The license is still Apache-2.0.
(From OE-Core rev: ef60510204013957d3aea3108ccd5fee72825293)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
- It is required by oeqa, add it so that we can add nativesdk-python3-subunit
to buildtools-tarball.
- The original one is python-subunit_1.1.0.bb which is from meta-openstack
layer, I divided it into python-subunit.inc and python3-subunit_1.1.0.bb
and extended to nativesdk.
(From OE-Core rev: f55d5a8022dda18f3e9aa4138dc1961d9ef979c4)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
- It is required by oeqa, add it so that we can add nativesdk-python3-testtools
to buildtools-tarball.
- The original one is python-testtools_2.3.0.bb which is from meta-openstack
layer, I divided it into python-testtools.inc and python3-testtools_2.2.0.bb,
and extended to nativesdk.
(From OE-Core rev: 50864df5b0eb3d1704e8fe75c06e957af0b41c38)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
