There's no need to delete the line that removes the profile data, as we're not
using it after the build. This reduces the size of the patch, making it easier
to maintain.
(From OE-Core rev: 561577add4b935d9bfb7b7dd6abdc3e26c1d9143)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a PACKAGECONFIG for profile-guided-optimisation, and default to whether
qemu-usermode is available.
Move --enable-optimizations to the pgo optimisation as all this currently does
is use the PGO rules, causing a performance hit if PGO isn't actually used.
(From OE-Core rev: 231e13c76d3ca7beebfbc8d8a856f72fa3780601)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We're repeating the same make invocation over and over, twice without setting
OPT=${CFLAGS} which doesn't seem right.
Centralise the make invocation to clean up the tasks.
(From OE-Core rev: d2b5bf97eedc67f77d87a44bab86ab086176b8db)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There was a bug where modules were being added twice to the
core package, this patches the manifest to reflect the fix on
the create_manifest script.
(From OE-Core rev: 9ddbff824052751b4d0bb30c453ecb5098d728c9)
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch intends to clean up the whole create_manifest script/task
for python3.
This is an effort to make the code more human friendly and facilitate
adoption, it not only cleans up the code but it also improves comments,
it should also be easier to upgrade the manifest after each python3
upgrade now, with these fixes the transition to python 3.7 should be
seamless.
It fixes a rather harmless bug where module dependencies were being
added twice to the core package and adds tests and sqlite3-tests
as special packages since we want specific dependencies on those.
It also fixes a bug that happened on a few packages that
contained a directory with the same name as the module itself
e.g. asyncio, where the script avoided checking that module for
dependencies.
Lastly, it improves the output, it errors out if a module is found
on more than one package, which is what usually happens when
python upstream introduces a new file, hence why the current
manifest is unaware of, it is better to exit with an error because
otherwise the user wouldnt know if anything went wrong unless the output
of the script was checked manually.
(From OE-Core rev: 658042073cb58c58ac4db4ff13689d1ffd89b72e)
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch improves the create_manifest script by making it
use PYTHON_MAJMIN instead of hard coded paths containing the
version number when looking at the necessary modules for
every package, the script should now be independent of the
python(3) version on which were working
(From OE-Core rev: b94af33b5ffdd62617cf69fca4d99e927447740a)
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently the manifest contains hard coded paths with the version number,
e.g. python3.5, this patch changes the paths to use the variable
PYTHON_MAJMIN instead, this should make the python upgrades easier
(From OE-Core rev: a6fab3ca81e3dc6ee226840ed7e9f1cd21ecbc2a)
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is no good reason to loop 10 times on the test since the profile
optimisation won't change between runs, we don't need/want a statistical
average. This is just burning cpu cycles, just make 1 run.
(From OE-Core rev: 78af3a3e6865cac8bc9fd881af6158ce5e21d4e5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since `6146b8c glibc: Disable crypt support in glibc' in oe-core,
python2/3 could not find symbol crypt which caused import crypt failed.
[snip]
>>> import crypt
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib64/python3.5/crypt.py", line 3, in <module>
import _crypt
ImportError: /usr/lib64/python3.5/lib-dynload/_crypt.cpython-35m-x86_64-linux-gnu.so: undefined symbol: crypt
[snip]
Add virtual/crypt to DEPENDS, and python's build system (setup.py)
will search libcrypt.so in recipe-sysroot and add `-lcrypt' if it
exists.
(From OE-Core rev: d2a929d932900f753af1a06a8cb98d953369e3b4)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
License-Update: Update checksum for copyright year changes
Update to the latest stable version
(From OE-Core rev: 30d0e8e895813bc422d60bd00076a7f16035feeb)
Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Enable profile guided optimization (pgo) for python3. Enabling pgo in
python is generally as simple as invoking the target profile-opt which:
- builds python binaries with profile instrumentation enabled,
- runs a specific profile task using that python to get the profile
data and,
- feeds the compiler with this profile data and rebuilds python.
This change invokes qemu-user for the second step of running a profile
task using target python. Depending on how long profile task takes to
run, this might add a significant time to compilation (which would be
true for native builds too). The default profile task can be changed by
the users depending on what makes sense for their use case (or can be
left empty). In case qemu-user isn't supported, profile task won't be run.
(From OE-Core rev: 05a2a53f9cc7e75b4a3838ab9368cadf0f15ba1b)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The license file changes from README to README.rst, and the lines which
contains license info is from 1 to 20. The license is still Apache-2.0.
(From OE-Core rev: ef60510204013957d3aea3108ccd5fee72825293)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
- It is required by oeqa, add it so that we can add nativesdk-python3-subunit
to buildtools-tarball.
- The original one is python-subunit_1.1.0.bb which is from meta-openstack
layer, I divided it into python-subunit.inc and python3-subunit_1.1.0.bb
and extended to nativesdk.
(From OE-Core rev: f55d5a8022dda18f3e9aa4138dc1961d9ef979c4)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
- It is required by oeqa, add it so that we can add nativesdk-python3-testtools
to buildtools-tarball.
- The original one is python-testtools_2.3.0.bb which is from meta-openstack
layer, I divided it into python-testtools.inc and python3-testtools_2.2.0.bb,
and extended to nativesdk.
(From OE-Core rev: 50864df5b0eb3d1704e8fe75c06e957af0b41c38)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
- The python3-testtools RDEPENDS on it.
- It is from meta-python, I extended it to nativesdk.
(From OE-Core rev: 4cc30de57dd70571005a1f31f9465d745c92aad2)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
- The python3-testtools RDEPENDS on it.
- The original one is python-extras_1.0.0.bb which is from meta-openstack
layer, I divided it into python-extras.inc and python3-extras_1.0.0.bb, and
extended to nativesdk.
(From OE-Core rev: 0176e798364689774bed4ac4bd0e345c32ea3078)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since unify var-DISTUTILS_INSTALL_ARGS in distutils and setuptools,
- The local DISTUTILS_INSTALL_ARGS assignment is obsolete;
- The site.py is not be generated any more;
- The layout is in a standard pip dir (such as /usr/lib/python3.5/site-
packages/pip rather than /lib/python3.5/site-packages/pip-10.0.1-
py3.5.egg/pip), the pth file is not required;
`#!/usr/bin/env python3' is already used, do not manually sed.
[YOCTO #8446]
(From OE-Core rev: 5797631dc7a68e10c848ef06dd9eed011bc06e34)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a python recipe is using setuptools and the setup_requires argument, where
setuptools will use easy_install to fetch the module if it isn't
present.
The build failed on a machine where a proxy was required, but succeeded on a
machine which had direct access to the internet
Add var-NO_FETCH_BUILD, and set it in distutils_do_compile which does not
allow to fetch code from internet during do_compile.
Example result:
...
ERROR: Do not try to fetch `pytest-runner1' for building. Please add its native recipe to DEPENDS.
Traceback (most recent call last):
File "setup.py", line 56, in <module>
...
The improvement is flexible for test_requirements argument (used at
`setup.py test'), where use easy_install also.
[YOCTO #12084]
(From OE-Core rev: 57414f4a1c549e6faaa110e6e95ff601b05b7361)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There's no need to do native- and nativesdk-specific DEPENDS as the
BBCLASSEXTENDs handles those renames for us.
There's no need to have a subset of RDEPENDS for class-native as all with the
new manifest the python-native PROVIDES are complete.
(From OE-Core rev: 3e3c5cc579482041f0233e3e03ace736b62fb364)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
An over-ride replaces the original value regardless of whether or
not it's set up with +=. As replacing the original value seems to be
the intention here, drop the += to make it more explicit. Also some
minor recipe formatting tweaks.
(From OE-Core rev: bda531e65bb6cc9f654c333e2f71283eccd17f27)
Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update the python3-pygobject package to the latest stable version
Tested on the qemu with core-image-minimal
(From OE-Core rev: 3c22f2080ece178b0cc68ae0b74310f81945ea52)
Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update the python{3}-setuptools to the latest stable version
Tested on the qemu with core-image-minimal
(From OE-Core rev: 96971531e0d17a05bfd2b14c95d366f9c499edf9)
Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove the PACKAGECONFIG for documentation as it now requires python3-sphinx which is not
provided in any existing recipe.
(From OE-Core rev: 32dded1bd2f6c5cf6437330830399b72f15096c4)
Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update to the latest stable version
License-Update: Copyright year updated to include 2018
Remove the alignment patch that is included upstream
(From OE-Core rev: 855020053906478cea164ed254c08bedce48479d)
Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
License-Update: Update checksum for whitespace (CRLF) changes
Update to the latest stable version
(From OE-Core rev: 467321703315f6e50aa8bc0e28cfe56f86c5fab7)
Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
python3-git was updated to the latest stable version
Tested on qemux86-64 using core-image-minimal
(From OE-Core rev: 6aa6f57c4d42d45ce1d3dbdca0e8f38ae31c0fbe)
Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update the python{3}-setuptools to the latest stable version
Tested on the qemu with core-image-minimal
(From OE-Core rev: 920dc165abe2484c240b76f95123f5a28f48f9ce)
Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Config file python3.5m-config conflicts between 32 and 64 bit packages.
Use update-alternatives to add base_libdir as suffix to avoid it.
[YOCTO #12511]
(From OE-Core rev: bbb5e55abd0e628a5b0dae90d5eace0a2483f1fc)
Signed-off-by: Zhang Xiao <xiao.zhang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, $HOME/.local is being added into sys.path in the Python SDK
causing subtle host contamination. Suppress this by exporting
PYTHONNOUSERSITE = "1" as documented in PEP 370.
This issue occurred in the past for python*-native and was fixed
similarly in OE-core commit 8fe9fb4d5a61dcbcb3fc5b9ee0234cc135af873f
("python*native.bbclass: suppress user site dirs").
(From OE-Core rev: 0dc36439cb9fe1cea50bed59da6302f78372a30b)
Signed-off-by: Martin Kelly <mkelly@xevo.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Redefiine regen-all in Makefile to invoke regen-importlib after
building other regen- targets. Change the recipe to not build it
before regen-all. This avoids trying to build it multiple times,
which can occasionally fail.
(From OE-Core rev: 72d62c9af07bf34bb8fbb3958742eb592985acc2)
Signed-off-by: Joe Slater <joe.slater@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Using 'basename' to strip the prefix fails when using multiarch style paths.
(From OE-Core rev: c61c416a6504f7e8885df3c94c839d1031920a1c)
Signed-off-by: Koen Kooi <koen.kooi@linaro.org>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update the python{3}-setuptools to the latest stable version
Tested on the qemu with core-image-minimal
(From OE-Core rev: 43b3a293c34e8bfc047bd61a2b4ce3b3586f0d71)
Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We have a problem when for example, a glibc 2.27 based system builds some
library like libpopt-native and puts it into sstate then it is reused
on a pre glibc-2.27 system to build something which depends on popt like
rpm-native. This results in an error like:
recipe-sysroot-native/usr/lib/libpopt.so: undefined reference to `glob@GLIBC_2.27'
In the past we've had this problem with new symbols like getrandom and
getentropy, here its with a more complex symbol where there is an old
version and a newer version.
We've looked into various options, basically we cannot link against our
uninative libc/ld.so since we don't have the right headers or compiler
link libraries. The compiler doesn't allow you to switch in a new set
either, even if we did want to ship them. Shipping a complete compiler,
dev headers and libs also isn't an option.
On the other hand if we follow the ld man page, it does say:
"""
The reasons for allowing undefined symbol references in shared libraries
specified at link time are that:
- A shared library specified at link time may not be the same as the one
that is available at load time, so the symbol might actually be
resolvable at load time.
"""
which is exactly this case. By the time the binary runs, it will use
our uninative loader and libc and the symbol will be available.
Therefore we basically have a choice, we get weird intermittent bugs,
we drop uninative entirely, or we pass this option.
If we pass the option, we can drop the other workarounds too.
(From OE-Core rev: 75a62ede393bf6b4972390ef5290d50add19341a)
(From OE-Core rev: d18bf7fa8e80d6cfaf3fdbe1ab06eec84b954432)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The python3 installation in the SDK did not include the minimum set
of modules to be functional, particularly in the case where Python
is brought in through dependencies. Rather than requiring the user
to explicitly add the modules, it's better to pull in the modules
through RRECOMMENDS. Note that the Python 2 recipe already does
this.
(From OE-Core rev: 5a88e59e488248b7ec53b072f934052b96c78a51)
Signed-off-by: Tom Hochstein <tom.hochstein@nxp.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Alejandro asked this be reverted as the patch causes more problems
than it solves.
This reverts commit 5d288d286e.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since gnomebase class already inherits autotools and pkgconfig,
there is no need to repeat it here.
(From OE-Core rev: 2eb7ba6cb68e171c880bcb1fc614f1ae6b300e4f)
Signed-off-by: Maxin B. John <maxin.john@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds a couple of prints to get a nicer log, and creates a
small summary or report after checking every module, it
makes it more feasible for adoption, easier to debug why
a module ended at a certain package and see how the
manifest was created.
(From OE-Core rev: 4c2af72f51a7bf187615fc0b3a229d25c3e191e9)
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some of the sqlite3 files ended up in python3-misc incorrectly,
this is caused becuse we couldnt add the whole ${libdir}/python3/sqlite3
folder on the package because we also have another sqlite3-tests
package that needs to include another folder from that directory.
This patch not only fixes the do_create_manifest script to handle this
situation, but also patches the manifest (created using the script)
which also fixes a hiddn runtime dependency that we wouldn't have seen.
(From OE-Core rev: 3324cb31670f33ffe193e550e3b3da8380b3c8c9)
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We have a couple of python modules that contain folders themselves,
for that reason they also contain a __pycache__ folder inside those
directories, since we include the whole folder in the manifest, the
pycache directories end up with the files and not the cache files.
This patch catches that and adds the directories to the correct
structure.
(From OE-Core rev: df9401e7e69ce162e257e827d67eb217666e532d)
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some of the sqlite3 module was in python3-misc by accident, move the files
into python3-sqlite3 where they belong.
(From OE-Core rev: f06a8d20560b8e93cf875ee58eddda0a976e7b14)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
