Add a fix for reproducibility issues where pyc files for python-config.py
may not always be generated.
(From OE-Core rev: 917f800368c6d452670d3ccf74057afae98013b0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit d1c3a87c48b598b6e5624d0affe8bd89320631bf)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Causes build failures on autobuilder
This reverts commit 8a59c47ce4c101b2470a06ecf101ca5ab7d1f82e.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The package python/cpython from 0 and before 3.6.13, from 3.7.0 and before
3.7.10, from 3.8.0 and before 3.8.8, from 3.9.0 and before 3.9.2 are vulnerable
to Web Cache Poisoning via urllib.parse.parse_qsl and urllib.parse.parse_qs by
using a vector called parameter cloaking. When the attacker can separate query
parameters using a semicolon (;), they can cause a difference in the
interpretation of the request between the proxy (running with default
configuration) and the server. This can result in malicious requests being
cached as completely safe ones, as the proxy would usually not see the
semicolon as a separator, and therefore would not include it in a cache key of
an unkeyed parameter.
References:
https://nvd.nist.gov/vuln/detail/CVE-2021-23336https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-23336
(From OE-Core rev: 8a59c47ce4c101b2470a06ecf101ca5ab7d1f82e)
Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Tim Orling <timothy.t.orling@intel.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The pathlib module is for Object-oriented filesystem paths
It also provides a lot of handy utilities for checking on
paths. This seems to justify adding it to the core package
along side os, sys, and the other *path libraries.
[YOCTO #13670]
(From OE-Core rev: 81bec2f08229723b550a0cc33d1c77f82432814d)
Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The original patch "bpo-36852: proper detection of mips architecture
for soft float" uses AC_CANONICAL_TARGET to determine the platform
triplet. While AC_CANONICAL_TARGET exports i686 as target_cpu, gcc
is using i386 instead. We fall back here to i386, as it is conform
to the previous behavior.
Upstream Status: Submitted [https://github.com/python/cpython/pull/13196]
(From OE-Core rev: 6beab388e73b3ac6157650855a6c1fb1d71e8015)
Signed-off-by: Matthias Schoepfer <matthias.schoepfer@ithinx.io>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The specific issue here is rpc/rpc.h, but its likely more general.
/usr/include is searched for rpc/rpc.h and if it exists on the
system, it changes behavior. If you are using the extended buildtools
tarball on a machine that has /usr/include/rpc/rpc.h, it will decide
that is good enough and not continue to search. nis fails to build
because /usr/include and /usr/lib are not part of the include/link
paths for the buildtools tarball compiler(nor should they be).
This makes it so python3-native will not build if you are using the
extended buildtools tarball, but from a larger issue perspective it
is building in likely different ways depending on what machine it
is building on.
libtirpc is already a depend so we shouldn't need the hosts rpc/rcp.h.
(From OE-Core rev: f37dfc7907ae7bac08d40468ddde2e5b8bba030c)
Signed-off-by: Jeremy A. Puhlman <jpuhlman@mvista.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When enable PACKAGECONFIG[tk], we should install _tkinter.*.so to
python3-tkinter package rather than python3-misc package.
Fixes:
ERROR: python3-3.8.1-r0 do_package_qa: QA Issue:
/usr/lib/python3.8/lib-dynload/_tkinter.cpython-38-x86_64-linux-gnu.so
contained in package python3-misc requires libtk8.6.so()(64bit), but no
providers found in RDEPENDS_python3-misc? [file-rdeps]
(From OE-Core rev: 991d7ced7262a1340878bada307c6d021ea9cb77)
Signed-off-by: Yi Zhao <yi.zhao@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Get the sys.lib from python3 itself and do not use
hardcoded value of 'lib' for distutils.
Solve the error below that occurs when run "python3 setup.py
install"
on lib64 multilib platform:
[Errno 2] No such file or directory:
'/usr/lib/python3.7/site-packages/test-easy-install-1828.write-test'
(From OE-Core rev: fb33127de5d80f5bcb84edf6cf4007bca73fa863)
Signed-off-by: Li Zhou <li.zhou@windriver.com>
Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix the installation path of libpython3.7m.a on mulitlib lib64
platform to lib64 instead of lib
(From OE-Core rev: 3081d3993095d2e4817bbef33c9c90dd814a45d7)
Signed-off-by: Li Zhou <li.zhou@windriver.com>
Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Particularly:
[ERROR|FAIL] was matching characters rather than strings.
Using (ERROR|FAIL) requires -r option.
(From OE-Core rev: c041c326a1dbf1b128fc32d887acd73c6f5fb415)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Drop backports, rebase other patches.
0001-main.c-if-OEPYTHON3HOME-is-set-use-instead-of-PYTHON.patch
is removed as the use case (allowing python 2 and 3 to coexist
in SDKs) is no longer relevant with Python 2.x reaching end of line
and upstream has refactored the code making a rebase difficult.
If needed, please re-add the patch to py2, rather than py3.
Python 3.8 no longer adds "m" to "3.8" in paths, so adjust the recipes
and classes accordingly.
The manifest for the 3.8.0 version is updated; particularly pkgutil
module is now packaged in -core (as other things in core need it);
this also necessitates allowing empty -pkgutil package to avoid
breakage across layers.
(From OE-Core rev: e6ab9f16b92aa1abdae82c535c1a452a1341b0e2)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In multilib builds python3-config gets renamed to eg python3-config-lib64 but
this ends up being packaged in python3-core not python3-dev.
The manifest uses an extended glob to package all python* binaries that are not
python-config into python3-core:
"${bindir}/python*[!-config]",
However, this doesn't do what was intended, as [] is a range match.
Replace the globs with more verbose but precise matches, and clear out
FILES_${PN} to ensure that new binaries don't end up in ${PN} (which shouldn't
exist).
[ YOCTO #13592 ]
(From OE-Core rev: c780f21c33b0684dafff3421600cd1c11ddd0c48)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
python3-config uses distutils:
Traceback (most recent call last):
File "/usr/bin/python3-config", line 9, in <module>
from distutils import sysconfig
ModuleNotFoundError: No module named 'distutils'
Add the dependency so that distutils is always present.
[ YOCTO #13592 ]
(From OE-Core rev: 04136dbac48986dce5b2b872b2c0b46c673c44f2)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The use case is building a gpl3-free image, without having
to rely on outdated recipes from meta-gplv2 layer.
(From OE-Core rev: 02eb487c8145e0f3d957c39cf16f6f805e95e536)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The runpy module is used to implement 'python3 -m foo', so move it to
python3-core as it's an essential part of the CLI.
(From OE-Core rev: eba857d3e52f83d426e95fa8373799da058f9484)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Before this patch:
# python3 -m test -v test_locale
[snip]
test_getsetlocale_issue1813 (test.test_locale.TestMiscellaneous) ... testing with ('tr_TR', 'ISO8859-9') ok
[snip]
After this patch:
# python3 -m test -v test_locale
[snip]
test_getsetlocale_issue1813 (test.test_locale.TestMiscellaneous) ... testing with ('tr_TR', 'ISO8859-9')... ok
[snip]
Make the test ended with "... ok" is common in python
unittest world, we should make it keep consistent
with other test cases in case it may be ignored to
record in the report if we use the common filter
"... ok".
[YOCTO #13298]
(From OE-Core rev: 4e698cf8f3a9c661f5b67c6001ad4d92b574d136)
Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Needing the Windows distutils installer stubs is quite a niche requirement, so
put them into a separate package and just recommend it.
This can save both space and legal pain, as the installers embed an old zlib
that has known CVEs.
(From OE-Core rev: 617331a42fc5bde0c2d8d0f4dd8df652daa28778)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
distutils doesn't ship any static libraries, and the files this should pick up
do not exist. Copy/paste gone wrong maybe?
(From OE-Core rev: afbca8da4ddd6c3dd01bb44af43aee9e80cae512)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Applies a patch to python that makes the pre-compiled .pyc files
generated during the build reproducible.
(From OE-Core rev: 148d54f91f43147f31b16a7c2cb1ade385832366)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Reformats the sysconfig file when packaging. This file is output by
using the python pprint function. This function will wrap long lines at
80 characters by default, and will even split strings at whitespace
boundaries to do so, e.g.:
'A': 'B is really'
' long'
This causes a problem for reproducibility however because there might be
lines of differing lengths depending on the build path. These
non-reproducible paths are removed, but their effect on string wrapping
from pprint remains.
To correct this, reformat the entire sysconfig file by re-printing using
pprint with an (effectively) unlimited line length.
(From OE-Core rev: ec8a2b310d5f0b42f60898a5c6d239949842b34c)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When building x86->x86 the system will try to execute .so and related items
from the default PYTHONPATH. This will fail if the target CPU contains
instructions that the host CPU does not have, add CROSSPYTHONPATH
into PYTHONPATH so we can prepend the list to find correct libs.
Fixes:
Illegal instruction (core dumped)
Makefile:625: recipe for target 'sharedmods' failed
make: *** [sharedmods] Error 132
make: *** Waiting for unfinished jobs....
(From OE-Core rev: 2106a567820bad438ff78d54a49e3d87da428dcf)
Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch originally only meant to correct the python3 build for mips
with softfloat, as the original test only checked for mips hardfloat.
Replaced custom C Program for triplet detection with autotools triplet
detection.
(From OE-Core rev: 203336486c84528e8779db93f64cc28e4b372aab)
Signed-off-by: Matthias Schoepfer <matthias.schoepfer@ithinx.io>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using make -j with the 'install' target, it's possible for altbininstall
(which normally creates BINDIR) and libainstall (which doesn't, though it
installs python-config there) to race, resulting in a failure due to
attempting to install python-config into a nonexistent BINDIR. Ensure it also
exists in the libainstall target.
(From OE-Core rev: 07aeaa4f3f88f575e4674145a7faab3ba8e97fad)
Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The newer python3 recipe no longer includes ntpath.py in core, leading
to failures in pkg_resources:
Traceback (most recent call last):
File "/usr/bin/bmaptool", line 6, in <module>
from pkg_resources import load_entry_point
File "/usr/lib/python3.7/site-packages/pkg_resources/__init__.py", line 42, in <module>
import ntpath
ModuleNotFoundError: No module named 'ntpath'
(From OE-Core rev: 9ff61fa83a0a4f2a7b5b0376b6c48fb1173c9ac7)
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the kernel TCP backlog queue coalescing bug fixed in the 5.0 kernels
we don't need this patch anymore and can run the tests.
(From OE-Core rev: 21bfbef23e1f4c90070017e19f70e488e6ff64be)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adding back the python wrapper and adding a patch to use OEPYTHON3HOME
instead of PYTHONHOME if set, for python3.
If we add back the wrapper as is, we would see the following error that
we also see in Thud:
ImportError: No module named site
OpenEmbedded requires 'python' to be python v2 (>= 2.7.3), not python
v3.
Please upgrade your python v2
This is because python3 would've set PYTHONHOME to use nativesdk
python3 libraries but when the oe-buildenv-internal script tries to call
python2 for the py_v27_check, there will be no python2 libraries in the
PYTHONHOME directory.
In other words, bitbake needs host python2 and the env variable set from
the wrapper contaminates the env and host python2 won't be able to find
its libraries
Creating another variable OEPYTHON3HOME and using this in the python3
wrapper to allow for a way to set a different paths for python3 and
python2
[YOCTO #13208]
(From OE-Core rev: 75d2a85e24ef9a2bf0e218521944523f0ff281e0)
Signed-off-by: Jaewon Lee <jaewon.lee@xilinx.com>
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
