The following commit to fix [Yocto #13039] causes regression of
the behavior of PREMIRRORS.
"bitbake: fetch2: fix premirror URI when downloadfilename defined"
Take meta-openembedded/meta-networking/recipes-protocols/freediameter/freediameter_1.4.0.bb
as an example.
SRC_URI = "\
http://www.freediameter.net/hg/${fd_pkgname}/archive/${PV}.tar.gz;downloadfilename=${fd_pkgname}-${PV}.tar.gz \
...
"
With the above commit, it now tries to fetch 1.4.0.tar.gz instead of
freeDiameter-1.4.0.tar.gz. This makes https://downloads.yoctoproject.org/mirror/sources
not work for freediameter, as it holds freeDiameter-1.4.0.tar.gz.
The commit above tries to avoid fetching from invalid url such as:
https://<some_mirror>/1.4.0.tar.gz/freeDiameter-1.4.0.tar.gz.
And its solution is to make basename to be 1.4.0.tar.gz, thus causing the
regression.
This patch fixes the above regression. For Yocto #13039, it now tries
to fetch from url: https://<some_mirror>/freeDiameter-1.4.0.tar.gz.
(Bitbake rev: 78949cf3fd31d8a408e93af7e27bcf26ae7942f4)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 96c30007dc0b32eee2b15771daec7948bc9bfd97)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a test case test_fetch_premirror_use_downloadfilename_to_fetch to ensure
that 'downloadfilename' is used when fetching from premirror.
Although the other two previous test cases, test_fetch_premirror_specify_downloadfilename_regex_uri
and test_fetch_premirror_specify_downloadfilename_specific_uri already
implicitly contain such verification, we still need to add a very clear
case to ensure no regression.
(Bitbake rev: 057cbba6b7ade134e4fa3584b9e896be025a6f46)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 20aabc3d53f69949810ecf02295725db947ffef8)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When downloadfilename is specified, it is used to fetch from premirror.
So fix the test cases accordingly.
(Bitbake rev: af573273e4a5b73550af9639da18906f13bfa1a9)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 3b4d2e3b5024324058360a2a28f33c34114218d0)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
github is dropping support for git protocol in Git urls. Add code to remap
this to https in a way that could be used in older bitbake versions.
(Bitbake rev: f19eefdaa5b43460f00d79d002f96112a6aa3c9a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
"make install" may attempt to compile the python modules but it uses the host python
and host paths which means the binaries are not reproducbile. Make things consistent.
If anyone needs compiling, it will beed to be fixed to be cross compile compatible.
(From OE-Core rev: 6ca6c9c12c93c6df7b18f49ebdbfb69433ff5158)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 1189f95e05c80286e009e1ab46a603ee5b7ca239)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
running the 'id' command inside the sstate_create_package
function shows that this funcion run inside the pseudo:
uid=0(root) gid=0(root) groups=0(root)
The check for touch files [ ! -w ${SSTATE_PKG} ]
will always return true and the touch can fail
when the real user don't have permission or
in readonly filesystem.
As the documentation refers, the file test operator "-w"
check if the file has write permission (for the user running the test).
We can avoid this test running the touch and mask any return errors
that we have.
(From OE-Core rev: 1092bb67737eff63c24c26c9f807bec5e6adffc9)
Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit f6e7445c94443544e92fda97a017ce93393c5f84)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We sometimes see exceptions from code seeing the hashserv DB files
being removed at directory cleanup time. Add a check to ensure the
hashserv has written the data base journal (and hence likely exited)
before cleaning up.
This will hopefully avoid errors like:
Traceback (most recent call last):
File "[...]/meta/lib/oeqa/sdk/buildtools-cases/build.py", line 30, in test_libc
delay = delay - 1
File "/usr/lib/python3.6/tempfile.py", line 948, in __exit__
self.cleanup()
File "/usr/lib/python3.6/tempfile.py", line 952, in cleanup
_rmtree(self.name)
File "/usr/lib/python3.6/shutil.py", line 486, in rmtree
_rmtree_safe_fd(fd, path, onerror)
File "/usr/lib/python3.6/shutil.py", line 424, in _rmtree_safe_fd
_rmtree_safe_fd(dirfd, fullname, onerror)
File "/usr/lib/python3.6/shutil.py", line 444, in _rmtree_safe_fd
onerror(os.unlink, fullname, sys.exc_info())
File "/usr/lib/python3.6/shutil.py", line 442, in _rmtree_safe_fd
os.unlink(name, dir_fd=topfd)
FileNotFoundError: [Errno 2] No such file or directory: 'hashserv.db-wal'
(From OE-Core rev: 635833734b4c61e453ca9843a9fb5cecf3eb1c97)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 0b07d9add687d78495176cda0f3011c10ffa4d4b)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are situations when the user have the 'patchdir' defined
as a parameter on SRC_URI. However he doesn't know that with this
the patch is applied relatively to the receipe source dir 'S'.
- When user have 'patchdir' defined check if this directory exist.
- If the patch fails show addition info to the user:
- Import: show the striplevel
- Resolver: show the expanded 'patchdir' to the user.
The next example is from opencv in meta-oe layer, here the
patch is applied on the target directory ${WORKDIR}/git/contrib.
S = "${WORKDIR}/git"
SRCREV_FORMAT = "opencv_contrib"
SRC_URI = "git://github.com/opencv/opencv.git;name=opencv \
git://github.com/opencv/opencv_contrib.git;destsuffix=contrib;name=contrib \
file://0001-sfm-link-with-Glog_LIBS.patch;patchdir=../contrib \
"
* When the patch fail there are no message that indicates the real reason.
patchdir=../no-found-on-file-system
ERROR: opencv-4.5.2-r0 do_patch: Command Error: 'quilt --quiltrc /build/tmp/work/core2-64-poky-linux/opencv/4.5.2-r0/recipe-sysroot-native/etc/quiltrc push' exited with 0 Output:
stdout: Applying patch 0001-sfm-link-with-Glog_LIBS.patch
can't find file to patch at input line 37
Perhaps you used the wrong -p or --strip option?
* The check of the patchdir will add a new fatal error
when the user specifies a wrong path than don't exist.
patchdir=../no-found-on-file-system
ERROR: opencv-4.5.2-r0 do_patch: Target directory '/build/tmp/work/core2-64-poky-linux/opencv/4.5.2-r0/git/../no-found-on-file-system' not found, patchdir '../no-found-on-file-system' is incorrect in patch file '0001-sfm-link-with-Glog_LIBS.patch'
* When we can't aplly the patch but the patchdir exist,
show the expanded patchdir on fatal error.
patchdir=../git
ERROR: opencv-4.5.2-r0 do_patch: Applying patch '0001-sfm-link-with-Glog_LIBS.patch' on target directory '/build/tmp/work/core2-64-poky-linux/opencv/4.5.2-r0/git/../git'
Command Error: 'quilt --quiltrc /build/tmp/work/core2-64-poky-linux/opencv/4.5.2-r0/recipe-sysroot-native/etc/quiltrc push' exited with 0 Output:
stdout: Applying patch 0001-sfm-link-with-Glog_LIBS.patch
can't find file to patch at input line 37
Perhaps you used the wrong -p or --strip option?
(From OE-Core rev: caf21ee38f7a96af6c10e80f9422611e317b29d6)
Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit c44bc7c0fb8b7c2e44dd93607a3bfd9733e1df80)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
DEPENDS here should match what 5.14 is using.
(From OE-Core rev: adc33c4bb8a0f5c542cb1da3b986e89ecea75714)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 899fd41723f41fe0a0cc24373c326b88cb385fe9)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
5.14 changed how the GCC plugins are built, which means they now
depend on both GMP and MPC to be built. We already depend on gmp-native,
so add libmpc-native aswell.
(From OE-Core rev: 0c15ed141ea3b23140d3aa4e6ae17ddee0947f3f)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit f242a6db0757b31c0d4eba5c362f616e1ace14d6)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, installed packages are listed for images in image-info.txt, but
not for SDKs in sdk-info.txt. Add TOOLCHAIN_HOST_TASK and
TOOLCHAIN_TARGET_TASK to the output variables in sdk-info.txt.
Moreover, package output files for the SDK host are empty because
PKGDATA_DIR defaults to the target directory. Fix this bug and create a new
variable called PKGDATA_DIR_SDK which stores the correct path for the SDK
host package data.
(From OE-Core rev: af7b5c664649d2c0d1b23eb1d553080b9d2a7864)
Signed-off-by: Andres Beltran <abeltran@linux.microsoft.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 82e6172c1df378dff4e503aa878501c08937b5bb)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We're seeing pthread being linked sometimes and not others leading to
non-reproducible target binaries. The reason is mixing the native python
config with the target one. We should use the target one.
(From OE-Core rev: 5d27faf68ff94519d6618351ce87a8b3818ba611)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 3fe5101b335384ef83e96ccc58687fd631164075)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We're seeing pthread being linked sometimes and not others leading to
non-reproducible target binaries. The reason is mixing the native python
config with the target one. We should use the target one.
(From OE-Core rev: 0a390b5b36bbd1b2a3aefa74d03e8e40240c68fb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 1bc5378db760963e2ad46542f2907dd6a592eb66)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Recently a number of CVEs have been logged against a nodejs project
called "node-tar". These appear as false positives against the GNU tar
being built by Yocto. Some of these have been manually excluded using
CVE_CHECK_WHITELIST.
To avoid this problem, use the vendor name (in addition to package name)
for filtering CVEs. The syntax for this is:
CVE_PRODUCT = "vendor:package"
When not specified, the vendor defaults to "%" which matches anything.
(From OE-Core rev: d11e970c6e2482ad0b21994e4ec85ddf2aea1ede)
Signed-off-by: Ralph Siemsen <ralph.siemsen@linaro.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 45d1a0bea0c628f84a00d641a4d323491988106f)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We occasionally see bitbake-worker failing and from the logs, an unpickle error
occurs. Add more debug so we can further debug this next time it fails.
[YOCTO #14595]
(Bitbake rev: 692fa35f4c23722f3179502cb965960cc230e709)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit fe8105cc06beca8240b76ea366a1eff5aa9c5412)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We've seen races where the socket may be gone but the server is still writing
out it's database. Handle that case too to avoid cleanup tracebacks.
[YOCTO #14440]
(Bitbake rev: 36b1b4c4fcee9dde628c7113203939730ab12ae5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit b9e4fb843cb9d3a4d4404af093a781fab5520465)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It turns out that for Ubuntu, lz4 is the valid package name for newer
versions (e.g. 20.04 LTS) but not for older ones (e.g. 18.04 LTS, where
the correct package is liblz4-tool). In 20.04 the lz4 package includes
liblz4-tool in its "provides" so it's best to list that one for now.
(From yocto-docs rev: 222af72b9ee307d43a8463283e058c6ebb18fefc)
Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
Reviewed-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
