We keep seeing hangs on the autobuilder with make 4.2.1 on Centos8, Alma8
and OpenSuse workers. The hang occurs in perl and kernel builds in particular.
The issue is fixed in 4.3 and has been patched on Ubuntu systems:
https://git.savannah.gnu.org/cgit/make.git/commit/?id=78b5fec6898c26956d00548427cda1101cb80f8ahttps://savannah.gnu.org/bugs/?51400
Add a sanity test for make 4.2.1 and error for non-ubuntu systems. We're
making a buildtools-make-tarball available which can be used to
allow systems with the broken version to use the project.
(From OE-Core rev: ad5829aa1f8a7369509542b913bfd8d21d1b1bc3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We're seeing a lot of users configuring an sstate mirror but not realising that
the default hash equivalenve setting will make this ineffective. Add a warning
to highlight this to the user for the common case.
(From OE-Core rev: ae4eb33b5a6a037348e8f66e3d4f19b58dba8fa2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
example.com is proving unreliable at present so switch to our own connectivity
page instead. That page is very simple avoiding app overhead on our web server
which was an original reason for switching to example.com.
(From OE-Core rev: dc6b043cb75c5751b5a98afd2201aa31f9b4b9f6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These were added nearly a decade ago but there are no users in OE-Core. I
checked with the likely users and they seem to have no current usage either.
Therefore remove them.
If needed for some prebuilt library somewhere, they could be implemented
in the layer using them instead but I doubt these are in use any longer.
(From OE-Core rev: 95e196babc3c18dcf0aedfb03e85493c8ae54700)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
distutils is going away and we have functionality in bitbake which can
handle these comparisions so switch to the bb.utils function.
(From OE-Core rev: fe624b520e6c75e16a8f394785ab0216341402f9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
qemu-native 6.1.0 requires at least gcc v7.5:
ERROR: You need at least GCC v7.5 or Clang v6.0 (or XCode Clang v10.0)
(From OE-Core rev: 0137c3da41f8d9328888167fb497539436ead4e9)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The automated conversion of OE-Core to use the new override sytax isn't
perfect. This patches some mis-converted lines and some lines which were missed
by the automation.
(From OE-Core rev: 4e9a06b64b43131b731fb59a0305f78a98e27fbd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is the result of automated script conversion:
scripts/contrib/convert-overrides.py <oe-core directory>
converting the metadata to use ":" as the override character instead of "_".
(From OE-Core rev: 42344347be29f0997cc2f7636d9603b1fe1875ae)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This expands the error message when a network failure is detected.
It happens that some ISPs or networks block the default example.com
domain. Therefore, instead of disabling network access, it
lets the user know how to modify the test URL.
(From OE-Core rev: 62c94bb925543c1e1c5af3c751913d9f06d9597d)
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add tests to ensure COREBASE/TMPDIR doon't contain ".." as this causes
hard to understand build failures.
Also rework the code to test TMPDIR and COREBASE for all the patterns
since they may be set differently and one may contain problematic
characters.
[YOCTO #14111]
(From OE-Core rev: f22a6e46d003aba516a9a0cc7f94eae678d846b7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It has been reported that '%' characters in build paths break with python
exceptions, probably due to confusion with python string escaping. Whilst it
is probably fixable, showing the user a human readable error is better given
it doesn't work.
[YOCTO #14282]
(From OE-Core rev: 31a3cf78452270131a657be45e76569515cff7ef)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added a sanity check for when PSEUDO_IGNORE_PATHS and paths under pseudo control overlap to avoid random failures generated.
[YOCTO #14193]
(From OE-Core rev: 6e4bd8cabcdedf4b52345ef5eb421f71d0f19b1d)
Signed-off-by: Dorinda Bassey <dorindabassey@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This reverts commit c5a6cc4146402620851e2a1f2b01d69989150ba2.
This was accientally added and shouldn't have been, it has a v2 and was then
determined not to be required.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
added a sanity check for when PSEUDO_IGNORE_PATHS and ${S} overlap to avoid random failures generated.
[YOCTO #14193]
(From OE-Core rev: c5a6cc4146402620851e2a1f2b01d69989150ba2)
Signed-off-by: Dorinda Bassey <dorindabassey@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a user executes the environment script instead of sourcing it,
there's an error about an empty element in PATH. This is because
bitbake isn't present in environment variable PATH. Hence, this
patch adds a sanity check to verify if bitbake is present in
PATH and if bitbake isn't present issue a warning message.
[YOCTO #13822]
(From OE-Core rev: e08799913a7f207bc63e085eb98196fd61ed57bc)
Signed-off-by: Dorinda Bassey <dorindabassey@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a user builds in a path in PSEUDO_IGNORE_PATHS, random failures
are generated. Hence this patch adds a sanity check in sanity.bbclass
to ensure that a user isn't building in PSEUDO_IGNORE_PATHS.
[YOCTO #14179]
(From OE-Core rev: 7a681525e904914e938de25df5cc64209097d15d)
Signed-off-by: Dorinda Bassey <dorindabassey@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are a number of reasons 3.6 is a good minimum version. Of our supported/tested
distros, only debian 9 still had python 3.5, the others have 3.6+ or already
required buildtools-tarball.
New versions of qemu need python 3.6 as a minimum. We could work around that
but it seems simper to require 3.6 which will allow other improvements.
As such, bump the minimum python version requirement to 3.6.
(From OE-Core rev: 09385dd8d6be3aac31a4d8b1ca935d4fadfef7ba)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After the recent path ignore changes, there are invalid entries in exiting
pseudo databases which will trip up users with the new abort() (proving
how common the pseudo potential corrpution is!). Inform them a clean TMPDIR
is needed.
(From OE-Core rev: 016ee90e210c9b15b80e8370d83f41a14867a413)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bumps the minimum python version to 3.5 to match bitbake and the test
matrix
(From OE-Core rev: df13c0f2348898023fb7ee1b229e9b5ccc893609)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
'inherit' directive may not be used in conf files as it's supposed
to be used for the inheritance of classes.
Correct form in conf file is INHERIT.
This commit adds:
- a sanity check to find whether the wrong case exists
- fail the build if so
- tell user about the difference in directives
[YOCTO #5426]
(From OE-Core rev: 07bf9b460fe97dec86439302a83bbefa8bac9d70)
Signed-off-by: Gregor Zatko <gzatko@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
New versions of rpm have issues with the libgomp support in gcc 5.x
so raise the minimum to 6 or later. This mainly affects Ubuntu 16.04.
(From OE-Core rev: aca2e5816203b54e0955eaa99fc980d010052d5d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Due to the architectural changes between Windows Subsystem for Linux v2,
and WSL v1 it should now be possible to run bitbake on the several distros
offered through the Microsoft Store.
WSLv2 is available on Windows 10 build number > 18917
The current build number may be checked by opening a cmd prompt on Windows
and running:
C:\Users\myuser>ver
Microsoft Windows [Version 10.0.19041.113]
If a distro has already been installed via the Microsoft Store, then we can
check which WSL version its using by opening a Windows Powershell (notice this
is a powershell and not a cmd prompt):
C:\WINDOWS\system32> wsl -l -v
NAME STATE VERSION
* Ubuntu Running 2
Debian Stopped 1
In this case it shows two distros installed, Ubuntu running WSLv2 and
Debian running WSLv1
To change the version of WSL being used by a certain distro run:
C:\WINDOWS\system32> wsl --set-version <Distro> 2
e.g
C:\WINDOWS\system32> wsl --set-version Debian 2
For more information on installing WSLv2 please look at:
https://docs.microsoft.com/en-us/windows/wsl/wsl2-install
There are some caveats related to the way storage is handled by WSLv2 though,
and at this point these have to be managed by the user manually, the storage
space used by WSL is not reflected immediately and since bitbake heavily uses
storage, after several builds this can prove to be a bit of an issue.
WSLv2 uses a VHDX file for storage, this issue can be easily avoided by
optimizing this file every now and then, this can be done via the following:
1.- Find the location of your VHDX file:
- Get the distro app package directory.
- Open Windows Powershell as Administrator and run:
Get-AppxPackage -Name "*<DISTRO>*" | Select PackageFamilyName
e.g.:
PS C:\WINDOWS\system32> Get-AppxPackage -Name "*Ubuntu*" | Select PackageFamilyName
PackageFamilyName
-----------------
CanonicalGroupLimited.UbuntuonWindows_79abcdefgh
Replace the PackageFamilyName (and your user) on the following path:
C:\Users\<user>\AppData\Local\Packages\<PackageFamilyName>\LocalState\
e.g.
ls C:\Users\<user>\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79abcdefgh\LocalState\
Mode LastWriteTime Length Name
-a---- 3/14/2020 9:52 PM 57418973184 ext4.vhdx
The VHDX file path is:
C:\Users\<user>\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79abcdefgh\LocalState\ext4.vhdx
2.- Optimize your VHDX file (Also on Powershell):
- Make sure WSL is shutdown
wsl --shutdown
- Optimize it
optimize-vhd -Path C:\Users\<user>\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79abcdefgh\LocalState\ext4.vhdx -Mode full
A progress bar should be shown while optimizing the VHDX file.
As an example, after building core-image-sato, removing the TMPDIR did not reflect
any changes on Windows Explorer for storage space being used, after optimizing the
VHDX file, 14 extra GB were shown as free.
So, as long as the the user optimizes its storage, the builds should run smoothly.
This patch warns the user that is running bitbake under WSLv2, that they should
optimize the VHDX file eventually to avoid storage issues.
The same check previoulsy used for WSLv1 works for WSLv2, checking for the kernel
version:
WSLv1:
Linux version 4.4.0-19041-Microsoft (Microsoft@Microsoft.com)
WSLv2:
Linux version 4.19.84-microsoft-standard (oe-user@oe-host)
Builds have been tested under Ubuntu and Debian distros offered and installed through
the Microsoft Store, and other distros should be able to run builds just as fine.
Performance wise, using the same hardware, and same configuration a comparison between
builds using native Linux vs WSLv2 for the following targets has been performed:
- core-image-minimal
- core-image-sato
- core-image-sato-sdk
- meta-toolchain
No real evidence of any performance changes could be found, with WSLv2 builds running even
faster in some cases.
Running a recently built image can be done just as smoothly, if using "nographic" as
argument for runqemu, or if its a graphical image, installing an X server and running
runqemu runs just as fine.
Happy bitbaking.
(From OE-Core rev: c42cec0c1c57c4e67dc7cdb07c5e4aba14a847d3)
Signed-off-by: Alejandro Hernandez Samaniego <alejandro@enedino.org>
Signed-off-by: Alejandro Hernandez Samaniego <alhe@linux.microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It can be frustrating if this sanity check triggers, but you
don't know why; you haven't explicitly set any SDK vars, or
similar.
At least echo out the offending value, so the end user has
a bit more information to go on.
Before:
SDK_VENDOR should be of the form '-foosdk' with a single dash
After:
SDK_VENDOR should be of the form '-foosdk' with a single dash; found '-overc-sdk'
Cc: Ross Burton <ross.burton@intel.com>
(From OE-Core rev: e238fa177bd72bc5d165fbe4f640132267a1d3fd)
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After polling various develoeprs, its clear that many layers are struggling
with gcc 4.8 and its better for the project to adopt 5.0 as a minimum
version at this point in time. We should have technology like
buildtools-extended-tarball available to ensure things still work on Centos 7
and Debian 8.
(From OE-Core rev: abc741af16311cb473b7e3185ae34265b243d804)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is known that old versions of gcc prior to 4.8 causes builds to fail.
Add a test for BUILD_CC == 'gcc' and gcc < 4.8 and recommend using
scripts/install-buildtools or user built buildtools-extended-tarball.
Use the new get_host_compiler_version function from lib/oe/utils.py
NOTE: another solution is to install devtoolset-6+ from scl [1], but
this is a rather large install (> 1 Gb) and fairly invasive.
[1] https://www.softwarecollections.org/en/scls/rhscl/devtoolset-6/
Adding this code means we can increase the minimum version easily
in the future too (which will soon be needed).
RP: Change minimum version from 5.0 to 4.8 for initial patch
(From OE-Core rev: 3bb3b9cbad82b2f09386153226d1d4e769b7347b)
Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For old tar version (< 1.28), recommend using
scripts/install-buildtools
Drop check for tar version 1.24. Dubious extra value.
(From OE-Core rev: 7a66434cf11b7f051699b774e4fccd6738351368)
Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a long standing 'odd' dependency which we've meant to
resolve. We shouldn't need it and it introduces reproducability issues.
We already have texinfo-dummy-native and texinfo-native which can
provide it but the work to remove the hosttool was never completed.
After cleaning up texinfo.bbclass, this can now be removed with
minimal impact on build time.
[YOCTO #13753]
(From OE-Core rev: 0c58c479af151969dfb84d8763696da657f7248d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since this file is written during recipe parsing, having it in the
${BUILDDIR}/conf directory, which is covered by an inotify watcher,
will trigger a re-parse the next time bitbake is run and the resident
bitbake server is enabled. This causes the sanity_info file to be
updated again, which triggers a new parse the next time bitbake is run
ad infinitum. Moving it to ${BUILDDIR}/cache should avoid this.
(From OE-Core rev: f98103b548aa7dba6b1be6c8e02ef41858a8e85c)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Older versions break opkg-build when reproducible builds are enabled.
Rather than trying to be selective based on which features are enabled,
lets just make this a minimum version.
(From OE-Core rev: 0fdc43da005c3c6102cf07383ad6f451d2203fa5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If SDK_VENDOR isn't formatted as -foosdk and is instead for example -foo-sdk
then the triple that are constructed are not in fact triples, which results in
mysterious compile errors.
Check in sanity.bbclass so this failure is detected early.
[ YOCTO #13573 ]
(From OE-Core rev: b0efd8d4d0dbc30e6505b42f5603f18fa764d732)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The pythonexception argument is no more, and passing True is the new behavior.
[ YOCTO #13468 ]
(From OE-Core rev: b7a34d2b8d684e5b98f5c286de67dc1b5d8df853)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix Python3 TypeError error in check_perl_modules:
Executing bitbake, the following error message will be throwed:
File ".../poky/meta/classes/sanity.bbclass", line
979, in check_sanity_eventhandler
check_sanity(sanity_data)
File ".../poky/meta/classes/sanity.bbclass", line
943, in check_sanity
check_sanity_version_change(status, sanity_data)
File ".../poky/meta/classes/sanity.bbclass", line
637, in check_sanity_version_change
status.addresult(check_perl_modules(d))
File ".../poky/meta/classes/sanity.bbclass", line
563, in check_perl_modules
errresult += e.output
TypeError: must be str, not bytes
So here, transfer e.output from bytes to str.
(From OE-Core rev: 2c6fff3fe315357d65d082679856615afc367d90)
Signed-off-by: Zheng Ruoqin <zhengrq.fnst@cn.fujitsu.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If TMPDIR is moved the error message says "move it back or rebuild" but the
obvious rebuild method of running 'bitbake [recipe]] -cclean' fails with the
same error.
Make it clear what we mean by adding "delete and".
(From OE-Core rev: 36805a628f8208ff6d7fba9955c5fb1ed6396395)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Users are starting to expect OE to work under WSL which it doesn't. Add a warning to
tell them about this up front and manage expectations.
(From OE-Core rev: 62fe04c2451f612b9ee5eba469c7724b63484499)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Replace usage of oe.utils.getstatusoutput() with direct subprocess
calls.
(From OE-Core rev: 2f44b9b5babf8c95340b141917c1142081f1e594)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Although get_filesystem_id() is a private API and never gets passed
a path containing spaces or other special characters, etc, quote the
path anyway for consistency.
(From OE-Core rev: 1a9878cdb1cdb807c47e852b780c8ef9b93a214e)
Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We depend on the host GNU patch, but patch < 2.7 can't handle git-style patches.
This results in patches that fail to apply, or worse apply incorrectly.
(From OE-Core rev: ddda57ab1dee989dce8754350471807c916a6f47)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This code is an error path so nobody noticed that oe.utils.getstatusoutput() is
just a wrapper around subprocess.getstatusoutput() which returns an (int,
string) pair not (string, string).
(From OE-Core rev: 33bf6e05af0a68da32f0484460b1de5f7f4eea98)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The core path variables (TMPDIR, DL_DIR, SSTATE_DIR) don't use tilde expansion
but if the user does then the errors are very mysterious, so check on startup.
(From OE-Core rev: 2fb74abbe07b6b82a715ac0fe16449bd8420110e)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Case-insensitive file systems fail during builds in very mysterious ways, such
as mpfr:
ERROR: patch_do_patch: Not a directory
The problem here being that mpfr has a PATCHES file, so when we try to copy the
patches into ${S}/patches/ it fails.
We can't and won't support case-insensitive file systems so add a sanity check
to abort the build if one is found.
(From OE-Core rev: 20ce04fb64f559e64490d53678fa00644a92894a)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
mirrors is a list after split() and results in:
AttributeError: 'list' object has no attribute 'strip'
when the 'mirror values are pairs' check fails.
(From OE-Core rev: 2b7232f2913cc3c8463f136bad7dd06b690c5141)
Signed-off-by: Mikko Ylinen <mikko.ylinen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As gcc 4.5 is very old now (released in 2010, gcc 4.6 released in 2011)
this check can be removed now.
(From OE-Core rev: 78ea1af6bc5d314781be4a3c2d28347312238115)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
qemu-arm and libsdl-native are not in HOSTTOOLS, so there's no point in
checking that they're on PATH.
Also qemu uses pkg-config to find SDL, so libsdl-native isn't required.
(From OE-Core rev: a8d8b0d9e1c2346d5a314ed0a7bf5be66044a51d)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
