For arm we enforce ARM mode regardless of ARM_INSTRUCTION_SET
choice from config metadata, glibc works fine with thumb2 for
armv7+ so limit the restriction to armv5 and lower, tested on
rpi3 works equally well as arm mode glibc and sheds about 0.5MB
in size for main package alone. Other glibc build packages also
gets smaller
ARM: 2696 KiB libc6
Thumb2: 2132 KiB libc6
(From OE-Core rev: a22b97aff811566399765d755daffddb28f82857)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
localedef has no way to specify which locale archive to use, and the
compile-time default isn't useful as it points to the work directory.
Add support to read an environmental variable for the path, and don't fail to
write a new locale archive.
(From OE-Core rev: bf0f205a3c3714926649bd69db29e4df1c0ea112)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
nativesdk is built with a specific prefix but this will be different at install
time, however glibc hard-codes the path to locale files. Expand these strings to 4K and move them to a magic segment which we can relocate when the SDK is installed.
(From OE-Core rev: 59e0679378aac27c4fea0b06721e0a184a93c100)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This will make it easy to backport to rocko if needed after 2.27 is landed in master
plus it fixes the aarch64 build issue seen with binutils 2.30
(From OE-Core rev: 774e372d95c9082766477ea6dbfcd10c48ac4658)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These functions were moved to meta/lib/oe in 2010 and the base_* functions in
utils.bbclass were intended to be a short-term compatibility layer. They're
still used in a few places, so update the callers to use the new functions.
(From OE-Core rev: c97acbd034532895ce57c6717ed1b3ccc7900b0d)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Change hardcoded /lib to ${nonarch_base_libdir} to correctly adapt the
code in do_install_append_aarch64() for when usrmerge is enabled in
DISTRO_FEATURES.
(From OE-Core rev: ac373c9f760463d989d6a1eb3a14b7c5b255b9d4)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
PACKAGES_DYNAMIC contained an identical set of packages twice.
I suspect this was a result of global search/replace when moving
recipes from eglibc to glibc long time ago.
(From OE-Core rev: 9d0f0fa80ffbafc2d4a1df0fa3ac5d84396a8182)
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Building various libraries (libc6, libc6-pic, libc6-staticdev, libc6-dbg, ...)
can be non-deterministic because they may be built with two different versions
of intl/plural.c. in two otherwise identical builds. We may or may not re-generate
the file plural.c from the file plural.y, based on bison being installed or not
and based on mtimes of those two files, as the Makefile contains:
plural.c: plural.y
$(BISON) $(BISONFLAGS) $@ $^
If the above rule does not fire, we use a "fallback" plural.c, otherwise
we use plural.c re-generated from plural.y.
The fix is to always require bison to be installed and unconditionally
re-generate plural.c. (This is achieved by touching plural.y).
[YOCTO #12291]
(From OE-Core rev: b79eeb68c6ad7c92a3e54c6f97a8bf49f0098206)
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Affects glibc < 2.27 including current master
hash 77f921dac17c5fa99bd9e926d926c327982895f7
(From OE-Core rev: a273d099c3bc73736579b7b6ead2572721f16d2a)
Signed-off-by: Huang Qiyu <huangqy.fnst@cn.fujitsu.com>
[v2]
Rebased on new master
[v3]
Fix typo in patch status
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't have /etc/netgroup by default, so do not
cache for netgroup by default to avoid:
nscd[529]: 529 disabled inotify-based monitoring for file `/etc/netgroup': No such file or directory
nscd[529]: 529 stat failed for file `/etc/netgroup'; will try again later: No such file or directory
(From OE-Core rev: 10007bcd30a96470059f9d5b19cf698243486f06)
(From OE-Core rev: 0adedfc2bf8981819fbbf8b1884da44c7082d1a6)
Signed-off-by: Jackie Huang <jackie.huang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* the libc.so.6 dependency is detected always:
$ grep FILERDEPENDS BUILD-*/pkgdata/qemux86/runtime/localedef
BUILD-bad/pkgdata/qemux86/runtime/localedef:FILERDEPENDSFLIST_localedef: /usr/bin/localedef
BUILD-bad/pkgdata/qemux86/runtime/localedef:FILERDEPENDS_/usr/bin/localedef_localedef: libc.so.6(GLIBC_2.15) libc.so.6(GLIBC_2.3) libc.so.6(GLIBC_2.2) libc.so.6(GLIBC_2.1) libc.so.6(GLIBC_2.0) libc.so.6
BUILD-ok/pkgdata/qemux86/runtime/localedef:FILERDEPENDSFLIST_localedef: /usr/bin/localedef
BUILD-ok/pkgdata/qemux86/runtime/localedef:FILERDEPENDS_/usr/bin/localedef_localedef: libc.so.6(GLIBC_2.15) libc.so.6(GLIBC_2.3) libc.so.6(GLIBC_2.2) libc.so.6(GLIBC_2.1) libc.so.6(GLIBC_2.0) libc.so.6
* but in some builds the glibc dependency isn't built soon enough:
$ diff -uNr BUILD-*/pkgdata/qemux86/runtime/localedef
--- BUILD-bad/pkgdata/qemux86/runtime/localedef 2017-09-02 21:17:50.000000000 +0000
+++ BUILD-ok/pkgdata/qemux86/runtime/localedef 2017-09-11 10:15:49.954381592 +0000
@@ -6,6 +6,7 @@
LICENSE: GPLv2 & LGPLv2.1
DESCRIPTION_localedef: glibc: compile locale definition files
SUMMARY: Locale data from glibc
+RDEPENDS_localedef: glibc (>= 2.26)
SECTION: base
PKG_localedef: localedef
FILES_localedef: /usr/bin/localedef
and the build fails with QA issues:
http://errors.yoctoproject.org/Errors/Details/155529/
ERROR: QA Issue: /usr/bin/localedef contained in package localedef requires libc.so.6(GLIBC_2.0), but no providers found in RDEPENDS_localedef? [file-rdeps]
ERROR: QA Issue: /usr/bin/localedef contained in package localedef requires libc.so.6(GLIBC_2.15), but no providers found in RDEPENDS_localedef? [file-rdeps]
ERROR: QA Issue: /usr/bin/localedef contained in package localedef requires libc.so.6(GLIBC_2.3), but no providers found in RDEPENDS_localedef? [file-rdeps]
ERROR: QA Issue: /usr/bin/localedef contained in package localedef requires libc.so.6(GLIBC_2.2), but no providers found in RDEPENDS_localedef? [file-rdeps]
ERROR: QA Issue: /usr/bin/localedef contained in package localedef requires libc.so.6(GLIBC_2.1), but no providers found in RDEPENDS_localedef? [file-rdeps]
ERROR: QA Issue: /usr/bin/localedef contained in package localedef requires libc.so.6, but no providers found in RDEPENDS_localedef? [file-rdeps]
ERROR: QA run found fatal errors. Please consider fixing them.
* easily reproducible with empty TMPDIR and:
bitbake -c cleansstate glibc-locale && bitbake -c package_qa glibc-locale
* reproducible with Yocto 2.2 Morty as well, with slightly different
error message:
ERROR: glibc-locale-2.24-r0 do_package_qa: QA Issue: /usr/bin/localedef contained in package localedef requires libc.so.6(GLIBC_2.4), but no providers found in RDEPENDS_localedef? [file-rdeps]
* with Yocto 2.3 Pyro it's not reproducible by default, because rpmdeps
aren't enabled to populate FILERDEPENDS, but once you add:
ENABLE_RPM_FILEDEPS_FOR_PYRO = "1"
similar QA issue is shown as well:
ERROR: glibc-locale-2.25-r0 do_package_qa: QA Issue: /usr/bin/localedef contained in package localedef requires libc.so.6(GLIBC_2.15), but no providers found in RDEPENDS_localedef? [file-rdeps]
ERROR: glibc-locale-2.25-r0 do_package_qa: QA Issue: /usr/bin/localedef contained in package localedef requires libc.so.6(GLIBC_2.4), but no providers found in RDEPENDS_localedef? [file-rdeps]
ERROR: glibc-locale-2.25-r0 do_package_qa: QA Issue: /usr/bin/localedef contained in package localedef requires libc.so.6, but no providers found in RDEPENDS_localedef? [file-rdeps]
(From OE-Core rev: 2d2b4d7383c93174fe8eeb72440e81345df71295)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Otherwise it will cause conflicts in mutlilib setting, as it
varies from one machine to another.
(From OE-Core rev: 30140990a8bbe920222204b4f19113f5a1662478)
Signed-off-by: Chunrong Guo <chunrong.guo@nxp.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The patch in this Bugzilla entry was requested by a customer:
https://sourceware.org/bugzilla/show_bug.cgi?id=4578https://www.sourceware.org/bugzilla/show_bug.cgi?id=19282
If a thread happens to hold dl_load_lock and have r_state set to RT_ADD or
RT_DELETE at the time another thread calls fork(), then the child exit code
from fork (in nptl/sysdeps/unix/sysv/linux/fork.c in our case) re-initializes
dl_load_lock but does not restore r_state to RT_CONSISTENT. If the child
subsequently requires ld.so functionality before calling exec(), then the
assertion will fire.
The patch acquires dl_load_lock on entry to fork() and releases it on exit
from the parent path. The child path is initialized as currently done.
This is essentially pthreads_atfork, but forced to be first because the
acquisition of dl_load_lock must happen before malloc_atfork is active
to avoid a deadlock.
The __libc_fork() code reset dl_load_lock, but it also needed to reset
dl_load_write_lock.
(From OE-Core rev: f2e586ebf59a9b7d5b216fc92aeb892069a4b0c1)
Signed-off-by: Zhixiong Chi <zhixiong.chi@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
libnsl has been obsoleted in 2.26 and will be removed in future
until them we enable it
(From OE-Core rev: e8047c772f49fe5a4a18d54289d98fb2f98a8176)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Eventually it will be released as 2.26 final
(From OE-Core rev: 5a58883258206893d15990953c8691b05473eecb)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This adds or fixes the Upstream-Status for all remaining patches missing it
in OE-Core.
(From OE-Core rev: 563cab8e823c3fde8ae4785ceaf4d68a5d3e25df)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix a variety of spelling and format mistakes to improve the ease of reading the
tags programatically.
(From OE-Core rev: 6e1aaf80b0d951b48cd25cb7161ec19448295094)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
uclibc support was removed a while ago and musl works much better. Start to
remove the various overrides and patches related to uclibc which are no longer
needed.
uclibc support in a layer would still be possible. I have strong reasons to
believe nobody is still using uclibc since patches are missing and I doubt
the metadata even parses anymore.
(From OE-Core rev: 653704e9cf325cb494eb23facca19e9f05132ffd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you accidentally create symlinks in the glibc sources directory, it
fails in very hard to understand ways. Whilst most people don't do this,
since I debugged it, specifify the list of plugins we use to avoid
this biting anyone else.
(From OE-Core rev: 38a1d4ae4df8dd21546b7442e5a3bddca32a4734)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The path to where to install and find the sysroot components is used
in many places. This warrants it to get its own variable.
(From OE-Core rev: 70a84b525470f72339568409daf84845904e4cab)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We've had a lot of users running into RSS issues where -initial recipes
were being installed into sysroots alongside their counterparts and
causing overlapping files issues.
In general this was through do_build dependencies. Such dependencies are
bad in general and I'd encourage people to compare the taskgraphs with
using a more specific dependency like do_populate_sysroot, do_image_complete
or do_deploy as often the more specific dependency will result in a much
cleaner build.
Regardless, we don't want -initial dependencies getting in the way like
this and there are cases a do_build dependency could make sense.
Deleting the do_build task in these cases makes sense since this is not
a build "endpoint" we'd ever want a user to use, its a behind the scenes
piece of bootstrappping.
Unfortunately to make this work, we need a newer bitbake version which
has a bitbake bug fixed.
(From OE-Core rev: 04c053d42ab05f77b2d1ca93a0fabae44073d57e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With this change (combined with the previous linux-libc-header fix), a
combined sysroot for 32 and 64 bit arm works meaning our SDK works
correctly for that multilib setup.
(From OE-Core rev: 4690cd8e34fc23de10400cc1c178b2c73c7690c7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If base_bindir=bindir, /usr/sbin/ldconfig gets packaged into glibc-utils instead
of glibc: Switch PACKAGES order to prevent this.
(From OE-Core rev: 2794b1cc12af051db404d46aea4db4c74e8e1fe3)
Signed-off-by: Jussi Kukkonen <jussi.kukkonen@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Otherwise it will cause conflicts in mutlilib setting, as it
varies from one machine to another.
(From OE-Core rev: a3b19882e2f50089c785b1e2591550356c12a175)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
USE_LDCONFIG could previously be set to 0 by distros which do not
require ldconfig or ld.so.conf on the target. Since more and more
recipes may need to respect that option, replace the ad-hoc variable
with a distro feature.
Distros which previously set:
USE_LDCONFIG = "0"
Should now instead use:
DISTRO_FEATURES_BACKFILL_CONSIDERED_append = " ldconfig"
(From OE-Core rev: a905df2dd8f43a2febffa64a39b6e508510326a0)
Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that the datastore works dynamically we don't need the update_data calls
so we can just remove them. They're not actually done anything at all for
a while.
(From OE-Core rev: 8de0c5d3bd01919e2bf0394f9c485936d6098cec)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The change to make do_stash_locale an sstate task between do_install and
do_package has some unforeseen problems since the function deletes part of
${D} but may or may not run depending on whether the task is installed from
sstate.
This cleans up the current "pre packaging" function to be more deterministic
and result in the same set of files, whichever code path we end up reaching
that point by. Its not an ideal sitation but it should avoid the race
failures we've seen on some builds.
(From OE-Core rev: 2a60bba00da9c4e2b1a1ce30e509dba82cc6c94a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Putting the locale and script files into the sysroot for use by their
specific recipes used to be a simple way to share the files. With RSS,
we don't want to copy these into many different recipes so put these
files in their own sstate task.
(From OE-Core rev: c283f2532851e2a65e08fe7b3fd71c5d9abb70f2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We used to have issues removing tasks like do_fetch due to implications
for targets like world and universe. These have now been resolved.
Removing uneeded tasks has advantages compared to noexec since it means
that accidentally left in dependencies are no longer needed/processed
(e.g. do_patch depends on quilt-native).
This cleans up a number of cases which local analysis highlighted as
being unneeded leading to slightly cleaner task graphs.
(From OE-Core rev: 4e6ee37e09c60e83c0dfd844ba9cf8a07507f099)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch is comparatively large and invasive. It does only do one thing, switching the
system to build using recipe specific sysroots and where changes could be isolated from it,
that has been done.
With the current single sysroot approach, its possible for software to find things which
aren't in their dependencies. This leads to a determinism problem and is a growing issue in
several of the market segments where OE makes sense. The way to solve this problem for OE is
to have seperate sysroots for each recipe and these will only contain the dependencies for
that recipe.
Its worth noting that this is not task specific sysroots and that OE's dependencies do vary
enormously by task. This did result in some implementation challenges. There is nothing stopping
the implementation of task specific sysroots at some later point based on this work but
that as deemed a bridge too far right now.
Implementation details:
* Rather than installing the sysroot artefacts into a combined sysroots, they are now placed in
TMPDIR/sysroot-components/PACKAGE_ARCH/PN.
* WORKDIR/recipe-sysroot and WORKDIR/recipe-sysroot-native are built by hardlinking in files
from the sysroot-component trees. These new directories are known as RECIPE_SYSROOT and
RECIPE_SYSROOT_NATIVE.
* This construction is primarily done by a new do_prepare_recipe_sysroot task which runs
before do_configure and consists of a call to the extend_recipe_sysroot function.
* Other tasks need things in the sysroot before/after this, e.g. do_patch needs quilt-native
and do_package_write_deb needs dpkg-native. The code therefore inspects the dependencies
for each task and adds extend_recipe_sysroot as a prefunc if it has populate_sysroot
dependencies.
* We have to do a search/replace 'fixme' operation on the files installed into the sysroot to
change hardcoded paths into the correct ones. We create a fixmepath file in the component
directory which lists the files which need this operation.
* Some files have "postinstall" commands which need to run against them, e.g. gdk-pixbuf each
time a new loader is added. These are handled by adding files in bindir with the name
prefixed by "postinst-" and are run in each sysroot as its created if they're present.
This did mean most sstate postinstalls have to be rewritten but there shouldn't be many of them.
* Since a recipe can have multiple tasks and these tasks can run against each other at the same
time we have to have a lock when we perform write operations against the sysroot. We also have
to maintain manifests of what we install against a task checksum of the dependency. If the
checksum changes, we remove its files and then add the new ones.
* The autotools logic for filtering the view of m4 files is no longer needed (and was the model
for the way extend_recipe_sysroot works).
* For autotools, we used to build a combined m4 macros directory which had both the native and
target m4 files. We can no longer do this so we use the target sysroot as the default and add
the native sysroot as an extra backup include path. If we don't do this, we'd have to build
target pkg-config before we could built anything using pkg-config for example (ditto gettext).
Such dependencies would be painful so we haven't required that.
* PKDDATA_DIR was moved out the sysroot and works as before using sstate to build a hybrid copy
for each machine. The paths therefore changed, the behaviour did not.
* The ccache class had to be reworked to function with rss.
* The TCBOOTSTRAP sysroot for compiler bootstrap is no longer needed but the -initial data
does have to be filtered out from the main recipe sysroots. Putting "-initial" in a normal
recipe name therefore remains a bad idea.
* The logic in insane needed tweaks to deal with the new path layout, as did the debug source
file extraction code in package.bbclass.
* The logic in sstate.bbclass had to be rewritten since it previously only performed search and
replace on extracted sstate and we now need this to happen even if the compiled path was
"correct". This in theory could cause a mild performance issue but since the sysroot data
was the main data that needed this and we'd have to do it there regardless with rss, I've opted
just to change the way the class for everything. The built output used to build the sstate output
is now retained and installed rather than deleted.
* The search and replace logic used in sstate objects also seemed weak/incorrect and didn't hold
up against testing. This has been rewritten too. There are some assumptions made about paths, we
save the 'proper' search and replace operations to fixmepath.cmd but then ignore this. What is
here works but is a little hardcoded and an area for future improvement.
* In order to work with eSDK we need a way to build something that looks like the old style sysroot.
"bitbake build-sysroots" will construct such a sysroot based on everything in the components
directory that matches the current MACHINE. It will allow transition of external tools and can
built target or native variants or both. It also supports a clean task. I'd suggest not relying on
this for anything other than transitional purposes though. To see XXX in that sysroot, you'd have
to have built that in a previous bitbake invocation.
* pseudo is run out of its components directory. This is fine as its statically linked.
* The hacks for wayland to see allarch dependencies in the multilib case are no longer needed
and can be dropped.
* wic needed more extensive changes to work with rss and the fixes are in a separate commit series
* Various oe-selftest tweaks were needed since tests did assume the location to binaries and the
combined sysroot in several cases.
* Most missing dependencies this work found have been sent out as separate patches as they were found
but a few tweaks are still included here.
* A late addition is that extend_recipe_sysroot became multilib aware and able to populate multilib
sysroots. I had hoped not to have to add that complexity but the meta-environment recipe forced my
hand. That implementation can probably be neater but this is on the list of things to cleanup later
at this point.
In summary, the impact people will likely see after this change:
* Recipes may fail with missing dependencies, particularly native tools like gettext-native,
glib-2.0-native and libxml2.0-native. Some hosts have these installed and will mask these errors
* Any recipe/class using SSTATEPOSTINSTFUNCS will need that code rewriting into a postinst
* There was a separate patch series dealing with roots postinst native dependency issues. Any postinst
which expects native tools at rootfs time will need to mark that dependency with PACKAGE_WRITE_DEPS.
There could well be other issues. This has been tested repeatedly against our autobuilders and oe-selftest
and issues found have been fixed. We believe at least OE-Core is in good shape but that doesn't mean
we've found all the issues.
Also, the logging is a bit chatty at the moment. It does help if something goes wrong and goes to the
task logfiles, not the console so I've intentionally left this like that for now. We can turn it down
easily enough in due course.
(From OE-Core rev: 809746f56df4b91af014bf6a3f28997d6698ac78)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the file is expected to exist, then we should always be using require
so that if it doesn't we get an error rather than some other more
obscure failure later on.
(From OE-Core rev: 603ae6eb487489e65da69c68e532cb767ccc1fc2)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
