- Drop backported patches
- Move common pieces between cross-localedef and glibc into a common file
- Move latest checksums to glibc-common.inc and remove duplicates from glibc recipe
- Detailed release notes [1]
[1] https://sourceware.org/ml/libc-alpha/2019-08/msg00029.html
(From OE-Core rev: fe75808dca4bb56ac703d18ebbad4004678f69da)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Inherit deploy bbclass and install files to DEPLOYDIR rather than in
DEPLOY_DIR_IMAGE.
(From OE-Core rev: f03ab9b21c2aeeae0bd020ee94ec9bb1d903500d)
Signed-off-by: Ming Liu <liu.ming50@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These comments are not valid any more, drop them.
(From OE-Core rev: 39f5a3030a97bdf567ee11091dd4e95ef04585d7)
Signed-off-by: Ming Liu <liu.ming50@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fathi has as well been inactive for a long time.
(From OE-Core rev: 29ac3e98fdfb7d0d3bb3884b97a0ba5255826904)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
groff has system headers overrides in its sourcecode which it
generates as fallbacks but in some cases when a system header includes
math.h via include <> directive it lands in the override header and
causes compile issues, seen with clang+libc++, this patch makes sure
that right defines are available before including it
(From OE-Core rev: e9beba2a33b46d31bfdd926211d22ebf2abb6c90)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The default works fine; not sure why the line was added.
(From OE-Core rev: 66bb9fd36128bb5ce12c7a5e224cfcd69dc2cd06)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This doesn't appear to be used anymore, as saving a few hundred k at the expense
of only using ASCII is quite the compromise in the modern world.
(From OE-Core rev: 1a502765db2093e573c9e3ecd6c1bc7621476963)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update the metadata to correspond to the bitbake siggen task specification
format change. This standardises on "<fn>:<task>" everywhere rather than
the "." delimiter that was being used in some places.
This is an API breaking change but means we now have a consistent format
being used throughout the codebase without compatibility APIs.
(From OE-Core rev: 23db236a054ee7a989cdbbcb42ad5c6eefd4a6ae)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bitbake has updated to a cleaned up form of BB_HASHCHECK_FUNCTION,
adapt to this cleanup. This is an API breaking change.
(From OE-Core rev: a4d413d8d809132b0e0a5dd673a36e2bd0e0be4c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update the code to handle BB_HASHSERV, an autostarted bitbake internal hash
equivalency server suited to local developer usage.
(From OE-Core rev: 9b3d7b0b0ff27281391c2b8f2511d312d6995ed6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Upstream bitbake now hanes preserving the unihash data itself, drop
this usage of persist_data which was extremely problematic due to
concurrent task access issues, particulary on loaded systems.
(From OE-Core rev: 034d91c2c94b201797a7830b0af6141132f9bad1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the hash server settings are incorrect, show the user useful
error messages instead of tracebacks.
(From OE-Core rev: 2df5d95b9b63e30ddaa7c72a4173b9a05b3f15f9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This code is closely tied with the hash server in bitbake and also means
we can't relibably test the hashserv runqueue functionality without OE
metadata. Moving this to bitbake as a MixIn class makes most sense
and encourages code collaboration and reuse as well as enabling easier
and more accurate testing of the APIs.
(From OE-Core rev: a2a9c6092d4dde706ed071b08a972d1d87184295)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The pythonexception argument is no more, and passing True is the new behavior.
[ YOCTO #13468 ]
(From OE-Core rev: b7a34d2b8d684e5b98f5c286de67dc1b5d8df853)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Whilst seemingly a good idea, this exception doesn't really serve any purpose
that bb.fatal() doesn't cover. Wrapping exceptions within exceptions isn't
pythonic.
Its not used in many places, lets clean up those and remove usage of it
entirely. It may ultimately be dropped form bitbake entirely.
(From OE-Core rev: efe87ce4b2154c6f1c591ed9d8f770c229b044ad)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We depend on a number of bitbake API changes, bump the min bitbake
version requirement.
(From OE-Core rev: 908cdd669b083b172f7cd53dd020629affee360c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a test which tests the runqueue adaptations for hash equivalency.
(Bitbake rev: 477321d0780df177c1582db119c2bb6795912fc6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Its hard to see what this exception adds in the current codebase. The logfile
attribute is effectively ignored, the exception doesn't serve a defined
purpose and mostly seems to be worked around.
Remove it entirely. If this does cause output problems, we'll figure
out better ways to address those.
(Bitbake rev: cfeffb602dd5319f071cd6bcf84139ec77f2d170)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently siggen uses the format "<filename>.<taskname>" for referencing tasks
whilst runqueue uses "<filename>:<taskname>". This converts to use ":" as the
separator everywhere.
This is an API breaking change since the cache is affected, as are siginfo files
and any custom signature handlers such as those in OE-Core.
Ultimately this will let us clean up and the accessor functions from runqueue,
removing all the ".rsplit(".", 1)[0]" type code currently all over the place.
Once a standard is used everwhere we can update the code over time to be more
optimal.
(Bitbake rev: 07e539e1c566ca3434901e1a00335cb76c69d496)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This function uses an old API which uses offsets into lists as a communication
mechanism. Update the API to use "tid" which is used universally in runqueue now.
We can also add kwargs support to the funciton definition to drop some of the
backwards compaiblility hoops we had to jump though with different function
argument combinations.
(Bitbake rev: dc23550047e5078da491ce9a6f30989cb5260df6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There were hard to debug lockups when trying to use threading to start
hashserv as a thread. Switch to multiprocessing which doesn't show the
same locking problems.
(Bitbake rev: be23d887c8e244f1ef961298fbc9214d0fd0968a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Experience with the prserv shows that having two threads, one accepting
and queueing connections and one handling the requests leads to much
more reliable behaviour than having everything in a single thread.
(Bitbake rev: a03d60671a53d9ff70e07cc42fe35f6f8776dac2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We've seen PR Server timeouts on the autobuilder, this is likely from the
journal being blocked on disk IO generated by the build.
Since we're running with synchronous off, we may as well put the journal
into memory and avoid any IO related stalls.
(Bitbake rev: ee3fc6030e653f3244b065fc89aafd2a7c36ae04)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We're seeing performance problems with hashserv running on a normal build
system. The cause seems to be the large amounts of file IO that builds involve
blocking writes to the database. Since sqlite blocks on the sync calls, this
causes a significant problem.
Since if we lose power we have bigger problems, run with synchronous=off
to avoid locking and put the jounral into memory to avoid any write issues
there too.
This took writes from 120s down to negligible in my tests, which means
hashserv then responds promptly to requests.
(Bitbake rev: 7ae56a4d4fcf66e1da1581c70f75e30bfdf3ed83)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Its useful, particularly in the local developer model of usage, for
bitbake to start and stop a hash equivalence server on local port,
rather than relying on one being started by the user before the build.
The new BB_HASHSERVE variable supports this.
The database handling is moved internally into the hashserv code so that
different threads/processes can be used for the server without errors.
(Bitbake rev: a4fa8f1bd88995ae60e10430316fbed63d478587)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Whilst this isn't strictly necessary, its helpful if the log output is
consistent and its also helpful if bugs either appear or don't appear
for a specific configuration. Ensuring the various iterations we make
are deterministic (sorted) helps with this.
(Bitbake rev: 6a901bb904a97ca90d88be2c6901d3d32346282f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is a compelling usecase for tasks being able to notify runqueue
that their "unihash" has changed. When this is recieved, the hashes of
all subsequent tasks should be recomputed and their new hashes checked
against existing setscene validity. Any newly available setscene tasks
should then be executed.
Making this work effectively needs several pieces. An event is added
which the cooker listen for. If a new hash becomes available it can
send an event to notify of this.
When such an event is seen, hash recomputations are made. A setscene
task can't be run until all the tasks it "covers" are stopped. The
notion of "holdoff" tasks is therefore added, these are removed from
the buildable list with the assumption that some setscene task will
run and cover them.
The workers need to be notified when taskhashes change to update their
own internal siggen data stores. A new worker command is added to do this
which will affect all newly spawned worker processes from that worker.
An example workflow which tests this code is:
Configuration:
BB_SIGNATURE_HANDLER = "OEEquivHash"
SSTATE_HASHEQUIV_SERVER = "http://localhost:8686"
$ bitbake-hashserv &
$ bitbake automake-native
$ bitbake autoconf-native automake-native -c clean
$ bitbake m4-native -c install -f
$ bitbake automake-native
with the test being whether automake-native is installed from sstate.
(Bitbake rev: 1f630fdf0260db08541d3ca9f25f852931c19905)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than metadata driven sqlite databases for communication, use
bitbake's unitaskhashes variable instead.
(Bitbake rev: a0d941c787cf3ef030d190903279d311bc05d752)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need to preserve unihash task hashes between runs. Use the new SimpleCache
class to create such a class within the signature generator which is loaded
at init time and saved when builds complete. The default is unpopulated but
metadata sig handlers can populate this cache.
(Bitbake rev: 1f326f2c29c2664a5daaeeb0c1fd332630efbdba)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than a special copy of the data structure which we change, compute
the logic using set operations from other data we have. This means
we can add tasks back into the scenequeue without having to worry about
reversing operations on this variable with all the potential bugs that
might involve.
(Bitbake rev: b707d0cbc25fa336a1e95ff588f1ea37eee063eb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This adds a simple version of the MultiProcessCache which can be used to
save and load cache data, useful for a new usecase we have in
sigdata/runqueue.
(Bitbake rev: 19a6e35600ae6d2d1bcecca6e68ab8c37674774e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This code is closely tied with the hash server in bitbake and also means
we can't relibably test the hashserv runqueue functionality without OE
metadata. Moving this to bitbake as a MixIn class makes most sense
and encourages code collaboration and reuse as well as enabling easier
and more accurate testing of the APIs.
(Bitbake rev: 7bb79099a6c1b463d6ae9226c4cab5e76a965675)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Implements a number of optimizations to the SQL used in the hash
equivalence server:
1) Two indexes are created for the two methods (method, taskhash and
method outhash) by which rows are found in order to speed up the
lookup
2) An extra SELECT to lookup the just inserted row was removed. This
SELECT is unnecessary since all of the information about the newly
inserted row is already available.
3) A uniqueness constraint was added to the table. This should allow
the server to be multithreaded in the future since duplicate inserts
can be detected (and ignored). This change requires bumping the
database version to '2', since a uniqueness constraint can't be
added to an existing table.
4) Some comments are added to clarify the trick SELECT statement used
when inserting new equivalent hashes
(Bitbake rev: 7aec8632e67b4f0ab7b72692c40a42f6926608c3)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When we write to the sstate directory we try to do so atomically so
consumers either see one version or another but never an imcomplete file.
Unfortunately this is reliant on filesystem support and with some NFS
configurations a replaced file would be lost from memory even if users held
open descriptors.
It makes sense to try and avoid replacing existing files where we can.
(From OE-Core rev: 18cdc087fd5da30e2b31f3d4e81b153cd36ca844)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix the build when the host doesn't have libc.a installed by reverting an
upstream commit, and cherry-pick a build fix from master.
Drop a redundant sed, the expression it removes doesn't exist anymore.
Move most patches to libx11.inc so that they're applied to both libx11 and libx11-diet.
(From OE-Core rev: 8bb48ac237315e83837973ecd1488060ec170588)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The generated source file sdksyms.c has a comment with the absolute build path,
which means xserver-xorg-src contains this build path. This is both potential
build information leakage and a source of unreproducibility, so remove the
comment.
(From OE-Core rev: 2086e0f08d920de15ab4065fb43c2281b1dcc57a)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
No modern drivers nor applications use DGA, so remove the client library.
(From OE-Core rev: 2d7e485fcfbc0dc3d76339baf02f50c3c9e22a91)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
DGA (Direct Graphics Access) is an obsolete extension that neither modern X
drivers nor applications support. We'll remove the client library, so remove
the optional dependency on DGA in xdpyinfo.
(From OE-Core rev: b00b1bb1f9394c05268fa10bf0114dc9defc70df)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ideally we'd disable this as no real world client applications actually use DGA
these days, but some drivers (for example fbdev and cirrus) still need DGA
enabled in the server to build.
(From OE-Core rev: e7b1a58a757334d5c73a9b7a8c67e6ead07166c7)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
xorgproto is mentioned in the PACKAGECONFIG build dependencies because in the
past it was many separate *proto recipes. Now they're all in one recipe, which
is in DEPENDS, so we don't need to depend on it several times.
(From OE-Core rev: 1c072b42525864e26d4ab17a64f925ce3803d583)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The extension this client library is for was removed from the X server over a
decade ago:
commit 22e64108ec63ba77779891f8df237913ef9ca731
Author: Daniel Stone <daniel@fooishbar.org>
Date: Wed Jul 16 06:25:26 2008 +0300
XFree86: Remove XFree86-Misc extension
Its last remaining purpose in life has been destroyed by input
properties. Au revoir: it's been fun, by which I mean awful.
Remove the obsolete dependencies from xset and xdpyinfo, and delete libxx86misc
entirely.
(From OE-Core rev: 4d7677b2f511b2d07fabb1f06213a569c53f839f)
Signed-off-by: Ross Burton <ross.burton@intel.com>
enter the commit message for your changes. Lines starting
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The changes include:
- Drop all autotools related patches.
- Move weston-launch setuid-install to do_install task since it's not
supported yet by meson build.
- Drop cairo-glesv2 package config, it's not supported by meson build,
the recommended value is hard-coded to cairo-image for now in weston
source.
- Introduce remoting package config, to give this option a explicit
value, or else the build would run into gstreamer missing error since
it defaults to be true but gstreamer is not in depends list.
- Default to headless backend if neither x11 nor wayland is in
DISTRO_FEATURES.
To be noticed, the packages built from meson would not be identical
with those built from autotools, for instance, libz dependency has been
dropped from meson build, and systemd-notify is always built in meson
when systemd packageconfig is enabled, which in autotools, it's
optional, and so on, it's not easy to list all the differences, but
you could get them by comparing the pkgdata directories built by meson
and autotools.
(From OE-Core rev: 8d68e3cb315137f73d1b0f5f80180c2d3526f99d)
Signed-off-by: Stefan Agner <stefan.agner@toradex.com>
Signed-off-by: Ming Liu <ming.liu@toradex.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
