Experience with the prserv shows that having two threads, one accepting
and queueing connections and one handling the requests leads to much
more reliable behaviour than having everything in a single thread.
(Bitbake rev: a03d60671a53d9ff70e07cc42fe35f6f8776dac2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We're seeing performance problems with hashserv running on a normal build
system. The cause seems to be the large amounts of file IO that builds involve
blocking writes to the database. Since sqlite blocks on the sync calls, this
causes a significant problem.
Since if we lose power we have bigger problems, run with synchronous=off
to avoid locking and put the jounral into memory to avoid any write issues
there too.
This took writes from 120s down to negligible in my tests, which means
hashserv then responds promptly to requests.
(Bitbake rev: 7ae56a4d4fcf66e1da1581c70f75e30bfdf3ed83)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Its useful, particularly in the local developer model of usage, for
bitbake to start and stop a hash equivalence server on local port,
rather than relying on one being started by the user before the build.
The new BB_HASHSERVE variable supports this.
The database handling is moved internally into the hashserv code so that
different threads/processes can be used for the server without errors.
(Bitbake rev: a4fa8f1bd88995ae60e10430316fbed63d478587)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Implements a number of optimizations to the SQL used in the hash
equivalence server:
1) Two indexes are created for the two methods (method, taskhash and
method outhash) by which rows are found in order to speed up the
lookup
2) An extra SELECT to lookup the just inserted row was removed. This
SELECT is unnecessary since all of the information about the newly
inserted row is already available.
3) A uniqueness constraint was added to the table. This should allow
the server to be multithreaded in the future since duplicate inserts
can be detected (and ignored). This change requires bumping the
database version to '2', since a uniqueness constraint can't be
added to an existing table.
4) Some comments are added to clarify the trick SELECT statement used
when inserting new equivalent hashes
(Bitbake rev: 7aec8632e67b4f0ab7b72692c40a42f6926608c3)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the introduction of SPDX-License-Identifier headers, we don't need a ton
of header boilerplate in every file. Simplify the files and rely on the top
level for the full licence text.
(Bitbake rev: 695d84397b68cc003186e22f395caa378b06bc75)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This adds the SPDX-License-Identifier license headers to the majority of
our source files to make it clearer exactly which license files are under.
The bulk of the files are under GPL v2.0 with one found to be under V2.0
or later, some under MIT and some have dual license. There are some files
which are potentially harder to classify where we've imported upstream code
and those can be handled specifically in later commits.
The COPYING file is replaced with LICENSE.X files which contain the full
license texts.
(Bitbake rev: ff237c33337f4da2ca06c3a2c49699bc26608a6b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Implements a reference implementation of the hash equivalence server.
This server has minimal dependencies (and no dependencies outside of the
standard Python library), and implements the minimum required to be a
conforming hash equivalence server.
[YOCTO #13030]
(Bitbake rev: 1bb2ad0b44b94ee04870bf3f7dac4e663bed6e4d)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
