Running `devtool search gcc` would result in errors like this for
virtual recipes:
ERROR: When reparsing
.../meta/recipes-devtools/gcc/libgcc-initial_13.2.bb:do_populate_sysroot,
the basehash value changed from
b1cd809ed98cef9db0fb1b17d34c4083e739c336f9d5619b89715b0294d81af5 to
44c2f92781dc4a20e98b7bb4724e204e64b101905fa75e71241a574b725997dc. The
metadata is not deterministic and this needs to be fixed.
ERROR: The following commands may help:
ERROR: $ bitbake libgcc-initial -cdo_populate_sysroot -Snone
ERROR: Then:
ERROR: $ bitbake libgcc-initial -cdo_populate_sysroot -Sprintdiff
The reason was the newly introduced :layer-<layername> override, which
is used, e.g., in meta/classes-global/insane.bbclass to add the
patch-status QA test only for the meta layer:
ERROR_QA:append:layer-core = " patch-status"
When tinfoil parsed the recipes using the parseRecipeFile() function, it
failed to properly identify the layername for virtual recipes, which
resulted in the error above.
The correct thing to do is to make parseRecipeFile() call
bb.cache.virtualfn2realfn() to convert the virtual filename into a real
filename and virtual class.
(Bitbake rev: da2aed134412f5de04d7b540f92d735983ad0108)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we expand the variable AUTOREV in OE-Core, it triggers side effects in the
fetcher. The situation isn't ideal and needs improvement but this breaks
and is blocking enabling BB_DEFAULT_EVENTLOG.
Hack around the issue for now so we unblock things until we can work out
a better plan for how to improve AUTOREV support.
(Bitbake rev: cb9b6530f3d12c56a8b48847af2e7461924205d2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently it is only possible to specify an eventlog on the bitbake
commandline. Add a variable that can be used in bitbake.conf so that
we can log data by default more easily.
(Bitbake rev: ee174b231897a53cdde0f68769518342e53210cf)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some of the autobuilder workers have python 3.11 or in the future 3.12.
Add these to allow toaster tests to work on those platforms.
(Bitbake rev: 04d5edd81f5e924370090d68fd4a254abaacbfe4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The event at the end of the function doesn't really belong there,
move it out so the function becomes cleaner. It also avoids having
to pass a number of parameters around, another sign this is the correct
thing to do. This continues refactoring started a while ago to improve
and modularise the code.
(Bitbake rev: f712b927a84e8b8deaa8c907e8f9bcdec681f2aa)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Conceptually, this belongs in sqdata so this is the next step in refactoring
this code, continuing what was started a while ago.
(Bitbake rev: 2cf267cb657cd1864db43955cc010abba7a64093)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Cleanup an extraneous # at the end of the hash bang line.
* Cleanup some trailing whitespace errors.
* Fix typo inf dunfull -> dunfell.
(Bitbake rev: 3938b53f40960ff4a96a55fe5ef35b69737538a3)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When running an asyncrpc server as a subprocess, it is often desired to
run it with a lower logging level since the normal logging of clients
connecting and disconnecting is not desired.
As such, add an option to set the logging level of the server when
running as a subprocess and set the level to WARNING when starting a
local hashserver or PRserver
(Bitbake rev: 61dac7b99ad6d2a858f85d8ed1b5524d558be6c8)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This reverts commit 99305c25fafa61064a1bb0849616e4627e3453d7.
After discussion we found various versions of selenium don't work
with this change.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a user is authenticated with the server, report them as the owner of
a report
(Bitbake rev: a9fd4a45bb6e5ac9832835897f594f3bbf67e1aa)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allows users to self-service deletion of their own user accounts
(meaning, they can delete their own accounts without special
permissions).
(Bitbake rev: 2d4439948a5328a9768bca9eaec221eb82af3cb2)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If BB_TEST_HASHSERV_USERNAME and BB_TEST_HASHSERV_PASSWORD are provided
for a server admin user, the authentication tests for the external
hashserver will run. In addition, any users that get created will now be
deleted when the test finishes.
(Bitbake rev: 0e945d3dec02479df1157f48fd44223c2bfb34a3)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Space separation is more natural when setting the value from an
environment variable, so allow that here for convenience.
(Bitbake rev: 07eb9176f8a7449c1d2cbfff072fa0873e97a336)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Outputting the stats in JSON format makes more sense as it's easier for
a downstream tool to parse if desired.
(Bitbake rev: 3a18066e479ab06bdb08e258fc4aacad5e73222e)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The bitbake-hashclient command-line tool now has a lot more features
which should be tested, so add some tests for them.
(Bitbake rev: 178cf99673d7ddf8e0bb63a5a43331a18f3286d5)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds an API to retrieve the columns that can be queried on from the
database backend. This prevents front end applications from needing to
hardcode the query columns
(Bitbake rev: abfce2b68bdab02ea2e9a63fbb3b9e270428a0a6)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds an API to query the server for the usage of the database (e.g. how
many rows are present in each table)
(Bitbake rev: c9c1224447e147e0de92953bc85cea75670b898c)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds API that allows a user admin to impersonate another user in the
system. This makes it easier to write external services that have
external authentication, since they can use a common user account to
access the server, then impersonate the logged in user.
(Bitbake rev: 71e2f5b52b686f34df364ae1f2fc058f45cd5e18)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds support for the hashserver to have per-user permissions. User
management is done via a new "auth" RPC API where a client can
authenticate itself with the server using a randomly generated token.
The user can then be given permissions to read, report, manage the
database, or manage other users.
In addition to explicit user logins, the server supports anonymous users
which is what all users start as before they make the "auth" RPC call.
Anonymous users can be assigned a set of permissions by the server,
making it unnecessary for users to authenticate to use the server. The
set of Anonymous permissions defines the default behavior of the server,
for example if set to "@read", Anonymous users are unable to report
equivalent hashes with authenticating. Similarly, setting the Anonymous
permissions to "@none" would require authentication for users to perform
any action.
User creation and management is entirely manual (although
bitbake-hashclient is very useful as a front end). There are many
different mechanisms that could be implemented to allow user
self-registration (e.g. OAuth, LDAP, etc.), and implementing these is
outside the scope of the server. Instead, it is recommended to
implement a registration service that validates users against the
necessary service, then adds them as a user in the hash equivalence
server.
(Bitbake rev: 69e5417413ee2414fffaa7dd38057573bac56e35)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds an API to explicitly disconnect a client. This can be useful for
testing the auto-reconnect behavior of clients
(Bitbake rev: cb2fec7eaca20608faf4b8ff11ec8590ac7f2229)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Invalidate the loop in the client close() call so that it is not closed
twice (which is an error in the asyncio code)
(Bitbake rev: ef22f8ee82c242383625f078baafb629e45dad7e)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds support for Invocation Errors (that is, errors raised by the actual
RPC call instead of at the protocol level) to propagate across the
connection. If a server RPC call raises an InvokeError, it will be sent
across the connection and then raised on the client side also. The
connection is still terminated on this error.
(Bitbake rev: 50ee68175e7cf20a32bfbb176db2c47d7859da04)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When the hash equivalence server is in read-only mode, it should still
return a unihash for a given "report" call if there is one.
(Bitbake rev: d0bbb98553f5f3451606bd5f089b36cfe4219dc2)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds an SQLAlchemy backend to the server. While this database backend is
slower than the more direct sqlite backend, it easily supports just
about any SQL server, which is useful for large scale deployments.
(Bitbake rev: e0b73466dd7478c77c82f46879246c1b68b228c0)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Abstracts the way the database backend is accessed by the hash
equivalence server to make it possible to use other backends
(Bitbake rev: 04b53deacf857488408bc82b9890b1e19874b5f1)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allows the arguments to the bitbake-hashserv command to be specified in
environment variables. This is a very common idiom when running services
in containers as it allows the arguments to be specified from different
sources as desired by the service administrator
(Bitbake rev: f65c87fab07a730f2f4588764cb64508c2149b40)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds a logging adaptor to the asyncrpc clients that prefixes log
messages with the client remote address to aid in debugging
(Bitbake rev: f4d64ce73c2449c008ff5d9b32376a2893ef7195)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds support for running the hash equivalence test suite against an
external hash equivalence implementation.
(Bitbake rev: c1fbc3f68b94905d19ffcf4a6da5b27f0bf14599)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds context manager API for the asyncrcp client class which allow
writing code that will automatically close the connection like so:
with hashserv.create_client(address) as client:
...
Rework the bitbake-hashclient tool and PR server to use this new API to
fix warnings about unclosed event loops when exiting
(Bitbake rev: d01d684a0f6398270fe35ed59b7d28f3fd9b7e41)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds support to the hash equivalence client and server to communicate
over websockets. Since websockets are message orientated instead of
stream orientated, and new connection class is needed to handle them.
Note that websocket support does require the 3rd party websockets python
module be installed on the host, but it should not be required unless
websockets are actually being used.
(Bitbake rev: 56dd2fdbfb6350a9eef43a12aa529c8637887a7e)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rewrites the asyncrpc client and server code to make it possible to have
other transport backends that are not stream based (e.g. websockets
which are message based). The connection handling classes are now shared
between both the client and server to make it easier to implement new
transport mechanisms
(Bitbake rev: 2aaeae53696e4c2f13a169830c3b7089cbad6eca)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Test that the followings link are visible and clickable:
- OpenEmbedded
- BitBake
- Yocto Project
- Read the Toaster manual
- Contribute to Toaster
- Fixed typo: using instead of usign line:97
- Remove whitespace
(Bitbake rev: d4c35eaff3c152ee864a886aac8ad5476f7f29f7)
Signed-off-by: Alassane Yattara <alassane.yattara@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
- check that the documentation link is visible
- check browser open new tab toaster manual when clicking on the documentation link
(Bitbake rev: befe970fcc7742cf266e03849be36c733bb09de6)
Signed-off-by: Alassane Yattara <alassane.yattara@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Check the information icon is visible and clickable
Also if popup appearing when clicked and contain "Toaster version information"
(Bitbake rev: 0508fd74ebd43359473a9718edf0d146002041ad)
Signed-off-by: Alassane Yattara <alassane.yattara@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
