The connection may have failed before the event queue has been setup.
Handle this correctly in the exception handler.
(Bitbake rev: db4d80b5c2d32117cdf06333b9627202998b1512)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the featureset didn't match the defaults, we'd pay the price of two
base configuration parses which showed up adversely in the performance
benchmarks. This also passes the feature set into the cooker creation so
in the common case we don't have to reset the server. This speeds up
both knotty and hob startup. If the featureset doesn't match, the system
will reset as before, this just streamlines the common case.
(Bitbake rev: 1249543c4dbf3edeac033d888497864cfc807a4e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is no easy way to make this change. We really need parameters for the -S
(dump signatures) handling code. Such a parameter can then be used within the
codebase to handle the signatures in different ways.
For now, "none" is the recommended default and "printdiff" will execute the
new (and more expensive) comparison algorithms.
(Bitbake rev: b9873588696507dfb6aade6821f6f75cb9a19e0a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When passing -t with an incorrect server type, the error message was using a
variable that doesn't exist.
(Bitbake rev: 98b991287df06cd89955c1d0591fce3b5d4403d1)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Toaster start script lunches multiple process components
of the toaster system.
This patch adds logic into the startup script to
safely fail startup and do proper cleanup on any error
that may happen during system start.
Bitbake needs to return 0 if it will successfully lunches
the server-mode.
(Bitbake rev: f43d284e7ae752049711d8215a6020bee6966d45)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The idea is to build on the --status-only option for bitbake and
expose a mechanism where the oe init scripts can easily switch between
memres server and the non-memres server.
In the case of the standard oe init script the following
can shut down the server:
if [ -z "$BBSERVER" ] && [ -f bitbake.lock ] ; then
grep ":" bitbake.lock > /dev/null && BBSERVER=`cat bitbake.lock` bitbake --status-only
if [ $? = 0 ] ; then
echo "Shutting down bitbake memory resident server with bitbake -m"
BBSERVER=`cat bitbake.lock` bitbake -m
fi
fi
A similar function can be used to automatically detect if the server
is already running for the oe memres init script. This new
functionality allows for the memres init script to be started in a new
shell and connect up to an alaready running server without seeing the
error of trying to start the server multiple times.
(Bitbake rev: b1803958de8d7c3c3279841e38604a08dc2316cc)
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch adds the ability to dynamically select a port for the
bitbake memory resident server when the BBSERVER port is set to -1.
This allows for running multiple instances of the bitbake memory
resident server on the same system in different build directories.
The client portion of the bitbake instance can also request that the
server automatically start when using the auto port feature. This is
to deal with a bitbake instance that eventually times out and exits or
that has died for some unknown reason.
The new functionality allows for lazy startup of the server after
sourcing the init script for the memory resident functionality.
(Bitbake rev: d6abc07ff385357d312d8435b89e0a9c1f965433)
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The --status-only option is purely to check the health of the server.
The idea is to use it from the oe environment script so as to know if
the server needs to be started or not.
(Bitbake rev: 46b26ef5c27bfa9c91680f494f02750305b8098c)
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Getting an error message about --remote-server being set when really
BBSERVER was is confusing, clarify the message.
(Bitbake rev: d7b5938a30a9b0ed83f899a06a88786e8392f8bd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When starting a server we don't want to show the debug event queue which
is reserved for when when errors have occurred. This patch copies the UI
code to ensure the user doesn't see confusing output.
(Bitbake rev: a886cda58415085981646fb9a024fa7641f55865)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For BBHandledExceptions, we've already displaced a sensible error to
the user so we don't need to do it again. Just exit with an error
value.
(Bitbake rev: 1ff5ec26eba70ab1c85674a60b7dac77317bf349)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If for example you try "bitbake -m" with an invalid BBSERVER, error
messages are not displayed. This change ensures logging is in place
to catch and display such errors.
(Bitbake rev: 719808f95adc7820fcc09743c592513414d03ce1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The --help text was rather inconsistent in style and plain incorrect in places,
using confusing terminology in others. I guess most people know what the options
do and don't read this but its confusing to new users.
This updates it to use the terms recipe and task consistently, remove
the references to stage, bbread and generally try and make the output
more useful.
[YOCTO #4856]
(Bitbake rev: 516311946c7bd14c84947dc44c3bb0563e5a9667)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Implementing feature set selection that allows a client
to enable specific features in the server at connection time.
Only enabling of features is supported, as there is
no way to safely remove data loaded into the cooker.
Once enabled, a feature will remain enabled for the
life of the cooker.
Client-server connection now supports specifying the feature
set required by the client. This is implemented in the Process
server using a managed proxy list, so the server cooker
will now load dynamically needed features based on what client
connects to it.
In the XMLRPC server the feature set is requested by
using a parameter for registerUIHandler function.
This allows observer-only clients to also specify features
for the server.
The server code configuration now is completly separated
from the client code. All hardcoding of client knowledge is
removed from the server.
The extra_caches is removed as the client can now specify
the caches it needs using the feature. The UI modules
now need to specify the desired featureSet. HOB is modified
to conform to the featureSet specification.
The only feature available is CookerFeatures.HOB_EXTRA_CACHES
which forces loading the bb.cache_extra:HobRecipeInfo class.
(Bitbake rev: 98e594837aab89ea042cfa9f3740d20a661b14e2)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This change allows bitbake to connect to a remote server
by reading the BBSERVER environment variable and effecting
the config. Basically a shortcut to the command line parameters.
(Bitbake rev: 81929f86c57ed0a4ad0cda7aaa820fceabaa61e9)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I add an option to terminate a remote server gracefully
as not to need a kill command.
(Bitbake rev: 7495f835666a9561c2c7d84da7aaa74e4df55b9a)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I add an observer only mode for the knotty UI and
the XMLRPC server that will allow the UI to register
a callback with a server in order to receive events.
The observer-UI is able to send read-only commands to the
server, and also is able to register as an event handler.
Read-only commands are the commands that do not change
the state of the server and have been marked as such in
the command module.
The observer can switch to a full client if it calls addClient
at any time, and the server has no other client running.
(Bitbake rev: 4de9ee21f1fa4d04937cc7430fb1fc8b7a8f61e2)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Two fixes in bitbake related to running remote servers -
* can now specify correctly the bind port
* the information in print conforms to common server infrastructure
(Bitbake rev: b657208ee15ae065e5fcc2dd6e0051e03d246727)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added code in XMLRPC server that creates a stub local server
for a client-only connection and is able to connect to
a remote server, and receive events from the remote server.
Added the option to start a client with a remote server in
bitbake.
Original code by Bogdan Marinescu <bogdan.a.marinescu@intel.com>
(Bitbake rev: 25b2af76104d5aaf6435de8c158e0407512f97ce)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a code sanitization targeted at making further
server-related changes easier (launch a server
separately or creating a mockup-server) to do.
(Bitbake rev: eac00258d213137ef73aed255c92b7981e2f1c75)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The collection of the extra caching data should not
be performed by the cooker, but supplied to it.
This patch will also streamline the code for launching servers
without a UI attached.
Based on a patch by Bogdan Marinescu <bogdan.a.marinescu@intel.com>
(Bitbake rev: f0b54280a6bce522508e4741e5f507bc284113a8)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Slight change in bitbake as to retrieve the entire UI module
for further processing instead of just the main function.
Based on a patch by Bogdan Marinescu <bogdan.a.marinescu@intel.com>
(Bitbake rev: f49341a9599d971829ef65b2b02732543740a3c9)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the removal of the none server type, we can remove the launchUI
method and simplify the code slightly.
(Bitbake rev: 9bef2f2dd0bcaa59528ebcb3c1ce053b7dff1ec6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The process server backend has been serving well as the default for a long
time now and the UI model is much better thought out that it used to be. With
the move to make bitbake a memory resident process, the none server is now
looking rather pointless and complicates the code needlessly. Lets therefore
now remove it.
(Bitbake rev: 9af03a89605e3db9bce3cea1e0f2d0b6cfaa6fe1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As the code stands today its hard to know which configuration variables
are used by which parts of the system. Some are used by the UIs, some
by bin/bitbake itself, some by cooker.
This patch changes the configuration to just contain the variables cooker
uses, and changes bin/bitbake to access the variables it needs directly
which hopefully lets us start to untangle this mess.
(Bitbake rev: e57497a24b6157c92519a34accd66035a39ad1f8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Building up a set of actions for the server is tricky since we depend upon the
commandline but fall back to values from the datastore. We should be able to build
a datastore without a commandline and vice versa. Ultimately the UI should send
the commands to the server.
This patch amounts to code rearranging, moving the heavy lifting to the UI, though
a helper in the configuration option. This will need further cleanup/tweaking but
this should be the only update needed to the UIs. The code now queries the server
for any missing data should it need to.
This code allows various knowledge of configuration variables to move to the UI side
only, partcularly pkgs_to_build but also all the command specifiers. It should also
be possible to move cmd eventually, I'm just unsure if any callers call the commands
expecting this to default to something sane right now.
(Bitbake rev: 2dbbb1d51dafd4451fef8fe16f095bcd4b8f1177)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently the UI and server configuration is one big incestuous mess. To
start to untangle this we creater cookerdata, a new module which contains
various confiuration modules and the code for building the base datastore.
To start with we add a ConfigParameters() class which contains information
about both the commandline configuration and the original environment.
The CookerConfiguration class is created to contain the cooker.configuration
options. This means we can transfer new paramters to the server over something
like XMLRPC and then build a new configuration from these on the server.
Based on a patch from Alexandru Damian <alexandru.damian@intel.com>
(Bitbake rev: 35bd5997e8d8e74bc36019030cc10c560a8134f9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Due to the internal event processing, this excepting handler usually raises an
Empty error, masking the underlying failure. Ensure the original exception is
raised.
(Bitbake rev: 7d548568a55adfe84a976f2a549995e42da1afef)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Python multiprocessing needs /dev/shm, and if it doesn't exist (e.g. in
a chroot environment) or is not writable, you'll get
"OSError: [Errno 38] Function not implemented" or
"OSError: [Errno 13] Permission denied" with a traceback, which doesn't
really help to explain what is wrong.
Implements [YOCTO #4407].
(Bitbake rev: e7460d01f281f913a94192e1f4bd20688164bdd4)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently if errors occur early in the init process, the errors may
not be shown to the user. This change ensures that if a failure does
occur, the messages are flushed from the queue and shown to the user.
(Bitbake rev: fda84c4285e0bc31c249b6dd5464aeb6ad622a9a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In case of an error in bitbake, the stack trace is limited to 5
items. This is an endless source of confusion and it makes bugs
reports impractical, since a full stack trace can't be included
in the bug report. This patch simply removes the depth limit.
(Bitbake rev: 02629c42fb09413d9da16cfe43e03338ce7db3ff)
Signed-off-by: Bogdan Marinescu <bogdan.a.marinescu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need to empty out the environment whilst we build the cooker but
we need the environment for the UIs since hob uses DISPLAY and other
session variables.
This patch adapts the utils functions to return removed environment
components so we can reinject them for use by the UI, allowing hob
to work again.
(Bitbake rev: fc330d810099c57fefd4e706159a73ad8401d97c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The method this older code was accessing doesn't exist. A library shouldn't
be changing behaviour based on the environment anyway when sepcific function
calls exist to correctly setup the logging. Fix the crash if BBDEBUG was used
by moving the code to bin/bitbake.
[YOCTO #3319]
(Bitbake rev: 3a1b75037fe1e5282ffd40b037e93353405faa45)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This code is dead and doesn't do anything so lets remove it.
(Bitbake rev: 8d45739f49618757a5d7d79782deda355e3981ec)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is possible to lose critical log data when python exits in an
unorderly fashion via segmentation fault or certain types of crashes.
This is because the buffer characteristics are inherited from the top
level stdout, which should be set to unbuffered, for the purpose of
all the forked children.
This pushes the buffering to the OS, instead of having python managing
the buffers in its stream handler class.
This change is also to provide the ability to tail logs written from
processes in "real time" because they would be written in an orderly
fashion depending upon the OS characteristics for the file I/O.
(Bitbake rev: c6a367bc3224adafca698a4ffc5414ad83842c16)
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Mainly intended for the purpose of debugging or forcing builds
from source, the --no-setscene will prevent any setscene
tasks from running.
(Bitbake rev: 440e479f3e248482c38c149643403c6907ac7034)
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The '-s' option shows (input) recipes, not (built/output) packages. Clarify
the help wording for this option so it is consistent with how bitbake defines
recipes and packages.
(Bitbake rev: 0cac6d647c58ae449323959220775fc1afd5bfb0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
