Starts the sync thread slightly earlier to give it some extra time to
dump out the caches while the main process tears down the parsing
processes
(Bitbake rev: 105f2897b0618713b036fc0f7a6e0f3e78d1707a)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Uses an event to terminate the parser threads instead of a queue. This
is not only simpler, but saves about 500ms on shutdown time
(Bitbake rev: 2aed34e1d4bf24bba6263f168ff31b55b5fbe982)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Being able to track siggen hash construction data can be useful for cache
debugging. For now, add an extra cache class which contains this information.
It can be enabled in the same way as the hob data cache through a feature flag
to cooker. This allows us to experiment with the data without carrying larger
patches around and ultimately may allow use to have a hash mismatch debugging
mode that is more easily enabled.
(Bitbake rev: 0736a8a03da8b774fafbd28f746bef4705378049)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some parts of functions in Cache() were broken and unused, there was
also a totally unused function. This was historical as a result of the
cooker parsing process needing to handle cached entries in the main
thread but parsing actions in seperate processes.
Document the way it works, update the function name to be clear about
what it now does and drop the old code which was unused.
(Bitbake rev: af83ee32df85c8e4144f022a1f58493eb72cb18e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When 'NoCache' was written, databuilder/cookerdata didn't exist. It does
now and the recipe parsing functionality contained in NoCache clearly
belongs there, it isn't a cache function. Move those functions, renaming
to match the style in databuilder but otherwise not changing functionality
for now. Fix up the callers to match (which make it clear this is the right
move).
(Bitbake rev: 783879319c6a4cf3639fcbf763b964e42f602eca)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
f68682a7 ("logging: Make bitbake logger compatible with python logger")
replaced several .debug() calls to make them comply with the standard
python logging API, but a few were missed.
(Bitbake rev: eb25cd4d64b9a4e8e2b2ce7fbccc123d00b2fc2b)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We have some confusion for users since some classes are meant to work
in the configuration space (or "globally") and some are meant to be
selected by recipes individually.
The cleanest way I could find to clarify this is to create "classes-global"
and "classes-recipe" directories which contain the approproate classes and
have bitbake switch scope between them at the appropriate point during
parsing. The existing "classes" directory is always searched as a fallback.
Once a class is moved to a specific directory, it will no longer be found
in the incorrect context. A good example from OE is that
INHERIT += "testimage"
will no longer work but
IMAGE_CLASSES += "testimage"
will, which makes the global scope cleaner by only including it where it
is useful and intended to be used (images).
(Bitbake rev: f33ce7e742f46635658c400b82558cf822690b5e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As reported by Martin Jansa <Martin.Jansa@gmail.com>:
bitbake/lib/bb/cooker.py:16: DeprecationWarning: module 'sre_constants' is deprecated
import sre_constants
it's deprecated since 3.11 with:
https://github.com/python/cpython/issues/91308
The correct replacement for our usage is re.error so use that instead.
(Bitbake rev: 3c0cd401472ffee06d5a93bdba566cb033851fcf)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The recent inotify changes can cause entire build trees to be monitored
which is suboptimal for performance.
Rather than trying increasingly convoluted tricks to try and handle add/removed
directories, rebuild the inotify watch when we reparse the configuration or
metadata.
(Bitbake rev: 3df322a200c28b45af1f2c92478c85eb7d20c38b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In some cases we'd never have setup a siggen so don't error in that case.
(Bitbake rev: bbaaf2cf7b5a9339d3790610e622020c19d52f5a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ensure any exiting hash server connection is terminated before we start
a new bitbake session. This avoids errors seen with memory resident bitbake
when the asyncio event loop isn't closed correctly.
(Bitbake rev: 42ff9de77f24e2a0bec48a14b64c4b538e00b4af)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When memory resident bitbake is active and we re-parse, the old module
configuration is present which can lead to strange errors. Reset this
when reparsing so the state is consistent. This fixes memory resident
bitbake errors.
(Bitbake rev: 951942c3c284ec2c62e730e145688033190af9b2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The previous fix for inotify wasn't quite correct as we need to modify
bbseen before calling add_filewatch(). We also need to ensure the parse
mtime cache is cleared when directories are added/removed. There was also
a typo in the original fix and the wrong watcher was being changed. Fix
the various issues which improves memory resident bitbake testing results.
(Bitbake rev: 66cadd6be58bce5f7a56556cf92efd8159fb0b0e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The "force" option to parser shutdown was often the cause of lockups and
there is no good reason we should have two different behaviours.
Change and unify the codepaths to always:
* Wait for longer for a controlled shutdown of a process (0.5s). Usually
it will be much faster if it has finished so the delay doesn't really matter.
* Send processes a SIGINT
* Failing that, send a SIGTERM
* Call .close() if available to release zombies
This means we no longer need the "force" parameter to the function so it is removed.
(Bitbake rev: de88c62ef873e9fce78ba162f5311d846de96f2b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a parser process is terminated while holding a write lock, then it
will lead to a deadlock (see
https://docs.python.org/3/library/multiprocessing.html#multiprocessing.Process.terminate).
With SIGTERM, we don't want to terminate holding the lock. We also don't
want a SIGINT to cause a partial write to the event stream.
I tried using signal masks to avoid this but it doesn't work, see
https://bugs.python.org/issue47139
Instead, add a signal handler and catch the calls around the critical section.
We also need a thread lock to ensure other threads in the same process don't
handle the signal until all the threads are not in the lock.
(Bitbake rev: a40efaa5556a188dfe46c8d060adde37dc400dcd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Not sure why this is so convoluted but we should simplify it!
(Bitbake rev: 6195343c46ba9d2685fc2d42366922f88ff3f369)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If an exception occurs in the parsing process, ensure the cleanup
is called via all codepaths using a try/finally.
(Bitbake rev: e1ba2a69f1fc02f01a851bce20b1badf0b991f03)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When parsing, the parser isn't servicing the main loop so a Ctrl+C in the
UI won't be seen on the cooker/server side. Fix this by returning when queue
timeouts occur. This helps where there is a hung or slow parsing thread.
(Bitbake rev: a2cde38311a51112dca5e7bb4e7feaf4e6a281b3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We shouldn't be generating exception inside a generator. Rearrange the
code to improve the handling of this. Also fix the misconverted code
from when multiconfig was added and pass the exception as "result".
(Bitbake rev: ae89e23696de2f27c00ae00922933395171de5d5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Thanks to great debugging from pavel@zhukoff.net we had a simpler reproducer
for the corruption see in oe-selftest when using BB_SERVER_TIMEOUT=60, i.e.
with bitbake in memory resident mode. This was effectively:
oe-selftest -r devtool.DevtoolUpgradeTests.test_devtool_upgrade devtool.DevtoolUpgradeTests.test_devtool_upgrade_git -j 1 -K
The issue is that if directories are removed (such as workspace), if
they are added again, we don't have the watches in place any more. This
patch adds some slightly paranoid checks to ensure we do the correct things
for directory additions and removals (we track directories, not files
specifically to avoid running out of watches).
[YOCTO #14023]
(Bitbake rev: 2c414f659d793d732041614caedd773959eb4f27)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We have shutdown functions within the async client code but the siggen
doesn't currently call them. We notice on python 3.10 (such as on fedora35)
that at exit, there is a stray asyncio process left behind. Usually this
doesn't cause problems but it could potentially be a cause of a hang.
For general cleanliness and completness, add in hooks to call the exit handler.
(Bitbake rev: 9ee3fb95330003878fbd64b3ce8897aad96fcd0f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake -e is not showing variable history correctly for all variables.
The issue was triggered by the change to knotty to default to enabling
variable tracking in the base datastore but that exposed another issue.
The real problem is that calling reset() inside cooker reverts to the
data tracking enabled by the UI for the base datastore, then turns
off tracking. In the case of the environment printing code, it needs
it to be left on.
Tweak the code to ensure data store tracking really is enabled.
The code here is clearly a bit of a mess but this at least fixes a
clear regression until more invasive improvements can be made.
(Bitbake rev: 9596bffc9903f023d1cc97ce531ddb1cf010c4d7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The "Parsing halted due to errors" is shown when user has old-style
variables requiring renaming. Make it clear that it isn't an additional
error, but a result of earlier errors.
(Bitbake rev: e81b6ae61dd8b5d0f77146263d79dfe66c66b060)
Signed-off-by: Marta Rybczynska <marta.rybczynska@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When a recipe raises a BBHandledException, it means the error was already
shown to the user. Adding an additional one here isn't helpful. What is
helpful is to mention that parsing was halted.
Tweak the code to do this with improves the messages the user sees
and helps understand what happened.
(Bitbake rev: fdf6ebc8b603fcfd3ed7c64baf486a4adabd25be)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The right hand side of dependencies in the task dependency file generated
by bitbake -g was missing multiconfig prefixes, corrupting the data. Fix
this.
[YOCTO #14621]
(Bitbake rev: 1d5ca721040c5e39aefa11219f62710de6587701)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The previous fix introduced a race where the queue might not be empty
but all the parser processes have exited. Handle this correctly to avoid
occasional errors.
(Bitbake rev: 8e7f2b6500e26610f52d128b48ca0a09bf6fb2cb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If one of the parse threads disappears during parsing for some reason, bitbake
currently hangs. Avoid this (and zombie threads hanging around) by joining()
threads which have exited.
(Bitbake rev: dc86a533d951d13643ce446533370da804782afc)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When the user specify an invalid upstream hash equivalence server in
BB_HASHSERVE_UPSTREAM notify the user that we can't connect the server.
(Bitbake rev: be45aeb9a84f30c28711e87e2d2a4a86320a8d94)
Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need a command genetating an event to test through the tinfoil API. The
current test has IO load issues so add a dummy version which won't have
the IO constraints.
(Bitbake rev: a144178584394ea0700ffc2f2bfac94e8690effc)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The hash equivalence server has had the option to support a read-only
upstream server for some time now when launched as a standalone program,
but there was no way to set the upstream when using a locally started
server. Add a new variable called BB_HASHSERVE_UPSTREAM that can be used
to specify an upstream server when a local hash equivalence server is
used (e.g. BB_HASHSERVE is "auto")
(Bitbake rev: 250fa17f1391ff1ee01ab9b51d2a4f9aa35c1d1e)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* When using PRSERV_HOST = "localhost:0" inside
Docker container (tested with ubuntu 20.04 and 21.04) the
self.loop.run_until_complete never completed, so self.address
wasn't ever assigned few lines bellow and then
self.port = int(self.prserv.address.rsplit(':', 1)[1])
in lib/prserv/serv.py caused a bit ugly exception:
bitbake@599696cd20aa:~/nodistro/honister$ bitbake -k pkgconfig-native
Traceback (most recent call last):
File "/OE/nodistro/honister/bitbake/bin/bitbake", line 35, in <module>
sys.exit(bitbake_main(BitBakeConfigParameters(sys.argv),
File "/OE/nodistro/honister/bitbake/lib/bb/main.py", line 385, in bitbake_main
return ui_module.main(server_connection.connection, server_connection.events,
File "/OE/nodistro/honister/bitbake/lib/bb/ui/knotty.py", line 397, in main
params.updateToServer(server, os.environ.copy())
File "/OE/nodistro/honister/bitbake/lib/bb/cookerdata.py", line 75, in updateToServer
raise Exception("Unable to update the server configuration with local parameters: %s" % error)
Exception: Unable to update the server configuration with local parameters: Traceback (most recent call last):
File "/OE/nodistro/honister/bitbake/lib/bb/command.py", line 90, in runCommand
result = command_method(self, commandline)
File "/OE/nodistro/honister/bitbake/lib/bb/command.py", line 286, in updateConfig
command.cooker.updateConfigOpts(options, environment, cmdline)
File "/OE/nodistro/honister/bitbake/lib/bb/cooker.py", line 491, in updateConfigOpts
self.reset()
File "/OE/nodistro/honister/bitbake/lib/bb/cooker.py", line 1717, in reset
self.handlePRServ()
File "/OE/nodistro/honister/bitbake/lib/bb/cooker.py", line 383, in handlePRServ
self.prhost = prserv.serv.auto_start(self.data)
File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 318, in auto_start
singleton.start()
File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 133, in start
self.port = int(self.prserv.address.rsplit(':', 1)[1])
AttributeError: 'NoneType' object has no attribute 'rsplit'
* the issue was caused by "localhost" being resolved as IPv6 address ::1
and then asyncio failing to bind it, the same is reproducible with hashserv, but
hashserve at least shows nice error message:
bitbake$ bitbake-hashserv -l DEBUG -b localhost:0
Traceback (most recent call last):
File "/OE/nodistro/honister/bitbake/bin/bitbake-hashserv", line 59, in <module>
ret = main()
File "/OE/nodistro/honister/bitbake/bin/bitbake-hashserv", line 53, in main
server.serve_forever()
File "/OE/nodistro/honister/bitbake/lib/bb/asyncrpc/serv.py", line 233, in serve_forever
self.start()
File "/OE/nodistro/honister/bitbake/lib/bb/asyncrpc/serv.py", line 144, in start_tcp
self.server = self.loop.run_until_complete(server_coro)
File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
return future.result()
File "/usr/lib/python3.8/asyncio/streams.py", line 94, in start_server
return await loop.create_server(factory, host, port, **kwds)
File "/usr/lib/python3.8/asyncio/base_events.py", line 1463, in create_server
raise OSError(err.errno, 'error while attempting '
OSError: [Errno 99] error while attempting to bind on address ('::1', 0, 0, 0): cannot assign requested address
* or by bitbake-prserv in prserv.log:
bitbake$ bitbake-prserv --start --host=localhost --port=42005
bitbake$ cat prserv.log
Traceback (most recent call last):
File "/OE/nodistro/honister/bitbake/bin/bitbake-prserv", line 55, in <module>
ret = main()
File "/OE/nodistro/honister/bitbake/bin/bitbake-prserv", line 46, in main
ret=prserv.serv.start_daemon(options.dbfile, options.host, options.port,os.path.abspath(options.logfile), options.read_only)
File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 226, in start_daemon
run_as_daemon(daemon_main, pidfile, os.path.abspath(logfile))
File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 202, in run_as_daemon
func()
File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 224, in daemon_main
server.serve_forever()
File "/OE/nodistro/honister/bitbake/lib/bb/asyncrpc/serv.py", line 233, in serve_forever
self.start()
File "/OE/nodistro/honister/bitbake/lib/bb/asyncrpc/serv.py", line 144, in start_tcp
self.server = self.loop.run_until_complete(server_coro)
File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
return future.result()
File "/usr/lib/python3.8/asyncio/streams.py", line 94, in start_server
return await loop.create_server(factory, host, port, **kwds)
File "/usr/lib/python3.8/asyncio/base_events.py", line 1463, in create_server
raise OSError(err.errno, 'error while attempting '
OSError: [Errno 99] error while attempting to bind on address ('::1', 42005, 0, 0): cannot assign requested address
* while 127.0.0.1 works fine:
bitbake$ bitbake-prserv --start --host=127.0.0.1 --port=42005
bitbake$ cat prserv.log
DEBUG: Listening on ('127.0.0.1', 42005)
2021-08-26 22:28:05,828 Listening on ('127.0.0.1', 42005)
DEBUG: Opening PRServ database 'file:/OE/nodistro/honister/prserv.sqlite3'
2021-08-26 22:28:05,829 Opening PRServ database 'file:/OE/nodistro/honister/prserv.sqlite3'
NOTE: Started PRServer with DBfile: /OE/nodistro/honister/prserv.sqlite3, Address: 127.0.0.1:42005, PID: 39
2021-08-26 22:28:05,831 Started PRServer with DBfile: /OE/nodistro/honister/prserv.sqlite3, Address: 127.0.0.1:42005, PID: 39
but 127.0.0.1:0 wasn't handled as "autostart" like localhost:0 is
update is_local_special to allow that
* /etc/hosts file generated by docker contails localhost for both IPv4 and IPv6:
$ grep localhost /etc/hosts
127.0.0.1 localhost
::1 localhost ip6-localhost ip6-loopback
even when ipv6 is disabled in dockerd as reported in:
https://github.com/docker/for-linux/issues/250
* add a check for self.prserv.address to provide better error message:
ERROR: Unable to start PR Server, exitting
when something bad happens, but in this case you still need to read
bitbake-cookerdaemon.log to see the actuall error, in this case:
90 22:30:39.008441 --- Starting bitbake server pid 90 at 2021-08-26 22:30:39.008419 ---
90 22:30:39.023734 Started bitbake server pid 90
90 22:30:39.024286 Entering server connection loop
90 22:30:39.024753 Accepting [<socket.socket fd=6, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0, laddr=bitbake.sock>] ([])
90 22:30:39.026314 Processing Client
90 22:30:39.026456 Connecting Client
90 22:30:39.027509 Running command ['setFeatures', [2]]
90 22:30:39.027757 Command Completed
90 22:30:39.028711 Running command ['updateConfig', {'abort': False, 'force': False, 'invalidate_stamp': None, 'dry_run': False, 'dump_signatures': [], 'extra_assume_provided': [], 'profile': False, 'prefile': [], 'postfile': [], 'server_timeout': None, 'nosetscene': False, 'setsceneonly': False, 'skipsetscene': False, 'runall': None, 'runonly': None, 'writeeventlog': None, 'build_verbose_shell': False, 'build_verbose_stdout': False, 'default_loglevel': 20, 'debug_domains': {}}, {'DISTRO': '', 'PWD': '/OE/nodistro/honister', 'HOME': '/OE', 'MACHINE': 'qemux86', 'BB_ENV_EXTRAWHITE': 'MACHINE DISTRO TCMODE TCLIBC http_proxy ftp_proxy https_proxy all_proxy ALL_PROXY no_proxy SSH_AGENT_PID SSH_AUTH_SOCK BB_SRCREV_POLICY SDKMACHINE BB_NUMBER_THREADS PARALLEL_MAKE GIT_PROXY_COMMAND GIT_PROXY_IGNORE SOCKS5_PASSWD SOCKS5_USER WEBOS_DISTRO_BUILD_ID PSEUDO_DISABLED PSEUDO_BUILD', 'PATH': '/OE/nodistro/honister/oe-core/scripts:/OE/nodistro/honister/bitbake/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'LC_ALL': 'en_US.UTF-8', 'MACHINES': 'qemux86', 'HOSTNAME': '6a439759e3c6', 'TOPDIR': '/OE/nodistro/honister', 'LANG': 'en_US.UTF-8', 'TERM': 'xterm', 'SHLVL': '1', 'BITBAKE_HOME': '/OE', 'BUILDDIR': '/OE/nodistro/honister/BUILD', 'OLDPWD': '/OE/nodistro/honister/bitbake', '_': '/OE/nodistro/honister/bitbake/bin/bitbake'}, ['/OE/nodistro/honister/bitbake/bin/bitbake', '-k', 'zlib-native']]
Process Process-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/OE/nodistro/honister/bitbake/lib/bb/asyncrpc/serv.py", line 255, in run
self.start()
File "/OE/nodistro/honister/bitbake/lib/bb/asyncrpc/serv.py", line 144, in start_tcp
self.server = self.loop.run_until_complete(server_coro)
File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
return future.result()
File "/usr/lib/python3.8/asyncio/streams.py", line 94, in start_server
return await loop.create_server(factory, host, port, **kwds)
File "/usr/lib/python3.8/asyncio/base_events.py", line 1463, in create_server
raise OSError(err.errno, 'error while attempting '
OSError: [Errno 99] error while attempting to bind on address ('::1', 0, 0, 0): cannot assign requested address
90 22:30:39.530037 Command Completed
90 22:30:39.530913 Processing Client
90 22:30:39.531023 Disconnecting Client
90 22:30:39.531638 No timeout, exiting.
90 22:30:39.632137 Exiting
90 22:30:39.637562 Original lockfile contents: ['90\n']
90 22:30:39.638107 Exiting as we could obtain the lock
(Bitbake rev: c2cdda0e5fc720c60d3b8537fc702cb118981bd2)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the SIGTERM signal is sent to an asyncrpc server before it has
installed the SIGTERM handler in the main loop, it may miss the signal
which will can cause the calling process to wait forever on the join().
To resolve this, the calling process should mask of SIGTERM before
forking the server process and the server should unmask the signal only
after the handler is installed. To simplify the usage of the server, an
new helper function called serve_as_process() is added to do this
automatically and correctly.
Thanks: Scott Murray <scott.murray@konsulko.com> for helping debug
(Bitbake rev: ef2865efa98ad20823267364f2159d8d8c931400)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add logic to check that if REQUIRED_VERSION has been set that the cooker
class method findBestProviders properly handles the case where the
REQUIRED_VERSION has not been found.
Fixes [YOCTO #10096]
(Bitbake rev: 5df201d746f26154213e6d15d2721cd35b38b59e)
Signed-off-by: Charlie Davies <charles.davies@whitetree.xyz>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit adds a new column to the Bitbake --show-versions command
called Required Version. This column will display any packages which
have a REQUIRED_VERSION successfully set.
Fixes [YOCTO #10096]
(Bitbake rev: 90c7d1815e41243323d32b9dbb865757a922578a)
Signed-off-by: Charlie Davies <charles.davies@whitetree.xyz>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Commit bebef58b21bdff7a3ee1fa2449b7df19144f26fd introduced forcing
parser shutdown as default in case of build abort.
In this case bitbake sometimes hangs after facing error during parsing,
waiting for child processes to finish. Killing it then will spawn zombie
processes.
Thus we force the shutdown after catching an exception.
(Bitbake rev: 915330e1dbae1ee8fd9a0358decf2c294f771961)
Signed-off-by: Jan Brzezanski <jan.brzezanski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The bitbake logger overrode the definition of the debug() logging call
to include a debug level, but this causes problems with code that may
be using standard python logging, since the extra argument is
interpreted differently.
Instead, change the bitbake loggers debug() call to match the python
logger call and add a debug2() and debug3() API to replace calls that
were logging to a different debug level.
[RP: Small fix to ensure bb.debug calls bbdebug()]
(Bitbake rev: f68682a79d83e6399eb403f30a1f113516575f51)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
From tinfoil, if you edit bblayers.conf and break it, then call
parseConfiguration (e.g. by adding a bad layer with bitbake-layers),
the system doens't show any parse error yet it should.
Add in a call to the updateCache function so that things really
are reparsed when requested.
Partially fixes [YOCTO #14054]
(Bitbake rev: e655f9361b9c3b77906b8e06b5cc76bc5180640e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When we run `devtool build mc` recipe's task dependencies are expanded
to "mc:do_populate_sysroot" where "mc" name is treated as multiconfig
and "do_package_sysroot" as multiconfigname.
| ERROR: Multiconfig dependency mc:do_populate_sysroot depends on
| nonexistent multiconfig configuration named do_populate_sysroot
(Bitbake rev: 3ce4b2caccfe608a54dff159459f3687ea610597)
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The provided packages by a skipped recipe are supposed to be listed in
SkippedPackage.rprovides, which is used when generating a meaningful
error message when a build fails because of a skipped package.
Previously this variable only contained the contents of ${RPROVIDES}.
However, most recipes don't define RPROVIDES, they define
RPROVIDES_<pkg> for each package they provide. Additionally, the recipe
provides the packages in PACKAGES without them being included in
${RPROVIDES}.
Before this change, having a runtime dependency on a skipped non-recipe
package would result in a build error stating that the build failed
because the package was skipped, but without providing any reason for
why it was skipped.
(Bitbake rev: efd026c26a377b826a49b945a8212bf7de8a480a)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Recent changes mean data might not be setup. If its not, avoid tracebacks.
(Bitbake rev: 3daff610d9f39d73c80c54d1df46f573666e20db)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Blocks SIGINT in the worker processes to prevent them from running the
parent process signal handler, which causes them to deadlock under
certain circumstances.
[YOCTO #14034]
(Bitbake rev: 9f4207f4b598f549cbd4159841c720276736f23b)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After the recent init changes, if a client disconnects before issuing a
command, the cooker can break in the reset handlers. Add some guards
in the code to prevent this.
(Bitbake rev: 12605e30e4c4e1ae6a67c97363b892ebf0b9566c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The original code saved BBFILES back to BBFILES without any changes which isn't
usefule, so remove that line. Now save prioritized BBFILES to
BBFILES_PRIORITIZED which can accelerate the query a lot for the one which
relies on it such as bb.utils.get_file_layer().
(Bitbake rev: 49bdb5dfa57b41b3ed399961e947c404f9195998)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
