Sometimes the peg revision functionality we use in the svn fetcher is
not the correct option. Add a parameter nopegrevision which can be added
to disable this behaviour.
[YOCTO #6258]
(Bitbake rev: ea26682b706f655a8e418f56bebe742e2d130a1d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a "None" value gets into the varlist, it doesn't display properly.
Ensure we don't add one to have the exception display properly.
(Bitbake rev: ee26e258888114143e66330c256b5bfe7d071c53)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix:
lib/bb/tests/fetch.py:1288: DeprecationWarning: invalid escape sequence
for several lines of the fetch tests.
(Bitbake rev: 9d84fd557a3fcbae2cdd70b24e69325ad737a01e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
os.rename will overwrite the destination file if present so we can use this
instead of the process call overhead.
(Bitbake rev: b3cccaa6a896c41d8c9be5eebc327f726542d16b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a multiconfig starts with a digit, users would see pages of
errors as we use the multiconfig as a python function name prefix
and python functions cannot start with a digit. We could avoid doing
that but it is easier just to ask users to name multiconfigs not
starting with digits.
This tweak ensures the user sees an easier to understand error.
(Bitbake rev: f9cddaeef35b2ea0dadf717101ed896f6b857abd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make usage of Bitbake's s3 fetcher more flexible with different AWS profiles
and switch between profiles as export of the AWS_PROFILE environment variable
(Bitbake rev: 0f35dc4dfc829fe9c51c239d15567f86c5c14e58)
Signed-off-by: Valentin Danaila <drlv85@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This becomes a hard error in python 3.10.
(Bitbake rev: ae219e1f7460077f4492b31ac91cef4cf9b17277)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These no longer even work, and it's much better to just see all warnings
and fix them as they happen.
(Bitbake rev: 62d96ea2afbb0cfdb0d1f932dc96643e7bcd7f26)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is a potential race over the mirror tarballs where a partial git repo
could be extracted causing fetcher failures if the tarball is being rewritten
whilst another build accesses it.
Create the mirror tarball atomically to avoid this.
[YOCTO #14441]
(Bitbake rev: 3250bc950c56bd7dd2114df26e5a8e13b04ceac8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The recent fixes to merge setscene and normal task accounting in runqueue
fixed some display issues but broke the task numbering of setscene tasks.
Add new accounting methods to the stats structure specifically designed
for setscene. This accounts for the fact that setscene tasks can rerun
multiple times in the build.
Then use the new data in the UI to correctly display the numbers the
user wants to see to understand progress.
(Bitbake rev: ed7e2da88bf4b7bfc7ebfc12b9bd6c0fb7d8c1aa)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The fetcher would fail with:
Could not find a fetcher which supports the URL: ftps://...
(Bitbake rev: 9e56710c7203b1ec6cbefa758c81b69b697fe1a4)
Signed-off-by: Daniel Ammann <daniel.ammann@bytesatwork.ch>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The hash equivalence server has had the option to support a read-only
upstream server for some time now when launched as a standalone program,
but there was no way to set the upstream when using a locally started
server. Add a new variable called BB_HASHSERVE_UPSTREAM that can be used
to specify an upstream server when a local hash equivalence server is
used (e.g. BB_HASHSERVE is "auto")
(Bitbake rev: 250fa17f1391ff1ee01ab9b51d2a4f9aa35c1d1e)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When we parallelised normal and setscene tasks, the task stats
handling was left separate pending further thought. We had to remove
handling of the setscene tasks from the UI in order to maintain
consistent task numbering.
Currently, "0 of 0" tasks can be shown as setscene tasks execute
until the first normal task runs.
The only use left for sq_stats is in the active task numbers which
we can use the length of sq_ive for instead. We can therefore
drop it and return stats in all cases. This removes the "0 of 0" task
problem since the stats in all normal and setscene tasks matches.
[YOCTO #14479]
(Bitbake rev: eae6e947e37e18cded053814bd2a268b44fb25cd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If there is no BBLAYERS set in bblayers.conf show a more helpful
error and exit.
[YOCTO #14340]
(Bitbake rev: 97183e10faf9862b5d9489d6e2c27ac77c3b697d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the core layer is missing from bblayers.conf, the message the user sees is
hard to understand. Improve it.
[YOCTO #14340]
(Bitbake rev: 5815a7258ebb8a989e0c6f5798853559d9413f02)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Users sometimes put ${S} references in ${SRC_URI} without realising this can be
problematic. Improve the error messages if they accidentally do.
[YOCTO #11593]
(Bitbake rev: 89e0b19ec0b245a6cd414088904c91808e8814ab)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If there are circular task references, error on them rather than show
a recursion error. A simple reproducer is:
"""
do_packageswu () {
:
}
addtask do_packageswu after do_image_complete before do_image_qa
"""
into image_types.bbclass. There is code in runqueue to detect these but
we never get that far with the current codebase.
[YOCTO #13140]
(Bitbake rev: 339d4d6be515a71311b81fb9e99742af0d8a5130)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With "bitbake -v", for task failures you'd see the log output twice. Avoid
this by using the existing "did we print info" switch.
(Bitbake rev: e2c1afda4cb8023ed4ffeb5dc5bee4f0055659a8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the caller is piping the logs, they likely don't want them in the error exception
as well. This removes duplicate output from the build output allowing the UI level
controls on whether to show logs to work correctly.
(Bitbake rev: fc58ad84a9deb2620ad90611684dad65dafedb11)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a python task fails with sys.exit(), we currently see no TaskFailed event.
The high level code does detect the exit code and fail the task but it can
leave the UI inconsistent with log output.
Fix this be intercepting SystemExit explicitly. This makes python
task failures consistent with shell task failures.
(Bitbake rev: 9eee9fd4f2f96789ad2b037e74d561bdc1426856)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The point of the injected text is to identify where the function comes from. Using
the correct function name would therefore be better.
(Bitbake rev: 30c6ff8551c235254ab90663ab88f66bb0c71edb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is now used from bb.parse everywhere so drop this long deprecated reference.
(Bitbake rev: aaa5292ef96ea27f505bc5c5a4b1eb4f497ed061)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This class has long since been deprecated and is unused, drop it. I'd love
to get rid of the rest of persist_data but it is still used by the fetcher,
sadly.
(Bitbake rev: 1c574aae8c8ec427f27ab0d68bac9e7483016f18)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In multiconfig builds with large numbers of identical tasks, builds were
deadlocking after recent runqueue changes upon rebuilds where there was
heavy sstate usage (i.e. on second builds after a first completed).
The issue was that deferred tasks were being left indefinitely on
the deferred list. The deadlock handler was then "breaking" things
by failing tasks that had already succeeded, leading to the task
being on both covered and not covered lists, giving a further error.
The fix is to clean up the deferred task list when each setscene task
completes. I'd previously been hoping to avoid iterating that list
but it appears unavoidable.
[YOCTO #14342]
(Bitbake rev: ae24a0f2d2d8b4b5ec10efabd0e9362e560832ea)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the deferred task deadlock avoidance code triggers, it could mark an executed
task as failed which leads to "covered and not covered" error messages. Improve
the logic so if the deadlock code is triggered, it doesn't cause the errors.
(Bitbake rev: 51bdd6cb3bd9e2c02e261fb578bb945b86b82c75)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This adds one new test that verifies the use of a specific filename
when defined in PREMIRRORS.
bb.tests.fetch.NPMTest:
- test_npm_premirrors_with_specified_filename
While testing bz#13039, it was found that test_npm_registry_alternate
fails with ENOTFOUND. This was corrected by using npmjs's public mirror.
The change to fetch2 for bz#13039 highlighted an issue with the
test_npm_premirrors test where the created file:// mirror was using the
downloadfilename rather than the tarball that is defined by the npm url.
(Bitbake rev: 5ba191a0407af9e652e3b86302dce3e952d6b587)
Signed-off-by: Scott Weaver <weaverjs@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This adds three new tests which evaluate different use cases of the
downloadfilename property.
bb.tests.fetch.FetcherNetworkTest:
- test_fetch_specify_downloadfilename
- test_fetch_premirror_specify_downloadfilename_regex_uri
- test_fetch_premirror_specify_downloadfilename_specific_uri
(Bitbake rev: 61db3e96530d650e098436fd086f0182d32998f7)
Signed-off-by: Scott Weaver <weaverjs@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When downloadfilename is defined in a recipe's SRC_URI and PREMIRRORS is also
defined using the same URI, the downloadfilename is appended to the mirror
URI and it should not be.
[YOCTO #13039]
(Bitbake rev: 8a3ff9f3eaf19d4258eb070c5dc230dface269b2)
Signed-off-by: Scott Weaver <weaverjs@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* When using PRSERV_HOST = "localhost:0" inside
Docker container (tested with ubuntu 20.04 and 21.04) the
self.loop.run_until_complete never completed, so self.address
wasn't ever assigned few lines bellow and then
self.port = int(self.prserv.address.rsplit(':', 1)[1])
in lib/prserv/serv.py caused a bit ugly exception:
bitbake@599696cd20aa:~/nodistro/honister$ bitbake -k pkgconfig-native
Traceback (most recent call last):
File "/OE/nodistro/honister/bitbake/bin/bitbake", line 35, in <module>
sys.exit(bitbake_main(BitBakeConfigParameters(sys.argv),
File "/OE/nodistro/honister/bitbake/lib/bb/main.py", line 385, in bitbake_main
return ui_module.main(server_connection.connection, server_connection.events,
File "/OE/nodistro/honister/bitbake/lib/bb/ui/knotty.py", line 397, in main
params.updateToServer(server, os.environ.copy())
File "/OE/nodistro/honister/bitbake/lib/bb/cookerdata.py", line 75, in updateToServer
raise Exception("Unable to update the server configuration with local parameters: %s" % error)
Exception: Unable to update the server configuration with local parameters: Traceback (most recent call last):
File "/OE/nodistro/honister/bitbake/lib/bb/command.py", line 90, in runCommand
result = command_method(self, commandline)
File "/OE/nodistro/honister/bitbake/lib/bb/command.py", line 286, in updateConfig
command.cooker.updateConfigOpts(options, environment, cmdline)
File "/OE/nodistro/honister/bitbake/lib/bb/cooker.py", line 491, in updateConfigOpts
self.reset()
File "/OE/nodistro/honister/bitbake/lib/bb/cooker.py", line 1717, in reset
self.handlePRServ()
File "/OE/nodistro/honister/bitbake/lib/bb/cooker.py", line 383, in handlePRServ
self.prhost = prserv.serv.auto_start(self.data)
File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 318, in auto_start
singleton.start()
File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 133, in start
self.port = int(self.prserv.address.rsplit(':', 1)[1])
AttributeError: 'NoneType' object has no attribute 'rsplit'
* the issue was caused by "localhost" being resolved as IPv6 address ::1
and then asyncio failing to bind it, the same is reproducible with hashserv, but
hashserve at least shows nice error message:
bitbake$ bitbake-hashserv -l DEBUG -b localhost:0
Traceback (most recent call last):
File "/OE/nodistro/honister/bitbake/bin/bitbake-hashserv", line 59, in <module>
ret = main()
File "/OE/nodistro/honister/bitbake/bin/bitbake-hashserv", line 53, in main
server.serve_forever()
File "/OE/nodistro/honister/bitbake/lib/bb/asyncrpc/serv.py", line 233, in serve_forever
self.start()
File "/OE/nodistro/honister/bitbake/lib/bb/asyncrpc/serv.py", line 144, in start_tcp
self.server = self.loop.run_until_complete(server_coro)
File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
return future.result()
File "/usr/lib/python3.8/asyncio/streams.py", line 94, in start_server
return await loop.create_server(factory, host, port, **kwds)
File "/usr/lib/python3.8/asyncio/base_events.py", line 1463, in create_server
raise OSError(err.errno, 'error while attempting '
OSError: [Errno 99] error while attempting to bind on address ('::1', 0, 0, 0): cannot assign requested address
* or by bitbake-prserv in prserv.log:
bitbake$ bitbake-prserv --start --host=localhost --port=42005
bitbake$ cat prserv.log
Traceback (most recent call last):
File "/OE/nodistro/honister/bitbake/bin/bitbake-prserv", line 55, in <module>
ret = main()
File "/OE/nodistro/honister/bitbake/bin/bitbake-prserv", line 46, in main
ret=prserv.serv.start_daemon(options.dbfile, options.host, options.port,os.path.abspath(options.logfile), options.read_only)
File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 226, in start_daemon
run_as_daemon(daemon_main, pidfile, os.path.abspath(logfile))
File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 202, in run_as_daemon
func()
File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 224, in daemon_main
server.serve_forever()
File "/OE/nodistro/honister/bitbake/lib/bb/asyncrpc/serv.py", line 233, in serve_forever
self.start()
File "/OE/nodistro/honister/bitbake/lib/bb/asyncrpc/serv.py", line 144, in start_tcp
self.server = self.loop.run_until_complete(server_coro)
File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
return future.result()
File "/usr/lib/python3.8/asyncio/streams.py", line 94, in start_server
return await loop.create_server(factory, host, port, **kwds)
File "/usr/lib/python3.8/asyncio/base_events.py", line 1463, in create_server
raise OSError(err.errno, 'error while attempting '
OSError: [Errno 99] error while attempting to bind on address ('::1', 42005, 0, 0): cannot assign requested address
* while 127.0.0.1 works fine:
bitbake$ bitbake-prserv --start --host=127.0.0.1 --port=42005
bitbake$ cat prserv.log
DEBUG: Listening on ('127.0.0.1', 42005)
2021-08-26 22:28:05,828 Listening on ('127.0.0.1', 42005)
DEBUG: Opening PRServ database 'file:/OE/nodistro/honister/prserv.sqlite3'
2021-08-26 22:28:05,829 Opening PRServ database 'file:/OE/nodistro/honister/prserv.sqlite3'
NOTE: Started PRServer with DBfile: /OE/nodistro/honister/prserv.sqlite3, Address: 127.0.0.1:42005, PID: 39
2021-08-26 22:28:05,831 Started PRServer with DBfile: /OE/nodistro/honister/prserv.sqlite3, Address: 127.0.0.1:42005, PID: 39
but 127.0.0.1:0 wasn't handled as "autostart" like localhost:0 is
update is_local_special to allow that
* /etc/hosts file generated by docker contails localhost for both IPv4 and IPv6:
$ grep localhost /etc/hosts
127.0.0.1 localhost
::1 localhost ip6-localhost ip6-loopback
even when ipv6 is disabled in dockerd as reported in:
https://github.com/docker/for-linux/issues/250
* add a check for self.prserv.address to provide better error message:
ERROR: Unable to start PR Server, exitting
when something bad happens, but in this case you still need to read
bitbake-cookerdaemon.log to see the actuall error, in this case:
90 22:30:39.008441 --- Starting bitbake server pid 90 at 2021-08-26 22:30:39.008419 ---
90 22:30:39.023734 Started bitbake server pid 90
90 22:30:39.024286 Entering server connection loop
90 22:30:39.024753 Accepting [<socket.socket fd=6, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0, laddr=bitbake.sock>] ([])
90 22:30:39.026314 Processing Client
90 22:30:39.026456 Connecting Client
90 22:30:39.027509 Running command ['setFeatures', [2]]
90 22:30:39.027757 Command Completed
90 22:30:39.028711 Running command ['updateConfig', {'abort': False, 'force': False, 'invalidate_stamp': None, 'dry_run': False, 'dump_signatures': [], 'extra_assume_provided': [], 'profile': False, 'prefile': [], 'postfile': [], 'server_timeout': None, 'nosetscene': False, 'setsceneonly': False, 'skipsetscene': False, 'runall': None, 'runonly': None, 'writeeventlog': None, 'build_verbose_shell': False, 'build_verbose_stdout': False, 'default_loglevel': 20, 'debug_domains': {}}, {'DISTRO': '', 'PWD': '/OE/nodistro/honister', 'HOME': '/OE', 'MACHINE': 'qemux86', 'BB_ENV_EXTRAWHITE': 'MACHINE DISTRO TCMODE TCLIBC http_proxy ftp_proxy https_proxy all_proxy ALL_PROXY no_proxy SSH_AGENT_PID SSH_AUTH_SOCK BB_SRCREV_POLICY SDKMACHINE BB_NUMBER_THREADS PARALLEL_MAKE GIT_PROXY_COMMAND GIT_PROXY_IGNORE SOCKS5_PASSWD SOCKS5_USER WEBOS_DISTRO_BUILD_ID PSEUDO_DISABLED PSEUDO_BUILD', 'PATH': '/OE/nodistro/honister/oe-core/scripts:/OE/nodistro/honister/bitbake/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'LC_ALL': 'en_US.UTF-8', 'MACHINES': 'qemux86', 'HOSTNAME': '6a439759e3c6', 'TOPDIR': '/OE/nodistro/honister', 'LANG': 'en_US.UTF-8', 'TERM': 'xterm', 'SHLVL': '1', 'BITBAKE_HOME': '/OE', 'BUILDDIR': '/OE/nodistro/honister/BUILD', 'OLDPWD': '/OE/nodistro/honister/bitbake', '_': '/OE/nodistro/honister/bitbake/bin/bitbake'}, ['/OE/nodistro/honister/bitbake/bin/bitbake', '-k', 'zlib-native']]
Process Process-1:
Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/OE/nodistro/honister/bitbake/lib/bb/asyncrpc/serv.py", line 255, in run
self.start()
File "/OE/nodistro/honister/bitbake/lib/bb/asyncrpc/serv.py", line 144, in start_tcp
self.server = self.loop.run_until_complete(server_coro)
File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
return future.result()
File "/usr/lib/python3.8/asyncio/streams.py", line 94, in start_server
return await loop.create_server(factory, host, port, **kwds)
File "/usr/lib/python3.8/asyncio/base_events.py", line 1463, in create_server
raise OSError(err.errno, 'error while attempting '
OSError: [Errno 99] error while attempting to bind on address ('::1', 0, 0, 0): cannot assign requested address
90 22:30:39.530037 Command Completed
90 22:30:39.530913 Processing Client
90 22:30:39.531023 Disconnecting Client
90 22:30:39.531638 No timeout, exiting.
90 22:30:39.632137 Exiting
90 22:30:39.637562 Original lockfile contents: ['90\n']
90 22:30:39.638107 Exiting as we could obtain the lock
(Bitbake rev: c2cdda0e5fc720c60d3b8537fc702cb118981bd2)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make versionVariableMatch() support pn-foo overrides using the new
override syntax.
(Bitbake rev: 653df4bc413d595d5611d5fa678e7f2e30bb7431)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
asyncio in older Python 3.x (seen with 3.7) can seemingly hang if
new_event_loop is called repeatedly in the same process. The
reuse of processes in the Bitbake thread pool during parsing seems
to be able to trigger this with the PR server export selftest.
It appears that calling set_event_loop with the new loop avoids the
issue, so that is now done in the asyncrpc Client initializer (with
an explanatory comment). This should be revisited when the day
arrives that Python 3.9 becomes the minimum required for BitBake.
Additionally, it was discovered that using get_event_loop in the
asyncrpc server initialization can trigger hangs in the hashserv
unittests when the second test is run. To avoid this, switch to
calling new_event_loop + set_event_loop in the initialization code
there as well.
(Bitbake rev: bb9a36563505652294b20b4c88199b24fa208342)
Signed-off-by: Scott Murray <scott.murray@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Reworks the async I/O API so that the async loop is only created in the
child process. This requires deferring the creation of the server until
the child process and a queue to transfer the bound address back to the
parent process
(Bitbake rev: 8555869cde39f9e9a9ced5a3e5788209640f6d50)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
[small loop -> self.loop fix in serv.py]
Signed-off-by: Scott Murray <scott.murray@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The urllib.request.ProxyHandler constructor only reads the $http_proxy
+ $https_proxy environment variables.
$no_proxy is evaluated later when the url is opened.
It is therefore not sufficient to just construct the proxy handler in
the
| with bb.utils.environment(**newenv):
context, but the 'opener.open(r)' call must also be made there.
(Bitbake rev: 076baf4fbd328d247508fd399866a397eb34f67e)
Signed-off-by: Enrico Scholz <enrico.scholz@sigma-chemnitz.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
OE-Core did this a while ago, it is simpler if bitbake matches.
(Bitbake rev: a3050aee21b6a23b55232d52f89980a3bbd3a290)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The days of broken certificates are behind us now, so instead of always
passing --no-check-certificate to wget, don't pass it by default and
instead only pass it BB_CHECK_SSL_CERTS = "0".
[ YOCTO #14108 ]
(Bitbake rev: 4104850dd36096a9ff01836c5fca9ac0e452bcf8)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of just exporting the proxy variables when calling into urllib,
use bb.utils.environment() to export all of the known variables that are
needed for proper connectivity.
Specifically, this ensures that SSL_CERT_FILE is set, so that libssl can
find the certificates in buildtools environments
(Bitbake rev: 116637b0e9aabae7f680b102dbf3577b8a58f049)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is a list of environment variable names that need to be exported
when executing external commands, such as 'http_proxy'. To avoid
duplication, make this a top-level variable.
Also add SSL_CERT_FILE, which is used by OpenSSL to locate the
certificate bundle. This is needed in buildtools environments where the
default path isn't valid.
(Bitbake rev: 5c2cf57fd11d91f749a3b562f6f0a863f15013ed)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bb.utils.environment() is a context manager to alter os.environ inside
a specific block, restoring it after the block is closed.
(Bitbake rev: 9974848f67581ff7d76cef52a94f505af99b4932)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If run on an empty build directory, taskexp wasn't working as it didn't
send the current environment to the server. This means HOSTTOOLS in oe-core
couldn't be built and gave an error. Add the missing updateToServer call in.
[YOCTO #14408]
(Bitbake rev: 06a0bbe746f879ae539223e7fdb6f07d55d13719)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When an exception occurs at startup, show it to the user.
[YOCTO #14408]
(Bitbake rev: cc1df1af67cfd3e223b39e2b7ea5f86b8cf78aee)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If an exception occurs in early setup, bitbake could just hang. Return
the exception rather than doing that.
[YOCTO #14408]
(Bitbake rev: c8a4107132ce51f84ae84bf1ceb1c3fd90f156d3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently the code can just show nothing as the exception if there was a double
fault, which in this code path is quite likely. This leads to an error log
which effectively says "it failed" with no information about how.
Improve things so we get a nice verbose traceback left in the logs/output
which is preferable to no logs.
(Bitbake rev: e5782b71647d1eb6de53bde7bc4f6019a5589f21)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix some references that missed during the overrides syntax migration or
were incorrect. Thanks to Quentin Schulz <foss@0leil.net> for the patch.
(Bitbake rev: 6184cb07dfa44f5f76f1c423533b4547d80b20ab)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The previous patches have exposed new issues with this code path,
the issues being around what should happen when the hash of a task
changes and the task is or is not on the deferred task list.
Rather than rebuilding the deferred task list during each rehash
event, build it once at the start of a build. This avoids the problem
of tasks being added back after they have run and also avoids problems
of always ensuring the same task is deferred. It also allows the
'outrightfail' codepath to be handled separately as the conditions
are subtly differnt.
One significant win for the new approch is the build is not continually
printing out lists of deferred tasks, that list remains fairly static
from the start of the build. Logic is added in to ensure a rehashed
task with a hash matching other deferred tasks is deferred along with
them as a small optimization.
An interesting test case for this code was reported by Mark Hatle
with four multiconfigs, each the same apart from TMPDIR and running a
build of:
bitbake buildtools-tarball mc:{one,two,three,four}:core-image-minimal
which is interesting in that the build of buildtools partially overlaps
core-image-minimal and the build has a rehash event for qemuwrapper-cross
even without any external hash equivalence server or preexisting data.
(Bitbake rev: bb424e0a6d274d398f434f7df63951da9ce305b3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Setting something like:
BAR:append:unusedoverride
should cause BAR to be None, not "" which was what the datastore was
returning. This caused problems when mixing variables like:
RDEPENDS:${PN}:inactiveoverride
RDEPENDS:${BPN}
since key expansion would report key overlap when there was none. This
is a bug in the datastore. Fix it and add a test too.
[YOCTO #14088]
(Bitbake rev: 699e36c270d863258502d315ed00a1b940bfbf96)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
