58155ea1a3
* when comparing 2 tmp/stamps/*do_configure.sigdata* I got
TypeError("filename must be a str or bytes object, or a file")
but both files I was comparing were
Zstandard compressed data (v0.8+), Dictionary ID: None
according to "file"
with TypeError catched to show which file it failed to open I got better
error which shows it was trying to read "do_prepare_recipe_sysroot.sigdata"
file now and after a while you might notice that it's not just the expected
file, but a dict with 'path', 'sstate', 'time'.
Fix that in bitbake-diffsigs but keep the TypeError and add OSError
in case it will eventually walk on file which isn't zstd compressed
pipecompress throws CompressionError.
ERROR: Failed to open {'path': '5.15.do_prepare_recipe_sysroot.sigdata.99b12a401341a0df7c3553cb00c87a7674295496bd5c25ed71764ee0d0fb8eb8', 'sstate': False, 'time': 1707136354.991718}: filename must be a str or bytes object, or a file
Traceback (most recent call last):
File "bitbake/bin/bitbake-diffsigs", line 192, in <module>
output = bb.siggen.compare_sigfiles(options.sigdatafile1, options.sigdatafile2, recursecb, color=color)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "bitbake/lib/bb/siggen.py", line 1039, in compare_sigfiles
recout = recursecb(dep, a[dep], b[dep])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "bitbake/bin/bitbake-diffsigs", line 102, in recursecb
out2 = bb.siggen.compare_sigfiles(hashfiles[hash1], hashfiles[hash2], recursecb, color=color)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "bitbake/lib/bb/siggen.py", line 857, in compare_sigfiles
raise err
File "bitbake/lib/bb/siggen.py", line 853, in compare_sigfiles
with bb.compress.zstd.open(a, "rt", encoding="utf-8", num_threads=1) as f:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "bitbake/lib/bb/compress/zstd.py", line 12, in open
return bb.compress._pipecompress.open_wrap(ZstdFile, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "bitbake/lib/bb/compress/_pipecompress.py", line 59, in open_wrap
raise TypeError("filename must be a str or bytes object, or a file")
TypeError: filename must be a str or bytes object, or a file
* if I replace zstd file with just plaintext it fails with:
$ echo foo > foo
$ echo foo > bar
$ bitbake-diffsigs foo bar
zstd: /*stdin*\: unsupported format
ERROR: Process died with 1
sys:1: ResourceWarning: unclosed file <_io.BufferedReader name='foo'>
with this change it shows the name of the file which it failed to uncompress:
$ bitbake-diffsigs foo bar
zstd: /*stdin*\: unsupported format
ERROR: Failed to open sigdata file 'foo': Process died with 1
ERROR: Process died with 1
sys:1: ResourceWarning: unclosed file <_io.BufferedReader name='foo'>
(Bitbake rev: f3f843a4fd6e0a9e8f6edef5dd3cf1fce29c50ba)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bitbake
BitBake is a generic task execution engine that allows shell and Python tasks to be run efficiently and in parallel while working within complex inter-task dependency constraints. One of BitBake's main users, OpenEmbedded, takes this core and builds embedded Linux software stacks using a task-oriented approach.
For information about Bitbake, see the OpenEmbedded website: https://www.openembedded.org/
Bitbake plain documentation can be found under the doc directory or its integrated html version at the Yocto Project website: https://docs.yoctoproject.org
Bitbake requires Python version 3.8 or newer.
Contributing
Please refer to our contributor guide here: https://docs.yoctoproject.org/dev/contributor-guide/ for full details on how to submit changes.
As a quick guide, patches should be sent to bitbake-devel@lists.openembedded.org The git command to do that would be:
git send-email -M -1 --to bitbake-devel@lists.openembedded.org
If you're sending a patch related to the BitBake manual, make sure you copy the Yocto Project documentation mailing list:
git send-email -M -1 --to bitbake-devel@lists.openembedded.org --cc docs@lists.yoctoproject.org
Mailing list:
https://lists.openembedded.org/g/bitbake-devel
Source code:
https://git.openembedded.org/bitbake/
Testing
Bitbake has a testsuite located in lib/bb/tests/ whichs aim to try and prevent regressions. You can run this with "bitbake-selftest". In particular the fetcher is well covered since it has so many corner cases. The datastore has many tests too. Testing with the testsuite is recommended before submitting patches, particularly to the fetcher and datastore. We also appreciate new test cases and may require them for more obscure issues.
To run the tests "zstd" and "git" must be installed.
The assumption is made that this testsuite is run from an initialized OpenEmbedded build
environment (i.e. source oe-init-build-env is used). If this is not the case, run the
testsuite as follows:
export PATH=$(pwd)/bin:$PATH
bin/bitbake-selftest
The testsuite can alternatively be executed using pytest, e.g. obtained from PyPI (in this case, the PATH is configured automatically):
pytest