ac5512b0ac
This fetcher allows BitBake to fetch from a Google Cloud Storage bucket. The fetcher expects a gs:// URI of the following form: SSTATE_MIRRORS = "file://.* gs://<bucket name>/PATH" The fetcher uses the Google Cloud Storage Python Client, and expects it to be installed, configured, and authenticated prior to use. If accepted, this patch should merge in with the corresponding oe-core patch titled "Add GCP fetcher to list of supported protocols". Some comments on the patch: There is also documentation for the fetcher added to the User Manual. I'm still not completely sure about the recommends_checksum() being set to True. As I've noted in the mailing list, it will throw warnings if the fetcher is used in recipes without specifying a checksum. Please let me know if this is intended behavior or if it should be modified. Here is how this fetcher conforms to the fetcher expectations described at this link: https://git.yoctoproject.org/poky/tree/bitbake/lib/bb/fetch2/README a) Yes, network fetching only happens in the fetcher b) The fetcher has nothing to do with the unpack phase so there is no network access there c) This change doesn't affect the behavior of DL_DIR. The GCP fetcher only downloads to the DL_DIR in the same way that other fetchers, namely the S3 and Azure fetchers do. d) The fetcher is identical to the S3 and Azure fetchers in this context e) Yes, the fetcher output is deterministic because it is downloading tarballs from a bucket and not modifying them in any way. f) I set up a local proxy using tinyproxy and set the http_proxy variable to test whether the Python API respected the proxy. It appears that it did as I could see traffic passing through the proxy. I also did some searching online and found posts indicating that the Google Cloud Python APIs supported the classic Linux proxy variables, namely: - https://github.com/googleapis/google-api-python-client/issues/1260 g) Access is minimal, only checking if the file exists and downloading it if it does. h) Not applicable, BitBake already knows which version it wants and the version infomation is encoded in the filename. The fetcher has no concept of versions. i) Not applicable j) Not applicable k) No tests were added as part of this change. I didn't see any tests for the S3 or Azure changes either, is that OK? l) I'm not 100% familiar but I don't believe this fetcher is using any tools during parse time. Please correct me if I'm wrong. (Bitbake rev: 8e7e5719c1de79eb488732818871add3a6fc238b) Signed-off-by: Emil Ekmečić <eekmecic@snap.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Documentation
This is the directory that contains the BitBake documentation.
Manual Organization
Folders exist for individual manuals as follows:
- bitbake-user-manual --- The BitBake User Manual
Each folder is self-contained regarding content and figures.
If you want to find HTML versions of the BitBake manuals on the web, go to https://www.openembedded.org/wiki/Documentation.
Sphinx
The BitBake documentation was migrated from the original DocBook format to Sphinx based documentation for the Yocto Project 3.2 release.
Additional information related to the Sphinx migration, and guidelines for developers willing to contribute to the BitBake documentation can be found in the Yocto Project Documentation README file:
https://git.yoctoproject.org/cgit/cgit.cgi/yocto-docs/tree/documentation/README
How to build the Yocto Project documentation
Sphinx is written in Python. While it might work with Python2, for obvious reasons, we will only support building the BitBake documentation with Python3.
Sphinx might be available in your Linux distro packages repositories, however it is not recommend using distro packages, as they might be old versions, especially if you are using an LTS version of your distro. The recommended method to install Sphinx and all required dependencies is to use the Python Package Index (pip).
To install all required packages run:
$ pip3 install sphinx sphinx_rtd_theme pyyaml
To build the documentation locally, run:
$ cd documentation $ make -f Makefile.sphinx html
The resulting HTML index page will be _build/html/index.html, and you can browse your own copy of the locally generated documentation with your browser.