Now that the DocBook files are removed, we can rename the top level
Makefile.
(Bitbake rev: a7c47f1eac8caac607a2b5f12d07235dff4d740f)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
The BitBake documentation was migrated to Sphinx. Let's remove the
deprecated DocBook files.
(Bitbake rev: 427721d8ff2c8e1db8cb490074f2eed88d03852a)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
From experience the body takes the whole space anyway and the text stays
within the screen boundaries by default, no need to make the min-width
640px then.
(Bitbake rev: d52190ea426d961f609c657dcb403baf59352969)
Signed-off-by: Quentin Schulz <foss@0leil.net>
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is similar to this change in yocto-docs:
9e468274eaad (docs: ref-manual: ref-variables: add links to terms in glossary)
(Bitbake rev: ebdeef2c185465ac9f7d7f01ae9e8531355b9e70)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Recent changes mean data might not be setup. If its not, avoid tracebacks.
(Bitbake rev: 3daff610d9f39d73c80c54d1df46f573666e20db)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We try to write sstate with group-write permissions so that sstate-cache
can be shared between multiple users. However the siginfo files are
created in various tasks which may set their own umask (such as
do_populate_sysroot, 0022). This results in no group write permission
on the intermediate directories, which is fatal when sharing a cache.
Fix this by wrapping the siginfo mkdir in a umask change to 0002.
(Bitbake rev: 75d9ef04a908e366633b255d23ab3275f6860d3a)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a umask context manager which can be used to temporarily change the
umask in a 'with' block.
(Bitbake rev: 6c601e68a27e1c60b04c2a61830d1812cc883e09)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Implements backslashes in local filenames.
A typical usecase for such a filename is a systemd unit.
Example: `dev-disk-by\x2dlabel-FOO.device`
(Bitbake rev: 14a35f273b579d5cd5fd92765b89c28f870dd577)
Signed-off-by: Leif Middelschulte <leif.middelschulte@klsmartin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The introspected API for GtkTreeModel and friends had some unexpected
quirks which have now been fixed, see[1] for details. However, for
example Ubuntu 20.04 has the fixed GTK but not an updated pygobject which
means taskexp raises an exception on startup.
Solve by manually looking at what functions are present and calling the
right one.
[ YOCTO #14055 ]
[1] 9cdbc56fba
(Bitbake rev: ac7d1114a7e99e6efd6a37b03d170faf678513fb)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This reverts commit 4fca9a07f2d6b0544977112672b786982d7bb8f2.
The default python handler is to ignore SIGPIPE errors yet by adding
this line, we see:
2020-09-23 22:10:20,325 - oe-selftest - INFO - recipetool.RecipetoolTests.test_recipetool_appendfile_subdir (subunit.RemotedTestCase)
2020-09-23 22:10:20,325 - oe-selftest - INFO - ... FAIL
2020-09-23 22:10:20,325 - oe-selftest - INFO - 11: 30/44 227/415 (14.85s) (recipetool.RecipetoolTests.test_recipetool_appendfile_subdir)
2020-09-23 22:10:20,325 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
File "/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/meta/lib/oeqa/selftest/cases/recipetool.py", line 226, in test_recipetool_appendfile_subdir
_, output = self._try_recipetool_appendfile('selftest-recipetool-appendfile', '/usr/share/selftest-replaceme-subdir', self.testfile, '', expectedlines, ['testfile'])
File "/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/meta/lib/oeqa/selftest/cases/recipetool.py", line 85, in _try_recipetool_appendfile
return self._try_recipetool_appendcmd(cmd, testrecipe, expectedfiles, expectedlines)
File "/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/meta/lib/oeqa/selftest/cases/recipetool.py", line 49, in _try_recipetool_appendcmd
bbappendfile = self._check_bbappend(testrecipe, recipefile, self.templayerdir)
File "/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/meta/lib/oeqa/selftest/cases/devtool.py", line 186, in _check_bbappend
result = runCmd('bitbake-layers show-appends', cwd=self.builddir)
File "/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/meta/lib/oeqa/utils/commands.py", line 201, in runCmd
raise AssertionError("Command '%s' returned non-zero exit status %d:\n%s" % (command, result.status, exc_output))
AssertionError: Command 'bitbake-layers show-appends' returned non-zero exit status 141:
NOTE: Reconnecting to bitbake server...
where status 141 is SIGPIPE.
We expect SIGPIPE not to be raised like this. I tried the commands in the original
commit to pipe through less with the change reverted and I don't see the error
shown. I therefore believe this is safe to revert.
(Bitbake rev: 94a18f1f2e51dd5d3928e6e0c17f2c19c8b5db49)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This change improves performance by reducing runtime about 33% for typical
inputs. (using test_clean_basepath_performance)
It is also easier to read, and slightly more resilient to future changes since it doesn't
mention 'virtual' anymore.
(Bitbake rev: 27b53186fa67d281d29b2f8e15bcff8dc2557b8a)
Signed-off-by: Jean-Francois Dagenais <jeff.dagenais@gmail.com>
Co-Developed-by: Maxime Roussin-Bélanger <maxime.roussinbelanger@gmail.com>
Signed-off-by: Maxime Roussin-Bélanger <maxime.roussinbelanger@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
While discussing with Richard we thought these might help document
and safeguard the basic requirements of clean_basepath.
A 'bonus' performance testcase is added but commented out since its
runtime is long and test machine specific. It is intended for developers
to test before and after their changes to the target function as a due
diligence verification.
(Bitbake rev: ee41549f26952d5f7af19a9b3d8a8b969866e2ef)
Signed-off-by: Jean-Francois Dagenais <jeff.dagenais@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Microsoft's TFS VCS system allows for spaces in a git repository url.
An example of a valid url is:
ssh://tfs-my-company.org:22/tfs/My Projects/FooBar
This commit adds support for such urls by implementing two changes.
Firstly, when bitbake makes a git command line call the url is
surrounded by quotes so that the url, regardless of spaces, is
treated as one argument. Secondly, additional parsing of various
filepath variables, which are based off of the url, are now
completed with any spaces in the url replaced with underscores.
(Bitbake rev: eb38b6f0935763f7ba19e5618f376fcae1dac41a)
Signed-off-by: Charlie Davies <charles.davies@whitetree.xyz>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Using localhost for direct builds on host is fine. A case with a
misbehavior has been sighted on a Docker build. Even when the host
supports IPv6, but Docker is not configured correspondingly - some
versions of the asyncio Python module seem to misbehave and try to
use IPv6 where it's not supported in the container. This happens at
least on some Ubuntu 18.04 based containers, resolving the IP
explicitly appears to be the fix.
(Bitbake rev: 0e20f91c11afdc17ea776aa02e0cc8b0d59a23d4)
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This more closely matches the original docbook style and is appropriate
given the way the manual uses this element.
(Bitbake rev: a0f37789493aeb1cd5e82c17459bfa4cf6c0ecd6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The 'verbatim' text was rendered with Courier font in DocBook
(e.g. when using the <filename> tag). With DocBook we are using the
``FOO`` notation which ends up in a <pre> class in the HTML
output. Configure the theme CSS to use Courier, to preserve the look
and feel of the original docs.
(Bitbake rev: cbc5ca48c7e79c47f055a47f8f9aa480490dd00f)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The initial theme override covers for tip and and note only.
(Bitbake rev: 99ba6fe98b9c7c4dbe70dce63c478c167c10d91a)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Review all pages, and fix up for formatting which was not covered by
pandoc, such as some links and code block sections.
(Bitbake rev: d99760cc687cc9c24d6d9a1c49fa094574476e0c)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This page has a list of all the previously released Bitbake user manual.
(Bitbake rev: 63adcaa527b931f4a9494729b26e40ddef8d3695)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Remove the pointless looking parts of breadcrumb navigtation
* Add a document type switcher to the breadcrumb navigation
* Add a version selection switch to the breadcrumb navigation
(Bitbake rev: cb19159cf8972713a2063387849804c5cea72d24)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Remove the 'generated by Sphinx' text on each page
* Add a 'last updated timestamp' on each page
* Remove the trailing 'dot' in TOC numbering
(Bitbake rev: 10a54678bf7a119e91df8160b0f43384181d636d)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch fixes a handful of remaining warnings reported by Sphinx.
(Bitbake rev: 35fdc18549607e195d424f554e8d4b74e87fd245)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For some notes with links, the links were lost during the pandoc
conversion.
(Bitbake rev: e11d2dd1adbcb35b45d6d0f3a47f8d06caab0b49)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since we converted the list of variables into a Sphinx glossary, the
automatic conversion from Pandoc does not produce proper links. We fix
them up using a Python regexp.
Similarly some http links were not converted correctly, and can also
be fixed up with a regexp.
This patch was generated by running the following regexp:
line = re.sub("` <(https?://.*)>`__",
"\\1",
line)
line = re.sub("`+(\w+)`* <#var-bb-\\1>`__",
":term:`\\1`",
line)
(Bitbake rev: 57300955beb200c81762880f44f1a499451b7432)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is possible to override CSS settings from the theme, by providing
custom snippets of CSS stylesheet. Support for that is added in
conf.py file.
Most of the CSS customization is inherited from the DocBook CSS style.
(Bitbake rev: fa304c0129538d7a0b7a3922cd1fde2e242d6cab)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To install this additional theme:
pip3 install sphinx_rtd_theme
(Bitbake rev: 29af1cd2a4f379096f5dd6f4cacc0a9c9f211e7c)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sphinx has a glossary directive. From the documentation:
This directive must contain a reST definition list with terms and
definitions. The definitions will then be referencable with the 'term'
role.
So anywhere in *any* manual, we can do :term:`VAR` to refer to an item
from the glossary, and create a link.
An HTML anchor is created for each term in the glossary, and can be
accessed as:
<link>/ref-variables.html#term-<NAME>
To convert to a glossary, we needed proper indentation (e.g. added 3
spaces to each line)
(Bitbake rev: e8359fd85ce0358019e2a32b4c47ba76613f48f0)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit is autogenerated pandoc to generate an inital set
of reST files based on DocBook XML files.
A .rst file is generated for each .xml files in all manuals with this
command:
cd <manual>
for i in *.xml; do \
pandoc -f docbook -t rst --shift-heading-level-by=-1 \
$i -o $(basename $i .xml).rst \
done
The conversion was done with: pandoc 2.9.2.1-91 (Arch Linux).
Also created an initial top level index file for each document, and
added all 'books' to the top leve index.rst file.
The YP manuals layout is organized as:
Book
Chapter
Section
Section
Section
Sphinx uses section headers to create the document structure.
ReStructuredText defines sections headers like that:
To break longer text up into sections, you use section headers. These
are a single line of text (one or more words) with adornment: an
underline alone, or an underline and an overline together, in dashes
"-----", equals "======", tildes "~~~~~~" or any of the
non-alphanumeric characters = - ` : ' " ~ ^ _ * + # < > that you feel
comfortable with. An underline-only adornment is distinct from an
overline-and-underline adornment using the same character. The
underline/overline must be at least as long as the title text. Be
consistent, since all sections marked with the same adornment style
are deemed to be at the same level:
Let's define the following convention when converting from Docbook:
Book => overline === (Title)
Chapter => overline *** (1.)
Section => ==== (1.1)
Section => ---- (1.1.1)
Section => ~~~~ (1.1.1.1)
Section => ^^^^ (1.1.1.1.1)
During the conversion with pandoc, we used --shift-heading-level=-1 to
convert most of DocBook headings automatically. However with this
setting, the Chapter header was removed, so I added it back
manually. Without this setting all headings were off by one, which was
more difficult to manually fix.
At least with this change, we now have the same TOC with Sphinx and
DocBook.
(Bitbake rev: 6bf6c8d63787aed7624793c24af3fa603b5ac961)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used sphinx-quickstart to generate top level config and
Makefile.sphinx, to allow side by side DocBook and Sphinx
co-existence.
(Bitbake rev: 84ccba0f4aff91528f764523fe1205a354c889ed)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Blocks SIGINT in the worker processes to prevent them from running the
parent process signal handler, which causes them to deadlock under
certain circumstances.
[YOCTO #14034]
(Bitbake rev: 9f4207f4b598f549cbd4159841c720276736f23b)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We're either hitting rate limiting with freedesktop.org or the
servers have intermittent network connections. Use our own mirror
of these repositories instead.
(Bitbake rev: a1b7ab5c9d5e64969f5ca0e41c0ac13c723e3761)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We're seeing leaking open socket connections when errors occur and
tinfoil is in use. Improve the exception handling so the sockets
are closed even if exceptions occur, allowing more robust behaviour
when things go wrong.
(Bitbake rev: cefbec9ff47ca973a74ec7300cd736f3e0f0bce0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After the recent init changes, if a client disconnects before issuing a
command, the cooker can break in the reset handlers. Add some guards
in the code to prevent this.
(Bitbake rev: 12605e30e4c4e1ae6a67c97363b892ebf0b9566c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The following code costs a lot of time when there are lot of layers and recipes:
for collection in collections:
collection_res[collection] = d.getVar('BBFILE_PATTERN_%s' % collection) or ''
My build has more than 100 layers and 3000 recipes, which calls d.getVar() 300K
(3000 * 100) times and makes 'bitbake-layers show-recipes' very slow, add a
keyword argument to get_file_layer() can fix the problem, it can save about 90%
time in my build (6min -> 40s).
(Bitbake rev: f08a6601c9bb09622855d62e1cedb92fafd2f71d)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This can make "$ bitbake-layers show-recipes" save about 60% time (14min ->
6min) in my build (more than 3000 recipes)
The command "bitbake-layers show-recipes" calls bb.utils.get_file_layer() with
each recipe, and get_file_layer() compare the file with each item in BBFILES
which makes it very time consuming when there are a lot of recipes and items in
BBFILES. So use BBFILES_PRIORITIZED and exit when file is matched, it doesn't
make sense to go on the loop when file is matched.
And use fnmatchcase to replace of fnmatch since the comparison should be
case-sensitive.
(Bitbake rev: 8d64181d29dc262e066a6114dd51e5f2d04f47de)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The original code saved BBFILES back to BBFILES without any changes which isn't
usefule, so remove that line. Now save prioritized BBFILES to
BBFILES_PRIORITIZED which can accelerate the query a lot for the one which
relies on it such as bb.utils.get_file_layer().
(Bitbake rev: 49bdb5dfa57b41b3ed399961e947c404f9195998)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
