assetTrue is a poor choice for "x in y" since assertIn gives much more
useful output upon failure.
Change such inserts to assertIn or assertEqual to make errors easier
to debug.
(Bitbake rev: dde78e0ff8af872fdc5cdf5354174fc713141102)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
GitHub Codespaces allow users to use GitHub-hosted pre-configured Docker
containers to work on GitHub repositories in VS Code within the browser
or using a locally running instance of VS Code [1].
Users can define access permissions for Codespaces containers, allowing
them to clone or work with other private repositories [2].
To do this, a git credential helper is injected into the container at
/.codespaces/bin/gitcredential_github.sh, which contains the following
lines:
#!/bin/sh
echo protocol=https
echo host=github.com
echo path=
echo username=PersonalAccessToken
echo password=$GITHUB_TOKEN
The `GITHUB_TOKEN` environment variable is automatically set when the
Codespaces container is created. If authorized by the user, it has the
specified permissions. It is thus required to be able to fetch private
git repos from a Codespace container, so add it to `FETCH_EXPORT_VARS`
to make it available for invocations of git.
[1] https://docs.github.com/en/codespaces/overview
[2] https://docs.github.com/en/codespaces/managing-your-codespaces/managing-repository-access-for-your-codespaces
(Bitbake rev: 71ca83bdd541f70737a3a85f05e40c222611af2f)
Signed-off-by: Oliver Kästner <git@oliver-kaestner.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
On faster machines these tests are failing as the tests are running
before the page has rendered. Add appripriate wait calls.
(Bitbake rev: c3a425ac3ccafa7b06b319c6a525773d04a2ddac)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The WebDriverException can occur if elements are not present in the DOM
yet. Catch and handle this. It is our most frequently occuring failure
in automated testing now.
(Bitbake rev: 006173cbd32116ff1cea59b2c99eead807be39bb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We should release locks in the opposite order to the order we locked
them in to avoid potentially problematic behaviour. This hasn't been
observed with our current usage and is just a preventative improvement.
(Bitbake rev: 4fbef87adaca2df8c1e877aec5588c2e5466f252)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of PRSERVER_UPSTREAM.
The intended variable name is PRSERV_UPSTREAM, as
already used in lib/prserv/serv.py, an consistently
with the PRSERV_HOST variable name.
(Bitbake rev: b0c277f16f9fae51914024c1daecd5d3e4fac5c2)
Signed-off-by: Michael Opdenacker <michael.opdenacker@rootcommit.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Prevents the following warning in bitbake-cookerdaemon.log:
2386729 10:54:16.195427 Exiting (socket: True)
2386729 10:54:16.201065 Exiting as we could obtain the lock
sys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name='/home/laplante/main_yocto/build/bitbake-cookerdaemon.log' mode='a+' encoding='UTF-8'>
(Bitbake rev: 8dbf1ec8139d9dd7f52c1773cccbe7696b3ec1b4)
Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
THe test fails on faster systems as we need to wait. Fix that and
a typo.
(Bitbake rev: 749b561196b0b1b388051b345fee2efb6ce68ff7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Using a hardcoded database ID for a recipe is a bad idea and causes tests
to fail when we change data/versions.
Remove magic numbers.
(Bitbake rev: 6e3a958666ae4821cfa1d43c064a2294e6881638)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Consistently use meta-poky as the test target, instead of a magic number
ID which can vary.
Fix incorrect comments.
Also ensure the viewport is scrolled to the top when changing tabs.
This breaks if other layers are selected like openembedded-core due
to the number of recipes.
(Bitbake rev: 0bcb7b4a8d8e89647aa6179cc61e5eb55d433585)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
"False is not True" is not a helpful error messgae. "XXX not in YYY" is
much more helpful. Tweak the test to give more meaninful failure messages
using assertIn rather than assertTrue.
(Bitbake rev: 6292b5a81ce1f3cf35980d0f669a1b52f3c56695)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When the fixtures were updated for the styhead release, the test cases
were not updated to match. Fix this.
(Bitbake rev: d40bd9d35fa3850e77a1e349dab2308a54d70d60)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In python 3 we should always have named tuples in this context. Drop
the obsolete exception handling.
(Bitbake rev: c11c9f18dad1cf4cb21e3bd873338ecc95652062)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It was never a great solution to persisting data and there are much better
ones now. The last user has been replaced so drop the code and tests.
(Bitbake rev: 681a7516e9f7027e0be6f489c54a7a5e19fa9f06)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The sqlite connection handling is causing problems with python 3.13. The
connection can be closed at gc time which causing warnings and those
can appear at 'random' points and break output, causing weird failures
in different tinfoil tools and other tests.
Using sqlite as an IPC was never a great idea so drop that usage entirely
and just use the standard cache mechanism we already have for other
situations.
(Bitbake rev: fdc55bb649cb77456d0ac48a9600ef289a52af18)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
>From re on python 3.13 onwards: "Passing count and flags as positional arguments is deprecated.
In future Python versions they will be keyword-only parameters."
Avoid the warning.
(Bitbake rev: 016d872dc3c36792af5c84da1d35a5c86996d795)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The COW tests started failing on python 3.13. Looks like it is time
to fix the FIXME and drop the magic numbers!
(Bitbake rev: 2e6608cec508b3b9bab3530f83e70665ff638182)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The pipes module is removed in python 3.13. It was already using the
quote function from shlex so use that directly instead.
The module already imports shlex too so it is an easy substitution.
(Bitbake rev: 9ef4f2a30127455562e38e869793a723eed6c07c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When copying files as part of the unpack we currently use cp -p, which
is a shortcut for --preserve=mode,ownership,timestamps.
We do want to preserve timestamps, because some fetchers set these
explicitly.
We don't care about ownership. If the files are owned by us then they
ill remain owned by us, and if they're not then the attempt to change
ownership will be silently ignored. In a shared DL_DIR where files have
group ownership this group access isn't relevant in the single-user build
tree.
We do want to preserve executable bits in the mode, but cp always does
this. The difference between --preserve=mode and no --preserve is that
the mode isn't preserved exactly (no sticky bits, no suid, umask is
applied) but this also isn't a relevant difference in a build tree.
Also expand the arguments to be clearer about what options are being
passed.
The impetus for this is that coreutils 9.4 includes a change in gnulib[1]
and will now try to preserve permission-based xattrs if asked to preserve
the mode. This can result in cp failing when copying a file from a
NFSv4 server with ACLs stored in xattrs to a non-NFS directory where
those xattrs cannot be written:
cp: preserving permissions for ‘./jquery-3.7.1.js’: Operation not supported
The error comes from the kernel refusing to write a system.nfs4_acl
xattr to a file on ext4.
This situation doesn't appear on all systems with coreutils 9.4, at the
time of writing it fails on Ubuntu 24.04 onwards but not Fedora 40. This
is because /etc/xattr.conf is used to determine which xattrs describe
permissions, and Fedora 40 has removed the NFSv4 attributes[2].
Also, use long-form options to make the cp command clearer.
[1] https://github.com/coreutils/gnulib/commit/eb6a8a4dfb
[2] https://src.fedoraproject.org/rpms/attr/blob/rawhide/f/0003-attr-2.4.48-xattr-conf-nfs4-acls.patch
[ YOCTO #15596 ]
(Bitbake rev: 2f35dac0c821ab231459922ed98e1b2cc599ca9a)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add test cases for a module path consisting of only a hostname and add
checksum verification of the unpacked go.mod files.
(Bitbake rev: 9380859b59923dee17469348f472a22e11be1779)
Signed-off-by: Christian Lindeberg <christian.lindeberg@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add styhead. Also fix a typo in the usage instructions.
(Bitbake rev: d3c84198771b7f79aa84dc73061d8ca071fe18f3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When calculating the module name for a gomod URI with only a hostname,
e.g.:
gomod://go.opencensus.io;version=v0.24.0;sha256sum=203a767d7f8e7c1ebe5588220ad168d1e15b14ae70a636de7ca9a4a88a7e0d0c
the non-existing path would actually be treated as "/", which resulted
in a trailing slash being added to the module name preventing the unpack
method from correctly locating the go.mod file.
(Bitbake rev: f0e02e1de4d649e647e4ab61341042dd38d0eeb0)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This reverts commit a4f42e396e2942fde94b8b4944487c1c45f7a295.
Unfortunately configparser is a bit more strict and fails to parse e.g.:
e104dd7f27/.gitmodules
[submodule "components/bt/controller/lib_esp32"]
path = components/bt/controller/lib_esp32
url = ../../espressif/esp32-bt-lib.git
The url is using 8 spaces while path 1 tab causing:
Exception: configparser.NoOptionError: No option 'url' in section: 'submodule "components/bt/controller/lib_esp32"'
It was fixed in:
62ca8e2fb4 (diff-fe7afb5c9c916e521401d3fcfb4277d5071798c3baf83baf11d6071742823584)
but mcuboot is using a bit older esp-idf revision in:
https://github.com/ATmobica/mcuboot/blame/main/.gitmodules
and mcuboot is then used as submodule in:
https://github.com/project-chip/connectedhomeip/blob/master/.gitmodules
so it might take a while for the fix to be propagated everywhere.
Not sure how common these issues are, but configparser parses
"url = ../../espressif/esp32-bt-lib.git" as 2nd line of "path"
value (because it's indented differently) while git submodule
and old gitsm implementation parses it as separate path and
url keys.
(Bitbake rev: d9eb2650256292a524a0c0bb1c0562d87bedcbc4)
Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously, incrementing "0.9" would result in "0.1.0", which
generally gets recognised as a lower version number. Even more
surprising, incrementing "0.99" returned "0.1.0.0".
This is due to the behaviour of the list function on a string
object; it adds each character as an element in a new list,
causing the new string '10' to become the list [ '1', '0' ].
Instead of converting a string to a list, add the string to a
new list, and concatenate it with the existing list slice. And
provide test cases for "0.9" -> "0.10" and related edge cases.
(Bitbake rev: 96ddeefa88ff4c37e9ea096726a7cdca5b5b4572)
Signed-off-by: Dan McGregor <dan.mcgregor@usask.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
.gitmodules is basically ini-style, so use configparser instead of manually
parsing by hand.
(Bitbake rev: a4f42e396e2942fde94b8b4944487c1c45f7a295)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a go module fetcher for downloading module dependencies to the
module cache directly from a git repository. The fetcher can be used
with the go-mod class in OE-Core.
A module dependency can be specified with:
SRC_URI += "gomodgit://golang.org/x/net;version=v0.9.0;srcrev=..."
(Bitbake rev: 29ff38ccf0d5389a5bee81e252a78548361a9d7c)
Signed-off-by: Christian Lindeberg <christian.lindeberg@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a go module fetcher for downloading module dependencies to the
module cache from a module proxy. The fetcher can be used with the
go-mod class in OE-Core.
A module dependency can be specified with:
SRC_URI += "gomod://golang.org/x/net;version=v0.9.0;sha256sum=..."
(Bitbake rev: 5ff4694bf305e266ebf0abab5d9745c6b6d07d67)
Signed-off-by: Christian Lindeberg <christian.lindeberg@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is a subtle issue with full pathnames being included in the
file checksums since the sorting might be different depending upon
how layers are being setup causing hash mismatches for recipes appeneded
from other layers with differing directory layouts.
Avoid this by filtering out to the path basename which is what is written
into the sig data anyway later in the code.
(Bitbake rev: 83acc21cdfdb410082c0871ac7693d29a7c5627d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow the metadata to define code visitor expressions which mean that
custom dependencies can be handled in function libraries.
An example is the qa.handle_error function in OE which can set something
like:
"""
def handle_error_visitorcode(name, args):
execs = set()
contains = {}
warn = None
if isinstance(args[0], ast.Constant) and isinstance(args[0].value, str):
for i in ["ERROR_QA", "WARN_QA"]:
if i not in contains:
contains[i] = set()
contains[i].add(args[0].value)
else:
warn = args[0]
execs.add(name)
return contains, execs, warn
handle_error.visitorcode = handle_error_visitorcode
"""
Meaning that it can have contains optimisations on ERROR and WARN_QA
instead of hard dependencies.
One drawback to this solution is the parsing order. Functions with
visitorcode need to be defined before anything else references them
or the visitor code will not function for the earlier references.
(Bitbake rev: 5bd0c65c217394cde4c8e382eba6cf7f4b909c97)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The '@types/ramda' [1] npm package has recently gained a dependency on
the 'types-ramda' [2] npm package. Both have the same version number.
The name mangling results in the tarballs of both packages sharing the same
name, but different contents.
Fix that by accepting '@' as valid character in the package name,
resulting in one package named @types-ramda and one called types-ramda.
[1]: https://www.npmjs.com/package/@types/ramda
[2]: https://www.npmjs.com/package/types-ramda
(Bitbake rev: 7c9573cb6ea2081bc585eb65267f3124fd4d7e43)
Signed-off-by: Leonard Göhrs <l.goehrs@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The changes of 1ab1d36c0af6fc58a974106b61ff4d37da6cb229 added calls to "gsutil stat" to avoid unhandled exceptions, however:
- in the case of checkstatus() this is redundant with the call to self.gcp_client.bucket(ud.host).blob(path).exists() which already returns True/False
and does not throw an exception in case the file does not exist.
- Also the call to gsutil stat is much slower than using the python client to call exists() so we should not replace the call to exists() with a call to gsutil stat.
- I think the intent of calling check_network_access in checkstatus() was to error-out in case the error is disabled. We can rather change the string "gsutil stat" to something else to make the code more readable.
- add a try/except block in download() instead of the extra call to gsutil
[RP: Tweak to avoid import until needed so google module isn't required for everyone]
(Bitbake rev: dd120f630e9ddadad95fe83728418335a14d3c3b)
Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We fetch npm dependencies from the npm-shrinkwrap.json file. They can
point to a package on the NPM registry with a version field, or to a
git/http/file URL with the resolved field. Such packages are allowed not
to declare a registry version field because they may not have been
published to the NPM registry. The previous implementation refuses to
fetch such packages and throws an error.
The resolved field contains the exact source, including the revision,
wich we can use to pass as SRC_URI to the git/http/file fetcher. The
integrity field is also mandatory for HTTP tarballs which will ensure
reproducibility. So even if the version field is not present, we are
still fetching a precise revision of the package.
Another commit published along this stack is also required in the npm
class to support these packages.
v5:
- improve commit message
v3:
- Split bitbake npmsw.py modification in another commit
Co-authored-by: Tanguy Raufflet <tanguy.raufflet@savoirfairelinux.com>
(Bitbake rev: 209982b5a3efc8081e65b4326bf9b64eef7f0ba0)
Signed-off-by: Tanguy Raufflet <tanguy.raufflet@savoirfairelinux.com>
Signed-off-by: Enguerrand de Ribaucourt <enguerrand.de-ribaucourt@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The NPM package.json documentation[1] states that git URLs may contain
a commit-ish suffix to specify a specific revision. When running
`npm install`, this revision will be looked for on any branch of the
repository.
The bitbake implementation however translates the URL stored in
package.json into a git URL to be fetch by the bitbake git fetcher. The
bitbake fetcher git.py, enforces the branch to be master by default. If
the revision specified in the package.json is not on the master branch,
the fetch will fail while the package.json is valid.
To fix this, append the ";nobranch=1" suffix to the revision in the git
URL to be fetched. This will make the bitbake git fetcher ignore the
branch and respect the behavior of `npm install``.
This can be tested with the following command:
$ devtool add --npm-dev https://github.com/seapath/cockpit-cluster-dashboard.git -B version
Which points to a project which has a package.json with a git URL:
```json
"devDependencies": {
"cockpit-repo": "git+https://github.com/cockpit-project/cockpit.git#d34cabacb8e5e1e028c7eea3d6e3b606d862b8ac"
}
```
In this repo, the specified revision is on the "main" branch, which
would fail without this fix.
[1] https://docs.npmjs.com/cli/v10/configuring-npm/package-json#git-urls-as-dependencies
Co-authored-by: Tanguy Raufflet <tanguy.raufflet@savoirfairelinux.com>
(Bitbake rev: 37a35adf7882f231c13643dbf9168497c6a242a1)
Signed-off-by: Tanguy Raufflet <tanguy.raufflet@savoirfairelinux.com>
Signed-off-by: Enguerrand de Ribaucourt <enguerrand.de-ribaucourt@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When tasks fail, it's very frustrating to have to scroll up to find the
log path(s). Many of us have the muscle memory to navigate to the 'temp'
directories under tmp/work/, but new users do not.
This change enhances the final summary to include log paths (reported
via bb.build.TaskFailed events). Here's an example:
NOTE: Tasks Summary: Attempted 856 tasks of which 853 didn't need to be rerun and 3 failed.
Summary: 3 tasks failed:
virtual:native:/home/chris/repos/poky/meta/recipes-core/ncurses/ncurses_6.5.bb:do_fetch
log: /home/chris/repos/poky/build/tmp/work/x86_64-linux/ncurses-native/6.5/temp/log.do_fetch.1253462
/home/chris/repos/poky/meta/recipes-core/ncurses/ncurses_6.5.bb:do_fetch
log: /home/chris/repos/poky/build/tmp/work/core2-64-poky-linux/ncurses/6.5/temp/log.do_fetch.1253466
virtual:nativesdk:/home/chris/repos/poky/meta/recipes-core/ncurses/ncurses_6.5.bb:do_fetch
log: /home/chris/repos/poky/build/tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-ncurses/6.5/temp/log.do_fetch.1253467
Summary: There were 3 WARNING messages.
Summary: There were 6 ERROR messages, returning a non-zero exit code.
Each log is rendered as a clickable hyperlink in the terminal. See
https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda
(Bitbake rev: 2852a478ab03a482989c3a7e247860ab4f0e9f3e)
Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When getting errors during build, they would be printed using repr(), which
doesnt have a lot of context in some cases.
For example FileNotFoundError(2, "file or directory not found"), would be
printed, without the path of the file not found.
This changes the build logging to use str() instead, which according to
the spec is fore human readable strings, whereas repr() is for string
representations that can be be used as valid python.
(Bitbake rev: 2a97024b8b9245ec47deace011a7560a25491207)
Signed-off-by: Troels Dalsgaard Hoffmeyer <tdah@bang-olufsen.dk>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
"addtask do_XXX before YYY after ZZZ "
where YYY or ZZZ is missing the "do_" prefix don't work as expected. Ajust the
code so that it doesn't just silently do the wrong thing but works as expected.
Expand a test case to cover this.
(Bitbake rev: 21670b9bb8936ec44aedff26163948bbc2ceb44a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The recent addtask improvement to handle comments complicated the regex significantly
and there are already a number of corner cases in that code which aren't handled well.
Instead of trying to complicate the regex further, switch to code logic instead. This
means the following cases are now handled:
* addtask with multiple task names
* addtask with multiple before constraints
* addtask with multiple after constraints
The testcase is updated to match the improvements.
(Bitbake rev: 417016b83c21fca7616b2ee768d5d08e1edd1e06)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Within tinfoil, the user can write to the configuration data but it won't
cause the data_hash checksum to be re-written, meaning cached parsing
data can be reused when it would now be incorrect.
Abstract out the data_hash code and add it to the invalidateCaches
command, called by tinfoil.modified_files() meaning that tinfoil can
instruct bitbake to update the caches and re-parse if necessary.
Also move the data_hash entirely into databuilder and drop the copy
in cooker as obsolete and not needed.
(Bitbake rev: d9ee77829f693ce75348fa64f406fcecfe4989aa)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I can't spot any users of this function and it is poking at variables
inside cooker that could and are about to change so drop it.
(Bitbake rev: 52491808706e9e58b5e6b59d2d792353d77c8b66)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Technically our syntax would allow for comments after an addtask/deltask.
Currently these get silently processed in various ways by the code which
is bad.
Tweak the regex to drop any comments and add test cases to ensure this
continues to work in the future.
[YOCTO #15250]
(Bitbake rev: 64f8796e55a8826ffec0d76b993c8256713f67a3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We missed on adding nanbield to the release manuals. It's now EOL so
let's add it directly to the oudated release manuals section.
(Bitbake rev: b891878a7f08b15ee5d6d037d99fbc769cc905e4)
Signed-off-by: Quentin Schulz <quentin.schulz@cherry.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixed:
- BBMULTICONFIG = "qemux86-64 qemuarm64" and more than 70 layers in BBLAYERS
$ bitbake -p -P
Check profile.log.processed, the record() cost more than 20 seconds, it is less
than 1 second when multiconfig is not enabled, and there would be the following
error when more muticonfigs are enabled:
Timeout while waiting for a reply from the bitbake server
Don't change the type of loginfo['detail'] or re-assign it can make record()
back to less than 1 second, this won't affect COW since loginfo is a mutable
type.
The time mainly affected by two factors:
1) The number of enabled layers, nearly 1 second added per layer when the
number is larger than 50.
2) The global var such as USER_CLASSES, about 1 ~ 2 seconds added per layer
when the layers number is larger than 50.
(Bitbake rev: 0596aa0d5b0e4ed3db11b5bd560f1d3439963a41)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Enforce the default remote name to "origin", as assumed in numerous
places.
This prevents build failures in case the system/user configuration sets
this to a different value.
(Bitbake rev: 1d7360031164f04887c792fb0b2dd86c6ccfcc23)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update the test cases since the implementation is changed:
* test_shallow_multi_one_uri()
The a_branch and v0.0 had the same revision, and it required fetch a_branch
and remove histories of v0.0 which were conflicted, and bitbake reported:
fatal: no commits selected for shallow requests
Make a_branch and v0.0 have different revs to fix the problem.
And now the 'rev^' is not needed, so update self.assertRevCount() as well.
* test_shallow_multi_one_uri_depths()
Update self.assertRevCount(), now git only fetches the required revs.
* test_shallow_fetch_missing_revs()
The command is:
$ git fetch --shallow-exclude=v0.0 master
But master and v0.0 uses the same revision, so there is no commit to fetch.
* test_shallow_fetch_missing_revs_fails()
Two unneeded committs are not fetched now:
- rev^
- One not specified or required tag.
So update self.assertRevCount()
(Bitbake rev: 48eff9d9a660ad6b9bd8b53a7dcec600ef42b1d1)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch can make the following settings much more faster:
BB_GIT_SHALLOW = "1"
BB_GENERATE_MIRROR_TARBALLS = "1"
* The previous implementation was:
- Make a full clone for the repo from local ud.clonedir
- Use git-make-shallow to remove unneeded revs
It was very slow for recipes which have a lot of SRC_URIs, for example
vulkan-samples and docker-compose, the docker-compose can't be done after 5
hours.
$ bitbake vulkan-samples -cfetch
Before: 12 minutes
Now: 2 minutes
$ bitbake docker-compose -cfetch
Before: More than 300 minutes
Now: 15 minutes
* The patch uses git shallow fetch to fetch the repo from local
ud.clonedir:
- For BB_GIT_SHALLOW_DEPTH: git fetch --depth <depth> rev
- For BB_GIT_SHALLOW_REVS: git fetch --shallow-exclude=<revs> rev
Then the git repo will be shallow, and git-make-shallow is not needed any
more.
And git shallow fetch will download less commits than before since it doesn't
need "rev^" to parse the dependencies, the previous code always need 'rev^'.
(Bitbake rev: a5a569c075224fe41707cfa9123c442d1fda2fbf)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
