- Address the absence of an initial full bare clone
- Utilize the initial shallow clone
- Modify existing test cases for this behavior
- Remove incompatible test cases
(Bitbake rev: 599fedacd7782dcb52825c22200f35344c102548)
Signed-off-by: Stefan Koch <stefan-koch@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to avoid a potentially confusing backtrace, check that the mcdepend
is valid when we add it.
Add a test case to ensure invalid configurations are caught and trigger an
error.
[RP: Reworked test case to simplify and improve code]
(Bitbake rev: ff523497270f37b484b44a4445c2194791bcb6ff)
Signed-off-by: Mark Hatle <mark.hatle@amd.com>
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Check default flags are correctly returned by getVarFlags and check all flags
are returned when internalflags is True.
Check delVarFlags also removes default value.
Check all flags are removed after delVar.
Run the test with:
$ bitbake-selftest -v bb.tests.data.TestFlags
test_delflag (bb.tests.data.TestFlags.test_delflag) ... ok
test_delvar (bb.tests.data.TestFlags.test_delvar) ... ok
test_setflag (bb.tests.data.TestFlags.test_setflag) ... ok
----------------------------------------------------------------------
Ran 3 tests in 0.000s
OK
This is a test case for [YOCTO #15685]
(Bitbake rev: ff8cae735cf489373af1aac7ee233d7b82d483d3)
Signed-off-by: Louis Rannou <louis.rannou@non.se.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The password must be quoted as per RFC3986 because it can contain
reserved characters.
(Bitbake rev: 5c53dbf0ad4385e1de7f2eef66565a1f05dae67a)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Do not use upper case hostnames because the hostname is case insensitive
and maybe decoded into lower case to follow the common style.
(Bitbake rev: 554b7048412c4c67bf895a8b98822b54ac3a66db)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Old code that parse variable names in assignment commands behave differently for
variables that ends with special symbol for single-character variable names and
multi-character variable names. For example:
A+="1" # Change variable ${A}, '+' glued to '='
A+ = "1" # Change variable ${A+}
+="1" # Change variable ${+}, the '+' symbol not part of assignment operator
+ = "1" # Change variable ${+}
New code would always assume that '.=', '+=', and ':=' is assignment operator.
As result code like the following would raise parsing error
+="value"
While code with extra spaces would work as before
+ = "value" # Change variable ${+}
This change allow to catch issues in code that generate bitbake configuration
files in a manner like "echo ${VARNAME}+=${VALUE} >> conf/local.conf"
(Bitbake rev: 93059aad13a12cd69d86368795c88e5349197d5d)
Signed-off-by: Nikolai Merinov <n.merinov@inango-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This test covers the handling of unclosed functions.
It tests that both whitespace and tabs generate the
correct exception if added before a closing bracket.
Additionally that a residue blocks generates a error
is tested as well.
[YOCTO #15470]
(Bitbake rev: 29e67acb87ae76879efe9688a69c961a96df10f1)
Signed-off-by: Savvas Etairidis <falital@hotmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The string value "default" for the default multiconfig is confusing since an
empty string is used pretty much everywhere in the code. Remove the few
remaining references to that to standarise.
This affects the default value of BB_CURRENT_MC and does have an impact
on metadata, particulalry bitbake.conf in openembedded-core. That said, the
number of bugs we'll avoid by trying to make "default" back to "" within
bitbake's code make fixing those extremely worthwhile.
(Bitbake rev: 0fa0d8d764bbeb8a44c47f79d7b849068d565199)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
From git 2.48 release notes:
"""
When "git fetch $remote" notices that refs/remotes/$remote/HEAD is
missing and discovers what branch the other side points with its
HEAD, refs/remotes/$remote/HEAD is updated to point to it.
"""
This means with git 2.48 onwards, there is a mystery "HEAD" revision
appearing in some of our shallow clone tests. We can avoid this by
using the same canonicalization as used for the reference revisions.
This resolves autobuilder failures on the Fedora 40 workers.
(Bitbake rev: c83444d1210740e27b1744d3aa7c5cad4e28db2f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Run the test with:
$ bitbake-selftest bb.tests.parse.ParseTest.test_parse_defaulttest
.
----------------------------------------------------------------------
Ran 1 test in 0.001s
OK
This is a test case for [YOCTO #15685]
(Bitbake rev: d12c80125d810df54379cc4f019a95ab9064ad2c)
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Reviewed-by: Ghislain Mangé <ghislain.mange@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The fetcher support entries with an embedded checksum like 'sha256sum'
in the SRC_URI. It adds the parameter 'name' as prefix to the checksums
if the parameter is set. This behavior is unexpected and leads to hacks
in fetchers. Fallback to the checksum without the useless prefix and
set the parameter 'name' in the gomod fetcher unconditional.
(Bitbake rev: 7a86c5a20ea2586f1ae240613644e065e7b21683)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The npm package lock and shrinkwrap file list bundled dependencies which
are supplied together with the parent dependency. The bundled
dependencies are marked by a flag. The flag and thereby test depends on
the lock file version. The old lock file version uses a `bundled` flag
and stores dependencies in the `dependencies` list. The new lock file
version uses an `inBundle` flag and stores dependencies in the
`packages` list.
(Bitbake rev: 34fd8ea6abe755e04220fe70b082aa620ae15f86)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The npmsw fetcher doesn't use the npm command. Remove the
`@skipIfNoNpm()` decorator from plain npmsw tests which don't use the
npm fetcher.
(Bitbake rev: 48f9964a19b8e2e3a73667c9889908eceecf7719)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adapt the npmsw tests to commit 84f102954e10 ("fetch/npmsw: The fetcher
shouldn't have any knowledge of S").
(Bitbake rev: 8194178291d9b47efb2079fe81b704ba6171efc1)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use the new listtasks() function (introduced in 185c4b) to avoid
accessing a private variable.
Also use assertSequenceEqual() directly as we don't really care about
the actual type returned, only that it's a sequence.
(Bitbake rev: 452289efc6d3608ceca321286fd5bf417a6e1bed)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
lz4c has been considered deprecated by upstream since at least 2018 [1]
and has been disabled by default recently [2]. openSUSE Tumbleweed is
no longer packaging the deprecated version and others will likely
follow.
Luckily, we're not using any legacy-specific options and, going back as
far as Ubuntu 16.04/Fedora 25, both lz4 and lz4cli are installed as part
of the same package
1. https://github.com/lz4/lz4/pull/553
2. https://github.com/lz4/lz4/pull/1479
(Bitbake rev: 907472034b344e4eb73cfd43059a413469f52e1c)
Signed-off-by: Justin Bronder <jsbronder@cold-front.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
GNOME gitlab has occasional downtime which impacts bitbake-selftest
and causes autobuilder failures. Switch to our own mirror for test
purposes to avoid those issues.
(Bitbake rev: 91e268b11ed683bd197026f9b36001f6d54ee05c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
GNOME gitlab has occasional downtime which impacts bitbake-selftest
and causes autobuilder failures. Switch to our own mirror for test
purposes to avoid those issues.
(Bitbake rev: 008808755ed6cfeb6c41273e69ce718f0833c26c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It was never a great solution to persisting data and there are much better
ones now. The last user has been replaced so drop the code and tests.
(Bitbake rev: 681a7516e9f7027e0be6f489c54a7a5e19fa9f06)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add test cases for a module path consisting of only a hostname and add
checksum verification of the unpacked go.mod files.
(Bitbake rev: 9380859b59923dee17469348f472a22e11be1779)
Signed-off-by: Christian Lindeberg <christian.lindeberg@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a go module fetcher for downloading module dependencies to the
module cache directly from a git repository. The fetcher can be used
with the go-mod class in OE-Core.
A module dependency can be specified with:
SRC_URI += "gomodgit://golang.org/x/net;version=v0.9.0;srcrev=..."
(Bitbake rev: 29ff38ccf0d5389a5bee81e252a78548361a9d7c)
Signed-off-by: Christian Lindeberg <christian.lindeberg@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a go module fetcher for downloading module dependencies to the
module cache from a module proxy. The fetcher can be used with the
go-mod class in OE-Core.
A module dependency can be specified with:
SRC_URI += "gomod://golang.org/x/net;version=v0.9.0;sha256sum=..."
(Bitbake rev: 5ff4694bf305e266ebf0abab5d9745c6b6d07d67)
Signed-off-by: Christian Lindeberg <christian.lindeberg@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
"addtask do_XXX before YYY after ZZZ "
where YYY or ZZZ is missing the "do_" prefix don't work as expected. Ajust the
code so that it doesn't just silently do the wrong thing but works as expected.
Expand a test case to cover this.
(Bitbake rev: 21670b9bb8936ec44aedff26163948bbc2ceb44a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The recent addtask improvement to handle comments complicated the regex significantly
and there are already a number of corner cases in that code which aren't handled well.
Instead of trying to complicate the regex further, switch to code logic instead. This
means the following cases are now handled:
* addtask with multiple task names
* addtask with multiple before constraints
* addtask with multiple after constraints
The testcase is updated to match the improvements.
(Bitbake rev: 417016b83c21fca7616b2ee768d5d08e1edd1e06)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Technically our syntax would allow for comments after an addtask/deltask.
Currently these get silently processed in various ways by the code which
is bad.
Tweak the regex to drop any comments and add test cases to ensure this
continues to work in the future.
[YOCTO #15250]
(Bitbake rev: 64f8796e55a8826ffec0d76b993c8256713f67a3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update the test cases since the implementation is changed:
* test_shallow_multi_one_uri()
The a_branch and v0.0 had the same revision, and it required fetch a_branch
and remove histories of v0.0 which were conflicted, and bitbake reported:
fatal: no commits selected for shallow requests
Make a_branch and v0.0 have different revs to fix the problem.
And now the 'rev^' is not needed, so update self.assertRevCount() as well.
* test_shallow_multi_one_uri_depths()
Update self.assertRevCount(), now git only fetches the required revs.
* test_shallow_fetch_missing_revs()
The command is:
$ git fetch --shallow-exclude=v0.0 master
But master and v0.0 uses the same revision, so there is no commit to fetch.
* test_shallow_fetch_missing_revs_fails()
Two unneeded committs are not fetched now:
- rev^
- One not specified or required tag.
So update self.assertRevCount()
(Bitbake rev: 48eff9d9a660ad6b9bd8b53a7dcec600ef42b1d1)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Upstream changed their urls, update our test to match.
(Bitbake rev: dc391b86540ec5e0a0f1d811c776a22d443b1c06)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
On Fedora40, "localhost" sometimes resolves to ::1 and sometimes to 127.0.0.1
and python only binds to one of the addresses, leading to test failures.
Use 127.0.0.1 explicitly to avoid problems of the name resolution, we're trying
to test things other than the host networking.
(Bitbake rev: 9adc6da42618f41bf0d6b558d62b2f3c13bedd61)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Tests quotes around `` and $() expansions, nested and multiple
expansions, and that escaped quotes are treated as characters by the
parser.
(Bitbake rev: d98130cb4d500c495bc692c56dde3e019f36320a)
Signed-off-by: Antonin Godard <antoningodard@pm.me>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add return false to supports_checksum for svn fetcher which fhis
fixes MIRROR usage for svn uris. Also add a testcase.
[YOCTO #15473]
(Bitbake rev: 21cfc7ae9a19f39ac8904e1c3466e7e499ac523f)
Signed-off-by: Kari Sivonen <kari.sivonen@live.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Whilst typically the URI query is a list of key-value pairs, that's not
actually required by the URI specification.
For example: http://example.com/foo?bar is a valid query, but this will
result in the fetcher raising an exception:
File "bitbake/lib/bb/fetch2/__init__.py", line 265, in __init__
self.query = self._param_str_split(urlp.query, "&")
File "bitbake/lib/bb/fetch2/__init__.py", line 293, in _param_str_split
for k, v in [x.split(kvdelim, 1) for x in string.split(elmdelim) if x]:
ValueError: not enough values to unpack (expected 2, got 1)
In this case the query is just "bar", but the fetcher is trying to split
it into a key-value pair.
The URI object exposes the parsed query explicitly as a dictionary of
key-value pairs, so we have to be a little creative here: if a value is
None then it isn't a key-value pair, but a bare key.
Fix this by handling elements without the deliminator in _param_str_split()
(by assigning the value to None), and handle a None value when formatting
the query in _param_str_join().
(Bitbake rev: eac583bd4c46f3bb9661852cb6a1448f16147ff1)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The current implementation only performs a git lfs fetch alongside of a
regular git fetch. This causes issues when the downloaded revision is
already part of the fetched repository (e.g. because of moving back in
history or the updated revision already being part of the repository at
the time of the initial clone).
Fix this by explicitly checking whether the required LFS objects are
available in the downloade directory before confirming that a downloaded
repository is up-to-date.
This issue previously went unnoticed as git lfs would silently fetch the
missing objects during the `unpack` task. With network isolation turned
on, this no longer works, and unpacking fails.
(Bitbake rev: cfae1556bf671acec119a6c8bbc4b667a856b9ae)
Signed-off-by: Philip Lorenz <philip.lorenz@bmw.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Before, everything up to the last slash was removed when extracting the
names of the tags. This would lead to that a tag such as "agent/11.0.0"
would be incorrectly identified as "11.0.0", which would then be treated
as a correct version matching "^(?P<pver>\d+(\.\d+)+)".
(Bitbake rev: 8b21024b9966d5158ac4a77e87ffb935c2a57764)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In some cases, the version found by latest_versionstring() may be higher
than the real version. Make it possible to specify a maximum version so
that this case can be detected.
(Bitbake rev: 9134d4777109bc78410c3e641420d9a78b485e33)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We had issue when BB_ALLOWED_NETWORKS is set and `bitbake grpc-native -c fetch'
failed even with all contents available in PREMIRRORS.
Add a test case to ensure no regression in the future.
(Bitbake rev: 80c91ceb81b1cae203067af58d3f1fe9c619ae83)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fetch from premirror few times to emulate multiple machines sharing same
clonedir or few rebuilds of the package from (pre)mirror
Regression test for [Yocto #15369]
(Bitbake rev: 7fcbac574c68f16b95ab7abb2874931d168d3c9e)
Signed-off-by: Pavel Zhukov <pavel@zhukoff.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When git is configured with safe.bareRepository=explicit [1], the
bitbake selftests fail miserably. LWN has an article about the
problem that this configuration option addresses and why it is useful
in [2].
It also seems that it is being rolled out in some environments as a
default for users.
In order to allow having this configuration turned on for a user's
environment in general, the fetcher has to be tought to use --git-dir=
for all relevent git operations.
The alternative, implemented here, is to forcibly turn off that option
for all git operations. In the future, we could look into converting
these to using the --git-dir= command line argument instead.
Link: https://git.kernel.org/pub/scm/git/git.git/tree/Documentation/config/safe.txt#n1 [1]
Link: https://lwn.net/Articles/892755/ [2]
(Bitbake rev: a45e14a7343e36101e45639931322e5649587f57)
Signed-off-by: André Draszik <andre.draszik@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* github no longer provides support for subversion clients:
https://docs.github.com/en/enterprise-server@3.11/get-started/working-with-subversion-on-github/support-for-subversion-clients
it was shut down on 2024-01-08:
https://github.blog/2023-01-20-sunsetting-subversion-support/
and this test was now failing with:
======================================================================
ERROR: test_external_svn (bb.tests.fetch.SVNTest.test_external_svn)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/OE/layers/bitbake/lib/bb/tests/fetch.py", line 1287, in test_external_svn
fetcher.download()
File "/OE/layers/bitbake/lib/bb/fetch2/__init__.py", line 1896, in download
raise FetchError("Unable to fetch URL from any source.", u)
bb.fetch2.FetchError: Fetcher failure for URL: 'svn:///tmp/bitbake-fetch-zdvedqt_/svnfetch_localrepo_wup8mgn6/project;module=trunk;protocol=file;externals=allowed;rev=2'. Unable to fetch URL from any source.
Stdout:
Fetch svn:///tmp/bitbake-fetch-zdvedqt_/svnfetch_localrepo_wup8mgn6/project;module=trunk;protocol=file;externals=allowed;rev=2
Failed to fetch URL svn:///tmp/bitbake-fetch-zdvedqt_/svnfetch_localrepo_wup8mgn6/project;module=trunk;protocol=file;externals=allowed;rev=2, attempting MIRRORS if available
Fetcher failure: Fetch command export PSEUDO_DISABLED=1; /usr/bin/env svn --non-interactive --trust-server-cert co --no-auth-cache -r 2 file:///tmp/bitbake-fetch-zdvedqt_/svnfetch_localrepo_wup8mgn6/project/trunk@2 trunk failed with exit code 1, output:
A trunk/README.md
U trunk
Checked out revision 2.
svn: warning: W205011: Error handling externals definition for 'trunk/bitbake':
svn: warning: W170013: Unable to connect to a repository at URL 'https://github.com/PhilipHazel/pcre2.git'
svn: E205011: Failure occurred processing one or more externals definitions
in the rare cases where subversion was still installed on the host
running bitbake-selftest :).
to avoid this use still alive repository from https://svn.apache.org/
and pick something rather small and only the trunk subdirectory which
is fast to fetch:
svn co https://svn.apache.org/repos/asf/serf/trunk
takes just 2 sec here
adjust expected dir/file to use "protocols/fcgi_buckets.h" instead of
"trunk/README"
(Bitbake rev: a735898abcf056f897c9350bb128a5637e6b4617)
Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you have two classes, both of which set EXPORT_FUNCTIONS for the same funciton
and a standard funciton definition for the function that is exported, the export
function can sometimes overwrite the standard one.
The issue is that the internal flag the code uses isn't ovweritten if the variable
is giving a new value. Fix the issue by using a comment in the code that is injected
so that we know if it is ours or not.
Also add some testing for EXPORT_FUNCTIONS, not perfect but a start.
(Bitbake rev: 66306d5151acb0a26a171c338d8f60eb9eb16c6b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add test_lineno_in_eventhandler to test lineno in eventhandler.
(Bitbake rev: 4e5de537bebb68180c5755858c81b095eb9ae2f6)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Found a duplicate test, added _2 suffix to one, 74 tests now pass up from 73.
(Bitbake rev: ae2a19dadb4f3065b8731a61f45f29e6a70af402)
Signed-off-by: Richard Haar <rh@richhaar.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When downloading a Git repository containing an LFS, the Git hooks
are not always initialized correctly to perform the download. This
change updates the Git downloader to run the "git lfs install" command
in order to "smudge" the LFS files. The tests have also been updated
to do checks for situations in which git lfs is not installed, as
the application was required to be installed for any of the tests
to run previously.
The Git LFS functionality was working to some degree previously,
but this change also updates the fetcher to allow LFS downloads for
nobranch SRC_URIs.
(Bitbake rev: 05f8529fb439db93d85a892704b6f2f0ac0c9217)
Signed-off-by: Desone Burns II <dburns@seegrid.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Create repository with few branches and test if fetcher can work
with such repository as PREMIRROR
(Bitbake rev: a1737610e5d5b61e126ec3632d7f27b337a87818)
Signed-off-by: Pavel Zhukov <pavel@zhukoff.net>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The infrastructure changes highlighed an issue where the top level directory
mtime stamp isn't included in the tarballs. This isn't an issue the test is
meant to check for so ignore it.
(Bitbake rev: 5a1e5b8dc55d6bb4d93bac3492f8c43ff957e712)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
