Add a test case to ensure git shallow fetch succeeds for SRC_URI
with tag containing slash.
For example, we want to succeed for SRC_URI like below:
SRC_URI = "git://salsa.debian.org/debian/debianutils.git;protocol=https;branch=master;tag=debian/${PV}"
See the following link for more information:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=15862
(Bitbake rev: 919d4cf6e688e67229c46d30c84d523b21936377)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a test case to verify that the gitsm fetcher properly handles
repositories storing objects with LFS.
The test case verifies that LFS objects are fetched on the initial clone
but also ensures that consecutive updates extend the original clone with
any newly referenced LFS objects.
(Bitbake rev: 2a8722ddd155596862029f6ea34e1e92c77e0b7f)
Signed-off-by: Philip Lorenz <philip.lorenz@bmw.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
So far, existence of `git-lfs` was only checked during unpacking. As the
binary is also used in earlier steps also check for its existence there.
Additionally, factor out the LFS existence check into a dedicated
function and call it wherever git-lfs is used for the first time.
(Bitbake rev: 5818367db9b261b7e07c347d38044e6cba8f9727)
Signed-off-by: Philip Lorenz <philip.lorenz@bmw.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Avoid multiple import statements for anything that is used more than
once. Additionally, drop no longer used imports.
(Bitbake rev: 7c74310440f4d6ec47cf5bacf597e18308b3bb20)
Signed-off-by: Philip Lorenz <philip.lorenz@bmw.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Using a shallow clone to simulate an outdated git mirror tarball does
not work in the intended way. A shallow clone already contains the
latest commit which can hide certain fetcher behavior. Simulate an
outdated mirror tarball, as the test titles indicate, by removing the
newer commits from the mirror.
(Bitbake rev: a51ee01f0a586fefd5a4061f4a1ca6cbf81b7046)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If SRCREV points at an annotated tag, the comparision code can fail
as the resolved tag might not be the same sha.
Handle this by also resolving the SRCREV. We only need to do this if
they don't match in the first place for a minor performance win.
Also add a test for this.
(Bitbake rev: 136c06e251de68ed64355ec6b47a522ff3a372e3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For example the miniupnpd recipe has a SRC_URI like this:
http://miniupnp.tuxfamily.org/files/download.php?file=${BP}.tar.gz
In this case the path is /files/download.php, which isn't useful when
the latest_upstream logic bails early if there is no version in the path.
The logic now also checks in the downloadfilename, so add a test that
this works as expected.
(Bitbake rev: fffbf5d5e1c8556cddf0794e0b303bb0106747a0)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some code paths in latest_versionstring() need PV to be set correctly.
(Bitbake rev: 0a9f90ff658e09feda63b398ec35715a65ff6193)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use a named tuple so the test can access named members instead of just
accessing the data via index, which is harder to understand.
(Bitbake rev: 4b15652c84b06f0506c757e2647875a9b1cc7bfe)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Marking the test iterations as subtests means that when one fails, it
can identify clearly which iteration has failed.
(Bitbake rev: 52c55e681332d7cdbe06f3c9d9c8d77cb0cb93f6)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Build the 'downloadfilename' parameter by replacing path separators in
the module path like the git fetcher builds the mirror tar ball name.
Copy the downloaded file in the fetcher's unpack method like the crate
fetcher instead of calling the base fetcher's unpack method.
(Bitbake rev: 7762cea087597019460d66b04268757bd46befdf)
Signed-off-by: Christian Lindeberg <christian.lindeberg@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We used to use this for bare clones where a single git url could handle multiple
revisions (which implied multiple branches).
We don't use this any more and I doubt we'd want to go back to it. If we
remove it, we can simplfy the looping in the code which seems desireable.
This patch does change the warning for missing branch parameters to a
error. The message has hinted about that for long enough.
Some test cases are removed since they are no longer needed.
(Bitbake rev: 2515fbd10824005fa7f34e87706000c079920366)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add tests for git tag verification in both standard and shallow clones.
(Bitbake rev: f47127066d67e2ad80974fa1e7c0fcc7409161af)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently bitbake disallows tag parameters along with revision parameters.
This isn't great since quite often, we'd like to verify that a given revision
does match some tag. At the same time we don't want to or need to access
the network to verify this, which normally a tag would require.
Rework the code so that tag and revisions can both be specified together.
Verify that any tag specified matches the revision in use at unpack time.
This means we can start requiring people to put tags in git SRC_URIs
when revisions are used, making review a little easier that it isn't
some random revision.
The test that is dropped looks like a different test but the comment
is a copy and paste error. The SRCREV/rev mismatch test remains, this
removes the rev and tag set test.
(Bitbake rev: d591d7633fe8d739ec00395920e44910b0b77e27)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This fixes a bug where escapes in the url path would not be properly
restored for the git commands in the git fetcher. For example, a
space which is encoded as '%20' was not properly encoded before the
clone command.
e.g.
SRC_URI="git://git.openembedded.org/bitbake%20example/bitbake;protocol=https"
resulted in
git clone 'https://git.openembedded.org/bitbake example/bitbake'
instead of
git clone 'https://git.openembedded.org/bitbake%20example/bitbake'
(Bitbake rev: be48024253b93215bb110cd1d05925e789ec9680)
Signed-off-by: Patrik Nordvall <patrik.nordvall95@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A regular expression "URL" in PREMIRRORS and MIRRORS may contain a ? as
part of the regular expression. Make sure this does not cause
problems.
(Bitbake rev: 5af7fe4473cd7e75d4eb7f8b93c499bd157ff156)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
- Address the absence of an initial full bare clone
- Utilize the initial shallow clone
- Modify existing test cases for this behavior
- Remove incompatible test cases
(Bitbake rev: 599fedacd7782dcb52825c22200f35344c102548)
Signed-off-by: Stefan Koch <stefan-koch@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to avoid a potentially confusing backtrace, check that the mcdepend
is valid when we add it.
Add a test case to ensure invalid configurations are caught and trigger an
error.
[RP: Reworked test case to simplify and improve code]
(Bitbake rev: ff523497270f37b484b44a4445c2194791bcb6ff)
Signed-off-by: Mark Hatle <mark.hatle@amd.com>
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Check default flags are correctly returned by getVarFlags and check all flags
are returned when internalflags is True.
Check delVarFlags also removes default value.
Check all flags are removed after delVar.
Run the test with:
$ bitbake-selftest -v bb.tests.data.TestFlags
test_delflag (bb.tests.data.TestFlags.test_delflag) ... ok
test_delvar (bb.tests.data.TestFlags.test_delvar) ... ok
test_setflag (bb.tests.data.TestFlags.test_setflag) ... ok
----------------------------------------------------------------------
Ran 3 tests in 0.000s
OK
This is a test case for [YOCTO #15685]
(Bitbake rev: ff8cae735cf489373af1aac7ee233d7b82d483d3)
Signed-off-by: Louis Rannou <louis.rannou@non.se.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The password must be quoted as per RFC3986 because it can contain
reserved characters.
(Bitbake rev: 5c53dbf0ad4385e1de7f2eef66565a1f05dae67a)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Do not use upper case hostnames because the hostname is case insensitive
and maybe decoded into lower case to follow the common style.
(Bitbake rev: 554b7048412c4c67bf895a8b98822b54ac3a66db)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Old code that parse variable names in assignment commands behave differently for
variables that ends with special symbol for single-character variable names and
multi-character variable names. For example:
A+="1" # Change variable ${A}, '+' glued to '='
A+ = "1" # Change variable ${A+}
+="1" # Change variable ${+}, the '+' symbol not part of assignment operator
+ = "1" # Change variable ${+}
New code would always assume that '.=', '+=', and ':=' is assignment operator.
As result code like the following would raise parsing error
+="value"
While code with extra spaces would work as before
+ = "value" # Change variable ${+}
This change allow to catch issues in code that generate bitbake configuration
files in a manner like "echo ${VARNAME}+=${VALUE} >> conf/local.conf"
(Bitbake rev: 93059aad13a12cd69d86368795c88e5349197d5d)
Signed-off-by: Nikolai Merinov <n.merinov@inango-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This test covers the handling of unclosed functions.
It tests that both whitespace and tabs generate the
correct exception if added before a closing bracket.
Additionally that a residue blocks generates a error
is tested as well.
[YOCTO #15470]
(Bitbake rev: 29e67acb87ae76879efe9688a69c961a96df10f1)
Signed-off-by: Savvas Etairidis <falital@hotmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The string value "default" for the default multiconfig is confusing since an
empty string is used pretty much everywhere in the code. Remove the few
remaining references to that to standarise.
This affects the default value of BB_CURRENT_MC and does have an impact
on metadata, particulalry bitbake.conf in openembedded-core. That said, the
number of bugs we'll avoid by trying to make "default" back to "" within
bitbake's code make fixing those extremely worthwhile.
(Bitbake rev: 0fa0d8d764bbeb8a44c47f79d7b849068d565199)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
From git 2.48 release notes:
"""
When "git fetch $remote" notices that refs/remotes/$remote/HEAD is
missing and discovers what branch the other side points with its
HEAD, refs/remotes/$remote/HEAD is updated to point to it.
"""
This means with git 2.48 onwards, there is a mystery "HEAD" revision
appearing in some of our shallow clone tests. We can avoid this by
using the same canonicalization as used for the reference revisions.
This resolves autobuilder failures on the Fedora 40 workers.
(Bitbake rev: c83444d1210740e27b1744d3aa7c5cad4e28db2f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Run the test with:
$ bitbake-selftest bb.tests.parse.ParseTest.test_parse_defaulttest
.
----------------------------------------------------------------------
Ran 1 test in 0.001s
OK
This is a test case for [YOCTO #15685]
(Bitbake rev: d12c80125d810df54379cc4f019a95ab9064ad2c)
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Reviewed-by: Ghislain Mangé <ghislain.mange@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The fetcher support entries with an embedded checksum like 'sha256sum'
in the SRC_URI. It adds the parameter 'name' as prefix to the checksums
if the parameter is set. This behavior is unexpected and leads to hacks
in fetchers. Fallback to the checksum without the useless prefix and
set the parameter 'name' in the gomod fetcher unconditional.
(Bitbake rev: 7a86c5a20ea2586f1ae240613644e065e7b21683)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The npm package lock and shrinkwrap file list bundled dependencies which
are supplied together with the parent dependency. The bundled
dependencies are marked by a flag. The flag and thereby test depends on
the lock file version. The old lock file version uses a `bundled` flag
and stores dependencies in the `dependencies` list. The new lock file
version uses an `inBundle` flag and stores dependencies in the
`packages` list.
(Bitbake rev: 34fd8ea6abe755e04220fe70b082aa620ae15f86)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The npmsw fetcher doesn't use the npm command. Remove the
`@skipIfNoNpm()` decorator from plain npmsw tests which don't use the
npm fetcher.
(Bitbake rev: 48f9964a19b8e2e3a73667c9889908eceecf7719)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adapt the npmsw tests to commit 84f102954e10 ("fetch/npmsw: The fetcher
shouldn't have any knowledge of S").
(Bitbake rev: 8194178291d9b47efb2079fe81b704ba6171efc1)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use the new listtasks() function (introduced in 185c4b) to avoid
accessing a private variable.
Also use assertSequenceEqual() directly as we don't really care about
the actual type returned, only that it's a sequence.
(Bitbake rev: 452289efc6d3608ceca321286fd5bf417a6e1bed)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
lz4c has been considered deprecated by upstream since at least 2018 [1]
and has been disabled by default recently [2]. openSUSE Tumbleweed is
no longer packaging the deprecated version and others will likely
follow.
Luckily, we're not using any legacy-specific options and, going back as
far as Ubuntu 16.04/Fedora 25, both lz4 and lz4cli are installed as part
of the same package
1. https://github.com/lz4/lz4/pull/553
2. https://github.com/lz4/lz4/pull/1479
(Bitbake rev: 907472034b344e4eb73cfd43059a413469f52e1c)
Signed-off-by: Justin Bronder <jsbronder@cold-front.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
GNOME gitlab has occasional downtime which impacts bitbake-selftest
and causes autobuilder failures. Switch to our own mirror for test
purposes to avoid those issues.
(Bitbake rev: 91e268b11ed683bd197026f9b36001f6d54ee05c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
GNOME gitlab has occasional downtime which impacts bitbake-selftest
and causes autobuilder failures. Switch to our own mirror for test
purposes to avoid those issues.
(Bitbake rev: 008808755ed6cfeb6c41273e69ce718f0833c26c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It was never a great solution to persisting data and there are much better
ones now. The last user has been replaced so drop the code and tests.
(Bitbake rev: 681a7516e9f7027e0be6f489c54a7a5e19fa9f06)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add test cases for a module path consisting of only a hostname and add
checksum verification of the unpacked go.mod files.
(Bitbake rev: 9380859b59923dee17469348f472a22e11be1779)
Signed-off-by: Christian Lindeberg <christian.lindeberg@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a go module fetcher for downloading module dependencies to the
module cache directly from a git repository. The fetcher can be used
with the go-mod class in OE-Core.
A module dependency can be specified with:
SRC_URI += "gomodgit://golang.org/x/net;version=v0.9.0;srcrev=..."
(Bitbake rev: 29ff38ccf0d5389a5bee81e252a78548361a9d7c)
Signed-off-by: Christian Lindeberg <christian.lindeberg@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a go module fetcher for downloading module dependencies to the
module cache from a module proxy. The fetcher can be used with the
go-mod class in OE-Core.
A module dependency can be specified with:
SRC_URI += "gomod://golang.org/x/net;version=v0.9.0;sha256sum=..."
(Bitbake rev: 5ff4694bf305e266ebf0abab5d9745c6b6d07d67)
Signed-off-by: Christian Lindeberg <christian.lindeberg@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
"addtask do_XXX before YYY after ZZZ "
where YYY or ZZZ is missing the "do_" prefix don't work as expected. Ajust the
code so that it doesn't just silently do the wrong thing but works as expected.
Expand a test case to cover this.
(Bitbake rev: 21670b9bb8936ec44aedff26163948bbc2ceb44a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The recent addtask improvement to handle comments complicated the regex significantly
and there are already a number of corner cases in that code which aren't handled well.
Instead of trying to complicate the regex further, switch to code logic instead. This
means the following cases are now handled:
* addtask with multiple task names
* addtask with multiple before constraints
* addtask with multiple after constraints
The testcase is updated to match the improvements.
(Bitbake rev: 417016b83c21fca7616b2ee768d5d08e1edd1e06)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Technically our syntax would allow for comments after an addtask/deltask.
Currently these get silently processed in various ways by the code which
is bad.
Tweak the regex to drop any comments and add test cases to ensure this
continues to work in the future.
[YOCTO #15250]
(Bitbake rev: 64f8796e55a8826ffec0d76b993c8256713f67a3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update the test cases since the implementation is changed:
* test_shallow_multi_one_uri()
The a_branch and v0.0 had the same revision, and it required fetch a_branch
and remove histories of v0.0 which were conflicted, and bitbake reported:
fatal: no commits selected for shallow requests
Make a_branch and v0.0 have different revs to fix the problem.
And now the 'rev^' is not needed, so update self.assertRevCount() as well.
* test_shallow_multi_one_uri_depths()
Update self.assertRevCount(), now git only fetches the required revs.
* test_shallow_fetch_missing_revs()
The command is:
$ git fetch --shallow-exclude=v0.0 master
But master and v0.0 uses the same revision, so there is no commit to fetch.
* test_shallow_fetch_missing_revs_fails()
Two unneeded committs are not fetched now:
- rev^
- One not specified or required tag.
So update self.assertRevCount()
(Bitbake rev: 48eff9d9a660ad6b9bd8b53a7dcec600ef42b1d1)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Upstream changed their urls, update our test to match.
(Bitbake rev: dc391b86540ec5e0a0f1d811c776a22d443b1c06)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
