Commit Graph

344 Commits

Author SHA1 Message Date
Justin Bronder
f700c1b774 bitbake: bb/compress: use lz4 instead of lz4c
lz4c has been considered deprecated by upstream since at least 2018 [1]
and has been disabled by default recently [2].  openSUSE Tumbleweed is
no longer packaging the deprecated version and others will likely
follow.

Luckily, we're not using any legacy-specific options and, going back as
far as Ubuntu 16.04/Fedora 25, both lz4 and lz4cli are installed as part
of the same package

1. https://github.com/lz4/lz4/pull/553
2. https://github.com/lz4/lz4/pull/1479

(Bitbake rev: 907472034b344e4eb73cfd43059a413469f52e1c)

Signed-off-by: Justin Bronder <jsbronder@cold-front.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-11-05 11:17:17 +00:00
Richard Purdie
59d4ee01ca bitbake: tests/fetch: Use our own mirror of mobile-broadband-provider to decouple from gnome gitlab
GNOME gitlab has occasional downtime which impacts bitbake-selftest
and causes autobuilder failures. Switch to our own mirror for test
purposes to avoid those issues.

(Bitbake rev: 91e268b11ed683bd197026f9b36001f6d54ee05c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-10-29 22:27:45 +00:00
Richard Purdie
2abe7cedbf bitbake: tests/fetch: Use our own mirror of sysprof to decouple from gnome gitlab
GNOME gitlab has occasional downtime which impacts bitbake-selftest
and causes autobuilder failures. Switch to our own mirror for test
purposes to avoid those issues.

(Bitbake rev: 008808755ed6cfeb6c41273e69ce718f0833c26c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-10-28 14:08:33 +00:00
Richard Purdie
2aad7b988e bitbake: persist_data: Remove it
It was never a great solution to persisting data and there are much better
ones now. The last user has been replaced so drop the code and tests.

(Bitbake rev: 681a7516e9f7027e0be6f489c54a7a5e19fa9f06)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-10-09 13:04:30 +01:00
Christian Lindeberg
6ae52edfbb bitbake: tests/fetch: Update GoModTest and GoModGitTest
Add test cases for a module path consisting of only a hostname and add
checksum verification of the unpacked go.mod files.

(Bitbake rev: 9380859b59923dee17469348f472a22e11be1779)

Signed-off-by: Christian Lindeberg <christian.lindeberg@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-09-20 13:57:32 +01:00
Christian Lindeberg
87c29b5a94 bitbake: fetch2: Add gomodgit fetcher
Add a go module fetcher for downloading module dependencies to the
module cache directly from a git repository. The fetcher can be used
with the go-mod class in OE-Core.

A module dependency can be specified with:

  SRC_URI += "gomodgit://golang.org/x/net;version=v0.9.0;srcrev=..."

(Bitbake rev: 29ff38ccf0d5389a5bee81e252a78548361a9d7c)

Signed-off-by: Christian Lindeberg <christian.lindeberg@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-09-10 13:05:00 +01:00
Christian Lindeberg
dd7631426c bitbake: fetch2: Add gomod fetcher
Add a go module fetcher for downloading module dependencies to the
module cache from a module proxy. The fetcher can be used with the
go-mod class in OE-Core.

A module dependency can be specified with:

  SRC_URI += "gomod://golang.org/x/net;version=v0.9.0;sha256sum=..."

(Bitbake rev: 5ff4694bf305e266ebf0abab5d9745c6b6d07d67)

Signed-off-by: Christian Lindeberg <christian.lindeberg@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-09-10 13:05:00 +01:00
Richard Purdie
997b11a5a1 bitbake: build: Ensure addtask before/after tasknames have prefix applied
"addtask do_XXX before YYY after ZZZ "

where YYY or ZZZ is missing the "do_" prefix don't work as expected. Ajust the
code so that it doesn't just silently do the wrong thing but works as expected.

Expand a test case to cover this.

(Bitbake rev: 21670b9bb8936ec44aedff26163948bbc2ceb44a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-13 15:47:55 +01:00
Richard Purdie
5226f46342 bitbake: BBHandler/ast: Improve addtask handling
The recent addtask improvement to handle comments complicated the regex significantly
and there are already a number of corner cases in that code which aren't handled well.

Instead of trying to complicate the regex further, switch to code logic instead. This
means the following cases are now handled:

* addtask with multiple task names
* addtask with multiple before constraints
* addtask with multiple after constraints

The testcase is updated to match the improvements.

(Bitbake rev: 417016b83c21fca7616b2ee768d5d08e1edd1e06)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-13 15:47:55 +01:00
Richard Purdie
60cba6a073 bitbake: BBHandler: Handle comments in addtask/deltask
Technically our syntax would allow for comments after an addtask/deltask.
Currently these get silently processed in various ways by the code which
is bad.

Tweak the regex to drop any comments and add test cases to ensure this
continues to work in the future.

[YOCTO #15250]

(Bitbake rev: 64f8796e55a8826ffec0d76b993c8256713f67a3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-13 15:47:55 +01:00
Robert Yang
7bc521ed34 bitbake: bitbake: tests/fetch: Update GitShallowTest for clone_shallow_local()
Update the test cases since the implementation is changed:

* test_shallow_multi_one_uri()
  The a_branch and v0.0 had the same revision, and it required fetch a_branch
  and remove histories of v0.0 which were conflicted, and bitbake reported:
  fatal: no commits selected for shallow requests

  Make a_branch and v0.0 have different revs to fix the problem.

  And now the 'rev^' is not needed, so update self.assertRevCount() as well.

* test_shallow_multi_one_uri_depths()
  Update self.assertRevCount(), now git only fetches the required revs.

* test_shallow_fetch_missing_revs()
  The command is:
  $ git fetch --shallow-exclude=v0.0 master

  But master and v0.0 uses the same revision, so there is no commit to fetch.

* test_shallow_fetch_missing_revs_fails()
  Two unneeded committs are not fetched now:
  - rev^
  - One not specified or required tag.

  So update self.assertRevCount()

(Bitbake rev: 48eff9d9a660ad6b9bd8b53a7dcec600ef42b1d1)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-13 23:30:07 +01:00
Richard Purdie
b6b66ca07b bitbake: tests/fetch: Tweak test to match upstream repo url change
Upstream changed their urls, update our test to match.

(Bitbake rev: dc391b86540ec5e0a0f1d811c776a22d443b1c06)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-06-05 18:33:42 +01:00
Richard Purdie
f24ffc087b bitbake: tests/fetch: Tweak to work on Fedora40
On Fedora40, "localhost" sometimes resolves to ::1 and sometimes to 127.0.0.1
and python only binds to one of the addresses, leading to test failures.

Use 127.0.0.1 explicitly to avoid problems of the name resolution, we're trying
to test things other than the host networking.

(Bitbake rev: 9adc6da42618f41bf0d6b558d62b2f3c13bedd61)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-31 16:56:25 +01:00
Antonin Godard
1aa8276c64 bitbake: tests.codeparser: add tests for shell expansions
Tests quotes around `` and $() expansions, nested and multiple
expansions, and that escaped quotes are treated as characters by the
parser.

(Bitbake rev: d98130cb4d500c495bc692c56dde3e019f36320a)

Signed-off-by: Antonin Godard <antoningodard@pm.me>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-21 14:14:41 +01:00
Kari Sivonen
9925db0c4f bitbake: fetch2/svn: Fix mirroring issue with svn
Add return false to supports_checksum for svn fetcher which fhis
fixes MIRROR usage for svn uris. Also add a testcase.

[YOCTO #15473]

(Bitbake rev: 21cfc7ae9a19f39ac8904e1c3466e7e499ac523f)

Signed-off-by: Kari Sivonen <kari.sivonen@live.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-08 14:51:43 +01:00
Alexander Kanavin
551fdabc54 bitbake: fetch2/crate: add upstream latest version check function
This is actually rather easy: crate web API provides a json
with all the versions, for example:
https://crates.io/api/v1/crates/cargo-c/versions

(Bitbake rev: f6c2755db9a1f88c8534193b420fa31d135945e6)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-08 14:51:43 +01:00
Ross Burton
e20ee877ed bitbake: fetch2: handle URIs with single-valued query parameters
Whilst typically the URI query is a list of key-value pairs, that's not
actually required by the URI specification.

For example:  http://example.com/foo?bar is a valid query, but this will
result in the fetcher raising an exception:

  File "bitbake/lib/bb/fetch2/__init__.py", line 265, in __init__
    self.query = self._param_str_split(urlp.query, "&")
  File "bitbake/lib/bb/fetch2/__init__.py", line 293, in _param_str_split
    for k, v in [x.split(kvdelim, 1) for x in string.split(elmdelim) if x]:
ValueError: not enough values to unpack (expected 2, got 1)

In this case the query is just "bar", but the fetcher is trying to split
it into a key-value pair.

The URI object exposes the parsed query explicitly as a dictionary of
key-value pairs, so we have to be a little creative here: if a value is
None then it isn't a key-value pair, but a bare key.

Fix this by handling elements without the deliminator in _param_str_split()
(by assigning the value to None), and handle a None value when formatting
the query in _param_str_join().

(Bitbake rev: eac583bd4c46f3bb9661852cb6a1448f16147ff1)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-03-22 16:26:45 +00:00
Philip Lorenz
0e3bcc5103 bitbake: fetch2: Ensure that git LFS objects are available
The current implementation only performs a git lfs fetch alongside of a
regular git fetch. This causes issues when the downloaded revision is
already part of the fetched repository (e.g. because of moving back in
history or the updated revision already being part of the repository at
the time of the initial clone).

Fix this by explicitly checking whether the required LFS objects are
available in the downloade directory before confirming that a downloaded
repository is up-to-date.

This issue previously went unnoticed as git lfs would silently fetch the
missing objects during the `unpack` task. With network isolation turned
on, this no longer works, and unpacking fails.

(Bitbake rev: cfae1556bf671acec119a6c8bbc4b667a856b9ae)

Signed-off-by: Philip Lorenz <philip.lorenz@bmw.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-23 14:34:05 +00:00
Peter Kjellerstedt
d25f6b970f bitbake: fetch2/git: Make latest_versionstring extract tags with slashes correctly
Before, everything up to the last slash was removed when extracting the
names of the tags. This would lead to that a tag such as "agent/11.0.0"
would be incorrectly identified as "11.0.0", which would then be treated
as a correct version matching "^(?P<pver>\d+(\.\d+)+)".

(Bitbake rev: 8b21024b9966d5158ac4a77e87ffb935c2a57764)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-19 15:08:30 +00:00
Peter Kjellerstedt
21603031df bitbake: tests/fetch: Make test_git_latest_versionstring support a max version
In some cases, the version found by latest_versionstring() may be higher
than the real version. Make it possible to specify a maximum version so
that this case can be detected.

(Bitbake rev: 9134d4777109bc78410c3e641420d9a78b485e33)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-19 15:08:30 +00:00
Chen Qi
09ecbcd307 bitbake: tests/fetch.py: add test case for using premirror in restricted network
We had issue when BB_ALLOWED_NETWORKS is set and `bitbake grpc-native -c fetch'
failed even with all contents available in PREMIRRORS.

Add a test case to ensure no regression in the future.

(Bitbake rev: 80c91ceb81b1cae203067af58d3f1fe9c619ae83)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-19 15:08:30 +00:00
Pavel Zhukov
de27ecd228 bitbake: tests/fetch.py: add multiple fetches test
Fetch from premirror few times to emulate multiple machines sharing same
clonedir or few rebuilds of the package from (pre)mirror
Regression test for [Yocto #15369]

(Bitbake rev: 7fcbac574c68f16b95ab7abb2874931d168d3c9e)

Signed-off-by: Pavel Zhukov <pavel@zhukoff.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-19 12:52:30 +00:00
André Draszik
577e73606a bitbake: tests/fetch: support git's safe.bareRepository
When git is configured with safe.bareRepository=explicit [1], the
bitbake selftests fail miserably. LWN has an article about the
problem that this configuration option addresses and why it is useful
in [2].

It also seems that it is being rolled out in some environments as a
default for users.

In order to allow having this configuration turned on for a user's
environment in general, the fetcher has to be tought to use --git-dir=
for all relevent git operations.

The alternative, implemented here, is to forcibly turn off that option
for all git operations. In the future, we could look into converting
these to using the --git-dir= command line argument instead.

Link: https://git.kernel.org/pub/scm/git/git.git/tree/Documentation/config/safe.txt#n1 [1]
Link: https://lwn.net/Articles/892755/ [2]
(Bitbake rev: a45e14a7343e36101e45639931322e5649587f57)

Signed-off-by: André Draszik <andre.draszik@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-19 12:00:59 +00:00
Martin Jansa
5276b7f548 bitbake: tests: fetch.py: use real subversion repository
* github no longer provides support for subversion clients:
  https://docs.github.com/en/enterprise-server@3.11/get-started/working-with-subversion-on-github/support-for-subversion-clients
  it was shut down on 2024-01-08:
  https://github.blog/2023-01-20-sunsetting-subversion-support/

  and this test was now failing with:

======================================================================
ERROR: test_external_svn (bb.tests.fetch.SVNTest.test_external_svn)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/OE/layers/bitbake/lib/bb/tests/fetch.py", line 1287, in test_external_svn
    fetcher.download()
  File "/OE/layers/bitbake/lib/bb/fetch2/__init__.py", line 1896, in download
    raise FetchError("Unable to fetch URL from any source.", u)
bb.fetch2.FetchError: Fetcher failure for URL: 'svn:///tmp/bitbake-fetch-zdvedqt_/svnfetch_localrepo_wup8mgn6/project;module=trunk;protocol=file;externals=allowed;rev=2'. Unable to fetch URL from any source.

Stdout:
Fetch svn:///tmp/bitbake-fetch-zdvedqt_/svnfetch_localrepo_wup8mgn6/project;module=trunk;protocol=file;externals=allowed;rev=2
Failed to fetch URL svn:///tmp/bitbake-fetch-zdvedqt_/svnfetch_localrepo_wup8mgn6/project;module=trunk;protocol=file;externals=allowed;rev=2, attempting MIRRORS if available
Fetcher failure: Fetch command export PSEUDO_DISABLED=1; /usr/bin/env svn --non-interactive --trust-server-cert co --no-auth-cache -r 2 file:///tmp/bitbake-fetch-zdvedqt_/svnfetch_localrepo_wup8mgn6/project/trunk@2 trunk failed with exit code 1, output:
A    trunk/README.md
 U   trunk
Checked out revision 2.

svn: warning: W205011: Error handling externals definition for 'trunk/bitbake':
svn: warning: W170013: Unable to connect to a repository at URL 'https://github.com/PhilipHazel/pcre2.git'
svn: E205011: Failure occurred processing one or more externals definitions

  in the rare cases where subversion was still installed on the host
  running bitbake-selftest :).

  to avoid this use still alive repository from https://svn.apache.org/
  and pick something rather small and only the trunk subdirectory which
  is fast to fetch:
  svn co https://svn.apache.org/repos/asf/serf/trunk
  takes just 2 sec here

  adjust expected dir/file to use "protocols/fcgi_buckets.h" instead of
  "trunk/README"

(Bitbake rev: a735898abcf056f897c9350bb128a5637e6b4617)

Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-08 10:59:32 +00:00
Richard Purdie
c665a2c933 bitbake: ast: Fix EXPORT_FUNCTIONS bug
If you have two classes, both of which set EXPORT_FUNCTIONS for the same funciton
and a standard funciton definition for the function that is exported, the export
function can sometimes overwrite the standard one.

The issue is that the internal flag the code uses isn't ovweritten if the variable
is giving a new value. Fix the issue by using a comment in the code that is injected
so that we know if it is ours or not.

Also add some testing for EXPORT_FUNCTIONS, not perfect but a start.

(Bitbake rev: 66306d5151acb0a26a171c338d8f60eb9eb16c6b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-10 13:55:33 +00:00
Robert Yang
4e2fcdb383 bitbake: bitbake: tests/event: Add test_lineno_in_eventhandler
Add test_lineno_in_eventhandler to test lineno in eventhandler.

(Bitbake rev: 4e5de537bebb68180c5755858c81b095eb9ae2f6)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-10 13:55:33 +00:00
Richard Purdie
029e82d418 bitbake: bb/toaster: Fix assertEquals deprecation warnings
Fix:
    DeprecationWarning: Please use assertEqual instead

(Bitbake rev: dd990ea6843685927954101feb729f3faa3a16d9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-06 22:28:03 +00:00
Richard Haar
64725a9c32 bitbake: bitbake: tests: Fix duplicate test_underscore_override test
Found a duplicate test, added  _2 suffix to one, 74 tests now pass up from 73.

(Bitbake rev: ae2a19dadb4f3065b8731a61f45f29e6a70af402)

Signed-off-by: Richard Haar <rh@richhaar.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-02 18:04:23 +00:00
Desone Burns
7aab731f1e bitbake: bitbake: fetch2: git: Update Git-LFS download and tests
When downloading a Git repository containing an LFS, the Git hooks
are not always initialized correctly to perform the download. This
change updates the Git downloader to run the "git lfs install" command
in order to "smudge" the LFS files. The tests have also been updated
to do checks for situations in which git lfs is not installed, as
the application was required to be installed for any of the tests
to run previously.

The Git LFS functionality was working to some degree previously,
but this change also updates the fetcher to allow LFS downloads for
nobranch SRC_URIs.

(Bitbake rev: 05f8529fb439db93d85a892704b6f2f0ac0c9217)

Signed-off-by: Desone Burns II <dburns@seegrid.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-23 17:37:25 +00:00
Pavel Zhukov
695ba46528 bitbake: tests/fetch.py: Add tests to cover multiple branch/name parameters
Create repository with few branches and test if fetcher can work
with such repository as PREMIRROR

(Bitbake rev: a1737610e5d5b61e126ec3632d7f27b337a87818)

Signed-off-by: Pavel Zhukov <pavel@zhukoff.net>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-23 11:46:36 +01:00
Richard Purdie
b8b5c786e7 bitbake: selftest/fetch: Ensure top level directory timestamp doesn't break test
The infrastructure changes highlighed an issue where the top level directory
mtime stamp isn't included in the tarballs. This isn't an issue the test is
meant to check for so ignore it.

(Bitbake rev: 5a1e5b8dc55d6bb4d93bac3492f8c43ff957e712)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-06 11:42:46 +01:00
Insu Park
6ad45f110e bitbake: data: Add missing dependency handling of remove operator
A recipe variable handles its dependencies even on the "contains"
variables within the "inline Python expressions" like bb.utils.filter().
And it also handles those in the append operator correctly, but the
problem is that it does not so in the remove operator.

Fix it by adding the missing dependencies every time the remove
operator has been handled.
Also add a test case to check if the override operators handle
dependencies correctly.

(Bitbake rev: b90520eedb1dbc7f6a3928d089fe74fafb864eb5)

Signed-off-by: Insu Park <insu0.park@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-20 08:57:41 +01:00
Richard Purdie
84a7485025 bitbake: siggen: Improve runtaskdeps data to fix sstate debugging
The runtaskdep data in siginfo files was written out with full paths to
the bb files, matching bitbake's internal "unique key" ID for recipes/tasks.
When originally implemented this made sense.

Over time, the main use for the data in siginfo files has become to match
against other siginfo files to debug changes of hash calcuations. The
recipename data is not useful for this as the siginfo filenames use PN
instead which can often be derived from the recipe filename but not always.

It is time to throw away the 'tid' data format and switch over the use a
hybrid PN form which includes the multiconfig. That can be easily stripped
off in the find_siginfo code in oe-core.

The other purpose of having a sortable dependency ID is retained and the
multiconfig needs to be included to allow the taskhashes to be processed
and calculated correctly. PN is meant to be unique between recipes, only
one would ever be built so using PN in this location is fine.

The one risk of this change is there isn't any compatibility to the old
format. I'm not convinced we should spend time complicating the code
with it. This change will change the taskhashes everywhere so the only
mixing of old and new siginfo files will be either through hash equivalence
or through users using the tool against old and new info files manually
which will give some weird output but it should be clear they're in
different formats as there would be large paths from the old files not
present in the new ones.

We have options to add backwards compatibility if some issue is found
to need that.

(Bitbake rev: 637933e2e5a59228a8d17aae4160551cab5f2f61)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-08-09 13:23:33 +01:00
Christopher Larson
0402e69853 bitbake: bb.tests.data: don't require the func flag for context functions
Update test_python_snippet_function_reference to not require the 'func'
flag, now that we know the real function will be returned for context
functions without the flag.

(Bitbake rev: 83f41281ec3d9b4327ffc8e2312e1fb8f53cbf02)

Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-08-04 11:37:56 +01:00
Christopher Larson
042b70da3b bitbake: tests.data: add test for builtin preferred over metadata value
This test makes sure that '${@eval()}' calls the eval builtin, even if an 'eval' variable is defined in the metadata.

(Bitbake rev: e9150447738a48f772240874b3512b08e982b19b)

Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-08-04 11:37:56 +01:00
Christopher Larson
a9d2012f50 bitbake: tests.codeparser: add test for exec of builtin from inline python
This ensures that any change to the presence of builtins in inline
python execs will be noticed.

(Bitbake rev: ee22d3d51c60db2da97422b2be1e42239b7a2324)

Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-08-04 11:37:55 +01:00
Christopher Larson
fc8468024c bitbake: tests.data: add test for inline python calling a def'd function
This is a test for an issue seen long ago, to avoid regressions, where a
reference to a def'd function in the metadata would return the string
value from the metadata rather than the function in inline python.

(Bitbake rev: 9f7cb22febd557817c164e25a93f5660e9c06358)

Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-08-04 11:37:55 +01:00
Dit Kozmaj
afed7e8747 bitbake: fetch2: Set maxsplit to match expected variables
Set the maxsplit value to match the expected number of variables.
This also avoids an unnecessary split as the parameters are in the form 'key=value'
and the 'value' could contain the '=' character.

(Bitbake rev: 3b17a7ed9bf6cd2808946c2d9c3ed9961af11f19)

Signed-off-by: Dit Kozmaj <dit.kozmaj@kynetics.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-07-27 21:09:14 +01:00
Peter Hoyes
fd1d2c6dc7 bitbake: bitbake: tests: Use assertLogs to test logging output
By default, pytest captures all stdout and exposes it using its built-in
fixtures (capsys, caplog etc), so stdout does not support getvalue().

To support running tests using both unittest and pytest, use assertLogs
to capture logging and assert on the log output instead.

(Bitbake rev: 2d28caa01bab9540d2bbaf713ae3e5c563d003f5)

Signed-off-by: Peter Hoyes <Peter.Hoyes@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-06-27 15:28:56 +01:00
Peter Hoyes
8b0fc4850d bitbake: bitbake: tests/fetch: Set git config if not already set
git config returns an error code if user.email or user.name are not set,
so bb.process.run raises an ExecutionError, failing the test.

Improve the logic by catching the ExectionError and using this to set a
default value.

Change a direct invocation of 'git init' to use self.git_init so that
it follows the code path above.

Remove the related comment from the README now that git sets up the user
details automatically.

(Bitbake rev: 0c7079d0609a13713b890a9675abd9fba032f199)

Signed-off-by: Peter Hoyes <Peter.Hoyes@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-06-27 15:28:56 +01:00
Peter Hoyes
baf345598d bitbake: bitbake: tests/fetch: Rename assertRaisesRegexp to assertRaisesRegex
TestCase.assertRaisesRegexp was renamed to assertRaisesRegex in Python
3.2, so rename to fix a warning during test execution.

(Bitbake rev: 9bc2ca3197bff7a5ef6e0ece6bfef0cda68b808e)

Signed-off-by: Peter Hoyes <Peter.Hoyes@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-06-27 15:28:56 +01:00
Peter Hoyes
a7b74c0b73 bitbake: bitbake: tests/fetch: Mark TestTimeout as not a test suite
pytest test collection attempts to collect all classes containing
"Test". TestTimeout is not a test class so add the __test__ = False
attribute to indicate this.

(Bitbake rev: b5455b98093a3eaf122ebe3d29c3bef949fd2440)

Signed-off-by: Peter Hoyes <Peter.Hoyes@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-06-27 15:28:56 +01:00
Bergin, Peter
5c3b6b6fd1 bitbake: lib/bb/tests/fetch.py: set initial branch
If you have a host where the git config for initial branch
name is something else than 'master' the unittest will fail
as they assume the default branch name is 'master'. Fix this
by explicitly set the intial branch name at 'git init'.

(Bitbake rev: 5bcd213c23da30a84baba88b775b1740e6bb77d0)

Signed-off-by: Peter Bergin <peter.bergin@windriver.com>
Signed-off-by: Peter Bergin <peter@berginkonsult.se>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-05-25 10:33:15 +01:00
Peter Kjellerstedt
5b4c7dc600 bitbake: fetch2/crate: Correct unpack for a crate that matches the recipe name
The crate fetcher handles a crate with a name that matches the recipe's
name specially by placing the unpacked code in the current directory
(which typically is ${S}) rather than together with the sources for the
other crates. This broke when the URI names for all crates were changed
recently to include the version in the name.

Correct the crate fetcher to test against ${BP} instead of ${BPN}.
Also add a test case to the selftests to avoid this breaking again.

[Yocto #15012]

(Bitbake rev: 9af52b748357d0c843ce2507ce4d119fd9c37008)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-05-25 10:33:15 +01:00
Richard Purdie
e53cf94d6e bitbake: tests/fetch: Default to https git protocol where possible
The recommendation from server maintainers is that the https protocol
is both faster and more reliable than the dedicated git protocol at this point.
Switch to it where possible.

(Bitbake rev: 879b830fcc60a022682ef22f58c04e7e07d03c63)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-05-05 11:07:42 +01:00
Kai Kang
a99566dac1 bitbake: bitbake: ConfHandler: Allow variable flag name with a single character
Update regex pattern to allow variable flag name with a single character.

Regression tests have also been updated in `bb.parse` and
`bin/bitbake-selftest -k ParseTest` has been successfully executed.

Eliminate a trailing space as well.

(Bitbake rev: bb9e523291a3cad6e1596ee6a1e715b5e5feba8f)

Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-04-13 12:01:45 +01:00
Enrico Scholz
b92918eb0e bitbake: fetch2/crate: create versioned 'name' entries
It is common for rust packages to depend on different versions of the
same crate.  E.g.

|    crate://crates.io/windows_x86_64_msvc/0.42.2 \
|    crate://crates.io/windows_x86_64_msvc/0.48.0 \

Identification only by the plain crate name makes the sha256sum
ambiguous

| SRC_URI[windows_x86_64_msvc.sha256sum] = "9aec5da331524158c6d1a4ac0ab1541149c0b9505fde06423b02f5ef0106b9f0"
| SRC_URI[windows_x86_64_msvc.sha256sum] = "1a515f5799fe4961cb532f983ce2b23082366b898e52ffbce459c86f67c8378a"

and requires lot of manual work to fix the SRC_URI list.

Making the 'name' property unique by appending the version allows
direct copy & paste of reported sha256sum errors to complete the
crates list.

(Bitbake rev: ae2efb05196f9e29ef56ad9a84e2eae5fbdd8030)

Signed-off-by: Enrico Scholz <enrico.scholz@sigma-chemnitz.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-04-06 14:28:59 +01:00
Frederic Martinsons
2b12c58724 bitbake: fetch2: Add checksum capability for crate fetcher
This change brings checksum verification of each crate
in a recipe, e.g

| SRC_URI += " \
|     crate://crates.io/aho-corasick/0.7.20 \
|     crate://crates.io/atomic-waker/1.1.0 \
|     crate://crates.io/cc/1.0.79 \
| "
|
| SRC_URI[aho-corasick.sha256sum] = "cc936419f96fa211c1b9166887b38e5e40b19958e5b895be7c1f93adec7071ac"
| SRC_URI[atomic-waker.sha256sum] = "debc29dde2e69f9e47506b525f639ed42300fc014a3e007832592448fa8e4599"
| SRC_URI[cc.sha256sum] = "50d30906286121d95be3d479533b458f87493b30a4b5f79a607db8f5d11aa91f"

That will require to move the checksum initialization
after the possible call to urldata_init method in order
for the crate fetcher to parse the url.

Another way of doing could have been implementing a decodeurl
method that would have been specific for each fetcher class.

(Bitbake rev: 4920686c13dd66f9bfa4f7dd38d6e955f153eeec)

Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-30 10:51:54 +01:00
James R T
f14328c22f bitbake: ConfHandler: Allow the '@' character in variable flag names
This patch enables the usage of the '@' character in variable flag
names.

One use case of variable flags is to assign the network namespaces of
some systemd services/targets to configure other parts of the build
process of some system. The filenames of systemd services/targets might
contain the '@' character if they are template unit files that can take
in a single parameter/argument and be instanced multiple times, as
indicated by systemd's official manual page.

The '@' character is disallowed as the first character in a variable
flag name. Imposing more restrictions on the first character is a
compromise to make parsing easier and to allow for more options in the
future to extend the syntax.

This patch is successfully verified by creating a custom BitBake recipe
that sets and unsets the value of a variable flag with the '@' character
in its name and ensuring that no ParseError is being thrown. Regression
tests have also been added to `bb.parse`.

`bin/bitbake-selftest` has also been successfully executed and all tests
passed.

(Bitbake rev: 00f9ab2cacfbd2a63b6b4959cf5401babae7e32a)

Signed-off-by: James Raphael Tiovalen <jamestiotio@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-23 22:41:32 +00:00
Richard Purdie
2b8761e5fc bitbake: fetch2: Rename __BBSEENSRCREV -> __BBSRCREV_SEEN
For readability of following patches, rename this internal variable to
allow for others in a similar format.

(Bitbake rev: 450b4a00895ed1f9396a8ff859dc1cc0eccc838f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-22 13:57:04 +00:00