1767 Commits

Author SHA1 Message Date
Richard Purdie
94efc6a14c resulttool/store: Fix permissions of logarchive
We want the results directory to be visable to other users, tweak the
permissions of the created directory to ensure this is the case.

(From OE-Core rev: bf121dc86f2c3443673f5ff3d486237678044479)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit ed9d887e8d71a800db19826264de552f7736dc6a)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2025-02-12 06:29:34 -08:00
Adrian Freihofer
6b2ad54a43 devtool: ide-sdk remove the plugin from eSDK installer
The ide-sdk command bootstraps the SDK from the bitbake environment
before the IDE configuration is generated. In the case of the eSDK
installer, the bootstrapping is performed during the installation of
the eSDK installer. Running the ide-sdk plugin from an eSDK installer
based setup would require skipping the bootstrapping and probably taking
some other differences into account when generating the IDE
configurations.

This would be possible. But it will probably never be implemented, as
running devtool ide-sdk directly from the bitbake environment is much
more flexible.
Also, some of the recent improvements that have made it into the core
have the potential to make the eSDK installer obsolete at some point in
the future:
- bitbake-layers create-layers-setup replicates the layers
- bitbake-config-build replicates the build configuration
- The new sstate mirror features replicate the sstate
- bblock locks the sstate more flexible than the eSDK installer
- devtool ide-sdk bootstraps the SDK directly from the bitbake
  environment. The same environment-setup... file is provided with
  --mode=shared.
  The devtool modify based workflow is supported since always by devtool
  and also the default --mode of devtool ide-sdk.
These functions essentially cover what the eSDK installer does without
a need for the current implementation of the eSDK installer and the
populate_sdk_ext, which is hard to maintain and takes a lot of time to
build.
This means that instead of making the ide-sdk plugin compatible with the
eSDK installer, we should rather replace the current implementation of
the eSDK installer and populate_sdk_ext with an implementation that can
replicate a normal bitbake environment in a convenient way where the
ide-sdk plugin also just works without additional complexity.

(From OE-Core rev: a5e7ff4424a68335fc768e49a232a58011a5b160)

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2025-02-03 06:24:37 -08:00
Adrian Freihofer
8a4f813c86 devtool: ide-sdk recommend DEBUG_BUILD
The debug_build_config function was never called. Compiling with debug
optimized compiler flags was not working. Even with the
--debug-build-config flag set, the build configuration from the recipe
was used.

The devtool ide-sdk --debug-build-config approach didn't work very well
anyway. The problem is that changing the bbappend file doesn't work
while bitbake uses the bbappend file. As a workaround, it would be
possible to parse the recipe, get DEBUG_BUILD and the path to the append
file, exit tinfoil, change the bbappend file, reopen tinfoil and do what
ide-sdk is supposed to do. Such an implementation would be complicated
and slow.
Therefore, the code that was originally supposed to implement this is
removed from ide-sdk and the new --debug-build function of devtool
modify is used instead. Additionally, a hint should be given on how to
manually add DEBUG_BUILD = '1' to bbappend.

This is compatible with the VSCode Bitbake plug-in, which does not
support this parameter anyway.

(From OE-Core rev: 8753ddc7a42a09eec9b12af97b2b511b2970d83c)

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2025-02-03 06:24:37 -08:00
Adrian Freihofer
e1f3ee328b devtool: ide-sdk sort cmake preset
Sort the keys of the generated CMakeUserPreset.json file to make it
easier to search and compare.

(From OE-Core rev: ad633eb3b90adb1087609c97ab8dc83ca0f08ad2)

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2025-02-03 06:24:37 -08:00
Adrian Freihofer
a221cda57c devtool: modify support debug-builds
Add a new option --debug-builds to automatically add DEBUG_BUILD = “1”
to the bbappend file of this recipe. This is especially useful when
invoking devtool modify before invoking devtool ide-sdk to perform a
remote debugging session.

(From OE-Core rev: c1bea1549d3bc47845b7c23158d48fad8e17277e)

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2025-02-03 06:24:37 -08:00
Richard Purdie
721a0e0103 resulttool: Improve repo layout for oeselftest results
Having all oe-selftest results on top of each other results in a large 640MB
json file which is hard to use. Split the results out per machine and test type.

This also stops the toolchain raw logs from overwriting each other meaning more
than one MACHINE is preserved.

(From OE-Core rev: 8717a2e2161378bab4e27d515a71396b2a6bcf00)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 4b890f04bc7d147b4a11b824a84f3d2abd75ac54)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-12-09 06:25:53 -08:00
Richard Purdie
ea0a01b8f2 resulttool: Trim the precision of duration information
The duration values have pointless amounts of precision. Removing some of the
least significant digits reduces result size and makes the results easier to read.

(From OE-Core rev: 7a95226404e41e878858afcc383d26fb534ccf16)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit a789a2e6d97bb8efd663226a17db8d1ca6c1e40f)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-12-09 06:25:53 -08:00
Richard Purdie
d329f42da2 resulttool: Clean up repoducible build logs
We've improved the data stored for reproduicible builds. Teach resulttool how
to apply those cleanups when reprocessing data so we can reduce results file
sizes and make the data easier to process.

(From OE-Core rev: 316535d421be32d773a4f98beea5bf2aabfccefb)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit b799c57ae6d61c1b1c7035c8a2c4ba6ee08d1a81)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-12-09 06:25:53 -08:00
Richard Purdie
4d496db35d resulttool: Handle ltp rawlogs as well as ptest
Improve the rawlogs handling to include ltp logs as well as the ptest ones to
reduce the size of the results git repos.

(From OE-Core rev: 8a34083bf097bf6a8362ce7f9b86fba5662a27bd)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit a0a1954d559609c2c1ca16936d0d68eb3c4c6b45)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-12-09 06:25:53 -08:00
Richard Purdie
e6b2aa1998 resulttool: Add --logfile-archive option to store mode
Storing the log files inside the testresults git repo isn't scaling and isn't
really appropriate use of a git repository. Allow these to be optionally stored
in a separate filesystem location so the git repo can remain managable.

(From OE-Core rev: 174e86868178f817f2bb6b3d9c7224a4352144ed)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 1afc0f3d7e93fa8496be241e9622d3b9a6904bd5)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-12-09 06:25:53 -08:00
Richard Purdie
0f8b48414d resulttool: Fix passthrough of --all files in store mode
When using store mode, --all was broken as not all files were being preserved.
Fix this by limiting the scope of the git rm command.

(From OE-Core rev: ad850837d3a45fa7cc094e3eb9c059120664c8e2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 9604561d2022b6c76b1cb4186d40800d1affdd2b)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-12-09 06:25:53 -08:00
Richard Purdie
4dec174239 resulttool: Use single space indentation in json output
Using 4 space indentation in resulted in hundreds of megabytes of extra file size
in general use. Reduce this to make filesizes more managable and reduce the processing
cost. Some level of indentation and spacing does make the files more readable and allows
use of git diff so we need to retain some of it.

(From OE-Core rev: b8be3ac506bb39665390c770f72de2ca359670b1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit a274cdcaf852cca9497f0358f44dda99c06aacbe)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-12-09 06:25:53 -08:00
Richard Purdie
417c3eb065 resulttool: Allow store to filter to specific revisions
We have a challenge on the autobuilder where test results from both OE-Core
and poky are being mixed together during result storage which is confusing the
data. Add a way to filter to specific revisions as the least worst way to fix
the various issues this is causing.

(From OE-Core rev: 42a21e6f942208da75ae1121468379044ece77dc)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 3f276a0dc65341668788853be2cf27ab6aa12b13)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-12-09 06:25:53 -08:00
Richard Purdie
0c29992ed6 efi-bootdisk.wks: Increase overhead-factor to avoid test failures
After commit d74bfb2d5c:
"linux-yocto: Enable l2tp drivers when ptest featuee is on"
was merged, oe-selftest efibootpartition.GenericEFITest.test_boot_efi
breaks due to space issues.

Increase the disk space available to avoid this and allow functional
automated testing again.

(From OE-Core rev: 76f91dba302330c89c4a742e4ef24ec1898d130f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit eb76c15de881a56ead0a18f6428c5564291249c9)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-10-29 05:51:03 -07:00
Adrian Freihofer
6e0dc47ce4 devtool: remove obsolete SRCTREECOVEREDTASKS handling
devtool modify generates a bbappend for kernel recipes which contains:
  SRCTREECOVEREDTASKS="\
    do_fetch \
    do_kernel_checkout \
    do_kernel_configcheck \
    do_unpack \
    do_validate_branches \
  "
  do_patch[noexec] = "1"

If the linux-yocto kernel is used, this is redundant. The
linux-yocto.bbclass already does the same:

linux-yocto sets SRCTREECOVEREDTASKS to
  SRCTREECOVEREDTASKS="\
    do_fetch \
    do_kernel_checkout \
    do_kernel_configcheck \
    do_patch \
    do_unpack \
    do_validate_branches \
  "

Also the do_patch[noexec] is redundant because the purpose of
SRCTREECOVEREDTASKS is to delete these tasks if the externalsrc.bbclass
is used.

The default value of SRCTREECOVEREDTASKS initialized in
externalsrc.bbclass is:
  SRCTREECOVEREDTASKS ?= "do_patch do_unpack do_fetch"
This is fine for kernels which do not inherit the linux-yocto.bbclass.

The code in devtool modify is redundant and therefore removed.

(From OE-Core rev: 94ff1be36a1eeef7ddceb4fcf20425a03cd052de)

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-09-04 12:38:44 +01:00
Adrian Freihofer
b161ca0008 devtool: modify kernel adds append twice
Drop the redundant generation of the do_configure:append section for the
kernel. The same append is generated twice:

if bb.data.inherits_class('kernel', rd):
    f.write('\ndo_configure:append() {\n'
    '    cp ${B}/.config ${S}/.config.baseline\n'
    '    ln -sfT ${B}/.config ${S}/.config.new\n'
    '}\n')

KCONFIG_CONFIG_ENABLE_MENUCONFIG ??= "true"
KCONFIG_CONFIG_ROOTDIR ??= "${B}"
if rd.getVarFlag('do_menuconfig', 'task'):
    f.write('\ndo_configure:append() {\n'
    '    if [ ${@oe.types.boolean(d.getVar("KCONFIG_CONFIG_ENABLE_MENUCONFIG"))} = True ]; then\n'
    '        cp ${KCONFIG_CONFIG_ROOTDIR}/.config ${S}/.config.baseline\n'
    '        ln -sfT ${KCONFIG_CONFIG_ROOTDIR}/.config ${S}/.config.new\n'
    '    fi\n'
    '}\n')

In contradiction to the first code block the second code block considers
the variables which is correct.

(From OE-Core rev: b0733c440e861ed7bf70efdd9b7a73afb4701218)

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-09-04 12:38:44 +01:00
Clara Kowalsky
5aeabd3217 resulttool: Add support to create test report in JUnit XML format
This adds the functionality to convert the results of the
testresults.json file to a unit test report in JUnit XML format. The
unit test report can be used in the CI/CD pipeline to display the test
results.

To use the resulttool scripts, first source oe environment, then run the
entry point script to look for help.
	$ resulttool

To generate the unit test report, execute the below
	$ resulttool junit <json_file>

By default the unit test report is stored as
<build_dir>/tmp/log/oeqa/junit.xml.

(From OE-Core rev: 3f9be03946243feaa09b908d7010899769091fe6)

Signed-off-by: Clara Kowalsky <clara.kowalsky@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-29 21:58:19 +01:00
Enguerrand de Ribaucourt
0a0caacfa5 recipetool: create_npm: reuse license utils
create_npm.py duplicated the logic for matching licenses from files and
also finding them. This patch refactors the code to reuse the license
utils. This will make the code more maintainable and also align both
behaviors. For instance, some licenses weren't matched properly because
the duplicate logic did not support the difference in format in the md5
tables for COMMON_LICENSE_DIR and licenses.csv.

This is also faster since the license files were being read twice.
The result is slightly more accurate since the utils have better
implementations, and I was able to reuse the logic for the root PN
package, as well as the base LICENSE variable.

I chose to extract generate_common_licenses_chksums into create.py
since it can be considered a general utility function to allow
other recipetool creators to refer to COMMON_LICENSE_DIR files.

I updated the wording in the code when appropriate.

v3:
 - added commit
 - this replaces the commit that added all the COMMON_LICENSE_DIR md5
   to licenses.csv

(From OE-Core rev: 7bc18bed63b94689890bcde63402d7cc1cedffa9)

Signed-off-by: Enguerrand de Ribaucourt <enguerrand.de-ribaucourt@savoirfairelinux.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-23 09:51:36 +01:00
Enguerrand de Ribaucourt
69bf37a3dd recipetool: create: split guess_license function
The npm recipetool handler redefines the license code the could be
unified. In order to do this refactoring, extract the bits we'll
need into separate functions.

guess_license() is renamed to find_licenses() and is split into
find_license_files() and match_licenses().

(From OE-Core rev: f1ec28feaea8ea6a2df894dd4ddba561c8a04ed2)

Signed-off-by: Enguerrand de Ribaucourt <enguerrand.de-ribaucourt@savoirfairelinux.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-23 09:51:36 +01:00
Enguerrand de Ribaucourt
01d17cd5d4 recipetool: create_npm: resolve licenses defined in package.json
Some npm packages do not copy the LICENSE or COPY file into their
git repository. They'll instead simply use SPDX identifiers in their
package.json. A fallback for those repositories attempted to match
the README file to a license file instead, which had a very low
probability of success.

This commit replaces this fallback with parsing the package.json and
looking for the license in COMMON_LICENSE_DIR. If the license is not
found, "Unknown" will still be produced.

This also generates "Unknown" for packages which had no README file,
which could silently not appear in the generated recipe. The user was
more likely to miss them.

Co-authored-by: Tanguy Raufflet <tanguy.raufflet@savoirfairelinux.com>
(From OE-Core rev: 445604cfc4a5813ea635f18053cd1f673bf0b830)

Signed-off-by: Tanguy Raufflet <tanguy.raufflet@savoirfairelinux.com>
Signed-off-by: Enguerrand de Ribaucourt <enguerrand.de-ribaucourt@savoirfairelinux.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-23 09:51:36 +01:00
Alexander Kanavin
18e4194539 devtool/upgrade: rename RECIPE_UPDATE_EXTRA_TASKS -> RECIPE_UPGRADE_EXTRA_TASKS
'UPDATE' as a name is somewhat unfortunate as the variable is intended only for
the 'devtool upgrade' operation and devtool also has an 'update-recipe' operation.

(From OE-Core rev: 4467aa0661e233f44c4ce029428c67d88fccfc07)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-03 07:56:10 +01:00
Marcus Folkesson
d045fed31a bootimg-partition: break out code to a common library.
Break out the code that parse IMAGE_BOOT_FILES to a common library.

(From OE-Core rev: 1e07fe51bdb24070308c85e83df0b80ab9f83cea)

Signed-off-by: Marcus Folkesson <marcus.folkesson@gmail.com>
Reviewed-by: Quentin Schulz <quentin.schulz@cherry.de>
Reviewed-by: Konrad Weihmann <kweihmann@outlook.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-23 11:17:11 +01:00
Alexander Kanavin
15e5193e9f devtool/upgrade: use PN instead of BPN for naming newly created upgraded recipes
BPN isn't correct, as it is set to 'cmake' when 'cmake-native' is being upgraded
(or libva for libva-initial etc.)

(From OE-Core rev: e634316547f86ce5662fa4899440a5af1047d494)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-22 16:53:06 +01:00
Alexander Kanavin
3d98aafc43 recipeutils/get_recipe_upgrade_status: group recipes when they need to be upgraded together
This will allow 'lockstep upgrades' of such recipes, improving success
rates in automated version updating process.

devtool check-upgrade-status now prints:

These recipes need to be upgraded together {
glib-2.0                  2.80.2          2.80.4          Anuj Mittal <anuj.mittal@intel.com>
glib-2.0-initial          2.80.2          2.80.4          Anuj Mittal <anuj.mittal@intel.com>
}
These recipes need to be upgraded together {
util-linux                2.39.3          2.40.2          Chen Qi <Qi.Chen@windriver.com>
util-linux-libuuid        2.39.3          2.40.2          Chen Qi <Qi.Chen@windriver.com>
}
These recipes need to be upgraded together {
cmake                     3.29.3          3.30.0          Unassigned <unassigned@yoctoproject.org>
cmake-native              3.29.3          3.30.0          Unassigned <unassigned@yoctoproject.org>
}

etc.

(From OE-Core rev: 7874aea5c62be3e8dbd19e04fce5389c5ed7aab6)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-22 16:53:06 +01:00
Alexander Kanavin
597b87a468 lib/oe/recipeutils: return a dict in get_recipe_upgrade_status() instead of a tuple
Putting various things in a tuple is an anti-pattern of sorts, as the consumers
have to unpack it into local variables for readability, or access items directly
with indexes, which makes code pretty much unreadable.

(From OE-Core rev: e86aa26d209eb9809198f6dd40cd058366318e3d)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-22 16:53:06 +01:00
Adithya Balakumar
c79c8ce71c wic/partition.py: Set hash_seed for empty ext partition
Although setting hash_seed is handled for the rootfs plugin case, but
this is missed when deploying an empty ext partition.

(From OE-Core rev: 0202fb594fb05098cb8d8b6088e63beb40b5906e)

Signed-off-by: Adithya Balakumar <adithya.balakumar@toshiba-tsip.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-06-18 08:44:41 +01:00
Tim Orling
82d07928fe devtool upgrade: enable RECIPE_UPDATE_EXTRA_TASKS
For some recipes, such as those that inherit cargo-update-recipe-crates,
we need to run additional tasks once the new sources have been unpacked.

Introduce a new variable RECIPE_UPDATE_EXTRA_TASKS which is a space-
delimited list of tasks to run after the new sources have been
unpacked in scripts/lib/devtool/upgrade.py ugrade() method.

(From OE-Core rev: 59894f3b5b0bc257837d7ce4ea684f1d8c382cec)

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-06-13 09:11:17 +01:00
Enrico Jörns
1d5b4d19c7 wic: bootimg-efi: fix error handling
If we check for empty variables, it does not make sense to print their
content in case they are empty.

Additionally, the error message in the 'kernel' check attempted to print
the wrong variable ('target') which was not even defined, yet.

Also, raising WicError doesn't require an extra newline.

(From OE-Core rev: f31cf475d0235f42d73aeec07694f79b9937fd76)

Signed-off-by: Enrico Jörns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-06-06 11:56:04 +01:00
Enrico Jörns
4f600110ee wic: engine.py: use raw string for escape sequence
Fixes:

| poky/scripts/lib/wic/engine.py:362: SyntaxWarning: invalid escape sequence '\/'

(From OE-Core rev: e33d7241f6c2897e930aff41e18b154891197ab9)

Signed-off-by: Enrico Jörns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-06-06 11:56:04 +01:00
Ninette Adhikari
f66f8fa9f6 oe-build-perf-report: Add commit hash link to chart tooltip
Update tooltip to include the commit hash link to the poky repository

(From OE-Core rev: 1187a460c2608369df8de08fe52a5542e4ab6ecd)

Signed-off-by: Ninette Adhikari <ninette@thehoodiefirm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-24 16:59:31 +01:00
Richard Purdie
569f8e09f0 recipetool/devtool: Update to work correctly with UNPACKDIR
Tweak recipetool and devtool to correctly use UNPACKDIR. This allows some
simplification of the code. This patch makes things basically work but there
are likely deeper improvements that can be made now that WORKDIR != UNPACKDIR.

(From OE-Core rev: d2eeaa88b27a2875c419591d1d91bcc85d7b129c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-22 22:26:30 +01:00
Richard Purdie
6d77296d83 devtool: Drop oe-local-files and simplify
The only real reason for oe-local-files was to support S = WORKDIR. With changes to
drop support for that, it makes sense to simplify devtool and to try and make both
the code and the processes/workflows simpler.

This patch drops support for S = WORKDIR, removes oe-local-files and then updates
the test cases to match this new situation.

At the code level, we assume we can always now track code changes using git and
that things committed into git are handled as patches (as before) but delta against
HEAD is saved as specific file level changes to the recipe.

One test is disabled as it is no longer approproate. It is being keped until we can
make WORKDIR != UNPACKDIR at which point it should be revisited.

(From OE-Core rev: ce8190c519052fed10b5233697b69a75868db45a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-21 12:08:04 +01:00
Anton Almqvist
9a894dac19 devtool: modify: Catch git submodule error for go code
One of the git submodule commands failed for source extracted for
recipes using go.bbclass.  The root cause is probably the path set up
by go_do_unpack which makes S and gitroot not match.

This patch does not fix the root problem, but at least it is no worse
than before the git submodule support.

The extracted source will still have two .git folders, one in S
created by devtool and one in the go path which will contain the tru
git history.

[ YOCTO #15483 ]

(From OE-Core rev: fe242408af40dd1f6e47d9b2b232bdc76756c80a)

Signed-off-by: Anton Almqvist <antonal@axis.com>
Signed-off-by: Ola x Nilsson <olani@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-15 16:39:19 +01:00
Ninette Adhikari
cf0c866b51 oe-build-perf-report: Add dark mode
Update css to add dark mode when window prefers-color-scheme is dark.

(From OE-Core rev: ed02a235d42202279ad5e4e3153247f9e5e2bba8)

Signed-off-by: Ninette Adhikari <ninette@thehoodiefirm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-15 16:39:19 +01:00
Ninette Adhikari
20fafa0854 oe-build-perf-report: Update chart tooltip and chart type
- Update chart tooltip format to show value as size in MB for 'rootfs size'
and timestamp for 'tmpdir size'
- Add commit number to tooltip
- Update chart type to 'step chart' instead of 'line chart'

(From OE-Core rev: fb41cbbe6008e442b6eac77308eadeae327eed7d)

Signed-off-by: Ninette Adhikari <ninette@thehoodiefirm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-15 16:39:19 +01:00
Ninette Adhikari
314a220280 oe-build-perf-report: Improve report styling and add descriptions
Styling updates are added including page margin, labels for x and y axis, tooltip, and section descriptions.

(From OE-Core rev: 2a2568fcf1a3f8e467bd814f4fb13dffae8ec61d)

Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-15 16:39:19 +01:00
Ninette Adhikari
99861c17e7 oe-build-perf-report: Display more than 300 commits and date instead of commit number
- This commit updates measurement statistics data to include start_time so that time can be displayed instead of commit numbers on the chart.
- It also updates default commit history length to 300.

(From OE-Core rev: 64275a41c37130aaaacc5e592f94a1afe057119b)

Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-15 16:39:19 +01:00
Ninette Adhikari
836b4b57d6 oe-build-perf-report: Add apache echarts to make report interactive
- Add Apache echarts (https://echarts.apache.org/en/index.html) library to create build performance charts.
- Restructure data to time and value array format so that it can be used by echarts.
- This commit also converts test duration to minutes to map against the values axis.
- Zoom is added to the line charts.

(From OE-Core rev: 63c9321832aae79d20a4ddd199a4a1385f81de53)

Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-15 16:39:19 +01:00
Antonin Godard
12c447b38a devtool: ide-sdk: correct help typo
(From OE-Core rev: 50e043387a2f0f9a5c2f7a5f914c465c830d329b)

Signed-off-by: Antonin Godard <antoningodard@pm.me>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-09 09:48:58 +01:00
Richard Purdie
808f700efc classes/lib/scripts: Initial WORKDIR -> UNPACKDIR updates
Work through the initial issues I found where we need to change WORKDIR
to UNPACKDIR.

(From OE-Core rev: 86fec41b1e809d1a2fa2feadc26d29020df53d39)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-02 15:15:51 +01:00
John Ripple
5756ac9936 rootfs.py: Fix logger error message format
--_000_PH7PR17MB61307CB6690EC00DEB4ED9B39C0F2PH7PR17MB6130namp_
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

This patch adds a missing "%s" format in a logger.error call. Without
this addition the logger itself would error out and not print a useful
message.

(From OE-Core rev: 2c892e5dd9ba72a51c0a8fb851599cc2dc3a8b5c)

Signed-off-by: John Ripple <john.ripple@keysight.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-04-25 10:19:28 +01:00
Julien Stephan
28516320d8 devtool: standard: update-recipe/finish: fix update localfile in another layer
When trying to use devtool update-recipe/finish on another layer, with modified
local file we have the following error:

  Traceback (most recent call last):
    File "<..>/poky/scripts/devtool", line 350, in <module>
      ret = main()
            ^^^^^^
    File "<..>/poky/scripts/devtool", line 337, in main
      ret = args.func(args, config, basepath, workspace)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "<..>/poky/scripts/lib/devtool/standard.py", line 1968, in update_recipe
      updated, _, _ = _update_recipe(args.recipename, workspace, rd, args.mode, args.append, args.wildcard_version, args.no_remove, args.initial_rev, dry_run_outdir=dry_run_outdir, no_overrides=args.no_overrides, force_patch_refresh=args.force_patch_refresh)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "<..>/poky/scripts/lib/devtool/standard.py", line 1930, in _update_recipe
      updated, appendf, removed = _update_recipe_patch(recipename, workspace, srctree, crd, appendlayerdir, wildcard_version, no_remove, no_report_remove, initial_rev, dry_run_outdir, force_patch_refresh)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "<..>/poky/scripts/lib/devtool/standard.py", line 1747, in _update_recipe_patch
      patchdir = param.get('patchdir', ".")
                 ^^^^^^^^^
  AttributeError: 'str' object has no attribute 'get'

This was introduced when adding support for git submodules.
No selftest case exists to catch this, so a selftest will be
added in another commit.

(From OE-Core rev: de7ca9f800e15e10271502da7e51e3ae08e0c85b)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-04-25 10:10:14 +01:00
Vincent Kriek
629063944c devtool: sync: Fix Execution error
When executing devtool sync on a recipe that was extract with devtool
extract earlier the following error occured:

    Traceback (most recent call last):
    [...]
    bb.process.ExecutionError: Execution of 'git fetch file:///home/vin/projects/poky/build/tmp/work/all-poky-linux/netbase/6.4/devtooltmp-figt1jmr/workdir/netbase devtool:devtool' failed with exit code 128:
    fatal: refusing to fetch into branch 'refs/heads/devtool' checked out at '/home/vin/projects/poky/build/netbase-src'

Fix this by adding --update-head-ok and --force to git fetch so it will
override the current head even if it is checked out and has changes.
Possible existing changes in the devtool branch can be retrieved by
checking out the devtool.bak branch

(From OE-Core rev: 4c89b061003a0f4143c6d3838e39fc2643edc8d6)

Signed-off-by: Vincent Kriek <vincent@coelebs.dev>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-04-25 10:10:14 +01:00
Sven Schwermer
8219eefdd8 recipetool: Handle several go-import tags in go resolver
When dynamically resolving go modules, the HTML page may contain several
go-import meta tags. We must handle all and pick the correct one based
on the module name. An example for such a behaviour is
gonum.org/v1/gonum:

<meta name="go-import" content="gonum.org/v1/exp git https://github.com/gonum/exp">
<meta name="go-import" content="gonum.org/v1/gonum git https://github.com/gonum/gonum">
<meta name="go-import" content="gonum.org/v1/hdf5 git https://github.com/gonum/hdf5">
<meta name="go-import" content="gonum.org/v1/netlib git https://github.com/gonum/netlib">
<meta name="go-import" content="gonum.org/v1/plot git https://github.com/gonum/plot">
<meta name="go-import" content="gonum.org/v1/tools git https://github.com/gonum/tools">

(From OE-Core rev: 9c36a61e29359067165bddc7f2accdf2c4c8a761)

Signed-off-by: Sven Schwermer <sven.schwermer@disruptive-technologies.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-04-23 13:40:24 +01:00
Sven Schwermer
e4c3483ecf recipetool: Handle unclean response in go resolver
It appears that some go modules repond with a 404 error when trying to
resolve them dynamically. The response body may still contain the
go-import meta tag. An example for such behaviour is gonum.org/v1/gonum.

(From OE-Core rev: 8f2e14ab6562a9a68819a960c66a258ea9dbe246)

Signed-off-by: Sven Schwermer <sven.schwermer@disruptive-technologies.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-04-23 13:40:24 +01:00
Julien Stephan
834e91ce53 devtool: standard: throws appropriate error if source is in detached HEAD
If source is in detached HEAD, we get the following error when using
detvool finish command:

    [...]
    File "<...>/poky/scripts/lib/devtool/standard.py", line 1938, in _update_recipe
      bb.process.run('git checkout %s' % startbranch, cwd=srctree)
    File "<...>/poky/bitbake/lib/bb/process.py", line 189, in run
      raise ExecutionError(cmd, pipe.returncode, stdout, stderr)
  bb.process.ExecutionError: Execution of 'git checkout (HEAD detached at 9bbf87e)' failed with exit code 2:
  /bin/sh: -c: line 1: syntax error near unexpected token `('
  /bin/sh: -c: line 1: `git checkout (HEAD detached at 9bbf87e)'

Check this and throws an appropriate error in this case

(From OE-Core rev: d9c686b5ff9f591ec6b928ed539084c02df4c8a5)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-04-12 17:27:53 +01:00
Anibal Limon
7c3f956a0a wic: bootimg-partition allow to set var to get boot files
This is a generic functionality that takes a set of files and create a
boot parition based on IMAGE_BOOT_FILES var, allow to sepcify the
variable name so a BSP plugin can extend this and re-use the
functionality.

(From OE-Core rev: 3a2339455be29a6ff65c8c5f6865a4cd35dd7f63)

Signed-off-by: Anibal Limon <anibal@limonsoftware.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-03-26 17:10:51 +00:00
Trevor Woerner
f15676592b bmaptool: now part of Yocto Project
The bmaptool (previously: bmap-tools, bmap-tool, bmaptool) has been moved
to be under the Yocto Project umbrella and is now hosted at:

	github.com/yoctoproject/bmaptool

[RP: Added a couple of missing renames]
(From OE-Core rev: 7a036b1a1ec7dcd27dbe18d4c2e703bd2a8af182)

Signed-off-by: Trevor Woerner <twoerner@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-03-05 12:24:50 +00:00
Adrian Freihofer
c82c57621b devtool: ide-sdk prefer sources from workspace
Improve the previous commit:
- log an error if some assumptions are not true
- Use TARGET_DBGSRC_DIR variable
- Do the same for ide none

Why the additional source mapping is required:

For example the cmake-example recipe refers to sources like this:
./recipe-sysroot-native/usr/bin/x86_64-poky-linux/x86_64-poky-linux-readelf \
  -wi image/usr/bin/cmake-example | grep -B1 DW_AT_comp_dir
    ...
    <560>   DW_AT_name        : (indirect line string, offset: 0x1da):
    /usr/src/debug/cmake-example/1.0/oe-local-files/cpp-example.cpp
    ...

Another example is powertop:
./recipe-sysroot-native/usr/bin/x86_64-poky-linux/x86_64-poky-linux-readelf \
  -wi image/usr/sbin/powertop | grep -B1 DW_AT_comp_dir
    ...
    <561>   DW_AT_name        : (indirect line string, offset: 0x1da):
    /usr/src/debug/powertop/2.15/src/devlist.cpp
    ...

For recipes with local files this works. The oe-local-files folder is
not available in the rootfs-dbg and therefore the sources are first
found in the workspace folder. GDB searches for source files in various
places:
https://sourceware.org/gdb/current/onlinedocs/gdb.html/Source-Path.html
However, for the powertop example the sources opened in the editor are
from the rootfs-dbg instead of from the workspace.

Bitbake calls the compiler with
  -fmacro-prefix-map=${S}=${TARGET_DBGSRC_DIR}
where TARGET_DBGSRC_DIR defaults to "/usr/src/debug/${PN}/${PV}".

A source map which maps the recipe specific path from TARGET_DBGSRC_DIR
to the workspace fixes this.
The already existing source map for /usr/src/debug applies for all other
recipes. It finds the sources (read only) in the rootfs-dbg folder.

(From OE-Core rev: 06601632c1879cb80276f9b36de91fb7808311a5)

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-27 11:38:46 +00:00
Enguerrand de Ribaucourt
a5b75e29ff devtool: ide-sdk source mapping for vscode
When launching the debug configuration, the source files from the debug
rootfs were openened in the editor instead of the local workspace files.
We add an exception to properly map them to the file being developed and
compiled by the IDE integration. This also more closely matches what the
user would expect compared to native development.

This is also true for the devtool fallback mode.

(From OE-Core rev: 24db2b8d0d7104960c1cdb2c7ee5216c830a6754)

Signed-off-by: Enguerrand de Ribaucourt <enguerrand.de-ribaucourt@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-27 11:38:46 +00:00