Commit Graph

195 Commits

Author SHA1 Message Date
Richard Purdie
07563ec1b5 buildhistory: Update for bitbake changes
Bitbake is dropping the need for fetcher name iteration and multiple revisions
per url. Update the code to match (removal of the for loop).

(From OE-Core rev: 571e8c6730eb5b89f732d879487ae578ef04aaee)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-03-20 13:50:40 +00:00
Peter Kjellerstedt
58deeacb07 buildhistory.bbclass: Do not create annotated tags if tag.gpgSign is set
If tag.gpgSign is configured in Git's configuration, then creating the
build-minus-X tags will fail (if Git's core.editor is not configured) or
it will hang (when trying to open the editor). This is beacause
tag.gpgSign causes git tag to create annotated tags. To avoid this,
specify --no-sign as argument to git tag.

(From OE-Core rev: 7595a0a63a933af9dd9d1e458dc34a4ba80d9eae)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-03-19 12:09:06 +00:00
Richard Purdie
e11cf39bf9 buildhistory: Drop git-gc call as obsolete
buildhistory used to result in many individual objects in the repo which
could affect performance. This gc call was added over a decade ago to avoid
those performance issues. Modern git calls git-gc when needed and this kind
of workaround should no longer be needed.

There is a small chance this is contributing to problems with the autobuilder
buildhistory repositories, so simplifying the code may help that too.

(From OE-Core rev: 0a88f9109f7670a6b188463a19392d445af6ff40)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-12-03 11:21:34 +00:00
Richard Purdie
ead03cae66 buildhistory: Simplify intercept call sites and drop SSTATEPOSTINSTFUNC usage
We planned to drop SSTATEPOSTINSTFUNC some time ago with the introduction of
postfuncs. Finally get around to doing that which should make the buildhistory
code a little more readable.

Unfortunately ordering the buildhistory function calls after the sstate ones is
difficult without coding that into the sstate class. This patch does that to
ensure everything functions as expected until we can find a better way. This is
still likely preferable than the generic sstate postfuncs support since the function
flow is much more readable.

(From OE-Core rev: c9e2a8fa2f0305ef1247ec405555612326f798f8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-09-04 12:38:44 +01:00
Pedro Ferreira
7355465f9e buildhistory: Restoring files from preserve list
This fix will ensure that, when we activate feature
`BUILDHISTORY_RESET`, files marked to keep on feature
`BUILDHISTORY_PRESERVE` will indeed exist is buildhistory
final path since they are moved to buildhistory/old but
not restored at any point.

(From OE-Core rev: 9f68a45aa238ae5fcdfaca71ba0e7015e9cb720e)

Signed-off-by: Pedro Ferreira <Pedro.Silva.Ferreira@criticaltechworks.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-07 15:47:15 +01:00
Pedro Ferreira
738b3b1e65 buildhistory: Fix intermittent package file list creation
The directory that buildhistory_list_pkg_files writes to during do_package
is created by do_packagedata so a clean buildhistory doesn't have
files-in-package written during the first build since packagedata happens
after do_package.

Ensure the output package folder is created to avoid missing
files-in-package.txt files.

Also it ensures that in case of `find` fails we leave with
a hard error instead of hiding the error on the for loop.

(From OE-Core rev: 8de9b8c1e199896b9a7bc5ed64967c6bfbf84bea)

Signed-off-by: Pedro Silva Ferreira <Pedro.Silva.Ferreira@criticaltechworks.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-07 15:47:15 +01:00
Ulrich Ölmann
b2bc061f7c buildhistory: fix typos
Fix some simple typos found while looking through the code.

(From OE-Core rev: c23cb2d89fcd014d1f08944468c7e2d8a7f3e8b0)

Signed-off-by: Ulrich Ölmann <u.oelmann@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-04 22:56:15 +01:00
Richard Purdie
6fd8af0d30 classes: Drop ';' delimiter from ROOTFS/IMAGE*COMMAND variables
Originally these were shell functions but they have long since been processed by
bb.build.exec_func(). Since we no longer need shell syntax, we can drop the ';'
delimiters and just use a space separated string.

This cleans up the variable and quietly removes any stray ';' that do happen to
still make it in.

(From OE-Core rev: c3365dfd9ddd7fbe70b62e0f11166e57a8ca6f73)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-09 22:14:41 +01:00
Richard Purdie
ff525695f2 classes: Add SPDX license identifiers
As stated in our top level license files, the license is MIT unless
otherwise stated. Add SPDX identifers accordingly. Replace older
license statementa with the standardised syntax. Also drop "All
Rights Reserved" expression as it isn't used now, doesn't mean anything
and is confusing.

(From OE-Core rev: 081a391fe09a21265881e39a2a496e4e10b4f80b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-12 11:58:01 +01:00
Richard Purdie
883aea9b6e buildhistory: Only use image-artifact-names as an image class
The class only makes sense in the conext of images, not globally so
include as such.

(From OE-Core rev: 4fdda4d6bd434c240ec1dfdbfc6eb7e25e2db1f3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-12 11:46:26 +01:00
Richard Purdie
38a9622ba3 base/buildhistory/image-buildinfo: Use common buildcfg function
Reduce code duplication to a common function in oe.buildcfg.

(From OE-Core rev: 0b191a1df4c3722defb09dde0c16b1d9e7fe7ef6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-11 10:30:57 +01:00
Richard Purdie
5266670b2d lib/buildcfg: Share common clean/dirty layer function
The comments even say this was copy/paste code. Move to a
shared library function.

(From OE-Core rev: ac3de2f850a418673b87e1c454970cb099e191b0)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-11 10:30:57 +01:00
Richard Purdie
ba85bb6055 classes/buildcfg: Move git/layer revision code into new OE module buildcfg
There is a load of duplicated git/layer/revision code which makes
most sesne as a python library, not bbclass code. Start to refactor as such.

(From OE-Core rev: 439cdf8a1e52fd2c4dc81dc40ce7e6af282ce7ac)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-11 10:30:56 +01:00
Martin Jansa
eadc2ba4cd buildhistory.bbclass: fix shell syntax when using dash
* fix:
  run.buildhistory_get_image_installed.791888: 198: [: unexpected operator

  introduced in:
  commit 82e6172c1df378dff4e503aa878501c08937b5bb
  Author: Andres Beltran <abeltran@linux.microsoft.com>
  Date:   Tue Oct 5 00:34:15 2021 +0000

    buildhistory: Fix package output files for SDKs

(From OE-Core rev: 9852bedcb9eb8306a3f82f805ec4f76abbbbdd0f)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-08 21:55:56 +01:00
Jose Quaresma
39aec4ac7f buildhistory.bbclass: create the buildhistory directory when needed
When the BUILDHISTORY_RESET is enabled we need to move the
content from BUILDHISTORY_DIR to BUILDHISTORY_OLD_DIR but
when we start a clean build in the first run we don't have the
BUILDHISTORY_DIR so the move of files will fail.

| ERROR: Command execution failed: Traceback (most recent call last):
|  File "/xxx/poky/bitbake/lib/bb/command.py", line 110, in runAsyncCommand
|    commandmethod(self.cmds_async, self, options)
|  File "/xxx/poky/bitbake/lib/bb/command.py", line 564, in buildTargets
|    command.cooker.buildTargets(pkgs_to_build, task)
|  File "/xxx/poky/bitbake/lib/bb/cooker.py", line 1481, in buildTargets
|    bb.event.fire(bb.event.BuildStarted(buildname, ntargets), self.databuilder.mcdata[mc])
|  File "/xxx/home/builder/src/base/poky/bitbake/lib/bb/event.py", line 214, in fire
|    fire_class_handlers(event, d)
|  File "/xxx/poky/bitbake/lib/bb/event.py", line 121, in fire_class_handlers
|    execute_handler(name, handler, event, d)
|  File "/xxx/poky/bitbake/lib/bb/event.py", line 93, in execute_handler
|    ret = handler(event)
|  File "/xxx/poky/meta/classes/buildhistory.bbclass", line 919, in buildhistory_eventhandler
|    entries = [ x for x in os.listdir(rootdir) if not x.startswith('.') ]
| FileNotFoundError: [Errno 2] No such file or directory: '/xxx/buildhistory'

(From OE-Core rev: 97bc2168da7dbacdfbf79cd70db674363ab84f6b)

Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-02 18:43:25 +00:00
Richard Purdie
71ef319193 meta/scripts: Automated conversion of OE renamed variables
(From OE-Core rev: aa52af4518604b5bf13f3c5e885113bf868d6c81)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-21 23:37:27 +00:00
Richard Purdie
492214104a meta/scripts: Change BB_ENV_EXTRA_WHITE -> BB_ENV_PASSTHROUGH_ADDITIONS
After the change to bitbake, update the references in OE-Core to match the updates.

(From OE-Core rev: 193affb9f28b0116c3fd619834f145326fee08c5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-21 23:37:26 +00:00
Richard Purdie
3ddbfe0b15 buildhistory: Fix do_package race issues
The buildhistory_list_pkg_files function uses data from do_package, not
do_packagedata. Usally the two are restored together but it may see
a half complete directory or other races issues depending on timing.

Rework the function so that it uses the correct task dependencies. This
should avoid races but means the data is only restored to buildhistory
if the do_package or do_package_setscene tasks are restored.

(From OE-Core rev: b83823ce44e7531bbd2bfa62062c04147a11f724)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-24 09:55:53 +00:00
Richard Purdie
96f552377a buildhistory: Fix srcrevs output
The code was assuming that the a recipe with only one srcrev wouldn't "name"
it. This isn't the case as the glibc or bzip2 recipes show, you can have
a single srcrev which is named.

We can pull the data from the fetcher and in fact we already have it, we just
need to handle the "default" case and make that code the default for all srcrev
regardless of length.

[YOCTO #14017]

(From OE-Core rev: 45ae567932ba52b758eb41754453e9828d9533a1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-23 10:53:15 +00:00
Richard Purdie
4819440904 buildhistory: Drop support for older bitbakes
We've bumped the minimum bitbake version past the point this fallback code
was needed, drop it.

(From OE-Core rev: c45ad27f3c43c9bbde5c4d19237411fdbc66920e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-23 10:53:15 +00:00
Mike Crowe
51b1611e20 lib/oe/qa,insane: Move extra error handling functions to library
Extract package_qa_write_error, package_qa_handle_error and
package_qa_add_message functions from insane.bbclass to lib/oe/qa.py and
drop the package_qa_ prefixes.

Update various bbclasses to use the new functions. No import is required
since base.bbclass puts oe.qa in OE_IMPORTS.

Stop requiring callers to manually track whether a fatal error has been
encountered via a "sane" flag. Instead replace the QA_SANE variable with
QA_ERRORS_FOUND and call oe.qa.exit_if_errors or
oe.qa.exit_with_message_if_errors at the end of each task.

Inspired by discussion resulting from
https://lists.openembedded.org/g/openembedded-core/message/156793 and
https://lists.openembedded.org/g/openembedded-core/message/156900

(From OE-Core rev: f0ad152ef4cc15c042bc9eeefb6af096d054b220)

Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-17 11:56:32 +01:00
Andres Beltran
f3518bcdab buildhistory: Fix package output files for SDKs
Currently, installed packages are listed for images in image-info.txt, but
not for SDKs in sdk-info.txt. Add TOOLCHAIN_HOST_TASK and
TOOLCHAIN_TARGET_TASK to the output variables in sdk-info.txt.

Moreover, package output files for the SDK host are empty because
PKGDATA_DIR defaults to the target directory. Fix this bug and create a new
variable called PKGDATA_DIR_SDK which stores the correct path for the SDK
host package data.

(From OE-Core rev: 82e6172c1df378dff4e503aa878501c08937b5bb)

Signed-off-by: Andres Beltran <abeltran@linux.microsoft.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-14 11:22:18 +01:00
Andres Beltran
157462fc2a buildhistory: Label packages providing per-file dependencies in depends.dot
Currently, depends.dot includes per-file dependencies but not the packages
providing those files. This makes it hard to obtain all package
dependencies by just looking at depends.dot.

Parse the RPROVIDES and FILERPROVIDES fields from pkgdata to map each of
their values to the package providing the component. Include runtime
packages as dependencies in depends.dot, together with the component
provided by the package as a label.

(From OE-Core rev: 2ba33093017574bbe29eeba699eb90628614d03a)

Signed-off-by: Andres Beltran <abeltran@linux.microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-01 14:06:18 +01:00
Andres Beltran
37e0565c11 buildhistory: Add output file listing package information
Currently, buildhistory does not produce a single file combining relevant
information of installed packages. Produce an output file
"installed-package-info.txt" listing a package's runtime name, buildtime name,
its recipe, version, and size to avoid having to look up each package externally.
Leave the existing package list files as-is for backwards compatibility.

In order to support this efficiently, extend oe-pkgdata-util to accept multiple keys
for its read-value argument.

(From OE-Core rev: 1e18b514bf1f960d324a21db608c8e8e5af007ef)

Signed-off-by: Andres Beltran <abeltran@linux.microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-27 11:54:34 +01:00
Richard Purdie
bb6ddc3691 Convert to new override syntax
This is the result of automated script conversion:

scripts/contrib/convert-overrides.py <oe-core directory>

converting the metadata to use ":" as the override character instead of "_".

(From OE-Core rev: 42344347be29f0997cc2f7636d9603b1fe1875ae)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-02 15:44:10 +01:00
Joshua Watt
ba491952ee classes/buildhistory: Add option to strip path prefix
Adds an option to strip a prefix from the paths reported in
buildhistory. This makes it easier to compare task signatures in the
build history when the builds were done from different directories.

(From OE-Core rev: 194e7a29212c4a29222730f47d3133dfe92447c1)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-03 23:09:41 +01:00
Devendra Tewari
b71375304f classes/lib/scripts: Use bb.utils.rename() instead of os.rename()
Incremental build in Docker fails with:

OSError: [Errno 18] Invalid cross-device link

when source and destination are on different overlay filesystems.

Rather than adding fallback code to every call site, use a new wrapper
in bitbake which detects this case and falls back to shutil.move
which is slower but will handtle the overlay docker filesystems correctly.

[YOCTO #14301]

(From OE-Core rev: 656a65b2b84e7d529b89cf5de7eb838f902d84a2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-06 11:16:23 +01:00
Christopher Larson
8eb55de821 buildhistory: add missing vardepsexcludes
For POPULATE_SDK_POST_TARGET_COMMAND, POPULATE_SDK_POST_HOST_COMMAND, and SDK_POSTPROCESS_COMMAND, the appropriate entries were added to vardepvalueexclude, but we want them in vardepsexclude as well.

(From OE-Core rev: 554b17e0bbe5190e4b03121f2ed06f4845012a71)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-31 08:51:02 +01:00
Peter Bergin
55ead1be58 buildhistory.bbclass: avoid exception for empty BUILDHISTORY_FEATURES variable
An exception is fired when a BuildStarted event is sent to buildhistory bbclass
and the variable BUILDHISTORY_FEATURES is not set.

    ERROR: Execution of event handler 'buildhistory_eventhandler' failed
    Traceback (most recent call last):
      File "<...>/meta/classes/buildhistory.bbclass", line 862, in buildhistory_eventhandler(e=<bb.event.BuildStarted object at 0x7f94c3810250>):
         python buildhistory_eventhandler() {
        >    if e.data.getVar('BUILDHISTORY_FEATURES').strip():
                 reset = e.data.getVar("BUILDHISTORY_RESET")
    AttributeError: 'NoneType' object has no attribute 'strip'

This can happen in a multiconfig build where the default configuration use the
buildhistory class but not the configuration in mc. It should be a rare case that
this happens and it was found in a missconfigured build.

(From OE-Core rev: a74e30a4de02c8efd3e7102ba7a4fe06df53cc34)

Signed-off-by: Peter Bergin <peter@berginkonsult.se>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-29 21:21:53 +00:00
Saul Wold
355ca8732c classes/buildhistory: record LICENSE
Buildhistory stores various bits of information at both recipe
and package level, while there is an associated license manifest
directory tree it would require additional scripting to extract
that information.

(From OE-Core rev: 909bafef282f00dd4a83fab0569885e9788a4ed9)

Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-29 20:24:32 +00:00
Richard Purdie
14335a18dd Revert "classes/buildhistory: also save recipe info for native recipes"
This reverts commit d123606c4bef85c2436b40f51e47b602b7600c0b.

This change contains races as it will start poking into do_package task
directories from do_populate_sysroot. If we want to do this for native
recipes, we need to add guards around the package code and only make
this happen for native in populate_sysroot, not target in
populate_sysroot too. Backtrace from an example problem below:

ERROR: openssl-1.1.1g-r0 do_populate_sysroot: Error executing a python function in exec_python_func() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:buildhistory_emit_pkghistory(d)
     0003:
File: '/home/ross/Yocto/poky/meta/classes/buildhistory.bbclass', lineno: 319, function: buildhistory_emit_pkghistory
     0315:
     0316:        write_pkghistory(pkginfo, d)
     0317:
     0318:    # Create files-in-<package-name>.txt files containing a list of files of each recipe's package
 *** 0319:    bb.build.exec_func("buildhistory_list_pkg_files", d)
     0320:}
     0321:
     0322:python buildhistory_emit_outputsigs() {
     0323:    if not "task" in (d.getVar('BUILDHISTORY_FEATURES') or "").split():
File: '/home/ross/Yocto/poky/bitbake/lib/bb/build.py', lineno: 256, function: exec_func
     0252:    with bb.utils.fileslocked(lockfiles):
     0253:        if ispython:
     0254:            exec_func_python(func, d, runfile, cwd=adir)
     0255:        else:
 *** 0256:            exec_func_shell(func, d, runfile, cwd=adir)
     0257:
     0258:    try:
     0259:        curcwd = os.getcwd()
     0260:    except:
File: '/home/ross/Yocto/poky/bitbake/lib/bb/build.py', lineno: 503, function: exec_func_shell
     0499:    with open(fifopath, 'r+b', buffering=0) as fifo:
     0500:        try:
     0501:            bb.debug(2, "Executing shell function %s" % func)
     0502:            with open(os.devnull, 'r+') as stdin, logfile:
 *** 0503:                bb.process.run(cmd, shell=False, stdin=stdin, log=logfile, extrafiles=[(fifo,readfifo)])
     0504:        except bb.process.ExecutionError as exe:
     0505:            # Find the backtrace that the shell trap generated
     0506:            backtrace_marker_regex = re.compile(r"WARNING: Backtrace \(BB generated script\)")
     0507:            stdout_lines = (exe.stdout or "").split("\n")
File: '/home/ross/Yocto/poky/bitbake/lib/bb/process.py', lineno: 184, function: run
     0180:        if not stderr is None:
     0181:            stderr = stderr.decode("utf-8")
     0182:
     0183:    if pipe.returncode != 0:
 *** 0184:        raise ExecutionError(cmd, pipe.returncode, stdout, stderr)
     0185:    return stdout, stderr
Exception: bb.process.ExecutionError: Execution of '/yocto/ross/build/tmp/work/neoversen1-poky-linux/openssl/1.1.1g-r0/temp/run.buildhistory_list_pkg_files.4158804' failed with exit code 2:
/yocto/ross/build/tmp/work/neoversen1-poky-linux/openssl/1.1.1g-r0/temp/run.buildhistory_list_pkg_files.4158804: 183: cd: can't cd to /yocto/ross/build/tmp/work/neoversen1-poky-linux/openssl/1.1.1g-r0/packages-split/openssl-engines

(From OE-Core rev: 59aac6e134289d657d80bfb1d6f25b388d539818)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-26 22:08:36 +00:00
Paul Eggleton
1932b04a1e classes/buildhistory: also save recipe info for native recipes
If we want to also collect SRC_URI for native recipes we need to ensure
that the code that writes out all of the recipe info is called - there
isn't a do_packagedata for native recipes so we need to piggyback on
do_populate_sysroot instead.

(From OE-Core rev: d123606c4bef85c2436b40f51e47b602b7600c0b)

Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-20 11:11:46 +01:00
Paul Eggleton
71191e8e7a classes/buildhistory: record SRC_URI
It can be useful to record SRC_URI into buildhistory for the purposes of
tracking exactly which sources got built (we already have SRCREV) as
well as getting an indication when changes to the SRC_URI relate to
changes in the output.

(From OE-Core rev: 70714795aafc98a6df1df7f944867093eb8cafc7)

Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-20 11:11:46 +01:00
Martin Jansa
43f1cf2d3d image-artifact-names: introduce new bbclass and move some variables into it
* similar to kernel-artifact-names for other recipes/bbclasses which
  need to use some deployed artifacts

* bitbake.conf: move IMAGE_BASENAME, IMAGE_VERSION_SUFFIX, IMAGE_NAME,
  IMAGE_LINK_NAME variables

* image_types.bbclass: move IMAGE_NAME_SUFFIX variable

* currently IMAGE_NAME_SUFFIX is used only by image.bbclass,
  image_types.bbclass and meta/recipes-core/images/build-appliance-image_15.0.0.bb
  but if it's needed by some recipe which isn't itself an image, then
  it's useful in bitbake.conf, e.g. we have a recipe for creating
  VirtualBox appliances which combines .wic.vmdk with .ovf file to
  create .zip with appliance, but for that we need the filename of
  .wic.vmdk which now contains IMAGE_NAME_SUFFIX
  4980ce52a4/meta-luneos/recipes-core/images/luneos-emulator-appliance.inc (L24)

* we were hardcoding .rootfs suffix where needed, but for quite long
  time it's configurable with IMAGE_NAME_SUFFIX since:

  commit 380ee36811939d947024bf78de907e3c071b834f
  Author: Patrick Ohly <patrick.ohly@intel.com>
  Date:   Mon Mar 7 18:07:52 2016 +0100

    image creation: allow overriding .rootfs suffix

  and might not match with hardcoded .rootfs, so make it easier to
  use IMAGE_NAME_SUFFIX where needed even without inheritting whole
  image_types.bbclass

[YOCTO #12937]

(From OE-Core rev: 456b700d51a5052a285a8477304f902c335223be)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-17 09:47:45 +01:00
Mark Hatle
5cde141ef1 buildhistory.bbclass: Rework to use read_subpackage_metadata
Using this mechanism ensures that we have a single point to implement
the loading of the package and subpackage meta data.  This also then
allows the buildhistory class to use the regular datastore vs it's
own custom arrays for processing history items.

(From OE-Core rev: 155ae6a3fe84c25904ffe7058e125bb4a28e2b45)

Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-08-28 07:08:37 +01:00
Lee Chee Yang
4eef15704a buildhistory: use pid for temporary txt file name
buildhistory create .txt file bh_installed_pkgs.txt and
bh_installed_pkgs_deps.txt while listing down installed package.
These file is later removed in buildhistory_get_installed().

when multiple process runs in parellal there are chances to race for
the file where one process created the file while another task remove
the file right after it.

using different file name for each process should avoid the race. So
add PID to the file name to make it unique.

[yocto #13709]

(From OE-Core rev: 214d236d93700abffa2d586cc744a00455818fa4)

Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-25 15:11:05 +01:00
Jan Luebbe
e4faf3b3d0 classes/buildhistory: capture package config
As the PACKAGECONFIG variable has a large influence on the resulting
package sizes and dependencies, it's useful to capture it in the
recipe-level buildhistory. This makes it straightforward to analyze the
impact of PACKAGECONFIG changes on the resulting image size.

(From OE-Core rev: 189fae9f2df58759e9d66a3877c9ef30df4854bf)

Signed-off-by: Jan Luebbe <jlu@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-04 13:27:32 +01:00
Daniel McGregor
0a99a2e2da buildhistory: Allow customising buildhistory tags
Allow setting custom buildhistory tag prefixes. This allows multiple
build directories to share one buildhistory git repository with multiple
worktrees.

(From OE-Core rev: 1bb2aa53a74e36dc3ba901b9d8ce780e7880cef8)

Signed-off-by: Daniel McGregor <daniel.mcgregor@vecima.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-04 15:56:29 +00:00
Denys Dmytriyenko
d71fb563c1 buildhistory: fix "version went backwards" QA error message
Fix parentheses placement in the message from:
Package version for package X went backwards which would break package feeds from (Y to Z)
to this one:
Package version for package X went backwards which would break package feeds (from Y to Z)

(From OE-Core rev: 3a5a61fb6b3f811bd4f7232ba902afcfd2019154)

Signed-off-by: Denys Dmytriyenko <denys@ti.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-14 13:20:59 +00:00
Ross Burton
628c03af41 buildhistory: filter out the unexpected prefix for native/cross sysroots
For various technical reasons, native and cross builds have a prefix that
includes the full path to the sysroot.  As these are stripped away before the
files are used in the sysroot, we should also filter them out of the
buildhistory report.  This both removes noise when sharing a buildhistory
repository between different build directories, and improves the accuracy of the
reports.

(From OE-Core rev: 8bf53fbb62749b5d77c246fab6e1246b93f8c50f)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-10 09:56:22 +01:00
Ross Burton
f74e8081ad buildhistory: don't output ownership for the sysroot
As the sysroot isn't ran inside pseudo the ownership is whoever is running the
builds.  In a setup where multiple builders all contribute to a shared
buildhistory writing the ownership data isn't useful, so just replace it with "-
-".

(From OE-Core rev: fadb7ae78876a7cf25c48481ff4ed3131e53415f)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-10 09:56:22 +01:00
Ross Burton
d533ffb82c buildhistory: write the contents of the sysroot
Changes to the sysroot are just as interesting during development, so write the
file listing for the sysroot to buildhistory too.

(From OE-Core rev: b3ac82a27ab70ed6996fe3087a578ac637820329)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-28 13:28:37 +01:00
Martin Jansa
37e5436e1e buildhistory: show time spent writting buildhistory
* especially when pushing longer history to slow remote git server or when
  it timeouts during the push, it's useful to see where the time was actually
  spent

(From OE-Core rev: 96f1225d47985d94d9ed91eb5e7affdd70671c79)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-19 12:46:43 +01:00
André Draszik
181e3a8446 classes/buildhistory: display modified git branches
When collecting the git revision of repositories, also take
note of whether or not that repository has uncommited
changes.

This makes it a bit clearer what went on when looking at
diffs.

(From OE-Core rev: b9d780c7eeda0fefb13edde8bdba4f1d91e7823c)

Signed-off-by: André Draszik <andre.draszik@jci.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-14 17:38:24 +00:00
Jacob Kroon
18c1b63cb2 buildhistory: simplify buildhistory_list_files()
Avoid duplicating shell code for the two cases, fakeroot/non-fakeroot.

(From OE-Core rev: c4a931df28f45f95f19a13062b8dc38db60da342)

Signed-off-by: Jacob Kroon <jacob.kroon@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-11 10:39:09 +00:00
Jacob Kroon
2b9c40e56b classes/buildhistory: avoid git diff churn by sorting more dictionaries
Sort 'srcrevs' and 'tag_srcrevs' before iterating over them in order
to avoid unnecessary changes in the build history.

(From OE-Core rev: 031d708aedda11d65ba9746af4f01b91264a1f86)

Signed-off-by: Jacob Kroon <jacob.kroon@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-07 10:54:13 +00:00
Paul Eggleton
8274ccae0d classes/buildhistory: properly process escaped chars from pkgdata
All values written out to pkgdata are escaped (see write_if_exists() in
package.bbclass). In practice there tend not to be characters that need
escaping except in the scriptlets (pkg_preinst, pkg_postinst, pkg_prerm
and pkg_postrm) where currently we still see the escape codes in the
corresponding files within buildhistory (e.g. \n and \t) and thus also
in the output of buildhistory-diff, hindering proper diffing of changes.
To fix this, when we read values from pkgdata and write them out to
buildhistory, we need to interpret the escape codes by doing the exact
reverse of what we do in package.bbclass.

(From OE-Core rev: c258379181a438cb01728d223b3d05e0ab205941)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-18 10:18:41 +01:00
Paul Eggleton
2e8787593f classes/buildhistory: handle packaged files with names containing spaces
The FILELIST field of the package info file in the buildhistory
repository is a space-separated list of all of the files in the package.
If a name of a file packaged by a recipe contains a space character then
of course the result was that we didn't handle its name properly. To fix
that, use quotes around any filename containing spaces and at the other
end use these quotes to extract the proper entries.

Fixes [YOCTO #12742].

(From OE-Core rev: 801b705957dc683030d11393f43407d0b3506b6a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-18 10:18:41 +01:00
Paul Eggleton
397abe3acd buildhistory: enable committing history by default
The most common usage for buildhistory is with commits enabled so that
you actually collect history, rather than just keeping a snapshot of the
most recent build state, therefore default BUILDHISTORY_COMMIT to "1".
This really ought to have been the default in the beginning, I can't
really explain why it wasn't.

(From OE-Core rev: 8018a2349b7ad5ab27731c93a49603adf5f72fc2)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-11-05 22:33:23 +00:00
Ming Liu
307f25c23e meta: drop True option to getVar calls
Search made with the following regex: getVar ?\((.*), True\).

(From OE-Core rev: dbc0eaf478feb3f752ae22fd184984494fc85d0a)

Signed-off-by: Ming Liu <liu.ming50@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-26 11:05:01 +01:00