Bumping the SRCREV to integrat the following kern-tools change:
commit 2d01f24bc78256c709728eb3f204491bce13e0e5
Author: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Date: Fri Nov 4 23:32:38 2022 +0000
kconf_check: store some files in tmpdir
Some file systems, like ZFS, are very slow at appending to existing
files. Due to Copy-On-Write nature, they create a new copy of a file
each time we do ">>" in a shell script. This becomes very noticeable
if shell script does lots and lots of appends, like sanitize_fragment()
function in kconf_check. On my setup, do_kernel_configcheck task takes
literally hours to complete.
To fix this issue, we can store sanitized_list and fragment_errors.txt
files on tmpfs, which is extremely fast at writing. As most distros
use tmpfs for /tmp, logical step is to use `mktemp` to create
temporary files.
After completing writing to temporary locations, we can move those two
files back to ${LOGDIR}.
Also, function 'cleanup' was added to remove temporary files in case
of abnormal exit.
With this patch, do_kernel_configcheck task completes in ~2 minutes on
my setup, which is a great improvement.
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
(From OE-Core rev: a592314318509e6367a7625caf88837480570fcd)
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 9d50e2606eb66019044ee176f355a84a65a1499c)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The previous fix for processing of paths with relative components, broke
uses cases that were a mix of patches and configuration fragments.
Updating the SRCREV to include a simplied fix for relative paths, and
a cleanup patch from Jose:
[
Author: Jose Quaresma <quaresma.jose@gmail.com>
Date: Thu Sep 29 16:37:23 2022 +0000
scc: only look for error in scc_output_file if it has valid content
When process_file function fails the output of the processed script is show to
the user, some parsing is performed as well to look for common errors so we
can point to the right input file.
This can only be done when the scc_output_file have some valid content
otherwise it will show invalid messages to the user.
Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Author: Bruce Ashfield <bruce.ashfield@gmail.com>
Date: Wed Oct 5 19:13:33 2022 +0000
spp: ensure that prefix check uses absolute paths
The previous fix for this issue was too broad, and impacted
all calls to the prefix check and removal. With this change,
we only expand the input on scc/spp operations that may
execute with relative paths.
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
]
(From OE-Core rev: 110bcaf1629ee89d7117ea1d4f860fe9f61ce225)
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 533720a1756454447341769c4a0969fce8d6f287)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
commit b627b8bd17501400649d38bf61c8a3d2121a1b8f (HEAD -> master,
origin/master, origin/HEAD)
Author: Bruce Ashfield <bruce.ashfield@gmail.com>
Date: Sun Sep 18 16:36:43 2022 -0400
symbol_why: allow options to be flagged as built-in or module
Some options that are hardware related, can either be built in or
modules, and meet the criteria of being acceptable for a BSP.
To allow these options to not throw warnings, we create a new
configuration fragment to go along with the 'hardware.cfg' and
'nonhardware.cfg' special fragments.
This new fragment is called 'y_or_m_enabled.cfg' and is effective
for the fragments in its directory. The format is the same as
hardware/nonhardware.cfg, is simply a list of CONFIG_ options.
(From OE-Core rev: 35c2debe35103f92993d9d7bb164b7bd64629c67)
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 9cd0623f89d8ff5ea95d6ab713dd4bf45742a8b1)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As reported by Peter Hoyes, in some very specific configurations
the processing done by the kern-tools results in errors. This is
due to the bad creation of relative paths to patches and configuration.
This is fixed by the following kern-tools commit (based on a similar
suggestion by Peter):
spp: ensure that relocations and patch paths are absolute
If include directories (locations of layers) have relative
components, and the path to a patch doesn't share that relative
location, the detection of a common prefix can fail.
When the common prefix is incorrectly specified, it can be the
entire path to a patch. This results in everything being removed
and subtle errors (as this shouldn't happen).
By ensuring that both the relocation directories and patch path
are absolute, we can avoid this issue.
(From OE-Core rev: 8f76f9f04bf50d9c80ed23bd8297c5c9236b90bb)
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit b9681ad7c5c72aa1e68b3cdc93788a03b3781d63)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In situations where a buid needs to be reproducible, it is
sometimes desireable to use a patches author date, versus the
time when it is applied. This generates a consistent hash
between different patch applications.
We leverage the existing KERNEL_DEBUG_TIMESTAMPS to trigger
the use of a new option to kgit-s2q. This allows us to use
the author date in a reproducible configuration, but disable
it if we need the current time/date.
(From OE-Core rev: 8dbce9b13960acfd6889567c11456e3c73a783b5)
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit ebe2411ce78d5e4ea49b9b4cb732b461ecc32ca6)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
add the Kconfiglib license, as this was missing in before.
Add MIT identifier to LICENSE
(From OE-Core rev: 8e337904b1f612dc461e64a57bcd7c2dff2a309b)
Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
An automated conversion using scripts/contrib/convert-spdx-licenses.py to
convert to use the standard SPDX license identifiers. Two recipes in meta-selftest
were not converted as they're that way specifically for testing. A change in
linux-firmware was also skipped and may need a more manual tweak.
(From OE-Core rev: ceda3238cdbf1beb216ae9ddb242470d5dfc25e0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is uncertainty about the default branch name in git going forward.
To try and cover the different possible outcomes, add branch names to all
git:// and gitsm:// SRC_URI entries.
This update was made with the script added to contrib in this patch which
aims to help others convert other layers.
(From OE-Core rev: b51c405faf6f8c0365f7533bfaf470d79152a463)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bumping the SRCREV to pickup the following for the kern-tools:
commit 6dd85ff178cd76851e2184b13e545f5a88d1be30 [kconfig: change
"modules" from sub-option to first-level attribute] broke parsing
in 5.13+ trees.
We add the new location to the parser so we can support both
types of module specifications.
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Without this, the kernel configuration audit for 5.13+ errors
immediately.
The older "option modules" parsing has been left in the code, so that
older kernels continue to work as well.
(From OE-Core rev: ba39e42681e7e3c6fe82686a42a9ee4b519003f9)
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The rewrite of the configuration audit code dropped the ability to
generate warnings for configuration options that didn't make it into
the final .config.
We integrated the following commit to restore those warnings:
symbol_why: classify based on config.queue hints
The config.queue has typing hints inline with each fragment,
we should be using them to further classify the options, and
not only relying on the special hardware.cfg, etc, files that
are part of the meta data
We also should be checking for options that were set to a
non 'no' value, and that don't make it into the final .config,
since without that check it means we are missing some warnings.
(From OE-Core rev: f5e8a8c52386317607e333e55f710bf0393186c8)
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There was a bad indent in symbol_why, which we fix with a submitted
patch. As part of getting that patch, it was pointed out that there's
no README telling people where to send changes. So we add a basic
README to avoid that in the future:
8f6aaab docs: add README for patch submission
9f1a6cb symbol_why: fix incorrect indentation of sys.exit(1)
(From OE-Core rev: eea536b4368abed2248d3d0c5727a680b7368ce7)
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Integrating the following commits:
1aa9046 merge_config.sh: Translate some env variables to make variables
6fdcd64 symbol_why: allow re-classification from non-hardare to hardware
With these, a non-gcc compiler can be used, since the invocation of the
kernel configuration explicitly passes the variables as command line
vars, versus environment variables. This means the kernel Makefile
assignments are overriden and the desired compiler used.
With option re-classification, we can inhibit warnings when an option
has incorrectly been classified has 'hardware', and hence triggers a
visible warning.
[merge_config.sh: Translate some env variables to make variables]
(From OE-Core rev: b5d33d13cd8618c10ae0a0379a1c8f22f6941eeb)
Signed-off-by: Tyler Hicks <tyhicks@linux.microsoft.com>
[symbol_why: allow re-classification from non-hardare to hardware]
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Integrating the following commit:
symbol_why: tighten 'is not set' matching
Commented lines that contained CONFIG_FOO ... were being picked up
as option lines and reported as invalid.
We make the regex more explicit and only match on spaces before
CONFIG_ to declare if a 'is not set' line is invalid.
(From OE-Core rev: 63c8681e6bf763800e797f6d37f2f133abb7a0e8)
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Enable the kernel-yocto bbclass to use enhanced capabilities from
the kern-tools symbol_why.pl.
We bump the kern-tools SRCREV to pickup the reworking of symbol_why,
which uses Kconfiglib to provide analysis on configuration values.
This is useful for debugging why a symbol specified in a fragment
did not end up in the final .config.
We introduce two ways to interact with the new symbol_why:
1) a replacement of the existing kconf_check script
2) a dedicated task that is explicitly invoked to dump details
on the configuration.
The kconf_check replacement is transparent to the user, and is
run in exactly the same way as it was previously. But we get better
output and more detailed diagnostics if there are symbols that
don't make it into the final .config
The second way to interact with symbol why is via the new task
do_config_analysis. This is invoked like any other task, and by
default will provide a full configuration analysis and point the
user at files to look at for details.
If a more targetted analysis is desired, then specific symbols
can be set in the CONFIG_ANALYSIS variable. When this variable
is set, the task will only run for the given symbols and provide
per-variable links to the user. This variable can be set like
any other, including specification in the local.conf:
CONFIG_ANALYSIS_pn-linux-yocto-dev = 'NF_CONNTRACK LOCALVERSION'
Which produces output as follows:
WARNING: linux-yocto-dev-5.8-rc++gitAUTOINC+d22beb8f8a_8fc484ed37-r0
do_config_analysis: Configuration analysis executed, see: tmp/work/qemuarm64-poky-linux/linux-yocto-dev/5.8-rc++gitAUTOINC+d22beb8f8a_8fc484ed37-r0/NF_CONNTRACK-config-analysis.txt for details
WARNING: linux-yocto-dev-5.8-rc++gitAUTOINC+d22beb8f8a_8fc484ed37-r0
do_config_analysis: Configuration audit executed, see: tmp/work/qemuarm64-poky-linux/linux-yocto-dev/5.8-rc++gitAUTOINC+d22beb8f8a_8fc484ed37-r0/NF_CONNTRACK-config-audit.txt for details
WARNING: linux-yocto-dev-5.8-rc++gitAUTOINC+d22beb8f8a_8fc484ed37-r0
do_config_analysis: Configuration analysis executed, see: tmp/work/qemuarm64-poky-linux/linux-yocto-dev/5.8-rc++gitAUTOINC+d22beb8f8a_8fc484ed37-r0/LOCALVERSION-config-analysis.txt for details
WARNING: linux-yocto-dev-5.8-rc++gitAUTOINC+d22beb8f8a_8fc484ed37-r0
do_config_analysis: Configuration audit executed, see: work/qemuarm64-poky-linux/linux-yocto-dev/5.8-rc++gitAUTOINC+d22beb8f8a_8fc484ed37-r0/LOCALVERSION-config-audit.txt for details
(From OE-Core rev: cbc896def4c8bab3150d3405969e5dd018d62d0c)
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To ensure that the kernel linker is used when allno/mod/yes config
merge_config steps were executed, the call to make was tweaked to
explicitly pass LD.
But since the variable wasn't quoted, any parameters to LD (like
the sysroot) were mistakenly passed to make, and hence could trigger
an error on some architectures.
We also tweak the logging to hightlight errors like this in the
future and avoid losing it in the noise of merge configs sometimes
overly verbose output.
(From OE-Core rev: a60c4c116efecd7a6ee5a11b1d366bb00b9d23ce)
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bumping the SRCREV to pickup the following fix:
Author: Martin Jansa <Martin.Jansa@gmail.com>
Date: Wed Feb 5 03:26:57 2020 +0100
merge_config.sh: pass LD variable from shell environment to make
* since 5.4 kernel Kconfig will fail immediately when it detects
that LD points to gold linker:
scripts/Kconfig.include:39: gold linker 'i686-oe-linux-ld' not supported
* in OE we already pass bfd linker in KERNEL_LD variable to merge_config.sh
but we need to pass it also into the make call here
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
(From OE-Core rev: fa964575f9e00d8530563e61075992c5b69df137)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Updating the SRCREV to pickup tweaks to symbol_why.py to be
python3 safe:
- we explicitly call /usr/bin/env python3
- we full specifiy our symbols
- do not assume that 'None' can be converted to a string
(From OE-Core rev: eaa83453fe206567253257fcefdbf6feb6d53d72)
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When a symbol doesn't make it into the final analysis is run
using Kconfiglib to show dependencies. v5.4 has introduced Kconfig
elements that Kconfiglib can't parse (hence we get no analysis).
Updating the Kconfiglib snapshot solves our problem.
(From OE-Core rev: 240e0ae13d35469eecafc82d2cd9cfd110095c55)
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As reported in https://bugzilla.yoctoproject.org/show_bug.cgi?id=12563,
the regex that matches valid CONFIG_ options was missing some of the
ones in net/netfilter/ipvs/Kconfig, and hence triggering invalid
option warnings.
By dropping the trailing space on the regex, we'll cover all the cases
for valid option.
(From OE-Core rev: 461a2f54751ca18e17f897b10d6c3c47dab1733a)
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Integrating the following kern tools patch:
The cmd line, <grep '^[ ]*\(menu\)*config '>, can't
match all expect config options.
This is because that it is not always a single space after 'config'
in kernel-source/*/.../Kconfig. e.g. "config IP_VS_IPV6" in
net/netfilter/ipvs/Kconfig
So we should change the cmd to grep '^[ ]*\(menu\)*config\s'.
Signed-off-by: Hongzhi.Song <hongzhi.song@windriver.com>
(From OE-Core rev: b476d1cddefb9468c41a4c1c08b4fae66d03b52f)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Recent kernels broke the ability of kconfiglib to parse the Kconfig
files and offer reasons why a symbol may not be set. To address this
issue, we update to Kconfiglib2 and adjust the symbol_why script to
work with the new API.
We also tweak the kconf_check script to allow the specification of
a list of option as "non-hardware". This allows a BSP to inhibit
warnings on options that it knows are mismatched for a valid reason
(i.e. -tiny kernels using common fragments with known missing
dependencies).
(From OE-Core rev: 6acfe7d9e431923124c5c4e743f39f9d7aea97c5)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Replace usage of os.utils.getstatusoutput() with direct subprocess
calls. Pass a modified environment and working directory where necessary
to bypass the need to execute in a shell.
(From OE-Core rev: 21de5cc43cfedc703e5bc0515507a6dae36afb74)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Integrating the following kern-tools commit:
tools/merge_config.sh: add CR after each fragment
If a fragment file doesn't contain a CR at the end, two config
options may be merged on the same line in the result file, leading to
misconfiguration.
This patch adds a CR after each fragment to ensure that config
options are well separated in the result file.
Bug-AGL: SPEC-1475
Signed-off-by: Stephane Desneux <stephane.desneux@iot.bzh>
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
(From OE-Core rev: c36727f2cad6c2d51aff8da6e2acd5642afe9848)
Signed-off-by: Stephane Desneux <stephane.desneux@iot.bzh>
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It was reported that BSPs that only matched the machine were
being returned as the configuration entry point. This could lead
to warnings, or unexpected runtime results.
Integrating the following commit to ensure that only strict matches
are returned by default, with a flag to do fuzzy matching
spp: make fuzzy matching optional
Add a flag that can be used to toggle wether or not a partial
match is an error.
--fuzz
When passed, partial patching will be used. If not passed the
default is to return nothing (which can be interpreted as an
error by the calling routines) if both the kernel type and
machine do not match.
(From OE-Core rev: f60d050fef2e4ac592bb5554e74b9573e3570d0f)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There was a bug in the search routines responsible for locating
BSP definitions which returned a valid match if only the ktype
matched.
This meant that someone looking for "qemux86foo" (which is an
invalid definition) would potentially end up building "qemuarm"
and be none the wiser (until it didn't boot).
With this fix to the tools search routine, and improved return
code testing, we will now stop the build and report and error to
the user.
[YOCTO: #11878]
(From OE-Core rev: 44aea7b87307795fe4e089c51d45afccaa2f6525)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is possible to inherit meta data for either patches + config
or for just config.
It is possible that the patch queue contains invalid (when
sourced) shell characters in the patch names, which throws a
syntax error and aborts processing.
The patch + config case was fixed some time ago, but we recently
stumbled onto the config-only case which was still not properly
quoted and hence safe.
This commit brings the config-only inherit in line with the
patch + config processing and we won't abort processing if
characters like () are in patch names.
(From OE-Core rev: ce7044be10597d53725e6917a2949f3ab3de0c96)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It was reported that do_validate_branches was failing with the following
error:
Log data follows:
| DEBUG: Executing shell function do_validate_branches
| HEAD is now at fe0fb8d Merge tag 'v4.10.9' into standard/base
| mkdir: cannot create directory .: File exists
|
| [ERROR] Can't find patch dir at ./patches/standard/base
| usage: kgit s2q
| WARNING: exit code 1 from a shell command.
| ERROR: Function failed: do_validate_branches
This was triggered by the execution of 'kgit-s2q --clean' after forcing
the SRCREV to something other than the tip of the branch. --clean is
being run to remove any sentinel files from previous kernel builds to
ensure that the tree is in a consistent state.
There were two bugs, --clean was being executed and not exiting the
script as it was supposed to. Hence validation for applying patches
was done, and threw the error that eventually makes it to the console.
And the second bug is that since do_validate_branches actually calls
kgit-s2q --clean, the dependency on kern-tools-native needs to be on
that function (versus do_kernel_metadata which runs later).
With the tweaked kern-tool + the dependency fix, we no longer see this
error.
(From OE-Core rev: 4d5890b54cbdac01ee748759578b7b22ed8e61a2)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In some cases it is seen that kernel_checkout and validate_branches are
run again in simultaneous builds. During do_patch the kgit-s2q mechanism
looks for a sentinel file inside the .git directory, finds a fence post
and starts picking up patches after that.
This can create trouble as validate_branches checks out the HEAD of the
branch and so the patches should be reapplied rather than skipped due to
finding of the fence post.
We can call kgit-s2q --clean to remove the sentinel file when the
branches are checked out.
(From OE-Core rev: d57aeafdd2b49010a9fa6d1cd9d10f3cfd5754a5)
Signed-off-by: Awais Belal <awais_belal@mentor.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The ability to merge two branches directly from a .scc file was
dropped during the streamlining of the tools.
As was pointed out by David Vincent <freesilicon@gmail.com>, there is
once again a valid use case for this functionality, so we restore the
capability.
(From OE-Core rev: a0059ebbb52c659282e355664bba1a2fa282170e)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Options with spacs around = signs will be droped/ignored by the
kernel.
The audit phase can detect this and warn the user:
% kconf_check --report -o .kernel-meta/cfg/ \
linux-qemux86-standard-build/.config `pwd` $cfgs
[errors (3): .kernel-meta/cfg/fragment_errors.txt
There are errors withing the config fragments.
% cat .kernel-meta/cfg/fragment_errors.txt
Warning: Ignoring "CONFIG_PARAVIRTT_CLOCK =y" -- spaces around equals are invalid
Warning: Ignoring "CONFIG_PARAVIRTTT_CLOCK = y" -- spaces around equals are invalid
Warning: Ignoring "CONFIG_PARAVIRTTTT_CLOCK= y" -- spaces around equals are invalid
(From OE-Core rev: 952a455c09eb88755b2d24ae05c2d6b886922b63)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Commit 626ceac135fa66277c2fa53197be33cc9d4d7614 broke the error
check in process_file by adding in three lines that stomp on $? which
print the output file when verbose is set.
Move output file on verbose print to an elif after the error check.
Signed-off-by: George McCollister <george.mccollister@gmail.com>
(From OE-Core rev: 01657e8023ee535420ab5ba4a5d1952e13bce0cd)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need to avoid using shared/common directories for any files that are
part of specific build, since permissions issues in multi user
environments will cause issues.
Integrating the following commit to solve the issue:
scc: move unused patch queue under output dir
(From OE-Core rev: cad65cc0eef2e06cb5ae08062ffae7a4d43a51ad)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Lernel meta-data that has patches, but no branches, can trigger an
error due to no branch specific patch queue.
This error then cascades to more issues since the tools are using
a named file in /tmp to store and display error messages to the
user.
We fix both issues though the following kern tools tweaks:
commit bd9e1d6c9b0a34ff3e19a06999aaf57ffadfd04c
Author: Bruce Ashfield <bruce.ashfield@windriver.com>
Date: Fri Dec 2 13:09:40 2016 -0500
scc: use mktemp for consolidated output capture
To provide useful error messages the tools dump pre-processed
files and messages to a temporary file. If multiple users are
doing builds, this means they either race, or can have permissions
issues.
By creating the temporary file via mktemp, we avoid both issues.
(We also make sure to clean these up on exit, or /tmp will get
polluted quickly).
commit a287da4bfe0b4acb8f2b0627bd8e7abd1a1dde26
Author: Bruce Ashfield <bruce.ashfield@windriver.com>
Date: Fri Dec 2 13:08:08 2016 -0500
patch: do not assume a branch specific patch queue is needed
When processing input files per-branch and global patch queues are
generated. If the meta-data has not created any branches in the
repo, no branch specific queue is required.
The tools assumed that one is always valid, and hence would throw a
non-zero exit code and stop processing.
By testing for a named per-branch queue, we avoid this issue.
(From OE-Core rev: 0fd7da7375f0dcc59b56791fd482de557507c04c)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
During processing of the kernel meta data the kern tools were
not properly exiting on syntax errors or invalid commands.
Noticing and debugging these issues wasn't trivial. To make this
easier, we now trap the error and dump the offending meta-data
for the user to see.
There was also an issue with creating branches during tree
generation, which is resolved by always switching to the
active branch.
The following are the commit logs of the changes themselves:
[
commit b36f6f9a5695f2084b83823393e13ca42284bed9
Author: Paul Gortmaker <paul.gortmaker@windriver.com>
Date: Sat Oct 22 17:23:25 2016 -0400
kgit-scc: dont mention meta-repo in help ; it doesnt exist
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
commit 08463d684c1952e74c25344cddace4c3f24c739d
Author: Bruce Ashfield <bruce.ashfield@windriver.com>
Date: Mon Oct 31 14:30:12 2016 -0400
scc: exit on error
If there is an error in the processing of the input files, scc
should exit and inform the user.
scc is executed on a combined/preprocessed file and as a result
it doesn't have the granularity to see each input file individually.
Rather than moving preprocessing into scc (from spp), we can trap
the line number of the error and dump context around the line.
This gives the user a pointer to the input file and the specific
line that caused the problem.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
commit bf99953e8ac14cee653e559f2d4a6022c847a182
Author: Bruce Ashfield <bruce.ashfield@windriver.com>
Date: Fri Oct 28 21:23:27 2016 -0400
kgit-meta: always checkout branches on branch commands
During a tree generation we must always make the branch active when
we see any kind of branch command. This ensures that any subsequent
patches are applied in the proper context.
Previously, only branch creation was changing the active branch, and
this mean that tree generation was not determinstic and relied
on the order of processing to generate a correct tree.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
]
(From OE-Core rev: 83d10e2acef936b1f38804988f10eafa48db36f9)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a patch contains characters like ; or (), it can cause processing
errors in the patch queue.
We had previously fixed this issue by renaming the patches to not
include invalid characters, but with this change to the kern tools
that ensures patch names are wrapped in quotes, we avoid shell
processing and hence they are valid.
(From OE-Core rev: 727afe9e6fb9ef4a24a36cc907ebb6336d12184e)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We've been running with a set of kern-tools that were designed to work
with build systems that knew nothing about git, trees, commits, etc.
As such, there's been a set of shims/wrappers in place to work with
within bitbake/oe-core. These were the *me scripts: createme, updateme,
patchme and configme.
With this commit, we strip that legacy code and use the tools directly.
This means less complexity, fewer corner cases .. and no surprises
when the tools are arunning. As another benefit, the tools consume
much less time during a typical build and have no noticeable impact
on the overall build time.
Existing .scc files, features, and processing are not impacted as
these tools are compatible with existing feature descriptions and
kerne configuration fragments.
The audit of kernel configuration fragments is now detached
from the linux-yocto build structure and process. This means that
they can eventually be tweaked to offer kernel audit to any type of
kernel build and configuration process.
Additionally, the kernel symbol audit phase can now resolve symbol
dependencies and offer guidance when a symbol is missing:
WARNING: linux-yocto-4.4.15+gitAUTOINC+b030d96c7b_f5e2c49d58-r0 do_kernel_configcheck: [kernel config]: specified values did not make it into the kernel's final configuration:
---------- CONFIG_BT_6LOWPAN -----------------
Config: CONFIG_BT_6LOWPAN
From: /home/bruce/poky/build/tmp/work-shared/qemux86-64/kernel-source/.kernel-meta/configs/standard/features/bluetooth/bluetooth.cfg
Requested value: CONFIG_BT_6LOWPAN=y
Actual value:
Config 'BT_6LOWPAN' has the following conditionals:
BT_LE && 6LOWPAN (value: "n")
Dependency values are:
BT_LE [y] 6LOWPAN [n]
(From OE-Core rev: 0f698dfd1c8bbc0d53ae7977e26685a7a3df52a3)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Robert P. J. Day reported that configuration fragments and kernel
features were not being found when organized in a particular manner:
linux
- $BOARD
- mm.patch
- mm.scc
- ssd_sil.cfg
- ssd_sil.patch
- ssd_sil.scc
- uio.cfg
.. etc
There was a bug in the tools that did not handle the mix of subdirs
properly and ended up leaving a trailing / on the elements *not* in
the $BOARD subdir. As a result, the configuration fragments were not
properly found when searching the include paths, and a configuration
failure was triggered (due to missing files).
This change tweaks the tools to always check a path with and without
a trailing / when processing config fragments so they can be later
found when processing the configuration of the kernel.
Reported-by: "Robert P. J. Day" <rpjday@crashcourse.ca>
(From OE-Core rev: 92ba77bea59a33b0ddbd5db36e2a1b42e8fd7190)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the recent changes to improve patch processing times, the ability
to skip already applied patches is not active by default.
The automatic detection and resume was hiding issues with the include
files generated by scripts like yocto-bsp.
If a .scc file that contains a patch is included twice, the patch is
applied twice, and the second appliation fails for obvious reasons.
We can partially fix this by ensuring that already included
configuration fragments are not forced into the meta-series.
.scc files that are explicitly listed twice will continue to fail, and
recipes must be modified to avoid this.
[YOCTO: #8486]
(From OE-Core rev: ed2da98bf3ac798009f58a53b91285b4dac69d5a)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Updating the kern-tools SRCREV to import the following fix:
kgit-meta: resume after last applied patch
When the auto-resume (resume point detection) was removed from the
processing of a meta-series, it ignored the fact that a single patch
series may in fact be processed a number of times.
Two layers patching a kernel will generate two different runs on the
same branch, which always start at patch one. This will obviously
break with duplicate patches.
To avoid this, we simply track the last patch applied, and
explicitly
tell the patch scripts where to start. This gets us resume
functionality, without the overhead of resume point detection.
(From OE-Core rev: 692f1333e257556e7462b2436dd60e865869349c)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Updating the kern-tools SRCREV to integrat the following commit:
patching: only validate user supplied patches by default
Previously the patching tools would consider both system and user
supplied patches in the same manner .. they are simply a series of
patches to be applied to a branch, and that the scripts should determine
where in the series to start (based on what is already on the
branch).
This detection was causing a few problems:
- time consuming
- starting in the middle of a series when intermediate patches
were merged to a branch.
To solve both the performance and start detection, we instead simply
note the transition from system (i.e. already defined features and
series) and user/recipe supplied patches. When the transition is noted,
the system will start pushing ALL patches without doing autoresume
detection.
Control in keeping the series up to date is passed to the user, and
consistent behaviour/performance is achieved.
(From OE-Core rev: 440ad49e53359ea800c179df105ab885873d7691)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Updating the kern-tools SRCREV to import the following changes:
cbd4b7102668 patchme/updateme: unify meta directory handling
b65075997152 configme: standalone operation
The change of note is [configme: standalone operation], which makes the
kernel configuration script free from dependencies on other parts of the
kern-tools.
With this change, we set the stage to extend kernel configuration
fragments and auditing to arbitrary trees.
(From OE-Core rev: 17f071dea78a08648eda71829c845104338510b9)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is possible that recipe specific tasks, or build processes drop
files into the kernel source directory. These files can cause problems
with the meta data detection in the kern-tools.
With this change, we have a single unified meta data detection routine,
that logs the result in a new file ".metadir", which subsequent scripts
can find, and use, thereby avoid repeating the same check many times.
We also enhance the check to look for a sentinel file in a proper meta
directory, to avoid false positives when an unexpected kernel process
leaves an uncommitted directory in the kernel dir.
[YOCTO: #7441]
(From OE-Core rev: 6b04ae2c0439b83c0445fd1b8cb9cba5cee6b9bc)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When fixing a kernel configuration warning, it is often necessary to
modify the kernel's meta-data and re-run the tools to update and
re-audit the config. This implies that the patch, config and audit
steps are run multiple times.
The tools had a bug that would incorrectly restore old meta-data
versus using updated configuration. Updating the kern-tools SRCREV
to fix the issue.
(From OE-Core rev: b903559daa847d2c56bf729fc5ca885113d0eecc)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update the SRCREV for the following incremental improvement in patch
processing time:
kgit-meta: skip patches on non-leaf nodes
In a similar way as commit 0768d697 [kgit-meta: dont run kgit-s2q
for
non-leaf nodes], we can save even more processing time by not even
analysing and linking patches if we aren't on the leaf node of the
tree.
This early exit can save nearly 95% of the time required to "patch"
a tree when no changes are actually applied.
(From OE-Core rev: 148c78e0f5de2689de3ad9beaa9f6de618d87758)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
