Commit Graph

6478 Commits

Author SHA1 Message Date
Konrad Weihmann
2eb0fb2fcf pypi.bbclass: mind package suffix on version check
Some pypi packages do have suffixes like dev, or a0 or b1.
When doing a version check on these, the version will get falsely
identified as major release versions.
Add a terminating slash to rule out those false positives

(From OE-Core rev: 13b145b9551884534f1dd2446eccfc55abc25f38)

Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
(cherry picked from commit 0603f6d9f2abfa67b99b1bc39228f6aa16a0370d)
[Yocto bug #13990]
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-08-12 10:53:51 +01:00
Ralph Siemsen
6d936d8d3d cve-check: include epoch in product version output
In the generated cve.log files, include the epoch in the product
version. This better matches how versions are displayed elsewhere,
in particular the bb.warn("Found unpatched CVE...") that appears
on the terminal when CVEs are found.

(From OE-Core rev: 99f6de1c74b581054c74c6b4598a5d47facc9964)

(From OE-Core rev: a8b69927485acbe504fcd75dc2cafc4c550cb66a)

Signed-off-by: Ralph Siemsen <ralph.siemsen@linaro.org>
(cherry picked from commit e1c3c0b6e5b01304e2127f5058986697e82adf93)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Adrian Bunk <bunk@stusta.de>
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-08-04 23:17:37 +01:00
Khem Raj
46809da0bb cve-check: Run it after do_fetch
Certain recipes e.g. bash readline ( from meta-gplv2 ) download patches instead of having them in
metadata, this could fail cve_check

ERROR: readline-5.2-r9 do_cve_check: File Not found: qemuarm/build/../downloads/readline52-001

This patch ensures that download is done before running CVE scan, even
though these will be external patches and may not contain CVE tags as it
expects, but it will fix the run failures as seen above

(From OE-Core rev: dbf143d79476e54e8da93101fc16eaedeec88362)

(From OE-Core rev: 4b65abc70e84187f666e2ea2ab4dcf6bf7216658)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
(cherry picked from commit e406fcb6c609a0d2456d7da0d2406d2d9fa52dd2)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Adrian Bunk <bunk@stusta.de>
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-08-04 23:17:37 +01:00
Peter Kjellerstedt
ab25229636 relocatable.bbclass: Avoid an exception if an empty pkgconfig dir exist
Rewrite relocatable_native_pcfiles() so that it can handle that any of
the checked pkgconfig directories are empty without causing an
exception.

(From OE-Core rev: d449e6b536c197b6723dab1d61e989706c14f19c)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit f9c5df6dc1c13e9b05ff1b47ad84ad339f6779a4)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-08 10:47:50 +01:00
Lili Li
f8fcc057d5 kernel.bbclass: Fix Module.symvers support
Starting from v5.8-rc1 commit 269a535ca931 (modpost: generate
vmlinux.symvers and reuse it for the second modpost"), kernel will
generate new vmlinux.symvers instead of dumping all the vmlinux symbols
into Module.symvers in the first pass.

Error log:
    'run.do_shared_workdir.16614' failed with exit code 1:
    DEBUG: cp: cannot stat 'Module.symvers': No such file or directory

This change will check the file Module.symvers existence before copying it.

(From OE-Core rev: 03248953d1eacede00971289dc9fff8d97d00464)

Signed-off-by: Lili Li <lili.li@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit cd2d62a08a1dfcd890a03ee55132b6d6c65f5ab7)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-08 10:47:50 +01:00
Richard Leitner
42fd2f8e4d kernel-fitimage: introduce FIT_SIGN_ALG
make fitImage configuration signature algorithm selectable with
FIT_SIGN_ALG.

(From OE-Core rev: e24b27a2b49e97cec6153f2d642d17a901b8ba12)

(From OE-Core rev: 12dceaba5a7e3afed2ccb43d0788e5b95c602951)

Signed-off-by: Richard Leitner <richard.leitner@skidata.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-08 10:47:50 +01:00
Lee Chee Yang
73738879a2 cve-check: CPE version '-' as all version
CPE version could be '-' to mean no version info.
Current cve_check treat it as not valid and does not report these
CVE but some of these could be a valid vulnerabilities.

Since non-valid CVE can be whitelisted, so treat '-' as all version
and report all these CVE to capture possible vulnerabilities.

Non-valid CVE to be whitelisted separately.

[YOCTO #13617]

(From OE-Core rev: 1e8e188514584f9f972c0eb733c10a61853ec3d0)

Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit c69ee3594079589d27c10db32bc288566ebde9ef)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-29 16:11:56 +01:00
Paul Barker
74f229160c kernelsrc.bbclass: Fix externalsrc support
When the externalsrc class is used the tasks listed in
SRCTREECOVEREDTASKS are deleted to prevent them being executed. If
externalsrc is used for the kernel then this will include
virtual/kernel:do_patch.

We can depend on do_shared_workdir instead as this will survive when
externalsrc is used.

(From OE-Core rev: 5ac98c776f9b77804a0ee812f47f3ecf593e432f)

Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 2c17d35cc7b9c5e01fd5829858d2f0234e7ac8d6)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-17 08:29:02 +01:00
Paul Barker
5b38b8af6a kernel-yocto.bbclass: Support config fragments with externalsrc
The merging of config fragments is performend in the do_kernel_configme
task and so config fragments will not be supported when this task is
removed from the dependency tree.

kernel-yocto adds additional tasks which may modify the source directory
to SRCTREECOVEREDTASKS so that they are removed when using externalsrc.
However, do_kernel_configme should be safe to use, the only modification
to the source tree is the potential creation of the '.kernel-meta'
directory and the '.metadir' file.

(From OE-Core rev: b7ae0fa668be008e46982190553d0738a5465efe)

Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 44f04c039a4d61dd18666e42b9b9865cbc3ada9e)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-17 08:29:02 +01:00
Ross Burton
4ccd0345c8 sanity: check for more bits of Python
MJ: icu in master doesn't need distutils anymore, because icu 65.1 currently in
dunfell/master doesn't depend on python3-distutils anymore since:
b4d41b0561

but the icu-64.2 in zeus and openembedded-core/meta/recipes-core/ovmf/ovmf_git.bb
still need python3-distutils as described in:
http://lists.openembedded.org/pipermail/openembedded-core/2020-March/293984.html

(From OE-Core rev: da2df0251098c46a1476983db379cf33010e3035)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-30 17:41:56 +01:00
Chee Yang Lee
c6f99b055e cve-check: fix ValueError
fix below error for whitelisted recipe and recipe skip cve check.

Error:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:do_cve_check(d)
     0003:
File: '/poky-master/meta/classes/cve-check.bbclass', lineno: 59, function: do_cve_check
     0055:        try:
     0056:            patched_cves = get_patches_cves(d)
     0057:        except FileNotFoundError:
     0058:            bb.fatal("Failure in searching patches")
 *** 0059:        whitelisted, patched, unpatched = check_cves(d, patched_cves)
     0060:        if patched or unpatched:
     0061:            cve_data = get_cve_info(d, patched + unpatched)
     0062:            cve_write_data(d, patched, unpatched, whitelisted, cve_data)
     0063:    else:
Exception: ValueError: not enough values to unpack (expected 3, got 2)

(From OE-Core rev: d8cfc309f9dd0dc8904ab18e5898770502ee2540)

Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 64a362bd2dd0b4f3165d5162adbc600826af66f8)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-16 16:44:54 +00:00
Chee Yang Lee
8b1eccc311 cve-check: show whitelisted status
change whitelisted CVE status from "Patched" to "Whitelisted".

[Yocto #13687]

(From OE-Core rev: 707b3a41b3cacfb7f1d1ed75f9a298ff4721735e)

Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 181bdd670492525f9488d52c3ebb9a1b142e35ea)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-16 16:44:54 +00:00
Armin Kuster
7d8d93cfb9 cve-check: fail gracefully when file not found
With out these changes, a traceback displayed when a file
is listed in the SRC_URI but the file does not exist.

raise FileNotFoundError and print the patch then mark the task as failed.

(From OE-Core rev: d313a5912d2ecbf7796e75ee7e7cd1d442b4fa40)

Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
(cherry picked from commit d4926c11a4ab9148bdb640a9367c9e1891491a5b)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-16 16:44:53 +00:00
Richard Purdie
5602cc200b patch: Extend to native/nativesdk and depend upon
There is a bug in patch 2.7.3 and earlier where index lines
in patches can change file modes when they shouldn't:
http://git.savannah.gnu.org/cgit/patch.git/patch/?id=82b800c9552a088a241457948219d25ce0a407a4

This leaks into debug sources in particular (e.g. tcp-wrappers where
source files are read-only). Add the dependency to target recipes
to avoid this problem until we can rely on 2.7.4 or later.

We could try and remove all index lines from patch files but it will be a
losing battle. We could try and identify all the recipes which change
modes on files in patches but again, its a losing battle.

Instead, compromise and have patch-native as a dependency
for target recipes. We use patch-replacement-native since patch-native
is in ASSUME_PROVIDED.

Also add nativesdk-patch to buildtools-tarball.

[YOCTO #13777]

(From OE-Core rev: 5ed0840c93804488cd1c1aba6cb382b2434714a5)

(From OE-Core rev: fd3bd61a6fe5190c575dc968f3a0be9c1cbf21ed)

(From OE-Core rev: 148f1f8caf5d9a262c1f55e437326ce6139a743e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-11 23:05:12 +00:00
Joshua Watt
632bbf4bd7 classes/reproducible_build: Read SDE file later
Defers the resolution of the SOURCE_DATE_EPOCH until the variable needs
to be actually realized with a value. The previous method of loading the
value in anonymous python had issues because it could occur before other
anonymous python functions that affect the location of the epoch file,
such as when a recipe uses AUTOINC/AUTOREV or allarch.bbclass.

Also adds more logging to help diagnose issues in the future.

[YOCTO #13763]

(From OE-Core rev: b3313a10a3eb93f0a3710a35de0404fb49cd6202)

(From OE-Core rev: 10515e5f7e38edbc4430e2599062a9ce6fdb42a8)

(From OE-Core rev: 81d3832728aeae0e02e775bab9fc13e159fb61d3)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-11 23:05:12 +00:00
Alex Kiernan
8fdf082c11 linuxloader: Correct loader for glibc on armhf
Fix the naming of the loader on armhf to match what glibc expects:

  #define GLIBC_DYNAMIC_LINKER_SOFT_FLOAT SYSTEMLIBS_DIR "ld-linux.so.3"
  #define GLIBC_DYNAMIC_LINKER_HARD_FLOAT SYSTEMLIBS_DIR "ld-linux-armhf.so.3"

(From OE-Core rev: 70efe880589df1dfe7d71451d62d9d8e4afd7981)

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 7ad531093d620f18c04d487d79e9dad9f5a96232)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-28 11:51:03 +00:00
Peter Kjellerstedt
375065fa17 license.bbclass: Introduce AVAILABLE_LICENSES that lists all licenses
Previously, there was SRC_DISTRIBUTE_LICENSES, an undocumented
variable that contained a static list of licenses. It was used by
expand_wildcard_licenses() to expand any wildcards used in, e.g.,
INCOMPATIBLE_LICENSE. However, since this static list of licenses has
not been kept up-to-date, many licenses were missing, with the result
that if one tried to use any of those licenses with a wildcard, no
licenses would be found, effectively ignoring that they should be
marked as incompatible.

This introduces a new (documented) variable, AVAILABLE_LICENSES, that
is automatically updated to contain all licenses found in any
directories specified by ${COMMON_LICENSE_DIR} and ${LICENSE_PATH},
and uses it instead of SRC_DISTRIBUTE_LICENSES when expanding
wildcards.

(From OE-Core rev: 5ed714139f91eb03871e01b68a4370784071234d)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 8c9ef587fe499c612a878a1ab42092eb79b334ef)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-28 11:51:03 +00:00
Peter Kjellerstedt
afc2ffee1a sanity.bbclass: Move sanity_info from conf to cache
Since this file is written during recipe parsing, having it in the
${BUILDDIR}/conf directory, which is covered by an inotify watcher,
will trigger a re-parse the next time bitbake is run and the resident
bitbake server is enabled. This causes the sanity_info file to be
updated again, which triggers a new parse the next time bitbake is run
ad infinitum. Moving it to ${BUILDDIR}/cache should avoid this.

(From OE-Core rev: f1a609e7176a37e7e5f86f0b786f7da915085f83)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit f98103b548aa7dba6b1be6c8e02ef41858a8e85c)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-28 11:51:03 +00:00
Peter Kjellerstedt
2071b7caf2 populate_sdk_ext.bbclass: No longer needed to clean away conf/sanity_info
Since the sanity_info file has moved from the conf directory to the
cache directory, there is no longer any need to clean it away
explicitly in clean_esdk_builddir() since the whole cache directory is
already cleaned away anyway.

(From OE-Core rev: f0ea8b49a183d9340aaf0c60ee959aa8ec3c041c)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 40c30990e1be72130819c040fe471e2bdc0c6e7d)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-28 11:51:03 +00:00
Peter Kjellerstedt
0cce5bc165 toaster.bbclass: Correct pkgdatadir path in toaster_package_dumpdata()
In commit 692b2046 (package: Fix race between do_package and
do_packagedata), the path used for do_packagedata[sstate-inputdirs]
was changed from "${PKGDESTWORK}" to "${WORKDIR}/pkgdata-pdata-input".
This commit adapts the path used for pkgdatadir in
toaster_package_dumpdata() accordingly to avoid setscene errors like:

  ERROR: libgcc-9.2.0-r0 do_packagedata_setscene: Error executing a
  python function in exec_python_func() autogenerated:

  The stack trace of python calls that resulted in this
  exception/failure was:
  File: 'exec_python_func() autogenerated', lineno: 2, function:
  <module>
       0001:
   *** 0002:toaster_package_dumpdata(d)
       0003:
  File: 'meta/classes/toaster.bbclass', lineno: 130, function:
  toaster_package_dumpdata
       0126:    lpkgdata = {}
       0127:    datadir = os.path.join(pkgdatadir, 'runtime')
       0128:
       0129:    # scan and send data for each generated package
   *** 0130:    for datafile in os.listdir(datadir):
       0131:        if not datafile.endswith('.packaged'):
       0132:            lpkgdata = _toaster_load_pkgdatafile(datadir,
       0133:            # Fire an event containing the pkg data
       0134:            bb.event.fire(bb.event.MetadataEvent(
  Exception: FileNotFoundError: [Errno 2] No such file or directory:
  'tmp/work/mips32r2el-nf-poky-linux/libgcc/9.2.0-r0/pkgdata/runtime'

(From OE-Core rev: c1d881448fea0bc48aba0bc0e5b8748442572226)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 5a0f6f631b86f7107aa72453b6d23f32ba39f713)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-28 11:51:03 +00:00
Vasyl Gomonovych
9151c84245 kernel: Make symbol link to vmlinux.64 in boot directory
Some mips 64 bit platforms use vmlinux.64 image name
Make a symbol link to vmlinux.64 in arch/mips/boot/

(From OE-Core rev: f9739e0f58f5d8a5dc01fdb0efaa778af4edd671)

Signed-off-by: Vasyl Gomonovych <gomonovych@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 70626b52e5e61c80018e9f1f85bc169d0434b986)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-11 11:06:22 +00:00
Ross Burton
5a3341083f podfix: only alter normal files
(From OE-Core rev: dcbaf2a7a444f07ab516833f4ed4a2322828bddd)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 23d38b2ad6f7a39e5c1ffd092322942474935c33)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-11 11:06:22 +00:00
Ross Burton
0d4ae87e43 reproducible_build_simple: inherit podfix
When doing reproducible builds inherit podfix to remove the possibility
of differing Perl versions causing manpages to change.

(From OE-Core rev: 3534f17752eccdd62eac8809d7842e9d25fa770a)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 5dc43edab4787906f2c9f51e51bf78d1dc966286)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-11 11:06:22 +00:00
Ross Burton
9a294101b6 podfix: class to remove Pod::Man versions from manpages
Manpages generated by Pod::Man contain the version number, which isn't
reproducible if we're using the host Perl to generate manpage.

One option is to always depend on perl-native when generating manpages
but this is a heavy dependency, so instead strip out the versions in
do_install().

(From OE-Core rev: 88255abe6bb4d10d50a660022ab3f9a1c2954ec7)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 18d8e5ac689d6eb6098f68ac785f43e9d5f5938a)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-11 11:06:22 +00:00
Martin Jansa
53b981ae2a sstate.bbclass: Fix for consistent whitespace
(From OE-Core rev: a15c05fa17bcce345ce254f02c0d6dcd97f60cb8)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit e874d067ace03774132ba3030a898678057c1304)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-11 11:06:21 +00:00
Richard Purdie
be02afefc5 sstate: Ensure mkdir happens before mktemp
This avoids a directory not present error.

Fix a comment typo whilst here.

(From OE-Core rev: c5f285f83ea1e322d5124847904adf488d3e790a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 1360d8d7d99b70a80c8cdbc1fc6d9e6752483139)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-11 11:06:21 +00:00
Richard Purdie
17e3fcab7a sstate: Ensure target sstate directory exists if unihash changes
The previous patches meant the mkdir might no longer match the final target
directory. Fix this.

(From OE-Core rev: 7ce42a30a7508e8fcb496ba05cf6967dc04a988e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 0af4dae84099e8632a9ea6a4afdbea2f232bb170)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-11 11:06:21 +00:00
Richard Purdie
14823a0bda sstate: Ensure SSTATE_PKG is reloaded when handling siginfo
STATE_PKG may have been changed by sstate_report_unihash so don't
cache the variable's value.

(From OE-Core rev: bfed8be91525478ecdf6b64e6308958b3271c8cc)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit be29a25400c4ea285ab3f588c5831f00ba5d4f63)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-11 11:06:21 +00:00
Richard Purdie
5947cc196f sstate: Stop overwriting SSTATE_PKG
Its rather antisocial to overwrite SSTATE_PKG with an expanded form for the variable
and it stops the value of BB_UNIHASH being changed when the package is written out.

Instead of expanding the variable, append to it instead to avoid this rather
hard to figure out behaviour and allow the siggen code to behave as expected.

(From OE-Core rev: fe454e9c7837a903ffcff4c6f8a25070a0f0af82)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 62eca02024b4c44d618ab9bcf87a3166c886dadb)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-11 11:06:21 +00:00
Richard Purdie
8d72948bf4 sstate: Add ability to hide summary output for sstate
Its confusing to keep seeing sstate summary messages when hash equivalency is
active. This adds an option to control it. A default value is given which
maintains compatibility with different bitbake versions.

(From OE-Core rev: a67bd96de2b253c1d4c3bd82120f28dee06a4bf6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 038004866ff6650bcff7bb1bde36de6c0f451d29)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-11 11:06:21 +00:00
Richard Purdie
ccff952d9c qemu-helper-native/systemtap-native: Ensure sysroots are populated with dependencies
As Alex Kanavin found, dependencies aren't always populated, particularly
with the hash equivalence server enabled locally:

'bitbake core-image-minimal' with gtk+ disabled.
can confirm with: $ tmp/work/x86_64-linux/qemu-helper-native/1.0-r1/recipe-sysroot-native/usr/bin/qemu-system-x86_64 -display gtk
qemu-system-x86_64: Display 'gtk' is not available.
Enable gtk in local.conf with: PACKAGECONFIG_append_pn-qemu-system-native = " gtk+"
'bitbake core-image-minimal', without deleting tmp/
$ tmp/work/x86_64-linux/qemu-helper-native/1.0-r1/recipe-sysroot-native/usr/bin/qemu-system-x86_64 -display gtk
qemu-system-x86_64: Display 'gtk' is not available.

This change ensures the dependencies are correctly handled as the full
sysroot is always depended upon even if things come from sstate.

(From OE-Core rev: d40853b10dd9f01d6a8dd4edcb941cfa8a544922)

(From OE-Core rev: 309424458c73b14a4a4020cd508c2bf37bf2bcbe)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-31 10:37:23 +00:00
Niko Mauno
4065420e5b cve-check: Switch to NVD CVE JSON feed version 1.1
Switch to recently released version 1.1 of NVD CVE JSON feed, as in
https://nvd.nist.gov/General/News/JSON-1-1-Vulnerability-Feed-Release
it is mentioned that

  Due to changes required to support CVSS v3.1 scoring, the JSON
  vulnerability feeds must be modified. This will require the consumers
  of this data to update their internal processes. We will be providing
  the JSON 1.1 schema on the data feeds page and the information below
  to prepare for this transition.
  ...
  The JSON 1.1 data feeds will be available on September 9th, 2019. At
  that time the current JSON 1.0 data feeds will no longer available.

This change was tested briefly by issuing 'bitbake core-image-minimal'
with 'cve-check.bbclass' inherited via local.conf, and then comparing
the content between the resulting two
'DEPLOY_DIR_IMAGE/core-image-minimal-qemux86.cve' files, which did not
seem to contain any other change, except total of 167 entries like

  CVSS v3 BASE SCORE: 0.0

were replaced with similar 'CVSS v3 BASE SCORE:' entries which had
scores that were greater than '0.0' (up to '9.8').

(From OE-Core rev: cc20e4d8ff2f3aa52a2658404af9a0ff358cc323)

(From OE-Core rev: c92b8804d6e59b2707332859957f0e6a46db0a73)

Signed-off-by: Niko Mauno <niko.mauno@iki.fi>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-16 23:08:52 +00:00
Peter Kjellerstedt
cd46d6826d meson.bbclass: Prevent meson from downloading wrapped projects
Meson has support for downloading subprojects using something called
wraps. This interferes with bitbake's expectations of all downloads
being done by the fetch task. To avoid this, tell meson to not
download any wraps.

Suggested-by: Mattias Jernberg <mattias.jernberg@axis.com>
(From OE-Core rev: b547637ad84bad8f7fe27193bf636541f8588ae8)

(From OE-Core rev: 4170718196ca734e5dd7635fc98b55ea47a74e88)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-16 23:08:52 +00:00
Mattias Hansson
a2ab2dcf12 base.bbclass: add dependency on pseudo from do_prepare_recipe_sysroot
do_prepare_recipe_sysroot may perform groupadd, which requires pseudo.
However, do_prepare_recipe_sysroot does not depend on pseudo explicitly,
which sometimes causes a build error when building a recipe that adds
groups.

This issue only occurs when executing do_prepare_recipe_sysroot for a
recipe that adds groups before finishing a task that depends on pseudo
for a recipe that doesn't add groups.

(From OE-Core rev: e20a67fe9cc7de042f39f3c39b74de61dc5c6dfe)

Signed-off-by: Mattias Hansson <mattihn@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-16 23:08:51 +00:00
Ross Burton
c1cbb6fd15 cve-check: fetch CVE data once at a time instead of in a single call
This code used to construct a single SQL statement that fetched the NVD data for
every CVE requested.  For recipes such as the kernel where there are over 2000
CVEs to report this can hit the variable count limit and the query fails with
"sqlite3.OperationalError: too many SQL variables".  The default limit is 999
variables, but some distributions such as Debian set the default to 250000.

As the NVD table has an index on the ID column, whilst requesting the data
CVE-by-CVE is five times slower when working with 2000 CVEs the absolute time
different is insignificant: 0.05s verses 0.01s on my machine.

(From OE-Core rev: 53d0cc1e9b7190fa66d7ff1c59518f91b0128d99)

(From OE-Core rev: 3ded9a64c95ae02df7562fc69e2af08c150d2452)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-25 21:37:40 +00:00
Ross Burton
1f4750c47f cve-check: neaten get_cve_info
Remove obsolete Python 2 code, and use convenience methods for neatness.

(From OE-Core rev: f19253cc9e70c974a8e21a142086c13d7cde04ff)

(From OE-Core rev: 98162c04c877925c737674a1635b08cf998b92f5)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-25 21:37:40 +00:00
Ross Burton
728f969be0 cve-check: rewrite look to fix false negatives
A previous optimisation was premature and resulted in false-negatives in the report.

Rewrite the checking algorithm to first get the list of potential CVEs by
vendor:product, then iterate through every matching CPE for that CVE to
determine if the bounds match or not.  By doing this in two stages we can know
if we've checked every CPE, instead of accidentally breaking out of the scan too
early.

(From OE-Core rev: d61aff9e22704ad69df1f7ab0f8784f4e7cc0c69)

(From OE-Core rev: 0f42a1d4dbb74ab39e81449cf222302bcc04f7db)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-25 21:37:40 +00:00
Ross Burton
9971e87cad cve-check: we don't actually need to unpack to check
The patch scanner works with patch files in the layer, not in the workdir, so it
doesn't need to unpack.

(From OE-Core rev: 2cba6ada970deb5156e1ba0182f4f372851e3c17)

(From OE-Core rev: 02e6b727bf62858be7dba061879a6d57bd5a725d)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-25 21:37:40 +00:00
Richard Purdie
aa72758866 sanity: Add check for tar older than 1.28
Older versions break opkg-build when reproducible builds are enabled.
Rather than trying to be selective based on which features are enabled,
lets just make this a minimum version.

(From OE-Core rev: 96f5c7c2f8dda7d47af5398b3463aa25921f5301)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-25 21:34:50 +00:00
Jacob Kroon
4ad64a5cf5 rm_work: Promote do_image_qa stamps to setscene versions
[ YOCTO #13212 ]

Suggested-by: Romuald Jeanne <romuald.jeanne@st.com>
(From OE-Core rev: 624897ee788fe026a04951baaf770daca43e892c)

Signed-off-by: Jacob Kroon <jacob.kroon@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-13 22:02:16 +00:00
Ross Burton
a8580a49b0 cve-check: failure to parse versions should be more visible
(From OE-Core rev: 6b5cadd1a5822641285946f7e2ad56e294658621)

Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-13 22:02:16 +00:00
Ross Burton
faf0ebf337 cve-check: ensure all known CVEs are in the report
CVEs that are whitelisted or were not vulnerable when there are version
comparisons were not included in the report, so alter the logic to ensure that
all relevant CVEs are in the report for completeness.

(From OE-Core rev: 29d926802e7f8b4614a2dafa0af4c923912e1811)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-13 22:02:16 +00:00
Douglas Royds via Openembedded-core
bfd2ab3d67 icecc: Export ICECC_CC and friends via wrapper-script
By exporting ICECC_CC, ICECC_CXX, and ICECC_VERSION in a wrapper-script,
and putting this wrapper-script in the PATH, the Makefiles generated by CMake or
the autotools are able to function correctly outside of bitbake.
This provides a convenient developer workflow in which the
modify-compile-unittest cycle can happen directly in the ${B} directory.

The `rm -f $ICE_PATH/$compiler` line is transitional,
and can go at some later date (October 2020 or later, perhaps).

(From OE-Core rev: 0f47d4ef50d6b0732ac40f7ede31f554a757864c)

Signed-off-by: Douglas Royds <douglas.royds@taitradio.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
(cherry picked from commit 40d74cb1d0ddce930267e49764cacb263b244091)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-29 09:08:18 +00:00
Stefan Agner
ad9606f877 uninative: check .done file instead of tarball
In case multiple builds share UNINATIVE_DLDIR's location, one build
might be in the process of downloading the tarball while another is
just checking whether the tarball exists. Check for the done file
instead and rely on the fetchers lockfile mechanism in case two
builds are running.

(From OE-Core rev: 956ae2c2a72914c7e9a1d59e5906c7fc75670a39)

Signed-off-by: Stefan Agner <stefan.agner@toradex.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit a1c95580549cb4f77601e62c7f026b19c752d853)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-29 09:08:17 +00:00
Ross Burton
c07d45d20c sanity: check the format of SDK_VENDOR
If SDK_VENDOR isn't formatted as -foosdk and is instead for example -foo-sdk
then the triple that are constructed are not in fact triples, which results in
mysterious compile errors.

Check in sanity.bbclass so this failure is detected early.

[ YOCTO #13573 ]

(From OE-Core rev: 95d3ca54453cdb0662fae2a2cf7e8173611c86f4)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit b0efd8d4d0dbc30e6505b42f5603f18fa764d732)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-29 09:08:17 +00:00
Trevor Gamblin
e9ab95da30 systemd.bbclass: add RMINITDIR for nativesdk builds
nativesdk builds such as nativesdk-dnf are installing systemd
service files, leading bitbake to throw installed-vs-shipped
warnings, but these are not needed in nativesdk:

 WARNING: nativesdk-dnf-4.2.2-r0 do_package: QA Issue: nativesdk-dnf: Files/directories were installed but not shipped in any package:
      /usr/local/oecore-x86_64/sysroots/x86_64-oesdk-linux/lib
      /usr/local/oecore-x86_64/sysroots/x86_64-oesdk-linux/lib/systemd
      /usr/local/oecore-x86_64/sysroots/x86_64-oesdk-linux/lib/systemd/system
      /usr/local/oecore-x86_64/sysroots/x86_64-oesdk-linux/lib/systemd/system/dnf-automatic.timer
      /usr/local/oecore-x86_64/sysroots/x86_64-oesdk-linux/lib/systemd/system/dnf-makecache.timer
      /usr/local/oecore-x86_64/sysroots/x86_64-oesdk-linux/lib/systemd/system/dnf-automatic.service
      /usr/local/oecore-x86_64/sysroots/x86_64-oesdk-linux/lib/systemd/system/dnf-makecache.service
      /usr/local/oecore-x86_64/sysroots/x86_64-oesdk-linux/lib/systemd/system/dnf-automatic-notifyonly.timer
      /usr/local/oecore-x86_64/sysroots/x86_64-oesdk-linux/lib/systemd/system/dnf-automatic-download.service
      /usr/local/oecore-x86_64/sysroots/x86_64-oesdk-linux/lib/systemd/system/dnf-automatic-download.timer
      /usr/local/oecore-x86_64/sysroots/x86_64-oesdk-linux/lib/systemd/system/dnf-automatic-notifyonly.service
      /usr/local/oecore-x86_64/sysroots/x86_64-oesdk-linux/lib/systemd/system/dnf-automatic-install.timer
      /usr/local/oecore-x86_64/sysroots/x86_64-oesdk-linux/lib/systemd/system/dnf-automatic-install.service
    Please set FILES such that these items are packaged. Alternatively if they are unneeded, avoid installing them or delete them within do_install.
    nativesdk-dnf: 13 installed and not shipped files. [installed-vs-shipped]

Rather than have each recipe remove the files, an RMINITDIR case
for nativesdk builds should be added to systemd.bbclass.

(From OE-Core rev: 86092fdac2ed2935b25a6c05f16055df77a10575)

Signed-off-by: Trevor Gamblin <trevor.gamblin@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-09 14:07:43 +01:00
Ross Burton
5d8648b058 toolchain-scripts: export READELF
The readelf binary is prefixed, so export READELF in the SDKs so that the
correct readelf binary can be used.

This fixes problems with the symbol extractor in Meson with multilib SDKs where
readelf can't be found correctly.

(From OE-Core rev: 2795e3bfdefc9f48294802ea7301dae1910553dc)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-09 14:07:43 +01:00
Ross Burton
a5a5708dbb insane: add check for perllocal.pod
perlocal.pod is an index file of locally installed modules and so shouldn't be
installed by any distribution packages.  cpan.bbclass already sets NO_PERLOCAL
to stop this file being generated by most Perl recipes, but if a recipe is using
MakeMaker directly (such as rrdtool) then they might not be doing this
correctly.

To avoid multiple packages shipping this file and then failing to install
together, add a QA test to check if this file exists and by default emit an
error if it does.

[ YOCTO #13491 ]

(From OE-Core rev: 5d16d265cc61c4e279fe3bf66016a00d9daa4068)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-08 11:22:24 +01:00
Martin Jansa
ec6c982949 kernel-devicetree.bbclass: add missing backslash
* in oe-core commit 1860d9d3c62e2e94cd68a809385873ffd8270b6d I've accidentally
  removed the backshash here

Reported-By: "Hilsdorf, Jan (LAWO)" <Jan.Hilsdorf@lawo.com>
(From OE-Core rev: 44f0047c0268f3e9f9fdf78f93bdad9bfe491174)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-08 11:22:24 +01:00
Alessio Igor Bogani
a558e507ae rootfs-postcommands: Avoid use of an hard-coded value
(From OE-Core rev: 520c6f30cd571166a0d890d86b461f24f7a6998f)

Signed-off-by: Alessio Igor Bogani <alessio.bogani@elettra.eu>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-02 20:22:25 +01:00