Commit Graph

93 Commits

Author SHA1 Message Date
Richard Purdie
2252b53ac6 cve_check: Use a local copy of the database during builds
Rtaher than trying to use a sqlite database over NFS from DL_DIR, work from
a local copy in STAGING DIR after fetching.

(From OE-Core rev: 9b6363994e5715f1d08b98956befd8915c128e85)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 03596904392d257572a905a182b92c780d636744)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-11-11 06:19:18 -08:00
Peter Marko
b87290f76d cve-check: add support for cvss v4.0
https://nvd.nist.gov/general/news/cvss-v4-0-official-support

CVSS v4.0 was released in November 2023
NVD announced support for it in June 2024

Current stats are:
* cvss v4 provided, but also v3, so cve-check showed a value
sqlite> select count(*) from nvd where scorev4 != 0.0 and scorev3 != 0.0;
2069
* only cvss v4 provided, so cve-check did not show any
sqlite> select count(*) from nvd where scorev4 != 0.0 and scorev3 = 0.0;
260

(From OE-Core rev: 358dbfcd80ae1fa414d294c865dd293670c287f0)

(From OE-Core rev: 8c20a7badb6e5d6c6c90176e45e90f776df25298)

Signed-off-by: Peter Marko <peter.marko@siemens.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-11-02 06:32:36 -07:00
Antoine Lubineau
24effee3d5 cve-check: add CVSS vector string to CVE database and reports
This allows building detailed vulnerability analysis tools without
relying on external resources.

(From OE-Core rev: 048ff0ad927f4d37cc5547ebeba9e0c221687ea6)

(From OE-Core rev: 3e47644d24d97c2541ccb70d91c144cf6530d5b0)

Signed-off-by: Antoine Lubineau <antoine.lubineau@easymile.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Peter Marko <peter.marko@siemens.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2024-11-02 06:32:36 -07:00
Ross Burton
ae28221a40 cve-check: sort the package list in the JSON report
The JSON report generated by the cve-check class is basically a huge
list of packages.  This list of packages is, however, unsorted.

To make things easier for people comparing the JSON, or more
specifically for git when archiving the JSON over time in a git
repository, we can sort the list by package name.

(From OE-Core rev: 1245649fd2725915154648a98584c908da07af18)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit e9861be0e5020830c2ecc24fd091f4f5b05da036)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-11-14 06:49:11 -10:00
Marta Rybczynska
075d712b28 cve-update-nvd2-native: new CVE database fetcher
Add new fetcher for the NVD database using the 2.0 API [1].
The implementation changes as little as possible, keeping the current
database format (but using a different database file for the transition
period), with a notable exception of not using the META table.

Minor changes that could be visible:
- the database starts in 1999 instead of 2002
- the complete fetch is longer (30 minutes typically)

[1] https://nvd.nist.gov/developers/vulnerabilities

(From OE-Core rev: b8ae26aabe825c1bc8d84fd7e0ee197695f1dee7)

Signed-off-by: Marta Rybczynska <marta.rybczynska@syslinbit.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit fb62c4c3dbca4e58f7ce6cf29d4b630a06411a97)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-06-21 04:00:58 -10:00
Geoffrey GIRY
8064466b45 cve-check: Fix false negative version issue
NVD DB store version and update in the same value, separated by '_'.
The proposed patch check if the version from NVD DB contains a "_",
ie 9.2.0_p1 is convert to 9.2.0p1 before version comparison.

[YOCTO #14127]

Reviewed-by: Yoann CONGAL <yoann.congal@smile.fr>
(From OE-Core rev: f331c80df6c447d3073ebe3f00102c78ced242f3)

Signed-off-by: Geoffrey GIRY <geoffrey.giry@smile.fr>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 7d00f6ec578084a0a0e5caf36241d53036d996c4)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-04-11 11:31:52 +01:00
Jermain Horsman
b899f258be cve-check: write the cve manifest to IMGDEPLOYDIR
When building an image cve_check_write_rootfs_manifest() would sometimes fail
with a FileNotFoundError when writing the manifest.cve due to the parent
directory (DEPLOY_DIR_IMAGE) not (yet) existing.

The image task will provide the manifest in the deploy directory afterwards,
so other recipes depending on the manifest being in DEPLOY_DIR_IMAGE should
continue to function properly.

(From OE-Core rev: 3e1ae171d89d0ac5487a8930bf6f8a97c43fdf57)

Signed-off-by: Jermain Horsman <jermain.horsman@nedap.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 00fb2aae22ce0d7ff5f3f8766fa770eeb4e73483)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-01-26 23:37:05 +00:00
Ross Burton
4384b8a13a cve-check: close cursors as soon as possible
We can have multiple processes reading the database at the same time, and
cursors only release their locks when they're garbage collected.

This might be the cause of random sqlite errors on the autobuilder, so
explicitly close the cursors when we're done with them.

(From OE-Core rev: 90917cadeb7201e56c74294e9156fe899d5455d7)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
(cherry picked from commit 5d2e90e4a58217a943ec21140bc2ecdd4357a98a)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-09-12 08:41:47 +01:00
Joshua Watt
8856232de4 classes: cve-check: Get shared database lock
The CVE check database needs to have a shared lock acquired on it before
it is accessed. This to prevent cve-update-db-native from deleting the
database file out from underneath it.

[YOCTO #14899]

(From OE-Core rev: 655bc5acdebdee9b4dfd3c964104a84b6845b2d4)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 20a9911b73df62a0d0d1884e57085f13ac5016dd)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-09-12 08:41:47 +01:00
Ross Burton
611d7ed2ac cve-check: hook cleanup to the BuildCompleted event, not CookerExit
The cve-check class writes temporary files to preserve state across the
build, and cleans them up in a CookerExit handler.

However, in memory-resident builds the cooker won't exit in between
builds, so the state isn't cleared and the CVE report generation fails:

NOTE: Generating JSON CVE summary
ERROR: Error adding the same package twice

Easily solved by hooking to BuildCompleted, instead of CookerExit.

(From OE-Core rev: 6f9c806dd1580a80856a687a2ad3a0b32d74dd17)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit fccdcfd301de281a427bfee48d8ff47fa07b7259)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-08 08:27:16 +01:00
Marta Rybczynska
c086c4b35a cve-check: add support for Ignored CVEs
Ignored CVEs aren't patched, but do not apply in our configuration
for some reason. Up till now they were only partially supported
and reported as "Patched".

This patch adds separate reporting of Ignored CVEs. The variable
CVE_CHECK_REPORT_PATCHED now manages reporting of both patched
and ignored CVEs.

(From OE-Core rev: d76957e4257a3bf934bdb94d65aa43267b8d9766)

Signed-off-by: Marta Rybczynska <marta.rybczynska@huawei.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit c773102d4828fc4ddd1024f6115d577e23f1afe4)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-24 23:57:42 +01:00
Marta Rybczynska
50ee11d88f cve-check: fix return type in check_cves
Make empty return types in check_cvs the same for all code paths.

(From OE-Core rev: 2fc716ad75dcc59000dcc18143f06bcdf43be242)

Signed-off-by: Marta Rybczynska <marta.rybczynska@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit f86393c93dec47b24e837d0c4c5761a716ecdbb6)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-11 10:06:13 +01:00
Marta Rybczynska
34eecaa053 cve-check: write empty fragment files in the text mode
In the cve-check text mode output, we didn't write fragment
files if there are no CVEs (if CVE_CHECK_REPORT_PATCHED is 1),
or no unpached CVEs otherwise.

However, in a system after multiple builds,
cve_check_write_rootfs_manifest might find older files and use
them as current, what leads to incorrect reporting.

Fix it by always writing a fragment file, even if empty.

(From OE-Core rev: 79c02facd3f248122b4b7a6bd00192151e4a6406)

Signed-off-by: Marta Rybczynska <marta.rybczynska@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit f1b7877acd0f6e3626faa57d9f89809cfcdfd0f1)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-11 10:06:13 +01:00
Marta Rybczynska
7f22fb48eb cve-check: move update_symlinks to a library
Move the function to a library, it could be useful in other places.

(From OE-Core rev: 66bf08f2df478a909a12756c650dcc2ae09a0a47)

Signed-off-by: Marta Rybczynska <marta.rybczynska@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit debd37abcdde8788761ebdb4a05bc61f7394cbb8)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-11 10:06:13 +01:00
Richard Purdie
cd12e92aab cve-check: Allow warnings to be disabled
When running CVE checks in CI we're usually not interested in warnings on the
console for any CVEs present. Add a configuration option CVE_CHECK_SHOW_WARNINGS
to allow this to be disabled (it is left enabled by default).

(From OE-Core rev: 8fd6a9f521ea6b1e10c80fe33968943db30991ba)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 1054d3366ba528f2ad52585cf951e508958c5c68)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-07 11:53:26 +01:00
Ernst Sjöstrand
e72a868593 cve-check: Only include installed packages for rootfs manifest
Before this the rootfs manifest and the summary were identical.
We should separate the summary and rootfs manifest more clearly,
now the summary is for all CVEs and the rootfs manifest is only for
things in that image. This is even more useful if you build multiple
images.

(From OE-Core rev: 65498411d73e8008d5550c2d0a1148f990717587)

Signed-off-by: Ernst Sjöstrand <ernstp@gmail.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 3b8cc6fc45f0ea5677729ee2b1819bdc7a441ab1)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-07 11:53:26 +01:00
Ernst Sjöstrand
ecac8cb4f7 cve-check: Add helper for symlink handling
(From OE-Core rev: af227e0c624c74ba90509829000cca607583e929)

Signed-off-by: Ernst Sjöstrand <ernstp@gmail.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 5046d54df2c3057be2afa4143a2833183fca0d67)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-07 11:53:26 +01:00
leimaohui
edf2b57a6b cve-check.bbclass: Added do_populate_sdk[recrdeptask].
As product, sdk should do cve check as well as rootfs.

(From OE-Core rev: 637d6ff44bc17565edae5466addbbe251f0cdadc)

Signed-off-by: Lei Maohui <leimaohui@fujitsu.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
(cherry picked from commit cc17753935c5f9e08aaa6c5886f059303147c07b)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-07 11:53:26 +01:00
Marta Rybczynska
31909cc347 cve-check: Fix report generation
The addition of summary output caused two issues: error when building
an image and the fact that JSON output was generated even when
CVE_CHECK_FORMAT_JSON.

When generating an image it caused an error like:
ERROR: core-image-minimal-1.0-r0 do_rootfs: Error executing a python function in exec_func_python() autogenerated:

  The stack trace of python calls that resulted in this exception/failure was:
  File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
       0001:
   *** 0002:cve_check_write_rootfs_manifest(d)
       0003:
  File: '/home/alexk/poky/meta/classes/cve-check.bbclass', lineno: 213, function: cve_check_write_rootfs_manifest
       0209:
       0210:        link_path = os.path.join(deploy_dir, "%s.json" % link_name)
       0211:        manifest_path = d.getVar("CVE_CHECK_MANIFEST_JSON")
       0212:        bb.note("Generating JSON CVE manifest")
   *** 0213:        generate_json_report(json_summary_name, json_summary_link_name)
       0214:        bb.plain("Image CVE JSON report stored in: %s" % link_path)
       0215:}
       0216:
       0217:ROOTFS_POSTPROCESS_COMMAND:prepend = "${@'cve_check_write_rootfs_manifest; ' if d.getVar('CVE_CHECK_CREATE_MANIFEST') == '1' else ''}"
  Exception: NameError: name 'json_summary_name' is not defined

The fix is to pass the d variable to the pure python function generate_json_report
to get correct values of variables and add conditions for the JSON
output where needed.

In addition clarify the message presenting the summary JSON file,
which isn't related to an image.

Uses partial fixes from Alex Kiernan, Ernst Sjöstrand (ernstp),
and Davide Gardenal.

Fixes: f2987891d315 ("cve-check: add JSON format to summary output")

(From OE-Core rev: 2fcc696e27d7f8c70ba60f5c7de8c48030a938d3)

Signed-off-by: Marta Rybczynska <marta.rybczynska@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 9015dec93233c7d45fd0c9885ff5d4ec23ad377d)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-25 22:45:50 +01:00
Davide Gardenal
dd08692cac cve-check: fix symlinks where link and output path are equal
An if statement now checks if the link and output path are
the same, if they are then the link is not created,
otherwise it is.

(From OE-Core rev: 2fd7f3b7dc964b59b268dd4a34761f9f71f61c25)

Signed-off-by: Davide Gardenal <davide.gardenal@huawei.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
(cherry picked from commit 2f024c0236c4806f0e59e4ce51a42f6b80fdf1b3)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-12 16:44:06 +01:00
Davide Gardenal
645c157bef cve-check: add JSON format to summary output
Create generate_json_report including all the code used to generate the JSON
manifest file.
Add to cve_save_summary_handler the ability to create the summary in JSON format.

(From OE-Core rev: 8a79c476706b25e5c707c65b4e46b6e940874bd6)

Signed-off-by: Davide Gardenal <davide.gardenal@huawei.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
(cherry picked from commit f2987891d315466b7ef180ecce81d15320ce8487)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-12 16:44:06 +01:00
Ross Burton
e163bed574 cve-check: no need to depend on the fetch task
The only part of the cve-check task which needs files is the patch
examination, and typically these patches are local so fetch isn't needed.

(From OE-Core rev: a76b642736d78cd4dec0ae264da6d0ffd4e9aaf7)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 2c9b3186d3b7c18cbea239ab9b06e85b7c243b54)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-12 16:44:06 +01:00
Marta Rybczynska
b3f96e6fea cve-check: add coverage statistics on recipes with/without CVEs
Until now the CVE checker was giving information about CVEs found for
a product (or more products) contained in a recipe. However, there was
no easy way to find out which products or recipes have no CVEs. Having
no reported CVEs might mean there are simply none, but can also mean
a product name (CPE) mismatch.

This patch adds CVE_CHECK_COVERAGE option enabling a new type of
statistics. Then we use the new JSON format to report the information.
The legacy text mode report does not contain it.

This option is expected to help with an identification of recipes with
mismatched CPEs, issues in the database and more.

This work is based on [1], but adding the JSON format makes it easier
to implement, without additional result files.

[1] https://lists.openembedded.org/g/openembedded-core/message/159873

(From OE-Core rev: d1849a1facd64fa0bcf8336a0ed5fbf71b2e3cb5)

Signed-off-by: Marta Rybczynska <marta.rybczynska@huawei.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-30 13:07:41 +01:00
Marta Rybczynska
777f1d42b6 cve-check: add json format
Add an option to output the CVE check in a JSON-based format.
This format is easier to parse in software than the original
text-based one and allows post-processing by other tools.

Output formats are now handed by CVE_CHECK_FORMAT_TEXT and
CVE_CHECK_FORMAT_JSON. Both of them are enabled by default.

The JSON output format gets generated in a similar way to the
text format with the exception of the manifest: appending to
JSON arrays requires parsing the file. Because of that we
first write JSON fragments and then assemble them in one pass
at the end.

(From OE-Core rev: df567de36ae5964bee433ebb97e8bf702034994a)

Signed-off-by: Marta Rybczynska <marta.rybczynska@huawei.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-30 13:07:41 +01:00
Saul Wold
d9e500f83d meta/scripts: Improve internal variable naming
Update internal variable names to improve the terms used.

(From OE-Core rev: f408068e5d7998ae165f3002e51bc54b380b8099)

Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-10 08:00:28 +00:00
Ross Burton
3f37a01275 Revert "cve-check: add lockfile to task"
Now that all of the functions in cve-check open the database read-only,
we can remove this lockfile.

This means cve-check can run in parallal again, improving runtimes
massively.

This reverts commit d55fbf4779.

(From OE-Core rev: d3d3e7b324698ec3e6fce1951aba389805f3edd3)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-27 12:34:40 +00:00
Ross Burton
440f07d211 cve-check: get_cve_info should open the database read-only
All of the function in cve-check should open the database read-only, as
the only writer is the fetch task in cve-update-db.  However,
get_cve_info() was failing to do this, which might be causing locking
issues with sqlite.

(From OE-Core rev: 8de517238f1f418d9af1ce312d99de04ce2e26fc)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-25 12:41:23 +00:00
Richard Purdie
71ef319193 meta/scripts: Automated conversion of OE renamed variables
(From OE-Core rev: aa52af4518604b5bf13f3c5e885113bf868d6c81)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-21 23:37:27 +00:00
Stefan Herbrechtsmeier
272212c2cb cve-check: create directory of CVE_CHECK_MANIFEST before copy
Create directory of the CVE_CHECK_MANIFEST variable before copy to it,
so that the variable can use an arbitrary directory name.

(From OE-Core rev: 9829c16301bf2dce39fa046401a984f112fa0322)

Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-07 10:08:59 +00:00
Konrad Weihmann
d55fbf4779 cve-check: add lockfile to task
this should prevent running into the very rare error
sqlite3.OperationalError: attempt to write a readonly database

As highlighted by https://www.sqlite.org/faq.html#q5
it is likely that the adapter won't allow use multiple exec calls
at the same time.

So it's best to prevent multiple accesses at a time, by reusing
the already in place CVE_CHECK_DB_FILE_LOCK

YOCTO #14110

(From OE-Core rev: 677f5741bd265be49d4a5bb933b3e8d8c4eec653)

Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-01-08 14:17:35 +00:00
Joshua Watt
fa6c07bc1a classes/cve-check: Move get_patches_cves to library
Moving the function will allow other classes to capture which CVEs have
been patched, in particular SBoM generation.

Also add a function to capture the CPE ID from the CVE Product and
Version

(From OE-Core rev: 75d34259a715120be1d023e4fd7b6b4b125f2443)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-13 14:44:06 +01:00
Michael Opdenacker
a99473976d cve-check: remove deprecated CVE_CHECK_CVE_WHITELIST
This variable has been deprecated since Yocto Project version 3.0.

(From OE-Core rev: f8ac58568b2dceef54a743369460019b3a3eeccd)

Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-12 06:26:15 +01:00
Michael Opdenacker
ae55dea68d cve-check: improve comment about CVE patch file names
(From OE-Core rev: 8aa613480663e11ecc62278d8c57ca719eb23899)

Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-06 06:34:58 +01:00
Michael Opdenacker
4c28645c04 cve-check: update link to NVD website for CVE details
The old URL schema
https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-ID
now redirects to
https://nvd.nist.gov/vuln/detail/CVE-ID

(From OE-Core rev: 57adb57a9d9b08c08ab606ec7b561792e4f4ff2d)

Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-06 06:34:58 +01:00
Michael Opdenacker
fce14b90e3 cve-check: fix comments
This implements various fixes in comments in cve-check.bbclass
In particular, the "whitlisted" typo is important as the "whitelisted"
word is going to be replaced in a near future.

(From OE-Core rev: 5eecd2bf942254d08c252388594e5ec7ae330f45)

Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-06 06:34:58 +01:00
Richard Purdie
bb6ddc3691 Convert to new override syntax
This is the result of automated script conversion:

scripts/contrib/convert-overrides.py <oe-core directory>

converting the metadata to use ":" as the override character instead of "_".

(From OE-Core rev: 42344347be29f0997cc2f7636d9603b1fe1875ae)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-02 15:44:10 +01:00
akuster
73befa8f41 cve-check: add include/exclude layers
There are times when exluding or including a layer
may be desired. This provide the framwork for that via
two variables. The default is all layers in bblayers.

CVE_CHECK_LAYER_INCLUDELIST
CVE_CHECK_LAYER_EXCLUDELIST

(From OE-Core rev: 5fdde65ef58b4c1048839e4f9462b34bab36fc22)

Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-09 08:56:10 +00:00
akuster
29e280f7ee cve-check.bbclass: add layer to cve log
Lets include whcih layer a package belongs to and
add it to the cve logs

(From OE-Core rev: 00d965bb42dc427749a4c3985af56ceffff80457)

Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-09 08:56:10 +00:00
Lee Chee Yang
86b42289bd cve_check: add CVE_VERSION_SUFFIX to indicate suffix in versioning
add CVE_VERSION_SUFFIX to indicate the version suffix type, currently
works in two value, "alphabetical" if the version string uses single
alphabetical character suffix as incremental release, blank to not
consider the unidentified suffixes. This can be expand when more suffix
pattern identified.

refactor cve_check.Version class to use functools and add parameter to
handle suffix condition.

Also update testcases to cover new changes.

(From OE-Core rev: 5dfd5ad5144708b474ef31eaa89a846c57be8ac0)

Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-30 10:41:04 +00:00
Lee Chee Yang
3807c6d9a7 cve-check: replace Looseversion with custom version class
The way distutils.version.LooseVersion compare version are tricky, it treat
all these ( "1.0-beta2", "1.0-rc1", "1.0A", "1.0p2" and "1.0pre1") as greater
version than "1.0". This might be right for "1.0A" and "1.0p1" but not for
the rest, also these version could be confusing, the "p" in "1.0p1" can be
"pre" or "patched" version or even other meaning.

Replace Looseversion with custom class, it uses regex to capture common
version format like "1.1.1" or tag format using date like "2020-12-12" as
release section, check for following known string/tags ( beta, rc, pre, dev,
alpha, preview) as pre-release section, any other trailing characters
are difficult to understand/define so ignore them. Compare release
section and pre-release section saperately.

included selftest for the version class.

[YOCTO#14127]

(From OE-Core rev: 6ced85e9ddd3569240f1e8b82130d1ac0fffbc40)

Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-23 17:08:54 +00:00
Ross Burton
0be2aedfd3 cve-check: show real PN/PV
The output currently shows the remapped product and version fields,
which may not be the actual recipe name/version. As this report is about
recipes, use the real values.

(From OE-Core rev: 18827d7f40db4a4f92680bd59ca655cca373ad65)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-24 10:27:45 +00:00
Chris Laplante
3d24ecf540 cve-check: add CVE_CHECK_REPORT_PATCHED variable to suppress reporting of patched CVEs
Default behavior is not changed. To suppress patched CVEs, set:

        CVE_CHECK_REPORT_PATCHED = ""

(From OE-Core rev: 05bd9f1f006cf94cf5324f96df29cd5862abaf45)

Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-30 15:01:51 +01:00
Chris Laplante
47a35a3843 cve-check: introduce CVE_CHECK_RECIPE_FILE variable to allow changing of per-recipe check file
The addition of this variable also makes it possible to change the
output suffix of the check files, e.g. in local.conf:

CVE_CHECK_MANIFEST_append = ".txt"
CVE_CHECK_RECIPE_FILE_append = ".txt"

(From OE-Core rev: 0d40f1482c6d87785ae47c46c2305e1df46f459a)

Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-30 15:01:51 +01:00
Ross Burton
33efd93517 cve-update-db-native: use fetch task
Instead of inventing a new task to fetch the CVE data, use the existing
fetch task.

(From OE-Core rev: f5f97d33a1703d75b9fd9760f2c7767081538e00)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-12 14:49:00 +01:00
Chris Laplante
06b20b01a0 cve-check: avoid FileNotFoundError if no do_cve_check task has run
For example, if you just run 'bitbake cve-update-db-native' in a clean
build system, |cve_tmp_file| won't exist yet.

(From OE-Core rev: dd4473f3d8e1c1a587b6de660775e4b46ddc5fad)

Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-10 13:48:26 +01:00
Chris Laplante
9ba2f3b8c3 cve-check/cve-update-db-native: use lockfile to fix usage under multiconfig
Previously CVE_CHECK_DB_FILE / CVE_CHECK_DB_DIR was the same across
multiconfigs which led to a race condition wherein multiple
cve-update-db-native:do_populate_cve_db tasks could attempt to write to
the same sqlite database. This led to the following task failure:

    Error executing a python function in exec_python_func() autogenerated:

    The stack trace of python calls that resulted in this exception/failure was:
    File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
         0001:
     *** 0002:do_populate_cve_db(d)
         0003:
    File: '/mnt/data/agent/work/74f119cccb44f133/yocto/sources/poky/meta/recipes-core/meta/cve-update-db-native.bb', lineno: 103, function: do_populate_cve_db
         0099:        if year == date.today().year:
         0100:            cve_f.write('CVE database update : %s\n\n' % date.today())
         0101:
         0102:    cve_f.close()
     *** 0103:    conn.commit()
         0104:    conn.close()
         0105:}
         0106:
         0107:def initialize_db(c):
    Exception: sqlite3.OperationalError: disk I/O error

Use a lockfile to ensure multiple tasks don't step over each other.

(From OE-Core rev: 24e9380643a2ae3fcae193519cb64aedaf682153)

Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-10 13:48:26 +01:00
akuster
fca05777a4 cve-check.bbclass: always save cve report
The cve-check file should be saved always, it has good info.

Put a copy in the log dir as cve-summary with symlinks to latest run.

[Yocto #13974]

(From OE-Core rev: 859849c7b594d844819ad8c3f7d8325388d94b93)

Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-18 11:06:31 +01:00
Ralph Siemsen
9fef505804 cve-check: include epoch in product version output
In the generated cve.log files, include the epoch in the product
version. This better matches how versions are displayed elsewhere,
in particular the bb.warn("Found unpatched CVE...") that appears
on the terminal when CVEs are found.

(From OE-Core rev: e1c3c0b6e5b01304e2127f5058986697e82adf93)

Signed-off-by: Ralph Siemsen <ralph.siemsen@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-04 13:27:31 +01:00
Khem Raj
d0b97f53f9 cve-check: Run it after do_fetch
Certain recipes e.g. bash readline ( from meta-gplv2 ) download patches instead of having them in
metadata, this could fail cve_check

ERROR: readline-5.2-r9 do_cve_check: File Not found: qemuarm/build/../downloads/readline52-001

This patch ensures that download is done before running CVE scan, even
though these will be external patches and may not contain CVE tags as it
expects, but it will fix the run failures as seen above

(From OE-Core rev: e406fcb6c609a0d2456d7da0d2406d2d9fa52dd2)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-24 23:56:03 +01:00
Lee Chee Yang
7c37e5c7cd cve-check: CPE version '-' as all version
CPE version could be '-' to mean no version info.
Current cve_check treat it as not valid and does not report these
CVE but some of these could be a valid vulnerabilities.

Since non-valid CVE can be whitelisted, so treat '-' as all version
and report all these CVE to capture possible vulnerabilities.

Non-valid CVE to be whitelisted separately.

[YOCTO #13617]

(From OE-Core rev: c69ee3594079589d27c10db32bc288566ebde9ef)

Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-01 11:44:23 +01:00