Fixes the following warning sometimes appearing during image builds:
WARNING: The license listed ABC was not in the licenses collected for recipe xyz
The files being looked for here, which runs during do_rootfs,
are written out by the do_populate_lic task for each recipe. However,
there was no explicit dependency between do_rootfs and all of the
do_populate_lic tasks to ensure they had run - only an implicit link via
do_build, so it is possible that sometimes they had not depending on how
the tasks were scheduled. Add an explicit set of dependencies to fix
this.
(From OE-Core master rev: ef7dc532e800d9b170246550cbc8703adf624beb)
(From OE-Core rev: f521d8d2d1ea495383f54e5e7c2754dde007f7eb)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A few short fixes to splitting/iteration done over package lists
in license.bbclass, package_manager.py and rootfs.py.
[YOCTO #7664]
(From OE-Core rev: a902e98c5938f52ec960e0518e0ceaf8f5ee610c)
Signed-off-by: Alex Franco <alejandro.franco@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use canonical_license when doing evaluation of license expresion since
INCOMPATIBLE_LICENSE are already canonized.
[YOCTO #8080]
(From OE-Core rev: 799b25e09a149303edc1c098935f38a438878a95)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than just use d.getVar(X), use the more explict d.getVar(X, False)
since at some point in the future, having the default of expansion would
be nice. This is the first step towards that.
This patch was mostly made using the command:
sed -e 's:\(getVar([^,()]*\)\s*):\1, False):g' -i `grep -ril getVar *`
(From OE-Core rev: ab7c1d239b122c8e549e8112c88fd46c9e2b061b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use image_list_installed_packges instead of open manually the package
manifest and iterate over it.
(From OE-Core rev: 7ffe10df73cc20d10fcd41b121074445273bd60e)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When INCOMPATIBLE_LICENSE's is specified it need to be removed from
license.manifest and also avoid copy to target image.
Add ManifestVisitor that walk the license string searching for
INCOMPATIBLE_LICENSE's if found remove it.
[YOCTO #6765]
(From OE-Core rev: d1278570041029d7c9fc6ce657e9a1701a421841)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add dont_want_licenses as parameter to license_ok function and move it
to oe.license module in order to use in other modules.
(From OE-Core rev: 243fe3a4583a21ad6c0b2a26196ed18d41740f7a)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Reimplemented license_manifest_create from shell to python for
INCOMPATIBLE_LICENSE handle using oe.license module.
Optimizations are made to avoid license copy now uses a hardlink
and symbolic link this helps to save space during build.
(From OE-Core rev: a7976cbae34dc1bd08395afb8d720d43c5f0062e)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some package like linux-firmware has many licenses that aren't in any
way common, and new ones will be added from time to time, in order to
avoid adding bunch of such common license files that are only applicable
to a specific package, NO_GENERIC_LICENSE is added to allow copying license
not in common licenses, it should be used in the recipe as:
NO_GENERIC_LICENSE[<license_name>] = "<license file in fetched source>"
e.g.
NO_GENERIC_LICENSE[Firmware-Abilis] = "LICENCE.Abilis.txt"
(From OE-Core rev: 56930227128d55dab22f79138152b29cf040ceff)
Signed-off-by: Jackie Huang <jackie.huang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the package doesn't contain any file, then the license isn't relevant
as far as the final image is concerned. So we skip the license checking
in license_create_manifest if such case.
(From OE-Core rev: f7b6684d2248f4a56e70b292891f9a454275449d)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixed:
ERROR: Build of do_populate_lic failed
ERROR: Traceback (most recent call last):
File "bitbake/lib/bb/build.py", line 497, in exec_task
return _exec_task(fn, task, d, quieterr)
File "bitbake/lib/bb/build.py", line 437, in _exec_task
exec_func(func, localdata)
File "bitbake/lib/bb/build.py", line 212, in exec_func
exec_func_python(func, d, runfile, cwd=adir)
File "/home/nxadm/nx/ala-blade44.1/builds-2015-03-09-163005/qemuppc_world_oe_bp/bitbake/lib/bb/build.py", line 237, in exec_func_python
os.chdir(cwd)
OSError: [Errno 2] No such file or directory: 'bitbake_build/tmp/work/ppc7400-wrs-linux/taglib/1.9.1-r0/build'
When running setscene, the cwd is $B which maybe removed by
autotools.bbclass or cmake.bbclass when rebuild.
(From OE-Core rev: b3262af11094f93c9588f2e3edc70a5058491e6d)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* add quotes around pkged_lic so that it works correctly with spaces
* fixes following error:
run.license_create_manifest.50601: 193: [: GPLv2: unexpected operator
(From OE-Core rev: 2bb8b2abb689d91b7b7e28e6bd528747bde94dd2)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Use argparse instead of optparse for standardised help output, options
and a much cleaner code structure
* Look up pkgdata directory automatically so the user doesn't have to
specify it
* Use standard logging
NOTE: this does mean a slight change in syntax - if you do want to
specify the pkgdata directory (usually only necessary if you're calling
it from within the build process) you need to use the parameter -p (or
--pkgdata-dir) and specify this before the command, not after it.
Examples:
oe-pkgdata-util find-path /sbin/mke2fs
oe-pkgdata-util lookup-recipe libelf1
oe-pkgdata-util read-value PKGSIZE libc6
oe-pkgdata-util -p /home/user/oe/build/tmp/sysroots/qemux86-64/pkgdata read-value PKGSIZE libc6
(From OE-Core rev: 04dc571ac7c26f0dcf1a1fcd466482e22519998d)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Strictly speaking not all of these characters are operators, so reword
the message to describe them as separators. Also use the standard
"recipename: message" format.
(From OE-Core rev: 9e5da3e83f2a5d43620e07a31728b7427329d8f4)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixed:
* The os.link() reqiures write permission on the src file (suppose the
src file belongs to another user, then you need write permission to harlink to
it since the link count would change)
* Print more info when failed to copy
The warning was like:
WARNING: Could not copy license file COPYING: [Errno 1] Operation not permitted
We couldn't know which recipe print the warning from this message.
(From OE-Core rev: ebc185186c36fe839008d94dbfb779383df960c7)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add check_license_format function that shows warning if LICENSE don't have
valid operators and also if have space separated entries without operator,
add check_license_format validation into base class.
[YOCTO #6758]
(From OE-Core rev: 346a023a42f127881476e760e8fa4e04303849b9)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Drop removal of [|&()*] operators in pkged_lic because this removal is only
needed to validate if license is collected.
[YOCTO #6757]
(From OE-Core rev: 57e5f74382d51f2a8df00e18b6008e3d2b44ad1a)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The whitelist processing in code in base.bbclass does not play well with
wildcards in INCOMPATIBLE_LICENSES. The code expects bad_licenses to
contain actual license names, not wildcards.
Add incompatible_license_contains to replace bb.utils.contains(
"INCOMPATIBLE_LICENSE", **, **, **, d)
[YOCTO #5592]
(From OE-Core rev: 3587653a8d8abc7cfed6a5c6ecfa72bee283e451)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If INCOMPATIBLE_LICENSE=GPLv3, GPLv3+ should be excluded
as well but not now since there is no SPDXLICENSEMAP for
licenses named with 'X+', we can add all the SPDXLICENSEMAP
settings for licenses named with 'X+' in licenses.conf,
but it's more like a duplication, so improve the canonical_license
function to auto map for 'X+' if SPDXLICENSEMAP for 'X' is
available, so GPLv3+ becomes GPL-3.0+.
(From OE-Core rev: 1d6dab1dbbbfbcb32e58dba3111130157ef2b24f)
Signed-off-by: Jackie Huang <jackie.huang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently copies of the license files are made which wastes disk space
and adversely affects performance. We can link these instead in most
cases for small performance gains.
(From OE-Core rev: 0b0f3631fd22f731b6aeedb73965e367b695028b)
(From OE-Core rev: fe00d4f479c4fb5e4be5dda616a4de0a257ef6c3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If INCOMPATIBLE_LICENSE=GPL-3.0 but the recipe sets LICENSE=GPLv3, the current
code won't trigger because they're different strings.
Fix this by attempting to canonicalise every license name to a SPDX name, so
both names in this example become GPL-3.0.
[ YOCTO #5622 ]
(From OE-Core rev: 8998e13fc95f11d15c34fb09d8451a9d4b69f2f1)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we don't fix this, the fix for bug 6502 will trigger warnings
that the write_package_manifest function contains tabs.
Related to fix for [YOCTO #6502].
(From OE-Core rev: 6d93be5338ca301caafbcd44bfe3da08c05610ec)
Signed-off-by: Roxana Ciobanu <roxana.ciobanu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The old wrapper got renamed to image_list_installed_packages().
(From OE-Core rev: 118a2a44bbe5ed2e9bbd0012970686be454e5d4c)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Wihtout this patch, FEEDS functionality is broken, and creating
a manifest from recipe metadata may be faulty when using FEEDS.
(From OE-Core rev: d926c0bdc58ab6dda55eed52b66dbd5834ced64f)
Signed-off-by: David Nyström <david.nystrom@enea.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since the list_installed_packages() function has refactored in
python, do the necessary adjustments to license_create_manifes-
t() and write_image_manifest() in license.bbclass and image.bb-
class respectively;
(From OE-Core rev: 4477991633d2971c17afbd0874b1ab2efb7d6eef)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When originally developed, it was thought a task may have more than one associated
sstate archive. The way the code has grown that idea is now not possible or needed.
We can therefore assume one sstate archive per task and drop the crazy name
mapping code. Simpler is better in this case.
The downside is that various sstate archives will change name so this forces a cache
rebuild. Given the other sstate changes going in at this time, this isn't really
a bad thing as things would rebuild anyway.
(From OE-Core rev: 5afe86a6854b21692fd97c5fc7fab50dbc068acb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For each recipe, it populated license files to ${LICENSE_DIRECTORY}/${PN},
such as kernel's license dir was ${LICENSE_DIRECTORY}/kernel-3.10.17-yocto-standard;
In do_rootfs task, it copied license directories from ${LICENSE_DIRECTORY}/
${pkg}, and ${pkg} was listed in ${INSTALLED_PKGS};
We got ${INSTALLED_PKGS} by rpm query, such as the kernel were 'kernel-*',
but the kernel's PN was linux-yocto, so searching ${LICENSE_DIRECTORY}/
kernel-* failed.
Copied license directories from ${LICENSE_DIRECTORY}/${PN} fixed this issue.
[YOCTO #5572]
(From OE-Core rev: 8968f9a3461912c8de217135f3691c86e2a58e86)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The license_create_manifest function contains bashism, this will lead
to unexpected results on ubuntu build host, as sh is linked to dash on
ubuntu. Even if COPY_LIC_MANIFEST and COPY_LIC_DIRS are enabled, the
license files will still be missing on target.
This patch fixes the above problem.
[YOCTO #5549]
(From OE-Core rev: 4df9daee5c732c0a20dabe8515577238a1508512)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The expected usage of COPY_LIC_MANIFEST and COPY_LIC_DIRS appears to be
to set them to "1" to enable; however the test here is just testing
whether they have a value at all, so setting them to "0" would also
enable them which is somewhat disingenuous. Actually check if they are
set to "1" instead in order to fix this.
(From OE-Core rev: 2d8e124adcf27af524eeeae61daf1b21a1c2f27c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bb.fetch.decodeurl() will throw if it doesn't like the look of the URL that
it's given. (Bitbake's idea of what constitutes a valid URL is somewhat
idiosyncratic so it is fairly easy to trip over this by mistake when writing
a recipe.)
If these exceptions are allowed to propagate all the way up to better_exec()
then we will get a large amount of python stack trace spew when they are
finally caught. Avoid that by catching them locally and throwing
bb.build.FuncFailed() with a suitable explanation instead.
(From OE-Core rev: ef35e164c62d89806367b822e3baeff482ec237f)
Signed-off-by: Phil Blundell <philb@gnu.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bb.utils.copyfile is for a specific purpose and more complicated than needed
here, so just use shutil.copyfile.
(From OE-Core rev: dc81df215cc94c279991df35125d94770a1bc3d2)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently we have a hierarchy of pkgdata directories and the code has to put together
a search path and look through each in turn until it finds the data it needs.
This has lead to a number of hardcoded paths and file globing which
is unpredictable and undesirable. Worse, certain tricks that should be
easy like a GL specific package architecture become problematic with the
curretn search paths.
With the modern sstate code, we can do better and construct a single pkgdata
directory for each machine in just the same way as we do for the sysroot. This
is already tried and well tested. With such a single directory, all the code that
iterated through multiple pkgdata directories and simply be removed and give
a significant simplification of the code. Even existing build directories adapt
to the change well since the package contents doesn't change, just the location
they're installed to and the stamp for them.
The only complication is the we need a different shlibs directory for each
multilib. These are only used by package.bbclass and the simple fix is to
add MLPREFIX to the shlib directory name. This means the multilib packages will
repackage and the sstate checksum will change but an existing build directory
will adapt to the changes safely.
It is close to release however I believe the benefits this patch give us
are worth consideration for inclusion and give us more options for dealing
with problems like the GL one. It also sets the ground work well for
shlibs improvements in 1.6.
(From OE-Core rev: 1b8e4abd2d9c0901d38d89d0f944fe1ffd019379)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These have been deprecated for a long time, convert the remaining
references to the correct modules and prepare for removal of the
compatibility support from bitbake.
(From OE-Core rev: 6a39835af2b2b3c7797fe05479341d71a3f3aaf6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When we don't have a generic license file for the license in question, we can
warn, but we should still include it in the manifest, otherwise the manifest
doesn't reflect reality. Failing to include a license listed in the recipe in
the manifest can't be allowed.
(From OE-Core rev: e87232828b761d56f1ce6a27e4009d350d68209c)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Package listing was implemented in the deb backend some time ago.
(From OE-Core rev: e2915b6e1d2088d3a791bf629dabc58f38940961)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* as reported by Enrico on #oe
11:06:50 < ensc|w> JaMa: might this be caused by dc78ef91a2bf01efb8028c9afbe69e506e016265
which checks for 'd.getVar('LICENSE_CREATE_PACKAGE', True)' evaluating to 'True' for every
string (including the default 0)
(From OE-Core rev: bc42585ad9ca3a9891459ec3234893dff420b95b)
(From OE-Core rev: 0ab56ad52937823dc66b541dd4eda09fb6a34407)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* move it from do_populate_lic to find_license_files so we can reuse it
to populate license in package itself
[YOCTO #3743]
(From OE-Core rev: 833f8c239aa475b3e0cacbd448a90079ac287468)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This gives us an error message (bb.fatal) rather than a traceback due to the
uncaught LicenseError.
(From OE-Core rev: 7fab4cd27a2ad896218123ca82252e2b59526726)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was causing it to only obey the last of the elements in
INCOMPATIBLE_LICENSE.
(From OE-Core rev: df4a4b9792e3bd13b396b94e232c69054fdf19b5)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Recipes define the latter, as that's what goes into the binary packages, not
the former.
(From OE-Core rev: 6a59b294b99b05e8973b5f2d60678988e90e2de3)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow INCOMPATIBLE_LICENSE to be a whitespace-separated list of
incompatible license strings and/or glob patterns.
Also fix wildcarding: the string in INCOMPATIBLE_LICENSE was clearly
intended to match with wildcards (e.g. "*GPLv3" to match both GPLv3
and LGPLv3), but this was broken because of a bug in return_spdx()
which would die with a runtime error when there was no SPDXLICENSEMAP
entry for the string.
(From OE-Core rev: 8a8d00f4c9f7fe5f9f173b43b86cd18a6c75435c)
Signed-off-by: Andy Ross <andy.ross@windriver.com>
Signed-off-by: Elizabeth Flanagan <elizabeth.flanagan@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As locale packages are installed on the image, we should
be including them in the package/license manifest.
This ensures that the manifests are accurate and complete.
[ YOCTO #2461 ]
(From OE-Core rev: 4578b708ada1ffb99ab542da793977ffb90a9b50)
Signed-off-by: Elizabeth Flanagan <elizabeth.flanagan@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some bad logic in license.bbclass misses certain package level
LICENSEs.
(From OE-Core rev: c5a171d5817233c0371e6f5b19f57f3c4b84f5ac)
Signed-off-by: Elizabeth Flanagan <elizabeth.flanagan@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This change means we have more flexibility about when to schedule the license
task and if it changes, we don't repackage everything (which is pointless).
(From OE-Core rev: ee1293446936c5444ece42b60e3ab94189b2fbc3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This requires the changes to bitbake.conf that allow parsing of
license.conf.
As we should now be parsing license.conf, we can move some globals
out of license.bblcass and into the conf file.
(From OE-Core rev: 03e6a7cd27ed109a011fac09cf04412f87f31c3a)
Signed-off-by: Elizabeth Flanagan <elizabeth.flanagan@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
