When using the Go 'vendor' mechanism to bring in
dependencies for a Go package, the default GO_INSTALL
setting, which uses the '...' wildcard, will include
the vendored packages in the build, which produces
incorrect results.
There are also some Go packages that are structured
poorly, so that the '...' wildcard results in building
example or test code that should not be included in
the build, or fail to build.
This patch adds a mechanism for filtering out a
subset of the sources. It defaults to filtering
out everything under the 'vendor' subdirectory
under package's main directory, which is the
normal location for vendored packages, but can
be overridden by a recipe to filter out other
subdirectories, if needed.
(From OE-Core rev: 9819353726d85780546158428bd97a253705017d)
Signed-off-by: Matt Madison <matt@madison.systems>
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Go source trees have a particular structure, with all
sources located under ${GOROOT}/src/<import-path>.
The fetcher step implemented by the 'go get' command
automatically follows this structure, so we need
to do the same here.
Since most Go packages are hosted in git repositories,
this adds a custom do_unpack() function that sets
the destsuffix to match the expected directory structure,
for any git SRC_URIs that haven't had a destsuffix
explicitly set in the recipe.
This simplifies recipe writing for the most common
cases.
(From OE-Core rev: efcf6513b71021ea4bfe6fbaa326e6591dee487d)
Signed-off-by: Matt Madison <matt@madison.systems>
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a do_configure task to populate ${B} by symlinking
in the src subdirectory under ${S}, which lets us point
GOPATH at ${B}. This lets us take advantage of the
automatic directory creation and cleaning for do_configure.
This necessitates a change to do_install to split the
installation of the sources and built artifacts. Taking
advantage of some additional tar options, we can eliminate
the extra staging area and extra recursive chown command.
So overall efficiency should be improved.
(From OE-Core rev: c62a083306c26b7e4deca1ff41336bb6b33d5b3a)
Signed-off-by: Matt Madison <matt@madison.systems>
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since this is a class, it should follow the
class function export mechanism for its task
functions, and should set directory-related
flags for directories they need.
(From OE-Core rev: 0369a99ad1c9e3a9a6394c723461795460dc2c76)
Signed-off-by: Matt Madison <matt@madison.systems>
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The previous logic applied a regex to TUNE_FEATURES which could
set the GOARM value to 7 incorrectly, for example when dealing
with an arm1176 core. Simplify to check for the presence of
"armv7" instead. At the same time add a check for "armv6" and
set GOARM to 6 in that case.
(From OE-Core rev: 07b60c15e9ef650940afdde37bf3f3b9c50a336d)
Signed-off-by: Will Newton <willn@resin.io>
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Go toolchain changes the installation path when building for the same
architecture as the build host. This was already been considered in
the GO_BUILD_BINDIR variable but was not being used by the go class.
This fixes following error:
,----
| ERROR: go-dep-0.3.0-r0 do_package: QA Issue: go-dep: Files/directories
| were installed but not shipped in any package:
| /usr/lib/x86_64-oel-linux/go/bin/dep
| Please set FILES such that these items are packaged. Alternatively if
| they are unneeded, avoid installing them or delete them within
| do_install.
`----
(From OE-Core rev: 56ce355b3d775e801d2ca89ee812571e794311cd)
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The d.getVar has the second argument as True by default, avoid passing
it here.
(From OE-Core rev: 1a5026db41929d42bece22bd0ae60c13219a98f5)
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The update-alternatives where using relative links so not being really
in use since December 2016 (see OE-Core:c7bc46b9 "kernel: Fix
symlinks") so instead we now generate the relative symlinks during the
do_install task and drop the update-alternatives use at all.
Acked-by: Saul Wold <sgw@linux.intel.com>
(From OE-Core rev: 10a1b293191268e6792ac8e27bd6427f1974c7ce)
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Current implementation does not handle possible exceptions coming from child
processes, the latter responsible for creating packages. With the aim to have more
control, use pipes to communicate exceptions and stop package creation in case
of failure.
Helps to debug [YOCTO #12012].
(From OE-Core rev: 11350a67ba137f560d04aa643ff500a7ff112c73)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Redirecting stderr to stdout helps debugging issues, i.e instead of just
getting the return code, get also the error log from the pkg manger
This commit is in the way to figure out the root cause of [YOCTO #12012],
where dpkg-deb fails with a 2 return code and according to the man page,
there are multiple issues leading to the same code.
(From OE-Core rev: 9ff023fb26f5f0ce19e757beda00ccc32c009b21)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* QA check which aren't included in WARN_QA and ERROR_QA are shown
during the build only as NOTE message (not shown at all with default
knotty setting), so it might be surprising to see them later in qa.log
file
(From OE-Core rev: 35ab2c7b08359f22f74106339841f8134123adf4)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
While poky master branch has been fixed so that all CVE patch files have
the:
CVE: CVE-2017-1234556
strings in the patch comments, many older versions of poky and other meta
layers are not, but the CVE patches quite often have the CVE id in the
patch file name.
If the CVE: string also found, there are no duplicates in the report.
(From OE-Core rev: 5ee5b0c66627c9e974c838b86e2e659c2f601f2a)
Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The do_rootfs log contains a number of unsatisfied package
recommendations. At the moment those are only visible when
reviewing the rootfs log.
This patch adds an extra check to surface any unsatisfied
recommendation as WARNINGS to the build output.
Enable this check with:
ROOTFS_POSTPROCESS_COMMAND += "rootfs_log_check_recommends;"
(From OE-Core rev: 9d049bf7941f30e35c51775684559e95185fba96)
Signed-off-by: Jose Alarcon <jose.alarcon@ge.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The top level LICENSE file is not actually a license, it refers
other licenses that are used by Bitbake and Meta-data. Relying
on this file could cause problems for recipes when this file
changes, which it is about to.
(From OE-Core rev: a1948ab38c9cb7f0b16cce9dadc03ae6e2fe44ad)
(From OE-Core rev: dd75644db2f662f0d88529a068bbfb599c9790a9)
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Perl dependencies may look as "Perl(Foo::Bar)", but dpkg does not
support the non-alphanumeric characters. There was already special
handling present for turning '(' and ')' into '__'. This change does
the same for ':'.
(From OE-Core rev: a34e397095a9c2f8d0af1168ceab295af659242d)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The silent rpmdeps failures fixed in a preceeding commit mean we
need to rerun all packaging.
(From OE-Core rev: 77abe012c496a85f56191abc769a7da07e9f8c0e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There was a logic error in the dependency cleanup code which meant
it would remove dependencies which other tasks still depended upon.
Fix the path names so the comparisions work as intended.
This fixes dependencies accidentally disappearing from sysroots
under certain reconfiguration situations.
(From OE-Core rev: 1634fe5148b3501f2c1b75cf7fb704a2ef60424e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In OE-Core commit a48fea275b, a check on
the value of a "sha256" parameter was added, however there was no
mention of this in the commit message and no corresponding code to
actually verify the checksum as sha256 was added along with it either,
so there's no point in getting the value. Additionally it was assuming
that a sha256 value would be present without checking first, with the
result that if you leave out the md5 value in a recipe intentionally in
order to get it to tell you the correct value on the next build, you got
a traceback instead of the appropriate error containing the information.
Drop this entirely - if we want to implement this we need to do it
properly.
(From OE-Core rev: e9eaa7d15fe7ab643ab19556dab84051f8f1974e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When compressing with "gzip", do not save the time stamp in
the compressed file metadata.
(From OE-Core rev: a88c16c2dc5dd7e2fbf7ca908a8b48da7be8b34c)
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The lines in qemuboot.conf are in random order.
This patch fixes this by printing the lines in sorted order.
This makes it easier to compare two different builds for
any differences.
(From OE-Core rev: 0868edace0750862168abc8d504891646afc8c76)
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For URIs with the npm:// transport but with no other slash in it, the
common MIRRORS and PREMIRRORS pattern of npm://.*/.* fails to match.
Make the last slash in the pattern optional in the mirros.bbclass and
own-mirrors.bbclass classes.
Many URIs with the npm:// transport have no slash after the host part:
npm://registry.npmjs.org;name=foo;version=0.1.2
This means that MIRRORS and PREMIRRORS containing entries like the
first one will not match these URIs:
npm://.*/.* # fails to match
npm://.*/?.* # matches this and URIs with path components
For normal regular expressions, a pattern like 'npm://.*(/.*)?' would
probably be preferred, but that won't work here: the pattern gets split
into the substrings 'npm', '.*(' and '/.*)?', which are not valid
regular expressions individually.
(From OE-Core rev: 0d1e2b4507af28fc451b8fa94130a39ac342637d)
Signed-off-by: Olaf Mandel <o.mandel@menlosystems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
gettext has a notoriously slow configuration step, and so in my testing
this greatly speeds up building core-image-minimal: from 21m36s to 19m2s
(empty sstate and tmp, but pre-populated downloads).
I have also built world, and core-image-sato to make sure it doesn't break
or modify the build, and there is no difference whatsoever in packages
and images content. Target gettext seems not to be used for anything.
Also fix up insane.bbclass to remove the corresponding QA check.
(From OE-Core rev: 1a7b843d575b290917d1e379c2ba106460988230)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since we disabled uppercase characters in overrides a few releases
ago, uppercase characters in recipe names (and for that matter, distro
and machine names) cannot be supported due to their reliance upon
overrides including the name.
QA check will produce an warning message when it verify that recipe
name is uppercase.
[YOCTO# 11592]
(From OE-Core rev: 4713f8b2c4f2c74239d284adcf1e59e61aa66576)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently items are added to the sysroot, the obsolete items are removed. If
a change such as pkgconfig -> pkgconf is made, this leads to conflicts of
overlapping files in the sysroot.
In order to better support this, handle removing items before adding them.
This requires some minor refactoring to construct the installed list
before the main function loop, otherwise there are no changes in this
patch other than reordering the operations.
(From OE-Core rev: add4f107c151d32d9ea914bb0b93c3d3c17c776c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently a task could remove a dependency needed by another task leading
to build failures, often due to missing dependencies (e.g. dynamic libraries
not being found). This was often seen for all-arch recipes in package_write_rpm.
When removing a dependency, first check that no other task active for the
recipe has that same dependency.
(From OE-Core rev: ff3617cc2cd5618f48a25aa4e3b2014430fcbe23)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Whilst this should work we see failures in testsdkext at the moment when
this is set. Add this to the blacklist for now until we can fix these issues
meaning we can at least test BB_SERVER_TIMEOUT in other scenarios.
Bug 119733 has been opened to track this.
(From OE-Core rev: 88cb174680cd37b5afb2b79e8248979895c35312)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
commit 44aea7b873 [kernel-yocto: ensure that only valid BSPs are built]
introduced a new check to ensure that a valid BSP definition was found,
rather than building something that 'closely' matched the current
MACHINE.
This check breaks valid configurations which do not have a bsp
definition but are otherwise completely configured machines.
To allow both elements to co-exist (and not add warnings or errors
to otherwise valid builds), we first check to see if an empty bsp
definition was found, but then check to see if a defconfig was
provided.
If a defconfig has been provided, that is a sign that the board
configuration is complete and we should continue the build without
otherwise bothering the user.
Tested on meta-raspberrypi and linux-yocto*
(From OE-Core rev: 156cf9e25f875a50f907e570d9b62cbc2c051133)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The ptest log will be saved to buildhistory/ptest, we can easily get
the regression result between builds by:
$ git show HEAD ptest/pass.fail.skip.*
[YOCTO #11547]
(From OE-Core rev: dcb6cd19fb8c639cb844d116fb83827267f37421)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It's very important to add IMAGE_CLASSES += "testimage" in local.conf firstly,
otherwise the var like TEST_LOG_DIR (defined in testimage.bbclass) will not be
in testdata.json.
[YOCTO #11547]
(From OE-Core rev: 5663ed989f0af5b1c61c74288ec421cbca2261e7)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This makes the user easier to know how to make commit in buildhistory.
[YOCTO #11547]
(From OE-Core rev: 164a0bd847f02ca65dcd53ddc789690060274191)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes error "Exception: NameError: name 'errno' is not defined"
during build-sysroots.bb:do_build_target_sysroot
(From OE-Core rev: b1e482ae20b9c714f4089d4da6470d041242e441)
Signed-off-by: Marko, Peter <peter.marko@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Refactor get_cc_option so it is simpler and also does not fail
when using external toolchain.
(From OE-Core rev: d4564ee59df907d1e01a3610ac88a35f9a78c1ce)
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There was a bug in the search routines responsible for locating
BSP definitions which returned a valid match if only the ktype
matched.
This meant that someone looking for "qemux86foo" (which is an
invalid definition) would potentially end up building "qemuarm"
and be none the wiser (until it didn't boot).
With this fix to the tools search routine, and improved return
code testing, we will now stop the build and report and error to
the user.
[YOCTO: #11878]
(From OE-Core rev: 44aea7b87307795fe4e089c51d45afccaa2f6525)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Hard depending on the kernel makes it impossible to install kernel
modules without getting the kernel image installed too. This is
inconvenient in e.g. initramdisks, where the kernel is loaded from
outside the initramdisk.
Making the kernel modules rrecommend kernel-image-<version> instead of
rdepending on it, makes it possible to install kernel modules without
the kernel image by setting "kernel-image" in BAD_RECOMMENDATIONS.
(From OE-Core rev: 5dd7ddb66a6846d9bb59dc7833e8318992d0e645)
Signed-off-by: Martin Hundebøll <mnhu@prevas.dk>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In images built with pam in DISTRO_FEATURES, we end up with dangling symlinks
if su is not packaged into image
$ ls /mnt/a/oe/build/tmp/work/raspberrypi3-bec-linux-gnueabi/core-image-minimal/1.0-r0/rootfs/etc/pam.d/su-l -l
lrwxrwxrwx 1 kraj users 2 Aug 9 07:56 /mnt/a/oe/build/tmp/work/raspberrypi3-bec-linux-gnueabi/core-image-minimal/1.0-r0/rootfs/etc/pam.d/su-l -> su
This causes image do_rootfs to fail
| sed: can't read /mnt/a/oe/build/tmp/work/raspberrypi3-bec-linux-gnueabi/core-image-minimal/1.0-r0/rootfs/etc/pam.d/s
u-l: No such file or directory
| WARNING: /mnt/a/oe/build/tmp/work/raspberrypi3-bec-linux-gnueabi/core-image-minimal/1.0-r0/temp/run.ssh_allow_empty_
password.19238:1 exit 2 from 'sed -i 's/nullok_secure/nullok/' /mnt/a/oe/build/tmp/work/raspberrypi3-bec-linux-gnueabi
/core-image-minimal/1.0-r0/rootfs/etc/pam.d/*'
Therefore we need to filter out dangling symlinks before sed'ing
things out
(From OE-Core rev: b92105e5a085c8cd3c650579644922ed97163e73)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
testdata and testdata_link may point to the same file, in particular
when IMAGE_LINK_NAME and IMAGE_NAME are equal.
Check if this is the case before creating a symlink that points to
itself and makes the next build fail.
(From OE-Core rev: b516394f9e7858062aa7b042aa4a1bdef9d3a941)
Signed-off-by: Mike Looijmans <mike.looijmans@topic.nl>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When IMAGE_NAME and IMAGE_LINK_NAME are equal, do_write_qemuboot_conf will
create a symlink that links to itself.
Check if this is the case before creating the link.
(From OE-Core rev: f46652e77f467861dc68c3a8e54f27d08659222d)
Signed-off-by: Mike Looijmans <mike.looijmans@topic.nl>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Besides providing the NATIVELSBSTRING, include distro info when creating
the (json) error report. This information provides better info than the
standard 'universal*' string for uninative builds.
[YOCTO #11824]
(From OE-Core rev: fdfbfc954a3e0ad79dc2bed9828f9f5dc40d506f)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Commit d58b1d196 moved from chunk to serial signing, but neither of both approaches
allowed the user to select the chunks size. This patch allows the user to select
a chunk size through RPM_GPG_SIGN_CHUNK defaulting to BB_NUMBER_THREADS, considered
a good default. Indirectly, this change reduces the number of processes spawn
to number-of-packages/RPM_GPG_SIGN_CHUNK.
(From OE-Core rev: f7f78e73f1cd15f4233a231364b14438af758628)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In commit 255caaad:
[ packagefeed-stability.bbclass: change a bb.plain to bb.note ]
I changed a bb.plain to bb.note to get a cleaner console, but there is
still one more bb.plain left which I forgot to change.
(From OE-Core rev: ccb96849e010883cb1cb4d384f017d05eb6bbcee)
Signed-off-by: Ming Liu <peter.x.liu@external.atlascopco.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Need to add ilp32 to known list of abi variants
(From OE-Core rev: 6c5e50d26cbb9172d7264e7a8fed3edf0c0e8414)
Signed-off-by: Vishal Bhoj <vishal.bhoj@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a new file, fs-perms-persistent-log.txt, which treats /var/log
as a directory instead of a link.
Modify package.bbclass to use this file if VOLATILE_LOG_DIR is set to boolean
false value.
[YOCTO #6132]
(From OE-Core rev: 6b22e247bf91f112401cad822cd7fe0b5621ffe0)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
