Fixes [YOCTO #11630]
The "Yocto Project Source Repositories" section is purely conceptual
and thus does not belong in the reconstituted dev-manual, which is
converting to a "how-to" manual. I have moved this section to the
ref-manual under "The Yocto Project Development Environment" section.
Moving the section affected several things. Three figures that where
in the dev-manual were copied and moved to the figures folder of the
ref-manual. This required deleting them from the dev-manual and
adding them to the ref-manual.
The makefile had to be edited to move those figures into the ref-manual
and delete them from the latest version of the dev-manual in order for
the tarball of files to be correct when the manuals are made.
Links to anchors within the moved section had to be fixed in various
manuals. Links to the main section "id" had to be fixed to point
to the ref-manual manual.
(From yocto-docs rev: c619ae2bbc6be93cb0324ca44e936eefec12b28c)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #11630]
The "Open Source Philosophy" section that was in the dev-manual is
really conceptual reference information and has no place in the
dev-manual, which is being re-written to be a "how-to" manual. I
moved the section into the new "ref-development-environment.xml"
chapter.
No links were affected by this.
(From yocto-docs rev: 0a3e65bf7a23eec6e36a3cda3c2011b70aef325b)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #11630]
The ref-manual needs expansion for the old "closer-look" chapter.
This chapter previously held a detailed look at what happens when
a user uses the YP to develop something. Now, the chapter needs
to also contain YP development environment concepts (e.g. open-
source philosophy, etc.), which are coming from the dev-manual.
Because of this, I renamed the "closer-look.xml" chapter to be
"ref-development-environment.xml". I also renamed the larger
section that was formerly the entire chapter into its own section
named "Development Concepts".
Both these changes caused a few links to break. I fixed all the
links from within the various manuals so they would find appropriate
targets.
I did some re-writing for introductory material to introduce the new
chapter and the section on "Development Concepts".
A new file ("ref-development-environment.xml") was added by basically
renaming the "closer-look.xml" chapter. And, the tracking for
"closer-look.xml" was deleted.
(From yocto-docs rev: e37806474578b4f0ed137f64d68a39a17ab60644)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I made some changes to better introduce this reference manual.
Clarified the best scenario for using the manual. Removed the
long list of manual descriptions as that can be referenced further
down in the manual.
(From yocto-docs rev: 8f4555aa387ab3bd3f90f5fcda1d343811ecc168)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #11630]
Moving the "Yocto Project Terms" section from dev-manual to
ref-manual broke all the links to the "Source Directory" term.
I fixed them.
(From yocto-docs rev: d3a206b141e0b0c623362622ec73bbf44118071a)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #11630]
Moving the "Yocto Project Terms" section from the dev-manual to
the ref-manual broke a single link to the "build-system-term"
id. I fixed it.
(From yocto-docs rev: 51bbfbb6030a5298fde1dc0e6f67332cfa59fae2)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #11630]
Moving the "Yocto Project Terms" section from the dev-manual
to the ref-manual broke the links to the "Metadata" term.
I fixed these.
(From yocto-docs rev: 190da4b4d44952d141db26ca72b5bc1a52d77023)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #11630]
Moving the "Yocto Project Terms" section from the dev-manual to the
ref-manual broke links for the "cross-development-toolchain" term.
I have fixed these.
(From yocto-docs rev: f2d2b5936bd339d691ab859cbe3df86e3da58f0c)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #11630]
Moving the "Yocto Project Terms" section from the dev-manual to the
ref-manual broke several links to the id "build-directory". I fixed
these to reference into the ref-manual now.
(From yocto-docs rev: debe2ae8ce203ebe0f1cee0de54124c15df8bb7e)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #11630]
Moving the "Yocto Project Terms" section from the dev-manual to
the ref-manual caused many links local to that section to be
incorrect. I scrubbed the section and fixed all the links.
(From yocto-docs rev: 4b795159aa80184f26ff1181a564516840c373b2)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #11630]
Moving the "Yocto Project Terms" section from the dev-manual to
the ref-manual. Doing so caused all the links to the id
"bitbake-term" to break. These had to be individually fixed.
Discovered two unresolved references that were a consequence of
moving that section to the ref-manual. These were fixed as well.
(From yocto-docs rev: 829ca6b64562f00a69f3956e9636c7edaa90ce16)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was only added for eglibc. That is gone so we can drop this too.
(From OE-Core rev: 7c36b4316ad185088c65a4db8a84b5fcba9eef06)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Patches patching content from other patches is bad. This folds the three
patches into one, improves some of what its doing to be a little
cleaner and adds a proper description to the patch header.
This also moves the STRIP override to the configure commandline rather
than patching and handles passing libdir the same way.
(From OE-Core rev: 186b30798cf74b738483e8708157cac628c1d3cd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some of these are clearly dead, e.g. one binutils patch reverts the effects
of the earlier one.
This also removes the uclibc site files. We now have mechanisms to allow these
to be extended from another layer should someone ever wish to do that.
(From OE-Core rev: e01e7c543a559c8926d72159b5cd55db0c661434)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We now have gcc 6 and gcc 7 recipes, the gcc 5 series can be dropped
as we're no longer going to support it for targets.
(From OE-Core rev: 98183e57dffaa155fc207a28e1c788ff50dc2054)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
uclibc support was removed a while ago and musl works much better. Start to
remove the various overrides and patches related to uclibc which are no longer
needed.
uclibc support in a layer would still be possible. I have strong reasons to
believe nobody is still using uclibc since patches are missing and I doubt
the metadata even parses anymore.
(From OE-Core rev: ec03023d2165b49a52b83bac1ea2f0bfded7b852)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These comments/variables appear to be long dead, remove them.
(From OE-Core rev: a50c8fa7c6c9d40279724a04fb616462b1b491ff)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
uclibc support was removed a while ago and musl works much better. Start to
remove the various overrides and patches related to uclibc which are no longer
needed.
uclibc support in a layer would still be possible. I have strong reasons to
believe nobody is still using uclibc since patches are missing and I doubt
the metadata even parses anymore.
(From OE-Core rev: 653704e9cf325cb494eb23facca19e9f05132ffd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow the creation of ipks to happen in parallel, making best use of resources
on multiprocessor systems.
(From OE-Core rev: 07f6c0b464f0671bc39116317138e4ddf27bdae9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow the creation of debs to happen in parallel, making best use of resources
on multiprocessor systems.
(From OE-Core rev: dd540fba6c65fb74df014f5d9d2965078314a790)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we have large amounts of parallelism, pseudo can end up with too
many open connections and will no longer accept further connections,
hanging. This patch works around that by closing some clients, allowing
turnover of connections and unblocking the system. The downside is a small
but theoretical window of data loss. This is likely better than locking
up entirely though. Discussions with Peter are onging about how we could
better fix this.
(From OE-Core rev: f3589f154dad1c92e599737623d392508810ae7e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The recent ovmf update broke secureboot because upstream changed the
way how openssl gets compiled into ovmf. It's now integrated directly
into the ovmf build process, without having to patch it first.
In addition, more recent OpenSSL releases are supported. 1.1.0e was
explicitly mentioned in the ovmf commits and because the current
1.1.0f only has minor build enhancements, 1.1.0e is used here.
(From OE-Core rev: 1d9a88f635549e68562de681e297b9270ad02d4e)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is the potential for sensitive information to leak through the urls
there and removing it brings this into the behavior of the other package
backends since filtering it is likely error prone.
Since ipks don't appear to be generated at all if we don't set this, set
the field to the recipe name used (basename only, no paths). This avoids
information leaking. We may want to drop the field if opkg can allow that
at a future point but the recipe name is a suitable identifier for now.
Reported-by: Andrej Valek <andrej.valek@siemens.com>
(From OE-Core rev: ec7feb9d315f357b9a073425a31b352ec24ddfd9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Just discovered a minor issue in the bitbake manual - under the variable
glossary entry for BBLAYERS_FETCH_DIR we say there's a default value of
${COREBASE} but that's not actually true in bitbake itself - there's no
default in bitbake, that default is set in OE-Core. We can keep this in the YP
ref manual but in the bitbake manual that sentence should be removed.
(Bitbake rev: 7b4e81772ed8789521661a3b32b576c91276ae34)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If an event handler for bb.event.BuildCompleted fails, we still need to
call finishAsyncCommand() or else BitBake will just exit immediately
without showing any error summary, or worse in the case of memory
resident mode BitBake will hang and if you Ctrl+C to break out, the
command won't be marked as finished which means that no further commands
will be able to be executed until the server is manually restarted.
(Bitbake rev: 5639faa3eef55cc476a82e810e61ca228cbdf221)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The main point of memory resident bitbake is to avoid loading data
unnecessarily on every bitbake invocation. Unfortunately the code that
updated options from the UI was simply treating the fact that either
of the "prefile" or "postfile" options were in the list of options
passed in as an indication that the configuration was invalid, which was
bad because these are always passed in. We only need to mark the
configuration as invalid and thus reload it (and thus reload the cache)
if the option value has actually changed.
At the same time, the recently handled "tracking" option needs to be
treated in a similar manner since the configuration needs to be reparsed
if that has changed. Also, add a few extra debug messages to aid
debugging this code in future.
(Bitbake rev: 18dfd144d0da6ff662308ce4f9d135dc11412133)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Differentiate between fn and taskfn in "execute".
This was somehow missed in "fakeroot" handling.
(Bitbake rev: d848bff7cf78f63986467b51f701a998a480eb25)
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When a process terminates, some messages may still remain in stdout or
stderr and do not make it into the log file.
In addition, the messages that do make it to the log file may end up in
the log file in incorrect order.
This patch flushes all messages into the log file after the
process terminates. Some additional log flushing is also needed
to keep the various messages showing up in the log file in proper order.
[YOCTO#10785]
(Bitbake rev: 1f6e6aa8262369eafc3bbf9f01f8d981f90becdf)
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is OE-specific customisation so set the status as such.
(From OE-Core rev: 059846662f1ea1c82804cfce5f91afcb2980ec8a)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* since this commit:
commit f5a1013ffa9815f22e13989e2bcb83f966e7ce2c
Author: Ross Burton <ross.burton@intel.com>
Date: Tue Apr 18 16:19:12 2017 +0100
package_manager: don't race on a file when installing complementary
packages
the file isn't closed before oe-pkgdata-util uses it and this
temporary file might look empty to oe-pkgdata-util, because it
wasn't flushed yet. Which resulted in almost empty debugfs tarballs
and no locale packages in regular rootfs.
* without this change:
124K May 30 07:41 core-image-full-cmdline-raspberrypi3-64-20170530054003-dbg.rootfs.tar.gz
* with this change:
173M May 30 07:29 core-image-full-cmdline-raspberrypi3-64-20170530052715-dbg.rootfs.tar.gz
(From OE-Core rev: 877d38db08aa7060d16405443cf70539c559fe82)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This achieves the same goal as the same change to bitbake.conf itself,
but because the class gets added later as part expanding INHERIT, this
new approach is less likely to run into problems when DISTRO_FEATURES
contains complex code.
Another difference is that the class currently does not get inherited
by default and thus is completely absent from a build unless some
layer or include file adds it to INHERIT.
Compared to the earlier code in bitbake.conf and a similar class in
intel-iot-refkit, additional overrides now get sorted. This makes the
final OVERRIDES more deterministic.
The lessons learned about unintentionally depending on OVERRIDES are
documented in the class because such problems are more likely to show
up as unexpected signature differences when using this class.
(From OE-Core rev: e982ca7f2d4fb0aef3fedc00af1e90f613e7b5ee)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This reverts commit 3b3ae91a22d6f685e804df4f32cdeebe1bd6bd88.
It turned out that the code which expands DISTRO_FEATURES early during
base config parsing can fail because some entries in DISTRO_FEATURES
might call Python functions like base_conditional() from base.bbclass
which aren't defined yet.
A different solution will be needed.
(From OE-Core rev: 0c6e4a14ba8d9d9701ec16ffe46d618f41633571)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* the /usr/lib/python3.5/_compression.py file is possibly incorrectly included
in python3-misc. This runtime dependency is needed in order to use e.g. gzip.py in runtime:
>>> import tarfile, zlib, gzip
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.5/gzip.py", line 12, in <module>
import _compression
ImportError: No module named '_compression'
* at least python3-tests and lzma and bz2 still in python3-misc are using this as well:
$ grep -R import.*_compression tmp-glibc/work/aarch64-oe-linux/python3/3.5.2-r1.0/packages-split/
tmp-glibc/work/aarch64-oe-linux/python3/3.5.2-r1.0/packages-split/python3-compression/usr/lib/python3.5/gzip.py:import _compression
tmp-glibc/work/aarch64-oe-linux/python3/3.5.2-r1.0/packages-split/python3-misc/usr/lib/python3.5/lzma.py:import _compression
tmp-glibc/work/aarch64-oe-linux/python3/3.5.2-r1.0/packages-split/python3-misc/usr/lib/python3.5/bz2.py:import _compression
tmp-glibc/work/aarch64-oe-linux/python3/3.5.2-r1.0/packages-split/python3-tests/usr/lib/python3.5/test/test_bz2.py:import _compression
tmp-glibc/work/aarch64-oe-linux/python3/3.5.2-r1.0/packages-split/python3-tests/usr/lib/python3.5/test/test_lzma.py:import _compression
and python3-tests are using it as well, so add new runtime dependency
on python3-compression
(From OE-Core rev: 987363c3c720b3764f4d64976d7455f6b0bae99c)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you attempt to build an image with both attr(-doc) and man-pages
packages your rootfs might fail to assemble. The error will be
something like:
Error: Transaction check error:
file /usr/share/man/man2/fgetxattr.2 from install of \
attr-doc-2.4.47-r0.core2_64 conflicts with file from \
package man-pages-4.11-r0.core2_64
(the error is usually only seen on builders which don't have manpages
installed, if you have /usr/share/man/man1/man.1.gz your build will
complete but you will have duplicate manpages, just one zipped and one
not)
Backporting changes from upstream attr removes the conflicted files in
favour of those in the man-pages package.
(From OE-Core rev: 5e6595ef42807c8d2a100da3d9862152daf68d3d)
Signed-off-by: Mark Asselstine <mark.asselstine@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If these are set to URLs then the errors produced are not helpful.
(From OE-Core rev: 946b6623154e748a0d75ff498802a720aeec27a6)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Goes down to 40 seconds from over 4 minutes :)
Note that there is no control over the amount of shell jobs; on my machine
this is not a problem, but if it's a problem on less capable hardware,
we can add some kind of limiter.
(From OE-Core rev: cd9af17028c069f52fb0616074170093dd63c143)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
runqemu uses stty to change terminal settings to give users
better control to qemu. However, stty does not work when
runqemu is run directly or indirectly via oe-selftest in
a Docker container (presumably some problems with Docker's
pseudo-tty implementation).
The error reported is:
stty: 'standard input': Inappropriate ioctl for device
As runqemu recently moved to subprocess.check_call() for
stty calls we now get thrown an error and all runqemu
runs fail.
sys.stdin.isatty() does proper job in detecting if the stty
calls can work so we use that check before running the stty
subprocess operations.
(From OE-Core rev: 06742ed59092530aedf03f65c3c9542c24ff7ac3)
Signed-off-by: Mikko Ylinen <mikko.ylinen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added leading space to vardeps to avoid flag value to be
added to the existing value without a separator.
(From OE-Core rev: f582773c2b1e8db441f397867d3c9665fd265cec)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Replaced setVarFlag calls to appendVarFlag to allow
modification of prefuncs, postfuncs and subimages flags
in inherited image classes.
[YOCTO #11372]
(From OE-Core rev: 6690f3ab43c04fa7cff7215d4a5d8d639e41aed8)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Replaced repeated expression "do_image_%s" % t with
a variable 'task' to simplify the code and increase
readability.
(From OE-Core rev: d24dd95f2c1c7f773875454bee3f2016c4e7553c)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The function is "sysroot_strip" instead of "split_and_strip_files".
(From OE-Core rev: f2d349887710dcae05db09dc2277223e85fa1c19)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
connman fails to start in systemd based read-only images while creating links:
Jun 08 12:53:56 qemux86-64 systemd[1]: Starting Create Volatile Files
and Directories...
Jun 08 12:53:56 qemux86-64 systemd-tmpfiles[366]:
[[0;1;31msymlink(/var/run/connman/resolv.conf, /etc/resolv.conf) failed:
Read-only file system[[0m
Fix this failure and make connman co-exist with systemd-resolved.
(From OE-Core rev: 732e1f74bb9f5ecc98b29197f6bcab117710adab)
Signed-off-by: Maxin B. John <maxin.john@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
