In order to better support multilib processing, switch from PN
to BPN.
(From OE-Core rev: 75f70ab534f6dfef961906fb5c252f9e5310d38f)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Refactor in order to:
* Deprecate the old interfaces, but keep them for compatibility
* Provide a new, interface -- capable of working with split packages
* Each update-alternative will now set proper "per-file" provides
Note: this adds a warning message when the older deprecated behavior is
used. The older behavior has been fully tested using oe-core.
(From OE-Core rev: 309117d26de6a87b16406a44a0cefcbaaf7b5d7a)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some image classes such as bootimg save files into ${S} as part of rootfs
generation. For correctness we should therefore clean this at the start of
image generation to ensure reproducibility.
I found this issue when some files I thought should disappear from my rootfs
would not disappear.
(From OE-Core rev: 363424cdc78cafa278008f973c2b4288297ebf8c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When trying to understand why a QA wanring such as:
ERROR: QA Issue: foo rdepends on bar-dev
it is very difficult to figure out where the bar-dev dependency
comes from, since many of them are added dynamically.
This adds a debug statement that says which dependency adds an
rdepends to the system.
Also, while doing this work, it was noted that the same dependencies
were being scanned for over and over. Instead we shorten the list
by only added to the dep list if the dependency was not already there.
(From OE-Core rev: 257b9e371143421b78a6991ef5401e564918c164)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The following functions in meta/classes/utils.bbclass were not
used by anyone, and they didn't work when I tried to use them:
def oe_popen_env(d):
def oe_run(d, cmd, **kwargs):
def oe_popen(d, cmd, **kwargs):
def oe_system(d, cmd, **kwargs):
There error was:
AttributeError: type object 'str' has no attribute 'getVar'
We have bitbake/lib/bb/process.py to instead, so remove them.
[YOCTO #2489]
(From OE-Core rev: 15ad62ab1be060d8a7cdc2d28167ea3af4cfd5d5)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This enables a user to use bitbake -e even when the sanity checks are
failing.
(From OE-Core rev: be317df78535120137ecaadf797e3f4dfe04119e)
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The call to bb.mkdirhier() in check_create_long_filename() can fail with an
OSError, explicitly catch this and report something useful to the user.
(From OE-Core rev: b066906477eb0496a2babb3d8e87682a1b7df0de)
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the user does not have write permissions to SSTATE_CACHE, detected by
the check_create_long_filename() test failing with a "Permission denied"
value in strerror, then suggest they might want to use the location as
an entry in SSTATE_MIRRORS.
(From OE-Core rev: 719d44305508c75a9f2decacdff6558ca14277bb)
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Each failure in the sanity message should be reported on a new line.
(From OE-Core rev: 9362702aa9dd82fed7e6d6e3cb1289521b802075)
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Replace calls to data.getVar(VARIABLE, data_object, expand) to
direct calls to the getVar method the the data_object.
(From OE-Core rev: 41ee978b62a9c40f36f8ad0acef147e36edffa17)
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
At the ConfigParsed event the datastore has yet to be finalised and thus
appends and overrides have not been set.
To ensure the sanity check is being run against the configuration values
the user has set call finalize() on a copy of the datastore and pass that
for all sanity checks.
(From OE-Core rev: 71142172efc0f44a50216550c2b6cc3094fdc21d)
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This has caused problems for several users, including the Yocto Project
autobuilder. Since the message was added in order to be more user friendly
revert the change.
This reverts commit 0c0c4efbf92bcf0f8942f17c18525a4b4ed1798c.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Deleting .pyo files causes them to get compiled on the target.
* First boot gets *really* slow for python based projects.
* No space gets saved on the target.
* The package manager doesn't know about the files and
therefore fails to uninstall them, occupying space and causing
uninstalled python scripts to remain executable.
* It's inconsistent, because python itself and autotools based
projects already ship .pyo files.
* Probably .pyo files were deleted because .pyc files were
available earlier, but this has changed and OE-Core's python
now only generates optimized .pyo files. Deletion of .pyo was
introduced in 2008, python/04-default-is-optimized.patch
was introduced in 2009.
(From OE-Core rev: 25e186ad5b75bd2f93435857580bd16698e18e21)
Signed-off-by: Andreas Oberritter <obi@opendreambox.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix the incremental rpm image generation, it didn't work since the code
has been changed.
The btmanifest should have a ".manifest" suffix, so that it can be moved
to ${T} by rootfs_rpm.bbclass:
mv ${IMAGE_ROOTFS}/install/*.manifest ${T}/
Note: The locale pkgs would always be re-installed.
[YOCTO #2440]
(From OE-Core rev: 5149630746626c6d416f26ab9dd1c7213fcd8c50)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Connect the new fetcher file checksum code so that we get a list of the
files to be checksummed at parse time.
The file-checksums flag will not be read unless we are using a version
of BitBake that supports the function we call within it, so it is safe
to include this change even when the metadata will still be used with
older versions of BitBake.
Implements [YOCTO #2044].
(From OE-Core rev: f26065629b6397d129db930268b72164f8e5d3e4)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
needed for new architecture support
(From OE-Core rev: 75167eb890d8e4683f1e3852e3b77c67d160bea7)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Jason Wessel noticed that a package without any fixmepath entries would
generate a sed warning about no input files. This patch resolves that
by ensuring that an empty fixmepath file never gets written into the
sstate archive. Also we avoid a second message by only doing xargs if
we got input.
(From OE-Core rev: 326563d5a897ae2dba7cfd8d73579d3d979d72c8)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added a new PACKAGEFUNCS function to check the multilib packages'
dependency.
(From OE-Core rev: f193729cdf0f3eccf96a8ce7bf5f599eca89a0d0)
Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added multilib prefix to the locale related package names/dependencies.
(From OE-Core rev: a52f7cf2cc90d918e3250c410995dcc1f3bfd5ee)
Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The user needs read and write permissions to SSTATE_DIR, check
whether they have sufficient permissions and if not recommend
use of SSTATE_MIRRORS.
(From OE-Core rev: 0c0c4efbf92bcf0f8942f17c18525a4b4ed1798c)
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add two default set(SOURCE_ARCHIVE_LOG_WITH_SCRIPTS, \
SOURCE_ARCHIVE_PACKAGE_TYPE) to archiver.bbclass for avoiding \
building error when forgetting to assign to them.
(From OE-Core rev: 5bda8f33b6763fb519543d0f9e35c970a31d39b8)
Signed-off-by: Xiaofeng Yan <xiaofeng.yan@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to make sharing sstate files easier chmod them with 0664 permissions so that they are readable and writable by the user and any other members of their group.
(From OE-Core rev: dae9ad8a0ba0343e3083694cdcb20f0d02927ad0)
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It's common to provide a non-machine-suffixed link in DEPLOY_DIR_IMAGE, so
let's be consistent and do so here as well.
(From OE-Core rev: c1c8d2f3cffc540380c0a5fcdda48d64cbec333a)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you point to snapshot.debian.net/archive/pool then it will fetch
you a html page which will end up in corrupt download. The locations
have changed for archives and here we point the mirror to right
location.
(From OE-Core rev: 15e8aa4cd773022b36a66f70e4f6436289f3d1bc)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The fixmepath file that is generated contains a list of all of the files
that need their paths fixed. In the previous version the fixmepath was
generated to include all of the files that sed may have changed. In the
new version, we first grep the files to see if they contain a path that
needs to be changed, only then do we perform the sed operation on those files.
This results in a modest performance increate in the creation of the sstate
file. The following numbers include the do_package and do_populate_sysroot
tasks on the perl recipe.
Before the change:
real 4m23.018s
user 1m57.067s
sys 1m33.327s
After the change:
real 4m13.083s
user 1m54.062s
sys 1m26.064s
However, a more significnt performance gain is felt during the
extraction/install of sstate cache files, as the fixmepaths file now has a
significantly smaller list of files to modify.
Before the change:
real 0m39.798s
user 0m11.158s
sys 0m12.642s
After the change:
real 0m25.511s
user 0m8.408s
sys 0m5.077s
(All numbers above were recorded with a cold filesystem cache on a machine
with 12 GB of ram.)
(From OE-Core rev: 46067264bedeff8248a2b2441733420fe6651f84)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a + character appears in a version specification within the list of
package dependencies, the version will not be removed from the list in
list_package_depends/recommends leading to garbage appearing in the
dependency graphs generated by buildhistory. To avoid any future
problems due to unusual characters appearing in versions, change the
regex to match almost any character.
Fixes [YOCTO #2451].
(From OE-Core rev: 36d1717e2ad4ca1620ee9f01b524b5ff2f499b26)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As the comment says, using packages[0] is rather broken and can be incorrect,
as demonstrated by the recent gdk-pixbuf change. Replacing it with PN is
a bit more correct and more likely to do what was originally intended
by this function.
(From OE-Core rev: 5d0144fe6bd1febc2d1e651bbcff891c80f20b88)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
One of the patch backends to linux-yocto is guilt, which normally
tracks patches under .git. But .git isn't something that can be
checked into a SCM and repeated. So it has been moved under meta/patches
and committed to the meta branch.
If devshell is used, GUILT_BASE isn't set, so patch manipulations will
fail. We export GUILT_BASE and point it at the meta directory when
devshell is invoked for linux-yocto.
(From OE-Core rev: 10546e42a0db6a9585488536e29759239ef6e73b)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The usage of this function renaming and it being called using
bb.build.exec_func() causes needless indirection loops, confusing log
files and seems generally pointless.
This simplification makes the process much simpler and faster. I can't
come up with a good reason why the export_functions functionality is
needed for this function.
(From OE-Core rev: 74daad03ca29a03b0005f7d2b90a0347d5b583a5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In a pathological case, lots of files to process, the sstate_installpkg
performance was very poor. It interated over each file and ran 3
individual sed commands per file. Changing this to keep iterating
but running only a single command took about 1/3 time time.
However, when looking at the corresponding sstate_hardcode_path
function, it was clear we could optimize this further.
Using the same encoding logic to specify only the minimumal sed
operation necessary, and using xargs to avoid the os.system call the
install step was able to be performed in 13% of the original time.
Example timing numbers for perl:
3m7s original code
1m20s single sed, but interating
0m26s using xargs and limited sed
(From OE-Core rev: d9f655753fbdc8cbd8e705577430fed4f23732b3)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Empty components in $PATH have the same effect as a . in $PATH,
and are a common side-effect of inserting a misspelled or unset
shell variable in $PATH.
(From OE-Core rev: ac4201b714c83c614113bfa735d0a2fc6f64db99)
Signed-off-by: Peter Seebach <peter.seebach@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Revise the handling from ANGSTROM_BLACKLIST to PNBLACKLIST[pn].
Refactor the code to eliminate references to the distribution and recipe
name in the message.
Change the skipPackage message message from:
ERROR: <recipe> was skipped: <distro> DOES NOT support <recipe> because <reason>
to:
ERROR: <recipe> was skipped: Recipe is blacklisted: <reason>
(From OE-Core rev: 0893457465e184550906844fbab2dec397201835)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need to add MLPREFIX when set RDEPENDS by setVar() function.
(From OE-Core rev: 9407895c88e46490d0ef9bb185f73466e7204c15)
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This introduces a sanity check for the toolchain, which verifies
each tuning (including any multilibs), producing meaningful diagnostics
for problems, and also provides some higher-level tuning features.
The TUNEVALID and TUNECONFLICT/TUNECONFLICTS settings were not
implemented. Listed one or two missing features in TUNEVALID,
also (in a previous patch) fixed the references to
features which didn't exist.
This patch also provides a whitelisting mechanism (which is completely
unused) to allow vendors providing prebuilt toolchain components to
restrict tunings to those based on or compatible with a particular ABI.
(From OE-Core rev: 2a91ff0ba0d587c516a5a972553280364853faa4)
Signed-off-by: Peter Seebach <peter.seebach@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to support repositories of various types (with or without
meta data, branched, pristine, custom, etc) information about the
type of processing that is required was passed to the processing
phases via variables.
The combination of variables involved in coordinating the processing
creates a learning curve and overly complicates recipe extensions.
With minor tweaks to the kern-tools, adding flexibility and keying
off the existence of the meta branch it is possible to remove all
of the variables that were added to support different repository
types.
(From OE-Core rev: 06e5f45c8f38925cd5902a3a3f436f5e9451dd16)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some versions of the screen utility provided from the host OS vendor
write the socket directory to $HOME/.screen. When using a shared home
directory across many servers, one sets the SCREENDIR environment
variable to avoid collisions in the shared home directory. This
results in problems launching a devshell where it is not entirely
obvious what happened because the SCREENDIR environment variable
got stripped from the environment prior to setting up the screen
in detached mode.
Example:
% bitbake -c devshell busybox
# ...Please connect in another terminal with "screen -r devshell"
% screen -r devshell
There is no screen to be resumed matching devshell.
The temporary work around was to do something like:
sh -c "unset SCREENDIR; screen -r devshell"
This patch adds SCREENDIR to the white list to ensure screen
works properly on systems where a developer needs to use
the SCREENDIR with shared home directories.
(From OE-Core rev: 5568a8f5a1c65bae021b2e36d735d3153acc6d72)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The echo line here was merging multiple lines into one, and the result
was that if both image and package changes had to be comitted then only
the image changes were being committed and the package changes could
potentially be merged into the next package change. Quoting the variable
reference fixes this.
Fixes [YOCTO #2411]
(From OE-Core rev: 2086bb86885951d0a74342e30ff205c24cb93f0d)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If KERNEL_IMAGETYPE is vmlinux, the expectation is most likely that there will
be no kernel image package, but we still want a vmlinux package for debugging,
so move kernel-vmlinux in front of kernel-image in PACKAGES.
(From OE-Core rev: b62a44a9bf7b268296aeed64469390aa7efc2e73)
Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There were a couple problems with the handling of precompiled locales.
- it gathered the list of locales from the directories - this breaks due to
the naming mismatch, e.g. en_US.UTF-8 vs en_US.utf8.
- it retained its hardcoded assumption that the non-suffixed locale (en_US, as
opposed to en_US.*) is UTF-8, while the others are otherwise. Hardcoding
this is both inflexible and just plain wrong for some toolchains. It's most
common in desktop distros for 'en_US' to be non-utf8, and ''en_US.UTF-8' is
utf8, and this is the case in some external toolchains as well.
The code now uses the SUPPORTED file to hold the knowledge it needs. This file
not only holds the list of locales to generate, but also maps the locale names
to the charsets they correspond to. The code now uses this to assemble its
charset map, falling back to the '.' suffix as charset when the locale is not
in the map. For precompiled, it now uses the locale->charset knowledge it has,
thereby allowing non-utf8 non-suffixed locale names, whereas for
non-precompiled, it reverts to the previous assumption, renaming the utf8
locale and forcibly suffixing the others.
So, a person maintaining an external toolchain recipe is responsible for
ensuring that the SUPPORTED file they provide matches up with the compiled
locales in the toolchain, if they want to utilize precompiled locales.
I believe in the long term the compiled case should do the same thing
precompiled does, and use SUPPORTED or a similar mechanism to encode the
knowledge, and if people want all the non-suffixed names to be utf8, they can
change that file to do so. This would avoid the hardcoded assumption in the
code, as well as consolidating the behavior between the compiled and
precompiled cases.
(From OE-Core rev: 3f36058923ccda25a3dd85046542e65b6034c09e)
Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This change farms the solvedb creation out to a separate script which
handles creation of the index, only if mtime of any of the packages
has changed.
For a core-image-minimal set of rpm's this saves ~20s of a 45s rootfs
build. For core-image-sato it saves 1 minute of a 5 minute rootfs build.
The more packages in the system, the bigger the saving will be.
(From OE-Core rev: 3021136e7b42ab64ca16f30c88467c4b00d51ee0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The existing shell script is a fork bomb and forks off hundreds of
grep/cur/wc calls as it reads from its input stream and iterates over
the file data table for each line of input. This patch replaces the
shell code with python code which doesn't exec anything and hence runs
much faster without the exec() overhead. This speeds up rpm packaging
considerably, as can be measured simply by timing it, or watching the
processor utilisation.
(From OE-Core rev: 2fb5f72b47840ed7f540df181595878e3f5ba4c6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We have currently no override to detect a recipe being build cross, crosssdk
or for target at times we can use virtclass-native and virtclass-nativesdk to
override stuff in recipes but we dont have way to modify a variables
based on recipe type always.
This patch adds in such an override and in particular makes a target override
class available.
With this change now we can say:
EXTRA_OECONF_class-target = "...."
EXTRA_OECONF_class-native = "..."
EXTRA_OECONF_class-nativesdk = "..."
EXTRA_OECONF_class-crosssdk= "..."
Based of an original patch by Khem Raj
(From OE-Core rev: cf332fd9bf685f6d42b11c1f0c37b934c7f5bcbe)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
