This reverts commit e83d8e58a6b107eea87df0ec233a1bc932b2c6e as the conversion
is not correct. Its replacing readlines() calls which generate an array with
what are effectively strings. There are split("\n") calls missing in many
cases so this needs to be reverted until it gets fixed.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, ccache is used if it is present. When building from scratch it gives
no performance improvement and creates a ton of empty directories even when its
not in use.
This change moves ccache support to a bbclass file which the user can choose to
enable. This should make builds more determinstic and make it easier/clearer
to the end user when its being used and when it is not.
(From OE-Core rev: 2acf8da4f13c175ea818b9514677b7059de1e3e2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently we copy the aclocal directory to the build so that autotools
doesn't see .m4 files disappear when its processing them. This can happen
if for example, package X is being rebuilt at the same time as Y and it
gets uninstalled from sstate (assuming there are no dependencies between
X and Y). This code making the copy was added to avoid races but introduces
a race of its own, namely that the files can disappear during the copy.
This patch adds a cp-noerror script which silently ignores such errors
and gives the behaviour we need in this case. It hence fixes issues which
crop up for users and the autobuilder occasionally.
[YOCTO #2485]
(From OE-Core rev: 0f81fbc0df73675aeb79c724858799a3b6a02f85)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to show a friendlier error message within Hob that does not
bury the actual sanity error in our typical preamble about disabling
sanity checks, use a separate event to indicate that sanity checks
failed.
This change is intended to work together with the related change to
BitBake, however it has a check to ensure that it does not fail with
older versions that do not include that change.
Fixes [YOCTO #2336].
(From OE-Core rev: 49d8b7b8c3b18da64583637db207f1f064a5bdb2)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Replace os.popen with subprocess.Popen since the older function would
fail (more or less) silently if the executed program cannot be found
There are both bb.process.run() and bb.process.Popen() which wraps the
subprocess module, use it for simplifying the code.
Note: We don't need the "2>/dev/null" or "2>&1" since bb.process.run()
can handle it, it will raise exception when error occurs, we should
handle the exception ourselves if we want to ignore the error.
More info:
http://docs.python.org/library/subprocess.html#subprocess-replacements
[YOCTO #2454]
(From OE-Core rev: e83d8e58a6b107eea87df0ec233a1bc932b2c6ea)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to better support multilib processing, switch from PN
to BPN.
(From OE-Core rev: 75f70ab534f6dfef961906fb5c252f9e5310d38f)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Refactor in order to:
* Deprecate the old interfaces, but keep them for compatibility
* Provide a new, interface -- capable of working with split packages
* Each update-alternative will now set proper "per-file" provides
Note: this adds a warning message when the older deprecated behavior is
used. The older behavior has been fully tested using oe-core.
(From OE-Core rev: 309117d26de6a87b16406a44a0cefcbaaf7b5d7a)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some image classes such as bootimg save files into ${S} as part of rootfs
generation. For correctness we should therefore clean this at the start of
image generation to ensure reproducibility.
I found this issue when some files I thought should disappear from my rootfs
would not disappear.
(From OE-Core rev: 363424cdc78cafa278008f973c2b4288297ebf8c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When trying to understand why a QA wanring such as:
ERROR: QA Issue: foo rdepends on bar-dev
it is very difficult to figure out where the bar-dev dependency
comes from, since many of them are added dynamically.
This adds a debug statement that says which dependency adds an
rdepends to the system.
Also, while doing this work, it was noted that the same dependencies
were being scanned for over and over. Instead we shorten the list
by only added to the dep list if the dependency was not already there.
(From OE-Core rev: 257b9e371143421b78a6991ef5401e564918c164)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The following functions in meta/classes/utils.bbclass were not
used by anyone, and they didn't work when I tried to use them:
def oe_popen_env(d):
def oe_run(d, cmd, **kwargs):
def oe_popen(d, cmd, **kwargs):
def oe_system(d, cmd, **kwargs):
There error was:
AttributeError: type object 'str' has no attribute 'getVar'
We have bitbake/lib/bb/process.py to instead, so remove them.
[YOCTO #2489]
(From OE-Core rev: 15ad62ab1be060d8a7cdc2d28167ea3af4cfd5d5)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This enables a user to use bitbake -e even when the sanity checks are
failing.
(From OE-Core rev: be317df78535120137ecaadf797e3f4dfe04119e)
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The call to bb.mkdirhier() in check_create_long_filename() can fail with an
OSError, explicitly catch this and report something useful to the user.
(From OE-Core rev: b066906477eb0496a2babb3d8e87682a1b7df0de)
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the user does not have write permissions to SSTATE_CACHE, detected by
the check_create_long_filename() test failing with a "Permission denied"
value in strerror, then suggest they might want to use the location as
an entry in SSTATE_MIRRORS.
(From OE-Core rev: 719d44305508c75a9f2decacdff6558ca14277bb)
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Each failure in the sanity message should be reported on a new line.
(From OE-Core rev: 9362702aa9dd82fed7e6d6e3cb1289521b802075)
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Replace calls to data.getVar(VARIABLE, data_object, expand) to
direct calls to the getVar method the the data_object.
(From OE-Core rev: 41ee978b62a9c40f36f8ad0acef147e36edffa17)
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
At the ConfigParsed event the datastore has yet to be finalised and thus
appends and overrides have not been set.
To ensure the sanity check is being run against the configuration values
the user has set call finalize() on a copy of the datastore and pass that
for all sanity checks.
(From OE-Core rev: 71142172efc0f44a50216550c2b6cc3094fdc21d)
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This has caused problems for several users, including the Yocto Project
autobuilder. Since the message was added in order to be more user friendly
revert the change.
This reverts commit 0c0c4efbf92bcf0f8942f17c18525a4b4ed1798c.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Deleting .pyo files causes them to get compiled on the target.
* First boot gets *really* slow for python based projects.
* No space gets saved on the target.
* The package manager doesn't know about the files and
therefore fails to uninstall them, occupying space and causing
uninstalled python scripts to remain executable.
* It's inconsistent, because python itself and autotools based
projects already ship .pyo files.
* Probably .pyo files were deleted because .pyc files were
available earlier, but this has changed and OE-Core's python
now only generates optimized .pyo files. Deletion of .pyo was
introduced in 2008, python/04-default-is-optimized.patch
was introduced in 2009.
(From OE-Core rev: 25e186ad5b75bd2f93435857580bd16698e18e21)
Signed-off-by: Andreas Oberritter <obi@opendreambox.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix the incremental rpm image generation, it didn't work since the code
has been changed.
The btmanifest should have a ".manifest" suffix, so that it can be moved
to ${T} by rootfs_rpm.bbclass:
mv ${IMAGE_ROOTFS}/install/*.manifest ${T}/
Note: The locale pkgs would always be re-installed.
[YOCTO #2440]
(From OE-Core rev: 5149630746626c6d416f26ab9dd1c7213fcd8c50)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Connect the new fetcher file checksum code so that we get a list of the
files to be checksummed at parse time.
The file-checksums flag will not be read unless we are using a version
of BitBake that supports the function we call within it, so it is safe
to include this change even when the metadata will still be used with
older versions of BitBake.
Implements [YOCTO #2044].
(From OE-Core rev: f26065629b6397d129db930268b72164f8e5d3e4)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
needed for new architecture support
(From OE-Core rev: 75167eb890d8e4683f1e3852e3b77c67d160bea7)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Jason Wessel noticed that a package without any fixmepath entries would
generate a sed warning about no input files. This patch resolves that
by ensuring that an empty fixmepath file never gets written into the
sstate archive. Also we avoid a second message by only doing xargs if
we got input.
(From OE-Core rev: 326563d5a897ae2dba7cfd8d73579d3d979d72c8)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added a new PACKAGEFUNCS function to check the multilib packages'
dependency.
(From OE-Core rev: f193729cdf0f3eccf96a8ce7bf5f599eca89a0d0)
Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added multilib prefix to the locale related package names/dependencies.
(From OE-Core rev: a52f7cf2cc90d918e3250c410995dcc1f3bfd5ee)
Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The user needs read and write permissions to SSTATE_DIR, check
whether they have sufficient permissions and if not recommend
use of SSTATE_MIRRORS.
(From OE-Core rev: 0c0c4efbf92bcf0f8942f17c18525a4b4ed1798c)
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add two default set(SOURCE_ARCHIVE_LOG_WITH_SCRIPTS, \
SOURCE_ARCHIVE_PACKAGE_TYPE) to archiver.bbclass for avoiding \
building error when forgetting to assign to them.
(From OE-Core rev: 5bda8f33b6763fb519543d0f9e35c970a31d39b8)
Signed-off-by: Xiaofeng Yan <xiaofeng.yan@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to make sharing sstate files easier chmod them with 0664 permissions so that they are readable and writable by the user and any other members of their group.
(From OE-Core rev: dae9ad8a0ba0343e3083694cdcb20f0d02927ad0)
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It's common to provide a non-machine-suffixed link in DEPLOY_DIR_IMAGE, so
let's be consistent and do so here as well.
(From OE-Core rev: c1c8d2f3cffc540380c0a5fcdda48d64cbec333a)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you point to snapshot.debian.net/archive/pool then it will fetch
you a html page which will end up in corrupt download. The locations
have changed for archives and here we point the mirror to right
location.
(From OE-Core rev: 15e8aa4cd773022b36a66f70e4f6436289f3d1bc)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The fixmepath file that is generated contains a list of all of the files
that need their paths fixed. In the previous version the fixmepath was
generated to include all of the files that sed may have changed. In the
new version, we first grep the files to see if they contain a path that
needs to be changed, only then do we perform the sed operation on those files.
This results in a modest performance increate in the creation of the sstate
file. The following numbers include the do_package and do_populate_sysroot
tasks on the perl recipe.
Before the change:
real 4m23.018s
user 1m57.067s
sys 1m33.327s
After the change:
real 4m13.083s
user 1m54.062s
sys 1m26.064s
However, a more significnt performance gain is felt during the
extraction/install of sstate cache files, as the fixmepaths file now has a
significantly smaller list of files to modify.
Before the change:
real 0m39.798s
user 0m11.158s
sys 0m12.642s
After the change:
real 0m25.511s
user 0m8.408s
sys 0m5.077s
(All numbers above were recorded with a cold filesystem cache on a machine
with 12 GB of ram.)
(From OE-Core rev: 46067264bedeff8248a2b2441733420fe6651f84)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a + character appears in a version specification within the list of
package dependencies, the version will not be removed from the list in
list_package_depends/recommends leading to garbage appearing in the
dependency graphs generated by buildhistory. To avoid any future
problems due to unusual characters appearing in versions, change the
regex to match almost any character.
Fixes [YOCTO #2451].
(From OE-Core rev: 36d1717e2ad4ca1620ee9f01b524b5ff2f499b26)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As the comment says, using packages[0] is rather broken and can be incorrect,
as demonstrated by the recent gdk-pixbuf change. Replacing it with PN is
a bit more correct and more likely to do what was originally intended
by this function.
(From OE-Core rev: 5d0144fe6bd1febc2d1e651bbcff891c80f20b88)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
One of the patch backends to linux-yocto is guilt, which normally
tracks patches under .git. But .git isn't something that can be
checked into a SCM and repeated. So it has been moved under meta/patches
and committed to the meta branch.
If devshell is used, GUILT_BASE isn't set, so patch manipulations will
fail. We export GUILT_BASE and point it at the meta directory when
devshell is invoked for linux-yocto.
(From OE-Core rev: 10546e42a0db6a9585488536e29759239ef6e73b)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The usage of this function renaming and it being called using
bb.build.exec_func() causes needless indirection loops, confusing log
files and seems generally pointless.
This simplification makes the process much simpler and faster. I can't
come up with a good reason why the export_functions functionality is
needed for this function.
(From OE-Core rev: 74daad03ca29a03b0005f7d2b90a0347d5b583a5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In a pathological case, lots of files to process, the sstate_installpkg
performance was very poor. It interated over each file and ran 3
individual sed commands per file. Changing this to keep iterating
but running only a single command took about 1/3 time time.
However, when looking at the corresponding sstate_hardcode_path
function, it was clear we could optimize this further.
Using the same encoding logic to specify only the minimumal sed
operation necessary, and using xargs to avoid the os.system call the
install step was able to be performed in 13% of the original time.
Example timing numbers for perl:
3m7s original code
1m20s single sed, but interating
0m26s using xargs and limited sed
(From OE-Core rev: d9f655753fbdc8cbd8e705577430fed4f23732b3)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Empty components in $PATH have the same effect as a . in $PATH,
and are a common side-effect of inserting a misspelled or unset
shell variable in $PATH.
(From OE-Core rev: ac4201b714c83c614113bfa735d0a2fc6f64db99)
Signed-off-by: Peter Seebach <peter.seebach@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Revise the handling from ANGSTROM_BLACKLIST to PNBLACKLIST[pn].
Refactor the code to eliminate references to the distribution and recipe
name in the message.
Change the skipPackage message message from:
ERROR: <recipe> was skipped: <distro> DOES NOT support <recipe> because <reason>
to:
ERROR: <recipe> was skipped: Recipe is blacklisted: <reason>
(From OE-Core rev: 0893457465e184550906844fbab2dec397201835)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need to add MLPREFIX when set RDEPENDS by setVar() function.
(From OE-Core rev: 9407895c88e46490d0ef9bb185f73466e7204c15)
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This introduces a sanity check for the toolchain, which verifies
each tuning (including any multilibs), producing meaningful diagnostics
for problems, and also provides some higher-level tuning features.
The TUNEVALID and TUNECONFLICT/TUNECONFLICTS settings were not
implemented. Listed one or two missing features in TUNEVALID,
also (in a previous patch) fixed the references to
features which didn't exist.
This patch also provides a whitelisting mechanism (which is completely
unused) to allow vendors providing prebuilt toolchain components to
restrict tunings to those based on or compatible with a particular ABI.
(From OE-Core rev: 2a91ff0ba0d587c516a5a972553280364853faa4)
Signed-off-by: Peter Seebach <peter.seebach@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to support repositories of various types (with or without
meta data, branched, pristine, custom, etc) information about the
type of processing that is required was passed to the processing
phases via variables.
The combination of variables involved in coordinating the processing
creates a learning curve and overly complicates recipe extensions.
With minor tweaks to the kern-tools, adding flexibility and keying
off the existence of the meta branch it is possible to remove all
of the variables that were added to support different repository
types.
(From OE-Core rev: 06e5f45c8f38925cd5902a3a3f436f5e9451dd16)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some versions of the screen utility provided from the host OS vendor
write the socket directory to $HOME/.screen. When using a shared home
directory across many servers, one sets the SCREENDIR environment
variable to avoid collisions in the shared home directory. This
results in problems launching a devshell where it is not entirely
obvious what happened because the SCREENDIR environment variable
got stripped from the environment prior to setting up the screen
in detached mode.
Example:
% bitbake -c devshell busybox
# ...Please connect in another terminal with "screen -r devshell"
% screen -r devshell
There is no screen to be resumed matching devshell.
The temporary work around was to do something like:
sh -c "unset SCREENDIR; screen -r devshell"
This patch adds SCREENDIR to the white list to ensure screen
works properly on systems where a developer needs to use
the SCREENDIR with shared home directories.
(From OE-Core rev: 5568a8f5a1c65bae021b2e36d735d3153acc6d72)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
