ref-manual: Corrected the "package_rpm.bbclass" section.

A cut-and-paste error had left a "package_deb" string in the
first sentence of the section.  Replaced with "package_rpm."

Reported-by: Geoffroy VanCutsem <geoffroy.vancutsem@intel.com>
(From yocto-docs rev: 8eb1ef532754def5d1d15b98b3b45f259af0edc0)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

README.hardware

@@ -75,14 +75,32 @@ Intel x86 based PCs and devices (genericx86)
 
 The genericx86 MACHINE is tested on the following platforms:
 
-  o Asus EeePC 901
-  o Acer Aspire One
-  o Toshiba NB305
-  o Intel Embedded Development Board 1-N450 (Black Sand)
+Intel Xeon/Core i-Series:
+  + Intel Romley Server: Sandy Bridge Xeon processor, C600 PCH (Patsburg), (Canoe Pass CRB)
+  + Intel Romley Server: Ivy Bridge Xeon processor, C600 PCH (Patsburg), (Intel SDP S2R3)
+  + Intel Crystal Forest Server: Sandy Bridge Xeon processor, DH89xx PCH (Cave Creek), (Stargo CRB)
+  + Intel Chief River Mobile: Ivy Bridge Mobile processor, QM77 PCH (Panther Point-M), (Emerald Lake II CRB, Sabino Canyon CRB)
+  + Intel Huron River Mobile: Sandy Bridge processor, QM67 PCH (Cougar Point), (Emerald Lake CRB, EVOC EC7-1817LNAR board)
+  + Intel Calpella Platform: Core i7 processor, QM57 PCH (Ibex Peak-M), (Red Fort CRB, Emerson MATXM CORE-411-B)
+  + Intel Nehalem/Westmere-EP Server: Xeon 56xx/55xx processors, 5520 chipset, ICH10R IOH (82801), (Hanlan Creek CRB)
+  + Intel Nehalem Workstation: Xeon 56xx/55xx processors, System SC5650SCWS (Greencity CRB)
+  + Intel Picket Post Server: Xeon 56xx/55xx processors (Jasper Forest), 3420 chipset (Ibex Peak), (Osage CRB)
+  + Intel Storage Platform: Sandy Bridge Xeon processor, C600 PCH (Patsburg), (Oak Creek Canyon CRB)
+  + Intel Shark Bay Client Platform: Haswell processor, LynxPoint PCH, (Walnut Canyon CRB, Lava Canyon CRB, Basking Ridge CRB, Flathead Creek CRB)
+  + Intel Shark Bay Ultrabook Platform: Haswell ULT processor, Lynx Point-LP PCH, (WhiteTip Mountain 1 CRB)
 
-and is likely to work on many unlisted Atom based devices. The MACHINE type
-supports ethernet, wifi, sound, and i915 graphics by default in addition to
-common PC input devices, busses, and so on.
+Intel Atom platforms:
+  + Intel embedded Menlow: Intel Atom Z510/530 CPU, System Controller Hub US15W (Portwell NANO-8044)
+  + Intel Luna Pier: Intel Atom N4xx/D5xx series CPU (aka: Pineview-D & -M), 82801HM I/O Hub (ICH8M), (Advantech AIMB-212, Moon Creek CRB)
+  + Intel Queens Bay platform: Intel Atom E6xx CPU (aka: Tunnel Creek), Topcliff EG20T I/O Hub (Emerson NITX-315, Crown Bay CRB, Minnow Board)
+  + Intel Fish River Island platform: Intel Atom E6xx CPU (aka: Tunnel Creek), Topcliff EG20T I/O Hub (Kontron KM2M806)
+  + Intel Cedar Trail platform: Intel Atom N2000 & D2000 series CPU (aka: Cedarview), NM10 Express Chipset (Norco kit BIS-6630, Cedar Rock CRB)
+
+and is likely to work on many unlisted Atom/Core/Xeon based devices. The MACHINE
+type supports ethernet, wifi, sound, and Intel/vesa graphics by default in
+addition to common PC input devices, busses, and so on.  Note that it does not
+included the binary-only graphic drivers used on some Atom platforms, for
+accelerated graphics on these machines please refer to meta-intel.
 
 Depending on the device, it can boot from a traditional hard-disk, a USB device,
 or over the network. Writing generated images to physical media is

bitbake/bin/bitbake

@@ -41,7 +41,7 @@ from bb import ui
 from bb import server
 from bb import cookerdata
 
-__version__ = "1.19.1"
+__version__ = "1.20.0"
 logger = logging.getLogger("BitBake")
 
 # Python multiprocessing requires /dev/shm
@@ -75,18 +75,6 @@ def get_ui(config):
         sys.exit("FATAL: Invalid user interface '%s' specified.\n"
                  "Valid interfaces: depexp, goggle, ncurses, hob, knotty [default]." % interface)
 
-def gather_extra_cache_data():
-    extra = []
-    interfaces = ['depexp', 'goggle', 'ncurses', 'hob', 'knotty']
-    for i in interfaces:
-        try:
-            ui = __import__("bb.ui." + i, fromlist = [i])
-            if hasattr(ui, "extraCaches"):
-                extra = extra + ui.extraCaches
-            del ui
-        except:
-            pass
-    return extra
 
 # Display bitbake/OE warnings via the BitBake.Warnings logger, ignoring others"""
 warnlog = logging.getLogger("BitBake.Warnings")
@@ -112,59 +100,58 @@ class BitBakeConfigParameters(cookerdata.ConfigParameters):
     def parseCommandLine(self):
         parser = optparse.OptionParser(
             version = "BitBake Build Tool Core version %s, %%prog version %s" % (bb.__version__, __version__),
-            usage = """%prog [options] [package ...]
+            usage = """%prog [options] [recipename/target ...]
 
-    Executes the specified task (default is 'build') for a given set of BitBake files.
-    It expects that BBFILES is defined, which is a space separated list of files to
-    be executed.  BBFILES does support wildcards.
-    Default BBFILES are the .bb files in the current directory.""")
+    Executes the specified task (default is 'build') for a given set of target recipes (.bb files).
+    It is assumed there is a conf/bblayers.conf available in cwd or in BBPATH which
+    will provide the layer, BBFILES and other configuration information.""")
 
-        parser.add_option("-b", "--buildfile", help = "execute the task against this .bb file, rather than a package from BBFILES. Does not handle any dependencies.",
+        parser.add_option("-b", "--buildfile", help = "Execute tasks from a specific .bb recipe directly. WARNING: Does not handle any dependencies from other recipes.",
                    action = "store", dest = "buildfile", default = None)
 
-        parser.add_option("-k", "--continue", help = "continue as much as possible after an error. While the target that failed, and those that depend on it, cannot be remade, the other dependencies of these targets can be processed all the same.",
+        parser.add_option("-k", "--continue", help = "Continue as much as possible after an error. While the target that failed and anything depending on it cannot be built, as much as possible will be built before stopping.",
                    action = "store_false", dest = "abort", default = True)
 
-        parser.add_option("-a", "--tryaltconfigs", help = "continue with builds by trying to use alternative providers where possible.",
+        parser.add_option("-a", "--tryaltconfigs", help = "Continue with builds by trying to use alternative providers where possible.",
                    action = "store_true", dest = "tryaltconfigs", default = False)
 
-        parser.add_option("-f", "--force", help = "force run of specified cmd, regardless of stamp status",
+        parser.add_option("-f", "--force", help = "Force the specified targets/task to run (invalidating any existing stamp file).",
                    action = "store_true", dest = "force", default = False)
 
-        parser.add_option("-c", "--cmd", help = "Specify task to execute. Note that this only executes the specified task for the providee and the packages it depends on, i.e. 'compile' does not implicitly call stage for the dependencies (IOW: use only if you know what you are doing). Depending on the base.bbclass a listtasks tasks is defined and will show available tasks",
+        parser.add_option("-c", "--cmd", help = "Specify the task to execute. The exact options available depend on the metadata. Some examples might be 'compile' or 'populate_sysroot' or 'listtasks' may give a list of the tasks available.",
                    action = "store", dest = "cmd")
 
-        parser.add_option("-C", "--clear-stamp", help = "Invalidate the stamp for the specified cmd such as 'compile' and run the default task for the specified target(s)",
+        parser.add_option("-C", "--clear-stamp", help = "Invalidate the stamp for the specified task such as 'compile' and then run the default task for the specified target(s).",
                     action = "store", dest = "invalidate_stamp")
 
-        parser.add_option("-r", "--read", help = "read the specified file before bitbake.conf",
+        parser.add_option("-r", "--read", help = "Read the specified file before bitbake.conf.",
                    action = "append", dest = "prefile", default = [])
 
-        parser.add_option("-R", "--postread", help = "read the specified file after bitbake.conf",
+        parser.add_option("-R", "--postread", help = "Read the specified file after bitbake.conf.",
                           action = "append", dest = "postfile", default = [])
 
-        parser.add_option("-v", "--verbose", help = "output more chit-chat to the terminal",
+        parser.add_option("-v", "--verbose", help = "Output more log message data to the terminal.",
                    action = "store_true", dest = "verbose", default = False)
 
         parser.add_option("-D", "--debug", help = "Increase the debug level. You can specify this more than once.",
                    action = "count", dest="debug", default = 0)
 
-        parser.add_option("-n", "--dry-run", help = "don't execute, just go through the motions",
+        parser.add_option("-n", "--dry-run", help = "Don't execute, just go through the motions.",
                    action = "store_true", dest = "dry_run", default = False)
 
-        parser.add_option("-S", "--dump-signatures", help = "don't execute, just dump out the signature construction information",
+        parser.add_option("-S", "--dump-signatures", help = "Don't execute, just dump out the signature construction information.",
                    action = "store_true", dest = "dump_signatures", default = False)
 
-        parser.add_option("-p", "--parse-only", help = "quit after parsing the BB files (developers only)",
+        parser.add_option("-p", "--parse-only", help = "Quit after parsing the BB recipes.",
                    action = "store_true", dest = "parse_only", default = False)
 
-        parser.add_option("-s", "--show-versions", help = "show current and preferred versions of all recipes",
+        parser.add_option("-s", "--show-versions", help = "Show current and preferred versions of all recipes.",
                    action = "store_true", dest = "show_versions", default = False)
 
-        parser.add_option("-e", "--environment", help = "show the global or per-package environment (this is what used to be bbread)",
+        parser.add_option("-e", "--environment", help = "Show the global or per-package environment complete with information about where variables were set/changed.",
                    action = "store_true", dest = "show_environment", default = False)
 
-        parser.add_option("-g", "--graphviz", help = "emit the dependency trees of the specified packages in the dot syntax, and the pn-buildlist to show the build list",
+        parser.add_option("-g", "--graphviz", help = "Save dependency tree information for the specified targets in the dot syntax.",
                     action = "store_true", dest = "dot_graph", default = False)
 
         parser.add_option("-I", "--ignore-deps", help = """Assume these dependencies don't exist and are already provided (equivalent to ASSUME_PROVIDED). Useful to make dependency graphs more appealing""",
@@ -173,34 +160,34 @@ class BitBakeConfigParameters(cookerdata.ConfigParameters):
         parser.add_option("-l", "--log-domains", help = """Show debug logging for the specified logging domains""",
                     action = "append", dest = "debug_domains", default = [])
 
-        parser.add_option("-P", "--profile", help = "profile the command and print a report",
+        parser.add_option("-P", "--profile", help = "Profile the command and save reports.",
                    action = "store_true", dest = "profile", default = False)
 
-        parser.add_option("-u", "--ui", help = "userinterface to use",
+        parser.add_option("-u", "--ui", help = "The user interface to use (e.g. knotty, hob, depexp).",
                    action = "store", dest = "ui")
 
-        parser.add_option("-t", "--servertype", help = "Choose which server to use, process or xmlrpc",
+        parser.add_option("-t", "--servertype", help = "Choose which server to use, process or xmlrpc.",
                    action = "store", dest = "servertype")
 
-        parser.add_option("", "--revisions-changed", help = "Set the exit code depending on whether upstream floating revisions have changed or not",
+        parser.add_option("", "--revisions-changed", help = "Set the exit code depending on whether upstream floating revisions have changed or not.",
                    action = "store_true", dest = "revisions_changed", default = False)
 
-        parser.add_option("", "--server-only", help = "Run bitbake without UI,  the frontend can connect with bitbake server itself",
+        parser.add_option("", "--server-only", help = "Run bitbake without a UI, only starting a server (cooker) process.",
                    action = "store_true", dest = "server_only", default = False)
 
-        parser.add_option("-B", "--bind", help = "The name/address for the bitbake server to bind to",
+        parser.add_option("-B", "--bind", help = "The name/address for the bitbake server to bind to.",
                    action = "store", dest = "bind", default = False)
 
-        parser.add_option("", "--no-setscene", help = "Do not run any setscene tasks, forces builds",
+        parser.add_option("", "--no-setscene", help = "Do not run any setscene tasks. sstate will be ignored and everything needed, built.",
                    action = "store_true", dest = "nosetscene", default = False)
 
-        parser.add_option("", "--remote-server", help = "Connect to the specified server",
+        parser.add_option("", "--remote-server", help = "Connect to the specified server.",
                    action = "store", dest = "remote_server", default = False)
 
-        parser.add_option("-m", "--kill-server", help = "Terminate the remote server",
+        parser.add_option("-m", "--kill-server", help = "Terminate the remote server.",
                     action = "store_true", dest = "kill_server", default = False)
 
-        parser.add_option("", "--observe-only", help = "Connect to a server as an observing-only client",
+        parser.add_option("", "--observe-only", help = "Connect to a server as an observing-only client.",
                    action = "store_true", dest = "observe_only", default = False)
 
         options, targets = parser.parse_args(sys.argv)
@@ -275,8 +262,8 @@ def main():
         if not configParams.bind:
             sys.exit("FATAL: The '--server-only' option requires a name/address to bind to with the -B option.\n")
         if configParams.remote_server:
-            sys.exit("FATAL: The '--server-only' option conflicts with the '--remote-server' option. %s\n" % 
-                ("Please check your BBSERVER environment" if "BBSERVER" in os.environ else "" ))
+            sys.exit("FATAL: The '--server-only' option conflicts with %s.\n" % 
+                ("the BBSERVER environment variable" if "BBSERVER" in os.environ else "the '--remote-server' option" ))
 
     if configParams.bind and configParams.servertype != "xmlrpc":
         sys.exit("FATAL: If '-B' or '--bind' is defined, we must set the servertype as 'xmlrpc'.\n")
@@ -302,30 +289,28 @@ def main():
     # Clear away any spurious environment variables while we stoke up the cooker
     cleanedvars = bb.utils.clean_environment()
 
-    # Collect all the caches we need
-    if configParams.server_only:
-        configuration.extra_caches = gather_extra_cache_data()
-    else:
-        configuration.extra_caches = getattr(ui_module, "extraCaches", [])
-
     if not configParams.remote_server:
         # we start a server with a given configuration
         server = start_server(servermodule, configParams, configuration)
+        bb.event.ui_queue = []
     else:
-        # we start a stub server that is actually a XMLRPClient to
+        # we start a stub server that is actually a XMLRPClient that connects to a real server
         server = servermodule.BitBakeXMLRPCClient(configParams.observe_only)
         server.saveConnectionDetails(configParams.remote_server)
 
-    logger.removeHandler(handler)
-
     if not configParams.server_only:
+        # Collect the feature set for the UI
+        featureset = getattr(ui_module, "featureSet", [])
+
         # Setup a connection to the server (cooker)
-        server_connection = server.establishConnection()
+        server_connection = server.establishConnection(featureset)
 
         # Restore the environment in case the UI needs it
         for k in cleanedvars:
             os.environ[k] = cleanedvars[k]
 
+        logger.removeHandler(handler)
+
         try:
             return ui_module.main(server_connection.connection, server_connection.events, configParams)
         finally:
@@ -339,6 +324,8 @@ def main():
 if __name__ == "__main__":
     try:
         ret = main()
+    except bb.BBHandledException:
+        ret = 1
     except Exception:
         ret = 1
         import traceback

bitbake/bin/bitbake-diffsigs

@@ -3,7 +3,7 @@
 # bitbake-diffsigs
 # BitBake task signature data comparison utility
 #
-# Copyright (C) 2012 Intel Corporation
+# Copyright (C) 2012-2013 Intel Corporation
 #
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License version 2 as
@@ -30,7 +30,18 @@ sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(sys.argv[0])), '
 import bb.tinfoil
 import bb.siggen
 
-logger = logging.getLogger('BitBake')
+def logger_create(name, output=sys.stderr):
+    logger = logging.getLogger(name)
+    console = logging.StreamHandler(output)
+    format = bb.msg.BBLogFormatter("%(levelname)s: %(message)s")
+    if output.isatty():
+        format.enable_color()
+    console.setFormatter(format)
+    logger.addHandler(console)
+    logger.setLevel(logging.INFO)
+    return logger
+
+logger = logger_create('bitbake-diffsigs')
 
 def find_compare_task(bbhandler, pn, taskname):
     """ Find the most recent signature files for the specified PN/task and compare them """
@@ -39,6 +50,9 @@ def find_compare_task(bbhandler, pn, taskname):
         logger.error('Metadata does not support finding signature data files')
         sys.exit(1)
 
+    if not taskname.startswith('do_'):
+        taskname = 'do_%s' % taskname
+
     filedates = bb.siggen.find_siginfo(pn, taskname, None, bbhandler.config_data)
     latestfiles = sorted(filedates.keys(), key=lambda f: filedates[f])[-2:]
     if not latestfiles:
@@ -71,6 +85,7 @@ def find_compare_task(bbhandler, pn, taskname):
 
 
 parser = optparse.OptionParser(
+    description = "Compares siginfo/sigdata files written out by BitBake",
     usage = """
   %prog -t recipename taskname
   %prog sigdatafile1 sigdatafile2
@@ -78,25 +93,30 @@ parser = optparse.OptionParser(
 
 parser.add_option("-t", "--task",
         help = "find the signature data files for last two runs of the specified task and compare them",
-        action="store_true", dest="taskmode")
+        action="store", dest="taskargs", nargs=2, metavar='recipename taskname')
 
 options, args = parser.parse_args(sys.argv)
 
-if len(args) == 1:
-    parser.print_help()
+if options.taskargs:
+    tinfoil = bb.tinfoil.Tinfoil()
+    tinfoil.prepare(config_only = True)
+    find_compare_task(tinfoil, options.taskargs[0], options.taskargs[1])
 else:
-    if options.taskmode:
-        tinfoil = bb.tinfoil.Tinfoil()
-        if len(args) < 3:
-            logger.error("Please specify a recipe and task name")
-            sys.exit(1)
-        tinfoil.prepare(config_only = True)
-        find_compare_task(tinfoil, args[1], args[2])
+    if len(args) == 1:
+        parser.print_help()
     else:
-        if len(args) == 2:
-            output = bb.siggen.dump_sigfile(sys.argv[1])
-        else:
-            output = bb.siggen.compare_sigfiles(sys.argv[1], sys.argv[2])
+        import cPickle
+        try:
+            if len(args) == 2:
+                output = bb.siggen.dump_sigfile(sys.argv[1])
+            else:
+                output = bb.siggen.compare_sigfiles(sys.argv[1], sys.argv[2])
+        except IOError as e:
+            logger.error(str(e))
+            sys.exit(1)
+        except cPickle.UnpicklingError, EOFError:
+            logger.error('Invalid signature data - ensure you are specifying sigdata/siginfo files')
+            sys.exit(1)
 
-        if output:
-            print '\n'.join(output)
+    if output:
+        print '\n'.join(output)

bitbake/bin/bitbake-dumpsig

@@ -1,11 +1,65 @@
 #!/usr/bin/env python
+
+# bitbake-dumpsig
+# BitBake task signature dump utility
+#
+# Copyright (C) 2013 Intel Corporation
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
 import os
 import sys
 import warnings
+import optparse
+import logging
+
 sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(sys.argv[0])), 'lib'))
 
 import bb.siggen
 
-output = bb.siggen.dump_sigfile(sys.argv[1])
-if output:
-    print '\n'.join(output)
+def logger_create(name, output=sys.stderr):
+    logger = logging.getLogger(name)
+    console = logging.StreamHandler(output)
+    format = bb.msg.BBLogFormatter("%(levelname)s: %(message)s")
+    if output.isatty():
+        format.enable_color()
+    console.setFormatter(format)
+    logger.addHandler(console)
+    logger.setLevel(logging.INFO)
+    return logger
+
+logger = logger_create('bitbake-dumpsig')
+
+parser = optparse.OptionParser(
+    description = "Dumps siginfo/sigdata files written out by BitBake",
+    usage = """
+  %prog sigdatafile""")
+
+options, args = parser.parse_args(sys.argv)
+
+if len(args) == 1:
+    parser.print_help()
+else:
+    import cPickle
+    try:
+        output = bb.siggen.dump_sigfile(args[1])
+    except IOError as e:
+        logger.error(str(e))
+        sys.exit(1)
+    except cPickle.UnpicklingError, EOFError:
+        logger.error('Invalid signature data - ensure you are specifying a sigdata/siginfo file')
+        sys.exit(1)
+
+    if output:
+        print '\n'.join(output)

bitbake/bin/bitbake-worker

@@ -97,7 +97,8 @@ def fork_off_task(cfg, data, workerdata, fn, task, taskname, appends, quieterror
         except TypeError:
              umask = taskdep['umask'][taskname]
 
-    if 'fakeroot' in taskdep and taskname in taskdep['fakeroot']:
+    # We can't use the fakeroot environment in a dry run as it possibly hasn't been built
+    if 'fakeroot' in taskdep and taskname in taskdep['fakeroot'] and not cfg.dry_run:
         envvars = (workerdata["fakerootenv"][fn] or "").split()
         for key, value in (var.split('=') for var in envvars):
             envbackup[key] = os.environ.get(key)
@@ -148,6 +149,8 @@ def fork_off_task(cfg, data, workerdata, fn, task, taskname, appends, quieterror
 
             data.setVar("BB_WORKERCONTEXT", "1")
             data.setVar("BUILDNAME", workerdata["buildname"])
+            data.setVar("DATE", workerdata["date"])
+            data.setVar("TIME", workerdata["time"])
             bb.parse.siggen.set_taskdata(workerdata["hashes"], workerdata["hash_deps"], workerdata["sigchecksums"])
             ret = 0
             try:

bitbake/contrib/vim/ftdetect/bitbake.vim

@@ -14,7 +14,7 @@ endif
 au BufNewFile,BufRead *.{bb,bbappend,bbclass}	set filetype=bitbake
 
 " .inc
-au BufNewFile,BufRead *.inc 		set filetype=bitbake
+au BufNewFile,BufRead *.inc		set filetype=bitbake
 
 " .conf
 au BufNewFile,BufRead *.conf

bitbake/contrib/vim/ftplugin/bitbake.vim

@@ -1 +1,2 @@
 set sts=4 sw=4 et
+set cms=#%s

bitbake/contrib/vim/plugin/newbb.vim

@@ -15,7 +15,7 @@ if &compatible || v:version < 600
 endif
 
 fun! <SID>GetUserName()
-    let l:user_name = system("git-config --get user.name")
+    let l:user_name = system("git config --get user.name")
     if v:shell_error
         return "Unknow User"
     else
@@ -23,7 +23,7 @@ fun! <SID>GetUserName()
 endfun
 
 fun! <SID>GetUserEmail()
-    let l:user_email = system("git-config --get user.email")
+    let l:user_email = system("git config --get user.email")
     if v:shell_error
         return "unknow@user.org"
     else

bitbake/contrib/vim/syntax/bitbake.vim

@@ -44,8 +44,8 @@ syn match bbArrayBrackets       "[\[\]]" contained
 
 " BitBake strings
 syn match bbContinue            "\\$"
-syn region bbString             matchgroup=bbQuote start=+"+ skip=+\\$+ excludenl end=+"+ contained keepend contains=bbTodo,bbContinue,bbVarDeref,bbVarPyValue,@Spell
-syn region bbString             matchgroup=bbQuote start=+'+ skip=+\\$+ excludenl end=+'+ contained keepend contains=bbTodo,bbContinue,bbVarDeref,bbVarPyValue,@Spell
+syn region bbString             matchgroup=bbQuote start=+"+ skip=+\\$+ end=+"+ contained contains=bbTodo,bbContinue,bbVarDeref,bbVarPyValue,@Spell
+syn region bbString             matchgroup=bbQuote start=+'+ skip=+\\$+ end=+'+ contained contains=bbTodo,bbContinue,bbVarDeref,bbVarPyValue,@Spell
 
 " Vars definition
 syn match bbExport            "^export" nextgroup=bbIdentifier skipwhite
@@ -55,11 +55,11 @@ syn match bbVarDeref            "${[a-zA-Z0-9\-_\.\/\+]\+}" contained
 syn match bbVarEq               "\(:=\|+=\|=+\|\.=\|=\.\|?=\|??=\|=\)" contained nextgroup=bbVarValue
 syn match bbVarDef              "^\(export\s*\)\?\([a-zA-Z0-9\-_\.\/\+]\+\(_[${}a-zA-Z0-9\-_\.\/\+]\+\)\?\)\s*\(:=\|+=\|=+\|\.=\|=\.\|?=\|??=\|=\)\@=" contains=bbExportFlag,bbIdentifier,bbVarDeref nextgroup=bbVarEq
 syn match bbVarValue            ".*$" contained contains=bbString,bbVarDeref,bbVarPyValue
-syn region bbVarPyValue         start=+${@+ skip=+\\$+ excludenl end=+}+ contained contains=@python
+syn region bbVarPyValue         start=+${@+ skip=+\\$+ end=+}+ contained contains=@python
 
 " Vars metadata flags
 syn match bbVarFlagDef          "^\([a-zA-Z0-9\-_\.]\+\)\(\[[a-zA-Z0-9\-_\.]\+\]\)\@=" contains=bbIdentifier nextgroup=bbVarFlagFlag
-syn region bbVarFlagFlag        matchgroup=bbArrayBrackets start="\[" end="\]\s*\(=\)\@=" keepend excludenl contained contains=bbIdentifier nextgroup=bbVarEq
+syn region bbVarFlagFlag        matchgroup=bbArrayBrackets start="\[" end="\]\s*\(=\|+=\|=+\|?=\)\@=" contained contains=bbIdentifier nextgroup=bbVarEq
 
 " Includes and requires
 syn keyword bbInclude           inherit include require contained 
@@ -83,13 +83,16 @@ if exists("b:current_syntax")
   unlet b:current_syntax
 endif
 syn keyword bbShFakeRootFlag    fakeroot contained
-syn match bbShFuncDef           "^\(fakeroot\s*\)\?\([0-9A-Za-z_-]\+\)\(python\)\@<!\(\s*()\s*\)\({\)\@=" contains=bbShFakeRootFlag,bbFunction,bbDelimiter nextgroup=bbShFuncRegion skipwhite
-syn region bbShFuncRegion       matchgroup=bbDelimiter start="{\s*$" end="^}\s*$" keepend contained contains=@shell
+syn match bbShFuncDef           "^\(fakeroot\s*\)\?\([0-9A-Za-z_${}-]\+\)\(python\)\@<!\(\s*()\s*\)\({\)\@=" contains=bbShFakeRootFlag,bbFunction,bbVarDeref,bbDelimiter nextgroup=bbShFuncRegion skipwhite
+syn region bbShFuncRegion       matchgroup=bbDelimiter start="{\s*$" end="^}\s*$" contained contains=@shell
+
+" Python value inside shell functions
+syn region shDeref         start=+${@+ skip=+\\$+ excludenl end=+}+ contained contains=@python
 
 " BitBake python metadata
 syn keyword bbPyFlag            python contained
-syn match bbPyFuncDef           "^\(python\s\+\)\([0-9A-Za-z_-]\+\)\?\(\s*()\s*\)\({\)\@=" contains=bbPyFlag,bbFunction,bbDelimiter nextgroup=bbPyFuncRegion skipwhite 
-syn region bbPyFuncRegion       matchgroup=bbDelimiter start="{\s*$" end="^}\s*$" keepend contained contains=@python
+syn match bbPyFuncDef           "^\(python\s\+\)\([0-9A-Za-z_${}-]\+\)\?\(\s*()\s*\)\({\)\@=" contains=bbPyFlag,bbFunction,bbVarDeref,bbDelimiter nextgroup=bbPyFuncRegion skipwhite
+syn region bbPyFuncRegion       matchgroup=bbDelimiter start="{\s*$" end="^}\s*$" contained contains=@python
 
 " BitBake 'def'd python functions
 syn keyword bbPyDef             def contained

bitbake/doc/manual/usermanual.xml

@@ -137,6 +137,24 @@ share common metadata between many packages.</para></listitem>
                 <para>In this example, <varname>B</varname> is now <literal>bvaladditionaldata</literal> and <varname>C</varname> is <literal>testcval</literal>. In contrast to the above appending and prepending operators, no additional space
 will be introduced.</para>
             </section>
+            <section>
+                <title>Appending and Prepending (override style syntax)</title>
+                    <para><screen><varname>B</varname> = "bval"
+<varname>B_append</varname> = " additional data"
+<varname>C</varname> = "cval"
+<varname>C_prepend</varname> = "additional data "</screen></para>
+                 <para>This example results in <varname>B</varname> becoming <literal>bval additional data</literal>
+and <varname>C</varname> becoming <literal>additional data cval</literal>. Note the spaces in the append.
+Unlike the += operator, additional space is not automatically added.  You must take steps to add space
+yourself.</para>
+            </section>
+            <section>
+                <title>Removing (override style syntax)</title>
+                <para><screen><varname>FOO</varname> = "123 456 789 123456 123 456 123 456"
+<varname>FOO_remove</varname> = "123"
+<varname>FOO_remove</varname> = "456"</screen></para>
+                <para>In this example, <varname>FOO</varname> is now <literal>789 123456</literal>.</para>
+            </section>
             <section>
                 <title>Conditional metadata set</title>
                 <para>OVERRIDES is a <quote>:</quote> separated variable containing each item you want to satisfy conditions.  So, if you have a variable which is conditional on <quote>arm</quote>, and <quote>arm</quote> is in OVERRIDES, then the <quote>arm</quote> specific version of the variable is used rather than the non-conditional version.  Example:</para>
@@ -536,7 +554,7 @@ options:
                 <example>
                     <title>Generating dependency graphs</title>
                     <para>BitBake is able to generate dependency graphs using the dot syntax. These graphs can be converted
-to images using the <application>dot</application> application from <ulink url="http://www.graphviz.org">Graphviz</ulink>. 
+to images using the <application>dot</application> application from <ulink url="http://www.graphviz.org">Graphviz</ulink>.
 Two files will be written into the current working directory, <emphasis>depends.dot</emphasis> containing dependency information at the package level and <emphasis>task-depends.dot</emphasis> containing a breakdown of the dependencies at the task level. To stop depending on common depends, one can use the <prompt>-I depend</prompt> to omit these from the graph. This can lead to more readable graphs.  This way, <varname>DEPENDS</varname> from inherited classes such as base.bbclass can be removed from the graph.</para>
                     <screen><prompt>$ </prompt>bitbake -g blah</screen>
                     <screen><prompt>$ </prompt>bitbake -g -I virtual/whatever -I bloom blah</screen>

bitbake/lib/bb/__init__.py

@@ -21,7 +21,7 @@
 # with this program; if not, write to the Free Software Foundation, Inc.,
 # 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
 
-__version__ = "1.19.1"
+__version__ = "1.20.0"
 
 import sys
 if sys.version_info < (2, 7, 3):

bitbake/lib/bb/build.py

@@ -69,9 +69,12 @@ class FuncFailed(Exception):
 class TaskBase(event.Event):
     """Base class for task events"""
 
-    def __init__(self, t, d ):
+    def __init__(self, t, logfile, d):
         self._task = t
         self._package = d.getVar("PF", True)
+        self.taskfile = d.getVar("FILE", True)
+        self.taskname = self._task
+        self.logfile = logfile
         event.Event.__init__(self)
         self._message = "recipe %s: task %s: %s" % (d.getVar("PF", True), t, self.getDisplayName())
 
@@ -96,16 +99,11 @@ class TaskFailed(TaskBase):
     """Task execution failed"""
 
     def __init__(self, task, logfile, metadata, errprinted = False):
-        self.logfile = logfile
         self.errprinted = errprinted
-        super(TaskFailed, self).__init__(task, metadata)
+        super(TaskFailed, self).__init__(task, logfile, metadata)
 
 class TaskFailedSilent(TaskBase):
     """Task execution failed (silently)"""
-    def __init__(self, task, logfile, metadata):
-        self.logfile = logfile
-        super(TaskFailedSilent, self).__init__(task, metadata)
-
     def getDisplayName(self):
         # Don't need to tell the user it was silent
         return "Failed"
@@ -113,7 +111,7 @@ class TaskFailedSilent(TaskBase):
 class TaskInvalid(TaskBase):
 
     def __init__(self, task, metadata):
-        super(TaskInvalid, self).__init__(task, metadata)
+        super(TaskInvalid, self).__init__(task, None, metadata)
         self._message = "No such task '%s'" % task
 
 
@@ -287,7 +285,7 @@ set -e
         if bb.msg.loggerVerboseLogs:
             script.write("set -x\n")
         if cwd:
-            script.write("cd %s\n" % cwd)
+            script.write("cd '%s'\n" % cwd)
         script.write("%s\n" % func)
         script.write('''
 # cleanup
@@ -348,6 +346,14 @@ def _exec_task(fn, task, d, quieterr):
     if not tempdir:
         bb.fatal("T variable not set, unable to build")
 
+    # Change nice level if we're asked to
+    nice = localdata.getVar("BB_TASK_NICE_LEVEL", True)
+    if nice:
+        curnice = os.nice(0)
+        nice = int(nice) - curnice
+        newnice = os.nice(nice)
+        logger.debug(1, "Renice to %s " % newnice)
+
     bb.utils.mkdirhier(tempdir)
 
     # Determine the logfile to generate
@@ -416,7 +422,7 @@ def _exec_task(fn, task, d, quieterr):
     localdata.setVar('BB_LOGFILE', logfn)
     localdata.setVar('BB_RUNTASK', task)
 
-    event.fire(TaskStarted(task, localdata), localdata)
+    event.fire(TaskStarted(task, logfn, localdata), localdata)
     try:
         for func in (prefuncs or '').split():
             exec_func(func, localdata)
@@ -453,7 +459,7 @@ def _exec_task(fn, task, d, quieterr):
             logger.debug(2, "Zero size logfn %s, removing", logfn)
             bb.utils.remove(logfn)
             bb.utils.remove(loglink)
-    event.fire(TaskSucceeded(task, localdata), localdata)
+    event.fire(TaskSucceeded(task, logfn, localdata), localdata)
 
     if not localdata.getVarFlag(task, 'nostamp') and not localdata.getVarFlag(task, 'selfstamp'):
         make_stamp(task, localdata)

bitbake/lib/bb/cache.py

@@ -43,7 +43,7 @@ except ImportError:
     logger.info("Importing cPickle failed. "
                 "Falling back to a very slow implementation.")
 
-__cache_version__ = "146"
+__cache_version__ = "147"
 
 def getCacheFile(path, filename, data_hash):
     return os.path.join(path, filename + "." + data_hash)

bitbake/lib/bb/cache_extra.py

@@ -35,6 +35,12 @@ class HobRecipeInfo(RecipeInfoCommon):
     # such as (bb_cache.dat, bb_extracache_hob.dat) 
     cachefile = "bb_extracache_" + classname +".dat"        
 
+    # override this member with the list of extra cache fields
+    # that this class will provide
+    cachefields = ['summary', 'license', 'section',
+            'description', 'homepage', 'bugtracker',
+            'prevision', 'files_info']
+
     def __init__(self, filename, metadata):
 
         self.summary = self.getvar('SUMMARY', metadata)

bitbake/lib/bb/command.py

@@ -86,6 +86,8 @@ class Command:
 
     def runAsyncCommand(self):
         try:
+            if self.cooker.state == bb.cooker.state.error:
+                return False
             if self.currentAsyncCommand is not None:
                 (command, options) = self.currentAsyncCommand
                 commandmethod = getattr(CommandsAsync, command)
@@ -117,14 +119,14 @@ class Command:
             return False
 
     def finishAsyncCommand(self, msg=None, code=None):
-        if msg:
+        if msg or msg == "":
             bb.event.fire(CommandFailed(msg), self.cooker.event_data)
         elif code:
             bb.event.fire(CommandExit(code), self.cooker.event_data)
         else:
             bb.event.fire(CommandCompleted(), self.cooker.event_data)
         self.currentAsyncCommand = None
-
+        self.cooker.finishcommand()
 
 class CommandsSync:
     """
@@ -137,13 +139,22 @@ class CommandsSync:
         """
         Trigger cooker 'shutdown' mode
         """
-        command.cooker.shutdown()
+        command.cooker.shutdown(False)
 
-    def stateStop(self, command, params):
+    def stateForceShutdown(self, command, params):
         """
         Stop the cooker
         """
-        command.cooker.stop()
+        command.cooker.shutdown(True)
+
+    def getAllKeysWithFlags(self, command, params):
+        """
+        Returns a dump of the global state. Call with
+        variable flags to be retrieved as params.
+        """
+        flaglist = params[0]
+        return command.cooker.getAllKeysWithFlags(flaglist)
+    getAllKeysWithFlags.readonly = True
 
     def getVariable(self, command, params):
         """
@@ -232,6 +243,13 @@ class CommandsSync:
         op = params[3]
         command.cooker.modifyConfigurationVar(var, val, default_file, op)
 
+    def removeVarFile(self, command, params):
+        """
+        Remove a variable declaration from a file
+        """
+        var = params[0]
+        command.cooker.removeConfigurationVar(var)
+
     def createConfigFile(self, command, params):
         """
         Create an extra configuration file

bitbake/lib/bb/cooker.py

@@ -61,7 +61,7 @@ class CollectionError(bb.BBHandledException):
     """
 
 class state:
-    initial, parsing, running, shutdown, stop = range(5)
+    initial, parsing, running, shutdown, forceshutdown, stopped, error = range(7)
 
 
 class SkippedPackage:
@@ -79,6 +79,29 @@ class SkippedPackage:
         elif reason:
             self.skipreason = reason
 
+
+class CookerFeatures(object):
+    _feature_list = [HOB_EXTRA_CACHES, SEND_DEPENDS_TREE] = range(2)
+
+    def __init__(self):
+        self._features=set()
+
+    def setFeature(self, f):
+        # validate we got a request for a feature we support
+        if f not in CookerFeatures._feature_list:
+            return
+        self._features.add(f)
+
+    def __contains__(self, f):
+        return f in self._features
+
+    def __iter__(self):
+        return self._features.__iter__()
+
+    def next(self):
+        return self._features.next()
+
+
 #============================================================================#
 # BBCooker
 #============================================================================#
@@ -90,25 +113,10 @@ class BBCooker:
     def __init__(self, configuration):
         self.recipecache = None
         self.skiplist = {}
+        self.featureset = CookerFeatures()
 
         self.configuration = configuration
 
-        self.caches_array = []
-
-        caches_name_array = ['bb.cache:CoreRecipeInfo'] + configuration.extra_caches
-
-        # At least CoreRecipeInfo will be loaded, so caches_array will never be empty!
-        # This is the entry point, no further check needed!
-        for var in caches_name_array:
-            try:
-                module_name, cache_name = var.split(':')
-                module = __import__(module_name, fromlist=(cache_name,))
-                self.caches_array.append(getattr(module, cache_name)) 
-            except ImportError as exc:
-                logger.critical("Unable to import extra RecipeInfo '%s' from '%s': %s" % (cache_name, module_name, exc))
-                sys.exit("FATAL: Failed to import extra cache class '%s'." % cache_name)
-
-        self.data = None
         self.loadConfigurationData()
 
         # Take a lock so only one copy of bitbake can run against a given build
@@ -118,13 +126,6 @@ class BBCooker:
         if not self.lock:
             bb.fatal("Only one copy of bitbake should be run against a build directory")
 
-        #
-        # Special updated configuration we use for firing events
-        #
-        self.event_data = bb.data.createCopy(self.data)
-        bb.data.update_data(self.event_data)
-        bb.parse.init_parser(self.event_data)
-
         # TOSTOP must not be set or our children will hang when they output
         fd = sys.stdout.fileno()
         if os.isatty(fd):
@@ -141,11 +142,30 @@ class BBCooker:
         self.parser = None
 
     def initConfigurationData(self):
-        worker = False
-        if not self.configuration.server_register_idlecallback:
-            worker = True
 
-        self.databuilder = bb.cookerdata.CookerDataBuilder(self.configuration, worker)
+        self.state = state.initial
+
+        self.caches_array = []
+
+        all_extra_cache_names = []
+        # We hardcode all known cache types in a single place, here.
+        if CookerFeatures.HOB_EXTRA_CACHES in self.featureset:
+            all_extra_cache_names.append("bb.cache_extra:HobRecipeInfo")
+
+        caches_name_array = ['bb.cache:CoreRecipeInfo'] + all_extra_cache_names
+
+        # At least CoreRecipeInfo will be loaded, so caches_array will never be empty!
+        # This is the entry point, no further check needed!
+        for var in caches_name_array:
+            try:
+                module_name, cache_name = var.split(':')
+                module = __import__(module_name, fromlist=(cache_name,))
+                self.caches_array.append(getattr(module, cache_name))
+            except ImportError as exc:
+                logger.critical("Unable to import extra RecipeInfo '%s' from '%s': %s" % (cache_name, module_name, exc))
+                sys.exit("FATAL: Failed to import extra cache class '%s'." % cache_name)
+
+        self.databuilder = bb.cookerdata.CookerDataBuilder(self.configuration, False)
         self.data = self.databuilder.data
 
     def enableDataTracking(self):
@@ -162,11 +182,21 @@ class BBCooker:
         self.data = self.databuilder.data
         self.data_hash = self.databuilder.data_hash
 
+        #
+        # Special updated configuration we use for firing events
+        #
+        self.event_data = bb.data.createCopy(self.data)
+        bb.data.update_data(self.event_data)
+        bb.parse.init_parser(self.event_data)
+
     def modifyConfigurationVar(self, var, val, default_file, op):
         if op == "append":
             self.appendConfigurationVar(var, val, default_file)
         elif op == "set":
-            self.saveConfigurationVar(var, val, default_file)
+            self.saveConfigurationVar(var, val, default_file, "=")
+        elif op == "earlyAssign":
+            self.saveConfigurationVar(var, val, default_file, "?=")
+
 
     def appendConfigurationVar(self, var, val, default_file):
         #add append var operation to the end of default_file
@@ -180,7 +210,7 @@ class BBCooker:
         for c in contents:
             total += c
 
-        total += "#added by bitbake"
+        total += "#added by hob"
         total += "\n%s += \"%s\"\n" % (var, val)
 
         with open(default_file, 'w') as f:
@@ -191,7 +221,7 @@ class BBCooker:
         loginfo = {"op":append, "file":default_file, "line":total.count("\n")}
         self.data.appendVar(var, val, **loginfo)
 
-    def saveConfigurationVar(self, var, val, default_file):
+    def saveConfigurationVar(self, var, val, default_file, op):
 
         replaced = False
         #do not save if nothing changed
@@ -233,8 +263,8 @@ class BBCooker:
                     #check if the variable was saved before in the same way
                     #if true it replace the place where the variable was declared
                     #else it comments it
-                    if contents[begin_line-1]== "#added by bitbake\n":
-                        contents[begin_line] = "%s = \"%s\"\n" % (var, val)
+                    if contents[begin_line-1]== "#added by hob\n":
+                        contents[begin_line] = "%s %s \"%s\"\n" % (var, op, val)
                         replaced = True
                     else:
                         for ii in range(begin_line, end_line):
@@ -263,8 +293,8 @@ class BBCooker:
                 total += c
 
             #add the variable on a single line, to be easy to replace the second time
-            total += "\n#added by bitbake"
-            total += "\n%s = \"%s\"\n" % (var, val)
+            total += "\n#added by hob"
+            total += "\n%s %s \"%s\"\n" % (var, op, val)
 
             with open(default_file, 'w') as f:
                 f.write(total)
@@ -274,6 +304,44 @@ class BBCooker:
             loginfo = {"op":set, "file":default_file, "line":total.count("\n")}
             self.data.setVar(var, val, **loginfo)
 
+    def removeConfigurationVar(self, var):
+        conf_files = self.data.varhistory.get_variable_files(var)
+        topdir = self.data.getVar("TOPDIR")
+
+        for conf_file in conf_files:
+            if topdir in conf_file:
+                with open(conf_file, 'r') as f:
+                    contents = f.readlines()
+                f.close()
+
+                lines = self.data.varhistory.get_variable_lines(var, conf_file)
+                for line in lines:
+                    total = ""
+                    i = 0
+                    for c in contents:
+                        total += c
+                        i = i + 1
+                        if i==int(line):
+                            end_index = len(total)
+                    index = total.rfind(var, 0, end_index)
+
+                    begin_line = total.count("\n",0,index)
+
+                    #check if the variable was saved before in the same way
+                    if contents[begin_line-1]== "#added by hob\n":
+                        contents[begin_line-1] = contents[begin_line] = "\n"
+                    else:
+                        contents[begin_line] = "\n"
+                    #remove var from history
+                    self.data.varhistory.del_var_history(var, conf_file, line)
+
+                total = ""
+                for c in contents:
+                    total += c
+                with open(conf_file, 'w') as f:
+                    f.write(total)
+                f.close()
+
     def createConfigFile(self, name):
         path = os.getcwd()
         confpath = os.path.join(path, "conf", name)
@@ -348,13 +416,7 @@ class BBCooker:
             if pkgs_to_build[0] in set(ignore.split()):
                 bb.fatal("%s is in ASSUME_PROVIDED" % pkgs_to_build[0])
 
-            localdata = data.createCopy(self.data)
-            bb.data.update_data(localdata)
-            bb.data.expandKeys(localdata)
-
-            taskdata = bb.taskdata.TaskData(self.configuration.abort)
-            taskdata.add_provider(localdata, self.recipecache, pkgs_to_build[0])
-            taskdata.add_unresolved(localdata, self.recipecache)
+            taskdata, runlist, pkgs_to_build = self.buildTaskData(pkgs_to_build, None, self.configuration.abort)
 
             targetid = taskdata.getbuild_id(pkgs_to_build[0])
             fnid = taskdata.build_targets[targetid][0]
@@ -386,34 +448,44 @@ class BBCooker:
             if data.getVarFlag( e, 'python', envdata ):
                 logger.plain("\npython %s () {\n%s}\n", e, data.getVar(e, envdata, 1))
 
-    def prepareTreeData(self, pkgs_to_build, task):
+
+    def buildTaskData(self, pkgs_to_build, task, abort):
         """
         Prepare a runqueue and taskdata object for iteration over pkgs_to_build
         """
         bb.event.fire(bb.event.TreeDataPreparationStarted(), self.data)
 
-        # If we are told to do the None task then query the default task
-        if (task == None):
+        # A task of None means use the default task
+        if task is None:
             task = self.configuration.cmd
 
-        pkgs_to_build = self.checkPackages(pkgs_to_build)
+        fulltargetlist = self.checkPackages(pkgs_to_build)
 
         localdata = data.createCopy(self.data)
         bb.data.update_data(localdata)
         bb.data.expandKeys(localdata)
+        taskdata = bb.taskdata.TaskData(abort, skiplist=self.skiplist)
+
+        current = 0
+        runlist = []
+        for k in fulltargetlist:
+            taskdata.add_provider(localdata, self.recipecache, k)
+            current += 1
+            runlist.append([k, "do_%s" % task])
+            bb.event.fire(bb.event.TreeDataPreparationProgress(current, len(fulltargetlist)), self.data)
+        taskdata.add_unresolved(localdata, self.recipecache)
+        bb.event.fire(bb.event.TreeDataPreparationCompleted(len(fulltargetlist)), self.data)
+        return taskdata, runlist, fulltargetlist
+
+    def prepareTreeData(self, pkgs_to_build, task):
+        """
+        Prepare a runqueue and taskdata object for iteration over pkgs_to_build
+        """
+
         # We set abort to False here to prevent unbuildable targets raising
         # an exception when we're just generating data
-        taskdata = bb.taskdata.TaskData(False, skiplist=self.skiplist)
+        taskdata, runlist, pkgs_to_build = self.buildTaskData(pkgs_to_build, task, False)
 
-        runlist = []
-        current = 0
-        for k in pkgs_to_build:
-            taskdata.add_provider(localdata, self.recipecache, k)
-            runlist.append([k, "do_%s" % task])
-            current += 1
-            bb.event.fire(bb.event.TreeDataPreparationProgress(current, len(pkgs_to_build)), self.data)
-        taskdata.add_unresolved(localdata, self.recipecache)
-        bb.event.fire(bb.event.TreeDataPreparationCompleted(len(pkgs_to_build)), self.data)
         return runlist, taskdata
     
     ######## WARNING : this function requires cache_extra to be enabled ########
@@ -426,7 +498,10 @@ class BBCooker:
         runlist, taskdata = self.prepareTreeData(pkgs_to_build, task)
         rq = bb.runqueue.RunQueue(self, self.data, self.recipecache, taskdata, runlist)
         rq.rqdata.prepare()
+        return self.buildDependTree(rq, taskdata)
 
+
+    def buildDependTree(self, rq, taskdata):
         seen_fnids = []
         depend_tree = {}
         depend_tree["depends"] = {}
@@ -447,6 +522,19 @@ class BBCooker:
                 depend_tree["pn"][pn] = {}
                 depend_tree["pn"][pn]["filename"] = fn
                 depend_tree["pn"][pn]["version"] = version
+
+                # if we have extra caches, list all attributes they bring in
+                extra_info = []
+                for cache_class in self.caches_array:
+                    if type(cache_class) is type and issubclass(cache_class, bb.cache.RecipeInfoCommon) and hasattr(cache_class, 'cachefields'):
+                        cachefields = getattr(cache_class, 'cachefields', [])
+                        extra_info = extra_info + cachefields
+
+                # for all attributes stored, add them to the dependency tree
+                for ei in extra_info:
+                    depend_tree["pn"][pn][ei] = vars(self.recipecache)[ei][fn]
+
+
             for dep in rq.rqdata.runq_depends[task]:
                 depfn = taskdata.fn_index[rq.rqdata.runq_fnid[dep]]
                 deppn = self.recipecache.pkg_fn[depfn]
@@ -509,35 +597,30 @@ class BBCooker:
         depend_tree["rdepends-pkg"] = {}
         depend_tree["rrecs-pkg"] = {}
 
+        # if we have extra caches, list all attributes they bring in
+        extra_info = []
+        for cache_class in self.caches_array:
+            if type(cache_class) is type and issubclass(cache_class, bb.cache.RecipeInfoCommon) and hasattr(cache_class, 'cachefields'):
+                cachefields = getattr(cache_class, 'cachefields', [])
+                extra_info = extra_info + cachefields
+
         for task in xrange(len(tasks_fnid)):
             fnid = tasks_fnid[task]
             fn = taskdata.fn_index[fnid]
             pn = self.recipecache.pkg_fn[fn]
-            version  = "%s:%s-%s" % self.recipecache.pkg_pepvpr[fn]
-            summary = self.recipecache.summary[fn]
-            lic = self.recipecache.license[fn]
-            section = self.recipecache.section[fn]
-            description = self.recipecache.description[fn]
-            homepage = self.recipecache.homepage[fn]
-            bugtracker = self.recipecache.bugtracker[fn]
-            files_info = self.recipecache.files_info[fn]
-            rdepends = self.recipecache.rundeps[fn]
-            rrecs = self.recipecache.runrecs[fn]
-            prevision = self.recipecache.prevision[fn]
-            inherits = self.recipecache.inherits.get(fn, None)
+
             if pn not in depend_tree["pn"]:
                 depend_tree["pn"][pn] = {}
                 depend_tree["pn"][pn]["filename"] = fn
+                version  = "%s:%s-%s" % self.recipecache.pkg_pepvpr[fn]
                 depend_tree["pn"][pn]["version"] = version
-                depend_tree["pn"][pn]["summary"] = summary
-                depend_tree["pn"][pn]["license"] = lic
-                depend_tree["pn"][pn]["section"] = section
-                depend_tree["pn"][pn]["description"] = description
-                depend_tree["pn"][pn]["inherits"] = inherits
-                depend_tree["pn"][pn]["homepage"] = homepage
-                depend_tree["pn"][pn]["bugtracker"] = bugtracker
-                depend_tree["pn"][pn]["files_info"] = files_info
-                depend_tree["pn"][pn]["revision"] = prevision
+                rdepends = self.recipecache.rundeps[fn]
+                rrecs = self.recipecache.runrecs[fn]
+                depend_tree["pn"][pn]["inherits"] = self.recipecache.inherits.get(fn, None)
+
+                # for all extra attributes stored, add them to the dependency tree
+                for ei in extra_info:
+                    depend_tree["pn"][pn][ei] = vars(self.recipecache)[ei][fn]
 
             if fnid not in seen_fnids:
                 seen_fnids.append(fnid)
@@ -1047,7 +1130,7 @@ class BBCooker:
 
         def buildFileIdle(server, rq, abort):
 
-            if abort or self.state == state.stop:
+            if abort or self.state == state.forceshutdown:
                 rq.finish_runqueue(True)
             elif self.state == state.shutdown:
                 rq.finish_runqueue(False)
@@ -1076,15 +1159,8 @@ class BBCooker:
         Attempt to build the targets specified
         """
 
-        # If we are told to do the NULL task then query the default task
-        if (task == None):
-            task = self.configuration.cmd
-
-        universe = ('universe' in targets)
-        targets = self.checkPackages(targets)
-
         def buildTargetsIdle(server, rq, abort):
-            if abort or self.state == state.stop:
+            if abort or self.state == state.forceshutdown:
                 rq.finish_runqueue(True)
             elif self.state == state.shutdown:
                 rq.finish_runqueue(False)
@@ -1108,27 +1184,32 @@ class BBCooker:
 
         self.buildSetVars()
 
+        taskdata, runlist, fulltargetlist = self.buildTaskData(targets, task, self.configuration.abort)
+
         buildname = self.data.getVar("BUILDNAME")
-        bb.event.fire(bb.event.BuildStarted(buildname, targets), self.data)
-
-        localdata = data.createCopy(self.data)
-        bb.data.update_data(localdata)
-        bb.data.expandKeys(localdata)
-
-        taskdata = bb.taskdata.TaskData(self.configuration.abort, skiplist=self.skiplist)
-
-        runlist = []
-        for k in targets:
-            taskdata.add_provider(localdata, self.recipecache, k)
-            runlist.append([k, "do_%s" % task])
-        taskdata.add_unresolved(localdata, self.recipecache)
+        bb.event.fire(bb.event.BuildStarted(buildname, fulltargetlist), self.data)
 
         rq = bb.runqueue.RunQueue(self, self.data, self.recipecache, taskdata, runlist)
-        if universe:
+        if 'universe' in targets:
             rq.rqdata.warn_multi_bb = True
 
         self.configuration.server_register_idlecallback(buildTargetsIdle, rq)
 
+
+    def getAllKeysWithFlags(self, flaglist):
+        dump = {}
+        for k in self.data.keys():
+            try:
+                v = self.data.getVar(k, True)
+                if not k.startswith("__") and not isinstance(v, bb.data_smart.DataSmart):
+                    dump[k] = { 'v' : v }
+                    for d in flaglist:
+                        dump[k][d] = self.data.getVarFlag(k, d)
+            except Exception as e:
+                print(e)
+        return dump
+
+
     def generateNewImage(self, image, base_image, package_queue, timestamp, description):
         '''
         Create a new image with a "require"/"inherit" base_image statement
@@ -1173,9 +1254,9 @@ class BBCooker:
         if self.state == state.running:
             return
 
-        if self.state in (state.shutdown, state.stop):
+        if self.state in (state.shutdown, state.forceshutdown):
             self.parser.shutdown(clean=False, force = True)
-            sys.exit(1)
+            raise bb.BBHandledException()
 
         if self.state != state.parsing:
             self.parseConfiguration ()
@@ -1197,7 +1278,7 @@ class BBCooker:
         if not self.parser.parse_next():
             collectlog.debug(1, "parsing complete")
             if self.parser.error:
-                sys.exit(1)
+                raise bb.BBHandledException()
             self.show_appends_with_no_recipes()
             self.handlePrefProviders()
             self.recipecache.bbfile_priority = self.collection.collection_priorities(self.recipecache.pkg_fn)
@@ -1208,6 +1289,9 @@ class BBCooker:
 
     def checkPackages(self, pkgs_to_build):
 
+        # Return a copy, don't modify the original
+        pkgs_to_build = pkgs_to_build[:]
+
         if len(pkgs_to_build) == 0:
             raise NothingToBuild
 
@@ -1237,24 +1321,26 @@ class BBCooker:
             self.prhost = prserv.serv.auto_start(self.data)
         except prserv.serv.PRServiceConfigError:
             bb.event.fire(CookerExit(), self.event_data)
+            self.state = state.error
         return
 
     def post_serve(self):
         prserv.serv.auto_shutdown(self.data)
         bb.event.fire(CookerExit(), self.event_data)
 
-    def shutdown(self):
-        self.state = state.shutdown
+    def shutdown(self, force = False):
+        if force:
+            self.state = state.forceshutdown
+        else:
+            self.state = state.shutdown
 
-    def stop(self):
-        self.state = state.stop
+    def finishcommand(self):
+        self.state = state.initial
 
     def initialize(self):
-        self.state = state.initial
         self.initConfigurationData()
 
     def reset(self):
-        self.state = state.initial
         self.loadConfigurationData()
 
 def server_main(cooker, func, *args):
@@ -1482,7 +1568,7 @@ class Feeder(multiprocessing.Process):
                 continue
 
 class Parser(multiprocessing.Process):
-    def __init__(self, jobs, results, quit, init):
+    def __init__(self, jobs, results, quit, init, profile):
         self.jobs = jobs
         self.results = results
         self.quit = quit
@@ -1490,8 +1576,28 @@ class Parser(multiprocessing.Process):
         multiprocessing.Process.__init__(self)
         self.context = bb.utils.get_context().copy()
         self.handlers = bb.event.get_class_handlers().copy()
+        self.profile = profile
 
     def run(self):
+
+        if not self.profile:
+            self.realrun()
+            return
+
+        try:
+            import cProfile as profile
+        except:
+            import profile
+        prof = profile.Profile()
+        try:
+            profile.Profile.runcall(prof, self.realrun)
+        finally:
+            logfile = "profile-parse-%s.log" % multiprocessing.current_process().name
+            prof.dump_stats(logfile)
+            bb.utils.process_profilelog(logfile)
+            print("Raw profiling information saved to %s and processed statistics to %s.processed" % (logfile, logfile))
+
+    def realrun(self):
         if self.init:
             self.init()
 
@@ -1592,7 +1698,7 @@ class CookerParser(object):
             self.feeder = Feeder(self.willparse, self.jobs, self.feeder_quit)
             self.feeder.start()
             for i in range(0, self.num_processes):
-                parser = Parser(self.jobs, self.result_queue, self.parser_quit, init)
+                parser = Parser(self.jobs, self.result_queue, self.parser_quit, init, self.cooker.configuration.profile)
                 parser.start()
                 self.processes.append(parser)
 

bitbake/lib/bb/cookerdata.py

@@ -127,7 +127,6 @@ class CookerConfiguration(object):
         self.dump_signatures = False
         self.dry_run = False
         self.tracking = False
-        self.extra_caches = []
 
         self.env = {}
 

bitbake/lib/bb/data.py

@@ -59,7 +59,7 @@ def init():
 def init_db(parent = None):
     """Return a new object representing the Bitbake data,
     optionally based on an existing object"""
-    if parent:
+    if parent is not None:
         return parent.createCopy()
     else:
         return _dict_type()
@@ -148,7 +148,7 @@ def expandKeys(alterdata, readdata = None):
         readdata = alterdata
 
     todolist = {}
-    for key in keys(alterdata):
+    for key in alterdata:
         if not '${' in key:
             continue
 
@@ -214,7 +214,7 @@ def emit_var(var, o=sys.__stdout__, d = init(), all=False):
         o.write('unset %s\n' % varExpanded)
         return 0
 
-    if not val:
+    if val is None:
         return 0
 
     val = str(val)
@@ -229,7 +229,7 @@ def emit_var(var, o=sys.__stdout__, d = init(), all=False):
 
     # if we're going to output this within doublequotes,
     # to a shell, we need to escape the quotes in the var
-    alter = re.sub('"', '\\"', val.strip())
+    alter = re.sub('"', '\\"', val)
     alter = re.sub('\n', ' \\\n', alter)
     o.write('%s="%s"\n' % (varExpanded, alter))
     return 0
@@ -285,20 +285,24 @@ def update_data(d):
     """Performs final steps upon the datastore, including application of overrides"""
     d.finalize(parent = True)
 
-def build_dependencies(key, keys, shelldeps, vardepvals, d):
+def build_dependencies(key, keys, shelldeps, varflagsexcl, d):
     deps = set()
-    vardeps = d.getVarFlag(key, "vardeps", True)
     try:
         if key[-1] == ']':
             vf = key[:-1].split('[')
             value = d.getVarFlag(vf[0], vf[1], False)
-        else:
-            value = d.getVar(key, False)
+            parser = d.expandWithRefs(value, key)
+            deps |= parser.references
+            deps = deps | (keys & parser.execs)
+            return deps, value
+        varflags = d.getVarFlags(key, ["vardeps", "vardepvalue", "vardepsexclude"]) or {}
+        vardeps = varflags.get("vardeps")
+        value = d.getVar(key, False)
 
-        if key in vardepvals:
-           value =  d.getVarFlag(key, "vardepvalue", True)
-        elif d.getVarFlag(key, "func"):
-            if d.getVarFlag(key, "python"):
+        if "vardepvalue" in varflags:
+           value = varflags.get("vardepvalue")
+        elif varflags.get("func"):
+            if varflags.get("python"):
                 parsedvar = d.expandWithRefs(value, key)
                 parser = bb.codeparser.PythonParser(key, logger)
                 if parsedvar.value and "\t" in parsedvar.value:
@@ -320,19 +324,16 @@ def build_dependencies(key, keys, shelldeps, vardepvals, d):
             deps = deps | (keys & parser.execs)
 
         # Add varflags, assuming an exclusion list is set
-        varflagsexcl = d.getVar('BB_SIGNATURE_EXCLUDE_FLAGS', True)
         if varflagsexcl:
             varfdeps = []
-            varflags = d.getVarFlags(key)
-            if varflags:
-                for f in varflags:
-                    if f not in varflagsexcl:
-                        varfdeps.append('%s[%s]' % (key, f))
+            for f in varflags:
+                if f not in varflagsexcl:
+                    varfdeps.append('%s[%s]' % (key, f))
             if varfdeps:
                 deps |= set(varfdeps)
 
         deps |= set((vardeps or "").split())
-        deps -= set((d.getVarFlag(key, "vardepsexclude", True) or "").split())
+        deps -= set(varflags.get("vardepsexclude", "").split())
     except Exception as e:
         raise bb.data_smart.ExpansionError(key, None, e)
     return deps, value
@@ -341,16 +342,16 @@ def build_dependencies(key, keys, shelldeps, vardepvals, d):
 
 def generate_dependencies(d):
 
-    keys = set(key for key in d.keys() if not key.startswith("__"))
-    shelldeps = set(key for key in keys if d.getVarFlag(key, "export") and not d.getVarFlag(key, "unexport"))
-    vardepvals = set(key for key in keys if d.getVarFlag(key, "vardepvalue"))
+    keys = set(key for key in d if not key.startswith("__"))
+    shelldeps = set(key for key in d.getVar("__exportlist", False) if d.getVarFlag(key, "export") and not d.getVarFlag(key, "unexport"))
+    varflagsexcl = d.getVar('BB_SIGNATURE_EXCLUDE_FLAGS', True)
 
     deps = {}
     values = {}
 
     tasklist = d.getVar('__BBTASKS') or []
     for task in tasklist:
-        deps[task], values[task] = build_dependencies(task, keys, shelldeps, vardepvals, d)
+        deps[task], values[task] = build_dependencies(task, keys, shelldeps, varflagsexcl, d)
         newdeps = deps[task]
         seen = set()
         while newdeps:
@@ -359,7 +360,7 @@ def generate_dependencies(d):
             newdeps = set()
             for dep in nextdeps:
                 if dep not in deps:
-                    deps[dep], values[dep] = build_dependencies(dep, keys, shelldeps, vardepvals, d)
+                    deps[dep], values[dep] = build_dependencies(dep, keys, shelldeps, varflagsexcl, d)
                 newdeps |=  deps[dep]
             newdeps -= seen
         #print "For %s: %s" % (task, str(deps[task]))

bitbake/lib/bb/data_smart.py

@@ -40,7 +40,7 @@ logger = logging.getLogger("BitBake.Data")
 
 __setvar_keyword__ = ["_append", "_prepend", "_remove"]
 __setvar_regexp__ = re.compile('(?P<base>.*?)(?P<keyword>_append|_prepend|_remove)(_(?P<add>.*))?$')
-__expand_var_regexp__ = re.compile(r"\${[^{}]+}")
+__expand_var_regexp__ = re.compile(r"\${[^{}@\n\t ]+}")
 __expand_python_regexp__ = re.compile(r"\${@.+?}")
 
 def infer_caller_details(loginfo, parent = False, varval = True):
@@ -94,9 +94,13 @@ class VariableParse:
             if self.varname and key:
                 if self.varname == key:
                     raise Exception("variable %s references itself!" % self.varname)
-            var = self.d.getVar(key, True)
+            if key in self.d.expand_cache:
+                varparse = self.d.expand_cache[key]
+                var = varparse.value
+            else:
+                var = self.d.getVar(key, True)
+            self.references.add(key)
             if var is not None:
-                self.references.add(key)
                 return var
             else:
                 return match.group()
@@ -277,9 +281,13 @@ class VariableHistory(object):
                 lines.append(line)
         return lines
 
-    def del_var_history(self, var):
+    def del_var_history(self, var, f=None, line=None):
+        """If file f and line are not given, the entire history of var is deleted"""
         if var in self.variables:
-            self.variables[var] = []
+            if f and line:
+                self.variables[var] = [ x for x in self.variables[var] if x['file']!=f and x['line']!=line]
+            else:
+                self.variables[var] = []
 
 class DataSmart(MutableMapping):
     def __init__(self, special = COWDictBase.copy(), seen = COWDictBase.copy() ):
@@ -505,12 +513,7 @@ class DataSmart(MutableMapping):
             self._seen_overrides[override].add( var )
 
     def getVar(self, var, expand=False, noweakdefault=False):
-        value = self.getVarFlag(var, "_content", False, noweakdefault)
-
-        # Call expand() separately to make use of the expand cache
-        if expand and value:
-            return self.expand(value, var)
-        return value
+        return self.getVarFlag(var, "_content", expand, noweakdefault)
 
     def renameVar(self, key, newkey, **loginfo):
         """
@@ -578,17 +581,30 @@ class DataSmart(MutableMapping):
         if flag == "defaultval" and '_' in var:
             self._setvar_update_overrides(var)
 
+        if flag == "unexport" or flag == "export":
+            if not "__exportlist" in self.dict:
+                self._makeShadowCopy("__exportlist")
+            if not "_content" in self.dict["__exportlist"]:
+                self.dict["__exportlist"]["_content"] = set()
+            self.dict["__exportlist"]["_content"].add(var)
+
     def getVarFlag(self, var, flag, expand=False, noweakdefault=False):
         local_var = self._findVar(var)
         value = None
-        if local_var:
+        if local_var is not None:
             if flag in local_var:
                 value = copy.copy(local_var[flag])
             elif flag == "_content" and "defaultval" in local_var and not noweakdefault:
                 value = copy.copy(local_var["defaultval"])
         if expand and value:
-            value = self.expand(value, None)
-        if value and flag == "_content" and local_var and "_removeactive" in local_var:
+            # Only getvar (flag == _content) hits the expand cache
+            cachename = None
+            if flag == "_content":
+                cachename = var
+            else:
+                cachename = var + "[" + flag + "]"
+            value = self.expand(value, cachename)
+        if value is not None and flag == "_content" and local_var is not None and "_removeactive" in local_var:
             filtered = filter(lambda v: v not in local_var["_removeactive"],
                               value.split(" "))
             value = " ".join(filtered)
@@ -636,16 +652,17 @@ class DataSmart(MutableMapping):
             self.varhistory.record(**loginfo)
             self.dict[var][i] = flags[i]
 
-    def getVarFlags(self, var):
+    def getVarFlags(self, var, expand = False, internalflags=False):
         local_var = self._findVar(var)
         flags = {}
 
         if local_var:
             for i in local_var:
-                if i.startswith("_"):
+                if i.startswith("_") and not internalflags:
                     continue
                 flags[i] = local_var[i]
-
+                if expand and i in expand:
+                    flags[i] = self.expand(flags[i], var + "[" + i + "]")
         if len(flags) == 0:
             return None
         return flags
@@ -751,13 +768,16 @@ class DataSmart(MutableMapping):
         for key in keys:
             if key in config_whitelist:
                 continue
+
             value = d.getVar(key, False) or ""
             data.update({key:value})
 
-            varflags = d.getVarFlags(key)
+            varflags = d.getVarFlags(key, internalflags = True)
             if not varflags:
                 continue
             for f in varflags:
+                if f == "_content":
+                    continue
                 data.update({'%s[%s]' % (key, f):varflags[f]})
 
         for key in ["__BBTASKS", "__BBANONFUNCS", "__BBHANDLERS"]:

bitbake/lib/bb/event.py

@@ -589,6 +589,16 @@ class PackageInfo(Event):
         Event.__init__(self)
         self._pkginfolist = pkginfolist
 
+class MetadataEvent(Event):
+    """
+    Generic event that target for OE-Core classes
+    to report information during asynchrous execution
+    """
+    def __init__(self, eventtype, eventdata):
+        Event.__init__(self)
+        self.type = eventtype
+        self.data = eventdata
+
 class SanityCheck(Event):
     """
     Event to issue sanity check

bitbake/lib/bb/fetch2/__init__.py

@@ -329,7 +329,7 @@ def decodeurl(url):
     user, password, parameters).
     """
 
-    m = re.compile('(?P<type>[^:]*)://((?P<user>.+)@)?(?P<location>[^;]+)(;(?P<parm>.*))?').match(url)
+    m = re.compile('(?P<type>[^:]*)://((?P<user>[^/]+)@)?(?P<location>[^;]+)(;(?P<parm>.*))?').match(url)
     if not m:
         raise MalformedUrl(url)
 
@@ -805,7 +805,11 @@ def try_mirror_url(newuri, origud, ud, ld, check = False):
             dest = os.path.join(dldir, os.path.basename(ud.localpath))
             if not os.path.exists(dest):
                 os.symlink(ud.localpath, dest)
-            return None
+            if not os.path.exists(origud.donestamp) or origud.method.need_update(origud.url, origud, ld):
+                origud.method.download(origud.url, origud, ld)
+                if hasattr(origud.method,"build_mirror_data"):
+                    origud.method.build_mirror_data(origud.url, origud, ld)
+            return ud.localpath
         # Otherwise the result is a local file:// and we symlink to it
         if not os.path.exists(origud.localpath):
             if os.path.islink(origud.localpath):

bitbake/lib/bb/fetch2/git.py

@@ -305,8 +305,8 @@ class Git(FetchMethod):
             username = ""
 
         basecmd = data.getVar("FETCHCMD_git", d, True) or "git"
-        cmd = "%s ls-remote %s://%s%s%s %s" % \
-              (basecmd, ud.proto, username, ud.host, ud.path, ud.branches[name])
+        cmd = "%s ls-remote %s://%s%s%s refs/heads/%s refs/tags/%s" % \
+              (basecmd, ud.proto, username, ud.host, ud.path, ud.branches[name], ud.branches[name])
         if ud.proto.lower() != 'file':
             bb.fetch2.check_network_access(d, cmd)
         output = runfetchcmd(cmd, d, True)

bitbake/lib/bb/fetch2/hg.py

@@ -92,7 +92,10 @@ class Hg(FetchMethod):
         if not ud.user:
             hgroot = host + ud.path
         else:
-            hgroot = ud.user + "@" + host + ud.path
+            if ud.pswd:
+                hgroot = ud.user + ":" + ud.pswd + "@" + host + ud.path
+            else:
+                hgroot = ud.user + "@" + host + ud.path
 
         if command == "info":
             return "%s identify -i %s://%s/%s" % (basecmd, proto, hgroot, ud.module)
@@ -112,7 +115,10 @@ class Hg(FetchMethod):
             # do not pass options list; limiting pull to rev causes the local
             # repo not to contain it and immediately following "update" command
             # will crash
-            cmd = "%s pull" % (basecmd)
+            if ud.user and ud.pswd:
+                cmd = "%s --config auth.default.prefix=* --config auth.default.username=%s --config auth.default.password=%s --config \"auth.default.schemes=%s\" pull" % (basecmd, ud.user, ud.pswd, proto)
+            else:
+                cmd = "%s pull" % (basecmd)
         elif command == "update":
             cmd = "%s update -C %s" % (basecmd, " ".join(options))
         else:

bitbake/lib/bb/fetch2/perforce.py

@@ -112,7 +112,7 @@ class Perforce(FetchMethod):
         base = path
         which = path.find('/...')
         if which != -1:
-            base = path[:which]
+            base = path[:which-1]
 
         base = self._strip_leading_slashes(base)
 

bitbake/lib/bb/fetch2/svn.py

@@ -27,6 +27,7 @@ import os
 import sys
 import logging
 import bb
+import re
 from   bb import data
 from   bb.fetch2 import FetchMethod
 from   bb.fetch2 import FetchError
@@ -91,6 +92,8 @@ class Svn(FetchMethod):
 
         if command == "info":
             svncmd = "%s info %s %s://%s/%s/" % (ud.basecmd, " ".join(options), proto, svnroot, ud.module)
+        elif command == "log1":
+            svncmd = "%s log --limit 1 %s %s://%s/%s/" % (ud.basecmd, " ".join(options), proto, svnroot, ud.module)
         else:
             suffix = ""
             if ud.revision:
@@ -167,14 +170,13 @@ class Svn(FetchMethod):
         """
         Return the latest upstream revision number
         """
-        bb.fetch2.check_network_access(d, self._buildsvncommand(ud, d, "info"))
+        bb.fetch2.check_network_access(d, self._buildsvncommand(ud, d, "log1"))
 
-        output = runfetchcmd("LANG=C LC_ALL=C " + self._buildsvncommand(ud, d, "info"), d, True)
+        output = runfetchcmd("LANG=C LC_ALL=C " + self._buildsvncommand(ud, d, "log1"), d, True)
 
-        revision = None
-        for line in output.splitlines():
-            if "Last Changed Rev" in line:
-                revision = line.split(":")[1].strip()
+        # skip the first line, as per output of svn log
+        # then we expect the revision on the 2nd line
+        revision = re.search('^r([0-9]*)', output.splitlines()[1]).group(1)
 
         return revision
 

bitbake/lib/bb/monitordisk.py

@@ -225,7 +225,7 @@ class diskMonitor:
                         self.preFreeS[k] = freeSpace
 
                     if action == "STOPTASKS" and not self.checked[k]:
-                        logger.error("No new tasks can be excuted since the disk space monitor action is \"STOPTASKS\"!")
+                        logger.error("No new tasks can be executed since the disk space monitor action is \"STOPTASKS\"!")
                         self.checked[k] = True
                         rq.finish_runqueue(False)
                         bb.event.fire(bb.event.DiskFull(dev, 'disk', freeSpace, path), self.configuration)
@@ -243,7 +243,7 @@ class diskMonitor:
                     # zero, this is a feature of the fs, we disable the inode
                     # checking for such a fs.
                     if st.f_files == 0:
-                        logger.warn("Inode check for %s is unavaliable, will remove it from disk monitor" % path)
+                        logger.info("Inode check for %s is unavaliable, will remove it from disk monitor" % path)
                         self.devDict[k][2] = None
                         continue
                     # Always show warning, the self.checked would always be False if the action is WARN
@@ -253,7 +253,7 @@ class diskMonitor:
                         self.preFreeI[k] = freeInode
 
                     if action  == "STOPTASKS" and not self.checked[k]:
-                        logger.error("No new tasks can be excuted since the disk space monitor action is \"STOPTASKS\"!")
+                        logger.error("No new tasks can be executed since the disk space monitor action is \"STOPTASKS\"!")
                         self.checked[k] = True
                         rq.finish_runqueue(False)
                         bb.event.fire(bb.event.DiskFull(dev, 'inode', freeInode, path), self.configuration)

bitbake/lib/bb/parse/__init__.py

@@ -73,9 +73,17 @@ def update_mtime(f):
 def mark_dependency(d, f):
     if f.startswith('./'):
         f = "%s/%s" % (os.getcwd(), f[2:])
-    deps = (d.getVar('__depends') or []) + [(f, cached_mtime(f))]
-    d.setVar('__depends', deps)
+    deps = (d.getVar('__depends') or [])
+    s = (f, cached_mtime_noerror(f))
+    if s not in deps:
+        deps.append(s)
+        d.setVar('__depends', deps)
 
+def check_dependency(d, f):
+    s = (f, cached_mtime_noerror(f))
+    deps = (d.getVar('__depends') or [])
+    return s in deps
+   
 def supports(fn, data):
     """Returns true if we have a handler for this file, false otherwise"""
     for h in handlers:
@@ -102,11 +110,14 @@ def init_parser(d):
 def resolve_file(fn, d):
     if not os.path.isabs(fn):
         bbpath = d.getVar("BBPATH", True)
-        newfn = bb.utils.which(bbpath, fn)
+        newfn, attempts = bb.utils.which(bbpath, fn, history=True)
+        for af in attempts:
+            mark_dependency(d, af)
         if not newfn:
             raise IOError("file %s not found in %s" % (fn, bbpath))
         fn = newfn
 
+    mark_dependency(d, fn)
     if not os.path.isfile(fn):
         raise IOError("file %s not found" % fn)
 

bitbake/lib/bb/parse/parse_py/BBHandler.py

@@ -77,7 +77,10 @@ def inherit(files, fn, lineno, d):
         if not os.path.isabs(file):
             dname = os.path.dirname(fn)
             bbpath = "%s:%s" % (dname, d.getVar("BBPATH", True))
-            abs_fn = bb.utils.which(bbpath, file)
+            abs_fn, attempts = bb.utils.which(bbpath, file, history=True)
+            for af in attempts:
+                if af != abs_fn:
+                    bb.parse.mark_dependency(d, af)
             if abs_fn:
                 file = abs_fn
 

bitbake/lib/bb/parse/parse_py/ConfHandler.py

@@ -82,9 +82,15 @@ def include(oldfn, fn, lineno, data, error_out):
     if not os.path.isabs(fn):
         dname = os.path.dirname(oldfn)
         bbpath = "%s:%s" % (dname, data.getVar("BBPATH", True))
-        abs_fn = bb.utils.which(bbpath, fn)
+        abs_fn, attempts = bb.utils.which(bbpath, fn, history=True)
+        if abs_fn and bb.parse.check_dependency(data, abs_fn):
+            bb.warn("Duplicate inclusion for %s in %s" % (abs_fn, data.getVar('FILE', True)))
+        for af in attempts:
+            bb.parse.mark_dependency(data, af)
         if abs_fn:
             fn = abs_fn
+    elif bb.parse.check_dependency(data, fn):
+        bb.warn("Duplicate inclusion for %s in %s" % (fn, data.getVar('FILE', True)))
 
     from bb.parse import handle
     try:
@@ -93,6 +99,7 @@ def include(oldfn, fn, lineno, data, error_out):
         if error_out:
             raise ParseError("Could not %(error_out)s file %(fn)s" % vars(), oldfn, lineno)
         logger.debug(2, "CONF file '%s' not found", fn)
+        bb.parse.mark_dependency(data, fn)
 
 # We have an issue where a UI might want to enforce particular settings such as
 # an empty DISTRO variable. If configuration files do something like assigning

bitbake/lib/bb/providers.py

@@ -341,7 +341,7 @@ def filterProvidersRunTime(providers, item, cfgData, dataCache):
     if numberPreferred > 1:
         logger.error("Trying to resolve runtime dependency %s resulted in conflicting PREFERRED_PROVIDER entries being found.\nThe providers found were: %s\nThe PREFERRED_PROVIDER entries resulting in this conflict were: %s", item, preferred, preferred_vars)
 
-    logger.debug(1, "sorted providers for %s are: %s", item, eligible)
+    logger.debug(1, "sorted runtime providers for %s are: %s", item, eligible)
 
     return eligible, numberPreferred
 

bitbake/lib/bb/runqueue.py

@@ -217,6 +217,15 @@ class RunQueueData:
             ret.extend([nam])
         return ret
 
+    def get_task_name(self, task):
+        return self.runq_task[task]
+
+    def get_task_file(self, task):
+        return self.taskData.fn_index[self.runq_fnid[task]]
+
+    def get_task_hash(self, task):
+        return self.runq_hash[task]
+
     def get_user_idstring(self, task, task_name_suffix = ""):
         fn = self.taskData.fn_index[self.runq_fnid[task]]
         taskname = self.runq_task[task] + task_name_suffix
@@ -696,6 +705,14 @@ class RunQueueData:
                     prov_list[prov].append(fn)
         for prov in prov_list:
             if len(prov_list[prov]) > 1 and prov not in self.multi_provider_whitelist:
+                seen_pn = []
+                # If two versions of the same PN are being built its fatal, we don't support it.
+                for fn in prov_list[prov]:
+                    pn = self.dataCache.pkg_fn[fn]
+                    if pn not in seen_pn:
+                        seen_pn.append(pn)
+                    else:
+                        bb.fatal("Multiple versions of %s are due to be built (%s). Only one version of a given PN should be built in any given build. You likely need to set PREFERRED_VERSION_%s to select the correct version or don't depend on multiple versions." % (pn, " ".join(prov_list[prov]), pn))
                 msg = "Multiple .bb files are due to be built which each provide %s (%s)." % (prov, " ".join(prov_list[prov]))
                 if self.warn_multi_bb:
                     logger.warn(msg)
@@ -703,7 +720,6 @@ class RunQueueData:
                     msg += "\n This usually means one provides something the other doesn't and should."
                     logger.error(msg)
 
-
         # Create a whitelist usable by the stamp checks
         stampfnwhitelist = []
         for entry in self.stampwhitelist.split():
@@ -853,6 +869,8 @@ class RunQueue:
             "logdefaultdomain" : bb.msg.loggerDefaultDomains,
             "prhost" : self.cooker.prhost,
             "buildname" : self.cfgData.getVar("BUILDNAME", True),
+            "date" : self.cfgData.getVar("DATE", True),
+            "time" : self.cfgData.getVar("TIME", True),
         }
 
         worker.stdin.write("<cookerconfig>" + pickle.dumps(self.cooker.configuration) + "</cookerconfig>")
@@ -990,6 +1008,11 @@ class RunQueue:
             else:
                 self.state = runQueueSceneInit
 
+                # we are ready to run, see if any UI client needs the dependency info
+                if bb.cooker.CookerFeatures.SEND_DEPENDS_TREE in self.cooker.featureset:
+                    depgraph = self.cooker.buildDependTree(self, self.rqdata.taskData)
+                    bb.event.fire(bb.event.DepTreeGenerated(depgraph), self.cooker.data)
+
         if self.state is runQueueSceneInit:
             if self.cooker.configuration.dump_signatures:
                 self.dump_signatures()
@@ -1181,6 +1204,8 @@ class RunQueueExecuteTasks(RunQueueExecute):
 
         self.stampcache = {}
 
+        initial_covered = self.rq.scenequeue_covered.copy()
+
         # Mark initial buildable tasks
         for task in xrange(self.stats.total):
             self.runq_running.append(0)
@@ -1234,13 +1259,28 @@ class RunQueueExecuteTasks(RunQueueExecute):
             except TypeError:
                 covered_remove = bb.utils.better_eval(call2, locs)
 
-        for task in covered_remove:
+        def removecoveredtask(task):
             fn = self.rqdata.taskData.fn_index[self.rqdata.runq_fnid[task]]
             taskname = self.rqdata.runq_task[task] + '_setscene'
             bb.build.del_stamp(taskname, self.rqdata.dataCache, fn)
-            logger.debug(1, 'Not skipping task %s due to setsceneverify', task)
             self.rq.scenequeue_covered.remove(task)
 
+        toremove = covered_remove
+        for task in toremove:
+            logger.debug(1, 'Not skipping task %s due to setsceneverify', task)
+        while toremove:
+            covered_remove = []
+            for task in toremove:
+                removecoveredtask(task)
+                for deptask in self.rqdata.runq_depends[task]:
+                    if deptask not in self.rq.scenequeue_covered:
+                        continue
+                    if deptask in toremove or deptask in covered_remove or deptask in initial_covered:
+                        continue
+                    logger.debug(1, 'Task %s depends on task %s so not skipping' % (task, deptask))
+                    covered_remove.append(deptask)
+            toremove = covered_remove
+
         logger.debug(1, 'Full skip list %s', self.rq.scenequeue_covered)
 
         event.fire(bb.event.StampUpdate(self.rqdata.target_pairs, self.rqdata.dataCache.stamp), self.cfgData)
@@ -1316,9 +1356,10 @@ class RunQueueExecuteTasks(RunQueueExecute):
         if self.rqdata.taskData.abort:
             self.rq.state = runQueueCleanUp
 
-    def task_skip(self, task):
+    def task_skip(self, task, reason):
         self.runq_running[task] = 1
         self.runq_buildable[task] = 1
+        bb.event.fire(runQueueTaskSkipped(task, self.stats, self.rq, reason), self.cfgData)
         self.task_completeoutright(task)
         self.stats.taskCompleted()
         self.stats.taskSkipped()
@@ -1343,13 +1384,13 @@ class RunQueueExecuteTasks(RunQueueExecute):
             if task in self.rq.scenequeue_covered:
                 logger.debug(2, "Setscene covered task %s (%s)", task,
                                 self.rqdata.get_user_idstring(task))
-                self.task_skip(task)
+                self.task_skip(task, "covered")
                 return True
 
             if self.rq.check_stamp_task(task, taskname, cache=self.stampcache):
                 logger.debug(2, "Stamp current task %s (%s)", task,
                                 self.rqdata.get_user_idstring(task))
-                self.task_skip(task)
+                self.task_skip(task, "existing")
                 return True
 
             taskdep = self.rqdata.dataCache.task_deps[fn]
@@ -1367,7 +1408,7 @@ class RunQueueExecuteTasks(RunQueueExecute):
                 bb.event.fire(startevent, self.cfgData)
 
             taskdep = self.rqdata.dataCache.task_deps[fn]
-            if 'fakeroot' in taskdep and taskname in taskdep['fakeroot']:
+            if 'fakeroot' in taskdep and taskname in taskdep['fakeroot'] and not self.cooker.configuration.dry_run:
                 if not self.rq.fakeworker:
                     self.rq.start_fakeworker(self)
                 self.rq.fakeworker.stdin.write("<runtask>" + pickle.dumps((fn, task, taskname, False, self.cooker.collection.get_file_appends(fn))) + "</runtask>")
@@ -1636,6 +1677,7 @@ class RunQueueExecuteScenequeue(RunQueueExecute):
 
     def task_complete(self, task):
         self.stats.taskCompleted()
+        bb.event.fire(sceneQueueTaskCompleted(task, self.stats, self.rq), self.cfgData)
         self.task_completeoutright(task)
 
     def task_fail(self, task, result):
@@ -1770,6 +1812,9 @@ class runQueueEvent(bb.event.Event):
     def __init__(self, task, stats, rq):
         self.taskid = task
         self.taskstring = rq.rqdata.get_user_idstring(task)
+        self.taskname = rq.rqdata.get_task_name(task)
+        self.taskfile = rq.rqdata.get_task_file(task)
+        self.taskhash = rq.rqdata.get_task_hash(task)
         self.stats = stats.copy()
         bb.event.Event.__init__(self)
 
@@ -1781,6 +1826,9 @@ class sceneQueueEvent(runQueueEvent):
         runQueueEvent.__init__(self, task, stats, rq)
         realtask = rq.rqdata.runq_setscene[task]
         self.taskstring = rq.rqdata.get_user_idstring(realtask, "_setscene")
+        self.taskname = rq.rqdata.get_task_name(realtask) + "_setscene"
+        self.taskfile = rq.rqdata.get_task_file(realtask)
+        self.taskhash = rq.rqdata.get_task_hash(task)
 
 class runQueueTaskStarted(runQueueEvent):
     """
@@ -1819,6 +1867,19 @@ class runQueueTaskCompleted(runQueueEvent):
     Event notifing a task completed
     """
 
+class sceneQueueTaskCompleted(sceneQueueEvent):
+    """
+    Event notifing a setscene task completed
+    """
+
+class runQueueTaskSkipped(runQueueEvent):
+    """
+    Event notifing a task was skipped
+    """
+    def __init__(self, task, stats, rq, reason):
+        runQueueEvent.__init__(self, task, stats, rq)
+        self.reason = reason
+
 class runQueuePipe():
     """
     Abstraction for a pipe between a worker thread and the server

bitbake/lib/bb/server/__init__.py

@@ -89,7 +89,7 @@ class BitBakeBaseServer(object):
     def detach(self):
         return
 
-    def establishConnection(self):
+    def establishConnection(self, featureset):
         raise   "Must redefine the %s.establishConnection()" % self.__class__.__name__
 
     def endSession(self):

bitbake/lib/bb/server/process.py

@@ -31,7 +31,7 @@ import sys
 import time
 import select
 from Queue import Empty
-from multiprocessing import Event, Process, util, Queue, Pipe, queues
+from multiprocessing import Event, Process, util, Queue, Pipe, queues, Manager
 
 from . import BitBakeBaseServer, BitBakeBaseServerConnection, BaseImplServer
 
@@ -78,12 +78,13 @@ class ProcessServer(Process, BaseImplServer):
     profile_filename = "profile.log"
     profile_processed_filename = "profile.log.processed"
 
-    def __init__(self, command_channel, event_queue):
+    def __init__(self, command_channel, event_queue, featurelist):
         BaseImplServer.__init__(self)
-        Process.__init__(self)
+        Process.__init__(self, args=(featurelist))
         self.command_channel = command_channel
         self.event_queue = event_queue
         self.event = EventAdapter(event_queue)
+        self.featurelist = featurelist
         self.quit = False
 
         self.keep_running = Event()
@@ -94,6 +95,14 @@ class ProcessServer(Process, BaseImplServer):
         for event in bb.event.ui_queue:
             self.event_queue.put(event)
         self.event_handle.value = bb.event.register_UIHhandler(self)
+
+        # process any feature changes based on what UI requested
+        original_featureset = list(self.cooker.featureset)
+        while len(self.featurelist)> 0:
+            self.cooker.featureset.setFeature(self.featurelist.pop())
+        if (original_featureset != list(self.cooker.featureset)):
+            self.cooker.reset()
+
         bb.cooker.server_main(self.cooker, self.main)
 
     def main(self):
@@ -113,7 +122,7 @@ class ProcessServer(Process, BaseImplServer):
         self.event_queue.close()
         bb.event.unregister_UIHhandler(self.event_handle.value)
         self.command_channel.close()
-        self.cooker.stop()
+        self.cooker.shutdown(True)
         self.idle_commands(.1)
 
     def idle_commands(self, delay, fds = []):
@@ -198,13 +207,17 @@ class BitBakeServer(BitBakeBaseServer):
         #
         self.ui_channel, self.server_channel = Pipe()
         self.event_queue = ProcessEventQueue(0)
-        self.serverImpl = ProcessServer(self.server_channel, self.event_queue)
+        manager = Manager()
+        self.featurelist = manager.list()
+        self.serverImpl = ProcessServer(self.server_channel, self.event_queue, self.featurelist)
 
     def detach(self):
         self.serverImpl.start()
         return
 
-    def establishConnection(self):
+    def establishConnection(self, featureset):
+        for f in featureset:
+            self.featurelist.append(f)
         self.connection = BitBakeProcessServerConnection(self.serverImpl, self.ui_channel, self.event_queue)
         signal.signal(signal.SIGTERM, lambda i, s: self.connection.terminate())
         return self.connection

bitbake/lib/bb/server/xmlrpc.py

@@ -89,12 +89,23 @@ class BitBakeServerCommands():
         self.server = server
         self.has_client = False
 
-    def registerEventHandler(self, host, port):
+    def registerEventHandler(self, host, port, featureset = []):
         """
         Register a remote UI Event Handler
         """
         s, t = _create_server(host, port)
 
+        # we don't allow connections if the cooker is running
+        if (self.cooker.state in [bb.cooker.state.parsing, bb.cooker.state.running]):
+            return None
+
+        original_featureset = list(self.cooker.featureset)
+        for f in featureset:
+            self.cooker.featureset.setFeature(f)
+
+        if (original_featureset != list(self.cooker.featureset)):
+            self.cooker.reset()
+
         self.event_handle = bb.event.register_UIHhandler(s)
         return self.event_handle
 
@@ -169,51 +180,6 @@ class BitBakeXMLRPCRequestHandler(SimpleXMLRPCRequestHandler):
         self.end_headers()
         self.wfile.write(response)
 
-class BitBakeUIEventServer(threading.Thread):
-    class EventAdapter():
-        """
-        Adapter to wrap our event queue since the caller (bb.event) expects to
-        call a send() method, but our actual queue only has put()
-        """
-        def __init__(self, notify):
-            self.queue = []
-            self.notify = notify
-            self.qlock = threading.Lock()
-
-        def send(self, event):
-            self.qlock.acquire()
-            self.queue.append(event)
-            self.qlock.release()
-            self.notify.set()
-
-        def get(self):
-            self.qlock.acquire()
-            if len(self.queue) == 0:
-                self.qlock.release()
-                return None
-            e = self.queue.pop(0)
-            if len(self.queue) == 0:
-                self.notify.clear()
-            self.qlock.release()
-            return e
-
-    def __init__(self, connection):
-        self.connection = connection
-        self.notify = threading.Event()
-        self.event = BitBakeUIEventServer.EventAdapter(self.notify)
-        self.quit = False
-        threading.Thread.__init__(self)
-
-    def terminateServer(self):
-        self.quit = True
-
-    def run(self):
-        while not self.quit:
-            self.notify.wait(0.1)
-            evt = self.event.get()
-            if evt:
-                self.connection.event.sendpickle(pickle.dumps(evt))
-
 
 class XMLRPCProxyServer(BaseImplServer):
     """ not a real working server, but a stub for a proxy server connection
@@ -308,11 +274,12 @@ class XMLRPCServer(SimpleXMLRPCServer, BaseImplServer):
         self.connection_token = token
 
 class BitBakeXMLRPCServerConnection(BitBakeBaseServerConnection):
-    def __init__(self, serverImpl, clientinfo=("localhost", 0), observer_only = False):
+    def __init__(self, serverImpl, clientinfo=("localhost", 0), observer_only = False, featureset = []):
         self.connection, self.transport = _create_server(serverImpl.host, serverImpl.port)
         self.clientinfo = clientinfo
         self.serverImpl = serverImpl
         self.observer_only = observer_only
+        self.featureset = featureset
 
     def connect(self):
         if not self.observer_only:
@@ -322,7 +289,8 @@ class BitBakeXMLRPCServerConnection(BitBakeBaseServerConnection):
         if token is None:
             return None
         self.transport.set_connection_token(token)
-        self.events = uievent.BBUIEventQueue(self.connection, self.clientinfo)
+
+        self.events = uievent.BBUIEventQueue(self.connection, self.clientinfo, self.featureset)
         for event in bb.event.ui_queue:
             self.events.queue_event(event)
         return self
@@ -346,14 +314,15 @@ class BitBakeXMLRPCServerConnection(BitBakeBaseServerConnection):
 
 class BitBakeServer(BitBakeBaseServer):
     def initServer(self, interface = ("localhost", 0)):
+        self.interface = interface
         self.serverImpl = XMLRPCServer(interface)
 
     def detach(self):
         daemonize.createDaemon(self.serverImpl.serve_forever, "bitbake-cookerdaemon.log")
         del self.cooker
 
-    def establishConnection(self):
-        self.connection = BitBakeXMLRPCServerConnection(self.serverImpl)
+    def establishConnection(self, featureset):
+        self.connection = BitBakeXMLRPCServerConnection(self.serverImpl, self.interface, False, featureset)
         return self.connection.connect()
 
     def set_connection_token(self, token):
@@ -363,12 +332,13 @@ class BitBakeXMLRPCClient(BitBakeBaseServer):
 
     def __init__(self, observer_only = False):
         self.observer_only = observer_only
+        # if we need extra caches, just tell the server to load them all
         pass
 
     def saveConnectionDetails(self, remote):
         self.remote = remote
 
-    def establishConnection(self):
+    def establishConnection(self, featureset):
         # The format of "remote" must be "server:port"
         try:
             [host, port] = self.remote.split(":")
@@ -384,9 +354,12 @@ class BitBakeXMLRPCClient(BitBakeBaseServer):
             s.close()
         except:
             return None
-        self.serverImpl = XMLRPCProxyServer(host, port)
-        self.connection = BitBakeXMLRPCServerConnection(self.serverImpl, (ip, 0), self.observer_only)
-        return self.connection.connect()
+        try:
+            self.serverImpl = XMLRPCProxyServer(host, port)
+            self.connection = BitBakeXMLRPCServerConnection(self.serverImpl, (ip, 0), self.observer_only, featureset)
+            return self.connection.connect()
+        except Exception as e:
+            bb.fatal("Could not connect to server at %s:%s (%s)" % (host, port, str(e)))
 
     def endSession(self):
         self.connection.removeClient()

bitbake/lib/bb/siggen.py

@@ -91,8 +91,7 @@ class SignatureGeneratorBasic(SignatureGenerator):
         basehash = {}
 
         for task in tasklist:
-            data = d.getVar(task, False)
-            lookupcache[task] = data
+            data = lookupcache[task]
 
             if data is None:
                 bb.error("Task %s from %s seems to be empty?!" % (task, fn))
@@ -115,16 +114,8 @@ class SignatureGeneratorBasic(SignatureGenerator):
             alldeps = sorted(seen)
             for dep in alldeps:
                 data = data + dep
-                if dep in lookupcache:
-                    var = lookupcache[dep]
-                elif dep[-1] == ']':
-                    vf = dep[:-1].split('[')
-                    var = d.getVarFlag(vf[0], vf[1], False)
-                    lookupcache[dep] = var
-                else:
-                    var = d.getVar(dep, False)
-                    lookupcache[dep] = var
-                if var:
+                var = lookupcache[dep]
+                if var is not None:
                     data = data + str(var)
             self.basehash[fn + "." + task] = hashlib.md5(data).hexdigest()
             taskdeps[task] = alldeps

bitbake/lib/bb/tests/data.py

@@ -27,21 +27,21 @@ import bb.data
 class DataExpansions(unittest.TestCase):
     def setUp(self):
         self.d = bb.data.init()
-        self.d["foo"] = "value of foo"
-        self.d["bar"] = "value of bar"
-        self.d["value of foo"] = "value of 'value of foo'"
+        self.d["foo"] = "value_of_foo"
+        self.d["bar"] = "value_of_bar"
+        self.d["value_of_foo"] = "value_of_'value_of_foo'"
 
     def test_one_var(self):
         val = self.d.expand("${foo}")
-        self.assertEqual(str(val), "value of foo")
+        self.assertEqual(str(val), "value_of_foo")
 
     def test_indirect_one_var(self):
         val = self.d.expand("${${foo}}")
-        self.assertEqual(str(val), "value of 'value of foo'")
+        self.assertEqual(str(val), "value_of_'value_of_foo'")
 
     def test_indirect_and_another(self):
         val = self.d.expand("${${foo}} ${bar}")
-        self.assertEqual(str(val), "value of 'value of foo' value of bar")
+        self.assertEqual(str(val), "value_of_'value_of_foo' value_of_bar")
 
     def test_python_snippet(self):
         val = self.d.expand("${@5*12}")
@@ -49,11 +49,11 @@ class DataExpansions(unittest.TestCase):
 
     def test_expand_in_python_snippet(self):
         val = self.d.expand("${@'boo ' + '${foo}'}")
-        self.assertEqual(str(val), "boo value of foo")
+        self.assertEqual(str(val), "boo value_of_foo")
 
     def test_python_snippet_getvar(self):
         val = self.d.expand("${@d.getVar('foo', True) + ' ${bar}'}")
-        self.assertEqual(str(val), "value of foo value of bar")
+        self.assertEqual(str(val), "value_of_foo value_of_bar")
 
     def test_python_snippet_syntax_error(self):
         self.d.setVar("FOO", "${@foo = 5}")
@@ -70,7 +70,7 @@ class DataExpansions(unittest.TestCase):
 
     def test_value_containing_value(self):
         val = self.d.expand("${@d.getVar('foo', True) + ' ${bar}'}")
-        self.assertEqual(str(val), "value of foo value of bar")
+        self.assertEqual(str(val), "value_of_foo value_of_bar")
 
     def test_reference_undefined_var(self):
         val = self.d.expand("${undefinedvar} meh")
@@ -109,7 +109,7 @@ class DataExpansions(unittest.TestCase):
 
     def test_rename(self):
         self.d.renameVar("foo", "newfoo")
-        self.assertEqual(self.d.getVar("newfoo"), "value of foo")
+        self.assertEqual(self.d.getVar("newfoo"), "value_of_foo")
         self.assertEqual(self.d.getVar("foo"), None)
 
     def test_deletion(self):
@@ -118,17 +118,17 @@ class DataExpansions(unittest.TestCase):
 
     def test_keys(self):
         keys = self.d.keys()
-        self.assertEqual(keys, ['value of foo', 'foo', 'bar'])
+        self.assertEqual(keys, ['value_of_foo', 'foo', 'bar'])
 
 class TestNestedExpansions(unittest.TestCase):
     def setUp(self):
         self.d = bb.data.init()
         self.d["foo"] = "foo"
         self.d["bar"] = "bar"
-        self.d["value of foobar"] = "187"
+        self.d["value_of_foobar"] = "187"
 
     def test_refs(self):
-        val = self.d.expand("${value of ${foo}${bar}}")
+        val = self.d.expand("${value_of_${foo}${bar}}")
         self.assertEqual(str(val), "187")
 
     #def test_python_refs(self):
@@ -154,11 +154,11 @@ class TestNestedExpansions(unittest.TestCase):
     #    self.assertEqual(str(val), str(depth + 1))
 
     def test_mixed(self):
-        val = self.d.expand("${value of ${@('${foo}'+'bar')[0:3]}${${@'BAR'.lower()}}}")
+        val = self.d.expand("${value_of_${@('${foo}'+'bar')[0:3]}${${@'BAR'.lower()}}}")
         self.assertEqual(str(val), "187")
 
     def test_runtime(self):
-        val = self.d.expand("${${@'value of' + ' f'+'o'+'o'+'b'+'a'+'r'}}")
+        val = self.d.expand("${${@'value_of' + '_f'+'o'+'o'+'b'+'a'+'r'}}")
         self.assertEqual(str(val), "187")
 
 class TestMemoize(unittest.TestCase):

bitbake/lib/bb/tests/fetch.py

@@ -407,7 +407,8 @@ class URLHandle(unittest.TestCase):
     datatable = {
        "http://www.google.com/index.html" : ('http', 'www.google.com', '/index.html', '', '', {}),
        "cvs://anoncvs@cvs.handhelds.org/cvs;module=familiar/dist/ipkg" : ('cvs', 'cvs.handhelds.org', '/cvs', 'anoncvs', '', {'module': 'familiar/dist/ipkg'}),
-       "cvs://anoncvs:anonymous@cvs.handhelds.org/cvs;tag=V0-99-81;module=familiar/dist/ipkg" : ('cvs', 'cvs.handhelds.org', '/cvs', 'anoncvs', 'anonymous', {'tag': 'V0-99-81', 'module': 'familiar/dist/ipkg'})
+       "cvs://anoncvs:anonymous@cvs.handhelds.org/cvs;tag=V0-99-81;module=familiar/dist/ipkg" : ('cvs', 'cvs.handhelds.org', '/cvs', 'anoncvs', 'anonymous', {'tag': 'V0-99-81', 'module': 'familiar/dist/ipkg'}),
+       "git://git.openembedded.org/bitbake;branch=@foo" : ('git', 'git.openembedded.org', '/bitbake', '', '', {'branch': '@foo'})
     }
 
     def test_decodeurl(self):

bitbake/lib/bb/ui/crumbs/builder.py

@@ -31,6 +31,7 @@ import re
 import logging
 import sys
 import signal
+import time
 from bb.ui.crumbs.imageconfigurationpage import ImageConfigurationPage
 from bb.ui.crumbs.recipeselectionpage import RecipeSelectionPage
 from bb.ui.crumbs.packageselectionpage import PackageSelectionPage
@@ -197,7 +198,7 @@ class Configuration:
         handler.set_var_in_file("BBLAYERS", self.layers, "bblayers.conf")
         # local.conf
         if not defaults:
-            handler.set_var_in_file("MACHINE", self.curr_mach, "local.conf")
+            handler.early_assign_var_in_file("MACHINE", self.curr_mach, "local.conf")
         handler.set_var_in_file("DISTRO", self.curr_distro, "local.conf")
         handler.set_var_in_file("DL_DIR", self.dldir, "local.conf")
         handler.set_var_in_file("SSTATE_DIR", self.sstatedir, "local.conf")
@@ -217,7 +218,7 @@ class Configuration:
         handler.set_var_in_file("SDKMACHINE", self.curr_sdk_machine, "local.conf")
         handler.set_var_in_file("CONF_VERSION", self.conf_version, "local.conf")
         handler.set_var_in_file("LCONF_VERSION", self.lconf_version, "bblayers.conf")
-        handler.set_var_in_file("EXTRA_SETTING", self.extra_setting, "local.conf")
+        handler.set_extra_config(self.extra_setting)
         handler.set_var_in_file("TOOLCHAIN_BUILD", self.toolchain_build, "local.conf")
         handler.set_var_in_file("IMAGE_FSTYPES", self.image_fstypes, "local.conf")
         if not defaults:
@@ -1466,3 +1467,10 @@ class Builder(gtk.Window):
 
     def get_topdir(self):
         return self.handler.get_topdir()
+
+    def wait(self, delay):
+        time_start = time.time()
+        time_end = time_start + delay
+        while time_end > time.time():
+            while gtk.events_pending():
+                gtk.main_iteration()

bitbake/lib/bb/ui/crumbs/hig/advancedsettingsdialog.py

@@ -234,7 +234,10 @@ class AdvancedSettingsDialog (CrumbsDialog, SettingsUIHelper):
             article = ""
             if image_type.startswith(("a", "e", "i", "o", "u")):
                 article = "n"
-            self.image_types_checkbuttons[image_type].set_tooltip_text("Build a%s %s image" % (article, image_type))
+            if image_type == "live":
+                self.image_types_checkbuttons[image_type].set_tooltip_text("Build iso and hddimg images")
+            else:
+                self.image_types_checkbuttons[image_type].set_tooltip_text("Build a%s %s image" % (article, image_type))
             table.attach(self.image_types_checkbuttons[image_type], j - 1, j + 3, i, i + 1)
             if image_type in self.configuration.image_fstypes.split():
                 self.image_types_checkbuttons[image_type].set_active(True)

bitbake/lib/bb/ui/crumbs/hig/saveimagedialog.py

@@ -70,9 +70,11 @@ class SaveImageDialog (CrumbsDialog):
         sub_label.set_alignment(0, 0)
         sub_label.set_markup("The description should be less than 150 characters long.")
         self.description_entry = gtk.TextView()
-        self.description_entry.get_buffer().set_text(self.description_field)
+        description_buffer = self.description_entry.get_buffer()
+        description_buffer.set_text(self.description_field)
+        description_buffer.connect("insert-text", self.limit_description_length)
         self.description_entry.set_wrap_mode(gtk.WRAP_WORD)
-        self.description_entry.set_size_request(350,150)
+        self.description_entry.set_size_request(350,50)
         sub_vbox.pack_start(label, expand=False, fill=False)
         sub_vbox.pack_start(sub_label, expand=False, fill=False)
         sub_vbox.pack_start(self.description_entry, expand=False, fill=False)
@@ -108,6 +110,13 @@ class SaveImageDialog (CrumbsDialog):
 
         self.show_all()
 
+    def limit_description_length(self, textbuffer, iter, text, length):
+        buffer_bounds = textbuffer.get_bounds()
+        entire_text = textbuffer.get_text(*buffer_bounds)
+        entire_text += text
+        if len(entire_text)>150 or text=="\n":
+            textbuffer.emit_stop_by_name("insert-text")
+
     def name_entry_changed(self, entry):
         text = entry.get_text()
         if text == '':

bitbake/lib/bb/ui/crumbs/hobeventhandler.py

@@ -21,6 +21,7 @@
 
 import gobject
 import logging
+import ast
 from bb.ui.crumbs.runningbuild import RunningBuild
 
 class HobHandler(gobject.GObject):
@@ -315,7 +316,7 @@ class HobHandler(gobject.GObject):
 
     def set_machine(self, machine):
         if machine:
-            self.set_var_in_file("MACHINE", machine, "local.conf")
+            self.early_assign_var_in_file("MACHINE", machine, "local.conf")
 
     def set_sdk_machine(self, sdk_machine):
         self.set_var_in_file("SDKMACHINE", sdk_machine, "local.conf")
@@ -357,7 +358,20 @@ class HobHandler(gobject.GObject):
     def set_incompatible_license(self, incompat_license):
         self.set_var_in_file("INCOMPATIBLE_LICENSE", incompat_license, "local.conf")
 
+    def set_extra_setting(self, extra_setting):
+        self.set_var_in_file("EXTRA_SETTING", extra_setting, "local.conf")
+
     def set_extra_config(self, extra_setting):
+        old_extra_setting = ast.literal_eval(self.runCommand(["getVariable", "EXTRA_SETTING"]) or "{}")
+        if extra_setting:
+            self.set_var_in_file("EXTRA_SETTING", extra_setting, "local.conf")
+        else:
+            self.remove_var_from_file("EXTRA_SETTING")
+
+        #remove not needed settings from conf
+        for key in old_extra_setting:
+            if key not in extra_setting:
+                self.remove_var_from_file(key)
         for key in extra_setting.keys():
             value = extra_setting[key]
             self.set_var_in_file(key, value, "local.conf")
@@ -440,12 +454,12 @@ class HobHandler(gobject.GObject):
         self.building = False
 
     def cancel_parse(self):
-        self.runCommand(["stateStop"])
+        self.runCommand(["stateForceShutdown"])
 
     def cancel_build(self, force=False):
         if force:
             # Force the cooker to stop as quickly as possible
-            self.runCommand(["stateStop"])
+            self.runCommand(["stateForceShutdown"])
         else:
             # Wait for tasks to complete before shutting down, this helps
             # leave the workdir in a usable state
@@ -472,6 +486,14 @@ class HobHandler(gobject.GObject):
         self.server.runCommand(["setVarFile", var, val, default_file, "set"])
         self.runCommand(["disableDataTracking"])
 
+    def early_assign_var_in_file(self, var, val, default_file=None):
+        self.runCommand(["enableDataTracking"])
+        self.server.runCommand(["setVarFile", var, val, default_file, "earlyAssign"])
+        self.runCommand(["disableDataTracking"])
+
+    def remove_var_from_file(self, var):
+        self.server.runCommand(["removeVarFile", var])
+
     def append_var_in_file(self, var, val, default_file=None):
         self.server.runCommand(["setVarFile", var, val, default_file, "append"])
 

bitbake/lib/bb/ui/crumbs/hoblistmodel.py

@@ -199,7 +199,9 @@ class PackageListModel(gtk.ListStore):
         return self.cmp_vals(val1, val2, user_data)
 
     def cmp_vals(self, val1, val2, user_data):
-        if val1.startswith(user_data) and not val2.startswith(user_data):
+        if val1 is None or val2 is None:
+            return 0
+        elif val1.startswith(user_data) and not val2.startswith(user_data):
             return -1
         elif not val1.startswith(user_data) and val2.startswith(user_data):
             return 1
@@ -575,7 +577,9 @@ class RecipeListModel(gtk.ListStore):
         return self.cmp_vals(val1, val2, user_data)
 
     def cmp_vals(self, val1, val2, user_data):
-        if val1.startswith(user_data) and not val2.startswith(user_data):
+        if val1 is None or val2 is None:
+            return 0
+        elif val1.startswith(user_data) and not val2.startswith(user_data):
             return -1
         elif not val1.startswith(user_data) and val2.startswith(user_data):
             return 1
@@ -690,7 +694,7 @@ class RecipeListModel(gtk.ListStore):
             inherits = event_model["pn"][item]["inherits"]
             summary = event_model["pn"][item]["summary"]
             version = event_model["pn"][item]["version"]
-            revision = event_model["pn"][item]["revision"]
+            revision = event_model["pn"][item]["prevision"]
             homepage = event_model["pn"][item]["homepage"]
             bugtracker = event_model["pn"][item]["bugtracker"]
             filename = event_model["pn"][item]["filename"]

bitbake/lib/bb/ui/crumbs/imageconfigurationpage.py

@@ -300,7 +300,12 @@ class ImageConfigurationPage (HobPage):
     def view_warnings_button_clicked_cb(self, button):
         self.builder.show_warning_dialog()
 
+    def machine_combo_changed_idle_cb(self):
+        self.builder.window.set_cursor(None)
+
     def machine_combo_changed_cb(self, machine_combo):
+        self.builder.window.set_cursor(gtk.gdk.Cursor(gtk.gdk.WATCH))
+        self.builder.wait(0.1) #wait for combo and cursor to update
         self.stopping = False
         self.builder.parsing_warnings = []
         combo_item = machine_combo.get_active_text()
@@ -324,6 +329,8 @@ class ImageConfigurationPage (HobPage):
         # Do reparse recipes
         self.builder.populate_recipe_package_info_async()
 
+        glib.idle_add(self.machine_combo_changed_idle_cb)
+
     def update_machine_combo(self):
         self.disable_warnings_bar()
         all_machines = [self.__dummy_machine__] + self.builder.parameters.all_machines
@@ -387,6 +394,10 @@ class ImageConfigurationPage (HobPage):
                 selected_image = self.__dummy_image__
                 self.update_image_combo(self.builder.recipe_model, None)
             dialog.destroy()
+        else:
+            if self.custom_image_selected:
+                self.custom_image_selected = None
+                self.update_image_combo(self.builder.recipe_model, selected_image)
 
         if not selected_image or (selected_image == self.__dummy_image__):
             self.builder.window_sensitive(True)
@@ -523,7 +534,10 @@ class ImageConfigurationPage (HobPage):
         if not response:
             return
         if settings_changed:
+            self.builder.window.set_cursor(gtk.gdk.Cursor(gtk.gdk.WATCH))
+            self.builder.wait(0.1) #wait for adv_settings_dialog to terminate
             self.builder.reparse_post_adv_settings()
+            self.builder.window.set_cursor(None)
 
     def just_bake_button_clicked_cb(self, button):
         self.builder.parsing_warnings = []

bitbake/lib/bb/ui/crumbs/imagedetailspage.py

@@ -355,9 +355,9 @@ class ImageDetailsPage (HobPage):
             vallist.append(base_image)
             i = 0
             for layer in layers:
-                varlist.append(" - ")
                 if i > layer_num_limit:
                     break
+                varlist.append(" - ")
                 i += 1
             vallist.append("")
             i = 0
@@ -635,6 +635,14 @@ class ImageDetailsPage (HobPage):
         images_dir = topdir + "/recipes/images/"
         self.builder.ensure_dir(images_dir)
 
+        self.name_field_template = self.builder.image_configuration_page.custom_image_selected
+        if self.name_field_template:
+            image_path = self.builder.recipe_model.pn_path[self.name_field_template]
+            image_iter = self.builder.recipe_model.get_iter(image_path)
+            self.description_field_template = self.builder.recipe_model.get_value(image_iter, self.builder.recipe_model.COL_DESC)
+        else:
+            self.name_field_template = ""
+
         dialog = SaveImageDialog(images_dir, self.name_field_template, self.description_field_template,
                  "Save image recipe", self.builder, gtk.DIALOG_MODAL | gtk.DIALOG_DESTROY_WITH_PARENT)
         response = dialog.run()

bitbake/lib/bb/ui/depexp.py

@@ -314,7 +314,7 @@ def main(server, eventHandler, params):
                 break
             if shutdown == 1:
                 print("\nSecond Keyboard Interrupt, stopping...\n")
-                _, error = server.runCommand(["stateStop"])
+                _, error = server.runCommand(["stateForceShutdown"])
                 if error:
                     print('Unable to cleanly stop: %s' % error)
             if shutdown == 0:

bitbake/lib/bb/ui/goggle.py

@@ -117,5 +117,5 @@ def main (server, eventHandler, params):
     except KeyboardInterrupt:
         pass
     finally:
-        server.runCommand(["stateStop"])
+        server.runCommand(["stateForceShutdown"])
 

bitbake/lib/bb/ui/hob.py

@@ -46,7 +46,7 @@ from bb.ui.crumbs.hoblistmodel import RecipeListModel, PackageListModel
 from bb.ui.crumbs.hobeventhandler import HobHandler
 from bb.ui.crumbs.builder import Builder
 
-extraCaches = ['bb.cache_extra:HobRecipeInfo']
+featureSet = [bb.cooker.CookerFeatures.HOB_EXTRA_CACHES]
 
 def event_handle_idle_func(eventHandler, hobHandler):
     # Consume as many messages as we can in the time available to us

bitbake/lib/bb/ui/knotty.py

@@ -306,8 +306,10 @@ def main(server, eventHandler, params, tf = TerminalFilter):
 
     while True:
         try:
-            termfilter.updateFooter()
-            event = eventHandler.waitEvent(0.25)
+            event = eventHandler.waitEvent(0)
+            if event is None:
+                termfilter.updateFooter()
+                event = eventHandler.waitEvent(0.25)
             if event is None:
                 if main.shutdown > 1:
                     break
@@ -403,8 +405,9 @@ def main(server, eventHandler, params, tf = TerminalFilter):
 
             if isinstance(event, bb.command.CommandFailed):
                 return_value = event.exitcode
-                errors = errors + 1
-                logger.error("Command execution failed: %s", event.error)
+                if event.error:
+                    errors = errors + 1
+                    logger.error("Command execution failed: %s", event.error)
                 main.shutdown = 2
                 continue
             if isinstance(event, bb.command.CommandExit):
@@ -469,8 +472,12 @@ def main(server, eventHandler, params, tf = TerminalFilter):
                              event.taskid, event.taskstring, event.exitcode)
                 continue
 
+            if isinstance(event, bb.event.DepTreeGenerated):
+                continue
+
             # ignore
             if isinstance(event, (bb.event.BuildBase,
+                                  bb.event.MetadataEvent,
                                   bb.event.StampUpdate,
                                   bb.event.ConfigParsed,
                                   bb.event.RecipeParsed,
@@ -497,7 +504,7 @@ def main(server, eventHandler, params, tf = TerminalFilter):
                 main.shutdown = 2
             if not params.observe_only and main.shutdown == 1:
                 print("\nSecond Keyboard Interrupt, stopping...\n")
-                _, error = server.runCommand(["stateStop"])
+                _, error = server.runCommand(["stateForceShutdown"])
                 if error:
                     logger.error("Unable to cleanly stop: %s" % error)
             if not params.observe_only and main.shutdown == 0:
@@ -518,7 +525,7 @@ def main(server, eventHandler, params, tf = TerminalFilter):
     if warnings:
         summary += pluralise("\nSummary: There was %s WARNING message shown.",
                              "\nSummary: There were %s WARNING messages shown.", warnings)
-    if return_value:
+    if return_value and errors:
         summary += pluralise("\nSummary: There was %s ERROR message shown, returning a non-zero exit code.",
                              "\nSummary: There were %s ERROR messages shown, returning a non-zero exit code.", errors)
     if summary:

bitbake/lib/bb/ui/ncurses.py

@@ -350,7 +350,7 @@ class NCursesUI:
                     exitflag = True
                 if shutdown == 1:
                     mw.appendText("Second Keyboard Interrupt, stopping...\n")
-                    _, error = server.runCommand(["stateStop"])
+                    _, error = server.runCommand(["stateForceShutdown"])
                     if error:
                         print("Unable to cleanly stop: %s" % error)
                 if shutdown == 0:

bitbake/lib/bb/ui/uievent.py

@@ -28,7 +28,7 @@ import socket, threading, pickle
 from SimpleXMLRPCServer import SimpleXMLRPCServer, SimpleXMLRPCRequestHandler
 
 class BBUIEventQueue:
-    def __init__(self, BBServer, clientinfo=("localhost, 0")):
+    def __init__(self, BBServer, clientinfo=("localhost, 0"), featureset=[]):
 
         self.eventQueue = []
         self.eventQueueLock = threading.Lock()
@@ -44,7 +44,10 @@ class BBUIEventQueue:
         server.register_function( self.send_event, "event.sendpickle" )
         server.socket.settimeout(1)
 
-        self.EventHandle = self.BBServer.registerEventHandler(self.host, self.port)
+        self.EventHandle = self.BBServer.registerEventHandler(self.host, self.port, featureset)
+
+        if (self.EventHandle == None):
+            bb.fatal("Could not register UI event handler")
 
         self.server = server
 

bitbake/lib/bb/utils.py

@@ -532,6 +532,8 @@ def approved_variables():
     Determine and return the list of whitelisted variables which are approved
     to remain in the envrionment.
     """
+    if 'BB_PRESERVE_ENV' in os.environ:
+        return os.environ.keys()
     approved = []
     if 'BB_ENV_WHITELIST' in os.environ:
         approved = os.environ['BB_ENV_WHITELIST'].split()
@@ -722,7 +724,7 @@ def copyfile(src, dest, newmtime = None, sstat = None):
         if not sstat:
             sstat = os.lstat(src)
     except Exception as e:
-        print("copyfile: Stating source file failed...", e)
+        logger.warn("copyfile: stat of %s failed (%s)" % (src, e))
         return False
 
     destexists = 1
@@ -749,7 +751,7 @@ def copyfile(src, dest, newmtime = None, sstat = None):
             #os.lchown(dest,sstat[stat.ST_UID],sstat[stat.ST_GID])
             return os.lstat(dest)
         except Exception as e:
-            print("copyfile: failed to properly create symlink:", dest, "->", target, e)
+            logger.warn("copyfile: failed to create symlink %s to %s (%s)" % (dest, target, e))
             return False
 
     if stat.S_ISREG(sstat[stat.ST_MODE]):
@@ -764,7 +766,7 @@ def copyfile(src, dest, newmtime = None, sstat = None):
             shutil.copyfile(src, dest + "#new")
             os.rename(dest + "#new", dest)
         except Exception as e:
-            print('copyfile: copy', src, '->', dest, 'failed.', e)
+            logger.warn("copyfile: copy %s to %s failed (%s)" % (src, dest, e))
             return False
         finally:
             if srcchown:
@@ -775,13 +777,13 @@ def copyfile(src, dest, newmtime = None, sstat = None):
         #we don't yet handle special, so we need to fall back to /bin/mv
         a = getstatusoutput("/bin/cp -f " + "'" + src + "' '" + dest + "'")
         if a[0] != 0:
-            print("copyfile: Failed to copy special file:" + src + "' to '" + dest + "'", a)
+            logger.warn("copyfile: failed to copy special file %s to %s (%s)" % (src, dest, a))
             return False # failure
     try:
         os.lchown(dest, sstat[stat.ST_UID], sstat[stat.ST_GID])
         os.chmod(dest, stat.S_IMODE(sstat[stat.ST_MODE])) # Sticky is reset on chown
     except Exception as e:
-        print("copyfile: Failed to chown/chmod/unlink", dest, e)
+        logger.warn("copyfile: failed to chown/chmod %s (%s)" % (dest, e))
         return False
 
     if newmtime:
@@ -791,22 +793,28 @@ def copyfile(src, dest, newmtime = None, sstat = None):
         newmtime = sstat[stat.ST_MTIME]
     return newmtime
 
-def which(path, item, direction = 0):
+def which(path, item, direction = 0, history = False):
     """
     Locate a file in a PATH
     """
 
+    hist = []
     paths = (path or "").split(':')
     if direction != 0:
         paths.reverse()
 
     for p in paths:
         next = os.path.join(p, item)
+        hist.append(next)
         if os.path.exists(next):
             if not os.path.isabs(next):
                 next = os.path.abspath(next)
+            if history:
+                return next, hist
             return next
 
+    if history:
+        return "", hist
     return ""
 
 def to_boolean(string, default=None):

bitbake/lib/prserv/db.py

@@ -43,6 +43,10 @@ class PRTable(object):
                     continue
                 raise exc
 
+    def sync(self):
+        self.conn.commit()
+        self._execute("BEGIN EXCLUSIVE TRANSACTION")
+
     def _getValueHist(self, version, pkgarch, checksum):
         data=self._execute("SELECT value FROM %s WHERE version=? AND pkgarch=? AND checksum=?;" % self.table,
                            (version, pkgarch, checksum))

bitbake/lib/prserv/serv.py

@@ -74,7 +74,7 @@ class PRServer(SimpleXMLRPCServer):
             except:
                 self.handle_error(request, client_address)
                 self.shutdown_request(request)
-
+                self.table.sync()
 
     def process_request(self, request, client_address):
         self.requestqueue.put((request, client_address))
@@ -120,6 +120,7 @@ class PRServer(SimpleXMLRPCServer):
         while not self.quit:
             self.handle_request()
 
+        self.table.sync()
         logger.info("PRServer: stopping...")
         self.server_close()
         return

documentation/Makefile

@@ -200,7 +200,8 @@ TARFILES = mega-manual.html mega-style.css figures/yocto-environment.png figures
 	figures/yocto-environment-ref.png figures/user-configuration.png figures/source-input.png \
 	figures/package-feeds.png figures/layer-input.png figures/images.png figures/sdk.png \
 	figures/source-fetching.png figures/patching.png figures/configuration-compile-autoreconf.png \
-	figures/analysis-for-package-splitting.png
+	figures/analysis-for-package-splitting.png figures/image-generation.png \
+	figures/sdk-generation.png
 	endif
 
 MANUALS = $(DOC)/$(DOC).html
@@ -219,7 +220,8 @@ TARFILES = ref-manual.html ref-style.css figures/poky-title.png \
 	figures/user-configuration.png figures/yocto-environment-ref.png \
 	figures/images.png figures/sdk.png figures/source-fetching.png \
 	figures/patching.png figures/configuration-compile-autoreconf.png \
-	figures/analysis-for-package-splitting.png
+	figures/analysis-for-package-splitting.png figures/image-generation.png \
+	figures/sdk-generation.png
 MANUALS = $(DOC)/$(DOC).html $(DOC)/$(DOC).pdf $(DOC)/eclipse
 FIGURES = figures
 STYLESHEET = $(DOC)/*.css
@@ -381,8 +383,8 @@ publish:
             echo " "; \
             echo "******** Publishing "$(DOC)".html"; \
             echo " "; \
-            scp -r $(MANUALS) $(STYLESHEET) www.yoctoproject.org:/srv/www/www.yoctoproject.org-docs/$(VER)/$(DOC); \
-            cd $(DOC); scp -r $(FIGURES) www.yoctoproject.org:/srv/www/www.yoctoproject.org-docs/$(VER)/$(DOC); \
+            scp -r $(MANUALS) $(STYLESHEET) docs.yp:/var/www/www.yoctoproject.org-docs/$(VER)/$(DOC); \
+            cd $(DOC); scp -r $(FIGURES) docs.yp:/var/www/www.yoctoproject.org-docs/$(VER)/$(DOC); \
 	else \
           echo " "; \
           echo $(DOC)".html missing.  Generate the file first then try again."; \

documentation/adt-manual/adt-command.xml

@@ -8,23 +8,32 @@
     <para>
         Recall that earlier the manual discussed how to use an existing toolchain
         tarball that had been installed into the default installation
-        directory, <filename>/opt/poky</filename>, which is outside of the
+        directory, <filename>/opt/poky/&DISTRO;</filename>, which is outside of the
         <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>
         (see the section "<link linkend='using-an-existing-toolchain-tarball'>Using a Cross-Toolchain Tarball)</link>".
         And, that sourcing your architecture-specific environment setup script
         initializes a suitable cross-toolchain development environment.
-        During the setup, locations for the compiler, QEMU scripts, QEMU binary,
+    </para>
+
+    <para>
+        During this setup, locations for the compiler, QEMU scripts, QEMU binary,
         a special version of <filename>pkgconfig</filename> and other useful
         utilities are added to the <filename>PATH</filename> variable.
-        Variables to assist <filename>pkgconfig</filename> and <filename>autotools</filename>
-        are also defined so that,
-        for example, <filename>configure.sh</filename> can find pre-generated
-        test results for tests that need target hardware on which to run.
-        These conditions allow you to easily use the toolchain outside of the
-        OpenEmbedded build environment on both autotools-based projects and
-        Makefile-based projects.
+        Also, variables to assist
+        <filename>pkgconfig</filename> and <filename>autotools</filename>
+        are also defined so that, for example, <filename>configure.sh</filename>
+        can find pre-generated test results for tests that need target hardware
+        on which to run.
     </para>
 
+    <para>
+        Collectively, these conditions allow you to easily use the toolchain
+        outside of the OpenEmbedded build environment on both autotools-based
+        projects and Makefile-based projects.
+        This chapter provides information for both these types of projects.
+    </para>
+
+
 <section id='autotools-based-projects'>
 <title>Autotools-Based Projects</title>
 
@@ -179,7 +188,7 @@
                 If <filename>configure</filename> script results in problems recognizing the
                 <filename>--with-libtool-sysroot=&lt;sysroot-dir&gt;</filename> option,
                 regenerate the script to enable the support by doing the following and then
-                re-running the script:
+                run the script again:
                 <literallayout class='monospaced'>
      $ libtoolize --automake
      $ aclocal -I ${OECORE_NATIVE_SYSROOT}/usr/share/aclocal \

documentation/adt-manual/adt-intro.xml

@@ -22,8 +22,8 @@
     and an introduction to the <trademark class='trade'>Eclipse</trademark> IDE
     Yocto Plug-in.
     <note>
-        The ADT is distribution-neutral and does not require the Yocto 
-        Project reference distribution, which is called Poky.  
+        The ADT is distribution-neutral and does not require the Yocto
+        Project reference distribution, which is called Poky.
         This manual, however, uses examples that use the Poky distribution.
     </note>
 </para>
@@ -43,7 +43,7 @@
         <itemizedlist>
             <listitem><para>An architecture-specific cross-toolchain and matching
                 sysroot both built by the OpenEmbedded build system.
-                The toolchain and sysroot are based on a 
+                The toolchain and sysroot are based on a
                 <ulink url='&YOCTO_DOCS_DEV_URL;#metadata'>Metadata</ulink>
                 configuration and extensions,
                 which allows you to cross-develop on the host machine for the target hardware.
@@ -149,8 +149,7 @@
                     that causes skips in audio,
                     stutters in your desktop experience, or situations that overload your server
                     even when you have plenty of CPU power left.
-                    You can find out more about LatencyTOP at
-                    <ulink url='https://latencytop.org/'></ulink>.</para></listitem>
+                    </para></listitem>
                 <listitem><para><emphasis>PowerTOP:</emphasis> Helps you determine what
                     software is using the most power.
                     You can find out more about PowerTOP at

documentation/adt-manual/adt-manual.xml

@@ -63,9 +63,29 @@
             </revision>
             <revision>
                 <revnumber>1.5</revnumber>
-                <date>Sometime in 2013</date>
+                <date>October 2013</date>
                 <revremark>Released with the Yocto Project 1.5 Release.</revremark>
             </revision>
+            <revision>
+                <revnumber>1.5.1</revnumber>
+                <date>January 2014</date>
+                <revremark>Released with the Yocto Project 1.5.1 Release.</revremark>
+            </revision>
+            <revision>
+                <revnumber>1.5.2</revnumber>
+                <date>May 2014</date>
+                <revremark>Released with the Yocto Project 1.5.2 Release.</revremark>
+            </revision>
+            <revision>
+                <revnumber>1.5.3</revnumber>
+                <date>July 2014</date>
+                <revremark>Released with the Yocto Project 1.5.3 Release.</revremark>
+            </revision>
+            <revision>
+                <revnumber>1.5.4</revnumber>
+                <date>December 2014</date>
+                <revremark>Released with the Yocto Project 1.5.4 Release.</revremark>
+            </revision>
        </revhistory>
 
     <copyright>
@@ -79,11 +99,10 @@
         the terms of the <ulink type="http" url="http://creativecommons.org/licenses/by-sa/2.0/uk/">Creative Commons Attribution-Share Alike 2.0 UK: England &amp; Wales</ulink> as published by Creative Commons.
       </para>
       <note>
-          Due to production processes, there could be differences between the Yocto Project
-          documentation bundled in the release tarball and the
-          <ulink url='&YOCTO_DOCS_ADT_URL;'>Yocto Project Application Developer's Guide</ulink> on
-          the <ulink url='&YOCTO_HOME_URL;'>Yocto Project</ulink> website.
-          For the latest version of this manual, see the manual on the website.
+          For the latest version of this manual associated with this
+          Yocto Project release, see the
+          <ulink url='&YOCTO_DOCS_ADT_URL;'>Yocto Project Application Developer's Guide</ulink>
+          from the Yocto Project website.
       </note>
 
     </legalnotice>

documentation/adt-manual/adt-prepare.xml

@@ -80,37 +80,48 @@
 
             <para>
                 The ADT Installer is contained in the ADT Installer tarball.
-                You can download the tarball into any directory from the
-                <ulink url='&YOCTO_DL_URL;/releases'>Index of Releases</ulink>, specifically
-                at
-                <ulink url='&YOCTO_ADTINSTALLER_DL_URL;'></ulink>.
-                Or, you can use BitBake to generate the tarball inside the existing
-                <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>.
-            </para>
-
-            <para>
-                If you use BitBake to generate the ADT Installer tarball, you must
-                <filename>source</filename> the environment setup script
-                (<ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink>)
-                located in the Source Directory before running the
-                BitBake command that creates the tarball.
-            </para>
-
-            <para>
-                The following example commands download the Poky tarball, set up the
-                <ulink url='&YOCTO_DOCS_DEV_URL;#source-directory'>Source Directory</ulink>,
-                set up the environment while also creating the default Build Directory,
-                and run the BitBake command that results in the tarball
-                <filename>~/yocto-project/build/tmp/deploy/sdk/adt_installer.tar.bz2</filename>:
-                <literallayout class='monospaced'>
+                You can get the tarball using either of these methods:
+                <itemizedlist>
+                    <listitem><para><emphasis>Download the Tarball:</emphasis>
+                        You can download the tarball from
+                        <ulink url='&YOCTO_ADTINSTALLER_DL_URL;'></ulink> into
+                        any directory.</para></listitem>
+                    <listitem><para><emphasis>Build the Tarball:</emphasis>
+                        You can use BitBake to generate the tarball inside an
+                        existing
+                        <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>.
+                        </para>
+                        <para>If you use BitBake to generate the ADT Installer
+                        tarball, you must <filename>source</filename> the
+                        environment setup script
+                        (<ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink>
+                        or
+                        <ulink url='&YOCTO_DOCS_REF_URL;#structure-memres-core-script'><filename>oe-init-build-env-memres</filename></ulink>)
+                        located in the Source Directory before running the
+                        BitBake command that creates the tarball.</para>
+                        <para>The following example commands establish
+                        the
+                        <ulink url='&YOCTO_DOCS_DEV_URL;#source-directory'>Source Directory</ulink>,
+                        check out the current release branch, set up the
+                        build environment while also creating the default
+                        Build Directory, and run the BitBake command that
+                        results in the tarball
+                        <filename>poky/build/tmp/deploy/sdk/adt_installer.tar.bz2</filename>:
+                        <note>
+                            Before using BitBake to build the ADT tarball, be
+                            sure to make sure your
+                            <filename>local.conf</filename> file is properly
+                            configured.
+                        </note>
+                        <literallayout class='monospaced'>
      $ cd ~
-     $ mkdir yocto-project
-     $ cd yocto-project
-     $ wget &YOCTO_RELEASE_DL_URL;/&YOCTO_POKY_TARBALL;
-     $ tar xjf &YOCTO_POKY_TARBALL;
-     $ source &OE_INIT_PATH;
+     $ git clone git://git.yoctoproject.org/poky
+     $ cd poky
+     $ git checkout -b &DISTRO_NAME; origin/&DISTRO_NAME;
+     $ source &OE_INIT_FILE;
      $ bitbake adt-installer
-                </literallayout>
+                        </literallayout></para></listitem>
+                </itemizedlist>
             </para>
         </section>
 
@@ -125,7 +136,7 @@
                 a top-level directory named <filename>adt-installer</filename>:
                 <literallayout class='monospaced'>
      $ cd ~
-     $ cp ~/poky/build/tmp/deploy/sdk/adt_installer.tar.bz2 $HOME
+     $ cp poky/build/tmp/deploy/sdk/adt_installer.tar.bz2 $HOME
      $ tar -xjf adt_installer.tar.bz2
                 </literallayout>
                 Unpacking it creates the directory <filename>adt-installer</filename>,
@@ -190,12 +201,9 @@
 
             <para>
                 After you have configured the <filename>adt_installer.conf</filename> file,
-                run the installer using the following command.
-                Be sure that you are not trying to use cross-compilation tools.
-                When you run the installer, the environment must use a
-                host <filename>gcc</filename>:
+                run the installer using the following command:
                 <literallayout class='monospaced'>
-     $ cd ~/adt-installer
+     $ cd adt-installer
      $ ./adt_installer
                 </literallayout>
                 Once the installer begins to run, you are asked to enter the
@@ -266,7 +274,7 @@
                             target, go into the <filename>x86_64</filename>
                             folder and download the following installer:
                             <literallayout class='monospaced'>
-     poky-eglibc-x86_64-core-image-sato-i586.sh
+     poky-eglibc-x86_64-core-image-sato-i586-toolchain-&DISTRO;.sh
                             </literallayout></para></listitem>
                         <listitem><para>Build your own toolchain installer.
                             For cases where you cannot use an installer
@@ -277,27 +285,32 @@
                     </itemizedlist></para></listitem>
                 <listitem><para>Once you have the installer, run it to install
                     the toolchain.
-                    You must change the permissions on the toolchain installer
-                    script so that it is executable.</para>
+                    <note>
+                        You must change the permissions on the toolchain
+                        installer script so that it is executable.
+                    </note></para>
                     <para>The following command shows how to run the installer
                     given a toolchain tarball for a 64-bit x86 development host
                     system and a 32-bit x86 target architecture.
                     The example assumes the toolchain installer is located
                     in <filename>~/Downloads/</filename>.
                     <literallayout class='monospaced'>
-     $ ~/Downloads/poky-eglibc-x86_64-core-image-sato-i586.sh
+     $ ~/Downloads/poky-eglibc-x86_64-core-image-sato-i586-toolchain-&DISTRO;.sh
                     </literallayout>
-                    <note>
-                        If you do not have write permissions for the directory
-                        into which you are installing the toolchain, the
-                        toolchain installer notifies you and exits.
-                        Be sure you have write permissions in the directory and
-                        run the installer again.
-                    </note>
-                    Once the tarball is expanded, the cross-toolchain is
+                    The first thing the installer prompts you for is the
+                    directory into which you want to install the toolchain.
+                    The default directory used is
+                    <filename>/opt/poky/&DISTRO;</filename>.
+                    If you do not have write permissions for the directory
+                    into which you are installing the toolchain, the
+                    toolchain installer notifies you and exits.
+                    Be sure you have write permissions in the directory and
+                    run the installer again.</para>
+                    <para>When the script finishes, the cross-toolchain is
                     installed.
                     You will notice environment setup files for the
-                    cross-toolchain in the directory.</para></listitem>
+                    cross-toolchain in the installation directory.
+                    </para></listitem>
             </orderedlist>
         </para>
     </section>
@@ -318,37 +331,43 @@
         <para>
             Follow these steps to generate the toolchain into the Build Directory:
             <orderedlist>
-                <listitem><para>Source the environment setup script
-                    <filename>&OE_INIT_FILE;</filename> located in the
+                <listitem><para><emphasis>Set up the Build Environment:</emphasis>
+                    Source the OpenEmbedded build environment setup
+                    script (i.e.
+                    <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink>
+                    or
+                    <ulink url='&YOCTO_DOCS_REF_URL;#structure-memres-core-script'><filename>oe-init-build-env-memres</filename></ulink>)
+                    located in the
                     <ulink url='&YOCTO_DOCS_DEV_URL;#source-directory'>Source Directory</ulink>.
                     </para></listitem>
-                <listitem><para>At this point, you should be sure that the
+                <listitem><para><emphasis>Check your Local Configuration File:</emphasis>
+                    At this point, you should be sure that the
                     <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink> variable
                     in the <filename>local.conf</filename> file found in the
                     <filename>conf</filename> directory of the Build Directory
                     is set for the target architecture.
-                    Comments within the <filename>local.conf</filename> file list the values you
-                    can use for the <filename>MACHINE</filename> variable.
-                    <note>You can populate the Build Directory with the cross-toolchains for more
-                        than a single architecture.
-                        You just need to edit the <filename>MACHINE</filename> variable in the
-                        <filename>local.conf</filename> file and re-run the BitBake
-                        command.</note></para></listitem>
-                <listitem><para>Run <filename>bitbake meta-ide-support</filename> to complete the
-                    cross-toolchain generation.
-                    <note>If you change out of your working directory after you
-                        <filename>source</filename> the environment setup script and before you run
-                        the BitBake command, the command might not work.
-                        Be sure to run the BitBake command immediately
-                        after checking or editing the <filename>local.conf</filename> but without
-                        changing out of your working directory.</note>
-                    Once the BitBake command finishes,
-                    the cross-toolchain is generated and populated within the Build Directory.
-                    You will notice environment setup files for the cross-toolchain in the
-                    Build Directory in the <filename>tmp</filename> directory.
-                    Setup script filenames contain the strings <filename>environment-setup</filename>.</para>
-                    <para>Be aware that when you use this method to install the toolchain you still need
-                    to separately extract and install the sysroot filesystem.
+                    Comments within the <filename>local.conf</filename> file
+                    list the values you can use for the
+                    <filename>MACHINE</filename> variable.
+                    <note>
+                        You can populate the Build Directory with the
+                        cross-toolchains for more than a single architecture.
+                        You just need to edit the <filename>MACHINE</filename>
+                        variable in the <filename>local.conf</filename> file and
+                        re-run the BitBake command.
+                    </note></para></listitem>
+                <listitem><para><emphasis>Generate the Cross-Toolchain:</emphasis>
+                    Run <filename>bitbake meta-ide-support</filename> to
+                    complete the cross-toolchain generation.
+                    Once the BitBake command finishes, the cross-toolchain is
+                    generated and populated within the Build Directory.
+                    You will notice environment setup files for the
+                    cross-toolchain that contain the string
+                    "<filename>environment-setup</filename>" in the
+                    Build Directory's <filename>tmp</filename> folder.</para>
+                    <para>Be aware that when you use this method to install the
+                    toolchain, you still need to separately extract and install
+                    the sysroot filesystem.
                     For information on how to do this, see the
                     "<link linkend='extracting-the-root-filesystem'>Extracting the Root Filesystem</link>" section.
                     </para></listitem>
@@ -374,10 +393,11 @@
     </para>
 
     <para>
-        Be sure to run the environment setup script that matches the architecture for
-        which you are developing.
-        Environment setup scripts begin with the string "<filename>environment-setup</filename>"
-        and include as part of their name the architecture.
+        Be sure to run the environment setup script that matches the
+        architecture for which you are developing.
+        Environment setup scripts begin with the string
+        "<filename>environment-setup</filename>" and include as part of their
+        name the architecture.
         For example, the toolchain environment setup script for a 64-bit
         IA-based architecture installed in the default installation directory
         would be the following:
@@ -434,8 +454,8 @@
             application from within the
             Eclipse IDE, you must have an image that contains the Yocto Target Communication
             Framework (TCF) agent (<filename>tcf-agent</filename>).
-            By default, the Yocto Project provides only one type pre-built image that contains the
-            <filename>tcf-agent</filename>.
+            By default, the Yocto Project provides only one type of pre-built
+            image that contains the <filename>tcf-agent</filename>.
             And, those images are SDK (e.g.<filename>core-image-sato-sdk</filename>).
         </para>
 
@@ -464,13 +484,16 @@
                             the following commands:
                             <literallayout class='monospaced'>
      $ git clone http://git.eclipse.org/gitroot/tcf/org.eclipse.tcf.agent.git
-     $ cd agent
+     $ cd org.eclipse.tcf.agent/agent
                             </literallayout></para></listitem>
-                        <listitem><para>Modify the <filename>Makefile.inc</filename> file
+                        <listitem><para>Locate the
+                            <filename>Makefile.inc</filename> file inside the
+                            <filename>agent</filename> folder and modify it
                             for the cross-compilation environment by setting the
                             <filename>OPSYS</filename> and
                             <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>
-                            variables according to your target.</para></listitem>
+                            variables according to your target.
+                            </para></listitem>
                         <listitem><para>Use the cross-development tools to build the
                             <filename>tcf-agent</filename>.
                             Before you "Make" the file, be sure your cross-tools are set up first.
@@ -490,32 +513,63 @@
         <title>Extracting the Root Filesystem</title>
 
         <para>
-            You must extract the root filesystem if you want to boot the image using NFS
-            or you want to use the root filesystem as the target sysroot.
-            For example, the Eclipse IDE environment with the Eclipse Yocto Plug-in installed allows you
-            to use QEMU to boot under NFS.
-            Another example is if you want to develop your target application using the
-            root filesystem as the target sysroot.
+            If you install your toolchain by hand or build it using BitBake and
+            you need a root filesystem, you need to extract it separately.
+            If you use the ADT Installer to install the ADT, the root
+            filesystem is automatically extracted and installed.
+        </para>
+
+        <para>
+            Here are some cases where you need to extract the root filesystem:
+            <itemizedlist>
+                <listitem><para>You want to boot the image using NFS.
+                    </para></listitem>
+                <listitem><para>You want to use the root filesystem as the
+                    target sysroot.
+                    For example, the Eclipse IDE environment with the Eclipse
+                    Yocto Plug-in installed allows you to use QEMU to boot
+                    under NFS.</para></listitem>
+                <listitem><para>You want to develop your target application
+                    using the root filesystem as the target sysroot.
+                    </para></listitem>
+            </itemizedlist>
         </para>
 
         <para>
             To extract the root filesystem, first <filename>source</filename>
-            the cross-development environment setup script and then
-            use the <filename>runqemu-extract-sdk</filename> command on the
+            the cross-development environment setup script.
+            If you built the toolchain in the Build Directory, you will find
+            the toolchain environment script in the
+            <filename>tmp</filename> directory.
+            If you installed the toolchain by hand, the environment setup
+            script is located in <filename>/opt/poky/&DISTRO;</filename>.
+        </para>
+
+        <para>
+            After sourcing the environment script, use the
+            <filename>runqemu-extract-sdk</filename> command and provide the
             filesystem image.
-            For example, the following commands set up the environment and then extract
-            the root filesystem from a previously built filesystem image tarball named
-            <filename>core-image-sato-sdk-qemux86-2011091411831.rootfs.tar.bz2</filename>.
-            The example extracts the root filesystem into the <filename>$HOME/qemux86-sato</filename>
-            directory:
+        </para>
+
+        <para>
+            Following is an example.
+            The second command sets up the environment.
+            In this case, the setup script is located in the
+            <filename>/opt/poky/&DISTRO;</filename> directory.
+            The third command extracts the root filesystem from a previously
+            built filesystem that is located in the
+            <filename>~/Downloads</filename> directory.
+            Furthermore, this command extracts the root filesystem into the
+            <filename>qemux86-sato</filename> directory:
             <literallayout class='monospaced'>
-     $ source $HOME/toolchain_dir/environment-setup-i586-poky-linux
+     $ cd ~
+     $ source /opt/poky/&DISTRO;/environment-setup-i586-poky-linux
      $ runqemu-extract-sdk \
-        ~Downloads/core-image-sato-sdk-qemux86-2011091411831.rootfs.tar.bz2 \
+        ~/Downloads/core-image-sato-sdk-qemux86-2011091411831.rootfs.tar.bz2 \
         $HOME/qemux86-sato
             </literallayout>
-            In this case, you could now point to the target sysroot at
-            <filename>$HOME/qemux86-sato</filename>.
+            You could now point to the target sysroot at
+            <filename>qemux86-sato</filename>.
         </para>
     </section>
 </section>
@@ -525,32 +579,32 @@
 
     <para>
         As an alternative to locating and downloading a toolchain installer,
-        you can build the toolchain installer if you have a
-        <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>.
+        you can build the toolchain installer one of two ways if you have a
+        <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>:
+        <itemizedlist>
+            <listitem><para>Use <filename>bitbake meta-toolchain</filename>.
+                This method requires you to still install the target
+                sysroot by installing and extracting it separately.
+                For information on how to install the sysroot, see the
+                "<link linkend='extracting-the-root-filesystem'>Extracting the Root Filesystem</link>"
+                section.</para></listitem>
+            <listitem><para>Use
+                <filename>bitbake image -c populate_sdk</filename>.
+                This method has significant advantages over the previous method
+                because it results in a toolchain installer that contains the
+                sysroot that matches your target root filesystem.
+                </para></listitem>
+        </itemizedlist>
     </para>
 
     <para>
-        You can build the toolchain
-        installer using <filename>bitbake meta-toolchain</filename>.
-        This method requires you to still install the target
-        sysroot by installing and extracting it separately.
-        For information on how to install the sysroot, see the
-        "<link linkend='extracting-the-root-filesystem'>Extracting the Root Filesystem</link>" section.
-    </para>
-
-    <para>
-        A final method of building the toolchain installer exists that has
-        significant advantages over the previous two methods.
-        This method results in a toolchain installer that contains the sysroot
-        that matches your target root filesystem.
-        To build this installer, use the
-        <filename>bitbake image -c populate_sdk</filename> command.
-    </para>
-
-    <para>
-         Remember, before using any <filename>bitbake</filename> command, you
-         must source the <filename>&OE_INIT_PATH;</filename> script located in
-         the Source Directory and you must make sure your
+         Remember, before using any BitBake command, you
+         must source the build environment setup script
+         (i.e.
+         <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink>
+         or
+         <ulink url='&YOCTO_DOCS_REF_URL;#structure-memres-core-script'><filename>oe-init-build-env-memres</filename></ulink>)
+         located in the Source Directory and you must make sure your
          <filename>conf/local.conf</filename> variables are correct.
          In particular, you need to be sure the
          <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>
@@ -560,12 +614,27 @@
          variable is correctly set if you are building a toolchain designed to
          run on an architecture that differs from your current development host
          machine (i.e. the build machine).
-     </para>
+    </para>
 
-     <para>
-         When the BitBake command completes, the toolchain installer will be in
-         <filename>tmp/deploy/sdk</filename> in the Build Directory.
-     </para>
+    <para>
+        When the BitBake command completes, the toolchain installer will be in
+        <filename>tmp/deploy/sdk</filename> in the Build Directory.
+        <note>
+            By default, this toolchain does not build static binaries.
+            If you want to use the toolchain to build these types of libraries,
+            you need to be sure your image has the appropriate static
+            development libraries.
+            Use the
+            <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'><filename>IMAGE_INSTALL</filename></ulink>
+            variable inside your <filename>local.conf</filename> file to
+            install the appropriate library packages.
+            Following is an example using <filename>eglibc</filename> static
+            development libraries:
+            <literallayout class='monospaced'>
+     IMAGE_INSTALL_append = " eglibc-staticdev"
+            </literallayout>
+        </note>
+    </para>
 </section>
 
 </chapter>

documentation/bsp-guide/bsp-guide.xml

@@ -75,9 +75,29 @@
             </revision>
             <revision>
                 <revnumber>1.5</revnumber>
-                <date>Sometime in 2013</date>
+                <date>October 2013</date>
                 <revremark>Released with the Yocto Project 1.5 Release.</revremark>
             </revision>
+            <revision>
+                <revnumber>1.5.1</revnumber>
+                <date>January 2014</date>
+                <revremark>Released with the Yocto Project 1.5.1 Release.</revremark>
+            </revision>
+            <revision>
+                <revnumber>1.5.2</revnumber>
+                <date>May 2014</date>
+                <revremark>Released with the Yocto Project 1.5.2 Release.</revremark>
+            </revision>
+            <revision>
+                <revnumber>1.5.3</revnumber>
+                <date>July 2014</date>
+                <revremark>Released with the Yocto Project 1.5.3 Release.</revremark>
+            </revision>
+            <revision>
+                <revnumber>1.5.4</revnumber>
+                <date>December 2014</date>
+                <revremark>Released with the Yocto Project 1.5.4 Release.</revremark>
+            </revision>
         </revhistory>
 
     <copyright>
@@ -91,11 +111,10 @@
         the terms of the <ulink type="http" url="http://creativecommons.org/licenses/by-nc-sa/2.0/uk/">Creative Commons Attribution-Non-Commercial-Share Alike 2.0 UK: England &amp; Wales</ulink> as published by Creative Commons.
       </para>
       <note>
-          Due to production processes, there could be differences between the Yocto Project
-          documentation bundled in the release tarball and the
-          <ulink url='&YOCTO_DOCS_BSP_URL;'>Yocto Project Board Support Package (BSP) Developer's Guide</ulink> on
-          the <ulink url='&YOCTO_HOME_URL;'>Yocto Project</ulink> website.
-          For the latest version of this manual, see the manual on the website.
+          For the latest version of this manual associated with this
+          Yocto Project release, see the
+          <ulink url='&YOCTO_DOCS_BSP_URL;'>Yocto Project Board Support Package (BSP) Developer's Guide</ulink>
+          from the Yocto Project website.
       </note>
     </legalnotice>
 

documentation/bsp-guide/bsp.xml

@@ -183,17 +183,15 @@
      meta-crownbay/recipes-graphics/xorg-xserver/xserver-xf86-config/
      meta-crownbay/recipes-graphics/xorg-xserver/xserver-xf86-config/crownbay/
      meta-crownbay/recipes-graphics/xorg-xserver/xserver-xf86-config/crownbay/xorg.conf
-     meta-crownbay/recipes-graphics/xorg-xserver/xserver-xf86-config/crownbay-noemgd/
-     meta-crownbay/recipes-graphics/xorg-xserver/xserver-xf86-config/crownbay-noemgd/xorg.conf
      meta-crownbay/recipes-kernel/
      meta-crownbay/recipes-kernel/linux/
-     meta-crownbay/recipes-kernel/linux/linux-yocto_3.2.bbappend
      meta-crownbay/recipes-kernel/linux/linux-yocto_3.4.bbappend
      meta-crownbay/recipes-kernel/linux/linux-yocto_3.8.bbappend
+     meta-crownbay/recipes-kernel/linux/linux-yocto_3.10.bbappend
      meta-crownbay/recipes-kernel/linux/linux-yocto-dev.bbappend
-     meta-crownbay/recipes-kernel/linux/linux-yocto-rt_3.2.bbappend
      meta-crownbay/recipes-kernel/linux/linux-yocto-rt_3.4.bbappend
      meta-crownbay/recipes-kernel/linux/linux-yocto-rt_3.8.bbappend
+     meta-crownbay/recipes-kernel/linux/linux-yocto-rt_3.10.bbappend
                 </literallayout>
             </para>
 
@@ -376,15 +374,6 @@
             <para>
                 Each BSP Layer requires at least one machine file.
                 However, you can supply more than one file.
-                For example, in the Crown Bay BSP shown earlier in this section, the
-                <filename>conf/machine</filename> directory contains two configuration files:
-                <filename>crownbay.conf</filename> and <filename>crownbay-noemgd.conf</filename>.
-                The <filename>crownbay.conf</filename> file is used for the Crown Bay BSP
-                that supports the <trademark class='registered'>Intel</trademark> Embedded
-                Media and Graphics Driver (<trademark class='registered'>Intel</trademark>
-                EMGD), while the <filename>crownbay-noemgd</filename> file is used for the
-                Crown Bay BSP that supports Video Electronics Standards Association (VESA)
-                graphics only.
             </para>
 
             <para>
@@ -405,12 +394,13 @@
 
             <para>
                 To use an include file, you simply include them in the machine configuration file.
-                For example, the Crown Bay BSP <filename>crownbay.conf</filename> has the
+                For example, the Crown Bay BSP <filename>crownbay.conf</filename> contains the
                 following statements:
                 <literallayout class='monospaced'>
      require conf/machine/include/tune-atom.inc
      require conf/machine/include/ia32-base.inc
      require conf/machine/include/meta-intel.inc
+     require conf/machine/include/meta-intel-emgd.inc
                 </literallayout>
             </para>
             </section>
@@ -428,15 +418,13 @@
                 This optional directory contains miscellaneous recipe files for the BSP.
                 Most notably would be the formfactor files.
                 For example, in the Crown Bay BSP there is the
-                <filename>formfactor_0.0.bbappend</filename> file, which is an append file used
-                to augment the recipe that starts the build.
-                Furthermore, there are machine-specific settings used during the build that are
-                defined by the <filename>machconfig</filename> files.
-                In the Crown Bay example, two <filename>machconfig</filename> files exist:
-                one that supports the
-                <trademark class='registered'>Intel</trademark> Embedded
-                Media and Graphics Driver (<trademark class='registered'>Intel</trademark>
-                EMGD) and one that does not:
+                <filename>formfactor_0.0.bbappend</filename> file, which is an
+                append file used to augment the recipe that starts the build.
+                Furthermore, there are machine-specific settings used during the
+                build that are defined by the <filename>machconfig</filename> file.
+                In the Crown Bay example, two <filename>machconfig</filename> files
+                exist: one that supports the Intel® Embedded Media and Graphics
+                Driver (Intel® EMGD) and one that does not:
                 <literallayout class='monospaced'>
      meta-crownbay/recipes-bsp/formfactor/formfactor/crownbay/machconfig
      meta-crownbay/recipes-bsp/formfactor/formfactor/crownbay-noemgd/machconfig
@@ -467,16 +455,13 @@
                 This optional directory contains recipes for the BSP if it has
                 special requirements for graphics support.
                 All files that are needed for the BSP to support a display are kept here.
-                For example, the Crown Bay BSP contains two versions of the
-                <filename>xorg.conf</filename> file.
-                The version in <filename>crownbay</filename> builds a BSP that supports the
-                <trademark class='registered'>Intel</trademark> Embedded Media Graphics Driver (EMGD),
-                while the version in <filename>crownbay-noemgd</filename> builds
-                a BSP that supports Video Electronics Standards Association (VESA) graphics only:
+                For example, the Crown Bay BSP's <filename>xorg.conf</filename> file
+                detects the graphics support needed (i.e. the Intel® Embedded Media
+                Graphics Driver (EMGD) or the Video Electronics Standards Association
+                (VESA) graphics):
                 <literallayout class='monospaced'>
      meta-crownbay/recipes-graphics/xorg-xserver/xserver-xf86-config_0.1.bbappend
      meta-crownbay/recipes-graphics/xorg-xserver/xserver-xf86-config/crownbay/xorg.conf
-     meta-crownbay/recipes-graphics/xorg-xserver/xserver-xf86-config/crownbay-noemgd/xorg.conf
                 </literallayout>
             </para>
             </section>
@@ -502,28 +487,28 @@
                 the <filename>meta-&lt;bsp_name&gt;/recipes-kernel/linux</filename> directory).
             </para>
             <para>
-                Suppose you are using the <filename>linux-yocto_3.8.bb</filename> recipe to build
+                Suppose you are using the <filename>linux-yocto_3.10.bb</filename> recipe to build
                 the kernel.
                 In other words, you have selected the kernel in your
                 <filename>&lt;bsp_name&gt;.conf</filename> file by adding these types
                 of statements:
                 <literallayout class='monospaced'>
      PREFERRED_PROVIDER_virtual/kernel ?= "linux-yocto"
-     PREFERRED_VERSION_linux-yocto ?= "3.8%"
+     PREFERRED_VERSION_linux-yocto ?= "3.10%"
                 </literallayout>
                 <note>
                     When the preferred provider is assumed by default, the
                     <filename>PREFERRED_PROVIDER</filename> statement does not appear in the
                     <filename>&lt;bsp_name&gt;.conf</filename> file.
                 </note>
-                You would use the <filename>linux-yocto_3.8.bbappend</filename> file to append
+                You would use the <filename>linux-yocto_3.10.bbappend</filename> file to append
                 specific BSP settings to the kernel, thus configuring the kernel for your particular BSP.
             </para>
             <para>
                 As an example, look at the existing Crown Bay BSP.
                 The append file used is:
                 <literallayout class='monospaced'>
-     meta-crownbay/recipes-kernel/linux/linux-yocto_3.8.bbappend
+     meta-crownbay/recipes-kernel/linux/linux-yocto_3.10.bbappend
                 </literallayout>
                 The following listing shows the file.
                 Be aware that the actual commit ID strings in this example listing might be different
@@ -532,46 +517,18 @@
                 <literallayout class='monospaced'>
      FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
 
-     COMPATIBLE_MACHINE_crownbay = "crownbay"
-     KMACHINE_crownbay = "crownbay"
-     KBRANCH_crownbay = "standard/crownbay"
-     KERNEL_FEATURES_crownbay_append = " features/drm-emgd/drm-emgd-1.16 cfg/vesafb"
-
      COMPATIBLE_MACHINE_crownbay-noemgd = "crownbay-noemgd"
      KMACHINE_crownbay-noemgd = "crownbay"
      KBRANCH_crownbay-noemgd = "standard/crownbay"
-     KERNEL_FEATURES_crownbay-noemgd_append = " cfg/vesafb"
+     KERNEL_FEATURES_append_crownbay-noemgd = " cfg/vesafb"
 
-     LINUX_VERSION = "3.8.4"
+     LINUX_VERSION = "3.10.11"
 
-     SRCREV_meta_crownbay = "2a6d36e75ca0a121570a389d7bab76ec240cbfda"
-     SRCREV_machine_crownbay = "47aed0c17c1c55988198ad39f86ae88894c8e0a4"
-     SRCREV_emgd_crownbay = "c780732f175ff0ec866fac2130175876b519b576"
-
-     SRCREV_meta_crownbay-noemgd = "2a6d36e75ca0a121570a389d7bab76ec240cbfda"
-     SRCREV_machine_crownbay-noemgd = "47aed0c17c1c55988198ad39f86ae88894c8e0a4"
-
-     SRC_URI_crownbay = "git://git.yoctoproject.org/linux-yocto-3.8.git;protocol=git;nocheckout=1;branch=${KBRANCH},${KMETA},emgd-1.16;name=machine,meta,emgd"
+     SRCREV_meta_crownbay-noemgd = "285f93bf942e8f6fa678ffc6cc53696ed5400718"
+     SRCREV_machine_crownbay-noemgd = "702040ac7c7ec66a29b4d147665ccdd0ff015577"
                 </literallayout>
-                This append file contains statements used to support the Crown Bay BSP for both
-                <trademark class='registered'>Intel</trademark> EMGD and the VESA graphics.
-                The build process, in this case, recognizes and uses only the statements that
-                apply to the defined machine name - <filename>crownbay</filename> in this case.
-                So, the applicable statements in the <filename>linux-yocto_3.8.bbappend</filename>
-                file are follows:
-                <literallayout class='monospaced'>
-     FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
-
-     COMPATIBLE_MACHINE_crownbay = "crownbay"
-     KMACHINE_crownbay = "crownbay"
-     KBRANCH_crownbay = "standard/crownbay"
-     KERNEL_FEATURES_crownbay_append = " features/drm-emgd/drm-emgd-1.16 cfg/vesafb"
-
-     SRCREV_meta_crownbay = "2a6d36e75ca0a121570a389d7bab76ec240cbfda"
-     SRCREV_machine_crownbay = "47aed0c17c1c55988198ad39f86ae88894c8e0a4"
-     SRCREV_emgd_crownbay = "c780732f175ff0ec866fac2130175876b519b576"
-                </literallayout>
-                The append file defines <filename>crownbay</filename> as the
+                This append file contains statements used to support the Crown Bay BSP.
+                The file defines <filename>crownbay</filename> as the
                 <ulink url='&YOCTO_DOCS_REF_URL;#var-COMPATIBLE_MACHINE'><filename>COMPATIBLE_MACHINE</filename></ulink>
                 and uses the
                 <ulink url='&YOCTO_DOCS_REF_URL;#var-KMACHINE'><filename>KMACHINE</filename></ulink> variable to
@@ -588,12 +545,6 @@
                 <ulink url='&YOCTO_DOCS_DEV_URL;#source-directory'>Source Directory</ulink> Git
                 repository and the <filename>meta</filename> Git repository branches to identify the
                 exact kernel needed to build the Crown Bay BSP.
-                <note>
-                    For <filename>crownbay</filename>, a specific commit is also needed to point
-                    to the branch that supports EMGD graphics.
-                    At a minimum, every BSP points to the
-                    <filename>machine</filename> and <filename>meta</filename> commits.
-                </note>
             </para>
 
             <para>
@@ -611,7 +562,7 @@
             <para>
                 For example, suppose you had some configuration options in a file called
                 <filename>network_configs.cfg</filename>.
-                You can place that file inside a directory named <filename>/linux-yocto</filename> and then add
+                You can place that file inside a directory named <filename>linux-yocto</filename> and then add
                 a <filename>SRC_URI</filename> statement such as the following to the append file.
                 When the OpenEmbedded build system builds the kernel, the configuration options are
                 picked up and applied.
@@ -797,7 +748,7 @@
                                 <listitem><para>Instructions on how to boot the BSP build from
                                     the BSP layer.</para></listitem>
                                 <listitem><para>Instructions on how to boot the binary images
-                                    contained in the <filename>/binary</filename> directory,
+                                    contained in the <filename>binary</filename> directory,
                                     if present.</para></listitem>
                                 <listitem><para>Information on any known bugs or issues that users
                                     should know about when either building or booting the BSP
@@ -808,7 +759,7 @@
                             <filename>meta-&lt;bsp_name&gt;</filename> directory.
                             This file specifies exactly where you can find the sources used to
                             generate the binary images contained in the
-                            <filename>/binary</filename> directory, if present.
+                            <filename>binary</filename> directory, if present.
                             See the
                             <ulink url='&YOCTO_GIT_URL;/cgit.cgi/meta-intel/tree/meta-fri2/README.sources'><filename>README.sources</filename></ulink>
                             file for the Fish River Island 2 BSP in the <filename>meta-fri2</filename> BSP layer
@@ -1281,31 +1232,35 @@
                     Following is the complete example:
                     <literallayout class='monospaced'>
      $ yocto-bsp create myarm qemu
+     Checking basic git connectivity...
+     Done.
+
      Which qemu architecture would you like to use? [default: i386]
-	     1) i386    (32-bit)
+     	     1) i386    (32-bit)
 	     2) x86_64  (64-bit)
 	     3) ARM     (32-bit)
 	     4) PowerPC (32-bit)
 	     5) MIPS    (32-bit)
      3
-     Would you like to use the default (3.8) kernel? (y/n) [default: y]
+     Would you like to use the default (3.10) kernel? (y/n) [default: y] y
      Do you need a new machine branch for this BSP (the alternative is to re-use an existing branch)? [y/n] [default: y]
-     Getting branches from remote repo git://git.yoctoproject.org/linux-yocto-3.8.git...
+     Getting branches from remote repo git://git.yoctoproject.org/linux-yocto-3.10.git...
      Please choose a machine branch to base your new BSP branch on: [default: standard/base]
-             1) standard/arm-versatile-926ejs
-             2) standard/base
-             3) standard/beagleboard
-             4) standard/ck
-             5) standard/crownbay
-             6) standard/edf
-             7) standard/emenlow
-             8) standard/fri2
-             9) standard/fsl-mpc8315e-rdb
-             10) standard/mti-malta32
-             11) standard/mti-malta64
-             12) standard/qemuppc
-             13) standard/routerstationpro
-             14) standard/sys940x
+	     1) standard/arm-versatile-926ejs
+	     2) standard/base
+	     3) standard/beagleboard
+	     4) standard/ck
+	     5) standard/crownbay
+	     6) standard/edf
+	     7) standard/emenlow
+	     8) standard/fri2
+	     9) standard/fsl-mpc8315e-rdb
+	     10) standard/minnow
+	     11) standard/mti-malta32
+	     12) standard/mti-malta64
+	     13) standard/qemuppc
+	     14) standard/routerstationpro
+	     15) standard/sys940x
      1
      Would you like SMP support? (y/n) [default: y]
      Does your BSP have a touchscreen? (y/n) [default: n]
@@ -1320,7 +1275,7 @@
                             In the example, we use the ARM architecture.
                             </para></listitem>
                         <listitem><para>The script then prompts you for the kernel.
-                            The default 3.8 kernel is acceptable.
+                            The default 3.10 kernel is acceptable.
                             So, the example accepts the default.
                             If you enter 'n', the script prompts you to further enter the kernel
                             you do want to use (e.g. 3.2, 3.2_preempt-rt, and so forth.).</para></listitem>

documentation/dev-manual/dev-manual-intro.xml

@@ -24,17 +24,6 @@
             set up your host development system and build an image, which you
             find in the
             <ulink url='&YOCTO_DOCS_QS_URL;'>Yocto Project Quick Start</ulink>.
-            <note>
-                By default, using the Yocto Project creates a Poky distribution.
-                However, you can create your own distribution by providing key
-                <link linkend='metadata'>Metadata</link>.
-                A good example is Angstrom, which has had a distribution
-                based on the Yocto Project since its inception.
-                Other examples include  commercial distributions like
-                Wind River Linux, Mentor Embedded Linux, and ENEA Linux.
-                See the "<link linkend='creating-your-own-distribution'>Creating Your Own Distribution</link>"
-                section for more information.
-            </note>
         </para>
 
         <para>
@@ -43,13 +32,25 @@
             reconfigure the kernel, and develop an application using the
             popular <trademark class='trade'>Eclipse</trademark> IDE.
         </para>
+
+        <note>
+            By default, using the Yocto Project creates a Poky distribution.
+            However, you can create your own distribution by providing key
+            <link linkend='metadata'>Metadata</link>.
+            A good example is Angstrom, which has had a distribution
+            based on the Yocto Project since its inception.
+            Other examples include  commercial distributions like
+            Wind River Linux, Mentor Embedded Linux, and ENEA Linux.
+            See the "<link linkend='creating-your-own-distribution'>Creating Your Own Distribution</link>"
+            section for more information.
+        </note>
     </section>
 
     <section id='what-this-manual-provides'>
         <title>What This Manual Provides</title>
 
         <para>
-            The following list describes what you can get from this guide:
+            The following list describes what you can get from this manual:
             <itemizedlist>
                 <listitem><para>Information that lets you get set
                     up to develop using the Yocto Project.</para></listitem>
@@ -75,17 +76,17 @@
         <para>
             This manual will not give you the following:
             <itemizedlist>
-                <listitem><para>Step-by-step instructions if those instructions exist in other Yocto
-                    Project documentation.
+                <listitem><para><emphasis>Step-by-step instructions when those instructions exist in other Yocto
+                    Project documentation:</emphasis>
                     For example, the Yocto Project Application Developer's Guide contains detailed
                     instructions on how to run the
-                    <ulink url='&YOCTO_DOCS_ADT_URL;#installing-the-adt'>Installing the ADT and Toolchains</ulink>,
+                    <ulink url='&YOCTO_DOCS_ADT_URL;#installing-the-adt'>ADT Installer</ulink>,
                     which is used to set up a cross-development environment.</para></listitem>
-                <listitem><para>Reference material.
+                <listitem><para><emphasis>Reference material:</emphasis>
                     This type of material resides in an appropriate reference manual.
                     For example, system variables are documented in the
                     <ulink url='&YOCTO_DOCS_REF_URL;'>Yocto Project Reference Manual</ulink>.</para></listitem>
-                <listitem><para>Detailed public information that is not specific to the Yocto Project.
+                <listitem><para><emphasis>Detailed public information that is not specific to the Yocto Project:</emphasis>
                     For example, exhaustive information on how to use Git is covered better through the
                     Internet than in this manual.</para></listitem>
             </itemizedlist>
@@ -109,7 +110,8 @@
                     with the Yocto Project and quickly begin building an image.</para></listitem>
                 <listitem><para><emphasis>
                     <ulink url='&YOCTO_DOCS_REF_URL;'>Yocto Project Reference Manual</ulink>:</emphasis> This manual is a reference
-                    guide to the OpenEmbedded build system known as "Poky."
+                    guide to the OpenEmbedded build system, which is based on BitBake.
+                    The build system is sometimes referred to as "Poky".
                     </para></listitem>
                 <listitem><para><emphasis>
                     <ulink url='&YOCTO_DOCS_ADT_URL;'>Yocto Project Application Developer's Guide</ulink>:</emphasis>

documentation/dev-manual/dev-manual-newbie.xml

@@ -254,9 +254,9 @@
                 <listitem><para><ulink url='http://sitaramc.github.com/gitolite/master-toc.html'>The <filename>gitolite</filename> master index</ulink>:
                     All topics for <filename>gitolite</filename>.
                     </para></listitem>
-                <listitem><para><ulink url='http://hjemli.net/git/cgit/tree/README'><filename>cgit</filename> index</ulink>:
-                    A <filename>README</filename> file on how to create a
-                    fast web interface for Git.</para></listitem>
+                <listitem><para><ulink url='https://git.wiki.kernel.org/index.php/Interfaces,_frontends,_and_tools'>Interfaces, frontends, and tools</ulink>:
+                    Documentation on how to create interfaces and frontends
+                    for Git.</para></listitem>
             </itemizedlist>
         </note>
     </section>
@@ -275,9 +275,8 @@
         </para>
 
         <para>
-            See "<ulink url='http://autobuilder.yoctoproject.org:8010/'>Welcome to the buildbot for the Yocto Project</ulink>"
-            for the Yocto Project's reference implementation that uses
-            buildbot.
+            See "<ulink url='http://autobuilder.yoctoproject.org'>Yocto Project Autobuilder</ulink>"
+            for more information and links to buildbot.
             The Yocto Project team has found this implementation
             works well in this role.
             A public example of this is the Yocto Project
@@ -394,7 +393,7 @@
                     For some guidance on mailing lists to use, see the list in the
                     "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>"
                     section.
-                    For a description of the available mailing lists, see
+                    For a description of the available mailing lists, see the
                     "<ulink url='&YOCTO_DOCS_REF_URL;#resources-mailinglist'>Mailing Lists</ulink>"
                     section in the Yocto Project Reference Manual.
                     </para></listitem>
@@ -430,15 +429,17 @@
     <para>
         For any supported release of Yocto Project, you can go to the
         <ulink url='&YOCTO_HOME_URL;'>Yocto Project Website</ulink> and
-        select the "Downloads" tab and get a tarball of the release.
-        You can also go to this site to download any supported BSP tarballs.
-        Unpacking the tarball gives you a hierarchical Source Directory that lets you develop
-        using the Yocto Project.
-    </para>
-
-    <para>
-        Once you are set up through either tarball extraction or a checkout of Git repositories,
-        you are ready to develop.
+        select the "Downloads" tab and get a released tarball of the
+        <filename>poky</filename> repository or any supported BSP tarballs.
+        Unpacking these tarballs gives you a snapshot of the released
+        files.
+        <note>
+            The recommended method for setting up the Yocto Project
+            <link linkend='source-directory'>Source Directory</link> and the
+            files for supported BSPs (e.g., <filename>meta-intel</filename>) is to
+            use <link linkend='git'>Git</link> to create a local copy of the
+            upstream repositories.
+        </note>
     </para>
 
     <para>
@@ -509,31 +510,46 @@
                 This term refers to the area used by the OpenEmbedded build system for builds.
                 The area is created when you <filename>source</filename> the setup
                 environment script that is found in the Source Directory
-                (i.e. <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink>).
+                (i.e. <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink>
+                or
+                <ulink url='&YOCTO_DOCS_REF_URL;#structure-memres-core-script'><filename>oe-init-build-env-memres</filename></ulink>).
                 The <ulink url='&YOCTO_DOCS_REF_URL;#var-TOPDIR'><filename>TOPDIR</filename></ulink>
                 variable points to the Build Directory.</para>
 
-                <para>You have a lot of flexibility when creating the Build Directory.
-                Following are some examples that show how to create the directory:
+                <para>
+                    You have a lot of flexibility when creating the Build
+                    Directory.
+                    Following are some examples that show how to create the
+                    directory.
+                    The examples assume your
+                    <link linkend='source-directory'>Source Directory</link> is
+                    named <filename>poky</filename>:
                    <itemizedlist>
-                        <listitem><para>Create the Build Directory in your current working directory
-                            and name it <filename>build</filename>.
-                            This is the default behavior.
+                        <listitem><para>Create the Build Directory inside your
+                            Source Directory and let the name of the Build
+                            Directory default to <filename>build</filename>:
                             <literallayout class='monospaced'>
-     $ source &OE_INIT_PATH;
+     $ cd $HOME/poky
+     $ source &OE_INIT_FILE;
                             </literallayout></para></listitem>
-                        <listitem><para>Provide a directory path and specifically name the build
-                            directory.
-                            This next example creates a Build Directory named <filename>YP-&POKYVERSION;</filename>
-                            in your home directory within the directory <filename>mybuilds</filename>.
-                            If <filename>mybuilds</filename> does not exist, the directory is created for you:
+                        <listitem><para>Create the Build Directory inside your
+                            home directory and specifically name it
+                            <filename>test-builds</filename>:
                             <literallayout class='monospaced'>
-     $ source &OE_INIT_PATH; $HOME/mybuilds/YP-&POKYVERSION;
+     $ cd $HOME
+     $ source poky/&OE_INIT_FILE; test-builds
                             </literallayout></para></listitem>
-                        <listitem><para>Provide an existing directory to use as the Build Directory
-                            and use the default <filename>build</filename> name.
+                        <listitem><para>Provide a directory path and
+                            specifically name the build directory.
+                            Any intermediate folders in the pathname must
+                            exist.
+                            This next example creates a Build Directory named
+                            <filename>YP-&POKYVERSION;</filename>
+                            in your home directory within the existing
+                            directory <filename>mybuilds</filename>:
                             <literallayout class='monospaced'>
-     $ source &OE_INIT_PATH; $HOME/mybuilds/
+     $cd $HOME
+     $ source $HOME/poky/&OE_INIT_FILE; $HOME/mybuilds/YP-&POKYVERSION;
                             </literallayout></para></listitem>
                     </itemizedlist>
                 </para></listitem>
@@ -613,7 +629,7 @@
                 "<ulink url='&YOCTO_DOCS_BSP_URL;#bsp-layers'>BSP Layers</ulink>"
                 section in the Yocto Project Board Support Packages (BSP)
                 Developer's Guide.</para></listitem>
-            <listitem><para id='meta-toochain'><emphasis>Meta-Toolchain:</emphasis>
+            <listitem><para id='meta-toolchain'><emphasis>Meta-Toolchain:</emphasis>
                 A term sometimes used for
                 <link linkend='cross-development-toolchain'>Cross-Development Toolchain</link>.
                 </para></listitem>
@@ -646,6 +662,17 @@
                 <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink>, and
                 <ulink url='&YOCTO_DOCS_REF_URL;#var-PE'><filename>PE</filename></ulink>).
                 </para></listitem>
+            <listitem><para><emphasis>Package Groups:</emphasis>
+                Arbitrary groups of software Recipes.
+                You use package groups to hold recipes that, when built,
+                usually accomplish a single task.
+                For example, a package group could contain the recipes for a
+                company’s proprietary or value-add software.
+                Or, the package group could contain the recipes that enable
+                graphics.
+                A package group is really just another recipe.
+                Because package group files are recipes, they end with the
+                <filename>.bb</filename> filename extension.</para></listitem>
             <listitem><para id='poky'><emphasis>Poky:</emphasis> The term "poky" can mean several things.
                 In its most general sense, it is an open-source project that was initially developed
                 by OpenedHand.  With OpenedHand, poky was developed off of the existing OpenEmbedded
@@ -687,7 +714,7 @@
                 For example, downloading and unpacking <filename>&YOCTO_POKY_TARBALL;</filename>
                 results in a Source Directory whose top-level folder is named
                 <filename>&YOCTO_POKY;</filename>.
-                If you create a local copy of the Git repository, then you can name the repository
+                If you create a local copy of the Git repository, you can name the repository
                 anything you like.
                 Throughout much of the documentation, <filename>poky</filename> is used as the name of
                 the top-level folder of the local copy of the poky Git repository.
@@ -715,13 +742,12 @@
                 see the
                 "<link linkend='repositories-tags-and-branches'>Repositories, Tags, and Branches</link>"
                 section.</para></listitem>
-            <listitem><para><emphasis>Tasks:</emphasis> Arbitrary groups of software Recipes.
-                You use tasks to hold recipes that, when built, usually accomplish a single task.
-                For example, a task could contain the recipes for a company’s proprietary or value-add software.
-                Or, the task could contain the recipes that enable graphics.
-                A task is really just another recipe.
-                Because task files are recipes, they end with the <filename>.bb</filename> filename
-                extension.</para></listitem>
+            <listitem><para><emphasis>Task:</emphasis>
+                A unit of execution for BitBake (e.g.
+                <filename>do_compile</filename>,
+                <filename>do_fetch</filename>, <filename>do_patch</filename>,
+                and so forth).
+                </para></listitem>
             <listitem><para><emphasis>Upstream:</emphasis> A reference to source code or repositories
                 that are not local to the development system but located in a master area that is controlled
                 by the maintainer of the source code.
@@ -806,7 +832,8 @@
     <title>Git</title>
 
     <para>
-        The Yocto Project uses Git, which is a free, open source distributed version control system.
+        The Yocto Project makes extensive use of Git,
+        which is a free, open source distributed version control system.
         Git supports distributed development, non-linear development, and can handle large projects.
         It is best that you have some fundamental understanding of how Git tracks projects and
         how to work with Git if you are going to use the Yocto Project for development.
@@ -863,8 +890,8 @@
             It is important to understand that Git tracks content change and not files.
             Git uses "branches" to organize different development efforts.
             For example, the <filename>poky</filename> repository has
-            <filename>bernard</filename>,
-            <filename>edison</filename>, <filename>denzil</filename>, <filename>danny</filename>
+            <filename>denzil</filename>, <filename>danny</filename>,
+            <filename>dylan</filename>, <filename>dora</filename>,
             and <filename>master</filename> branches among others.
             You can see all the branches by going to
             <ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/'></ulink> and
@@ -910,7 +937,7 @@
             local working branch based on a branch name,
             your local environment matches the "tip" of that development branch
             at the time you created your local branch, which could be
-            different than the files at the time of a similarly named release.
+            different from the files at the time of a similarly named release.
             In other words, creating and checking out a local branch based on the
             <filename>&DISTRO_NAME;</filename> branch name is not the same as
             cloning and checking out the <filename>master</filename> branch.
@@ -1003,7 +1030,7 @@
                     will allow the change, and for ultimately pushing the change from your local Git repository
                     into the project’s upstream (or master) repository.</para></listitem>
                 <listitem><para><emphasis><filename>git status</filename>:</emphasis> Reports any modified files that
-                    possibly need staged and committed.</para></listitem>
+                    possibly need to be staged and committed.</para></listitem>
                 <listitem><para><emphasis><filename>git checkout &lt;branch-name&gt;</filename>:</emphasis> Changes
                     your working branch.
                     This command is analogous to "cd".</para></listitem>
@@ -1231,6 +1258,10 @@
                 occurred.</para></listitem>
             <listitem><para>Be sure to indicate the Severity of the bug.
                 Severity communicates how the bug impacted your work.</para></listitem>
+            <listitem><para>Select the appropriate "Documentation change" item
+                for the bug.
+                Fixing a bug may or may not affect the Yocto Project
+                documentation.</para></listitem>
             <listitem><para>Provide a brief summary of the issue.
                 Try to limit your summary to just a line or two and be sure to capture the
                 essence of the issue.</para></listitem>
@@ -1276,10 +1307,9 @@
             <listitem><para><emphasis>Board Support Package (BSP) README Files:</emphasis>
                 For BSP maintainers of supported BSPs, you can examine
                 individual BSP <filename>README</filename> files.
-                Alternatively, you can examine the
-                <filename>MAINTAINERS</filename> file, which is found in the
-                <filename>meta-intel</filename>, for a list of all supported
-                BSP maintainers.
+                In addition, some layers (such as the <filename>meta-intel</filename> layer),
+                include a <filename>MAINTAINERS</filename> file which contains
+                a list of all supported BSP maintainers for that layer.
                 </para></listitem>
             <listitem><para><emphasis>Search by File:</emphasis>
                 Using <link linkend='git'>Git</link>, you can enter the
@@ -1454,8 +1484,8 @@
                     <para>For help on using these scripts, simply provide the
                     <filename>-h</filename> argument as follows:
                     <literallayout class='monospaced'>
-     $ ~/poky/scripts/create-pull-request -h
-     $ ~/poky/scripts/send-pull-request -h
+     $ poky/scripts/create-pull-request -h
+     $ poky/scripts/send-pull-request -h
                     </literallayout></para></listitem>
             </itemizedlist>
         </para>

documentation/dev-manual/dev-manual-start.xml

@@ -32,7 +32,7 @@
 
     <para>
         You can use the OpenEmbedded build system, which uses
-        BitBake to develop complete Linux
+        BitBake, to develop complete Linux
         images and associated user-space applications for architectures based
         on ARM, MIPS, PowerPC, x86 and x86-64.
         <note>
@@ -55,12 +55,12 @@
     <title>Getting Set Up</title>
 
     <para>
-        Here is what you need to get set up to use the Yocto Project:
+        Here is what you need to use the Yocto Project:
         <itemizedlist>
-            <listitem><para><emphasis>Host System:</emphasis>  You should have a reasonably current
-                Linux-based host system.
+            <listitem><para><emphasis>Host System:</emphasis>
+                You should have a reasonably current Linux-based host system.
                 You will have the best results with a recent release of Fedora,
-                OpenSUSE, Debian, Ubuntu, or CentOS as these releases are frequently tested against the Yocto Project
+                openSUSE, Debian, Ubuntu, or CentOS as these releases are frequently tested against the Yocto Project
                 and officially supported.
                 For a list of the distributions under validation and their status, see the
                 "<ulink url='&YOCTO_DOCS_REF_URL;#detailed-supported-distros'>Supported Linux Distributions</ulink>" section
@@ -69,23 +69,35 @@
                 <para>
                 You should also have about 100 gigabytes of free disk space for building images.
                 </para></listitem>
-            <listitem><para><emphasis>Packages:</emphasis>  The OpenEmbedded build system
-                requires certain packages exist on your development system (e.g. Python 2.6 or 2.7).
+            <listitem><para><emphasis>Packages:</emphasis>
+                The OpenEmbedded build system requires that certain packages
+                exist on your development system (e.g. Python 2.6 or 2.7).
                 See "<ulink url='&YOCTO_DOCS_QS_URL;#packages'>The Packages</ulink>"
                 section in the Yocto Project Quick Start and the
                 "<ulink url='&YOCTO_DOCS_REF_URL;#required-packages-for-the-host-development-system'>Required Packages for the Host Development System</ulink>"
                 section in the Yocto Project Reference Manual for the exact
                 package requirements and the installation commands to install
-                them for the supported distributions.
-                </para></listitem>
+                them for the supported distributions.</para></listitem>
             <listitem id='local-yp-release'><para><emphasis>Yocto Project Release:</emphasis>
-                You need a release of the Yocto Project.
-                You set that up with a local <link linkend='source-directory'>Source Directory</link>
-                one of two ways depending on whether you
-                are going to contribute back into the Yocto Project or not.
-                <note>
-                    Regardless of the method you use, this manual refers to the resulting local
-                    hierarchical set of files as the "Source Directory."
+                You need a release of the Yocto Project installed locally on
+                your development system.
+                This local area is referred to as the
+                <link linkend='source-directory'>Source Directory</link>
+                and is created when you use
+                <link linkend='git'>Git</link> to clone a local copy
+                of the upstream <filename>poky</filename> repository,
+                or when you download an official release of the corresponding
+                tarball.</para>
+                <para>Working from a copy of the upstream repository allows you
+                to contribute back into the Yocto Project or simply work with
+                the latest software on a development branch.
+                Because Git maintains and creates an upstream repository with
+                a complete history of changes and you are working with a local
+                clone of that repository, you have access to all the Yocto
+                Project development branches and tag names used in the upstream
+                repository.
+                <note>You can view the Yocto Project Source Repositories at
+                    <ulink url='&YOCTO_GIT_URL;/cgit.cgi'></ulink>
                 </note>
                 <itemizedlist>
                     <listitem><para><emphasis>Tarball Extraction:</emphasis>
@@ -97,46 +109,62 @@
                         directory of your choice.</para>
                         <para>For example, the following command extracts the
                         Yocto Project &DISTRO; release tarball
-                        into the current working directory and sets up the local Source Directory
-                        with a top-level folder named <filename>&YOCTO_POKY;</filename>:
+                        into the current working directory and sets up the local
+                        Source Directory
+                        with a top-level folder named
+                        <filename>&YOCTO_POKY;</filename>:
                         <literallayout class='monospaced'>
      $ tar xfj &YOCTO_POKY_TARBALL;
                         </literallayout></para>
-                        <para>This method does not produce a local Git repository.
-                        Instead, you simply end up with a snapshot of the release.</para></listitem>
-                    <listitem><para><emphasis>Git Repository Method:</emphasis>  If you are going to be contributing
-                        back into the Yocto Project or you simply want to keep up
-                        with the latest developments, you should use Git commands to set up a local
-                        Git repository of the upstream <filename>poky</filename> source repository.
-                        Doing so creates a repository with a complete history of changes and allows
-                        you to easily submit your changes upstream to the project.
-                        Because you clone the repository, you have access to all the Yocto Project development
-                        branches and tag names used in the upstream repository.</para>
-                        <note>You can view the Yocto Project Source Repositories at
-                        <ulink url='&YOCTO_GIT_URL;/cgit.cgi'></ulink></note>
-                        <para>The following transcript shows how to clone the <filename>poky</filename>
-                        Git repository into the current working directory.
-                        The command creates the local repository in a directory named <filename>poky</filename>.
-                        For information on Git used within the Yocto Project, see the
+                        <para>This method does not produce a local Git
+                        repository.
+                        Instead, you simply end up with a snapshot of the
+                        release.</para></listitem>
+                    <listitem><para><emphasis>Git Repository Method:</emphasis>
+                        If you are going to be contributing back into the Yocto
+                        Project or you simply want to keep up with the latest
+                        developments, you should use Git commands to set up a
+                        local Git repository of the upstream
+                        <filename>poky</filename> source repository.
+                        Doing so creates a repository with a complete history
+                        of changes and allows you to easily submit your changes
+                        upstream to the project.
+                        Because you clone the repository, you have access to all
+                        the Yocto Project development branches and tag names
+                        used in the upstream repository.
+                        <note>You can view the Yocto Project Source Repositories
+                            at
+                            <ulink url='&YOCTO_GIT_URL;/cgit.cgi'></ulink>
+                        </note></para>
+                        <para>The following transcript shows how to clone the
+                        <filename>poky</filename> Git repository into the
+                        current working directory.
+                        The command creates the local repository in a directory
+                        named <filename>poky</filename>.
+                        For information on Git used within the Yocto Project,
+                        see the
                         "<link linkend='git'>Git</link>" section.
                         <literallayout class='monospaced'>
      $ git clone git://git.yoctoproject.org/poky
      Cloning into 'poky'...
-     remote: Counting objects: 183981, done.
-     remote: Compressing objects: 100% (47428/47428), done.
-     remote: Total 183981 (delta 132271), reused 183703 (delta 132044)
-     Receiving objects: 100% (183981/183981), 89.71 MiB | 2.93 MiB/s, done.
-     Resolving deltas: 100% (132271/132271), done.
-                        </literallayout></para>
-                        <para>For another example of how to set up your own local Git repositories, see this
-                        <ulink url='&YOCTO_WIKI_URL;/wiki/Transcript:_from_git_checkout_to_meta-intel_BSP'>
-                        wiki page</ulink>, which describes how to create both <filename>poky</filename>
-                        and <filename>meta-intel</filename> Git repositories.</para></listitem>
+     remote: Counting objects: 203728, done.
+     remote: Compressing objects: 100% (52371/52371), done.
+     remote: Total 203728 (delta 147444), reused 202891 (delta 146614)
+     Receiving objects: 100% (203728/203728), 95.54 MiB | 308 KiB/s, done.
+     Resolving deltas: 100% (147444/147444), done.
+                        </literallayout>
+                        For another example of how to set up your own local
+                        Git repositories, see this
+                        <ulink url='&YOCTO_WIKI_URL;/wiki/Transcript:_from_git_checkout_to_meta-intel_BSP'>wiki page</ulink>,
+                        which describes how to create both
+                        <filename>poky</filename> and
+                        <filename>meta-intel</filename> Git repositories.
+                        </para></listitem>
                 </itemizedlist></para></listitem>
             <listitem id='local-kernel-files'><para><emphasis>Yocto Project Kernel:</emphasis>
                 If you are going to be making modifications to a supported Yocto Project kernel, you
                 need to establish local copies of the source.
-                You can find Git repositories of supported Yocto Project Kernels organized under
+                You can find Git repositories of supported Yocto Project kernels organized under
                 "Yocto Linux Kernel" in the Yocto Project Source Repositories at
                 <ulink url='&YOCTO_GIT_URL;/cgit.cgi'></ulink>.</para>
                 <para>This setup can involve creating a bare clone of the Yocto Project kernel and then
@@ -145,27 +173,28 @@
                 For simplicity, it is recommended that you create these structures outside of the
                 Source Directory (usually <filename>poky</filename>).</para>
                 <para>As an example, the following transcript shows how to create the bare clone
-                of the <filename>linux-yocto-3.8</filename> kernel and then create a copy of
+                of the <filename>linux-yocto-3.10</filename> kernel and then create a copy of
                 that clone.
                 <note>When you have a local Yocto Project kernel Git repository, you can
                 reference that repository rather than the upstream Git repository as
                 part of the <filename>clone</filename> command.
                 Doing so can speed up the process.</note></para>
                 <para>In the following example, the bare clone is named
-                <filename>linux-yocto-3.8.git</filename>, while the
-                copy is named <filename>my-linux-yocto-3.8-work</filename>:
+                <filename>linux-yocto-3.10.git</filename>, while the
+                copy is named <filename>my-linux-yocto-3.10-work</filename>:
                 <literallayout class='monospaced'>
-     $ git clone --bare git://git.yoctoproject.org/linux-yocto-3.8 linux-yocto-3.8.git
-     Cloning into bare repository 'linux-yocto-3.8.git'...
-     remote: Counting objects: 2847090, done.
-     remote: Compressing objects: 100% (454675/454675), done.
-     remote: Total 2847090 (delta 2386170), reused 2825793 (delta 2364886)
-     Receiving objects: 100% (2847090/2847090), 603.19 MiB | 3.54 MiB/s, done.
-     Resolving deltas: 100% (2386170/2386170), done.                </literallayout></para>
+     $ git clone &dash;&dash;bare git://git.yoctoproject.org/linux-yocto-3.10 linux-yocto-3.10.git
+     Cloning into bare repository 'linux-yocto-3.10.git'...
+     remote: Counting objects: 3364487, done.
+     remote: Compressing objects: 100% (507178/507178), done.
+     remote: Total 3364487 (delta 2827715), reused 3364481 (delta 2827709)
+     Receiving objects: 100% (3364487/3364487), 722.95 MiB | 423 KiB/s, done.
+     Resolving deltas: 100% (2827715/2827715), done.
+                </literallayout></para>
                 <para>Now create a clone of the bare clone just created:
                 <literallayout class='monospaced'>
-     $ git clone linux-yocto-3.8.git my-linux-yocto-3.8-work
-     Cloning into 'my-linux-yocto-3.8-work'...
+     $ git clone linux-yocto-3.10.git my-linux-yocto-3.10-work
+     Cloning into 'my-linux-yocto-3.10-work'...
      done.
                 </literallayout></para></listitem>
             <listitem id='meta-yocto-kernel-extras-repo'><para><emphasis>
@@ -189,11 +218,12 @@
      $ cd ~/poky
      $ git clone git://git.yoctoproject.org/meta-yocto-kernel-extras meta-yocto-kernel-extras
      Cloning into 'meta-yocto-kernel-extras'...
-     remote: Counting objects: 690, done.
-     remote: Compressing objects: 100% (431/431), done.
-     remote: Total 690 (delta 238), reused 690 (delta 238)
-     Receiving objects: 100% (690/690), 532.60 KiB, done.
-     Resolving deltas: 100% (238/238), done.                </literallayout></para></listitem>
+     remote: Counting objects: 727, done.
+     remote: Compressing objects: 100% (452/452), done.
+     remote: Total 727 (delta 260), reused 719 (delta 252)
+     Receiving objects: 100% (727/727), 536.36 KiB | 102 KiB/s, done.
+     Resolving deltas: 100% (260/260), done.
+               </literallayout></para></listitem>
            <listitem><para id='supported-board-support-packages-(bsps)'><emphasis>Supported Board
                 Support Packages (BSPs):</emphasis>
                 The Yocto Project provides a layer called <filename>meta-intel</filename> and
@@ -219,43 +249,54 @@
                 </literallayout>
                 See the
                 "<ulink url='&YOCTO_DOCS_BSP_URL;#bsp-layers'>BSP Layers</ulink>"
-                section in the Yocto Project Board Support Package (BSP) Developer's Guide for more
-                information on BSP Layers.
+                section in the Yocto Project Board Support Package (BSP)
+                Developer's Guide for more information on BSP Layers.
                 <itemizedlist>
-                    <listitem><para><emphasis>Tarball Extraction:</emphasis>  You can download any released
-                        BSP tarball from the same "Downloads" page of the
-                        <ulink url='&YOCTO_HOME_URL;'>Yocto Project Website</ulink>
+                    <listitem><para><emphasis>Tarball Extraction:</emphasis>
+                        You can download any released BSP tarball from the same
+                        "Downloads" page of the Yocto Project
+                        <ulink url='https://www.yoctoproject.org/downloads'>Website</ulink>
                         to get the Yocto Project release.
-                        Once on the "Download" page, look for "BSP" under the
-                        "Type" heading.</para>
-                        <para>Once you have the tarball, just extract it into a directory of your choice.
-                        Again, this method just produces a snapshot of the BSP layer in the form
-                        of a hierarchical directory structure.</para></listitem>
-                    <listitem><para><emphasis>Git Repository Method:</emphasis>  If you are working
-                        with a local Git repository for your Source Directory, you should also use this method
-                        to set up the <filename>meta-intel</filename> Git repository.
-                        You can locate the <filename>meta-intel</filename> Git repository in the
-                        "Yocto Metadata Layers" area of the Yocto Project Source Repositories at
+                        Once on the "Download" page, look to the right of the
+                        page and scroll down to find the BSP tarballs.</para>
+                        <para>Once you have the tarball, just extract it into a
+                        directory of your choice.
+                        Again, this method just produces a snapshot of the BSP
+                        layer in the form of a hierarchical directory
+                        structure.</para></listitem>
+                    <listitem><para><emphasis>Git Repository Method:</emphasis>
+                        If you are working with a local Git repository for your
+                        Source Directory, you should also use this method to
+                        set up the <filename>meta-intel</filename> Git
+                        repository.
+                        You can locate the <filename>meta-intel</filename> Git
+                        repository in the "Yocto Metadata Layers" area of the
+                        Yocto Project Source Repositories at
                         <ulink url='&YOCTO_GIT_URL;/cgit.cgi'></ulink>.</para>
-                        <para>Typically, you set up the <filename>meta-intel</filename> Git repository inside
+                        <para>Using
+                        <link linkend='git'>Git</link> to create a local clone
+                        of the upstream repository can be helpful if you are
+                        working with BSPs.
+                        Typically, you set up the
+                        <filename>meta-intel</filename> Git repository inside
                         the Source Directory.
-                        For example, the following transcript shows the steps to clone the
-                        <filename>meta-intel</filename>
-                        Git repository inside the local <filename>poky</filename> Git repository.
+                        For example, the following transcript shows the steps
+                        to clone <filename>meta-intel</filename>.
                         <literallayout class='monospaced'>
      $ cd ~/poky
      $ git clone git://git.yoctoproject.org/meta-intel.git
      Cloning into 'meta-intel'...
-     remote: Counting objects: 6264, done.
-     remote: Compressing objects: 100% (2135/2135), done.
-     remote: Total 6264 (delta 3321), reused 6235 (delta 3293)
-     Receiving objects: 100% (6264/6264), 2.17 MiB | 2.63 MiB/s, done.
-     Resolving deltas: 100% (3321/3321), done.
-                        </literallayout></para>
-                        <para>The same
-                        <ulink url='&YOCTO_WIKI_URL;/wiki/Transcript:_from_git_checkout_to_meta-intel_BSP'>
-                        wiki page</ulink> referenced earlier covers how to
-                        set up the <filename>meta-intel</filename> Git repository.</para></listitem>
+     remote: Counting objects: 7366, done.
+     remote: Compressing objects: 100% (2491/2491), done.
+     remote: Total 7366 (delta 3997), reused 7299 (delta 3930)
+     Receiving objects: 100% (7366/7366), 2.31 MiB | 95 KiB/s, done.
+     Resolving deltas: 100% (3997/3997), done.
+                        </literallayout>
+                        The same
+                        <ulink url='&YOCTO_WIKI_URL;/wiki/Transcript:_from_git_checkout_to_meta-intel_BSP'>wiki page</ulink>
+                        referenced earlier covers how to
+                        set up the <filename>meta-intel</filename> Git
+                        repository.</para></listitem>
                 </itemizedlist></para></listitem>
             <listitem><para><emphasis>Eclipse Yocto Plug-in:</emphasis>  If you are developing
                 applications using the Eclipse Integrated Development Environment (IDE),

documentation/dev-manual/dev-manual.xml

@@ -53,9 +53,29 @@
             </revision>
             <revision>
                 <revnumber>1.5</revnumber>
-                <date>Sometime in 2013</date>
+                <date>October 2013</date>
                 <revremark>Released with the Yocto Project 1.5 Release.</revremark>
             </revision>
+            <revision>
+                <revnumber>1.5.1</revnumber>
+                <date>January 2014</date>
+                <revremark>Released with the Yocto Project 1.5.1 Release.</revremark>
+            </revision>
+            <revision>
+                <revnumber>1.5.2</revnumber>
+                <date>May 2014</date>
+                <revremark>Released with the Yocto Project 1.5.2 Release.</revremark>
+            </revision>
+            <revision>
+                <revnumber>1.5.3</revnumber>
+                <date>July 2014</date>
+                <revremark>Released with the Yocto Project 1.5.3 Release.</revremark>
+            </revision>
+            <revision>
+                <revnumber>1.5.4</revnumber>
+                <date>December 2014</date>
+                <revremark>Released with the Yocto Project 1.5.4 Release.</revremark>
+            </revision>
         </revhistory>
 
     <copyright>
@@ -72,11 +92,10 @@
       </para>
 
       <note>
-          Due to production processes, there could be differences between the Yocto Project
-          documentation bundled in the release tarball and the
-          <ulink url='&YOCTO_DOCS_DEV_URL;'>Yocto Project Development Manual</ulink> on
-          the <ulink url='&YOCTO_HOME_URL;'>Yocto Project</ulink> website.
-          For the latest version of this manual, see the manual on the website.
+          For the latest version of this manual associated with this
+          Yocto Project release, see the
+          <ulink url='&YOCTO_DOCS_DEV_URL;'>Yocto Project Development Manual</ulink>
+          from the Yocto Project website.
       </note>
     </legalnotice>
 

documentation/kernel-dev/kernel-dev-advanced.xml

@@ -296,6 +296,20 @@
      $ git commit --allow-empty -m "Create orphan meta branch"
             </literallayout>
         </para>
+
+        <para>
+            If you modify the Metadata in the linux-yocto 
+            <filename>meta</filename> branch, you must not forget to update 
+            the 
+            <ulink url='&YOCTO_DOCS_REF_URL;#var-SRCREV'><filename>SRCREV</filename></ulink>
+            statements in the kernel's recipe.
+            In particular, you need to update the 
+            <filename>SRCREV_meta</filename> variable to match the commit in 
+            the <filename>KMETA</filename> branch you wish to use.
+            Changing the data in these branches and not updating the 
+            <filename>SRCREV</filename> statements to match will cause the 
+            build to fetch an older commit.
+        </para>
     </section>
 </section>
 

documentation/kernel-dev/kernel-dev-common.xml

@@ -95,7 +95,7 @@
                 <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS'><filename>FILESEXTRAPATHS</filename></ulink>
                 variable as follows:
                 <literallayout class='monospaced'>
-     FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}"
+     FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
                 </literallayout>
                 The path <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-THISDIR'><filename>THISDIR</filename></ulink><filename>}/${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink><filename>}</filename>
                 expands to "linux-yocto" in the current directory for this

documentation/kernel-dev/kernel-dev-maint-appx.xml

@@ -45,7 +45,7 @@
             Here is an example that assumes the local Git repository for the kernel is in
             a top-level directory named <filename>linux-yocto-3.4</filename>:
             <literallayout class='monospaced'>
-     $ cd ~/linux-yocto-3.4
+     $ cd linux-yocto-3.4
      $ git checkout -b meta origin/meta
             </literallayout>
             Once you have checked out and switched to the <filename>meta</filename> branch,
@@ -208,7 +208,7 @@
             the build tree directory.
             The files include the final <filename>.config</filename> file, all the <filename>.o</filename>
             files, the <filename>.a</filename> files, and so forth.
-            Since each machine or BSP has its own separate 
+            Since each machine or BSP has its own separate
             <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>
             in its own separate branch
             of the Git repository, you can easily switch between different builds.

documentation/kernel-dev/kernel-dev.xml

@@ -38,9 +38,29 @@
             </revision>
             <revision>
                 <revnumber>1.5</revnumber>
-                <date>Sometime in 2013</date>
+                <date>October 2013</date>
                 <revremark>Released with the Yocto Project 1.5 Release.</revremark>
             </revision>
+            <revision>
+                <revnumber>1.5.1</revnumber>
+                <date>January 2014</date>
+                <revremark>Released with the Yocto Project 1.5.1 Release.</revremark>
+            </revision>
+            <revision>
+                <revnumber>1.5.2</revnumber>
+                <date>May 2014</date>
+                <revremark>Released with the Yocto Project 1.5.2 Release.</revremark>
+            </revision>
+            <revision>
+                <revnumber>1.5.3</revnumber>
+                <date>July 2014</date>
+                <revremark>Released with the Yocto Project 1.5.3 Release.</revremark>
+            </revision>
+            <revision>
+                <revnumber>1.5.4</revnumber>
+                <date>December 2014</date>
+                <revremark>Released with the Yocto Project 1.5.4 Release.</revremark>
+            </revision>
         </revhistory>
 
     <copyright>
@@ -54,11 +74,10 @@
         the terms of the <ulink type="http" url="http://creativecommons.org/licenses/by-sa/2.0/uk/">Creative Commons Attribution-Share Alike 2.0 UK: England &amp; Wales</ulink> as published by Creative Commons.
       </para>
       <note>
-          Due to production processes, there could be differences between the Yocto Project
-          documentation bundled in the release tarball and the
-          <ulink url='&YOCTO_DOCS_KERNEL_DEV_URL;'>Yocto Project Linux Kernel Development Manual</ulink> on
-          the <ulink url='&YOCTO_HOME_URL;'>Yocto Project</ulink> website.
-          For the latest version of this manual, see the manual on the website.
+          For the latest version of this manual associated with this
+          Yocto Project release, see the
+          <ulink url='&YOCTO_DOCS_KERNEL_DEV_URL;'>Yocto Project Linux Kernel Development Manual</ulink>
+          from the Yocto Project website.
       </note>
     </legalnotice>
 

documentation/poky.ent

@@ -1,11 +1,11 @@
-<!ENTITY DISTRO "1.5">
-<!ENTITY DISTRO_COMPRESSED "15">
-<!ENTITY DISTRO_NAME "tbd">
-<!ENTITY YOCTO_DOC_VERSION "1.5">
-<!ENTITY POKYVERSION "10.0.0">
-<!ENTITY POKYVERSION_COMPRESSED "1000">
+<!ENTITY DISTRO "1.5.4">
+<!ENTITY DISTRO_COMPRESSED "154">
+<!ENTITY DISTRO_NAME "dora">
+<!ENTITY YOCTO_DOC_VERSION "1.5.4">
+<!ENTITY POKYVERSION "10.0.4">
+<!ENTITY POKYVERSION_COMPRESSED "1004">
 <!ENTITY YOCTO_POKY "poky-&DISTRO_NAME;-&POKYVERSION;">
-<!ENTITY COPYRIGHT_YEAR "2010-2013">
+<!ENTITY COPYRIGHT_YEAR "2010-2014">
 <!ENTITY YOCTO_DL_URL "http://downloads.yoctoproject.org">
 <!ENTITY YOCTO_HOME_URL "http://www.yoctoproject.org">
 <!ENTITY YOCTO_LISTS_URL "http://lists.yoctoproject.org">
@@ -14,9 +14,9 @@
 <!ENTITY YOCTO_AB_URL "http://autobuilder.yoctoproject.org">
 <!ENTITY YOCTO_GIT_URL "http://git.yoctoproject.org">
 <!ENTITY YOCTO_ADTREPO_URL "http://adtrepo.yoctoproject.org">
-<!ENTITY YOCTO_RELEASE_NOTES "&YOCTO_HOME_URL;/download/yocto-project-&DISTRO_COMPRESSED;-poky-&POKYVERSION_COMPRESSED;">
+<!ENTITY YOCTO_RELEASE_NOTES "&YOCTO_HOME_URL;/downloads/core/&DISTRO_NAME;&DISTRO_COMPRESSED;">
 <!ENTITY OE_HOME_URL "http://www.openembedded.org">
-<!ENTITY OE_LISTS_URL "http://lists.linuxtogo.org/cgi-bin/mailman">
+<!ENTITY OE_LISTS_URL "http://lists.openembedded.org/mailman">
 <!ENTITY OE_DOCS_URL "http://docs.openembedded.org">
 <!ENTITY OH_HOME_URL "http://o-hand.com">
 <!ENTITY BITBAKE_HOME_URL "http://developer.berlios.de/projects/bitbake/">
@@ -26,6 +26,7 @@
 <!ENTITY ECLIPSE_UPDATES_URL "&ECLIPSE_DL_URL;/tm/updates/3.3">
 <!ENTITY ECLIPSE_INDIGO_URL "&ECLIPSE_DL_URL;/releases/indigo">
 <!ENTITY ECLIPSE_JUNO_URL "&ECLIPSE_DL_URL;/releases/juno">
+<!ENTITY ECLIPSE_KEPLER_URL "&ECLIPSE_DL_URL;/releases/kepler">
 <!ENTITY ECLIPSE_INDIGO_CDT_URL "&ECLIPSE_DL_URL;tools/cdt/releases/indigo">
 <!ENTITY YOCTO_DOCS_URL "&YOCTO_HOME_URL;/docs">
 <!ENTITY YOCTO_SOURCES_URL "&YOCTO_HOME_URL;/sources/">
@@ -35,7 +36,7 @@
 <!ENTITY YOCTO_RELEASE_DL_URL "&YOCTO_DL_URL;/releases/yocto/yocto-&DISTRO;">
 <!ENTITY YOCTO_TOOLCHAIN_DL_URL "&YOCTO_RELEASE_DL_URL;/toolchain/">
 <!ENTITY YOCTO_ECLIPSE_DL_URL "&YOCTO_RELEASE_DL_URL;/eclipse-plugin/indigo;">
-<!ENTITY YOCTO_ADTINSTALLER_DL_URL "&YOCTO_RELEASE_DL_URL;/adt_installer">
+<!ENTITY YOCTO_ADTINSTALLER_DL_URL "&YOCTO_RELEASE_DL_URL;/adt-installer">
 <!ENTITY YOCTO_POKY_DL_URL "&YOCTO_RELEASE_DL_URL;/&YOCTO_POKY;.tar.bz2">
 <!ENTITY YOCTO_MACHINES_DL_URL "&YOCTO_RELEASE_DL_URL;/machines">
 <!ENTITY YOCTO_QEMU_DL_URL "&YOCTO_MACHINES_DL_URL;/qemu">
@@ -53,11 +54,11 @@
 <!ENTITY YOCTO_POKY_TARBALL "&YOCTO_POKY;.tar.bz2">
 <!ENTITY OE_INIT_PATH "&YOCTO_POKY;/oe-init-build-env">
 <!ENTITY OE_INIT_FILE "oe-init-build-env">
-<!ENTITY UBUNTU_HOST_PACKAGES_ESSENTIAL "gawk wget git-core diffstat unzip texinfo \
+<!ENTITY UBUNTU_HOST_PACKAGES_ESSENTIAL "gawk wget git-core diffstat unzip texinfo gcc-multilib \
      build-essential chrpath">
 <!ENTITY FEDORA_HOST_PACKAGES_ESSENTIAL "gawk make wget tar bzip2 gzip python unzip perl patch \
-     diffutils diffstat git cpp gcc gcc-c++ eglibc-devel texinfo chrpath \
-     ccache">
+     diffutils diffstat git cpp gcc gcc-c++ glibc-devel texinfo chrpath \
+     ccache perl-Data-Dumper perl-Text-ParseWords">
 <!ENTITY OPENSUSE_HOST_PACKAGES_ESSENTIAL "python gcc gcc-c++ git chrpath make wget python-xml \
      diffstat texinfo python-curses patch">
 <!ENTITY CENTOS_HOST_PACKAGES_ESSENTIAL "gawk make wget tar bzip2 gzip python unzip perl patch \

documentation/profile-manual/profile-manual-intro.xml

@@ -4,7 +4,7 @@
 
 <chapter id='profile-manual-intro'>
 
-<title>Yocto Project Tracing and Profiling Manual</title>
+<title>Yocto Project Profiling and Tracing Manual</title>
     <section id='intro'>
         <title>Introduction</title>
 
@@ -78,7 +78,7 @@
         </para>
 
         <para>
-            If you've already build a stripped image, you can generate
+            If you've already built a stripped image, you can generate
             debug packages (xxx-dbg) which you can manually install as
             needed.
         </para>

documentation/profile-manual/profile-manual-usage.xml

@@ -22,7 +22,7 @@
     <para>
         Don't let the fact that it's part of the kernel fool you into thinking
         that it's only for tracing and profiling the kernel - you can indeed
-        use it to trace and profile just the kernel , but you can also use it
+        use it to trace and profile just the kernel, but you can also use it
         to profile specific applications separately (with or without kernel
         context), and you can also use it to trace and profile the kernel
         and all applications on the system simultaneously to gain a system-wide
@@ -30,10 +30,10 @@
     </para>
 
     <para>
-        In many ways, it aims to be a superset of all the tracing and profiling
+        In many ways, perf aims to be a superset of all the tracing and profiling
         tools available in Linux today, including all the other tools covered
         in this HOWTO. The past couple of years have seen perf subsume a lot
-        of the functionality of those other tools, and at the same time those
+        of the functionality of those other tools and, at the same time, those
         other tools have removed large portions of their previous functionality
         and replaced it with calls to the equivalent functionality now
         implemented by the perf subsystem. Extrapolation suggests that at
@@ -126,7 +126,7 @@
      wget <ulink url='http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2'>http://downloads.yoctoproject.org/mirror/sources/linux-2.6.19.2.tar.bz2</ulink>
                 </literallayout>
                 The quickest and easiest way to get some basic overall data about
-                what's going on for a particular workload it to profile it using
+                what's going on for a particular workload is to profile it using
                 'perf stat'. 'perf stat' basically profiles using a few default
                 counters and displays the summed counts at the end of the run:
                 <literallayout class='monospaced'>
@@ -201,7 +201,7 @@
                 As our first attempt at profiling this workload, we'll simply
                 run 'perf record', handing it the workload we want to profile
                 (everything after 'perf record' and any perf options we hand
-                it - here none - will be executedin a new shell). perf collects
+                it - here none - will be executed in a new shell). perf collects
                 samples until the process exits and records them in a file named
                 'perf.data' in the current working directory.
                 <literallayout class='monospaced'>
@@ -241,7 +241,7 @@
                 Notice also that the above report shows an entry for 'busybox',
                 which is the executable that implements 'wget' in Yocto, but that
                 instead of a useful function name in that entry, it displays
-                an not-so-friendly hex value instead. The steps below will show
+                a not-so-friendly hex value instead. The steps below will show
                 how to fix that problem.
             </para>
 
@@ -308,7 +308,7 @@
             </para>
 
             <para>
-                Notice also that here there's also a case where the a hex value
+                Notice also that here there's also a case where the hex value
                 is displayed in the callstack, here in the expanded
                 sys_clock_gettime() function. Later we'll see it resolve to a
                 userspace function call in busybox.
@@ -367,7 +367,7 @@
 
             <para>
                 To generate the debug info for the packages in the image, we can
-                to add dbg-pkgs to EXTRA_IMAGE_FEATURES in local.conf. For example:
+                add dbg-pkgs to EXTRA_IMAGE_FEATURES in local.conf. For example:
                 <literallayout class='monospaced'>
      EXTRA_IMAGE_FEATURES = "debug-tweaks tools-profile dbg-pkgs"
                 </literallayout>
@@ -462,7 +462,7 @@
                 The tracing and profiling infrastructure in Linux has become
                 unified in a way that allows us to use the same tool with a
                 completely different set of counters, not just the standard
-                hardware counters that traditionally tools have had to restrict
+                hardware counters that traditional tools have had to restrict
                 themselves to (of course the traditional tools can also make use
                 of the expanded possibilities now available to them, and in some
                 cases have, as mentioned previously).
@@ -828,7 +828,7 @@
             </para>
 
             <para>
-                Luckily, there is general-purpose way to handle such needs,
+                Luckily, there is a general-purpose way to handle such needs,
                 called 'programming languages'. Making programming languages
                 easily available to apply to such problems given the specific
                 format of data is called a 'programming language binding' for
@@ -842,7 +842,7 @@
                 idea.  One of the first projects to do this was IBM's DProbes
                 dpcc compiler, an ANSI C compiler which targeted a low-level
                 assembly language running on an in-kernel interpreter on the
-                target system.  This is exactly analagous to what Sun's DTrace
+                target system.  This is exactly analogous to what Sun's DTrace
                 did, except that DTrace invented its own language for the purpose.
                 Systemtap, heavily inspired by DTrace, also created its own
                 one-off language, but rather than running the product on an
@@ -925,9 +925,9 @@
                 </literallayout>
                 Each event handler function in the generated code is modified
                 to do this. For convenience, we define a common function called
-                inc_counts() that each handler calls; inc_counts simply tallies
+                inc_counts() that each handler calls; inc_counts() simply tallies
                 a count for each event using the 'counts' hash, which is a
-                specialized has function that does Perl-like autovivification, a
+                specialized hash function that does Perl-like autovivification, a
                 capability that's extremely useful for kinds of multi-level
                 aggregation commonly used in processing traces (see perf's
                 documentation on the Python language binding for details):
@@ -1275,7 +1275,7 @@
             </para>
 
             <informalexample>
-                <emphasis>Tying it Together:</emphasis> The trace events subsystem accomodate static
+                <emphasis>Tying it Together:</emphasis> The trace events subsystem accommodate static
                 and dynamic tracepoints in exactly the same way - there's no
                 difference as far as the infrastructure is concerned.  See the
                 ftrace section for more details on the trace event subsystem.
@@ -1377,7 +1377,7 @@
             the /tracing directory of the mounted debugfs filesystem
             (Yocto follows the standard convention and mounts it
             at /sys/kernel/debug). Here's a listing of all the files
-            found in /sys/kernel/debug/tracing on a Yocto system.:
+            found in /sys/kernel/debug/tracing on a Yocto system:
             <literallayout class='monospaced'>
      root@sugarbay:/sys/kernel/debug/tracing# ls
      README                      kprobe_events               trace
@@ -1634,7 +1634,7 @@
             Also notice that there are various annotations on the left
             hand side of the display. For example if the total time it
             took for a given function to execute is above a certain
-            threshold, and exclamation point or plus sign appears on the
+            threshold, an exclamation point or plus sign appears on the
             left hand side. Please see the ftrace documentation for
             details on all these fields.
         </para>
@@ -1842,7 +1842,7 @@
             </literallayout>
             You can enable any number of events or complete subsystems
             (by using the 'enable' file in the subsystem directory) and
-            get am arbitrarily fine-grained idea of what's going on in the
+            get an arbitrarily fine-grained idea of what's going on in the
             system by enabling as many of the appropriate tracepoints
             as applicable.
         </para>
@@ -1878,14 +1878,14 @@
             in /sys/kernel/debug/tracing, allowing users to specify
             specific particular events within the
             /sys/kernel/debug/tracing/events/ subdirectory and to collect
-            traces and avoiding having to deal with those details directly.
+            traces and avoid having to deal with those details directly.
         </para>
 
         <para>
             As yet another layer on top of that, kernelshark provides a GUI
             that allows users to start and stop traces and specify sets
             of events using an intuitive interface, and view the
-            output as both trace events and as a per-cpu graphical
+            output as both trace events and as a per-CPU graphical
             display. It directly uses 'trace-cmd' as the plumbing
             that accomplishes all that underneath the covers (and
             actually displays the trace-cmd command it uses, as we'll see).
@@ -1896,13 +1896,13 @@
             <literallayout class='monospaced'>
      root@sugarbay:~# kernelshark
             </literallayout>
-            The bring up the 'Capture' dialog by choosing from the
+            Then bring up the 'Capture' dialog by choosing from the
             kernelshark menu:
             <literallayout class='monospaced'>
      Capture | Record
             </literallayout>
             That will display the following dialog, which allows you to
-            choose on or more events (or even one or more complete
+            choose one or more events (or even one or more complete
             subsystems) to trace:
         </para>
 
@@ -1911,7 +1911,7 @@
         </para>
 
         <para>
-            Note that these are exactly the same set of events described
+            Note that these are exactly the same sets of events described
             in the previous trace events subsystem section, and in fact
             is where trace-cmd gets them for kernelshark.
         </para>
@@ -1980,13 +1980,15 @@
             <literallayout class='monospaced'>
      Documentation/trace/events.txt
             </literallayout>
-            There are a nice series of articles on using
+            There is a nice series of articles on using
             ftrace and trace-cmd at LWN:
             <itemizedlist>
                 <listitem><para><ulink url='http://lwn.net/Articles/365835/'>Debugging the kernel using Ftrace - part 1</ulink>
                     </para></listitem>
                 <listitem><para><ulink url='http://lwn.net/Articles/366796/'>Debugging the kernel using Ftrace - part 2</ulink>
                     </para></listitem>
+                <listitem><para><ulink url='http://lwn.net/Articles/370423/'>Secrets of the Ftrace function tracer</ulink>
+                    </para></listitem>
                 <listitem><para><ulink url='https://lwn.net/Articles/410200/'>trace-cmd: A front-end for Ftrace</ulink>
                     </para></listitem>
             </itemizedlist>
@@ -2022,7 +2024,7 @@
         <ulink url='http://sourceware.org/systemtap/tutorial/'>SystemTap tutorial</ulink>
         simply prints a line every time any process on the system open()s
         a file. For each line, it prints the executable name of the
-        program that opened the file, along with its pid, and the name
+        program that opened the file, along with its PID, and the name
         of the file it opened (or tried to open), which it extracts
         from the open syscall's argstr.
         <literallayout class='monospaced'>
@@ -2096,6 +2098,15 @@
             booted. The 'crosstap' script provides details on how
             to do this if you run the script on the host without having
             done a build:
+            <note>
+                SystemTap, which uses 'crosstap', assumes you can establish an
+                ssh connection to the remote target.
+                Please refer to the crosstap wiki page for details on verifying
+                ssh connections at
+                <ulink url='https://wiki.yoctoproject.org/wiki/Tracing_and_Profiling#systemtap'></ulink>.
+                Also, the ability to ssh into the target system is not enabled
+                by default in *-minimal images.
+            </note>
             <literallayout class='monospaced'>
      $ crosstap root@192.168.1.88 trace_open.stp
 
@@ -2122,9 +2133,6 @@
             the EXTRA_IMAGE_FEATURES variable ]
           $ bitbake core-image-sato
 
-      [ NOTE that 'crosstap' needs to be able to ssh into the target
-        system, which isn't enabled by default in -minimal images. ]
-
      Once you've build the image on the host system, you're ready to
      boot it (or the equivalent pre-built image) and use 'crosstap'
      to probe it (you need to source the environment as usual first):
@@ -2195,7 +2203,7 @@
         <para>
             If everything worked as planned, you should see something
             like this (enter the password when prompted, or press enter
-            if its set up to use no password):
+            if it's set up to use no password):
             <literallayout class='monospaced'>
      $ crosstap root@192.168.7.2 trace_open.stp
      root@192.168.7.2's password:
@@ -2240,7 +2248,7 @@
         </para>
 
         <para>
-            For the the section that deals with oprofile from the command-line,
+            For the section that deals with running oprofile from the command-line,
             we assume you've ssh'ed to the host and will be running
             oprofile on the target.
         </para>
@@ -2260,7 +2268,7 @@
             Oprofile as configured in Yocto is a system-wide profiler
             (i.e. the version in Yocto doesn't yet make use of the
             perf_events interface which would allow it to profile
-            specific processes and workloads). It's relies on hardware
+            specific processes and workloads). It relies on hardware
             counter support in the hardware (but can fall back to a
             timer-based mode), which means that it doesn't take
             advantage of tracepoints or other event sources for example.
@@ -2281,8 +2289,8 @@
         <para>
             The oprofile daemon should already be running, but before
             you start profiling, you may need to change some settings
-            and some of these settings may require the daemon not
-            be running. One of these settings is the path the the
+            and some of these settings may require the daemon to not
+            be running. One of these settings is the path to the
             vmlinux file, which you'll want to set using the --vmlinux
             option if you want the kernel profiled:
             <literallayout class='monospaced'>
@@ -2313,7 +2321,7 @@
      Using log file /var/lib/oprofile/samples/oprofiled.log
      Daemon started.
             </literallayout>
-            If we get the status again we now see our updated settings:
+            If we check the status again we now see our updated settings:
             <literallayout class='monospaced'>
      root@crownbay:~# opcontrol --status
      Daemon paused: pid 1649
@@ -2322,7 +2330,7 @@
      Image filter: none
      Call-graph depth: 6
             </literallayout>
-            We're now in a position to run a profile. For that we used
+            We're now in a position to run a profile. For that we use
             'opcontrol --start':
             <literallayout class='monospaced'>
      root@crownbay:~# opcontrol --start
@@ -2334,10 +2342,10 @@
      Connecting to downloads.yoctoproject.org (140.211.169.59:80)
      linux-2.6.19.2.tar.b 100% |*******************************| 41727k  0:00:00 ETA
             </literallayout>
-            To stop the profile we use 'opcontrol --shudown', which not
+            To stop the profile we use 'opcontrol --shutdown', which not
             only stops the profile but shuts down the daemon as well:
             <literallayout class='monospaced'>
-     root@crownbay:~# opcontrol --start
+     root@crownbay:~# opcontrol --shutdown
      Stopping profiling.
      Killing daemon.
             </literallayout>
@@ -2896,7 +2904,7 @@
 
         <para>
             Once you've applied the above commits and built and booted your
-            image (you need to build the core-image-sato-sdk image or the
+            image (you need to build the core-image-sato-sdk image or use one of the
             other methods described in the General Setup section), you're
             ready to start tracing.
         </para>
@@ -2905,7 +2913,7 @@
             <title>Collecting and viewing a trace on the target (inside a shell)</title>
 
             <para>
-                First, from the target, ssh to the target:
+                First, from the host, ssh to the target:
                 <literallayout class='monospaced'>
      $ ssh -l root 192.168.1.47
      The authenticity of host '192.168.1.47 (192.168.1.47)' can't be established.
@@ -3006,7 +3014,7 @@
             <title>Collecting and viewing a userspace trace on the target (inside a shell)</title>
 
             <para>
-                For lttng userspace tracing, you need to have a properly
+                For LTTng userspace tracing, you need to have a properly
                 instrumented userspace program. For this example, we'll use
                 the 'hello' test program generated by the lttng-ust build.
             </para>
@@ -3028,7 +3036,7 @@
             </para>
 
             <para>
-                First, from the target, ssh to the target:
+                First, from the host, ssh to the target:
                 <literallayout class='monospaced'>
      $ ssh -l root 192.168.1.47
      The authenticity of host '192.168.1.47 (192.168.1.47)' can't be established.
@@ -3094,7 +3102,7 @@
                 If you already have an LTTng trace on a remote target and
                 would like to view it in Eclipse on the host, you can easily
                 copy it from the target to the host and import it into
-                Eclipse to view it using the LTTng Eclipse plugin already
+                Eclipse to view it using the LTTng Eclipse plug-in already
                 bundled in the Eclipse (Juno SR1 or greater).
             </para>
 
@@ -3172,7 +3180,7 @@
 
             <para>
                 You can access extensive help information on how to use
-                the LTTng plugin to search and analyze captured traces via
+                the LTTng plug-in to search and analyze captured traces via
                 the Eclipse help system:
                 <literallayout class='monospaced'>
      Help | Help Contents | LTTng Plug-in User Guide
@@ -3250,15 +3258,25 @@
         <title>Documentation</title>
 
         <para>
-            There doesn't seem to be any current documentation covering
-            LTTng 2.0, but maybe that's because the project is in transition.
-            The LTTng 2.0 website, however, is here:
-            <ulink url='http://lttng.org/lttng2.0'>LTTng Project</ulink>
+            You can find the primary LTTng Documentation on the
+            <ulink url='https://lttng.org/docs/'>LTTng Documentation</ulink>
+            site.
+            The documentation on this site is appropriate for intermediate to
+            advanced software developers who are working in a Linux environment
+            and are interested in efficient software tracing.
         </para>
 
         <para>
-            You can access extensive help information on how to use the
-            LTTng plug-in to search and analyze captured traces via the
+            For information on LTTng in general, visit the
+            <ulink url='http://lttng.org/lttng2.0'>LTTng Project</ulink>
+            site.
+            You can find a "Getting Started" link on this site that takes
+            you to an LTTng Quick Start.
+        </para>
+
+        <para>
+            Finally, you can access extensive help information on how to use
+            the LTTng plug-in to search and analyze captured traces via the
             Eclipse help system:
             <literallayout class='monospaced'>
      Help | Help Contents | LTTng Plug-in User Guide
@@ -3594,7 +3612,7 @@
                 It's also possible to trace block I/O using only
                 <link linkend='the-trace-events-subsystem'>trace events subsystem</link>,
                 which can be useful for casual tracing
-                if you don't want bother dealing with the userspace tools.
+                if you don't want to bother dealing with the userspace tools.
             </para>
 
             <para>

documentation/profile-manual/profile-manual.xml

@@ -38,9 +38,29 @@
             </revision>
             <revision>
                 <revnumber>1.5</revnumber>
-                <date>Sometime in 2013</date>
+                <date>October 2013</date>
                 <revremark>Released with the Yocto Project 1.5 Release.</revremark>
             </revision>
+            <revision>
+                <revnumber>1.5.1</revnumber>
+                <date>January 2014</date>
+                <revremark>Released with the Yocto Project 1.5.1 Release.</revremark>
+            </revision>
+            <revision>
+                <revnumber>1.5.2</revnumber>
+                <date>May 2014</date>
+                <revremark>Released with the Yocto Project 1.5.2 Release.</revremark>
+            </revision>
+            <revision>
+                <revnumber>1.5.3</revnumber>
+                <date>July 2014</date>
+                <revremark>Released with the Yocto Project 1.5.3 Release.</revremark>
+            </revision>
+            <revision>
+                <revnumber>1.5.4</revnumber>
+                <date>December 2014</date>
+                <revremark>Released with the Yocto Project 1.5.4 Release.</revremark>
+            </revision>
         </revhistory>
 
     <copyright>
@@ -57,11 +77,10 @@
       </para>
 
       <note>
-          Due to production processes, there could be differences between the Yocto Project
-          documentation bundled in the release tarball and the
-          <ulink url='&YOCTO_DOCS_PROF_URL;'>Yocto Project Tracing and Profiling Manual</ulink> on
-          the <ulink url='&YOCTO_HOME_URL;'>Yocto Project</ulink> website.
-          For the latest version of this manual, see the manual on the website.
+          For the latest version of this manual associated with this
+          Yocto Project release, see the
+          <ulink url='&YOCTO_DOCS_PROF_URL;'>Yocto Project Profiling and Tracing Manual</ulink>
+          from the Yocto Project website.
       </note>
     </legalnotice>
 

documentation/ref-manual/faq.xml

@@ -31,31 +31,19 @@
     <qandaentry>
         <question>
             <para>
-                I only have Python 2.4 or 2.5 but BitBake requires Python 2.6 or 2.7.
+                My development system does not have Python 2.7.3 or greater,
+                which the Yocto Project requires.
                 Can I still use the Yocto Project?
             </para>
         </question>
         <answer>
             <para>
-                You can use a stand-alone tarball to provide Python 2.6.
-                You can find pre-built 32 and 64-bit versions of Python 2.6 at the following locations:
-                <itemizedlist>
-                    <listitem><para><ulink url='&YOCTO_PYTHON-i686_DL_URL;'>32-bit tarball</ulink></para></listitem>
-                    <listitem><para><ulink url='&YOCTO_PYTHON-x86_64_DL_URL;'>64-bit tarball</ulink></para></listitem>
-                </itemizedlist>
-            </para>
-            <para>
-                These tarballs are self-contained with all required libraries and should work
-                on most Linux systems.
-                To use the tarballs extract them into the root
-                directory and run the appropriate command:
-                <literallayout class='monospaced'>
-     $ export PATH=/opt/poky/sysroots/i586-pokysdk-linux/usr/bin/:$PATH
-     $ export PATH=/opt/poky/sysroots/x86_64-pokysdk-linux/usr/bin/:$PATH
-                </literallayout>
-            </para>
-            <para>
-                Once you run the command, BitBake uses Python 2.6.
+                You can get the required tools on your host development
+                system a couple different ways (i.e. building a tarball or
+                downloading a tarball).
+                See the
+                "<link linkend='required-git-tar-and-python-versions'>Required Git, tar, and Python Versions</link>"
+                section for steps on how to update your build tools.
             </para>
         </answer>
     </qandaentry>
@@ -211,7 +199,7 @@
         </answer>
     </qandaentry>
 
-    <qandaentry>
+<!--    <qandaentry>
         <question>
             <para>
                 How do I make the Yocto Project work in RHEL/CentOS?
@@ -258,7 +246,7 @@
                 Wiki page.</para>
             </note>
         </answer>
-    </qandaentry>
+    </qandaentry> -->
 
     <qandaentry>
         <question>
@@ -682,9 +670,11 @@
             <para>
                 Yes - you can easily do this.
                 When you use BitBake to build an image, all the build output
-                goes into the directory created when you source the
+                goes into the directory created when you run the
+                build environment setup script (i.e.
                 <link linkend='structure-core-script'><filename>&OE_INIT_FILE;</filename></link>
-                setup script.
+                or
+                <link linkend='structure-memres-core-script'><filename>oe-init-build-env-memres</filename></link>).
                 By default, this <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>
                 is named <filename>build</filename> but can be named
                 anything you want.