build-appliance: Update SRCREV for release

(From OE-Core rev: 1149b1fef8912f77d971242dfec151fff5a3aa51)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

README.hardware

@@ -75,14 +75,32 @@ Intel x86 based PCs and devices (genericx86)
 
 The genericx86 MACHINE is tested on the following platforms:
 
-  o Asus EeePC 901
-  o Acer Aspire One
-  o Toshiba NB305
-  o Intel Embedded Development Board 1-N450 (Black Sand)
+Intel Xeon/Core i-Series:
+  + Intel Romley Server: Sandy Bridge Xeon processor, C600 PCH (Patsburg), (Canoe Pass CRB)
+  + Intel Romley Server: Ivy Bridge Xeon processor, C600 PCH (Patsburg), (Intel SDP S2R3)
+  + Intel Crystal Forest Server: Sandy Bridge Xeon processor, DH89xx PCH (Cave Creek), (Stargo CRB)
+  + Intel Chief River Mobile: Ivy Bridge Mobile processor, QM77 PCH (Panther Point-M), (Emerald Lake II CRB, Sabino Canyon CRB)
+  + Intel Huron River Mobile: Sandy Bridge processor, QM67 PCH (Cougar Point), (Emerald Lake CRB, EVOC EC7-1817LNAR board)
+  + Intel Calpella Platform: Core i7 processor, QM57 PCH (Ibex Peak-M), (Red Fort CRB, Emerson MATXM CORE-411-B)
+  + Intel Nehalem/Westmere-EP Server: Xeon 56xx/55xx processors, 5520 chipset, ICH10R IOH (82801), (Hanlan Creek CRB)
+  + Intel Nehalem Workstation: Xeon 56xx/55xx processors, System SC5650SCWS (Greencity CRB)
+  + Intel Picket Post Server: Xeon 56xx/55xx processors (Jasper Forest), 3420 chipset (Ibex Peak), (Osage CRB)
+  + Intel Storage Platform: Sandy Bridge Xeon processor, C600 PCH (Patsburg), (Oak Creek Canyon CRB)
+  + Intel Shark Bay Client Platform: Haswell processor, LynxPoint PCH, (Walnut Canyon CRB, Lava Canyon CRB, Basking Ridge CRB, Flathead Creek CRB)
+  + Intel Shark Bay Ultrabook Platform: Haswell ULT processor, Lynx Point-LP PCH, (WhiteTip Mountain 1 CRB)
 
-and is likely to work on many unlisted Atom based devices. The MACHINE type
-supports ethernet, wifi, sound, and i915 graphics by default in addition to
-common PC input devices, busses, and so on.
+Intel Atom platforms:
+  + Intel embedded Menlow: Intel Atom Z510/530 CPU, System Controller Hub US15W (Portwell NANO-8044)
+  + Intel Luna Pier: Intel Atom N4xx/D5xx series CPU (aka: Pineview-D & -M), 82801HM I/O Hub (ICH8M), (Advantech AIMB-212, Moon Creek CRB)
+  + Intel Queens Bay platform: Intel Atom E6xx CPU (aka: Tunnel Creek), Topcliff EG20T I/O Hub (Emerson NITX-315, Crown Bay CRB, Minnow Board)
+  + Intel Fish River Island platform: Intel Atom E6xx CPU (aka: Tunnel Creek), Topcliff EG20T I/O Hub (Kontron KM2M806)
+  + Intel Cedar Trail platform: Intel Atom N2000 & D2000 series CPU (aka: Cedarview), NM10 Express Chipset (Norco kit BIS-6630, Cedar Rock CRB)
+
+and is likely to work on many unlisted Atom/Core/Xeon based devices. The MACHINE
+type supports ethernet, wifi, sound, and Intel/vesa graphics by default in
+addition to common PC input devices, busses, and so on.  Note that it does not
+included the binary-only graphic drivers used on some Atom platforms, for
+accelerated graphics on these machines please refer to meta-intel.
 
 Depending on the device, it can boot from a traditional hard-disk, a USB device,
 or over the network. Writing generated images to physical media is

bitbake/bin/bitbake

@@ -41,7 +41,7 @@ from bb import ui
 from bb import server
 from bb import cookerdata
 
-__version__ = "1.19.1"
+__version__ = "1.20.0"
 logger = logging.getLogger("BitBake")
 
 # Python multiprocessing requires /dev/shm
@@ -75,18 +75,6 @@ def get_ui(config):
         sys.exit("FATAL: Invalid user interface '%s' specified.\n"
                  "Valid interfaces: depexp, goggle, ncurses, hob, knotty [default]." % interface)
 
-def gather_extra_cache_data():
-    extra = []
-    interfaces = ['depexp', 'goggle', 'ncurses', 'hob', 'knotty']
-    for i in interfaces:
-        try:
-            ui = __import__("bb.ui." + i, fromlist = [i])
-            if hasattr(ui, "extraCaches"):
-                extra = extra + ui.extraCaches
-            del ui
-        except:
-            pass
-    return extra
 
 # Display bitbake/OE warnings via the BitBake.Warnings logger, ignoring others"""
 warnlog = logging.getLogger("BitBake.Warnings")
@@ -112,59 +100,58 @@ class BitBakeConfigParameters(cookerdata.ConfigParameters):
     def parseCommandLine(self):
         parser = optparse.OptionParser(
             version = "BitBake Build Tool Core version %s, %%prog version %s" % (bb.__version__, __version__),
-            usage = """%prog [options] [package ...]
+            usage = """%prog [options] [recipename/target ...]
 
-    Executes the specified task (default is 'build') for a given set of BitBake files.
-    It expects that BBFILES is defined, which is a space separated list of files to
-    be executed.  BBFILES does support wildcards.
-    Default BBFILES are the .bb files in the current directory.""")
+    Executes the specified task (default is 'build') for a given set of target recipes (.bb files).
+    It is assumed there is a conf/bblayers.conf available in cwd or in BBPATH which
+    will provide the layer, BBFILES and other configuration information.""")
 
-        parser.add_option("-b", "--buildfile", help = "execute the task against this .bb file, rather than a package from BBFILES. Does not handle any dependencies.",
+        parser.add_option("-b", "--buildfile", help = "Execute tasks from a specific .bb recipe directly. WARNING: Does not handle any dependencies from other recipes.",
                    action = "store", dest = "buildfile", default = None)
 
-        parser.add_option("-k", "--continue", help = "continue as much as possible after an error. While the target that failed, and those that depend on it, cannot be remade, the other dependencies of these targets can be processed all the same.",
+        parser.add_option("-k", "--continue", help = "Continue as much as possible after an error. While the target that failed and anything depending on it cannot be built, as much as possible will be built before stopping.",
                    action = "store_false", dest = "abort", default = True)
 
-        parser.add_option("-a", "--tryaltconfigs", help = "continue with builds by trying to use alternative providers where possible.",
+        parser.add_option("-a", "--tryaltconfigs", help = "Continue with builds by trying to use alternative providers where possible.",
                    action = "store_true", dest = "tryaltconfigs", default = False)
 
-        parser.add_option("-f", "--force", help = "force run of specified cmd, regardless of stamp status",
+        parser.add_option("-f", "--force", help = "Force the specified targets/task to run (invalidating any existing stamp file).",
                    action = "store_true", dest = "force", default = False)
 
-        parser.add_option("-c", "--cmd", help = "Specify task to execute. Note that this only executes the specified task for the providee and the packages it depends on, i.e. 'compile' does not implicitly call stage for the dependencies (IOW: use only if you know what you are doing). Depending on the base.bbclass a listtasks tasks is defined and will show available tasks",
+        parser.add_option("-c", "--cmd", help = "Specify the task to execute. The exact options available depend on the metadata. Some examples might be 'compile' or 'populate_sysroot' or 'listtasks' may give a list of the tasks available.",
                    action = "store", dest = "cmd")
 
-        parser.add_option("-C", "--clear-stamp", help = "Invalidate the stamp for the specified cmd such as 'compile' and run the default task for the specified target(s)",
+        parser.add_option("-C", "--clear-stamp", help = "Invalidate the stamp for the specified task such as 'compile' and then run the default task for the specified target(s).",
                     action = "store", dest = "invalidate_stamp")
 
-        parser.add_option("-r", "--read", help = "read the specified file before bitbake.conf",
+        parser.add_option("-r", "--read", help = "Read the specified file before bitbake.conf.",
                    action = "append", dest = "prefile", default = [])
 
-        parser.add_option("-R", "--postread", help = "read the specified file after bitbake.conf",
+        parser.add_option("-R", "--postread", help = "Read the specified file after bitbake.conf.",
                           action = "append", dest = "postfile", default = [])
 
-        parser.add_option("-v", "--verbose", help = "output more chit-chat to the terminal",
+        parser.add_option("-v", "--verbose", help = "Output more log message data to the terminal.",
                    action = "store_true", dest = "verbose", default = False)
 
         parser.add_option("-D", "--debug", help = "Increase the debug level. You can specify this more than once.",
                    action = "count", dest="debug", default = 0)
 
-        parser.add_option("-n", "--dry-run", help = "don't execute, just go through the motions",
+        parser.add_option("-n", "--dry-run", help = "Don't execute, just go through the motions.",
                    action = "store_true", dest = "dry_run", default = False)
 
-        parser.add_option("-S", "--dump-signatures", help = "don't execute, just dump out the signature construction information",
+        parser.add_option("-S", "--dump-signatures", help = "Don't execute, just dump out the signature construction information.",
                    action = "store_true", dest = "dump_signatures", default = False)
 
-        parser.add_option("-p", "--parse-only", help = "quit after parsing the BB files (developers only)",
+        parser.add_option("-p", "--parse-only", help = "Quit after parsing the BB recipes.",
                    action = "store_true", dest = "parse_only", default = False)
 
-        parser.add_option("-s", "--show-versions", help = "show current and preferred versions of all recipes",
+        parser.add_option("-s", "--show-versions", help = "Show current and preferred versions of all recipes.",
                    action = "store_true", dest = "show_versions", default = False)
 
-        parser.add_option("-e", "--environment", help = "show the global or per-package environment (this is what used to be bbread)",
+        parser.add_option("-e", "--environment", help = "Show the global or per-package environment complete with information about where variables were set/changed.",
                    action = "store_true", dest = "show_environment", default = False)
 
-        parser.add_option("-g", "--graphviz", help = "emit the dependency trees of the specified packages in the dot syntax, and the pn-buildlist to show the build list",
+        parser.add_option("-g", "--graphviz", help = "Save dependency tree information for the specified targets in the dot syntax.",
                     action = "store_true", dest = "dot_graph", default = False)
 
         parser.add_option("-I", "--ignore-deps", help = """Assume these dependencies don't exist and are already provided (equivalent to ASSUME_PROVIDED). Useful to make dependency graphs more appealing""",
@@ -173,34 +160,34 @@ class BitBakeConfigParameters(cookerdata.ConfigParameters):
         parser.add_option("-l", "--log-domains", help = """Show debug logging for the specified logging domains""",
                     action = "append", dest = "debug_domains", default = [])
 
-        parser.add_option("-P", "--profile", help = "profile the command and print a report",
+        parser.add_option("-P", "--profile", help = "Profile the command and save reports.",
                    action = "store_true", dest = "profile", default = False)
 
-        parser.add_option("-u", "--ui", help = "userinterface to use",
+        parser.add_option("-u", "--ui", help = "The user interface to use (e.g. knotty, hob, depexp).",
                    action = "store", dest = "ui")
 
-        parser.add_option("-t", "--servertype", help = "Choose which server to use, process or xmlrpc",
+        parser.add_option("-t", "--servertype", help = "Choose which server to use, process or xmlrpc.",
                    action = "store", dest = "servertype")
 
-        parser.add_option("", "--revisions-changed", help = "Set the exit code depending on whether upstream floating revisions have changed or not",
+        parser.add_option("", "--revisions-changed", help = "Set the exit code depending on whether upstream floating revisions have changed or not.",
                    action = "store_true", dest = "revisions_changed", default = False)
 
-        parser.add_option("", "--server-only", help = "Run bitbake without UI,  the frontend can connect with bitbake server itself",
+        parser.add_option("", "--server-only", help = "Run bitbake without a UI, only starting a server (cooker) process.",
                    action = "store_true", dest = "server_only", default = False)
 
-        parser.add_option("-B", "--bind", help = "The name/address for the bitbake server to bind to",
+        parser.add_option("-B", "--bind", help = "The name/address for the bitbake server to bind to.",
                    action = "store", dest = "bind", default = False)
 
-        parser.add_option("", "--no-setscene", help = "Do not run any setscene tasks, forces builds",
+        parser.add_option("", "--no-setscene", help = "Do not run any setscene tasks. sstate will be ignored and everything needed, built.",
                    action = "store_true", dest = "nosetscene", default = False)
 
-        parser.add_option("", "--remote-server", help = "Connect to the specified server",
+        parser.add_option("", "--remote-server", help = "Connect to the specified server.",
                    action = "store", dest = "remote_server", default = False)
 
-        parser.add_option("-m", "--kill-server", help = "Terminate the remote server",
+        parser.add_option("-m", "--kill-server", help = "Terminate the remote server.",
                     action = "store_true", dest = "kill_server", default = False)
 
-        parser.add_option("", "--observe-only", help = "Connect to a server as an observing-only client",
+        parser.add_option("", "--observe-only", help = "Connect to a server as an observing-only client.",
                    action = "store_true", dest = "observe_only", default = False)
 
         options, targets = parser.parse_args(sys.argv)
@@ -275,8 +262,8 @@ def main():
         if not configParams.bind:
             sys.exit("FATAL: The '--server-only' option requires a name/address to bind to with the -B option.\n")
         if configParams.remote_server:
-            sys.exit("FATAL: The '--server-only' option conflicts with the '--remote-server' option. %s\n" % 
-                ("Please check your BBSERVER environment" if "BBSERVER" in os.environ else "" ))
+            sys.exit("FATAL: The '--server-only' option conflicts with %s.\n" % 
+                ("the BBSERVER environment variable" if "BBSERVER" in os.environ else "the '--remote-server' option" ))
 
     if configParams.bind and configParams.servertype != "xmlrpc":
         sys.exit("FATAL: If '-B' or '--bind' is defined, we must set the servertype as 'xmlrpc'.\n")
@@ -302,30 +289,28 @@ def main():
     # Clear away any spurious environment variables while we stoke up the cooker
     cleanedvars = bb.utils.clean_environment()
 
-    # Collect all the caches we need
-    if configParams.server_only:
-        configuration.extra_caches = gather_extra_cache_data()
-    else:
-        configuration.extra_caches = getattr(ui_module, "extraCaches", [])
-
     if not configParams.remote_server:
         # we start a server with a given configuration
         server = start_server(servermodule, configParams, configuration)
+        bb.event.ui_queue = []
     else:
-        # we start a stub server that is actually a XMLRPClient to
+        # we start a stub server that is actually a XMLRPClient that connects to a real server
         server = servermodule.BitBakeXMLRPCClient(configParams.observe_only)
         server.saveConnectionDetails(configParams.remote_server)
 
-    logger.removeHandler(handler)
-
     if not configParams.server_only:
+        # Collect the feature set for the UI
+        featureset = getattr(ui_module, "featureSet", [])
+
         # Setup a connection to the server (cooker)
-        server_connection = server.establishConnection()
+        server_connection = server.establishConnection(featureset)
 
         # Restore the environment in case the UI needs it
         for k in cleanedvars:
             os.environ[k] = cleanedvars[k]
 
+        logger.removeHandler(handler)
+
         try:
             return ui_module.main(server_connection.connection, server_connection.events, configParams)
         finally:
@@ -339,6 +324,8 @@ def main():
 if __name__ == "__main__":
     try:
         ret = main()
+    except bb.BBHandledException:
+        ret = 1
     except Exception:
         ret = 1
         import traceback

bitbake/bin/bitbake-diffsigs

@@ -3,7 +3,7 @@
 # bitbake-diffsigs
 # BitBake task signature data comparison utility
 #
-# Copyright (C) 2012 Intel Corporation
+# Copyright (C) 2012-2013 Intel Corporation
 #
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License version 2 as
@@ -30,7 +30,18 @@ sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(sys.argv[0])), '
 import bb.tinfoil
 import bb.siggen
 
-logger = logging.getLogger('BitBake')
+def logger_create(name, output=sys.stderr):
+    logger = logging.getLogger(name)
+    console = logging.StreamHandler(output)
+    format = bb.msg.BBLogFormatter("%(levelname)s: %(message)s")
+    if output.isatty():
+        format.enable_color()
+    console.setFormatter(format)
+    logger.addHandler(console)
+    logger.setLevel(logging.INFO)
+    return logger
+
+logger = logger_create('bitbake-diffsigs')
 
 def find_compare_task(bbhandler, pn, taskname):
     """ Find the most recent signature files for the specified PN/task and compare them """
@@ -39,6 +50,9 @@ def find_compare_task(bbhandler, pn, taskname):
         logger.error('Metadata does not support finding signature data files')
         sys.exit(1)
 
+    if not taskname.startswith('do_'):
+        taskname = 'do_%s' % taskname
+
     filedates = bb.siggen.find_siginfo(pn, taskname, None, bbhandler.config_data)
     latestfiles = sorted(filedates.keys(), key=lambda f: filedates[f])[-2:]
     if not latestfiles:
@@ -71,6 +85,7 @@ def find_compare_task(bbhandler, pn, taskname):
 
 
 parser = optparse.OptionParser(
+    description = "Compares siginfo/sigdata files written out by BitBake",
     usage = """
   %prog -t recipename taskname
   %prog sigdatafile1 sigdatafile2
@@ -78,25 +93,30 @@ parser = optparse.OptionParser(
 
 parser.add_option("-t", "--task",
         help = "find the signature data files for last two runs of the specified task and compare them",
-        action="store_true", dest="taskmode")
+        action="store", dest="taskargs", nargs=2, metavar='recipename taskname')
 
 options, args = parser.parse_args(sys.argv)
 
-if len(args) == 1:
-    parser.print_help()
+if options.taskargs:
+    tinfoil = bb.tinfoil.Tinfoil()
+    tinfoil.prepare(config_only = True)
+    find_compare_task(tinfoil, options.taskargs[0], options.taskargs[1])
 else:
-    if options.taskmode:
-        tinfoil = bb.tinfoil.Tinfoil()
-        if len(args) < 3:
-            logger.error("Please specify a recipe and task name")
-            sys.exit(1)
-        tinfoil.prepare(config_only = True)
-        find_compare_task(tinfoil, args[1], args[2])
+    if len(args) == 1:
+        parser.print_help()
     else:
-        if len(args) == 2:
-            output = bb.siggen.dump_sigfile(sys.argv[1])
-        else:
-            output = bb.siggen.compare_sigfiles(sys.argv[1], sys.argv[2])
+        import cPickle
+        try:
+            if len(args) == 2:
+                output = bb.siggen.dump_sigfile(sys.argv[1])
+            else:
+                output = bb.siggen.compare_sigfiles(sys.argv[1], sys.argv[2])
+        except IOError as e:
+            logger.error(str(e))
+            sys.exit(1)
+        except cPickle.UnpicklingError, EOFError:
+            logger.error('Invalid signature data - ensure you are specifying sigdata/siginfo files')
+            sys.exit(1)
 
-        if output:
-            print '\n'.join(output)
+    if output:
+        print '\n'.join(output)

bitbake/bin/bitbake-dumpsig

@@ -1,11 +1,65 @@
 #!/usr/bin/env python
+
+# bitbake-dumpsig
+# BitBake task signature dump utility
+#
+# Copyright (C) 2013 Intel Corporation
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
 import os
 import sys
 import warnings
+import optparse
+import logging
+
 sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(sys.argv[0])), 'lib'))
 
 import bb.siggen
 
-output = bb.siggen.dump_sigfile(sys.argv[1])
-if output:
-    print '\n'.join(output)
+def logger_create(name, output=sys.stderr):
+    logger = logging.getLogger(name)
+    console = logging.StreamHandler(output)
+    format = bb.msg.BBLogFormatter("%(levelname)s: %(message)s")
+    if output.isatty():
+        format.enable_color()
+    console.setFormatter(format)
+    logger.addHandler(console)
+    logger.setLevel(logging.INFO)
+    return logger
+
+logger = logger_create('bitbake-dumpsig')
+
+parser = optparse.OptionParser(
+    description = "Dumps siginfo/sigdata files written out by BitBake",
+    usage = """
+  %prog sigdatafile""")
+
+options, args = parser.parse_args(sys.argv)
+
+if len(args) == 1:
+    parser.print_help()
+else:
+    import cPickle
+    try:
+        output = bb.siggen.dump_sigfile(args[1])
+    except IOError as e:
+        logger.error(str(e))
+        sys.exit(1)
+    except cPickle.UnpicklingError, EOFError:
+        logger.error('Invalid signature data - ensure you are specifying a sigdata/siginfo file')
+        sys.exit(1)
+
+    if output:
+        print '\n'.join(output)

bitbake/bin/bitbake-worker

@@ -148,6 +148,8 @@ def fork_off_task(cfg, data, workerdata, fn, task, taskname, appends, quieterror
 
             data.setVar("BB_WORKERCONTEXT", "1")
             data.setVar("BUILDNAME", workerdata["buildname"])
+            data.setVar("DATE", workerdata["date"])
+            data.setVar("TIME", workerdata["time"])
             bb.parse.siggen.set_taskdata(workerdata["hashes"], workerdata["hash_deps"], workerdata["sigchecksums"])
             ret = 0
             try:

bitbake/contrib/vim/ftdetect/bitbake.vim

@@ -14,7 +14,7 @@ endif
 au BufNewFile,BufRead *.{bb,bbappend,bbclass}	set filetype=bitbake
 
 " .inc
-au BufNewFile,BufRead *.inc 		set filetype=bitbake
+au BufNewFile,BufRead *.inc		set filetype=bitbake
 
 " .conf
 au BufNewFile,BufRead *.conf

bitbake/contrib/vim/ftplugin/bitbake.vim

@@ -1 +1,2 @@
 set sts=4 sw=4 et
+set cms=#%s

bitbake/contrib/vim/plugin/newbb.vim

@@ -15,7 +15,7 @@ if &compatible || v:version < 600
 endif
 
 fun! <SID>GetUserName()
-    let l:user_name = system("git-config --get user.name")
+    let l:user_name = system("git config --get user.name")
     if v:shell_error
         return "Unknow User"
     else
@@ -23,7 +23,7 @@ fun! <SID>GetUserName()
 endfun
 
 fun! <SID>GetUserEmail()
-    let l:user_email = system("git-config --get user.email")
+    let l:user_email = system("git config --get user.email")
     if v:shell_error
         return "unknow@user.org"
     else

bitbake/contrib/vim/syntax/bitbake.vim

@@ -44,8 +44,8 @@ syn match bbArrayBrackets       "[\[\]]" contained
 
 " BitBake strings
 syn match bbContinue            "\\$"
-syn region bbString             matchgroup=bbQuote start=+"+ skip=+\\$+ excludenl end=+"+ contained keepend contains=bbTodo,bbContinue,bbVarDeref,bbVarPyValue,@Spell
-syn region bbString             matchgroup=bbQuote start=+'+ skip=+\\$+ excludenl end=+'+ contained keepend contains=bbTodo,bbContinue,bbVarDeref,bbVarPyValue,@Spell
+syn region bbString             matchgroup=bbQuote start=+"+ skip=+\\$+ end=+"+ contained contains=bbTodo,bbContinue,bbVarDeref,bbVarPyValue,@Spell
+syn region bbString             matchgroup=bbQuote start=+'+ skip=+\\$+ end=+'+ contained contains=bbTodo,bbContinue,bbVarDeref,bbVarPyValue,@Spell
 
 " Vars definition
 syn match bbExport            "^export" nextgroup=bbIdentifier skipwhite
@@ -55,11 +55,11 @@ syn match bbVarDeref            "${[a-zA-Z0-9\-_\.\/\+]\+}" contained
 syn match bbVarEq               "\(:=\|+=\|=+\|\.=\|=\.\|?=\|??=\|=\)" contained nextgroup=bbVarValue
 syn match bbVarDef              "^\(export\s*\)\?\([a-zA-Z0-9\-_\.\/\+]\+\(_[${}a-zA-Z0-9\-_\.\/\+]\+\)\?\)\s*\(:=\|+=\|=+\|\.=\|=\.\|?=\|??=\|=\)\@=" contains=bbExportFlag,bbIdentifier,bbVarDeref nextgroup=bbVarEq
 syn match bbVarValue            ".*$" contained contains=bbString,bbVarDeref,bbVarPyValue
-syn region bbVarPyValue         start=+${@+ skip=+\\$+ excludenl end=+}+ contained contains=@python
+syn region bbVarPyValue         start=+${@+ skip=+\\$+ end=+}+ contained contains=@python
 
 " Vars metadata flags
 syn match bbVarFlagDef          "^\([a-zA-Z0-9\-_\.]\+\)\(\[[a-zA-Z0-9\-_\.]\+\]\)\@=" contains=bbIdentifier nextgroup=bbVarFlagFlag
-syn region bbVarFlagFlag        matchgroup=bbArrayBrackets start="\[" end="\]\s*\(=\)\@=" keepend excludenl contained contains=bbIdentifier nextgroup=bbVarEq
+syn region bbVarFlagFlag        matchgroup=bbArrayBrackets start="\[" end="\]\s*\(=\|+=\|=+\|?=\)\@=" contained contains=bbIdentifier nextgroup=bbVarEq
 
 " Includes and requires
 syn keyword bbInclude           inherit include require contained 
@@ -83,13 +83,16 @@ if exists("b:current_syntax")
   unlet b:current_syntax
 endif
 syn keyword bbShFakeRootFlag    fakeroot contained
-syn match bbShFuncDef           "^\(fakeroot\s*\)\?\([0-9A-Za-z_-]\+\)\(python\)\@<!\(\s*()\s*\)\({\)\@=" contains=bbShFakeRootFlag,bbFunction,bbDelimiter nextgroup=bbShFuncRegion skipwhite
-syn region bbShFuncRegion       matchgroup=bbDelimiter start="{\s*$" end="^}\s*$" keepend contained contains=@shell
+syn match bbShFuncDef           "^\(fakeroot\s*\)\?\([0-9A-Za-z_${}-]\+\)\(python\)\@<!\(\s*()\s*\)\({\)\@=" contains=bbShFakeRootFlag,bbFunction,bbVarDeref,bbDelimiter nextgroup=bbShFuncRegion skipwhite
+syn region bbShFuncRegion       matchgroup=bbDelimiter start="{\s*$" end="^}\s*$" contained contains=@shell
+
+" Python value inside shell functions
+syn region shDeref         start=+${@+ skip=+\\$+ excludenl end=+}+ contained contains=@python
 
 " BitBake python metadata
 syn keyword bbPyFlag            python contained
-syn match bbPyFuncDef           "^\(python\s\+\)\([0-9A-Za-z_-]\+\)\?\(\s*()\s*\)\({\)\@=" contains=bbPyFlag,bbFunction,bbDelimiter nextgroup=bbPyFuncRegion skipwhite 
-syn region bbPyFuncRegion       matchgroup=bbDelimiter start="{\s*$" end="^}\s*$" keepend contained contains=@python
+syn match bbPyFuncDef           "^\(python\s\+\)\([0-9A-Za-z_${}-]\+\)\?\(\s*()\s*\)\({\)\@=" contains=bbPyFlag,bbFunction,bbVarDeref,bbDelimiter nextgroup=bbPyFuncRegion skipwhite
+syn region bbPyFuncRegion       matchgroup=bbDelimiter start="{\s*$" end="^}\s*$" contained contains=@python
 
 " BitBake 'def'd python functions
 syn keyword bbPyDef             def contained

bitbake/doc/manual/usermanual.xml

@@ -137,6 +137,24 @@ share common metadata between many packages.</para></listitem>
                 <para>In this example, <varname>B</varname> is now <literal>bvaladditionaldata</literal> and <varname>C</varname> is <literal>testcval</literal>. In contrast to the above appending and prepending operators, no additional space
 will be introduced.</para>
             </section>
+            <section>
+                <title>Appending and Prepending (override style syntax)</title>
+                    <para><screen><varname>B</varname> = "bval"
+<varname>B_append</varname> = " additional data"
+<varname>C</varname> = "cval"
+<varname>C_prepend</varname> = "additional data "</screen></para>
+                 <para>This example results in <varname>B</varname> becoming <literal>bval additional data</literal>
+and <varname>C</varname> becoming <literal>additional data cval</literal>. Note the spaces in the append.
+Unlike the += operator, additional space is not automatically added.  You must take steps to add space
+yourself.</para>
+            </section>
+            <section>
+                <title>Removing (override style syntax)</title>
+                <para><screen><varname>FOO</varname> = "123 456 789 123456 123 456 123 456"
+<varname>FOO_remove</varname> = "123"
+<varname>FOO_remove</varname> = "456"</screen></para>
+                <para>In this example, <varname>FOO</varname> is now <literal>789 123456</literal>.</para>
+            </section>
             <section>
                 <title>Conditional metadata set</title>
                 <para>OVERRIDES is a <quote>:</quote> separated variable containing each item you want to satisfy conditions.  So, if you have a variable which is conditional on <quote>arm</quote>, and <quote>arm</quote> is in OVERRIDES, then the <quote>arm</quote> specific version of the variable is used rather than the non-conditional version.  Example:</para>
@@ -536,7 +554,7 @@ options:
                 <example>
                     <title>Generating dependency graphs</title>
                     <para>BitBake is able to generate dependency graphs using the dot syntax. These graphs can be converted
-to images using the <application>dot</application> application from <ulink url="http://www.graphviz.org">Graphviz</ulink>. 
+to images using the <application>dot</application> application from <ulink url="http://www.graphviz.org">Graphviz</ulink>.
 Two files will be written into the current working directory, <emphasis>depends.dot</emphasis> containing dependency information at the package level and <emphasis>task-depends.dot</emphasis> containing a breakdown of the dependencies at the task level. To stop depending on common depends, one can use the <prompt>-I depend</prompt> to omit these from the graph. This can lead to more readable graphs.  This way, <varname>DEPENDS</varname> from inherited classes such as base.bbclass can be removed from the graph.</para>
                     <screen><prompt>$ </prompt>bitbake -g blah</screen>
                     <screen><prompt>$ </prompt>bitbake -g -I virtual/whatever -I bloom blah</screen>

bitbake/lib/bb/__init__.py

@@ -21,7 +21,7 @@
 # with this program; if not, write to the Free Software Foundation, Inc.,
 # 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
 
-__version__ = "1.19.1"
+__version__ = "1.20.0"
 
 import sys
 if sys.version_info < (2, 7, 3):

bitbake/lib/bb/build.py

@@ -69,9 +69,12 @@ class FuncFailed(Exception):
 class TaskBase(event.Event):
     """Base class for task events"""
 
-    def __init__(self, t, d ):
+    def __init__(self, t, logfile, d):
         self._task = t
         self._package = d.getVar("PF", True)
+        self.taskfile = d.getVar("FILE", True)
+        self.taskname = self._task
+        self.logfile = logfile
         event.Event.__init__(self)
         self._message = "recipe %s: task %s: %s" % (d.getVar("PF", True), t, self.getDisplayName())
 
@@ -96,16 +99,11 @@ class TaskFailed(TaskBase):
     """Task execution failed"""
 
     def __init__(self, task, logfile, metadata, errprinted = False):
-        self.logfile = logfile
         self.errprinted = errprinted
-        super(TaskFailed, self).__init__(task, metadata)
+        super(TaskFailed, self).__init__(task, logfile, metadata)
 
 class TaskFailedSilent(TaskBase):
     """Task execution failed (silently)"""
-    def __init__(self, task, logfile, metadata):
-        self.logfile = logfile
-        super(TaskFailedSilent, self).__init__(task, metadata)
-
     def getDisplayName(self):
         # Don't need to tell the user it was silent
         return "Failed"
@@ -113,7 +111,7 @@ class TaskFailedSilent(TaskBase):
 class TaskInvalid(TaskBase):
 
     def __init__(self, task, metadata):
-        super(TaskInvalid, self).__init__(task, metadata)
+        super(TaskInvalid, self).__init__(task, None, metadata)
         self._message = "No such task '%s'" % task
 
 
@@ -348,6 +346,14 @@ def _exec_task(fn, task, d, quieterr):
     if not tempdir:
         bb.fatal("T variable not set, unable to build")
 
+    # Change nice level if we're asked to
+    nice = localdata.getVar("BB_TASK_NICE_LEVEL", True)
+    if nice:
+        curnice = os.nice(0)
+        nice = int(nice) - curnice
+        newnice = os.nice(nice)
+        logger.debug(1, "Renice to %s " % newnice)
+
     bb.utils.mkdirhier(tempdir)
 
     # Determine the logfile to generate
@@ -416,7 +422,7 @@ def _exec_task(fn, task, d, quieterr):
     localdata.setVar('BB_LOGFILE', logfn)
     localdata.setVar('BB_RUNTASK', task)
 
-    event.fire(TaskStarted(task, localdata), localdata)
+    event.fire(TaskStarted(task, logfn, localdata), localdata)
     try:
         for func in (prefuncs or '').split():
             exec_func(func, localdata)
@@ -453,7 +459,7 @@ def _exec_task(fn, task, d, quieterr):
             logger.debug(2, "Zero size logfn %s, removing", logfn)
             bb.utils.remove(logfn)
             bb.utils.remove(loglink)
-    event.fire(TaskSucceeded(task, localdata), localdata)
+    event.fire(TaskSucceeded(task, logfn, localdata), localdata)
 
     if not localdata.getVarFlag(task, 'nostamp') and not localdata.getVarFlag(task, 'selfstamp'):
         make_stamp(task, localdata)

bitbake/lib/bb/cache.py

@@ -43,7 +43,7 @@ except ImportError:
     logger.info("Importing cPickle failed. "
                 "Falling back to a very slow implementation.")
 
-__cache_version__ = "146"
+__cache_version__ = "147"
 
 def getCacheFile(path, filename, data_hash):
     return os.path.join(path, filename + "." + data_hash)

bitbake/lib/bb/cache_extra.py

@@ -35,6 +35,12 @@ class HobRecipeInfo(RecipeInfoCommon):
     # such as (bb_cache.dat, bb_extracache_hob.dat) 
     cachefile = "bb_extracache_" + classname +".dat"        
 
+    # override this member with the list of extra cache fields
+    # that this class will provide
+    cachefields = ['summary', 'license', 'section',
+            'description', 'homepage', 'bugtracker',
+            'prevision', 'files_info']
+
     def __init__(self, filename, metadata):
 
         self.summary = self.getvar('SUMMARY', metadata)

bitbake/lib/bb/command.py

@@ -117,14 +117,14 @@ class Command:
             return False
 
     def finishAsyncCommand(self, msg=None, code=None):
-        if msg:
+        if msg or msg == "":
             bb.event.fire(CommandFailed(msg), self.cooker.event_data)
         elif code:
             bb.event.fire(CommandExit(code), self.cooker.event_data)
         else:
             bb.event.fire(CommandCompleted(), self.cooker.event_data)
         self.currentAsyncCommand = None
-
+        self.cooker.finishcommand()
 
 class CommandsSync:
     """
@@ -137,13 +137,22 @@ class CommandsSync:
         """
         Trigger cooker 'shutdown' mode
         """
-        command.cooker.shutdown()
+        command.cooker.shutdown(False)
 
-    def stateStop(self, command, params):
+    def stateForceShutdown(self, command, params):
         """
         Stop the cooker
         """
-        command.cooker.stop()
+        command.cooker.shutdown(True)
+
+    def getAllKeysWithFlags(self, command, params):
+        """
+        Returns a dump of the global state. Call with
+        variable flags to be retrieved as params.
+        """
+        flaglist = params[0]
+        return command.cooker.getAllKeysWithFlags(flaglist)
+    getAllKeysWithFlags.readonly = True
 
     def getVariable(self, command, params):
         """
@@ -232,6 +241,13 @@ class CommandsSync:
         op = params[3]
         command.cooker.modifyConfigurationVar(var, val, default_file, op)
 
+    def removeVarFile(self, command, params):
+        """
+        Remove a variable declaration from a file
+        """
+        var = params[0]
+        command.cooker.removeConfigurationVar(var)
+
     def createConfigFile(self, command, params):
         """
         Create an extra configuration file

bitbake/lib/bb/cooker.py

@@ -61,7 +61,7 @@ class CollectionError(bb.BBHandledException):
     """
 
 class state:
-    initial, parsing, running, shutdown, stop = range(5)
+    initial, parsing, running, shutdown, forceshutdown, stopped = range(6)
 
 
 class SkippedPackage:
@@ -79,6 +79,29 @@ class SkippedPackage:
         elif reason:
             self.skipreason = reason
 
+
+class CookerFeatures(object):
+    _feature_list = [HOB_EXTRA_CACHES, SEND_DEPENDS_TREE] = range(2)
+
+    def __init__(self):
+        self._features=set()
+
+    def setFeature(self, f):
+        # validate we got a request for a feature we support
+        if f not in CookerFeatures._feature_list:
+            return
+        self._features.add(f)
+
+    def __contains__(self, f):
+        return f in self._features
+
+    def __iter__(self):
+        return self._features.__iter__()
+
+    def next(self):
+        return self._features.next()
+
+
 #============================================================================#
 # BBCooker
 #============================================================================#
@@ -90,25 +113,10 @@ class BBCooker:
     def __init__(self, configuration):
         self.recipecache = None
         self.skiplist = {}
+        self.featureset = CookerFeatures()
 
         self.configuration = configuration
 
-        self.caches_array = []
-
-        caches_name_array = ['bb.cache:CoreRecipeInfo'] + configuration.extra_caches
-
-        # At least CoreRecipeInfo will be loaded, so caches_array will never be empty!
-        # This is the entry point, no further check needed!
-        for var in caches_name_array:
-            try:
-                module_name, cache_name = var.split(':')
-                module = __import__(module_name, fromlist=(cache_name,))
-                self.caches_array.append(getattr(module, cache_name)) 
-            except ImportError as exc:
-                logger.critical("Unable to import extra RecipeInfo '%s' from '%s': %s" % (cache_name, module_name, exc))
-                sys.exit("FATAL: Failed to import extra cache class '%s'." % cache_name)
-
-        self.data = None
         self.loadConfigurationData()
 
         # Take a lock so only one copy of bitbake can run against a given build
@@ -118,13 +126,6 @@ class BBCooker:
         if not self.lock:
             bb.fatal("Only one copy of bitbake should be run against a build directory")
 
-        #
-        # Special updated configuration we use for firing events
-        #
-        self.event_data = bb.data.createCopy(self.data)
-        bb.data.update_data(self.event_data)
-        bb.parse.init_parser(self.event_data)
-
         # TOSTOP must not be set or our children will hang when they output
         fd = sys.stdout.fileno()
         if os.isatty(fd):
@@ -141,11 +142,30 @@ class BBCooker:
         self.parser = None
 
     def initConfigurationData(self):
-        worker = False
-        if not self.configuration.server_register_idlecallback:
-            worker = True
 
-        self.databuilder = bb.cookerdata.CookerDataBuilder(self.configuration, worker)
+        self.state = state.initial
+
+        self.caches_array = []
+
+        all_extra_cache_names = []
+        # We hardcode all known cache types in a single place, here.
+        if CookerFeatures.HOB_EXTRA_CACHES in self.featureset:
+            all_extra_cache_names.append("bb.cache_extra:HobRecipeInfo")
+
+        caches_name_array = ['bb.cache:CoreRecipeInfo'] + all_extra_cache_names
+
+        # At least CoreRecipeInfo will be loaded, so caches_array will never be empty!
+        # This is the entry point, no further check needed!
+        for var in caches_name_array:
+            try:
+                module_name, cache_name = var.split(':')
+                module = __import__(module_name, fromlist=(cache_name,))
+                self.caches_array.append(getattr(module, cache_name))
+            except ImportError as exc:
+                logger.critical("Unable to import extra RecipeInfo '%s' from '%s': %s" % (cache_name, module_name, exc))
+                sys.exit("FATAL: Failed to import extra cache class '%s'." % cache_name)
+
+        self.databuilder = bb.cookerdata.CookerDataBuilder(self.configuration, False)
         self.data = self.databuilder.data
 
     def enableDataTracking(self):
@@ -162,11 +182,21 @@ class BBCooker:
         self.data = self.databuilder.data
         self.data_hash = self.databuilder.data_hash
 
+        #
+        # Special updated configuration we use for firing events
+        #
+        self.event_data = bb.data.createCopy(self.data)
+        bb.data.update_data(self.event_data)
+        bb.parse.init_parser(self.event_data)
+
     def modifyConfigurationVar(self, var, val, default_file, op):
         if op == "append":
             self.appendConfigurationVar(var, val, default_file)
         elif op == "set":
-            self.saveConfigurationVar(var, val, default_file)
+            self.saveConfigurationVar(var, val, default_file, "=")
+        elif op == "earlyAssign":
+            self.saveConfigurationVar(var, val, default_file, "?=")
+
 
     def appendConfigurationVar(self, var, val, default_file):
         #add append var operation to the end of default_file
@@ -180,7 +210,7 @@ class BBCooker:
         for c in contents:
             total += c
 
-        total += "#added by bitbake"
+        total += "#added by hob"
         total += "\n%s += \"%s\"\n" % (var, val)
 
         with open(default_file, 'w') as f:
@@ -191,7 +221,7 @@ class BBCooker:
         loginfo = {"op":append, "file":default_file, "line":total.count("\n")}
         self.data.appendVar(var, val, **loginfo)
 
-    def saveConfigurationVar(self, var, val, default_file):
+    def saveConfigurationVar(self, var, val, default_file, op):
 
         replaced = False
         #do not save if nothing changed
@@ -233,8 +263,8 @@ class BBCooker:
                     #check if the variable was saved before in the same way
                     #if true it replace the place where the variable was declared
                     #else it comments it
-                    if contents[begin_line-1]== "#added by bitbake\n":
-                        contents[begin_line] = "%s = \"%s\"\n" % (var, val)
+                    if contents[begin_line-1]== "#added by hob\n":
+                        contents[begin_line] = "%s %s \"%s\"\n" % (var, op, val)
                         replaced = True
                     else:
                         for ii in range(begin_line, end_line):
@@ -263,8 +293,8 @@ class BBCooker:
                 total += c
 
             #add the variable on a single line, to be easy to replace the second time
-            total += "\n#added by bitbake"
-            total += "\n%s = \"%s\"\n" % (var, val)
+            total += "\n#added by hob"
+            total += "\n%s %s \"%s\"\n" % (var, op, val)
 
             with open(default_file, 'w') as f:
                 f.write(total)
@@ -274,6 +304,44 @@ class BBCooker:
             loginfo = {"op":set, "file":default_file, "line":total.count("\n")}
             self.data.setVar(var, val, **loginfo)
 
+    def removeConfigurationVar(self, var):
+        conf_files = self.data.varhistory.get_variable_files(var)
+        topdir = self.data.getVar("TOPDIR")
+
+        for conf_file in conf_files:
+            if topdir in conf_file:
+                with open(conf_file, 'r') as f:
+                    contents = f.readlines()
+                f.close()
+
+                lines = self.data.varhistory.get_variable_lines(var, conf_file)
+                for line in lines:
+                    total = ""
+                    i = 0
+                    for c in contents:
+                        total += c
+                        i = i + 1
+                        if i==int(line):
+                            end_index = len(total)
+                    index = total.rfind(var, 0, end_index)
+
+                    begin_line = total.count("\n",0,index)
+
+                    #check if the variable was saved before in the same way
+                    if contents[begin_line-1]== "#added by hob\n":
+                        contents[begin_line-1] = contents[begin_line] = "\n"
+                    else:
+                        contents[begin_line] = "\n"
+                    #remove var from history
+                    self.data.varhistory.del_var_history(var, conf_file, line)
+
+                total = ""
+                for c in contents:
+                    total += c
+                with open(conf_file, 'w') as f:
+                    f.write(total)
+                f.close()
+
     def createConfigFile(self, name):
         path = os.getcwd()
         confpath = os.path.join(path, "conf", name)
@@ -348,13 +416,7 @@ class BBCooker:
             if pkgs_to_build[0] in set(ignore.split()):
                 bb.fatal("%s is in ASSUME_PROVIDED" % pkgs_to_build[0])
 
-            localdata = data.createCopy(self.data)
-            bb.data.update_data(localdata)
-            bb.data.expandKeys(localdata)
-
-            taskdata = bb.taskdata.TaskData(self.configuration.abort)
-            taskdata.add_provider(localdata, self.recipecache, pkgs_to_build[0])
-            taskdata.add_unresolved(localdata, self.recipecache)
+            taskdata, runlist, pkgs_to_build = self.buildTaskData(pkgs_to_build, None, self.configuration.abort)
 
             targetid = taskdata.getbuild_id(pkgs_to_build[0])
             fnid = taskdata.build_targets[targetid][0]
@@ -386,34 +448,44 @@ class BBCooker:
             if data.getVarFlag( e, 'python', envdata ):
                 logger.plain("\npython %s () {\n%s}\n", e, data.getVar(e, envdata, 1))
 
-    def prepareTreeData(self, pkgs_to_build, task):
+
+    def buildTaskData(self, pkgs_to_build, task, abort):
         """
         Prepare a runqueue and taskdata object for iteration over pkgs_to_build
         """
         bb.event.fire(bb.event.TreeDataPreparationStarted(), self.data)
 
-        # If we are told to do the None task then query the default task
-        if (task == None):
+        # A task of None means use the default task
+        if task is None:
             task = self.configuration.cmd
 
-        pkgs_to_build = self.checkPackages(pkgs_to_build)
+        fulltargetlist = self.checkPackages(pkgs_to_build)
 
         localdata = data.createCopy(self.data)
         bb.data.update_data(localdata)
         bb.data.expandKeys(localdata)
+        taskdata = bb.taskdata.TaskData(abort, skiplist=self.skiplist)
+
+        current = 0
+        runlist = []
+        for k in fulltargetlist:
+            taskdata.add_provider(localdata, self.recipecache, k)
+            current += 1
+            runlist.append([k, "do_%s" % task])
+            bb.event.fire(bb.event.TreeDataPreparationProgress(current, len(fulltargetlist)), self.data)
+        taskdata.add_unresolved(localdata, self.recipecache)
+        bb.event.fire(bb.event.TreeDataPreparationCompleted(len(fulltargetlist)), self.data)
+        return taskdata, runlist, fulltargetlist
+
+    def prepareTreeData(self, pkgs_to_build, task):
+        """
+        Prepare a runqueue and taskdata object for iteration over pkgs_to_build
+        """
+
         # We set abort to False here to prevent unbuildable targets raising
         # an exception when we're just generating data
-        taskdata = bb.taskdata.TaskData(False, skiplist=self.skiplist)
+        taskdata, runlist, pkgs_to_build = self.buildTaskData(pkgs_to_build, task, False)
 
-        runlist = []
-        current = 0
-        for k in pkgs_to_build:
-            taskdata.add_provider(localdata, self.recipecache, k)
-            runlist.append([k, "do_%s" % task])
-            current += 1
-            bb.event.fire(bb.event.TreeDataPreparationProgress(current, len(pkgs_to_build)), self.data)
-        taskdata.add_unresolved(localdata, self.recipecache)
-        bb.event.fire(bb.event.TreeDataPreparationCompleted(len(pkgs_to_build)), self.data)
         return runlist, taskdata
     
     ######## WARNING : this function requires cache_extra to be enabled ########
@@ -426,7 +498,10 @@ class BBCooker:
         runlist, taskdata = self.prepareTreeData(pkgs_to_build, task)
         rq = bb.runqueue.RunQueue(self, self.data, self.recipecache, taskdata, runlist)
         rq.rqdata.prepare()
+        return self.buildDependTree(rq, taskdata)
 
+
+    def buildDependTree(self, rq, taskdata):
         seen_fnids = []
         depend_tree = {}
         depend_tree["depends"] = {}
@@ -447,6 +522,19 @@ class BBCooker:
                 depend_tree["pn"][pn] = {}
                 depend_tree["pn"][pn]["filename"] = fn
                 depend_tree["pn"][pn]["version"] = version
+
+                # if we have extra caches, list all attributes they bring in
+                extra_info = []
+                for cache_class in self.caches_array:
+                    if type(cache_class) is type and issubclass(cache_class, bb.cache.RecipeInfoCommon) and hasattr(cache_class, 'cachefields'):
+                        cachefields = getattr(cache_class, 'cachefields', [])
+                        extra_info = extra_info + cachefields
+
+                # for all attributes stored, add them to the dependency tree
+                for ei in extra_info:
+                    depend_tree["pn"][pn][ei] = vars(self.recipecache)[ei][fn]
+
+
             for dep in rq.rqdata.runq_depends[task]:
                 depfn = taskdata.fn_index[rq.rqdata.runq_fnid[dep]]
                 deppn = self.recipecache.pkg_fn[depfn]
@@ -509,35 +597,30 @@ class BBCooker:
         depend_tree["rdepends-pkg"] = {}
         depend_tree["rrecs-pkg"] = {}
 
+        # if we have extra caches, list all attributes they bring in
+        extra_info = []
+        for cache_class in self.caches_array:
+            if type(cache_class) is type and issubclass(cache_class, bb.cache.RecipeInfoCommon) and hasattr(cache_class, 'cachefields'):
+                cachefields = getattr(cache_class, 'cachefields', [])
+                extra_info = extra_info + cachefields
+
         for task in xrange(len(tasks_fnid)):
             fnid = tasks_fnid[task]
             fn = taskdata.fn_index[fnid]
             pn = self.recipecache.pkg_fn[fn]
-            version  = "%s:%s-%s" % self.recipecache.pkg_pepvpr[fn]
-            summary = self.recipecache.summary[fn]
-            lic = self.recipecache.license[fn]
-            section = self.recipecache.section[fn]
-            description = self.recipecache.description[fn]
-            homepage = self.recipecache.homepage[fn]
-            bugtracker = self.recipecache.bugtracker[fn]
-            files_info = self.recipecache.files_info[fn]
-            rdepends = self.recipecache.rundeps[fn]
-            rrecs = self.recipecache.runrecs[fn]
-            prevision = self.recipecache.prevision[fn]
-            inherits = self.recipecache.inherits.get(fn, None)
+
             if pn not in depend_tree["pn"]:
                 depend_tree["pn"][pn] = {}
                 depend_tree["pn"][pn]["filename"] = fn
+                version  = "%s:%s-%s" % self.recipecache.pkg_pepvpr[fn]
                 depend_tree["pn"][pn]["version"] = version
-                depend_tree["pn"][pn]["summary"] = summary
-                depend_tree["pn"][pn]["license"] = lic
-                depend_tree["pn"][pn]["section"] = section
-                depend_tree["pn"][pn]["description"] = description
-                depend_tree["pn"][pn]["inherits"] = inherits
-                depend_tree["pn"][pn]["homepage"] = homepage
-                depend_tree["pn"][pn]["bugtracker"] = bugtracker
-                depend_tree["pn"][pn]["files_info"] = files_info
-                depend_tree["pn"][pn]["revision"] = prevision
+                rdepends = self.recipecache.rundeps[fn]
+                rrecs = self.recipecache.runrecs[fn]
+                depend_tree["pn"][pn]["inherits"] = self.recipecache.inherits.get(fn, None)
+
+                # for all extra attributes stored, add them to the dependency tree
+                for ei in extra_info:
+                    depend_tree["pn"][pn][ei] = vars(self.recipecache)[ei][fn]
 
             if fnid not in seen_fnids:
                 seen_fnids.append(fnid)
@@ -1047,7 +1130,7 @@ class BBCooker:
 
         def buildFileIdle(server, rq, abort):
 
-            if abort or self.state == state.stop:
+            if abort or self.state == state.forceshutdown:
                 rq.finish_runqueue(True)
             elif self.state == state.shutdown:
                 rq.finish_runqueue(False)
@@ -1076,15 +1159,8 @@ class BBCooker:
         Attempt to build the targets specified
         """
 
-        # If we are told to do the NULL task then query the default task
-        if (task == None):
-            task = self.configuration.cmd
-
-        universe = ('universe' in targets)
-        targets = self.checkPackages(targets)
-
         def buildTargetsIdle(server, rq, abort):
-            if abort or self.state == state.stop:
+            if abort or self.state == state.forceshutdown:
                 rq.finish_runqueue(True)
             elif self.state == state.shutdown:
                 rq.finish_runqueue(False)
@@ -1108,27 +1184,32 @@ class BBCooker:
 
         self.buildSetVars()
 
+        taskdata, runlist, fulltargetlist = self.buildTaskData(targets, task, self.configuration.abort)
+
         buildname = self.data.getVar("BUILDNAME")
-        bb.event.fire(bb.event.BuildStarted(buildname, targets), self.data)
-
-        localdata = data.createCopy(self.data)
-        bb.data.update_data(localdata)
-        bb.data.expandKeys(localdata)
-
-        taskdata = bb.taskdata.TaskData(self.configuration.abort, skiplist=self.skiplist)
-
-        runlist = []
-        for k in targets:
-            taskdata.add_provider(localdata, self.recipecache, k)
-            runlist.append([k, "do_%s" % task])
-        taskdata.add_unresolved(localdata, self.recipecache)
+        bb.event.fire(bb.event.BuildStarted(buildname, fulltargetlist), self.data)
 
         rq = bb.runqueue.RunQueue(self, self.data, self.recipecache, taskdata, runlist)
-        if universe:
+        if 'universe' in targets:
             rq.rqdata.warn_multi_bb = True
 
         self.configuration.server_register_idlecallback(buildTargetsIdle, rq)
 
+
+    def getAllKeysWithFlags(self, flaglist):
+        dump = {}
+        for k in self.data.keys():
+            try:
+                v = self.data.getVar(k, True)
+                if not k.startswith("__") and not isinstance(v, bb.data_smart.DataSmart):
+                    dump[k] = { 'v' : v }
+                    for d in flaglist:
+                        dump[k][d] = self.data.getVarFlag(k, d)
+            except Exception as e:
+                print(e)
+        return dump
+
+
     def generateNewImage(self, image, base_image, package_queue, timestamp, description):
         '''
         Create a new image with a "require"/"inherit" base_image statement
@@ -1173,9 +1254,9 @@ class BBCooker:
         if self.state == state.running:
             return
 
-        if self.state in (state.shutdown, state.stop):
+        if self.state in (state.shutdown, state.forceshutdown):
             self.parser.shutdown(clean=False, force = True)
-            sys.exit(1)
+            raise bb.BBHandledException()
 
         if self.state != state.parsing:
             self.parseConfiguration ()
@@ -1197,7 +1278,7 @@ class BBCooker:
         if not self.parser.parse_next():
             collectlog.debug(1, "parsing complete")
             if self.parser.error:
-                sys.exit(1)
+                raise bb.BBHandledException()
             self.show_appends_with_no_recipes()
             self.handlePrefProviders()
             self.recipecache.bbfile_priority = self.collection.collection_priorities(self.recipecache.pkg_fn)
@@ -1208,6 +1289,9 @@ class BBCooker:
 
     def checkPackages(self, pkgs_to_build):
 
+        # Return a copy, don't modify the original
+        pkgs_to_build = pkgs_to_build[:]
+
         if len(pkgs_to_build) == 0:
             raise NothingToBuild
 
@@ -1243,18 +1327,19 @@ class BBCooker:
         prserv.serv.auto_shutdown(self.data)
         bb.event.fire(CookerExit(), self.event_data)
 
-    def shutdown(self):
-        self.state = state.shutdown
+    def shutdown(self, force = False):
+        if force:
+            self.state = state.forceshutdown
+        else:
+            self.state = state.shutdown
 
-    def stop(self):
-        self.state = state.stop
+    def finishcommand(self):
+        self.state = state.initial
 
     def initialize(self):
-        self.state = state.initial
         self.initConfigurationData()
 
     def reset(self):
-        self.state = state.initial
         self.loadConfigurationData()
 
 def server_main(cooker, func, *args):
@@ -1482,7 +1567,7 @@ class Feeder(multiprocessing.Process):
                 continue
 
 class Parser(multiprocessing.Process):
-    def __init__(self, jobs, results, quit, init):
+    def __init__(self, jobs, results, quit, init, profile):
         self.jobs = jobs
         self.results = results
         self.quit = quit
@@ -1490,8 +1575,28 @@ class Parser(multiprocessing.Process):
         multiprocessing.Process.__init__(self)
         self.context = bb.utils.get_context().copy()
         self.handlers = bb.event.get_class_handlers().copy()
+        self.profile = profile
 
     def run(self):
+
+        if not self.profile:
+            self.realrun()
+            return
+
+        try:
+            import cProfile as profile
+        except:
+            import profile
+        prof = profile.Profile()
+        try:
+            profile.Profile.runcall(prof, self.realrun)
+        finally:
+            logfile = "profile-parse-%s.log" % multiprocessing.current_process().name
+            prof.dump_stats(logfile)
+            bb.utils.process_profilelog(logfile)
+            print("Raw profiling information saved to %s and processed statistics to %s.processed" % (logfile, logfile))
+
+    def realrun(self):
         if self.init:
             self.init()
 
@@ -1592,7 +1697,7 @@ class CookerParser(object):
             self.feeder = Feeder(self.willparse, self.jobs, self.feeder_quit)
             self.feeder.start()
             for i in range(0, self.num_processes):
-                parser = Parser(self.jobs, self.result_queue, self.parser_quit, init)
+                parser = Parser(self.jobs, self.result_queue, self.parser_quit, init, self.cooker.configuration.profile)
                 parser.start()
                 self.processes.append(parser)
 

bitbake/lib/bb/cookerdata.py

@@ -127,7 +127,6 @@ class CookerConfiguration(object):
         self.dump_signatures = False
         self.dry_run = False
         self.tracking = False
-        self.extra_caches = []
 
         self.env = {}
 

bitbake/lib/bb/data.py

@@ -59,7 +59,7 @@ def init():
 def init_db(parent = None):
     """Return a new object representing the Bitbake data,
     optionally based on an existing object"""
-    if parent:
+    if parent is not None:
         return parent.createCopy()
     else:
         return _dict_type()
@@ -148,7 +148,7 @@ def expandKeys(alterdata, readdata = None):
         readdata = alterdata
 
     todolist = {}
-    for key in keys(alterdata):
+    for key in alterdata:
         if not '${' in key:
             continue
 
@@ -285,20 +285,24 @@ def update_data(d):
     """Performs final steps upon the datastore, including application of overrides"""
     d.finalize(parent = True)
 
-def build_dependencies(key, keys, shelldeps, vardepvals, d):
+def build_dependencies(key, keys, shelldeps, varflagsexcl, d):
     deps = set()
-    vardeps = d.getVarFlag(key, "vardeps", True)
     try:
         if key[-1] == ']':
             vf = key[:-1].split('[')
             value = d.getVarFlag(vf[0], vf[1], False)
-        else:
-            value = d.getVar(key, False)
+            parser = d.expandWithRefs(value, key)
+            deps |= parser.references
+            deps = deps | (keys & parser.execs)
+            return deps, value
+        varflags = d.getVarFlags(key, ["vardeps", "vardepvalue", "vardepsexclude"]) or {}
+        vardeps = varflags.get("vardeps")
+        value = d.getVar(key, False)
 
-        if key in vardepvals:
-           value =  d.getVarFlag(key, "vardepvalue", True)
-        elif d.getVarFlag(key, "func"):
-            if d.getVarFlag(key, "python"):
+        if "vardepvalue" in varflags:
+           value = varflags.get("vardepvalue")
+        elif varflags.get("func"):
+            if varflags.get("python"):
                 parsedvar = d.expandWithRefs(value, key)
                 parser = bb.codeparser.PythonParser(key, logger)
                 if parsedvar.value and "\t" in parsedvar.value:
@@ -320,19 +324,16 @@ def build_dependencies(key, keys, shelldeps, vardepvals, d):
             deps = deps | (keys & parser.execs)
 
         # Add varflags, assuming an exclusion list is set
-        varflagsexcl = d.getVar('BB_SIGNATURE_EXCLUDE_FLAGS', True)
         if varflagsexcl:
             varfdeps = []
-            varflags = d.getVarFlags(key)
-            if varflags:
-                for f in varflags:
-                    if f not in varflagsexcl:
-                        varfdeps.append('%s[%s]' % (key, f))
+            for f in varflags:
+                if f not in varflagsexcl:
+                    varfdeps.append('%s[%s]' % (key, f))
             if varfdeps:
                 deps |= set(varfdeps)
 
         deps |= set((vardeps or "").split())
-        deps -= set((d.getVarFlag(key, "vardepsexclude", True) or "").split())
+        deps -= set(varflags.get("vardepsexclude", "").split())
     except Exception as e:
         raise bb.data_smart.ExpansionError(key, None, e)
     return deps, value
@@ -341,16 +342,16 @@ def build_dependencies(key, keys, shelldeps, vardepvals, d):
 
 def generate_dependencies(d):
 
-    keys = set(key for key in d.keys() if not key.startswith("__"))
-    shelldeps = set(key for key in keys if d.getVarFlag(key, "export") and not d.getVarFlag(key, "unexport"))
-    vardepvals = set(key for key in keys if d.getVarFlag(key, "vardepvalue"))
+    keys = set(key for key in d if not key.startswith("__"))
+    shelldeps = set(key for key in d.getVar("__exportlist", False) if d.getVarFlag(key, "export") and not d.getVarFlag(key, "unexport"))
+    varflagsexcl = d.getVar('BB_SIGNATURE_EXCLUDE_FLAGS', True)
 
     deps = {}
     values = {}
 
     tasklist = d.getVar('__BBTASKS') or []
     for task in tasklist:
-        deps[task], values[task] = build_dependencies(task, keys, shelldeps, vardepvals, d)
+        deps[task], values[task] = build_dependencies(task, keys, shelldeps, varflagsexcl, d)
         newdeps = deps[task]
         seen = set()
         while newdeps:
@@ -359,7 +360,7 @@ def generate_dependencies(d):
             newdeps = set()
             for dep in nextdeps:
                 if dep not in deps:
-                    deps[dep], values[dep] = build_dependencies(dep, keys, shelldeps, vardepvals, d)
+                    deps[dep], values[dep] = build_dependencies(dep, keys, shelldeps, varflagsexcl, d)
                 newdeps |=  deps[dep]
             newdeps -= seen
         #print "For %s: %s" % (task, str(deps[task]))

bitbake/lib/bb/data_smart.py

@@ -40,7 +40,7 @@ logger = logging.getLogger("BitBake.Data")
 
 __setvar_keyword__ = ["_append", "_prepend", "_remove"]
 __setvar_regexp__ = re.compile('(?P<base>.*?)(?P<keyword>_append|_prepend|_remove)(_(?P<add>.*))?$')
-__expand_var_regexp__ = re.compile(r"\${[^{}]+}")
+__expand_var_regexp__ = re.compile(r"\${[^{}@\n\t ]+}")
 __expand_python_regexp__ = re.compile(r"\${@.+?}")
 
 def infer_caller_details(loginfo, parent = False, varval = True):
@@ -94,9 +94,13 @@ class VariableParse:
             if self.varname and key:
                 if self.varname == key:
                     raise Exception("variable %s references itself!" % self.varname)
-            var = self.d.getVar(key, True)
+            if key in self.d.expand_cache:
+                varparse = self.d.expand_cache[key]
+                var = varparse.value
+            else:
+                var = self.d.getVar(key, True)
+            self.references.add(key)
             if var is not None:
-                self.references.add(key)
                 return var
             else:
                 return match.group()
@@ -277,9 +281,13 @@ class VariableHistory(object):
                 lines.append(line)
         return lines
 
-    def del_var_history(self, var):
+    def del_var_history(self, var, f=None, line=None):
+        """If file f and line are not given, the entire history of var is deleted"""
         if var in self.variables:
-            self.variables[var] = []
+            if f and line:
+                self.variables[var] = [ x for x in self.variables[var] if x['file']!=f and x['line']!=line]
+            else:
+                self.variables[var] = []
 
 class DataSmart(MutableMapping):
     def __init__(self, special = COWDictBase.copy(), seen = COWDictBase.copy() ):
@@ -505,12 +513,7 @@ class DataSmart(MutableMapping):
             self._seen_overrides[override].add( var )
 
     def getVar(self, var, expand=False, noweakdefault=False):
-        value = self.getVarFlag(var, "_content", False, noweakdefault)
-
-        # Call expand() separately to make use of the expand cache
-        if expand and value:
-            return self.expand(value, var)
-        return value
+        return self.getVarFlag(var, "_content", expand, noweakdefault)
 
     def renameVar(self, key, newkey, **loginfo):
         """
@@ -578,17 +581,30 @@ class DataSmart(MutableMapping):
         if flag == "defaultval" and '_' in var:
             self._setvar_update_overrides(var)
 
+        if flag == "unexport" or flag == "export":
+            if not "__exportlist" in self.dict:
+                self._makeShadowCopy("__exportlist")
+            if not "_content" in self.dict["__exportlist"]:
+                self.dict["__exportlist"]["_content"] = set()
+            self.dict["__exportlist"]["_content"].add(var)
+
     def getVarFlag(self, var, flag, expand=False, noweakdefault=False):
         local_var = self._findVar(var)
         value = None
-        if local_var:
+        if local_var is not None:
             if flag in local_var:
                 value = copy.copy(local_var[flag])
             elif flag == "_content" and "defaultval" in local_var and not noweakdefault:
                 value = copy.copy(local_var["defaultval"])
         if expand and value:
-            value = self.expand(value, None)
-        if value and flag == "_content" and local_var and "_removeactive" in local_var:
+            # Only getvar (flag == _content) hits the expand cache
+            cachename = None
+            if flag == "_content":
+                cachename = var
+            else:
+                cachename = var + "[" + flag + "]"
+            value = self.expand(value, cachename)
+        if value is not None and flag == "_content" and local_var is not None and "_removeactive" in local_var:
             filtered = filter(lambda v: v not in local_var["_removeactive"],
                               value.split(" "))
             value = " ".join(filtered)
@@ -636,16 +652,17 @@ class DataSmart(MutableMapping):
             self.varhistory.record(**loginfo)
             self.dict[var][i] = flags[i]
 
-    def getVarFlags(self, var):
+    def getVarFlags(self, var, expand = False, internalflags=False):
         local_var = self._findVar(var)
         flags = {}
 
         if local_var:
             for i in local_var:
-                if i.startswith("_"):
+                if i.startswith("_") and not internalflags:
                     continue
                 flags[i] = local_var[i]
-
+                if expand and i in expand:
+                    flags[i] = self.expand(flags[i], var + "[" + i + "]")
         if len(flags) == 0:
             return None
         return flags
@@ -751,13 +768,16 @@ class DataSmart(MutableMapping):
         for key in keys:
             if key in config_whitelist:
                 continue
+
             value = d.getVar(key, False) or ""
             data.update({key:value})
 
-            varflags = d.getVarFlags(key)
+            varflags = d.getVarFlags(key, internalflags = True)
             if not varflags:
                 continue
             for f in varflags:
+                if f == "_content":
+                    continue
                 data.update({'%s[%s]' % (key, f):varflags[f]})
 
         for key in ["__BBTASKS", "__BBANONFUNCS", "__BBHANDLERS"]:

bitbake/lib/bb/event.py

@@ -589,6 +589,16 @@ class PackageInfo(Event):
         Event.__init__(self)
         self._pkginfolist = pkginfolist
 
+class MetadataEvent(Event):
+    """
+    Generic event that target for OE-Core classes
+    to report information during asynchrous execution
+    """
+    def __init__(self, eventtype, eventdata):
+        Event.__init__(self)
+        self.type = eventtype
+        self.data = eventdata
+
 class SanityCheck(Event):
     """
     Event to issue sanity check

bitbake/lib/bb/monitordisk.py

@@ -225,7 +225,7 @@ class diskMonitor:
                         self.preFreeS[k] = freeSpace
 
                     if action == "STOPTASKS" and not self.checked[k]:
-                        logger.error("No new tasks can be excuted since the disk space monitor action is \"STOPTASKS\"!")
+                        logger.error("No new tasks can be executed since the disk space monitor action is \"STOPTASKS\"!")
                         self.checked[k] = True
                         rq.finish_runqueue(False)
                         bb.event.fire(bb.event.DiskFull(dev, 'disk', freeSpace, path), self.configuration)
@@ -253,7 +253,7 @@ class diskMonitor:
                         self.preFreeI[k] = freeInode
 
                     if action  == "STOPTASKS" and not self.checked[k]:
-                        logger.error("No new tasks can be excuted since the disk space monitor action is \"STOPTASKS\"!")
+                        logger.error("No new tasks can be executed since the disk space monitor action is \"STOPTASKS\"!")
                         self.checked[k] = True
                         rq.finish_runqueue(False)
                         bb.event.fire(bb.event.DiskFull(dev, 'inode', freeInode, path), self.configuration)

bitbake/lib/bb/providers.py

@@ -341,7 +341,7 @@ def filterProvidersRunTime(providers, item, cfgData, dataCache):
     if numberPreferred > 1:
         logger.error("Trying to resolve runtime dependency %s resulted in conflicting PREFERRED_PROVIDER entries being found.\nThe providers found were: %s\nThe PREFERRED_PROVIDER entries resulting in this conflict were: %s", item, preferred, preferred_vars)
 
-    logger.debug(1, "sorted providers for %s are: %s", item, eligible)
+    logger.debug(1, "sorted runtime providers for %s are: %s", item, eligible)
 
     return eligible, numberPreferred
 

bitbake/lib/bb/runqueue.py

@@ -217,6 +217,15 @@ class RunQueueData:
             ret.extend([nam])
         return ret
 
+    def get_task_name(self, task):
+        return self.runq_task[task]
+
+    def get_task_file(self, task):
+        return self.taskData.fn_index[self.runq_fnid[task]]
+
+    def get_task_hash(self, task):
+        return self.runq_hash[task]
+
     def get_user_idstring(self, task, task_name_suffix = ""):
         fn = self.taskData.fn_index[self.runq_fnid[task]]
         taskname = self.runq_task[task] + task_name_suffix
@@ -696,6 +705,14 @@ class RunQueueData:
                     prov_list[prov].append(fn)
         for prov in prov_list:
             if len(prov_list[prov]) > 1 and prov not in self.multi_provider_whitelist:
+                seen_pn = []
+                # If two versions of the same PN are being built its fatal, we don't support it.
+                for fn in prov_list[prov]:
+                    pn = self.dataCache.pkg_fn[fn]
+                    if pn not in seen_pn:
+                        seen_pn.append(pn)
+                    else:
+                        bb.fatal("Multiple versions of %s are due to be built (%s). Only one version of a given PN should be built in any given build. You likely need to set PREFERRED_VERSION_%s to select the correct version or don't depend on multiple versions." % (pn, " ".join(prov_list[prov]), pn))
                 msg = "Multiple .bb files are due to be built which each provide %s (%s)." % (prov, " ".join(prov_list[prov]))
                 if self.warn_multi_bb:
                     logger.warn(msg)
@@ -703,7 +720,6 @@ class RunQueueData:
                     msg += "\n This usually means one provides something the other doesn't and should."
                     logger.error(msg)
 
-
         # Create a whitelist usable by the stamp checks
         stampfnwhitelist = []
         for entry in self.stampwhitelist.split():
@@ -853,6 +869,8 @@ class RunQueue:
             "logdefaultdomain" : bb.msg.loggerDefaultDomains,
             "prhost" : self.cooker.prhost,
             "buildname" : self.cfgData.getVar("BUILDNAME", True),
+            "date" : self.cfgData.getVar("DATE", True),
+            "time" : self.cfgData.getVar("TIME", True),
         }
 
         worker.stdin.write("<cookerconfig>" + pickle.dumps(self.cooker.configuration) + "</cookerconfig>")
@@ -990,6 +1008,11 @@ class RunQueue:
             else:
                 self.state = runQueueSceneInit
 
+                # we are ready to run, see if any UI client needs the dependency info
+                if bb.cooker.CookerFeatures.SEND_DEPENDS_TREE in self.cooker.featureset:
+                    depgraph = self.cooker.buildDependTree(self, self.rqdata.taskData)
+                    bb.event.fire(bb.event.DepTreeGenerated(depgraph), self.cooker.data)
+
         if self.state is runQueueSceneInit:
             if self.cooker.configuration.dump_signatures:
                 self.dump_signatures()
@@ -1316,9 +1339,10 @@ class RunQueueExecuteTasks(RunQueueExecute):
         if self.rqdata.taskData.abort:
             self.rq.state = runQueueCleanUp
 
-    def task_skip(self, task):
+    def task_skip(self, task, reason):
         self.runq_running[task] = 1
         self.runq_buildable[task] = 1
+        bb.event.fire(runQueueTaskSkipped(task, self.stats, self.rq, reason), self.cfgData)
         self.task_completeoutright(task)
         self.stats.taskCompleted()
         self.stats.taskSkipped()
@@ -1343,13 +1367,13 @@ class RunQueueExecuteTasks(RunQueueExecute):
             if task in self.rq.scenequeue_covered:
                 logger.debug(2, "Setscene covered task %s (%s)", task,
                                 self.rqdata.get_user_idstring(task))
-                self.task_skip(task)
+                self.task_skip(task, "covered")
                 return True
 
             if self.rq.check_stamp_task(task, taskname, cache=self.stampcache):
                 logger.debug(2, "Stamp current task %s (%s)", task,
                                 self.rqdata.get_user_idstring(task))
-                self.task_skip(task)
+                self.task_skip(task, "existing")
                 return True
 
             taskdep = self.rqdata.dataCache.task_deps[fn]
@@ -1636,6 +1660,7 @@ class RunQueueExecuteScenequeue(RunQueueExecute):
 
     def task_complete(self, task):
         self.stats.taskCompleted()
+        bb.event.fire(sceneQueueTaskCompleted(task, self.stats, self.rq), self.cfgData)
         self.task_completeoutright(task)
 
     def task_fail(self, task, result):
@@ -1770,6 +1795,9 @@ class runQueueEvent(bb.event.Event):
     def __init__(self, task, stats, rq):
         self.taskid = task
         self.taskstring = rq.rqdata.get_user_idstring(task)
+        self.taskname = rq.rqdata.get_task_name(task)
+        self.taskfile = rq.rqdata.get_task_file(task)
+        self.taskhash = rq.rqdata.get_task_hash(task)
         self.stats = stats.copy()
         bb.event.Event.__init__(self)
 
@@ -1781,6 +1809,9 @@ class sceneQueueEvent(runQueueEvent):
         runQueueEvent.__init__(self, task, stats, rq)
         realtask = rq.rqdata.runq_setscene[task]
         self.taskstring = rq.rqdata.get_user_idstring(realtask, "_setscene")
+        self.taskname = rq.rqdata.get_task_name(realtask) + "_setscene"
+        self.taskfile = rq.rqdata.get_task_file(realtask)
+        self.taskhash = rq.rqdata.get_task_hash(task)
 
 class runQueueTaskStarted(runQueueEvent):
     """
@@ -1819,6 +1850,19 @@ class runQueueTaskCompleted(runQueueEvent):
     Event notifing a task completed
     """
 
+class sceneQueueTaskCompleted(sceneQueueEvent):
+    """
+    Event notifing a setscene task completed
+    """
+
+class runQueueTaskSkipped(runQueueEvent):
+    """
+    Event notifing a task was skipped
+    """
+    def __init__(self, task, stats, rq, reason):
+        runQueueEvent.__init__(self, task, stats, rq)
+        self.reason = reason
+
 class runQueuePipe():
     """
     Abstraction for a pipe between a worker thread and the server

bitbake/lib/bb/server/__init__.py

@@ -89,7 +89,7 @@ class BitBakeBaseServer(object):
     def detach(self):
         return
 
-    def establishConnection(self):
+    def establishConnection(self, featureset):
         raise   "Must redefine the %s.establishConnection()" % self.__class__.__name__
 
     def endSession(self):

bitbake/lib/bb/server/process.py

@@ -31,7 +31,7 @@ import sys
 import time
 import select
 from Queue import Empty
-from multiprocessing import Event, Process, util, Queue, Pipe, queues
+from multiprocessing import Event, Process, util, Queue, Pipe, queues, Manager
 
 from . import BitBakeBaseServer, BitBakeBaseServerConnection, BaseImplServer
 
@@ -78,12 +78,13 @@ class ProcessServer(Process, BaseImplServer):
     profile_filename = "profile.log"
     profile_processed_filename = "profile.log.processed"
 
-    def __init__(self, command_channel, event_queue):
+    def __init__(self, command_channel, event_queue, featurelist):
         BaseImplServer.__init__(self)
-        Process.__init__(self)
+        Process.__init__(self, args=(featurelist))
         self.command_channel = command_channel
         self.event_queue = event_queue
         self.event = EventAdapter(event_queue)
+        self.featurelist = featurelist
         self.quit = False
 
         self.keep_running = Event()
@@ -94,6 +95,14 @@ class ProcessServer(Process, BaseImplServer):
         for event in bb.event.ui_queue:
             self.event_queue.put(event)
         self.event_handle.value = bb.event.register_UIHhandler(self)
+
+        # process any feature changes based on what UI requested
+        original_featureset = list(self.cooker.featureset)
+        while len(self.featurelist)> 0:
+            self.cooker.featureset.setFeature(self.featurelist.pop())
+        if (original_featureset != list(self.cooker.featureset)):
+            self.cooker.reset()
+
         bb.cooker.server_main(self.cooker, self.main)
 
     def main(self):
@@ -113,7 +122,7 @@ class ProcessServer(Process, BaseImplServer):
         self.event_queue.close()
         bb.event.unregister_UIHhandler(self.event_handle.value)
         self.command_channel.close()
-        self.cooker.stop()
+        self.cooker.shutdown(True)
         self.idle_commands(.1)
 
     def idle_commands(self, delay, fds = []):
@@ -198,13 +207,17 @@ class BitBakeServer(BitBakeBaseServer):
         #
         self.ui_channel, self.server_channel = Pipe()
         self.event_queue = ProcessEventQueue(0)
-        self.serverImpl = ProcessServer(self.server_channel, self.event_queue)
+        manager = Manager()
+        self.featurelist = manager.list()
+        self.serverImpl = ProcessServer(self.server_channel, self.event_queue, self.featurelist)
 
     def detach(self):
         self.serverImpl.start()
         return
 
-    def establishConnection(self):
+    def establishConnection(self, featureset):
+        for f in featureset:
+            self.featurelist.append(f)
         self.connection = BitBakeProcessServerConnection(self.serverImpl, self.ui_channel, self.event_queue)
         signal.signal(signal.SIGTERM, lambda i, s: self.connection.terminate())
         return self.connection

bitbake/lib/bb/server/xmlrpc.py

@@ -89,12 +89,23 @@ class BitBakeServerCommands():
         self.server = server
         self.has_client = False
 
-    def registerEventHandler(self, host, port):
+    def registerEventHandler(self, host, port, featureset = []):
         """
         Register a remote UI Event Handler
         """
         s, t = _create_server(host, port)
 
+        # we don't allow connections if the cooker is running
+        if (self.cooker.state in [bb.cooker.state.parsing, bb.cooker.state.running]):
+            return None
+
+        original_featureset = list(self.cooker.featureset)
+        for f in featureset:
+            self.cooker.featureset.setFeature(f)
+
+        if (original_featureset != list(self.cooker.featureset)):
+            self.cooker.reset()
+
         self.event_handle = bb.event.register_UIHhandler(s)
         return self.event_handle
 
@@ -169,51 +180,6 @@ class BitBakeXMLRPCRequestHandler(SimpleXMLRPCRequestHandler):
         self.end_headers()
         self.wfile.write(response)
 
-class BitBakeUIEventServer(threading.Thread):
-    class EventAdapter():
-        """
-        Adapter to wrap our event queue since the caller (bb.event) expects to
-        call a send() method, but our actual queue only has put()
-        """
-        def __init__(self, notify):
-            self.queue = []
-            self.notify = notify
-            self.qlock = threading.Lock()
-
-        def send(self, event):
-            self.qlock.acquire()
-            self.queue.append(event)
-            self.qlock.release()
-            self.notify.set()
-
-        def get(self):
-            self.qlock.acquire()
-            if len(self.queue) == 0:
-                self.qlock.release()
-                return None
-            e = self.queue.pop(0)
-            if len(self.queue) == 0:
-                self.notify.clear()
-            self.qlock.release()
-            return e
-
-    def __init__(self, connection):
-        self.connection = connection
-        self.notify = threading.Event()
-        self.event = BitBakeUIEventServer.EventAdapter(self.notify)
-        self.quit = False
-        threading.Thread.__init__(self)
-
-    def terminateServer(self):
-        self.quit = True
-
-    def run(self):
-        while not self.quit:
-            self.notify.wait(0.1)
-            evt = self.event.get()
-            if evt:
-                self.connection.event.sendpickle(pickle.dumps(evt))
-
 
 class XMLRPCProxyServer(BaseImplServer):
     """ not a real working server, but a stub for a proxy server connection
@@ -308,11 +274,12 @@ class XMLRPCServer(SimpleXMLRPCServer, BaseImplServer):
         self.connection_token = token
 
 class BitBakeXMLRPCServerConnection(BitBakeBaseServerConnection):
-    def __init__(self, serverImpl, clientinfo=("localhost", 0), observer_only = False):
+    def __init__(self, serverImpl, clientinfo=("localhost", 0), observer_only = False, featureset = []):
         self.connection, self.transport = _create_server(serverImpl.host, serverImpl.port)
         self.clientinfo = clientinfo
         self.serverImpl = serverImpl
         self.observer_only = observer_only
+        self.featureset = featureset
 
     def connect(self):
         if not self.observer_only:
@@ -322,7 +289,8 @@ class BitBakeXMLRPCServerConnection(BitBakeBaseServerConnection):
         if token is None:
             return None
         self.transport.set_connection_token(token)
-        self.events = uievent.BBUIEventQueue(self.connection, self.clientinfo)
+
+        self.events = uievent.BBUIEventQueue(self.connection, self.clientinfo, self.featureset)
         for event in bb.event.ui_queue:
             self.events.queue_event(event)
         return self
@@ -346,14 +314,15 @@ class BitBakeXMLRPCServerConnection(BitBakeBaseServerConnection):
 
 class BitBakeServer(BitBakeBaseServer):
     def initServer(self, interface = ("localhost", 0)):
+        self.interface = interface
         self.serverImpl = XMLRPCServer(interface)
 
     def detach(self):
         daemonize.createDaemon(self.serverImpl.serve_forever, "bitbake-cookerdaemon.log")
         del self.cooker
 
-    def establishConnection(self):
-        self.connection = BitBakeXMLRPCServerConnection(self.serverImpl)
+    def establishConnection(self, featureset):
+        self.connection = BitBakeXMLRPCServerConnection(self.serverImpl, self.interface, False, featureset)
         return self.connection.connect()
 
     def set_connection_token(self, token):
@@ -363,12 +332,13 @@ class BitBakeXMLRPCClient(BitBakeBaseServer):
 
     def __init__(self, observer_only = False):
         self.observer_only = observer_only
+        # if we need extra caches, just tell the server to load them all
         pass
 
     def saveConnectionDetails(self, remote):
         self.remote = remote
 
-    def establishConnection(self):
+    def establishConnection(self, featureset):
         # The format of "remote" must be "server:port"
         try:
             [host, port] = self.remote.split(":")
@@ -384,9 +354,12 @@ class BitBakeXMLRPCClient(BitBakeBaseServer):
             s.close()
         except:
             return None
-        self.serverImpl = XMLRPCProxyServer(host, port)
-        self.connection = BitBakeXMLRPCServerConnection(self.serverImpl, (ip, 0), self.observer_only)
-        return self.connection.connect()
+        try:
+            self.serverImpl = XMLRPCProxyServer(host, port)
+            self.connection = BitBakeXMLRPCServerConnection(self.serverImpl, (ip, 0), self.observer_only, featureset)
+            return self.connection.connect()
+        except Exception as e:
+            bb.fatal("Could not connect to server at %s:%s (%s)" % (host, port, str(e)))
 
     def endSession(self):
         self.connection.removeClient()

bitbake/lib/bb/siggen.py

@@ -91,8 +91,7 @@ class SignatureGeneratorBasic(SignatureGenerator):
         basehash = {}
 
         for task in tasklist:
-            data = d.getVar(task, False)
-            lookupcache[task] = data
+            data = lookupcache[task]
 
             if data is None:
                 bb.error("Task %s from %s seems to be empty?!" % (task, fn))
@@ -115,16 +114,8 @@ class SignatureGeneratorBasic(SignatureGenerator):
             alldeps = sorted(seen)
             for dep in alldeps:
                 data = data + dep
-                if dep in lookupcache:
-                    var = lookupcache[dep]
-                elif dep[-1] == ']':
-                    vf = dep[:-1].split('[')
-                    var = d.getVarFlag(vf[0], vf[1], False)
-                    lookupcache[dep] = var
-                else:
-                    var = d.getVar(dep, False)
-                    lookupcache[dep] = var
-                if var:
+                var = lookupcache[dep]
+                if var is not None:
                     data = data + str(var)
             self.basehash[fn + "." + task] = hashlib.md5(data).hexdigest()
             taskdeps[task] = alldeps

bitbake/lib/bb/tests/data.py

@@ -27,21 +27,21 @@ import bb.data
 class DataExpansions(unittest.TestCase):
     def setUp(self):
         self.d = bb.data.init()
-        self.d["foo"] = "value of foo"
-        self.d["bar"] = "value of bar"
-        self.d["value of foo"] = "value of 'value of foo'"
+        self.d["foo"] = "value_of_foo"
+        self.d["bar"] = "value_of_bar"
+        self.d["value_of_foo"] = "value_of_'value_of_foo'"
 
     def test_one_var(self):
         val = self.d.expand("${foo}")
-        self.assertEqual(str(val), "value of foo")
+        self.assertEqual(str(val), "value_of_foo")
 
     def test_indirect_one_var(self):
         val = self.d.expand("${${foo}}")
-        self.assertEqual(str(val), "value of 'value of foo'")
+        self.assertEqual(str(val), "value_of_'value_of_foo'")
 
     def test_indirect_and_another(self):
         val = self.d.expand("${${foo}} ${bar}")
-        self.assertEqual(str(val), "value of 'value of foo' value of bar")
+        self.assertEqual(str(val), "value_of_'value_of_foo' value_of_bar")
 
     def test_python_snippet(self):
         val = self.d.expand("${@5*12}")
@@ -49,11 +49,11 @@ class DataExpansions(unittest.TestCase):
 
     def test_expand_in_python_snippet(self):
         val = self.d.expand("${@'boo ' + '${foo}'}")
-        self.assertEqual(str(val), "boo value of foo")
+        self.assertEqual(str(val), "boo value_of_foo")
 
     def test_python_snippet_getvar(self):
         val = self.d.expand("${@d.getVar('foo', True) + ' ${bar}'}")
-        self.assertEqual(str(val), "value of foo value of bar")
+        self.assertEqual(str(val), "value_of_foo value_of_bar")
 
     def test_python_snippet_syntax_error(self):
         self.d.setVar("FOO", "${@foo = 5}")
@@ -70,7 +70,7 @@ class DataExpansions(unittest.TestCase):
 
     def test_value_containing_value(self):
         val = self.d.expand("${@d.getVar('foo', True) + ' ${bar}'}")
-        self.assertEqual(str(val), "value of foo value of bar")
+        self.assertEqual(str(val), "value_of_foo value_of_bar")
 
     def test_reference_undefined_var(self):
         val = self.d.expand("${undefinedvar} meh")
@@ -109,7 +109,7 @@ class DataExpansions(unittest.TestCase):
 
     def test_rename(self):
         self.d.renameVar("foo", "newfoo")
-        self.assertEqual(self.d.getVar("newfoo"), "value of foo")
+        self.assertEqual(self.d.getVar("newfoo"), "value_of_foo")
         self.assertEqual(self.d.getVar("foo"), None)
 
     def test_deletion(self):
@@ -118,17 +118,17 @@ class DataExpansions(unittest.TestCase):
 
     def test_keys(self):
         keys = self.d.keys()
-        self.assertEqual(keys, ['value of foo', 'foo', 'bar'])
+        self.assertEqual(keys, ['value_of_foo', 'foo', 'bar'])
 
 class TestNestedExpansions(unittest.TestCase):
     def setUp(self):
         self.d = bb.data.init()
         self.d["foo"] = "foo"
         self.d["bar"] = "bar"
-        self.d["value of foobar"] = "187"
+        self.d["value_of_foobar"] = "187"
 
     def test_refs(self):
-        val = self.d.expand("${value of ${foo}${bar}}")
+        val = self.d.expand("${value_of_${foo}${bar}}")
         self.assertEqual(str(val), "187")
 
     #def test_python_refs(self):
@@ -154,11 +154,11 @@ class TestNestedExpansions(unittest.TestCase):
     #    self.assertEqual(str(val), str(depth + 1))
 
     def test_mixed(self):
-        val = self.d.expand("${value of ${@('${foo}'+'bar')[0:3]}${${@'BAR'.lower()}}}")
+        val = self.d.expand("${value_of_${@('${foo}'+'bar')[0:3]}${${@'BAR'.lower()}}}")
         self.assertEqual(str(val), "187")
 
     def test_runtime(self):
-        val = self.d.expand("${${@'value of' + ' f'+'o'+'o'+'b'+'a'+'r'}}")
+        val = self.d.expand("${${@'value_of' + '_f'+'o'+'o'+'b'+'a'+'r'}}")
         self.assertEqual(str(val), "187")
 
 class TestMemoize(unittest.TestCase):

bitbake/lib/bb/ui/crumbs/builder.py

@@ -31,6 +31,7 @@ import re
 import logging
 import sys
 import signal
+import time
 from bb.ui.crumbs.imageconfigurationpage import ImageConfigurationPage
 from bb.ui.crumbs.recipeselectionpage import RecipeSelectionPage
 from bb.ui.crumbs.packageselectionpage import PackageSelectionPage
@@ -197,7 +198,7 @@ class Configuration:
         handler.set_var_in_file("BBLAYERS", self.layers, "bblayers.conf")
         # local.conf
         if not defaults:
-            handler.set_var_in_file("MACHINE", self.curr_mach, "local.conf")
+            handler.early_assign_var_in_file("MACHINE", self.curr_mach, "local.conf")
         handler.set_var_in_file("DISTRO", self.curr_distro, "local.conf")
         handler.set_var_in_file("DL_DIR", self.dldir, "local.conf")
         handler.set_var_in_file("SSTATE_DIR", self.sstatedir, "local.conf")
@@ -217,7 +218,7 @@ class Configuration:
         handler.set_var_in_file("SDKMACHINE", self.curr_sdk_machine, "local.conf")
         handler.set_var_in_file("CONF_VERSION", self.conf_version, "local.conf")
         handler.set_var_in_file("LCONF_VERSION", self.lconf_version, "bblayers.conf")
-        handler.set_var_in_file("EXTRA_SETTING", self.extra_setting, "local.conf")
+        handler.set_extra_config(self.extra_setting)
         handler.set_var_in_file("TOOLCHAIN_BUILD", self.toolchain_build, "local.conf")
         handler.set_var_in_file("IMAGE_FSTYPES", self.image_fstypes, "local.conf")
         if not defaults:
@@ -1466,3 +1467,10 @@ class Builder(gtk.Window):
 
     def get_topdir(self):
         return self.handler.get_topdir()
+
+    def wait(self, delay):
+        time_start = time.time()
+        time_end = time_start + delay
+        while time_end > time.time():
+            while gtk.events_pending():
+                gtk.main_iteration()

bitbake/lib/bb/ui/crumbs/hig/advancedsettingsdialog.py

@@ -234,7 +234,10 @@ class AdvancedSettingsDialog (CrumbsDialog, SettingsUIHelper):
             article = ""
             if image_type.startswith(("a", "e", "i", "o", "u")):
                 article = "n"
-            self.image_types_checkbuttons[image_type].set_tooltip_text("Build a%s %s image" % (article, image_type))
+            if image_type == "live":
+                self.image_types_checkbuttons[image_type].set_tooltip_text("Build iso and hddimg images")
+            else:
+                self.image_types_checkbuttons[image_type].set_tooltip_text("Build a%s %s image" % (article, image_type))
             table.attach(self.image_types_checkbuttons[image_type], j - 1, j + 3, i, i + 1)
             if image_type in self.configuration.image_fstypes.split():
                 self.image_types_checkbuttons[image_type].set_active(True)

bitbake/lib/bb/ui/crumbs/hig/saveimagedialog.py

@@ -70,9 +70,11 @@ class SaveImageDialog (CrumbsDialog):
         sub_label.set_alignment(0, 0)
         sub_label.set_markup("The description should be less than 150 characters long.")
         self.description_entry = gtk.TextView()
-        self.description_entry.get_buffer().set_text(self.description_field)
+        description_buffer = self.description_entry.get_buffer()
+        description_buffer.set_text(self.description_field)
+        description_buffer.connect("insert-text", self.limit_description_length)
         self.description_entry.set_wrap_mode(gtk.WRAP_WORD)
-        self.description_entry.set_size_request(350,150)
+        self.description_entry.set_size_request(350,50)
         sub_vbox.pack_start(label, expand=False, fill=False)
         sub_vbox.pack_start(sub_label, expand=False, fill=False)
         sub_vbox.pack_start(self.description_entry, expand=False, fill=False)
@@ -108,6 +110,13 @@ class SaveImageDialog (CrumbsDialog):
 
         self.show_all()
 
+    def limit_description_length(self, textbuffer, iter, text, length):
+        buffer_bounds = textbuffer.get_bounds()
+        entire_text = textbuffer.get_text(*buffer_bounds)
+        entire_text += text
+        if len(entire_text)>150 or text=="\n":
+            textbuffer.emit_stop_by_name("insert-text")
+
     def name_entry_changed(self, entry):
         text = entry.get_text()
         if text == '':

bitbake/lib/bb/ui/crumbs/hobeventhandler.py

@@ -21,6 +21,7 @@
 
 import gobject
 import logging
+import ast
 from bb.ui.crumbs.runningbuild import RunningBuild
 
 class HobHandler(gobject.GObject):
@@ -315,7 +316,7 @@ class HobHandler(gobject.GObject):
 
     def set_machine(self, machine):
         if machine:
-            self.set_var_in_file("MACHINE", machine, "local.conf")
+            self.early_assign_var_in_file("MACHINE", machine, "local.conf")
 
     def set_sdk_machine(self, sdk_machine):
         self.set_var_in_file("SDKMACHINE", sdk_machine, "local.conf")
@@ -357,7 +358,20 @@ class HobHandler(gobject.GObject):
     def set_incompatible_license(self, incompat_license):
         self.set_var_in_file("INCOMPATIBLE_LICENSE", incompat_license, "local.conf")
 
+    def set_extra_setting(self, extra_setting):
+        self.set_var_in_file("EXTRA_SETTING", extra_setting, "local.conf")
+
     def set_extra_config(self, extra_setting):
+        old_extra_setting = ast.literal_eval(self.runCommand(["getVariable", "EXTRA_SETTING"]) or "{}")
+        if extra_setting:
+            self.set_var_in_file("EXTRA_SETTING", extra_setting, "local.conf")
+        else:
+            self.remove_var_from_file("EXTRA_SETTING")
+
+        #remove not needed settings from conf
+        for key in old_extra_setting:
+            if key not in extra_setting:
+                self.remove_var_from_file(key)
         for key in extra_setting.keys():
             value = extra_setting[key]
             self.set_var_in_file(key, value, "local.conf")
@@ -440,12 +454,12 @@ class HobHandler(gobject.GObject):
         self.building = False
 
     def cancel_parse(self):
-        self.runCommand(["stateStop"])
+        self.runCommand(["stateForceShutdown"])
 
     def cancel_build(self, force=False):
         if force:
             # Force the cooker to stop as quickly as possible
-            self.runCommand(["stateStop"])
+            self.runCommand(["stateForceShutdown"])
         else:
             # Wait for tasks to complete before shutting down, this helps
             # leave the workdir in a usable state
@@ -472,6 +486,14 @@ class HobHandler(gobject.GObject):
         self.server.runCommand(["setVarFile", var, val, default_file, "set"])
         self.runCommand(["disableDataTracking"])
 
+    def early_assign_var_in_file(self, var, val, default_file=None):
+        self.runCommand(["enableDataTracking"])
+        self.server.runCommand(["setVarFile", var, val, default_file, "earlyAssign"])
+        self.runCommand(["disableDataTracking"])
+
+    def remove_var_from_file(self, var):
+        self.server.runCommand(["removeVarFile", var])
+
     def append_var_in_file(self, var, val, default_file=None):
         self.server.runCommand(["setVarFile", var, val, default_file, "append"])
 

bitbake/lib/bb/ui/crumbs/hoblistmodel.py

@@ -690,7 +690,7 @@ class RecipeListModel(gtk.ListStore):
             inherits = event_model["pn"][item]["inherits"]
             summary = event_model["pn"][item]["summary"]
             version = event_model["pn"][item]["version"]
-            revision = event_model["pn"][item]["revision"]
+            revision = event_model["pn"][item]["prevision"]
             homepage = event_model["pn"][item]["homepage"]
             bugtracker = event_model["pn"][item]["bugtracker"]
             filename = event_model["pn"][item]["filename"]

bitbake/lib/bb/ui/crumbs/imageconfigurationpage.py

@@ -300,7 +300,12 @@ class ImageConfigurationPage (HobPage):
     def view_warnings_button_clicked_cb(self, button):
         self.builder.show_warning_dialog()
 
+    def machine_combo_changed_idle_cb(self):
+        self.builder.window.set_cursor(None)
+
     def machine_combo_changed_cb(self, machine_combo):
+        self.builder.window.set_cursor(gtk.gdk.Cursor(gtk.gdk.WATCH))
+        self.builder.wait(0.1) #wait for combo and cursor to update
         self.stopping = False
         self.builder.parsing_warnings = []
         combo_item = machine_combo.get_active_text()
@@ -324,6 +329,8 @@ class ImageConfigurationPage (HobPage):
         # Do reparse recipes
         self.builder.populate_recipe_package_info_async()
 
+        glib.idle_add(self.machine_combo_changed_idle_cb)
+
     def update_machine_combo(self):
         self.disable_warnings_bar()
         all_machines = [self.__dummy_machine__] + self.builder.parameters.all_machines
@@ -387,6 +394,10 @@ class ImageConfigurationPage (HobPage):
                 selected_image = self.__dummy_image__
                 self.update_image_combo(self.builder.recipe_model, None)
             dialog.destroy()
+        else:
+            if self.custom_image_selected:
+                self.custom_image_selected = None
+                self.update_image_combo(self.builder.recipe_model, selected_image)
 
         if not selected_image or (selected_image == self.__dummy_image__):
             self.builder.window_sensitive(True)
@@ -523,7 +534,10 @@ class ImageConfigurationPage (HobPage):
         if not response:
             return
         if settings_changed:
+            self.builder.window.set_cursor(gtk.gdk.Cursor(gtk.gdk.WATCH))
+            self.builder.wait(0.1) #wait for adv_settings_dialog to terminate
             self.builder.reparse_post_adv_settings()
+            self.builder.window.set_cursor(None)
 
     def just_bake_button_clicked_cb(self, button):
         self.builder.parsing_warnings = []

bitbake/lib/bb/ui/crumbs/imagedetailspage.py

@@ -635,6 +635,14 @@ class ImageDetailsPage (HobPage):
         images_dir = topdir + "/recipes/images/"
         self.builder.ensure_dir(images_dir)
 
+        self.name_field_template = self.builder.image_configuration_page.custom_image_selected
+        if self.name_field_template:
+            image_path = self.builder.recipe_model.pn_path[self.name_field_template]
+            image_iter = self.builder.recipe_model.get_iter(image_path)
+            self.description_field_template = self.builder.recipe_model.get_value(image_iter, self.builder.recipe_model.COL_DESC)
+        else:
+            self.name_field_template = ""
+
         dialog = SaveImageDialog(images_dir, self.name_field_template, self.description_field_template,
                  "Save image recipe", self.builder, gtk.DIALOG_MODAL | gtk.DIALOG_DESTROY_WITH_PARENT)
         response = dialog.run()

bitbake/lib/bb/ui/depexp.py

@@ -314,7 +314,7 @@ def main(server, eventHandler, params):
                 break
             if shutdown == 1:
                 print("\nSecond Keyboard Interrupt, stopping...\n")
-                _, error = server.runCommand(["stateStop"])
+                _, error = server.runCommand(["stateForceShutdown"])
                 if error:
                     print('Unable to cleanly stop: %s' % error)
             if shutdown == 0:

bitbake/lib/bb/ui/goggle.py

@@ -117,5 +117,5 @@ def main (server, eventHandler, params):
     except KeyboardInterrupt:
         pass
     finally:
-        server.runCommand(["stateStop"])
+        server.runCommand(["stateForceShutdown"])
 

bitbake/lib/bb/ui/hob.py

@@ -46,7 +46,7 @@ from bb.ui.crumbs.hoblistmodel import RecipeListModel, PackageListModel
 from bb.ui.crumbs.hobeventhandler import HobHandler
 from bb.ui.crumbs.builder import Builder
 
-extraCaches = ['bb.cache_extra:HobRecipeInfo']
+featureSet = [bb.cooker.CookerFeatures.HOB_EXTRA_CACHES]
 
 def event_handle_idle_func(eventHandler, hobHandler):
     # Consume as many messages as we can in the time available to us

bitbake/lib/bb/ui/knotty.py

@@ -306,8 +306,10 @@ def main(server, eventHandler, params, tf = TerminalFilter):
 
     while True:
         try:
-            termfilter.updateFooter()
-            event = eventHandler.waitEvent(0.25)
+            event = eventHandler.waitEvent(0)
+            if event is None:
+                termfilter.updateFooter()
+                event = eventHandler.waitEvent(0.25)
             if event is None:
                 if main.shutdown > 1:
                     break
@@ -403,8 +405,9 @@ def main(server, eventHandler, params, tf = TerminalFilter):
 
             if isinstance(event, bb.command.CommandFailed):
                 return_value = event.exitcode
-                errors = errors + 1
-                logger.error("Command execution failed: %s", event.error)
+                if event.error:
+                    errors = errors + 1
+                    logger.error("Command execution failed: %s", event.error)
                 main.shutdown = 2
                 continue
             if isinstance(event, bb.command.CommandExit):
@@ -469,8 +472,12 @@ def main(server, eventHandler, params, tf = TerminalFilter):
                              event.taskid, event.taskstring, event.exitcode)
                 continue
 
+            if isinstance(event, bb.event.DepTreeGenerated):
+                continue
+
             # ignore
             if isinstance(event, (bb.event.BuildBase,
+                                  bb.event.MetadataEvent,
                                   bb.event.StampUpdate,
                                   bb.event.ConfigParsed,
                                   bb.event.RecipeParsed,
@@ -497,7 +504,7 @@ def main(server, eventHandler, params, tf = TerminalFilter):
                 main.shutdown = 2
             if not params.observe_only and main.shutdown == 1:
                 print("\nSecond Keyboard Interrupt, stopping...\n")
-                _, error = server.runCommand(["stateStop"])
+                _, error = server.runCommand(["stateForceShutdown"])
                 if error:
                     logger.error("Unable to cleanly stop: %s" % error)
             if not params.observe_only and main.shutdown == 0:
@@ -518,7 +525,7 @@ def main(server, eventHandler, params, tf = TerminalFilter):
     if warnings:
         summary += pluralise("\nSummary: There was %s WARNING message shown.",
                              "\nSummary: There were %s WARNING messages shown.", warnings)
-    if return_value:
+    if return_value and errors:
         summary += pluralise("\nSummary: There was %s ERROR message shown, returning a non-zero exit code.",
                              "\nSummary: There were %s ERROR messages shown, returning a non-zero exit code.", errors)
     if summary:

bitbake/lib/bb/ui/ncurses.py

@@ -350,7 +350,7 @@ class NCursesUI:
                     exitflag = True
                 if shutdown == 1:
                     mw.appendText("Second Keyboard Interrupt, stopping...\n")
-                    _, error = server.runCommand(["stateStop"])
+                    _, error = server.runCommand(["stateForceShutdown"])
                     if error:
                         print("Unable to cleanly stop: %s" % error)
                 if shutdown == 0:

bitbake/lib/bb/ui/uievent.py

@@ -28,7 +28,7 @@ import socket, threading, pickle
 from SimpleXMLRPCServer import SimpleXMLRPCServer, SimpleXMLRPCRequestHandler
 
 class BBUIEventQueue:
-    def __init__(self, BBServer, clientinfo=("localhost, 0")):
+    def __init__(self, BBServer, clientinfo=("localhost, 0"), featureset=[]):
 
         self.eventQueue = []
         self.eventQueueLock = threading.Lock()
@@ -44,7 +44,10 @@ class BBUIEventQueue:
         server.register_function( self.send_event, "event.sendpickle" )
         server.socket.settimeout(1)
 
-        self.EventHandle = self.BBServer.registerEventHandler(self.host, self.port)
+        self.EventHandle = self.BBServer.registerEventHandler(self.host, self.port, featureset)
+
+        if (self.EventHandle == None):
+            bb.fatal("Could not register UI event handler")
 
         self.server = server
 

bitbake/lib/bb/utils.py

@@ -532,6 +532,8 @@ def approved_variables():
     Determine and return the list of whitelisted variables which are approved
     to remain in the envrionment.
     """
+    if 'BB_PRESERVE_ENV' in os.environ:
+        return os.environ.keys()
     approved = []
     if 'BB_ENV_WHITELIST' in os.environ:
         approved = os.environ['BB_ENV_WHITELIST'].split()
@@ -722,7 +724,7 @@ def copyfile(src, dest, newmtime = None, sstat = None):
         if not sstat:
             sstat = os.lstat(src)
     except Exception as e:
-        print("copyfile: Stating source file failed...", e)
+        logger.warn("copyfile: stat of %s failed (%s)" % (src, e))
         return False
 
     destexists = 1
@@ -749,7 +751,7 @@ def copyfile(src, dest, newmtime = None, sstat = None):
             #os.lchown(dest,sstat[stat.ST_UID],sstat[stat.ST_GID])
             return os.lstat(dest)
         except Exception as e:
-            print("copyfile: failed to properly create symlink:", dest, "->", target, e)
+            logger.warn("copyfile: failed to create symlink %s to %s (%s)" % (dest, target, e))
             return False
 
     if stat.S_ISREG(sstat[stat.ST_MODE]):
@@ -764,7 +766,7 @@ def copyfile(src, dest, newmtime = None, sstat = None):
             shutil.copyfile(src, dest + "#new")
             os.rename(dest + "#new", dest)
         except Exception as e:
-            print('copyfile: copy', src, '->', dest, 'failed.', e)
+            logger.warn("copyfile: copy %s to %s failed (%s)" % (src, dest, e))
             return False
         finally:
             if srcchown:
@@ -775,13 +777,13 @@ def copyfile(src, dest, newmtime = None, sstat = None):
         #we don't yet handle special, so we need to fall back to /bin/mv
         a = getstatusoutput("/bin/cp -f " + "'" + src + "' '" + dest + "'")
         if a[0] != 0:
-            print("copyfile: Failed to copy special file:" + src + "' to '" + dest + "'", a)
+            logger.warn("copyfile: failed to copy special file %s to %s (%s)" % (src, dest, a))
             return False # failure
     try:
         os.lchown(dest, sstat[stat.ST_UID], sstat[stat.ST_GID])
         os.chmod(dest, stat.S_IMODE(sstat[stat.ST_MODE])) # Sticky is reset on chown
     except Exception as e:
-        print("copyfile: Failed to chown/chmod/unlink", dest, e)
+        logger.warn("copyfile: failed to chown/chmod %s (%s)" % (dest, e))
         return False
 
     if newmtime:

bitbake/lib/prserv/db.py

@@ -43,6 +43,10 @@ class PRTable(object):
                     continue
                 raise exc
 
+    def sync(self):
+        self.conn.commit()
+        self._execute("BEGIN EXCLUSIVE TRANSACTION")
+
     def _getValueHist(self, version, pkgarch, checksum):
         data=self._execute("SELECT value FROM %s WHERE version=? AND pkgarch=? AND checksum=?;" % self.table,
                            (version, pkgarch, checksum))

bitbake/lib/prserv/serv.py

@@ -74,7 +74,7 @@ class PRServer(SimpleXMLRPCServer):
             except:
                 self.handle_error(request, client_address)
                 self.shutdown_request(request)
-
+                self.table.sync()
 
     def process_request(self, request, client_address):
         self.requestqueue.put((request, client_address))
@@ -120,6 +120,7 @@ class PRServer(SimpleXMLRPCServer):
         while not self.quit:
             self.handle_request()
 
+        self.table.sync()
         logger.info("PRServer: stopping...")
         self.server_close()
         return

documentation/Makefile

@@ -200,7 +200,8 @@ TARFILES = mega-manual.html mega-style.css figures/yocto-environment.png figures
 	figures/yocto-environment-ref.png figures/user-configuration.png figures/source-input.png \
 	figures/package-feeds.png figures/layer-input.png figures/images.png figures/sdk.png \
 	figures/source-fetching.png figures/patching.png figures/configuration-compile-autoreconf.png \
-	figures/analysis-for-package-splitting.png
+	figures/analysis-for-package-splitting.png figures/image-generation.png \
+	figures/sdk-generation.png
 	endif
 
 MANUALS = $(DOC)/$(DOC).html
@@ -219,7 +220,8 @@ TARFILES = ref-manual.html ref-style.css figures/poky-title.png \
 	figures/user-configuration.png figures/yocto-environment-ref.png \
 	figures/images.png figures/sdk.png figures/source-fetching.png \
 	figures/patching.png figures/configuration-compile-autoreconf.png \
-	figures/analysis-for-package-splitting.png
+	figures/analysis-for-package-splitting.png figures/image-generation.png \
+	figures/sdk-generation.png
 MANUALS = $(DOC)/$(DOC).html $(DOC)/$(DOC).pdf $(DOC)/eclipse
 FIGURES = figures
 STYLESHEET = $(DOC)/*.css
@@ -381,8 +383,8 @@ publish:
             echo " "; \
             echo "******** Publishing "$(DOC)".html"; \
             echo " "; \
-            scp -r $(MANUALS) $(STYLESHEET) www.yoctoproject.org:/srv/www/www.yoctoproject.org-docs/$(VER)/$(DOC); \
-            cd $(DOC); scp -r $(FIGURES) www.yoctoproject.org:/srv/www/www.yoctoproject.org-docs/$(VER)/$(DOC); \
+            scp -r $(MANUALS) $(STYLESHEET) docs.yp:/var/www/www.yoctoproject.org-docs/$(VER)/$(DOC); \
+            cd $(DOC); scp -r $(FIGURES) docs.yp:/var/www/www.yoctoproject.org-docs/$(VER)/$(DOC); \
 	else \
           echo " "; \
           echo $(DOC)".html missing.  Generate the file first then try again."; \

documentation/adt-manual/adt-manual.xml

@@ -63,7 +63,7 @@
             </revision>
             <revision>
                 <revnumber>1.5</revnumber>
-                <date>Sometime in 2013</date>
+                <date>October 2013</date>
                 <revremark>Released with the Yocto Project 1.5 Release.</revremark>
             </revision>
        </revhistory>

documentation/adt-manual/adt-prepare.xml

@@ -91,7 +91,9 @@
             <para>
                 If you use BitBake to generate the ADT Installer tarball, you must
                 <filename>source</filename> the environment setup script
-                (<ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink>)
+                (<ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink>
+                or
+                <ulink url='&YOCTO_DOCS_REF_URL;#structure-memres-core-script'><filename>oe-init-build-env-memres</filename></ulink>)
                 located in the Source Directory before running the
                 BitBake command that creates the tarball.
             </para>
@@ -266,7 +268,7 @@
                             target, go into the <filename>x86_64</filename>
                             folder and download the following installer:
                             <literallayout class='monospaced'>
-     poky-eglibc-x86_64-core-image-sato-i586.sh
+     poky-eglibc-x86_64-core-image-sato-i586-&DISTRO;.sh
                             </literallayout></para></listitem>
                         <listitem><para>Build your own toolchain installer.
                             For cases where you cannot use an installer
@@ -285,7 +287,7 @@
                     The example assumes the toolchain installer is located
                     in <filename>~/Downloads/</filename>.
                     <literallayout class='monospaced'>
-     $ ~/Downloads/poky-eglibc-x86_64-core-image-sato-i586.sh
+     $ ~/Downloads/poky-eglibc-x86_64-core-image-sato-i586-&DISTRO;.sh
                     </literallayout>
                     <note>
                         If you do not have write permissions for the directory
@@ -374,10 +376,11 @@
     </para>
 
     <para>
-        Be sure to run the environment setup script that matches the architecture for
-        which you are developing.
-        Environment setup scripts begin with the string "<filename>environment-setup</filename>"
-        and include as part of their name the architecture.
+        Be sure to run the environment setup script that matches the
+        architecture for which you are developing.
+        Environment setup scripts begin with the string
+        "<filename>environment-setup</filename>" and include as part of their
+        name the architecture.
         For example, the toolchain environment setup script for a 64-bit
         IA-based architecture installed in the default installation directory
         would be the following:
@@ -540,7 +543,7 @@
 
     <para>
         A final method of building the toolchain installer exists that has
-        significant advantages over the previous two methods.
+        significant advantages over the previous method.
         This method results in a toolchain installer that contains the sysroot
         that matches your target root filesystem.
         To build this installer, use the
@@ -549,8 +552,12 @@
 
     <para>
          Remember, before using any <filename>bitbake</filename> command, you
-         must source the <filename>&OE_INIT_PATH;</filename> script located in
-         the Source Directory and you must make sure your
+         must source the build environment setup script
+         (i.e.
+         <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink>
+         or
+         <ulink url='&YOCTO_DOCS_REF_URL;#structure-memres-core-script'><filename>oe-init-build-env-memres</filename></ulink>)
+         located in the Source Directory and you must make sure your
          <filename>conf/local.conf</filename> variables are correct.
          In particular, you need to be sure the
          <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>

documentation/bsp-guide/bsp-guide.xml

@@ -75,7 +75,7 @@
             </revision>
             <revision>
                 <revnumber>1.5</revnumber>
-                <date>Sometime in 2013</date>
+                <date>October 2013</date>
                 <revremark>Released with the Yocto Project 1.5 Release.</revremark>
             </revision>
         </revhistory>

documentation/bsp-guide/bsp.xml

@@ -183,17 +183,15 @@
      meta-crownbay/recipes-graphics/xorg-xserver/xserver-xf86-config/
      meta-crownbay/recipes-graphics/xorg-xserver/xserver-xf86-config/crownbay/
      meta-crownbay/recipes-graphics/xorg-xserver/xserver-xf86-config/crownbay/xorg.conf
-     meta-crownbay/recipes-graphics/xorg-xserver/xserver-xf86-config/crownbay-noemgd/
-     meta-crownbay/recipes-graphics/xorg-xserver/xserver-xf86-config/crownbay-noemgd/xorg.conf
      meta-crownbay/recipes-kernel/
      meta-crownbay/recipes-kernel/linux/
-     meta-crownbay/recipes-kernel/linux/linux-yocto_3.2.bbappend
      meta-crownbay/recipes-kernel/linux/linux-yocto_3.4.bbappend
      meta-crownbay/recipes-kernel/linux/linux-yocto_3.8.bbappend
+     meta-crownbay/recipes-kernel/linux/linux-yocto_3.10.bbappend
      meta-crownbay/recipes-kernel/linux/linux-yocto-dev.bbappend
-     meta-crownbay/recipes-kernel/linux/linux-yocto-rt_3.2.bbappend
      meta-crownbay/recipes-kernel/linux/linux-yocto-rt_3.4.bbappend
      meta-crownbay/recipes-kernel/linux/linux-yocto-rt_3.8.bbappend
+     meta-crownbay/recipes-kernel/linux/linux-yocto-rt_3.10.bbappend
                 </literallayout>
             </para>
 
@@ -376,15 +374,6 @@
             <para>
                 Each BSP Layer requires at least one machine file.
                 However, you can supply more than one file.
-                For example, in the Crown Bay BSP shown earlier in this section, the
-                <filename>conf/machine</filename> directory contains two configuration files:
-                <filename>crownbay.conf</filename> and <filename>crownbay-noemgd.conf</filename>.
-                The <filename>crownbay.conf</filename> file is used for the Crown Bay BSP
-                that supports the <trademark class='registered'>Intel</trademark> Embedded
-                Media and Graphics Driver (<trademark class='registered'>Intel</trademark>
-                EMGD), while the <filename>crownbay-noemgd</filename> file is used for the
-                Crown Bay BSP that supports Video Electronics Standards Association (VESA)
-                graphics only.
             </para>
 
             <para>
@@ -428,15 +417,13 @@
                 This optional directory contains miscellaneous recipe files for the BSP.
                 Most notably would be the formfactor files.
                 For example, in the Crown Bay BSP there is the
-                <filename>formfactor_0.0.bbappend</filename> file, which is an append file used
-                to augment the recipe that starts the build.
-                Furthermore, there are machine-specific settings used during the build that are
-                defined by the <filename>machconfig</filename> files.
-                In the Crown Bay example, two <filename>machconfig</filename> files exist:
-                one that supports the
-                <trademark class='registered'>Intel</trademark> Embedded
-                Media and Graphics Driver (<trademark class='registered'>Intel</trademark>
-                EMGD) and one that does not:
+                <filename>formfactor_0.0.bbappend</filename> file, which is an
+                append file used to augment the recipe that starts the build.
+                Furthermore, there are machine-specific settings used during the
+                build that are defined by the <filename>machconfig</filename>.
+                In the Crown Bay example, two <filename>machconfig</filename> files
+                exist: one that supports the Intel® Embedded Media and Graphics
+                Driver (Intel® EMGD) and one that does not:
                 <literallayout class='monospaced'>
      meta-crownbay/recipes-bsp/formfactor/formfactor/crownbay/machconfig
      meta-crownbay/recipes-bsp/formfactor/formfactor/crownbay-noemgd/machconfig
@@ -467,15 +454,12 @@
                 This optional directory contains recipes for the BSP if it has
                 special requirements for graphics support.
                 All files that are needed for the BSP to support a display are kept here.
-                For example, the Crown Bay BSP contains two versions of the
-                <filename>xorg.conf</filename> file.
-                The version in <filename>crownbay</filename> builds a BSP that supports the
-                <trademark class='registered'>Intel</trademark> Embedded Media Graphics Driver (EMGD),
-                while the version in <filename>crownbay-noemgd</filename> builds
-                a BSP that supports Video Electronics Standards Association (VESA) graphics only:
+                For example, the Crown Bay BSP's <filename>xorg.conf</filename> file
+                detects the graphics support needed (i.e. the Intel® Embedded Media
+                Graphics Driver (EMGD) or the Video Electronics Standards Association
+                (VESA) graphics):
                 <literallayout class='monospaced'>
      meta-crownbay/recipes-graphics/xorg-xserver/xserver-xf86-config_0.1.bbappend
-     meta-crownbay/recipes-graphics/xorg-xserver/xserver-xf86-config/crownbay/xorg.conf
      meta-crownbay/recipes-graphics/xorg-xserver/xserver-xf86-config/crownbay-noemgd/xorg.conf
                 </literallayout>
             </para>
@@ -502,28 +486,28 @@
                 the <filename>meta-&lt;bsp_name&gt;/recipes-kernel/linux</filename> directory).
             </para>
             <para>
-                Suppose you are using the <filename>linux-yocto_3.8.bb</filename> recipe to build
+                Suppose you are using the <filename>linux-yocto_3.10.bb</filename> recipe to build
                 the kernel.
                 In other words, you have selected the kernel in your
                 <filename>&lt;bsp_name&gt;.conf</filename> file by adding these types
                 of statements:
                 <literallayout class='monospaced'>
      PREFERRED_PROVIDER_virtual/kernel ?= "linux-yocto"
-     PREFERRED_VERSION_linux-yocto ?= "3.8%"
+     PREFERRED_VERSION_linux-yocto ?= "3.10%"
                 </literallayout>
                 <note>
                     When the preferred provider is assumed by default, the
                     <filename>PREFERRED_PROVIDER</filename> statement does not appear in the
                     <filename>&lt;bsp_name&gt;.conf</filename> file.
                 </note>
-                You would use the <filename>linux-yocto_3.8.bbappend</filename> file to append
+                You would use the <filename>linux-yocto_3.10.bbappend</filename> file to append
                 specific BSP settings to the kernel, thus configuring the kernel for your particular BSP.
             </para>
             <para>
                 As an example, look at the existing Crown Bay BSP.
                 The append file used is:
                 <literallayout class='monospaced'>
-     meta-crownbay/recipes-kernel/linux/linux-yocto_3.8.bbappend
+     meta-crownbay/recipes-kernel/linux/linux-yocto_3.10.bbappend
                 </literallayout>
                 The following listing shows the file.
                 Be aware that the actual commit ID strings in this example listing might be different
@@ -532,46 +516,18 @@
                 <literallayout class='monospaced'>
      FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
 
-     COMPATIBLE_MACHINE_crownbay = "crownbay"
-     KMACHINE_crownbay = "crownbay"
-     KBRANCH_crownbay = "standard/crownbay"
-     KERNEL_FEATURES_crownbay_append = " features/drm-emgd/drm-emgd-1.16 cfg/vesafb"
-
      COMPATIBLE_MACHINE_crownbay-noemgd = "crownbay-noemgd"
      KMACHINE_crownbay-noemgd = "crownbay"
      KBRANCH_crownbay-noemgd = "standard/crownbay"
-     KERNEL_FEATURES_crownbay-noemgd_append = " cfg/vesafb"
+     KERNEL_FEATURES_append_crownbay-noemgd = " cfg/vesafb"
 
-     LINUX_VERSION = "3.8.4"
+     LINUX_VERSION = "3.10.11"
 
-     SRCREV_meta_crownbay = "2a6d36e75ca0a121570a389d7bab76ec240cbfda"
-     SRCREV_machine_crownbay = "47aed0c17c1c55988198ad39f86ae88894c8e0a4"
-     SRCREV_emgd_crownbay = "c780732f175ff0ec866fac2130175876b519b576"
-
-     SRCREV_meta_crownbay-noemgd = "2a6d36e75ca0a121570a389d7bab76ec240cbfda"
-     SRCREV_machine_crownbay-noemgd = "47aed0c17c1c55988198ad39f86ae88894c8e0a4"
-
-     SRC_URI_crownbay = "git://git.yoctoproject.org/linux-yocto-3.8.git;protocol=git;nocheckout=1;branch=${KBRANCH},${KMETA},emgd-1.16;name=machine,meta,emgd"
+     SRCREV_meta_crownbay-noemgd = "285f93bf942e8f6fa678ffc6cc53696ed5400718"
+     SRCREV_machine_crownbay-noemgd = "702040ac7c7ec66a29b4d147665ccdd0ff015577"
                 </literallayout>
-                This append file contains statements used to support the Crown Bay BSP for both
-                <trademark class='registered'>Intel</trademark> EMGD and the VESA graphics.
-                The build process, in this case, recognizes and uses only the statements that
-                apply to the defined machine name - <filename>crownbay</filename> in this case.
-                So, the applicable statements in the <filename>linux-yocto_3.8.bbappend</filename>
-                file are follows:
-                <literallayout class='monospaced'>
-     FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
-
-     COMPATIBLE_MACHINE_crownbay = "crownbay"
-     KMACHINE_crownbay = "crownbay"
-     KBRANCH_crownbay = "standard/crownbay"
-     KERNEL_FEATURES_crownbay_append = " features/drm-emgd/drm-emgd-1.16 cfg/vesafb"
-
-     SRCREV_meta_crownbay = "2a6d36e75ca0a121570a389d7bab76ec240cbfda"
-     SRCREV_machine_crownbay = "47aed0c17c1c55988198ad39f86ae88894c8e0a4"
-     SRCREV_emgd_crownbay = "c780732f175ff0ec866fac2130175876b519b576"
-                </literallayout>
-                The append file defines <filename>crownbay</filename> as the
+                This append file contains statements used to support the Crown Bay BSP.
+                The file defines <filename>crownbay</filename> as the
                 <ulink url='&YOCTO_DOCS_REF_URL;#var-COMPATIBLE_MACHINE'><filename>COMPATIBLE_MACHINE</filename></ulink>
                 and uses the
                 <ulink url='&YOCTO_DOCS_REF_URL;#var-KMACHINE'><filename>KMACHINE</filename></ulink> variable to
@@ -588,12 +544,6 @@
                 <ulink url='&YOCTO_DOCS_DEV_URL;#source-directory'>Source Directory</ulink> Git
                 repository and the <filename>meta</filename> Git repository branches to identify the
                 exact kernel needed to build the Crown Bay BSP.
-                <note>
-                    For <filename>crownbay</filename>, a specific commit is also needed to point
-                    to the branch that supports EMGD graphics.
-                    At a minimum, every BSP points to the
-                    <filename>machine</filename> and <filename>meta</filename> commits.
-                </note>
             </para>
 
             <para>
@@ -1281,31 +1231,35 @@
                     Following is the complete example:
                     <literallayout class='monospaced'>
      $ yocto-bsp create myarm qemu
+     Checking basic git connectivity...
+     Done.
+
      Which qemu architecture would you like to use? [default: i386]
-	     1) i386    (32-bit)
+     	     1) i386    (32-bit)
 	     2) x86_64  (64-bit)
 	     3) ARM     (32-bit)
 	     4) PowerPC (32-bit)
 	     5) MIPS    (32-bit)
      3
-     Would you like to use the default (3.8) kernel? (y/n) [default: y]
+     Would you like to use the default (3.10) kernel? (y/n) [default: y] y
      Do you need a new machine branch for this BSP (the alternative is to re-use an existing branch)? [y/n] [default: y]
-     Getting branches from remote repo git://git.yoctoproject.org/linux-yocto-3.8.git...
+     Getting branches from remote repo git://git.yoctoproject.org/linux-yocto-3.10.git...
      Please choose a machine branch to base your new BSP branch on: [default: standard/base]
-             1) standard/arm-versatile-926ejs
-             2) standard/base
-             3) standard/beagleboard
-             4) standard/ck
-             5) standard/crownbay
-             6) standard/edf
-             7) standard/emenlow
-             8) standard/fri2
-             9) standard/fsl-mpc8315e-rdb
-             10) standard/mti-malta32
-             11) standard/mti-malta64
-             12) standard/qemuppc
-             13) standard/routerstationpro
-             14) standard/sys940x
+	     1) standard/arm-versatile-926ejs
+	     2) standard/base
+	     3) standard/beagleboard
+	     4) standard/ck
+	     5) standard/crownbay
+	     6) standard/edf
+	     7) standard/emenlow
+	     8) standard/fri2
+	     9) standard/fsl-mpc8315e-rdb
+	     10) standard/minnow
+	     11) standard/mti-malta32
+	     12) standard/mti-malta64
+	     13) standard/qemuppc
+	     14) standard/routerstationpro
+	     15) standard/sys940x
      1
      Would you like SMP support? (y/n) [default: y]
      Does your BSP have a touchscreen? (y/n) [default: n]
@@ -1320,7 +1274,7 @@
                             In the example, we use the ARM architecture.
                             </para></listitem>
                         <listitem><para>The script then prompts you for the kernel.
-                            The default 3.8 kernel is acceptable.
+                            The default 3.10 kernel is acceptable.
                             So, the example accepts the default.
                             If you enter 'n', the script prompts you to further enter the kernel
                             you do want to use (e.g. 3.2, 3.2_preempt-rt, and so forth.).</para></listitem>

documentation/dev-manual/dev-manual-intro.xml

@@ -24,17 +24,6 @@
             set up your host development system and build an image, which you
             find in the
             <ulink url='&YOCTO_DOCS_QS_URL;'>Yocto Project Quick Start</ulink>.
-            <note>
-                By default, using the Yocto Project creates a Poky distribution.
-                However, you can create your own distribution by providing key
-                <link linkend='metadata'>Metadata</link>.
-                A good example is Angstrom, which has had a distribution
-                based on the Yocto Project since its inception.
-                Other examples include  commercial distributions like
-                Wind River Linux, Mentor Embedded Linux, and ENEA Linux.
-                See the "<link linkend='creating-your-own-distribution'>Creating Your Own Distribution</link>"
-                section for more information.
-            </note>
         </para>
 
         <para>
@@ -43,13 +32,25 @@
             reconfigure the kernel, and develop an application using the
             popular <trademark class='trade'>Eclipse</trademark> IDE.
         </para>
+
+        <note>
+            By default, using the Yocto Project creates a Poky distribution.
+            However, you can create your own distribution by providing key
+            <link linkend='metadata'>Metadata</link>.
+            A good example is Angstrom, which has had a distribution
+            based on the Yocto Project since its inception.
+            Other examples include  commercial distributions like
+            Wind River Linux, Mentor Embedded Linux, and ENEA Linux.
+            See the "<link linkend='creating-your-own-distribution'>Creating Your Own Distribution</link>"
+            section for more information.
+        </note>
     </section>
 
     <section id='what-this-manual-provides'>
         <title>What This Manual Provides</title>
 
         <para>
-            The following list describes what you can get from this guide:
+            The following list describes what you can get from this manual:
             <itemizedlist>
                 <listitem><para>Information that lets you get set
                     up to develop using the Yocto Project.</para></listitem>
@@ -75,17 +76,17 @@
         <para>
             This manual will not give you the following:
             <itemizedlist>
-                <listitem><para>Step-by-step instructions if those instructions exist in other Yocto
-                    Project documentation.
+                <listitem><para><emphasis>Step-by-step instructions when those instructions exist in other Yocto
+                    Project documentation:</emphasis>
                     For example, the Yocto Project Application Developer's Guide contains detailed
                     instructions on how to run the
-                    <ulink url='&YOCTO_DOCS_ADT_URL;#installing-the-adt'>Installing the ADT and Toolchains</ulink>,
+                    <ulink url='&YOCTO_DOCS_ADT_URL;#installing-the-adt'>ADT Installer</ulink>,
                     which is used to set up a cross-development environment.</para></listitem>
-                <listitem><para>Reference material.
+                <listitem><para><emphasis>Reference material:</emphasis>
                     This type of material resides in an appropriate reference manual.
                     For example, system variables are documented in the
                     <ulink url='&YOCTO_DOCS_REF_URL;'>Yocto Project Reference Manual</ulink>.</para></listitem>
-                <listitem><para>Detailed public information that is not specific to the Yocto Project.
+                <listitem><para><emphasis>Detailed public information that is not specific to the Yocto Project:</emphasis>
                     For example, exhaustive information on how to use Git is covered better through the
                     Internet than in this manual.</para></listitem>
             </itemizedlist>
@@ -109,7 +110,8 @@
                     with the Yocto Project and quickly begin building an image.</para></listitem>
                 <listitem><para><emphasis>
                     <ulink url='&YOCTO_DOCS_REF_URL;'>Yocto Project Reference Manual</ulink>:</emphasis> This manual is a reference
-                    guide to the OpenEmbedded build system known as "Poky."
+                    guide to the OpenEmbedded build system, which is based on BitBake.
+                    The build system is sometimes referred to as "Poky".
                     </para></listitem>
                 <listitem><para><emphasis>
                     <ulink url='&YOCTO_DOCS_ADT_URL;'>Yocto Project Application Developer's Guide</ulink>:</emphasis>

documentation/dev-manual/dev-manual-newbie.xml

@@ -254,9 +254,9 @@
                 <listitem><para><ulink url='http://sitaramc.github.com/gitolite/master-toc.html'>The <filename>gitolite</filename> master index</ulink>:
                     All topics for <filename>gitolite</filename>.
                     </para></listitem>
-                <listitem><para><ulink url='http://hjemli.net/git/cgit/tree/README'><filename>cgit</filename> index</ulink>:
-                    A <filename>README</filename> file on how to create a
-                    fast web interface for Git.</para></listitem>
+                <listitem><para><ulink url='https://git.wiki.kernel.org/index.php/Interfaces,_frontends,_and_tools'>Interfaces, frontends, and tools</ulink>:
+                    Documentation on how to create interfaces and frontends
+                    for Git.</para></listitem>
             </itemizedlist>
         </note>
     </section>
@@ -275,9 +275,8 @@
         </para>
 
         <para>
-            See "<ulink url='http://autobuilder.yoctoproject.org:8010/'>Welcome to the buildbot for the Yocto Project</ulink>"
-            for the Yocto Project's reference implementation that uses
-            buildbot.
+            See "<ulink url='http://autobuilder.yoctoproject.org'>Yocto Project Autobuilder</ulink>"
+            for more information and links to buildbot.
             The Yocto Project team has found this implementation
             works well in this role.
             A public example of this is the Yocto Project
@@ -394,7 +393,7 @@
                     For some guidance on mailing lists to use, see the list in the
                     "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>"
                     section.
-                    For a description of the available mailing lists, see
+                    For a description of the available mailing lists, see the
                     "<ulink url='&YOCTO_DOCS_REF_URL;#resources-mailinglist'>Mailing Lists</ulink>"
                     section in the Yocto Project Reference Manual.
                     </para></listitem>
@@ -509,7 +508,9 @@
                 This term refers to the area used by the OpenEmbedded build system for builds.
                 The area is created when you <filename>source</filename> the setup
                 environment script that is found in the Source Directory
-                (i.e. <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink>).
+                (i.e. <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink>
+                or
+                <ulink url='&YOCTO_DOCS_REF_URL;#structure-memres-core-script'><filename>oe-init-build-env-memres</filename></ulink>).
                 The <ulink url='&YOCTO_DOCS_REF_URL;#var-TOPDIR'><filename>TOPDIR</filename></ulink>
                 variable points to the Build Directory.</para>
 
@@ -613,7 +614,7 @@
                 "<ulink url='&YOCTO_DOCS_BSP_URL;#bsp-layers'>BSP Layers</ulink>"
                 section in the Yocto Project Board Support Packages (BSP)
                 Developer's Guide.</para></listitem>
-            <listitem><para id='meta-toochain'><emphasis>Meta-Toolchain:</emphasis>
+            <listitem><para id='meta-toolchain'><emphasis>Meta-Toolchain:</emphasis>
                 A term sometimes used for
                 <link linkend='cross-development-toolchain'>Cross-Development Toolchain</link>.
                 </para></listitem>
@@ -687,7 +688,7 @@
                 For example, downloading and unpacking <filename>&YOCTO_POKY_TARBALL;</filename>
                 results in a Source Directory whose top-level folder is named
                 <filename>&YOCTO_POKY;</filename>.
-                If you create a local copy of the Git repository, then you can name the repository
+                If you create a local copy of the Git repository, you can name the repository
                 anything you like.
                 Throughout much of the documentation, <filename>poky</filename> is used as the name of
                 the top-level folder of the local copy of the poky Git repository.
@@ -863,8 +864,8 @@
             It is important to understand that Git tracks content change and not files.
             Git uses "branches" to organize different development efforts.
             For example, the <filename>poky</filename> repository has
-            <filename>bernard</filename>,
-            <filename>edison</filename>, <filename>denzil</filename>, <filename>danny</filename>
+            <filename>denzil</filename>, <filename>danny</filename>,
+            <filename>dylan</filename>, <filename>dora</filename>,
             and <filename>master</filename> branches among others.
             You can see all the branches by going to
             <ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/'></ulink> and
@@ -1231,6 +1232,10 @@
                 occurred.</para></listitem>
             <listitem><para>Be sure to indicate the Severity of the bug.
                 Severity communicates how the bug impacted your work.</para></listitem>
+            <listitem><para>Select the appropriate "Documentation change" item
+                for the bug.
+                Fixing a bug may or may not affect the Yocto Project
+                documentation.</para></listitem>
             <listitem><para>Provide a brief summary of the issue.
                 Try to limit your summary to just a line or two and be sure to capture the
                 essence of the issue.</para></listitem>

documentation/dev-manual/dev-manual-start.xml

@@ -32,7 +32,7 @@
 
     <para>
         You can use the OpenEmbedded build system, which uses
-        BitBake to develop complete Linux
+        BitBake, to develop complete Linux
         images and associated user-space applications for architectures based
         on ARM, MIPS, PowerPC, x86 and x86-64.
         <note>
@@ -122,11 +122,11 @@
                         <literallayout class='monospaced'>
      $ git clone git://git.yoctoproject.org/poky
      Cloning into 'poky'...
-     remote: Counting objects: 183981, done.
-     remote: Compressing objects: 100% (47428/47428), done.
-     remote: Total 183981 (delta 132271), reused 183703 (delta 132044)
-     Receiving objects: 100% (183981/183981), 89.71 MiB | 2.93 MiB/s, done.
-     Resolving deltas: 100% (132271/132271), done.
+     remote: Counting objects: 203728, done.
+     remote: Compressing objects: 100% (52371/52371), done.
+     remote: Total 203728 (delta 147444), reused 202891 (delta 146614)
+     Receiving objects: 100% (203728/203728), 95.54 MiB | 308 KiB/s, done.
+     Resolving deltas: 100% (147444/147444), done.
                         </literallayout></para>
                         <para>For another example of how to set up your own local Git repositories, see this
                         <ulink url='&YOCTO_WIKI_URL;/wiki/Transcript:_from_git_checkout_to_meta-intel_BSP'>
@@ -145,27 +145,28 @@
                 For simplicity, it is recommended that you create these structures outside of the
                 Source Directory (usually <filename>poky</filename>).</para>
                 <para>As an example, the following transcript shows how to create the bare clone
-                of the <filename>linux-yocto-3.8</filename> kernel and then create a copy of
+                of the <filename>linux-yocto-3.10</filename> kernel and then create a copy of
                 that clone.
                 <note>When you have a local Yocto Project kernel Git repository, you can
                 reference that repository rather than the upstream Git repository as
                 part of the <filename>clone</filename> command.
                 Doing so can speed up the process.</note></para>
                 <para>In the following example, the bare clone is named
-                <filename>linux-yocto-3.8.git</filename>, while the
-                copy is named <filename>my-linux-yocto-3.8-work</filename>:
+                <filename>linux-yocto-3.10.git</filename>, while the
+                copy is named <filename>my-linux-yocto-3.10-work</filename>:
                 <literallayout class='monospaced'>
-     $ git clone --bare git://git.yoctoproject.org/linux-yocto-3.8 linux-yocto-3.8.git
-     Cloning into bare repository 'linux-yocto-3.8.git'...
-     remote: Counting objects: 2847090, done.
-     remote: Compressing objects: 100% (454675/454675), done.
-     remote: Total 2847090 (delta 2386170), reused 2825793 (delta 2364886)
-     Receiving objects: 100% (2847090/2847090), 603.19 MiB | 3.54 MiB/s, done.
-     Resolving deltas: 100% (2386170/2386170), done.                </literallayout></para>
+     $ git clone --bare git://git.yoctoproject.org/linux-yocto-3.10 linux-yocto-3.10.git
+     Cloning into bare repository 'linux-yocto-3.10.git'...
+     remote: Counting objects: 3364487, done.
+     remote: Compressing objects: 100% (507178/507178), done.
+     remote: Total 3364487 (delta 2827715), reused 3364481 (delta 2827709)
+     Receiving objects: 100% (3364487/3364487), 722.95 MiB | 423 KiB/s, done.
+     Resolving deltas: 100% (2827715/2827715), done.
+                </literallayout></para>
                 <para>Now create a clone of the bare clone just created:
                 <literallayout class='monospaced'>
-     $ git clone linux-yocto-3.8.git my-linux-yocto-3.8-work
-     Cloning into 'my-linux-yocto-3.8-work'...
+     $ git clone linux-yocto-3.10.git my-linux-yocto-3.10-work
+     Cloning into 'my-linux-yocto-3.10-work'...
      done.
                 </literallayout></para></listitem>
             <listitem id='meta-yocto-kernel-extras-repo'><para><emphasis>
@@ -189,11 +190,12 @@
      $ cd ~/poky
      $ git clone git://git.yoctoproject.org/meta-yocto-kernel-extras meta-yocto-kernel-extras
      Cloning into 'meta-yocto-kernel-extras'...
-     remote: Counting objects: 690, done.
-     remote: Compressing objects: 100% (431/431), done.
-     remote: Total 690 (delta 238), reused 690 (delta 238)
-     Receiving objects: 100% (690/690), 532.60 KiB, done.
-     Resolving deltas: 100% (238/238), done.                </literallayout></para></listitem>
+     remote: Counting objects: 727, done.
+     remote: Compressing objects: 100% (452/452), done.
+     remote: Total 727 (delta 260), reused 719 (delta 252)
+     Receiving objects: 100% (727/727), 536.36 KiB | 102 KiB/s, done.
+     Resolving deltas: 100% (260/260), done.
+               </literallayout></para></listitem>
            <listitem><para id='supported-board-support-packages-(bsps)'><emphasis>Supported Board
                 Support Packages (BSPs):</emphasis>
                 The Yocto Project provides a layer called <filename>meta-intel</filename> and
@@ -226,8 +228,8 @@
                         BSP tarball from the same "Downloads" page of the
                         <ulink url='&YOCTO_HOME_URL;'>Yocto Project Website</ulink>
                         to get the Yocto Project release.
-                        Once on the "Download" page, look for "BSP" under the
-                        "Type" heading.</para>
+                        Once on the "Download" page, look to the right of the
+                        page and scroll down to find the BSP tarballs.</para>
                         <para>Once you have the tarball, just extract it into a directory of your choice.
                         Again, this method just produces a snapshot of the BSP layer in the form
                         of a hierarchical directory structure.</para></listitem>
@@ -246,16 +248,17 @@
      $ cd ~/poky
      $ git clone git://git.yoctoproject.org/meta-intel.git
      Cloning into 'meta-intel'...
-     remote: Counting objects: 6264, done.
-     remote: Compressing objects: 100% (2135/2135), done.
-     remote: Total 6264 (delta 3321), reused 6235 (delta 3293)
-     Receiving objects: 100% (6264/6264), 2.17 MiB | 2.63 MiB/s, done.
-     Resolving deltas: 100% (3321/3321), done.
+     remote: Counting objects: 7366, done.
+     remote: Compressing objects: 100% (2491/2491), done.
+     remote: Total 7366 (delta 3997), reused 7299 (delta 3930)
+     Receiving objects: 100% (7366/7366), 2.31 MiB | 95 KiB/s, done.
+     Resolving deltas: 100% (3997/3997), done.
                         </literallayout></para>
                         <para>The same
-                        <ulink url='&YOCTO_WIKI_URL;/wiki/Transcript:_from_git_checkout_to_meta-intel_BSP'>
-                        wiki page</ulink> referenced earlier covers how to
-                        set up the <filename>meta-intel</filename> Git repository.</para></listitem>
+                        <ulink url='&YOCTO_WIKI_URL;/wiki/Transcript:_from_git_checkout_to_meta-intel_BSP'>wiki page</ulink>
+                        referenced earlier covers how to
+                        set up the <filename>meta-intel</filename> Git repository.
+                        </para></listitem>
                 </itemizedlist></para></listitem>
             <listitem><para><emphasis>Eclipse Yocto Plug-in:</emphasis>  If you are developing
                 applications using the Eclipse Integrated Development Environment (IDE),

documentation/dev-manual/dev-manual.xml

@@ -53,7 +53,7 @@
             </revision>
             <revision>
                 <revnumber>1.5</revnumber>
-                <date>Sometime in 2013</date>
+                <date>October 2013</date>
                 <revremark>Released with the Yocto Project 1.5 Release.</revremark>
             </revision>
         </revhistory>

documentation/kernel-dev/kernel-dev-advanced.xml

@@ -296,6 +296,20 @@
      $ git commit --allow-empty -m "Create orphan meta branch"
             </literallayout>
         </para>
+
+        <para>
+            If you modify the Metadata in the linux-yocto 
+            <filename>meta</filename> branch, you must not forget to update 
+            the 
+            <ulink url='&YOCTO_DOCS_REF_URL;#var-SRCREV'><filename>SRCREV</filename></ulink>
+            statements in the kernel's recipe.
+            In particular, you need to update the 
+            <filename>SRCREV_meta</filename> variable to match the commit in 
+            the <filename>KMETA</filename> branch you wish to use.
+            Changing the data in these branches and not updating the 
+            <filename>SRCREV</filename> statements to match will cause the 
+            build to fetch an older commit.
+        </para>
     </section>
 </section>
 

documentation/kernel-dev/kernel-dev.xml

@@ -38,7 +38,7 @@
             </revision>
             <revision>
                 <revnumber>1.5</revnumber>
-                <date>Sometime in 2013</date>
+                <date>October 2013</date>
                 <revremark>Released with the Yocto Project 1.5 Release.</revremark>
             </revision>
         </revhistory>

documentation/poky.ent

@@ -1,6 +1,6 @@
 <!ENTITY DISTRO "1.5">
 <!ENTITY DISTRO_COMPRESSED "15">
-<!ENTITY DISTRO_NAME "tbd">
+<!ENTITY DISTRO_NAME "dora">
 <!ENTITY YOCTO_DOC_VERSION "1.5">
 <!ENTITY POKYVERSION "10.0.0">
 <!ENTITY POKYVERSION_COMPRESSED "1000">
@@ -26,6 +26,7 @@
 <!ENTITY ECLIPSE_UPDATES_URL "&ECLIPSE_DL_URL;/tm/updates/3.3">
 <!ENTITY ECLIPSE_INDIGO_URL "&ECLIPSE_DL_URL;/releases/indigo">
 <!ENTITY ECLIPSE_JUNO_URL "&ECLIPSE_DL_URL;/releases/juno">
+<!ENTITY ECLIPSE_KEPLER_URL "&ECLIPSE_DL_URL;/releases/kepler">
 <!ENTITY ECLIPSE_INDIGO_CDT_URL "&ECLIPSE_DL_URL;tools/cdt/releases/indigo">
 <!ENTITY YOCTO_DOCS_URL "&YOCTO_HOME_URL;/docs">
 <!ENTITY YOCTO_SOURCES_URL "&YOCTO_HOME_URL;/sources/">
@@ -35,7 +36,7 @@
 <!ENTITY YOCTO_RELEASE_DL_URL "&YOCTO_DL_URL;/releases/yocto/yocto-&DISTRO;">
 <!ENTITY YOCTO_TOOLCHAIN_DL_URL "&YOCTO_RELEASE_DL_URL;/toolchain/">
 <!ENTITY YOCTO_ECLIPSE_DL_URL "&YOCTO_RELEASE_DL_URL;/eclipse-plugin/indigo;">
-<!ENTITY YOCTO_ADTINSTALLER_DL_URL "&YOCTO_RELEASE_DL_URL;/adt_installer">
+<!ENTITY YOCTO_ADTINSTALLER_DL_URL "&YOCTO_RELEASE_DL_URL;/adt-installer">
 <!ENTITY YOCTO_POKY_DL_URL "&YOCTO_RELEASE_DL_URL;/&YOCTO_POKY;.tar.bz2">
 <!ENTITY YOCTO_MACHINES_DL_URL "&YOCTO_RELEASE_DL_URL;/machines">
 <!ENTITY YOCTO_QEMU_DL_URL "&YOCTO_MACHINES_DL_URL;/qemu">
@@ -56,8 +57,8 @@
 <!ENTITY UBUNTU_HOST_PACKAGES_ESSENTIAL "gawk wget git-core diffstat unzip texinfo \
      build-essential chrpath">
 <!ENTITY FEDORA_HOST_PACKAGES_ESSENTIAL "gawk make wget tar bzip2 gzip python unzip perl patch \
-     diffutils diffstat git cpp gcc gcc-c++ eglibc-devel texinfo chrpath \
-     ccache">
+     diffutils diffstat git cpp gcc gcc-c++ glibc-devel texinfo chrpath \
+     ccache perl-Data-Dumper perl-Text-ParseWords">
 <!ENTITY OPENSUSE_HOST_PACKAGES_ESSENTIAL "python gcc gcc-c++ git chrpath make wget python-xml \
      diffstat texinfo python-curses patch">
 <!ENTITY CENTOS_HOST_PACKAGES_ESSENTIAL "gawk make wget tar bzip2 gzip python unzip perl patch \

documentation/profile-manual/profile-manual-usage.xml

@@ -3094,7 +3094,7 @@
                 If you already have an LTTng trace on a remote target and
                 would like to view it in Eclipse on the host, you can easily
                 copy it from the target to the host and import it into
-                Eclipse to view it using the LTTng Eclipse plugin already
+                Eclipse to view it using the LTTng Eclipse plug-in already
                 bundled in the Eclipse (Juno SR1 or greater).
             </para>
 
@@ -3172,7 +3172,7 @@
 
             <para>
                 You can access extensive help information on how to use
-                the LTTng plugin to search and analyze captured traces via
+                the LTTng plug-in to search and analyze captured traces via
                 the Eclipse help system:
                 <literallayout class='monospaced'>
      Help | Help Contents | LTTng Plug-in User Guide

documentation/profile-manual/profile-manual.xml

@@ -38,7 +38,7 @@
             </revision>
             <revision>
                 <revnumber>1.5</revnumber>
-                <date>Sometime in 2013</date>
+                <date>October 2013</date>
                 <revremark>Released with the Yocto Project 1.5 Release.</revremark>
             </revision>
         </revhistory>

documentation/ref-manual/faq.xml

@@ -682,9 +682,11 @@
             <para>
                 Yes - you can easily do this.
                 When you use BitBake to build an image, all the build output
-                goes into the directory created when you source the
+                goes into the directory created when you run the 
+                build environment setup script (i.e.
                 <link linkend='structure-core-script'><filename>&OE_INIT_FILE;</filename></link>
-                setup script.
+                or
+                <link linkend='structure-memres-core-script'><filename>oe-init-build-env-memres</filename></link>).
                 By default, this <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>
                 is named <filename>build</filename> but can be named
                 anything you want.

documentation/ref-manual/introduction.xml

@@ -133,17 +133,26 @@
                 </para>
             </note>
             <itemizedlist>
-                <listitem><para>Ubuntu 10.04</para></listitem>
-                <listitem><para>Ubuntu 11.10</para></listitem>
+<!--                <listitem><para>Ubuntu 10.04</para></listitem>
+                <listitem><para>Ubuntu 11.10</para></listitem> -->
                 <listitem><para>Ubuntu 12.04 (LTS)</para></listitem>
                 <listitem><para>Ubuntu 12.10</para></listitem>
                 <listitem><para>Ubuntu 13.04</para></listitem>
-                <listitem><para>Fedora release 17 (Beefy Miracle)</para></listitem>
+<!--                <listitem><para>Fedora 16 (Verne)</para></listitem>
+                <listitem><para>Fedora 17 (Spherical)</para></listitem> -->
                 <listitem><para>Fedora release 18 (Spherical Cow)</para></listitem>
-                <listitem><para>CentOS release 6.3 (Final)</para></listitem>
-                <listitem><para>CentOS release 6.4 (Final)</para></listitem>
-                <listitem><para>Debian GNU/Linux 6.0 (Squeeze)</para></listitem>
+                <listitem><para>Fedora release 19 (Schrödinger's Cat)</para></listitem>
+<!--                <listitem><para>CentOS release 5.6 (Final)</para></listitem>
+                <listitem><para>CentOS release 5.7 (Final)</para></listitem>
+                <listitem><para>CentOS release 5.8 (Final)</para></listitem>
+                <listitem><para>CentOS release 6.3 (Final)</para></listitem> -->
+                <listitem><para>CentOS release 6.4</para></listitem>
+<!--                <listitem><para>Debian GNU/Linux 6.0 (Squeeze)</para></listitem> -->
+                <listitem><para>Debian GNU/Linux 6.0.7 (Squeeze)</para></listitem>
                 <listitem><para>Debian GNU/Linux 7.0 (Wheezy)</para></listitem>
+                <listitem><para>Debian GNU/Linux 7.1 (Wheezy)</para></listitem>
+<!--                <listitem><para>openSUSE 11.4</para></listitem>
+                <listitem><para>openSUSE 12.1</para></listitem> -->
                 <listitem><para>openSUSE 12.2</para></listitem>
                 <listitem><para>openSUSE 12.3</para></listitem>
             </itemizedlist>

documentation/ref-manual/migration.xml

@@ -230,6 +230,30 @@
             </para>
         </section>
     </section>
+
+    <section id='1.3-linux-kernel-naming'>
+        <title>Linux Kernel Naming</title>
+
+        <para>
+            The naming scheme for kernel output binaries has been changed to
+            now include
+            <link linkend='var-PE'><filename>PE</filename></link> as part of the
+            filename:
+            <literallayout class='monospaced'>
+     KERNEL_IMAGE_BASE_NAME ?= "${KERNEL_IMAGETYPE}-${PE}-${PV}-${PR}-${MACHINE}-${DATETIME}"
+            </literallayout>
+        </para>
+
+        <para>
+            Because the <filename>PE</filename> variable is not set by default,
+            these binary files could result with names that include two dash
+            characters.
+            Here is an example:
+            <literallayout class='monospaced'>
+     bzImage--3.10.9+git0+cd502a8814_7144bcc4b8-r0-qemux86-64-20130830085431.bin
+            </literallayout>
+        </para>
+    </section>
 </section>
 
 <section id='moving-to-the-yocto-project-1.4-release'>
@@ -556,6 +580,509 @@
         </para>
     </section>
 </section>
+
+<section id='moving-to-the-yocto-project-1.5-release'>
+    <title>Moving to the Yocto Project 1.5 Release</title>
+
+    <para>
+        This section provides migration information for moving to the
+        Yocto Project 1.5 Release from the prior release.
+    </para>
+
+    <section id='migration-1.5-host-dependency-changes'>
+        <title>Host Dependency Changes</title>
+
+        <para>
+            The OpenEmbedded build system now has some additional requirements
+            on the host system:
+            <itemizedlist>
+                <listitem><para>Python 2.7.3+</para></listitem>
+                <listitem><para>Tar 1.24+</para></listitem>
+                <listitem><para>Git 1.7.5+</para></listitem>
+                <listitem><para>Patched version of Make if you are using
+                    3.82.
+                    Most distributions that provide Make 3.82 use the patched
+                    version.</para></listitem>
+            </itemizedlist>
+            If the Linux distribution you are using on your build host
+            does not provide packages for these, you can install and use
+            the Buildtools tarball, which provides an SDK-like environment
+            containing them.
+        </para>
+
+        <para>
+            For more information on this requirement, see the
+            "<link linkend='required-git-tar-and-python-versions'>Required Git, tar, and Python Versions</link>"
+            section.
+        </para>
+    </section>
+
+    <section id='migration-1.5-atom-pc-bsp'>
+        <title><filename>atom-pc</filename> Board Support Package (BSP)</title>
+
+        <para>
+            The <filename>atom-pc</filename> hardware reference BSP has been
+            replaced by a <filename>genericx86</filename> BSP.
+            This BSP is not necessarily guaranteed to work on all x86
+            hardware, but it will run on a wider range of systems than the
+            <filename>atom-pc</filename> did.
+            <note>
+                Additionally, a <filename>genericx86-64</filename> BSP has been
+                added for 64-bit systems.
+            </note>
+        </para>
+    </section>
+
+    <section id='migration-1.5-bitbake'>
+        <title>BitBake</title>
+
+        <para>
+            The following changes have been made that relate to BitBake:
+            <itemizedlist>
+                <listitem><para>
+                    BitBake now supports a <filename>_remove</filename>
+                    operator.
+                    The addition of this operator means you will have to
+                    rename any items in recipe space (functions, variables)
+                    whose names currently contain
+                    <filename>_remove_</filename> or end with
+                    <filename>_remove</filename> to avoid unexpected behavior.
+                    </para></listitem>
+                <listitem><para>
+                    BitBake's global method pool has been removed.
+                    This method is not particularly useful and led to clashes
+                    between recipes containing functions that had the
+                    same name.</para></listitem>
+                <listitem><para>
+                    The "none" server backend has been removed.
+                    The "process" server backend has been serving well as the
+                    default for a long time now.</para></listitem>
+                <listitem><para>
+                    The <filename>bitbake-runtask</filename> script has been
+                    removed.</para></listitem>
+                <listitem><para>
+                    <filename>${</filename><link linkend='var-P'><filename>P</filename></link><filename>}</filename>
+                    and
+                    <filename>${</filename><link linkend='var-PF'><filename>PF</filename></link><filename>}</filename>
+                    are no longer added to
+                    <link linkend='var-PROVIDES'><filename>PROVIDES</filename></link>
+                    by default in <filename>bitbake.conf</filename>.
+                    These version-specific <filename>PROVIDES</filename>
+                    items were seldom used.
+                    Attempting to use them could result in two versions being
+                    built simultaneously rather than just one version due to
+                    the way BitBake resolves dependencies.</para></listitem>
+            </itemizedlist>
+        </para>
+    </section>
+
+    <section id='migration-1.5-qa-warnings'>
+        <title>QA Warnings</title>
+
+        <para>
+            The following changes have been made to the package QA checks:
+            <itemizedlist>
+                <listitem><para>
+                    If you have customized
+                    <link linkend='var-ERROR_QA'><filename>ERROR_QA</filename></link>
+                    or <link linkend='var-WARN_QA'><filename>WARN_QA</filename></link>
+                    values in your configuration, check that they contain all of
+                    the issues that you wish to be reported.
+                    Previous Yocto Project versions contained a bug that meant
+                    that any item not mentioned in <filename>ERROR_QA</filename>
+                    or <filename>WARN_QA</filename> would be treated as a
+                    warning.
+                    Consequently, several important items were not already in
+                    the default value of <filename>WARN_QA</filename>.
+                    All of the possible QA checks are now documented in the
+                    "<link linkend='ref-classes-insane'><filename>insane.bbclass</filename></link>"
+                    section.</para></listitem>
+                <listitem><para>
+                    An additional QA check has been added to check if
+                    <filename>/usr/share/info/dir</filename> is being installed.
+                    Your recipe should delete this file within
+                    <filename>do_install</filename> if "make install" is
+                    installing it.</para></listitem>
+                <listitem><para>
+                    If you are using the buildhistory class, the check for the
+                    package version going backwards is now controlled using a
+                    standard QA check.
+                    Thus, if you have customized your
+                    <filename>ERROR_QA</filename> or
+                    <filename>WARN_QA</filename> values and still wish to have
+                    this check performed, you should add
+                    "version-going-backwards" to your value for one or the
+                    other variables depending on how you wish it to be handled.
+                    See the documented QA checks in the
+                    "<link linkend='ref-classes-insane'><filename>insane.bbclass</filename></link>"
+                    section.
+                    </para></listitem>
+            </itemizedlist>
+        </para>
+    </section>
+
+    <section id='migration-1.5-directory-layout-changes'>
+        <title>Directory Layout Changes</title>
+
+        <para>
+            The following directory changes exist:
+            <itemizedlist>
+                <listitem><para>
+                    Output SDK installer files are now named to include the
+                    image name and tuning architecture through the
+                    <link linkend='var-SDK_NAME'><filename>SDK_NAME</filename></link>
+                    variable.</para></listitem>
+                <listitem><para>
+                    Images and related files are now installed into a directory
+                    that is specific to the machine, instead of a parent
+                    directory containing output files for multiple machines.
+                    The
+                    <link linkend='var-DEPLOY_DIR_IMAGE'><filename>DEPLOY_DIR_IMAGE</filename></link>
+                    variable continues to point to the directory containing
+                    images for the current
+                    <link linkend='var-MACHINE'><filename>MACHINE</filename></link>
+                    and should be used anywhere there is a need to refer to
+                    this directory.
+                    The <filename>runqemu</filename> script now uses this
+                    variable to find images and kernel binaries and will use
+                    BitBake to determine the directory.
+                    Alternatively, you can set the
+                    <filename>DEPLOY_DIR_IMAGE</filename> variable in the
+                    external environment.</para></listitem>
+                <listitem><para>
+                    When buildhistory is enabled, its output is now written
+                    under the
+                    <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>
+                    rather than
+                    <link linkend='var-TMPDIR'><filename>TMPDIR</filename></link>.
+                    Doing so makes it a easier to delete
+                    <filename>TMPDIR</filename> and preserve the build history.
+                    Additionally, data for produced SDKs is now split by
+                    <link linkend='var-IMAGE_NAME'><filename>IMAGE_NAME</filename></link>.
+                    </para></listitem>
+                <listitem><para>
+                    The <filename>pkgdata</filename> directory produced as
+                    part of the packaging process has been collapsed into a
+                    single machine-specific directory.
+                    This directory is located under
+                    <filename>sysroots</filename> and uses a machine-specific
+                    name (i.e.
+                    <filename>tmp/sysroots/&lt;machine&gt;/pkgdata</filename>).
+                    </para></listitem>
+            </itemizedlist>
+        </para>
+    </section>
+
+    <section id='migration-1.5-shortened-git-srcrev-values'>
+        <title>Shortened Git <filename>SRCREV</filename> Values</title>
+
+        <para>
+            BitBake will now shorten revisions from Git repositories from the
+            normal 40 characters down to 10 characters within
+            <link linkend='var-SRCPV'><filename>SRCPV</filename></link>
+            for improved usability in path and file names.
+            This change should be safe within contexts where these revisions
+            are used because the chances of spatially close collisions
+            is very low.
+            Distant collisions are not a major issue in the way
+            the values are used.
+        </para>
+    </section>
+
+    <section id='migration-1.5-image-features'>
+        <title><filename>IMAGE_FEATURES</filename></title>
+
+        <para>
+            The following changes have been made that relate to
+            <link linkend='var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></link>:
+            <itemizedlist>
+                <listitem><para>
+                    The value of
+                    <link linkend='var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></link>
+                    is now validated to ensure invalid feature items are not
+                    added.
+                    Some users mistakenly add package names to this variable
+                    instead of using
+                    <link linkend='var-IMAGE_INSTALL'><filename>IMAGE_INSTALL</filename></link>
+                    in order to have the package added to the image, which does
+                    not work.
+                    This change is intended to catch those kinds of situations.
+                    Valid <filename>IMAGE_FEATURES</filename> are drawn from
+                    <link linkend='var-PACKAGE_GROUP'><filename>PACKAGE_GROUP</filename></link>
+                    definitions,
+                    <link linkend='var-COMPLEMENTARY_GLOB'><filename>COMPLEMENTARY_GLOB</filename></link>
+                    and a new "validitems" varflag on
+                    <filename>IMAGE_FEATURES</filename>.
+                    The "validitems" varflag change allows additional features
+                    to be added if they are not provided using the previous
+                    two mechanisms.
+                    </para></listitem>
+                <listitem><para>
+                    The previously deprecated "apps-console-core"
+                    <filename>IMAGE_FEATURES</filename> item is no longer
+                    supported.
+                    Add "splash" to <filename>IMAGE_FEATURES</filename> if you
+                    wish to have the splash screen enabled, since this is
+                    all that apps-console-core was doing.</para></listitem>
+            </itemizedlist>
+        </para>
+    </section>
+
+    <section id='migration-1.5-run'>
+        <title><filename>/run</filename></title>
+
+        <para>
+            The <filename>/run</filename> directory from the Filesystem
+            Hierarchy Standard 3.0 has been introduced.
+            You can find some of the implications for this change
+            <ulink url='http://cgit.openembedded.org/openembedded-core/commit/?id=0e326280a15b0f2c4ef2ef4ec441f63f55b75873'>here</ulink>.
+            The change also means that recipes that install files to
+            <filename>/var/run</filename> must be changed.
+            You can find a guide on how to make these changes
+            <ulink url='http://permalink.gmane.org/gmane.comp.handhelds.openembedded/58530'>here</ulink>.
+        </para>
+    </section>
+
+    <section id='migration-1.5-removal-of-package-manager-database-within-image-recipes'>
+        <title>Removal of Package Manager Database Within Image Recipes</title>
+
+        <para>
+            The image <filename>core-image-minimal</filename> no longer adds
+            <filename>remove_packaging_data_files</filename> to
+            <link linkend='var-ROOTFS_POSTPROCESS_COMMAND'><filename>ROOTFS_POSTPROCESS_COMMAND</filename></link>.
+            This addition is now handled automatically when "package-management"
+            is not in
+            <link linkend='var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></link>.
+            If you have custom image recipes that make this addition,
+            you should remove the lines, as they are not needed and might
+            interfere with correct operation of postinstall scripts.
+        </para>
+    </section>
+
+    <section id='migration-1.5-images-now-rebuild-only-on-changes-instead-of-every-time'>
+        <title>Images Now Rebuild Only on Changes Instead of Every Time</title>
+
+        <para>
+            The <filename>do_rootfs</filename> and other related image
+            construction tasks are no longer marked as "nostamp".
+            Consequently, they will only be re-executed when their inputs have
+            changed.
+            Previous versions of the OpenEmbedded build system always rebuilt
+            the image when requested rather when necessary.
+        </para>
+    </section>
+
+    <section id='migration-1.5-task-recipes'>
+        <title>Task Recipes</title>
+
+        <para>
+            The previously deprecated <filename>task.bbclass</filename> has
+            now been dropped.
+            For recipes that previously inherited from this task, you should
+            rename them from <filename>task-*</filename> to
+            <filename>packagegroup-*</filename> and inherit packagegroup
+            instead.
+        </para>
+
+        <para>
+            For more information, see the
+            "<link linkend='ref-classes-packagegroup'>Package Groups - <filename>packagegroup.bbclass</filename></link>"
+            section.
+        </para>
+    </section>
+
+    <section id='migration-1.5-busybox'>
+        <title>BusyBox</title>
+
+        <para>
+            By default, we now split BusyBox into two binaries:
+            one that is suid root for those components that need it, and
+            another for the rest of the components.
+            Splitting BusyBox allows for optimization that eliminates the
+            <filename>tinylogin</filename> recipe as recommended by upstream.
+            You can disable this split by setting
+            <link linkend='var-BUSYBOX_SPLIT_SUID'><filename>BUSYBOX_SPLIT_SUID</filename></link>
+            to "0".
+        </para>
+    </section>
+
+    <section id='migration-1.5-automated-image-testing'>
+        <title>Automated Image Testing</title>
+
+        <para>
+            A new automated image testing framework has been added
+            through the
+            <link linkend='ref-classes-testimage'><filename>testimage*.bbclass</filename></link>
+            class.
+            This framework replaces the older
+            <filename>imagetest-qemu</filename> framework.
+        </para>
+
+        <para>
+            You can learn more about performing automated image tests in the
+            "<ulink url='&YOCTO_DOCS_DEV_URL;#performing-automated-runtime-testing'>Performing Automated Runtime Testing</ulink>"
+            section.
+        </para>
+    </section>
+
+    <section id='migration-1.5-build-history'>
+        <title>Build History</title>
+
+        <para>
+            Following are changes to Build History:
+            <itemizedlist>
+                <listitem><para>
+                     Installed package sizes:
+                     <filename>installed-package-sizes.txt</filename> for an
+                     image now records the size of the files installed by each
+                     package instead of the size of each compressed package
+                     archive file.</para></listitem>
+                <listitem><para>
+                    The dependency graphs (<filename>depends*.dot</filename>)
+                    now use the actual package names instead of replacing
+                    dashes, dots and plus signs with underscores.
+                    </para></listitem>
+                <listitem><para>
+                    The <filename>buildhistory-diff</filename> and
+                    <filename>buildhistory-collect-srcrevs</filename>
+                    utilities have improved command-line handling.
+                    Use the <filename>&dash;&dash;help</filename> option for
+                    each utility for more information on the new syntax.
+                    </para></listitem>
+            </itemizedlist>
+            For more information on Build History, see the
+            "<link linkend='maintaining-build-output-quality'>Maintaining Build Output Quality</link>"
+            section.
+        </para>
+    </section>
+
+    <section id='migration-1.5-udev'>
+        <title><filename>udev</filename></title>
+
+        <para>
+            Following are changes to <filename>udev</filename>:
+            <itemizedlist>
+                <listitem><para>
+                    <filename>udev</filename> no longer brings in
+                    <filename>udev-extraconf</filename> automatically
+                    through
+                    <link linkend='var-RRECOMMENDS'><filename>RRECOMMENDS</filename></link>,
+                    since this was originally intended to be optional.
+                    If you need the extra rules, then add
+                    <filename>udev-extraconf</filename> to your image.
+                    </para></listitem>
+                <listitem><para>
+                    <filename>udev</filename> no longer brings in
+                    <filename>pciutils-ids</filename> or
+                    <filename>usbutils-ids</filename> through
+                    <filename>RRECOMMENDS</filename>.
+                    These are not needed by <filename>udev</filename> itself
+                    and removing them saves around 350KB.
+                    </para></listitem>
+            </itemizedlist>
+        </para>
+    </section>
+
+    <section id='removed-renamed-recipes'>
+        <title>Removed and Renamed Recipes</title>
+
+        <itemizedlist>
+            <listitem><para>
+                The <filename>linux-yocto</filename> 3.2 kernel has been
+                removed.</para></listitem>
+            <listitem><para>
+                <filename>libtool-nativesdk</filename> has been renamed to
+                <filename>nativesdk-libtool</filename>.</para></listitem>
+            <listitem><para>
+                <filename>tinylogin</filename> has been removed.
+                It has been replaced by a suid portion of Busybox.
+                See the
+                "<link linkend='migration-1.5-busybox'>BusyBox</link>" section
+                for more information.</para></listitem>
+            <listitem><para>
+                <filename>external-python-tarball</filename> has been renamed
+                to <filename>buildtools-tarball</filename>.
+                </para></listitem>
+            <listitem><para>
+                <filename>web-webkit</filename> has been removed.
+                It has been functionally replaced by
+                <filename>midori</filename>.</para></listitem>
+            <listitem><para>
+                <filename>imake</filename> has been removed.
+                It is no longer needed by any other recipe.
+                </para></listitem>
+            <listitem><para>
+                <filename>transfig-native</filename> has been removed.
+                It is no longer needed by any other recipe.
+                </para></listitem>
+            <listitem><para>
+                <filename>anjuta-remote-run</filename> has been removed.
+                Anjuta IDE integration has not been officially supported for
+                several releases.</para></listitem>
+        </itemizedlist>
+    </section>
+
+    <section id='migration-1.5-other-changes'>
+        <title>Other Changes</title>
+
+        <para>
+            Following is a list of short entries describing other changes:
+            <itemizedlist>
+                <listitem><para>
+                    <filename>run-postinsts</filename>: Make this generic.
+                    </para></listitem>
+                <listitem><para>
+                    <filename>base-files</filename>: Remove the unnecessary
+                    <filename>/media/xxx</filename> directories.
+                    </para></listitem>
+                <listitem><para>
+                    <filename>alsa-state</filename>: Provide an empty
+                    <filename>asound.conf</filename> by default.
+                    </para></listitem>
+                <listitem><para>
+                    <filename>classes/image</filename>: Ensure
+                    <link linkend='var-BAD_RECOMMENDATIONS'><filename>BAD_RECOMMENDATIONS</filename></link>
+                    supports pre-renamed package names.</para></listitem>
+                <listitem><para>
+                    <filename>classes/rootfs_rpm</filename>: Implement
+                    <link linkend='var-BAD_RECOMMENDATIONS'><filename>BAD_RECOMMENDATIONS</filename></link>
+                    for RPM.</para></listitem>
+                <listitem><para>
+                    <filename>systemd</filename>: Remove
+                    <filename>systemd_unitdir</filename> if
+                    <filename>systemd</filename> is not in
+                    <link linkend='var-DISTRO_FEATURES'><filename>DISTRO_FEATURES</filename></link>.
+                    </para></listitem>
+                <listitem><para>
+                    <filename>systemd</filename>: Remove
+                    <filename>init.d</filename> dir if
+                    <filename>systemd</filename> unit file is present and
+                    <filename>sysvinit</filename> is not a distro feature.
+                    </para></listitem>
+                <listitem><para>
+                    <filename>libpam</filename>: Deny all services for the
+                    <filename>OTHER</filename> entries.
+                    </para></listitem>
+                <listitem><para>
+                    <filename>image.bbclass</filename>: Move
+                    <filename>runtime_mapping_rename</filename> to avoid
+                    conflict with <filename>multilib</filename>.
+                    See
+                    <ulink url='https://bugzilla.yoctoproject.org/show_bug.cgi?id=4993'><filename>YOCTO #4993</filename></ulink>
+                    in Bugzilla for more information.
+                    </para></listitem>
+                <listitem><para>
+                    <filename>linux-dtb</filename>: Use kernel build system
+                    to generate the <filename>dtb</filename> files.
+                    </para></listitem>
+                <listitem><para>
+                    <filename>kern-tools</filename>: Switch from guilt to
+                    new <filename>kgit-s2q</filename> tool.
+                    </para></listitem>
+            </itemizedlist>
+        </para>
+    </section>
+</section>
 </chapter>
 <!--
 vim: expandtab tw=80 ts=4

documentation/ref-manual/ref-bitbake.xml

@@ -23,7 +23,7 @@
     <note>
         BitBake strives to be a generic "task" executor that is capable of handling complex dependency relationships.
         As such, it has no real knowledge of what the tasks being executed actually do.
-        BitBake just considers a list of tasks with dependencies and handles 
+        BitBake just considers a list of tasks with dependencies and handles
         <ulink url='&YOCTO_DOCS_DEV_URL;#metadata'>Metadata</ulink>
         consisting of variables in a certain format that get passed to the tasks.
     </note>
@@ -320,63 +320,74 @@
 
         <screen>
 $ bitbake --help
-Usage: bitbake [options] [package ...]
+Usage: bitbake [options] [recipename/target ...]
 
-Executes the specified task (default is 'build') for a given set of BitBake files.
-It expects that BBFILES is defined, which is a space separated list of files to
-be executed.  BBFILES does support wildcards.
-Default BBFILES are the .bb files in the current directory.
+    Executes the specified task (default is 'build') for a given set of target recipes (.bb files).
+    It is assumed there is a conf/bblayers.conf available in cwd or in BBPATH which
+    will provide the layer, BBFILES and other configuration information.
 
 Options:
   --version             show program's version number and exit
   -h, --help            show this help message and exit
   -b BUILDFILE, --buildfile=BUILDFILE
-                        execute the task against this .bb file, rather than a
-                        package from BBFILES. Does not handle any
-                        dependencies.
-  -k, --continue        continue as much as possible after an error. While the
-                        target that failed, and those that depend on it,
-                        cannot be remade, the other dependencies of these
-                        targets can be processed all the same.
-  -a, --tryaltconfigs   continue with builds by trying to use alternative
+                        Execute tasks from a specific .bb recipe directly.
+                        WARNING: Does not handle any dependencies from other
+                        recipes.
+  -k, --continue        Continue as much as possible after an error. While the
+                        target that failed and anything depending on it cannot
+                        be built, as much as possible will be built before
+                        stopping.
+  -a, --tryaltconfigs   Continue with builds by trying to use alternative
                         providers where possible.
-  -f, --force           force run of specified cmd, regardless of stamp status
-  -c CMD, --cmd=CMD     Specify task to execute. Note that this only executes
-                        the specified task for the providee and the packages
-                        it depends on, i.e. 'compile' does not implicitly call
-                        stage for the dependencies (IOW: use only if you know
-                        what you are doing). Depending on the base.bbclass a
-                        listtasks tasks is defined and will show available
-                        tasks
+  -f, --force           Force the specified targets/task to run (invalidating
+                        any existing stamp file).
+  -c CMD, --cmd=CMD     Specify the task to execute. The exact options
+                        available depend on the metadata. Some examples might
+                        be 'compile' or 'populate_sysroot' or 'listtasks' may
+                        give a list of the tasks available.
+  -C INVALIDATE_STAMP, --clear-stamp=INVALIDATE_STAMP
+                        Invalidate the stamp for the specified task such as
+                        'compile' and then run the default task for the
+                        specified target(s).
   -r PREFILE, --read=PREFILE
-                        read the specified file before bitbake.conf
+                        Read the specified file before bitbake.conf.
   -R POSTFILE, --postread=POSTFILE
-                        read the specified file after bitbake.conf
-  -v, --verbose         output more chit-chat to the terminal
+                        Read the specified file after bitbake.conf.
+  -v, --verbose         Output more log message data to the terminal.
   -D, --debug           Increase the debug level. You can specify this more
                         than once.
-  -n, --dry-run         don't execute, just go through the motions
+  -n, --dry-run         Don't execute, just go through the motions.
   -S, --dump-signatures
-                        don't execute, just dump out the signature
-                        construction information
-  -p, --parse-only      quit after parsing the BB files (developers only)
-  -s, --show-versions   show current and preferred versions of all packages
-  -e, --environment     show the global or per-package environment (this is
-                        what used to be bbread)
-  -g, --graphviz        emit the dependency trees of the specified packages in
-                        the dot syntax
+                        Don't execute, just dump out the signature
+                        construction information.
+  -p, --parse-only      Quit after parsing the BB recipes.
+  -s, --show-versions   Show current and preferred versions of all recipes.
+  -e, --environment     Show the global or per-package environment complete
+                        with information about where variables were
+                        set/changed.
+  -g, --graphviz        Save dependency tree information for the specified
+                        targets in the dot syntax.
   -I EXTRA_ASSUME_PROVIDED, --ignore-deps=EXTRA_ASSUME_PROVIDED
                         Assume these dependencies don't exist and are already
                         provided (equivalent to ASSUME_PROVIDED). Useful to
                         make dependency graphs more appealing
   -l DEBUG_DOMAINS, --log-domains=DEBUG_DOMAINS
                         Show debug logging for the specified logging domains
-  -P, --profile         profile the command and print a report
-  -u UI, --ui=UI        userinterface to use
+  -P, --profile         Profile the command and save reports.
+  -u UI, --ui=UI        The user interface to use (e.g. knotty, hob, depexp).
   -t SERVERTYPE, --servertype=SERVERTYPE
-                        Choose which server to use, none, process or xmlrpc
+                        Choose which server to use, process or xmlrpc.
   --revisions-changed   Set the exit code depending on whether upstream
-                        floating revisions have changed or not
+                        floating revisions have changed or not.
+  --server-only         Run bitbake without a UI, only starting a server
+                        (cooker) process.
+  -B BIND, --bind=BIND  The name/address for the bitbake server to bind to.
+  --no-setscene         Do not run any setscene tasks. sstate will be ignored
+                        and everything needed, built.
+  --remote-server=REMOTE_SERVER
+                        Connect to the specified server.
+  -m, --kill-server     Terminate the remote server.
+  --observe-only        Connect to a server as an observing-only client.
         </screen>
     </section>
 

documentation/ref-manual/ref-classes.xml

@@ -86,9 +86,9 @@
                 the <filename><link linkend='var-EXTRA_OEMAKE'>EXTRA_OEMAKE</link></filename> variable.
                 </para></listitem>
             <listitem><para><filename>do_install</filename> &dash; Runs <filename>make install</filename>
-                and passes a destination directory option, which takes its value
-                from the standard
-                <filename><link linkend='var-DESTDIR'>DESTDIR</link></filename> variable.
+                and passes in
+                <filename>${</filename><link linkend='var-D'><filename>D</filename></link><filename>}</filename>
+                as <filename>DESTDIR</filename>.
                 </para></listitem>
         </itemizedlist>
     </para>
@@ -341,9 +341,9 @@
         on the development host that can be used by Smart, you can
         install packages from the feed while you are running the image
         on the target (i.e. runtime installation of packages).
-        For information on how to set up this repository, see the
-        "<ulink url='&YOCTO_DOCS_DEV_URL;#setting-up-runtime-package-management'>Setting Up Runtime Package Management</ulink>"
-         in the Yocto Project Development Manual.
+        For more information, see the
+        "<ulink url='&YOCTO_DOCS_DEV_URL;#using-runtime-package-management'>Using Runtime Package Management</ulink>"
+         section in the Yocto Project Development Manual.
     </para>
 
     <para>
@@ -797,6 +797,21 @@
             <listitem><para><emphasis><filename>perms:</filename></emphasis>
                 Currently, this check is unused but reserved.
                 </para></listitem>
+            <listitem><para><emphasis><filename>version-going-backwards:</filename></emphasis>
+                If Build History is enabled, reports when a package
+                being written out has a lower version than the previously
+                written package under the same name.
+                If you are placing output packages into a feed and
+                upgrading packages on a target system using that feed, the
+                version of a package going backwards can result in the target
+                system not correctly upgrading to the "new" version of the
+                package.
+                <note>
+                    If you are not using runtime package management on your
+                    target system, then you do not need to worry about
+                    this situation.
+                </note>
+                </para></listitem>
         </itemizedlist>
     </para>
 </section>
@@ -935,6 +950,35 @@
     </para>
 </section>
 
+<section id='ref-classes-testimage'>
+    <title><filename>testimage.bbclass</filename></title>
+
+    <para>
+        You can use this class to enable running a series of automated tests
+        for images.
+        The class handles loading the tests and starting the image.
+        <note>
+            Currently, there is only support for running these tests
+            under QEMU.
+        </note>
+    </para>
+
+    <para>
+        To use the class, you need to perform steps to set up the
+        environment.
+        The tests are commands that run on the target system over
+        <filename>ssh</filename>.
+        they are written in Python and make use of the
+        <filename>unittest</filename> module.
+    </para>
+
+    <para>
+        For information on how to enable, run, and create new tests, see the
+        "<ulink url='&YOCTO_DOCS_DEV_URL;#performing-automated-runtime-testing'>Performing Automated Runtime Testing</ulink>"
+        section.
+    </para>
+</section>
+
 <section id='ref-classes-others'>
     <title>Other Classes</title>
 
@@ -952,6 +996,7 @@
         allarch.bbclass
         archive*.bbclass
         binconfig.bbclass
+        bin_package.bbclass
         blacklist.bbclass
         bootimg.bbclass
         boot-directdisk.bbclass
@@ -960,6 +1005,7 @@
         buildstats.bbclass
         ccache.bbclass
         chrpath.bbclass
+        clutter.bbclass
         cmake.bbclass
         cml1.bbclass
         copyleft_compliance.bbclass
@@ -969,7 +1015,9 @@
         crosssdk.bbclass
         deploy.bbclass
         distrodata.bbclass
+        distro_features_check.bbclass
         dummy.bbclass
+        extrausers.bbclass
         fontcache.bbclass
         gconf.bbclass
         gettext.bbclass
@@ -988,12 +1036,11 @@
         image-mklibs.bbclass
         image-prelink.bbclass
         image-swab.bbclass
-        imagetest-dummy.bbclass
-        imagetest-qemu.bbclass
         image_types.bbclass
         image_types_uboot.bbclass
         insserv.bbclass
         kernel-arch.bbclass
+        kernel-module-split.bbclass
         kernel-yocto.bbclass
         lib_package.bbclass
         linux-kernel-base.bbclass
@@ -1034,6 +1081,7 @@
         sip.bbclass
         siteconfig.bbclass
         sourcepkg.bbclass
+        spdx.bbclass
         sstate.bbclass
         staging.bbclass
         syslinux.bbclass
@@ -1042,8 +1090,11 @@
         tinderclient.bbclass
         toolchain-scripts.bbclass
         typecheck.bbclass
+        uboot-config.bbclass
         utility-tasks.bbclass
         utils.bbclass
+        vala.bbclass
+        waf.bbclass
 -->
 
 

documentation/ref-manual/ref-features.xml

@@ -194,7 +194,7 @@
                 <listitem><para><emphasis>doc-pkgs:</emphasis> Installs documentation packages for all packages
                     installed in a given image.</para></listitem>
                 <listitem><para><emphasis>nfs-server:</emphasis> Installs an NFS server.</para></listitem>
-                <listitem><para><emphasis>read-only-fsroot:</emphasis> Creates
+                <listitem><para><emphasis>read-only-rootfs:</emphasis> Creates
                     an image whose root filesystem is read-only.
                     See the
                     "<ulink url='&YOCTO_DOCS_DEV_URL;#creating-a-read-only-root-filesystem'>Creating a Read-Only Root Filesystem</ulink>"

documentation/ref-manual/ref-manual.xml

@@ -69,7 +69,7 @@
             </revision>
             <revision>
                 <revnumber>1.5</revnumber>
-                <date>Sometime in 2013</date>
+                <date>October 2013</date>
                 <revremark>Released with the Yocto Project 1.5 Release.</revremark>
             </revision>
         </revhistory>
@@ -99,6 +99,8 @@
 
     <xi:include href="usingpoky.xml"/>
 
+    <xi:include href="closer-look.xml"/>
+
     <xi:include href="technical-details.xml"/>
 
     <xi:include href="migration.xml"/>

documentation/ref-manual/ref-structure.xml

@@ -74,7 +74,11 @@
             the source tree is combined with the output.
             The <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>
             is created initially when you <filename>source</filename>
-            the OpenEmbedded build environment setup script <filename>&OE_INIT_FILE;</filename>.
+            the OpenEmbedded build environment setup script 
+            (i.e.
+            <link linkend='structure-core-script'><filename>&OE_INIT_FILE;</filename></link>
+            or
+            <link linkend='structure-memres-core-script'><filename>oe-init-build-env-memres</filename></link>).
         </para>
 
         <para>
@@ -85,8 +89,10 @@
             the setup script.
             For information on separating output from your local
             Source Directory files, see the
-            "<link linkend='structure-core-script'><filename>&OE_INIT_FILE;</filename></link>"
-            section.
+            "<link linkend='structure-core-script'><filename>&OE_INIT_FILE;</filename></link>
+            and
+            "<link linkend='structure-memres-core-script'><filename>oe-init-build-env-memres</filename></link>"
+            sections.
         </para>
     </section>
 
@@ -143,7 +149,7 @@
             which is a Yocto Project build user interface.
             For more information on the Hob, see the
             <ulink url='&YOCTO_HOME_URL;/tools-resources/projects/hob'>Hob Project</ulink>
-            webpage.
+            web page.
         </para>
     </section>
 
@@ -162,7 +168,9 @@
             This directory contains various integration scripts that implement
             extra functionality in the Yocto Project environment (e.g. QEMU scripts).
             The <link linkend="structure-core-script"><filename>&OE_INIT_FILE;</filename></link>
-            script appends this directory to the shell's
+            and
+            <link linkend='structure-memres-core-script'><filename>oe-init-build-env-memres</filename></link>
+            scripts append this directory to the shell's
             <filename>PATH</filename> environment variable.
         </para>
 
@@ -177,11 +185,19 @@
         <title><filename>&OE_INIT_FILE;</filename></title>
 
         <para>
-            This script sets up the OpenEmbedded build environment.
+            This script is one of two scripts that set up the OpenEmbedded build 
+            environment.
+            For information on the other script, see the 
+            "<link linkend='structure-memres-core-script'><filename>oe-init-build-env-memres</filename></link>"
+            section.
+        </para>
+
+        <para>
             Running this script with the <filename>source</filename> command in
             a shell makes changes to <filename>PATH</filename> and sets other
             core BitBake variables based on the current working directory.
-            You need to run this script before running BitBake commands.
+            You need to run an environment setup script before running BitBake 
+            commands.
             The script uses other scripts within the
             <filename>scripts</filename> directory to do the bulk of the work.
         </para>
@@ -191,7 +207,7 @@
             <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>
             argument creates the <filename>build</filename> directory.
             If you provide a Build Directory argument when you
-            <filename>source</filename> the script, you direct OpenEmbedded
+            <filename>source</filename> the script, you direct the OpenEmbedded
             build system to create a Build Directory of your choice.
             For example, the following command creates a Build Directory named
             <filename>mybuilds</filename> that is outside of the
@@ -211,6 +227,81 @@
         </para>
     </section>
 
+    <section id='structure-memres-core-script'>
+        <title><filename>oe-init-build-env-memres</filename></title>
+
+        <para>
+            This script is one of two scripts that set up the OpenEmbedded build 
+            environment.
+            Setting up the environment with this script uses a 
+            memory-resident BitBake.
+            For information on the other setup script, see the 
+            "<link linkend='structure-core-script'><filename>&OE_INIT_FILE;</filename></link>"
+            section.
+        </para>
+
+        <para>
+            Memory-resident BitBake resides in memory until you specifically
+            remove it using the following BitBake command:
+            <literallayout class='monospaced'>
+     $ bitbake -m
+            </literallayout>
+        </para>
+
+        <para>
+            Running this script with the <filename>source</filename> command in
+            a shell makes changes to <filename>PATH</filename> and sets other
+            core BitBake variables based on the current working directory.
+            One of these variables is the 
+            <link linkend='var-BBSERVER'><filename>BBSERVER</filename></link>
+            variable, which allows the OpenEmbedded build system to locate 
+            the server that is running BitBake.
+        </para>
+
+        <para>
+            You need to run an environment setup script before running BitBake 
+            commands.
+            Following is the script syntax:
+            <literallayout class='monospaced'>
+     $ source oe-init-build-env-memres &lt;port_number&gt; &lt;build_dir&gt;
+            </literallayout>
+            The script uses other scripts within the
+            <filename>scripts</filename> directory to do the bulk of the work.
+        </para>
+
+        <para>
+            If you do not provide a port number with the script, the default
+            port "12345" is used.
+        </para>
+
+        <para>
+            By default, running this script without a
+            <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>
+            argument creates the <filename>build</filename> directory.
+            If you provide a Build Directory argument when you
+            <filename>source</filename> the script, you direct the OpenEmbedded
+            build system to create a Build Directory of your choice.
+            For example, the following command uses the default port number
+            "12345" and creates a Build Directory named
+            <filename>mybuilds</filename> that is outside of the
+            <ulink url='&YOCTO_DOCS_DEV_URL;#source-directory'>Source Directory</ulink>:
+            <literallayout class='monospaced'>
+     $ source oe-init-build-env-memres ~/mybuilds
+            </literallayout>
+            <note>
+                The OpenEmbedded build system does not support file or 
+                directory names that contain spaces.
+                If you attempt to run the 
+                <filename>oe-init-build-env-memres</filename> script
+                from a Source Directory that contains spaces in either the 
+                filenames or directory names, the script returns an error 
+                indicating no such file or directory.
+                Be sure to use a Source Directory free of names containing 
+                spaces.
+            </note>
+        </para>
+    </section>
+
     <section id='structure-basic-top-level'>
         <title><filename>LICENSE, README, and README.hardware</filename></title>
 
@@ -243,47 +334,123 @@
         <title><filename>build/conf/local.conf</filename></title>
 
         <para>
-            This file contains all the local user configuration for your build environment.
-            If there is no <filename>local.conf</filename> present, it is created from
-            <filename>local.conf.sample</filename>.
-            The <filename>local.conf</filename> file contains documentation on the various configuration options.
-            Any variable set here overrides any variable set elsewhere within the environment unless
-            that variable is hard-coded within a file (e.g. by using '=' instead of '?=').
-            Some variables are hard-coded for various reasons but these variables are
-            relatively rare.
+            This configuration file contains all the local user configurations 
+            for your build environment.
+            The <filename>local.conf</filename> file contains documentation on 
+            the various configuration options.
+            Any variable set here overrides any variable set elsewhere within 
+            the environment unless that variable is hard-coded within a file 
+            (e.g. by using '=' instead of '?=').
+            Some variables are hard-coded for various reasons but these 
+            variables are relatively rare.
         </para>
 
         <para>
-            Edit this file to set the <filename><link linkend='var-MACHINE'>MACHINE</link></filename>
+            Edit this file to set the 
+            <filename><link linkend='var-MACHINE'>MACHINE</link></filename>
             for which you want to build, which package types you wish to use
             (<link linkend='var-PACKAGE_CLASSES'><filename>PACKAGE_CLASSES</filename></link>),
             the location from which you want to downloaded files
             (<filename><link linkend='var-DL_DIR'>DL_DIR</link></filename>),
             and how you want your host machine to use resources
-            (<link linkend='var-BB_NUMBER_THREADS'><filename>BB_NUMBER_THREADS</filename></link> and
+            (<link linkend='var-BB_NUMBER_THREADS'><filename>BB_NUMBER_THREADS</filename></link> 
+            and
             <link linkend='var-PARALLEL_MAKE'><filename>PARALLEL_MAKE</filename></link>).
         </para>
+
+        <para>
+            If <filename>local.conf</filename> is not present when you 
+            start the build, the OpenEmbedded build system creates it from 
+            <filename>local.conf.sample</filename> when
+            you <filename>source</filename> the top-level build environment 
+            setup script (i.e.
+            <link linkend='structure-core-script'><filename>&OE_INIT_FILE;</filename></link>
+            or 
+            <link linkend='structure-memres-core-script'><filename>oe-init-build-env-memres</filename></link>).
+        </para>
+
+        <para>
+            The source <filename>local.conf.sample</filename> file used 
+            depends on the <filename>$TEMPLATECONF</filename> script variable,
+            which defaults to <filename>/meta-yocto/conf</filename>
+            when you are building from the Yocto Project development 
+            environment and defaults to <filename>/meta/conf</filename> when 
+            you are building from the OpenEmbedded Core environment.
+            Because the script variable points to the source of the 
+            <filename>local.conf.sample</filename> file, this implies that 
+            you can configure your build environment from any layer by setting     
+            the variable in the top-level build environment setup script as 
+            follows:
+            <literallayout class='monospaced'>
+     TEMPLATECONF=&lt;your_layer&gt;/conf
+            </literallayout>
+            Once the build process gets the sample file, it uses 
+            <filename>sed</filename> to substitute final 
+            <filename>${</filename><link linkend='var-OEROOT'><filename>OEROOT</filename></link><filename>}</filename> 
+            values for all <filename>##OEROOT##</filename> values.
+            <note>
+                You can see how the <filename>TEMPLATECONF</filename> variable
+                is used by looking at the       
+                <filename>/scripts/oe-setup-builddir</filename> script in the 
+                <ulink url='&YOCTO_DOCS_DEV_URL;#source-directory'>Source Directory</ulink>.
+                You can find the Yocto Project version of the 
+                <filename>local.conf.sample</filename> file in the 
+                <filename>/meta-yocto/conf</filename> directory.                
+            </note>
+        </para>
     </section>
 
     <section id='structure-build-conf-bblayers.conf'>
         <title><filename>build/conf/bblayers.conf</filename></title>
 
         <para>
-            This file defines
+            This configuration file defines
             <ulink url='&YOCTO_DOCS_DEV_URL;#understanding-and-creating-layers'>layers</ulink>,
             which are directory trees, traversed (or walked) by BitBake.
-            If <filename>bblayers.conf</filename>
-            is not present, it is created from <filename>bblayers.conf.sample</filename> when
-            you <filename>source</filename> the environment setup script.
+            The <filename>bblayers.conf</filename> file uses the
+            <link linkend='var-BBLAYERS'><filename>BBLAYERS</filename></link> 
+            variable to list the layers BitBake tries to find, and uses the
+            <link linkend='var-BBLAYERS_NON_REMOVABLE'><filename>BBLAYERS_NON_REMOVABLE</filename></link>
+            variable to list layers that must not be removed.
         </para>
 
         <para>
-            The <filename>bblayers.conf</filename> file uses the
-            <link linkend='var-BBLAYERS'><filename>BBLAYERS</filename></link> variable to
-            list the layers BitBake tries to find.
-            The file uses the
-            <link linkend='var-BBLAYERS_NON_REMOVABLE'><filename>BBLAYERS_NON_REMOVABLE</filename></link>
-            variable to list layers that must not be removed.
+            If <filename>bblayers.conf</filename> is not present when you 
+            start the build, the OpenEmbedded build system creates it from 
+            <filename>bblayers.conf.sample</filename> when
+            you <filename>source</filename> the top-level build environment 
+            setup script (i.e.
+            <link linkend='structure-core-script'><filename>&OE_INIT_FILE;</filename></link>
+            or 
+            <link linkend='structure-memres-core-script'><filename>oe-init-build-env-memres</filename></link>).
+        </para>
+
+        <para>
+            The source <filename>bblayers.conf.sample</filename> file used 
+            depends on the <filename>$TEMPLATECONF</filename> script variable,
+            which defaults to <filename>/meta-yocto/conf</filename>
+            when you are building from the Yocto Project development 
+            environment and defaults to <filename>/meta/conf</filename> when 
+            you are building from the OpenEmbedded Core environment.
+            Because the script variable points to the source of the 
+            <filename>bblayers.conf.sample</filename> file, this implies that 
+            you can base your build from any layer by setting the variable in
+            the top-level build environment setup script as follows:
+            <literallayout class='monospaced'>
+     TEMPLATECONF=&lt;your_layer&gt;/conf
+            </literallayout>
+            Once the build process gets the sample file, it uses 
+            <filename>sed</filename> to substitute final 
+            <filename>${</filename><link linkend='var-OEROOT'><filename>OEROOT</filename></link><filename>}</filename> 
+            values for all <filename>##OEROOT##</filename> values.
+            <note>
+                You can see how the <filename>TEMPLATECONF</filename> variable
+                <filename>/scripts/oe-setup-builddir</filename> script in the 
+                <ulink url='&YOCTO_DOCS_DEV_URL;#source-directory'>Source Directory</ulink>.
+                You can find the Yocto Project version of the 
+                <filename>bblayers.conf.sample</filename> file in the 
+                <filename>/meta-yocto/conf</filename> directory.                
+            </note>
         </para>
     </section>