bitbake: cooker, toaster: variable definition tracking

In order to track the file where a configuration
variable was defined, this patch bring these changes:

* a new feature is defined in CookerFeatures, named
BASEDATASTORE_TRACKING. When a UI requests BASEDATASTORE_TRACKING,
the base variable definition are tracked when configuration
is parsed.

* getAllKeysWithFlags now includes variable history in the
data dump

* toaster_ui.py will record the operation, file path
and line number where the variable was changes

* toaster Simple UI will display the file path
and line number for Configuration page

There is a change in the models to accomodate the recording
of variable change history.

[YOCTO #5227]

(Bitbake rev: 78e58fed82f2a71f052485de0052d7b9cca53ffd)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

README.hardware

@@ -75,14 +75,32 @@ Intel x86 based PCs and devices (genericx86)
 
 The genericx86 MACHINE is tested on the following platforms:
 
-  o Asus EeePC 901
+Intel Xeon/Core i-Series:
-  o Acer Aspire One
+  + Intel Romley Server: Sandy Bridge Xeon processor, C600 PCH (Patsburg), (Canoe Pass CRB)
-  o Toshiba NB305
+  + Intel Romley Server: Ivy Bridge Xeon processor, C600 PCH (Patsburg), (Intel SDP S2R3)
-  o Intel Embedded Development Board 1-N450 (Black Sand)
+  + Intel Crystal Forest Server: Sandy Bridge Xeon processor, DH89xx PCH (Cave Creek), (Stargo CRB)
+  + Intel Chief River Mobile: Ivy Bridge Mobile processor, QM77 PCH (Panther Point-M), (Emerald Lake II CRB, Sabino Canyon CRB)
+  + Intel Huron River Mobile: Sandy Bridge processor, QM67 PCH (Cougar Point), (Emerald Lake CRB, EVOC EC7-1817LNAR board)
+  + Intel Calpella Platform: Core i7 processor, QM57 PCH (Ibex Peak-M), (Red Fort CRB, Emerson MATXM CORE-411-B)
+  + Intel Nehalem/Westmere-EP Server: Xeon 56xx/55xx processors, 5520 chipset, ICH10R IOH (82801), (Hanlan Creek CRB)
+  + Intel Nehalem Workstation: Xeon 56xx/55xx processors, System SC5650SCWS (Greencity CRB)
+  + Intel Picket Post Server: Xeon 56xx/55xx processors (Jasper Forest), 3420 chipset (Ibex Peak), (Osage CRB)
+  + Intel Storage Platform: Sandy Bridge Xeon processor, C600 PCH (Patsburg), (Oak Creek Canyon CRB)
+  + Intel Shark Bay Client Platform: Haswell processor, LynxPoint PCH, (Walnut Canyon CRB, Lava Canyon CRB, Basking Ridge CRB, Flathead Creek CRB)
+  + Intel Shark Bay Ultrabook Platform: Haswell ULT processor, Lynx Point-LP PCH, (WhiteTip Mountain 1 CRB)
 
-and is likely to work on many unlisted Atom based devices. The MACHINE type
+Intel Atom platforms:
-supports ethernet, wifi, sound, and i915 graphics by default in addition to
+  + Intel embedded Menlow: Intel Atom Z510/530 CPU, System Controller Hub US15W (Portwell NANO-8044)
-common PC input devices, busses, and so on.
+  + Intel Luna Pier: Intel Atom N4xx/D5xx series CPU (aka: Pineview-D & -M), 82801HM I/O Hub (ICH8M), (Advantech AIMB-212, Moon Creek CRB)
+  + Intel Queens Bay platform: Intel Atom E6xx CPU (aka: Tunnel Creek), Topcliff EG20T I/O Hub (Emerson NITX-315, Crown Bay CRB, Minnow Board)
+  + Intel Fish River Island platform: Intel Atom E6xx CPU (aka: Tunnel Creek), Topcliff EG20T I/O Hub (Kontron KM2M806)
+  + Intel Cedar Trail platform: Intel Atom N2000 & D2000 series CPU (aka: Cedarview), NM10 Express Chipset (Norco kit BIS-6630, Cedar Rock CRB)
+
+and is likely to work on many unlisted Atom/Core/Xeon based devices. The MACHINE
+type supports ethernet, wifi, sound, and Intel/vesa graphics by default in
+addition to common PC input devices, busses, and so on.  Note that it does not
+included the binary-only graphic drivers used on some Atom platforms, for
+accelerated graphics on these machines please refer to meta-intel.
 
 Depending on the device, it can boot from a traditional hard-disk, a USB device,
 or over the network. Writing generated images to physical media is

bitbake/LICENSE

@@ -0,0 +1,10 @@
+BitBake is licensed under the GNU General Public License version 2.0. See COPYING for further details.
+
+The following external components are distributed with this software:
+
+* The Toaster Simple UI application is based upon the Django project template, the files of which are covered by the BSD license and are copyright (c) Django Software
+Foundation and individual contributors.
+
+* Twitter Bootstrap (including Glyphicons), redistributed under the Apache License 2.0.
+
+* jQuery is redistributed under the MIT license.

bitbake/bin/bitbake

@@ -41,7 +41,7 @@ from bb import ui
 from bb import server
 from bb import cookerdata
 
-__version__ = "1.19.1"
+__version__ = "1.20.0"
 logger = logging.getLogger("BitBake")
 
 # Python multiprocessing requires /dev/shm
@@ -75,18 +75,6 @@ def get_ui(config):
         sys.exit("FATAL: Invalid user interface '%s' specified.\n"
                  "Valid interfaces: depexp, goggle, ncurses, hob, knotty [default]." % interface)
 
-def gather_extra_cache_data():
-    extra = []
-    interfaces = ['depexp', 'goggle', 'ncurses', 'hob', 'knotty']
-    for i in interfaces:
-        try:
-            ui = __import__("bb.ui." + i, fromlist = [i])
-            if hasattr(ui, "extraCaches"):
-                extra = extra + ui.extraCaches
-            del ui
-        except:
-            pass
-    return extra
 
 # Display bitbake/OE warnings via the BitBake.Warnings logger, ignoring others"""
 warnlog = logging.getLogger("BitBake.Warnings")
@@ -112,59 +100,58 @@ class BitBakeConfigParameters(cookerdata.ConfigParameters):
     def parseCommandLine(self):
         parser = optparse.OptionParser(
             version = "BitBake Build Tool Core version %s, %%prog version %s" % (bb.__version__, __version__),
-            usage = """%prog [options] [package ...]
+            usage = """%prog [options] [recipename/target ...]
 
-    Executes the specified task (default is 'build') for a given set of BitBake files.
+    Executes the specified task (default is 'build') for a given set of target recipes (.bb files).
-    It expects that BBFILES is defined, which is a space separated list of files to
+    It is assumed there is a conf/bblayers.conf available in cwd or in BBPATH which
-    be executed.  BBFILES does support wildcards.
+    will provide the layer, BBFILES and other configuration information.""")
-    Default BBFILES are the .bb files in the current directory.""")
 
-        parser.add_option("-b", "--buildfile", help = "execute the task against this .bb file, rather than a package from BBFILES. Does not handle any dependencies.",
+        parser.add_option("-b", "--buildfile", help = "Execute tasks from a specific .bb recipe directly. WARNING: Does not handle any dependencies from other recipes.",
                    action = "store", dest = "buildfile", default = None)
 
-        parser.add_option("-k", "--continue", help = "continue as much as possible after an error. While the target that failed, and those that depend on it, cannot be remade, the other dependencies of these targets can be processed all the same.",
+        parser.add_option("-k", "--continue", help = "Continue as much as possible after an error. While the target that failed and anything depending on it cannot be built, as much as possible will be built before stopping.",
                    action = "store_false", dest = "abort", default = True)
 
-        parser.add_option("-a", "--tryaltconfigs", help = "continue with builds by trying to use alternative providers where possible.",
+        parser.add_option("-a", "--tryaltconfigs", help = "Continue with builds by trying to use alternative providers where possible.",
                    action = "store_true", dest = "tryaltconfigs", default = False)
 
-        parser.add_option("-f", "--force", help = "force run of specified cmd, regardless of stamp status",
+        parser.add_option("-f", "--force", help = "Force the specified targets/task to run (invalidating any existing stamp file).",
                    action = "store_true", dest = "force", default = False)
 
-        parser.add_option("-c", "--cmd", help = "Specify task to execute. Note that this only executes the specified task for the providee and the packages it depends on, i.e. 'compile' does not implicitly call stage for the dependencies (IOW: use only if you know what you are doing). Depending on the base.bbclass a listtasks tasks is defined and will show available tasks",
+        parser.add_option("-c", "--cmd", help = "Specify the task to execute. The exact options available depend on the metadata. Some examples might be 'compile' or 'populate_sysroot' or 'listtasks' may give a list of the tasks available.",
                    action = "store", dest = "cmd")
 
-        parser.add_option("-C", "--clear-stamp", help = "Invalidate the stamp for the specified cmd such as 'compile' and run the default task for the specified target(s)",
+        parser.add_option("-C", "--clear-stamp", help = "Invalidate the stamp for the specified task such as 'compile' and then run the default task for the specified target(s).",
                     action = "store", dest = "invalidate_stamp")
 
-        parser.add_option("-r", "--read", help = "read the specified file before bitbake.conf",
+        parser.add_option("-r", "--read", help = "Read the specified file before bitbake.conf.",
                    action = "append", dest = "prefile", default = [])
 
-        parser.add_option("-R", "--postread", help = "read the specified file after bitbake.conf",
+        parser.add_option("-R", "--postread", help = "Read the specified file after bitbake.conf.",
                           action = "append", dest = "postfile", default = [])
 
-        parser.add_option("-v", "--verbose", help = "output more chit-chat to the terminal",
+        parser.add_option("-v", "--verbose", help = "Output more log message data to the terminal.",
                    action = "store_true", dest = "verbose", default = False)
 
         parser.add_option("-D", "--debug", help = "Increase the debug level. You can specify this more than once.",
                    action = "count", dest="debug", default = 0)
 
-        parser.add_option("-n", "--dry-run", help = "don't execute, just go through the motions",
+        parser.add_option("-n", "--dry-run", help = "Don't execute, just go through the motions.",
                    action = "store_true", dest = "dry_run", default = False)
 
-        parser.add_option("-S", "--dump-signatures", help = "don't execute, just dump out the signature construction information",
+        parser.add_option("-S", "--dump-signatures", help = "Don't execute, just dump out the signature construction information.",
                    action = "store_true", dest = "dump_signatures", default = False)
 
-        parser.add_option("-p", "--parse-only", help = "quit after parsing the BB files (developers only)",
+        parser.add_option("-p", "--parse-only", help = "Quit after parsing the BB recipes.",
                    action = "store_true", dest = "parse_only", default = False)
 
-        parser.add_option("-s", "--show-versions", help = "show current and preferred versions of all recipes",
+        parser.add_option("-s", "--show-versions", help = "Show current and preferred versions of all recipes.",
                    action = "store_true", dest = "show_versions", default = False)
 
-        parser.add_option("-e", "--environment", help = "show the global or per-package environment (this is what used to be bbread)",
+        parser.add_option("-e", "--environment", help = "Show the global or per-package environment complete with information about where variables were set/changed.",
                    action = "store_true", dest = "show_environment", default = False)
 
-        parser.add_option("-g", "--graphviz", help = "emit the dependency trees of the specified packages in the dot syntax, and the pn-buildlist to show the build list",
+        parser.add_option("-g", "--graphviz", help = "Save dependency tree information for the specified targets in the dot syntax.",
                     action = "store_true", dest = "dot_graph", default = False)
 
         parser.add_option("-I", "--ignore-deps", help = """Assume these dependencies don't exist and are already provided (equivalent to ASSUME_PROVIDED). Useful to make dependency graphs more appealing""",
@@ -173,34 +160,34 @@ class BitBakeConfigParameters(cookerdata.ConfigParameters):
         parser.add_option("-l", "--log-domains", help = """Show debug logging for the specified logging domains""",
                     action = "append", dest = "debug_domains", default = [])
 
-        parser.add_option("-P", "--profile", help = "profile the command and print a report",
+        parser.add_option("-P", "--profile", help = "Profile the command and save reports.",
                    action = "store_true", dest = "profile", default = False)
 
-        parser.add_option("-u", "--ui", help = "userinterface to use",
+        parser.add_option("-u", "--ui", help = "The user interface to use (e.g. knotty, hob, depexp).",
                    action = "store", dest = "ui")
 
-        parser.add_option("-t", "--servertype", help = "Choose which server to use, process or xmlrpc",
+        parser.add_option("-t", "--servertype", help = "Choose which server to use, process or xmlrpc.",
                    action = "store", dest = "servertype")
 
-        parser.add_option("", "--revisions-changed", help = "Set the exit code depending on whether upstream floating revisions have changed or not",
+        parser.add_option("", "--revisions-changed", help = "Set the exit code depending on whether upstream floating revisions have changed or not.",
                    action = "store_true", dest = "revisions_changed", default = False)
 
-        parser.add_option("", "--server-only", help = "Run bitbake without UI,  the frontend can connect with bitbake server itself",
+        parser.add_option("", "--server-only", help = "Run bitbake without a UI, only starting a server (cooker) process.",
                    action = "store_true", dest = "server_only", default = False)
 
-        parser.add_option("-B", "--bind", help = "The name/address for the bitbake server to bind to",
+        parser.add_option("-B", "--bind", help = "The name/address for the bitbake server to bind to.",
                    action = "store", dest = "bind", default = False)
 
-        parser.add_option("", "--no-setscene", help = "Do not run any setscene tasks, forces builds",
+        parser.add_option("", "--no-setscene", help = "Do not run any setscene tasks. sstate will be ignored and everything needed, built.",
                    action = "store_true", dest = "nosetscene", default = False)
 
-        parser.add_option("", "--remote-server", help = "Connect to the specified server",
+        parser.add_option("", "--remote-server", help = "Connect to the specified server.",
                    action = "store", dest = "remote_server", default = False)
 
-        parser.add_option("-m", "--kill-server", help = "Terminate the remote server",
+        parser.add_option("-m", "--kill-server", help = "Terminate the remote server.",
                     action = "store_true", dest = "kill_server", default = False)
 
-        parser.add_option("", "--observe-only", help = "Connect to a server as an observing-only client",
+        parser.add_option("", "--observe-only", help = "Connect to a server as an observing-only client.",
                    action = "store_true", dest = "observe_only", default = False)
 
         options, targets = parser.parse_args(sys.argv)
@@ -275,8 +262,8 @@ def main():
         if not configParams.bind:
             sys.exit("FATAL: The '--server-only' option requires a name/address to bind to with the -B option.\n")
         if configParams.remote_server:
-            sys.exit("FATAL: The '--server-only' option conflicts with the '--remote-server' option. %s\n" % 
+            sys.exit("FATAL: The '--server-only' option conflicts with %s.\n" % 
-                ("Please check your BBSERVER environment" if "BBSERVER" in os.environ else "" ))
+                ("the BBSERVER environment variable" if "BBSERVER" in os.environ else "the '--remote-server' option" ))
 
     if configParams.bind and configParams.servertype != "xmlrpc":
         sys.exit("FATAL: If '-B' or '--bind' is defined, we must set the servertype as 'xmlrpc'.\n")
@@ -302,30 +289,28 @@ def main():
     # Clear away any spurious environment variables while we stoke up the cooker
     cleanedvars = bb.utils.clean_environment()
 
-    # Collect all the caches we need
-    if configParams.server_only:
-        configuration.extra_caches = gather_extra_cache_data()
-    else:
-        configuration.extra_caches = getattr(ui_module, "extraCaches", [])
-
     if not configParams.remote_server:
         # we start a server with a given configuration
         server = start_server(servermodule, configParams, configuration)
+        bb.event.ui_queue = []
     else:
-        # we start a stub server that is actually a XMLRPClient to
+        # we start a stub server that is actually a XMLRPClient that connects to a real server
         server = servermodule.BitBakeXMLRPCClient(configParams.observe_only)
         server.saveConnectionDetails(configParams.remote_server)
 
-    logger.removeHandler(handler)
-
     if not configParams.server_only:
+        # Collect the feature set for the UI
+        featureset = getattr(ui_module, "featureSet", [])
+
         # Setup a connection to the server (cooker)
-        server_connection = server.establishConnection()
+        server_connection = server.establishConnection(featureset)
 
         # Restore the environment in case the UI needs it
         for k in cleanedvars:
             os.environ[k] = cleanedvars[k]
 
+        logger.removeHandler(handler)
+
         try:
             return ui_module.main(server_connection.connection, server_connection.events, configParams)
         finally:
@@ -339,6 +324,8 @@ def main():
 if __name__ == "__main__":
     try:
         ret = main()
+    except bb.BBHandledException:
+        ret = 1
     except Exception:
         ret = 1
         import traceback

bitbake/bin/bitbake-diffsigs

@@ -3,7 +3,7 @@
 # bitbake-diffsigs
 # BitBake task signature data comparison utility
 #
-# Copyright (C) 2012 Intel Corporation
+# Copyright (C) 2012-2013 Intel Corporation
 #
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License version 2 as
@@ -30,7 +30,18 @@ sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(sys.argv[0])), '
 import bb.tinfoil
 import bb.siggen
 
-logger = logging.getLogger('BitBake')
+def logger_create(name, output=sys.stderr):
+    logger = logging.getLogger(name)
+    console = logging.StreamHandler(output)
+    format = bb.msg.BBLogFormatter("%(levelname)s: %(message)s")
+    if output.isatty():
+        format.enable_color()
+    console.setFormatter(format)
+    logger.addHandler(console)
+    logger.setLevel(logging.INFO)
+    return logger
+
+logger = logger_create('bitbake-diffsigs')
 
 def find_compare_task(bbhandler, pn, taskname):
     """ Find the most recent signature files for the specified PN/task and compare them """
@@ -39,6 +50,9 @@ def find_compare_task(bbhandler, pn, taskname):
         logger.error('Metadata does not support finding signature data files')
         sys.exit(1)
 
+    if not taskname.startswith('do_'):
+        taskname = 'do_%s' % taskname
+
     filedates = bb.siggen.find_siginfo(pn, taskname, None, bbhandler.config_data)
     latestfiles = sorted(filedates.keys(), key=lambda f: filedates[f])[-2:]
     if not latestfiles:
@@ -71,6 +85,7 @@ def find_compare_task(bbhandler, pn, taskname):
 
 
 parser = optparse.OptionParser(
+    description = "Compares siginfo/sigdata files written out by BitBake",
     usage = """
   %prog -t recipename taskname
   %prog sigdatafile1 sigdatafile2
@@ -78,25 +93,30 @@ parser = optparse.OptionParser(
 
 parser.add_option("-t", "--task",
         help = "find the signature data files for last two runs of the specified task and compare them",
-        action="store_true", dest="taskmode")
+        action="store", dest="taskargs", nargs=2, metavar='recipename taskname')
 
 options, args = parser.parse_args(sys.argv)
 
-if len(args) == 1:
+if options.taskargs:
-    parser.print_help()
+    tinfoil = bb.tinfoil.Tinfoil()
+    tinfoil.prepare(config_only = True)
+    find_compare_task(tinfoil, options.taskargs[0], options.taskargs[1])
 else:
-    if options.taskmode:
+    if len(args) == 1:
-        tinfoil = bb.tinfoil.Tinfoil()
+        parser.print_help()
-        if len(args) < 3:
-            logger.error("Please specify a recipe and task name")
-            sys.exit(1)
-        tinfoil.prepare(config_only = True)
-        find_compare_task(tinfoil, args[1], args[2])
     else:
-        if len(args) == 2:
+        import cPickle
-            output = bb.siggen.dump_sigfile(sys.argv[1])
+        try:
-        else:
+            if len(args) == 2:
-            output = bb.siggen.compare_sigfiles(sys.argv[1], sys.argv[2])
+                output = bb.siggen.dump_sigfile(sys.argv[1])
+            else:
+                output = bb.siggen.compare_sigfiles(sys.argv[1], sys.argv[2])
+        except IOError as e:
+            logger.error(str(e))
+            sys.exit(1)
+        except cPickle.UnpicklingError, EOFError:
+            logger.error('Invalid signature data - ensure you are specifying sigdata/siginfo files')
+            sys.exit(1)
 
-        if output:
+    if output:
-            print '\n'.join(output)
+        print '\n'.join(output)

bitbake/bin/bitbake-dumpsig

@@ -1,11 +1,65 @@
 #!/usr/bin/env python
+
+# bitbake-dumpsig
+# BitBake task signature dump utility
+#
+# Copyright (C) 2013 Intel Corporation
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
 import os
 import sys
 import warnings
+import optparse
+import logging
+
 sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(sys.argv[0])), 'lib'))
 
 import bb.siggen
 
-output = bb.siggen.dump_sigfile(sys.argv[1])
+def logger_create(name, output=sys.stderr):
-if output:
+    logger = logging.getLogger(name)
-    print '\n'.join(output)
+    console = logging.StreamHandler(output)
+    format = bb.msg.BBLogFormatter("%(levelname)s: %(message)s")
+    if output.isatty():
+        format.enable_color()
+    console.setFormatter(format)
+    logger.addHandler(console)
+    logger.setLevel(logging.INFO)
+    return logger
+
+logger = logger_create('bitbake-dumpsig')
+
+parser = optparse.OptionParser(
+    description = "Dumps siginfo/sigdata files written out by BitBake",
+    usage = """
+  %prog sigdatafile""")
+
+options, args = parser.parse_args(sys.argv)
+
+if len(args) == 1:
+    parser.print_help()
+else:
+    import cPickle
+    try:
+        output = bb.siggen.dump_sigfile(args[1])
+    except IOError as e:
+        logger.error(str(e))
+        sys.exit(1)
+    except cPickle.UnpicklingError, EOFError:
+        logger.error('Invalid signature data - ensure you are specifying a sigdata/siginfo file')
+        sys.exit(1)
+
+    if output:
+        print '\n'.join(output)

bitbake/bin/toaster

@@ -0,0 +1,164 @@
+#!/bin/bash
+# (c) 2013 Intel Corp.
+
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+
+
+# This script enables toaster event logging and
+# starts bitbake resident server
+# use as:  source toaster [start|stop]
+
+# Helper function to kill a background toaster development server
+
+function webserverKillAll()
+{
+	local pidfile
+	for pidfile in ${BUILDDIR}/.toastermain.pid; do
+		if [ -f ${pidfile} ]; then
+		while kill -0 $(< ${pidfile}) 2>/dev/null; do
+			kill -SIGTERM -$(< ${pidfile}) 2>/dev/null
+			sleep 1;
+		done;
+		rm  ${pidfile}
+		fi
+	done
+}
+
+function webserverStartAll()
+{
+        retval=0
+        python $BBBASEDIR/lib/toaster/manage.py syncdb || retval=1
+        if [ $retval -eq 1 ]; then
+                echo "Failed db sync, stopping system start" 1>&2
+        else
+            python $BBBASEDIR/lib/toaster/manage.py runserver 0.0.0.0:8000 </dev/null >${BUILDDIR}/toaster_web.log 2>&1 & echo $! >${BUILDDIR}/.toastermain.pid
+        fi
+        return $retval
+}
+
+# Helper functions to add a special configuration file
+
+function addtoConfiguration()
+{
+        echo "#Created by toaster start script" > ${BUILDDIR}/conf/$2
+        echo $1 >> ${BUILDDIR}/conf/$2
+}
+
+# define the stop command
+function stop_system()
+{
+    if [ -f ${BUILDDIR}/.toasterui.pid ]; then
+        kill $(< ${BUILDDIR}/.toasterui.pid )
+        rm ${BUILDDIR}/.toasterui.pid
+    fi
+    BBSERVER=localhost:8200 bitbake -m
+    unset BBSERVER
+    webserverKillAll
+    # force stop any misbehaving bitbake server
+    lsof bitbake.lock | awk '{print $2}' | grep "[0-9]\+" | xargs -n1 -r kill
+}
+
+# We make sure we're running in the current shell and in a good environment
+
+if [ -z "$ZSH_NAME" ] && [ `basename \"$0\"` = `basename \"$BASH_SOURCE\"` ]; then
+    echo "Error: This script needs to be sourced. Please run as 'source toaster [start|stop]'" 1>&2;
+    exit 1
+fi
+
+if [ -z "$BUILDDIR" ] || [ -z `which bitbake` ]; then
+    echo "Error: Build environment is not setup or bitbake is not in path." 1>&2;
+    return 2
+fi
+
+BBBASEDIR=`dirname ${BASH_SOURCE}`/..
+
+
+# Verify prerequisites
+
+if ! echo "import django; print (1,4,5) == django.VERSION[0:3]" | python 2>/dev/null | grep True >/dev/null; then
+    echo -e "This program needs Django 1.4.5. Please install with\n\nsudo pip install django==1.4.5"
+    return 2
+fi
+
+
+
+# Determine the action. If specified by arguments, fine, if not, toggle it
+if [ "x$1" == "xstart" ] || [ "x$1" == "xstop" ]; then
+    CMD="$1"
+else
+    if [ -z "$BBSERVER" ]; then
+        CMD="start"
+    else
+        CMD="stop"
+    fi;
+fi
+
+NOTOASTERUI=0
+if [ "x$2" == "xnoui" ]; then
+    NOTOASTERUI=1
+fi
+
+echo "The system will $CMD."
+
+# Make sure it's safe to run by checking bitbake lock
+
+lock=1
+if [ -e $BUILDDIR/bitbake.lock ]; then
+    (flock -n 200 ) 200<$BUILDDIR/bitbake.lock || lock=0
+fi
+
+if [ ${CMD} == "start" ] && ( [ $lock -eq 0 ] || [ -e $BUILDDIR/.toastermain.pid ] ); then
+    echo "Error: bitbake lock state error. System is already on." 2>&1
+    return 3
+elif [ ${CMD} == "stop" ] && ( [ $lock -eq 1 ] || ! [ -e $BUILDDIR/.toastermain.pid ] ) ; then
+    echo "Error: bitbake lock state error. Trying to stop a stopped system ?
+If you think the system is hanged up, you can try to manually stop system with the commands
+
+# BBSERVER=localhost:8200 bitbake -m
+
+and
+
+# webserverKillAll
+" 2>&1
+    return 3
+fi
+
+
+
+
+
+# Execute the commands
+
+case $CMD in
+    start )
+        addtoConfiguration "INHERIT+=\"toaster buildhistory\"" toaster.conf
+        webserverStartAll || return 4
+        unset BBSERVER
+        bitbake --postread conf/toaster.conf --server-only -t xmlrpc -B localhost:8200
+        export BBSERVER=localhost:8200
+        if [ $NOTOASTERUI == 0 ]; then        # we start the TOASTERUI only if not inhibited
+            bitbake --observe-only -u toasterui >${BUILDDIR}/toaster_ui.log 2>&1 & echo $! >${BUILDDIR}/.toasterui.pid
+        fi
+        # stop system on terminal exit
+        trap stop_system SIGHUP
+    ;;
+    stop )
+        stop_system
+        trap '' SIGHUP
+    ;;
+esac
+
+echo "Successful ${CMD}."
+

bitbake/doc/manual/usermanual.xml

@@ -137,6 +137,24 @@ share common metadata between many packages.</para></listitem>
                 <para>In this example, <varname>B</varname> is now <literal>bvaladditionaldata</literal> and <varname>C</varname> is <literal>testcval</literal>. In contrast to the above appending and prepending operators, no additional space
 will be introduced.</para>
             </section>
+            <section>
+                <title>Appending and Prepending (override style syntax)</title>
+                    <para><screen><varname>B</varname> = "bval"
+<varname>B_append</varname> = " additional data"
+<varname>C</varname> = "cval"
+<varname>C_prepend</varname> = "additional data "</screen></para>
+                 <para>This example results in <varname>B</varname> becoming <literal>bval additional data</literal>
+and <varname>C</varname> becoming <literal>additional data cval</literal>. Note the spaces in the append.
+Unlike the += operator, additional space is not automatically added.  You must take steps to add space
+yourself.</para>
+            </section>
+            <section>
+                <title>Removing (override style syntax)</title>
+                <para><screen><varname>FOO</varname> = "123 456 789 123456 123 456 123 456"
+<varname>FOO_remove</varname> = "123"
+<varname>FOO_remove</varname> = "456"</screen></para>
+                <para>In this example, <varname>FOO</varname> is now <literal>789 123456</literal>.</para>
+            </section>
             <section>
                 <title>Conditional metadata set</title>
                 <para>OVERRIDES is a <quote>:</quote> separated variable containing each item you want to satisfy conditions.  So, if you have a variable which is conditional on <quote>arm</quote>, and <quote>arm</quote> is in OVERRIDES, then the <quote>arm</quote> specific version of the variable is used rather than the non-conditional version.  Example:</para>

bitbake/lib/bb/__init__.py

@@ -21,7 +21,7 @@
 # with this program; if not, write to the Free Software Foundation, Inc.,
 # 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
 
-__version__ = "1.19.1"
+__version__ = "1.20.0"
 
 import sys
 if sys.version_info < (2, 7, 3):

bitbake/lib/bb/build.py

@@ -72,6 +72,8 @@ class TaskBase(event.Event):
     def __init__(self, t, logfile, d):
         self._task = t
         self._package = d.getVar("PF", True)
+        self.taskfile = d.getVar("FILE", True)
+        self.taskname = self._task
         self.logfile = logfile
         event.Event.__init__(self)
         self._message = "recipe %s: task %s: %s" % (d.getVar("PF", True), t, self.getDisplayName())
@@ -89,6 +91,9 @@ class TaskBase(event.Event):
 
 class TaskStarted(TaskBase):
     """Task execution started"""
+    def __init__(self, t, logfile, taskflags, d):
+        super(TaskStarted, self).__init__(t, logfile, d)
+        self.taskflags = taskflags
 
 class TaskSucceeded(TaskBase):
     """Task execution completed"""
@@ -344,6 +349,14 @@ def _exec_task(fn, task, d, quieterr):
     if not tempdir:
         bb.fatal("T variable not set, unable to build")
 
+    # Change nice level if we're asked to
+    nice = localdata.getVar("BB_TASK_NICE_LEVEL", True)
+    if nice:
+        curnice = os.nice(0)
+        nice = int(nice) - curnice
+        newnice = os.nice(nice)
+        logger.debug(1, "Renice to %s " % newnice)
+
     bb.utils.mkdirhier(tempdir)
 
     # Determine the logfile to generate
@@ -412,7 +425,9 @@ def _exec_task(fn, task, d, quieterr):
     localdata.setVar('BB_LOGFILE', logfn)
     localdata.setVar('BB_RUNTASK', task)
 
-    event.fire(TaskStarted(task, logfn, localdata), localdata)
+    flags = localdata.getVarFlags(task)
+
+    event.fire(TaskStarted(task, logfn, flags, localdata), localdata)
     try:
         for func in (prefuncs or '').split():
             exec_func(func, localdata)

bitbake/lib/bb/cache.py

@@ -43,7 +43,7 @@ except ImportError:
     logger.info("Importing cPickle failed. "
                 "Falling back to a very slow implementation.")
 
-__cache_version__ = "146"
+__cache_version__ = "147"
 
 def getCacheFile(path, filename, data_hash):
     return os.path.join(path, filename + "." + data_hash)

bitbake/lib/bb/cache_extra.py

@@ -35,6 +35,12 @@ class HobRecipeInfo(RecipeInfoCommon):
     # such as (bb_cache.dat, bb_extracache_hob.dat) 
     cachefile = "bb_extracache_" + classname +".dat"        
 
+    # override this member with the list of extra cache fields
+    # that this class will provide
+    cachefields = ['summary', 'license', 'section',
+            'description', 'homepage', 'bugtracker',
+            'prevision', 'files_info']
+
     def __init__(self, filename, metadata):
 
         self.summary = self.getvar('SUMMARY', metadata)

bitbake/lib/bb/command.py

@@ -145,6 +145,15 @@ class CommandsSync:
         """
         command.cooker.shutdown(True)
 
+    def getAllKeysWithFlags(self, command, params):
+        """
+        Returns a dump of the global state. Call with
+        variable flags to be retrieved as params.
+        """
+        flaglist = params[0]
+        return command.cooker.getAllKeysWithFlags(flaglist)
+    getAllKeysWithFlags.readonly = True
+
     def getVariable(self, command, params):
         """
         Read the value of a variable from data
@@ -232,6 +241,13 @@ class CommandsSync:
         op = params[3]
         command.cooker.modifyConfigurationVar(var, val, default_file, op)
 
+    def removeVarFile(self, command, params):
+        """
+        Remove a variable declaration from a file
+        """
+        var = params[0]
+        command.cooker.removeConfigurationVar(var)
+
     def createConfigFile(self, command, params):
         """
         Create an extra configuration file

bitbake/lib/bb/cooker.py

@@ -79,6 +79,29 @@ class SkippedPackage:
         elif reason:
             self.skipreason = reason
 
+
+class CookerFeatures(object):
+    _feature_list = [HOB_EXTRA_CACHES, SEND_DEPENDS_TREE, BASEDATASTORE_TRACKING] = range(3)
+
+    def __init__(self):
+        self._features=set()
+
+    def setFeature(self, f):
+        # validate we got a request for a feature we support
+        if f not in CookerFeatures._feature_list:
+            return
+        self._features.add(f)
+
+    def __contains__(self, f):
+        return f in self._features
+
+    def __iter__(self):
+        return self._features.__iter__()
+
+    def next(self):
+        return self._features.next()
+
+
 #============================================================================#
 # BBCooker
 #============================================================================#
@@ -90,6 +113,7 @@ class BBCooker:
     def __init__(self, configuration):
         self.recipecache = None
         self.skiplist = {}
+        self.featureset = CookerFeatures()
 
         self.configuration = configuration
 
@@ -122,7 +146,13 @@ class BBCooker:
         self.state = state.initial
 
         self.caches_array = []
-        caches_name_array = ['bb.cache:CoreRecipeInfo'] + self.configuration.extra_caches
+
+        all_extra_cache_names = []
+        # We hardcode all known cache types in a single place, here.
+        if CookerFeatures.HOB_EXTRA_CACHES in self.featureset:
+            all_extra_cache_names.append("bb.cache_extra:HobRecipeInfo")
+
+        caches_name_array = ['bb.cache:CoreRecipeInfo'] + all_extra_cache_names
 
         # At least CoreRecipeInfo will be loaded, so caches_array will never be empty!
         # This is the entry point, no further check needed!
@@ -147,6 +177,9 @@ class BBCooker:
         self.data.disableTracking()
 
     def loadConfigurationData(self):
+        if CookerFeatures.BASEDATASTORE_TRACKING in self.featureset:
+            self.enableDataTracking()
+
         self.initConfigurationData()
         self.databuilder.parseBaseConfiguration()
         self.data = self.databuilder.data
@@ -159,11 +192,18 @@ class BBCooker:
         bb.data.update_data(self.event_data)
         bb.parse.init_parser(self.event_data)
 
+        if CookerFeatures.BASEDATASTORE_TRACKING in self.featureset:
+            self.disableDataTracking()
+
+
     def modifyConfigurationVar(self, var, val, default_file, op):
         if op == "append":
             self.appendConfigurationVar(var, val, default_file)
         elif op == "set":
-            self.saveConfigurationVar(var, val, default_file)
+            self.saveConfigurationVar(var, val, default_file, "=")
+        elif op == "earlyAssign":
+            self.saveConfigurationVar(var, val, default_file, "?=")
+
 
     def appendConfigurationVar(self, var, val, default_file):
         #add append var operation to the end of default_file
@@ -177,7 +217,7 @@ class BBCooker:
         for c in contents:
             total += c
 
-        total += "#added by bitbake"
+        total += "#added by hob"
         total += "\n%s += \"%s\"\n" % (var, val)
 
         with open(default_file, 'w') as f:
@@ -188,7 +228,7 @@ class BBCooker:
         loginfo = {"op":append, "file":default_file, "line":total.count("\n")}
         self.data.appendVar(var, val, **loginfo)
 
-    def saveConfigurationVar(self, var, val, default_file):
+    def saveConfigurationVar(self, var, val, default_file, op):
 
         replaced = False
         #do not save if nothing changed
@@ -230,8 +270,8 @@ class BBCooker:
                     #check if the variable was saved before in the same way
                     #if true it replace the place where the variable was declared
                     #else it comments it
-                    if contents[begin_line-1]== "#added by bitbake\n":
+                    if contents[begin_line-1]== "#added by hob\n":
-                        contents[begin_line] = "%s = \"%s\"\n" % (var, val)
+                        contents[begin_line] = "%s %s \"%s\"\n" % (var, op, val)
                         replaced = True
                     else:
                         for ii in range(begin_line, end_line):
@@ -260,8 +300,8 @@ class BBCooker:
                 total += c
 
             #add the variable on a single line, to be easy to replace the second time
-            total += "\n#added by bitbake"
+            total += "\n#added by hob"
-            total += "\n%s = \"%s\"\n" % (var, val)
+            total += "\n%s %s \"%s\"\n" % (var, op, val)
 
             with open(default_file, 'w') as f:
                 f.write(total)
@@ -271,13 +311,50 @@ class BBCooker:
             loginfo = {"op":set, "file":default_file, "line":total.count("\n")}
             self.data.setVar(var, val, **loginfo)
 
+    def removeConfigurationVar(self, var):
+        conf_files = self.data.varhistory.get_variable_files(var)
+        topdir = self.data.getVar("TOPDIR")
+
+        for conf_file in conf_files:
+            if topdir in conf_file:
+                with open(conf_file, 'r') as f:
+                    contents = f.readlines()
+                f.close()
+
+                lines = self.data.varhistory.get_variable_lines(var, conf_file)
+                for line in lines:
+                    total = ""
+                    i = 0
+                    for c in contents:
+                        total += c
+                        i = i + 1
+                        if i==int(line):
+                            end_index = len(total)
+                    index = total.rfind(var, 0, end_index)
+
+                    begin_line = total.count("\n",0,index)
+
+                    #check if the variable was saved before in the same way
+                    if contents[begin_line-1]== "#added by hob\n":
+                        contents[begin_line-1] = contents[begin_line] = "\n"
+                    else:
+                        contents[begin_line] = "\n"
+                    #remove var from history
+                    self.data.varhistory.del_var_history(var, conf_file, line)
+
+                total = ""
+                for c in contents:
+                    total += c
+                with open(conf_file, 'w') as f:
+                    f.write(total)
+                f.close()
+
     def createConfigFile(self, name):
         path = os.getcwd()
         confpath = os.path.join(path, "conf", name)
         open(confpath, 'w').close()
 
     def parseConfiguration(self):
-
         # Set log file verbosity
         verboselogs = bb.utils.to_boolean(self.data.getVar("BB_VERBOSE_LOGS", "0"))
         if verboselogs:
@@ -427,7 +504,10 @@ class BBCooker:
         runlist, taskdata = self.prepareTreeData(pkgs_to_build, task)
         rq = bb.runqueue.RunQueue(self, self.data, self.recipecache, taskdata, runlist)
         rq.rqdata.prepare()
+        return self.buildDependTree(rq, taskdata)
 
+
+    def buildDependTree(self, rq, taskdata):
         seen_fnids = []
         depend_tree = {}
         depend_tree["depends"] = {}
@@ -437,6 +517,7 @@ class BBCooker:
         depend_tree["packages"] = {}
         depend_tree["rdepends-pkg"] = {}
         depend_tree["rrecs-pkg"] = {}
+        depend_tree["layer-priorities"] = self.recipecache.bbfile_config_priorities
 
         for task in xrange(len(rq.rqdata.runq_fnid)):
             taskname = rq.rqdata.runq_task[task]
@@ -448,6 +529,20 @@ class BBCooker:
                 depend_tree["pn"][pn] = {}
                 depend_tree["pn"][pn]["filename"] = fn
                 depend_tree["pn"][pn]["version"] = version
+                depend_tree["pn"][pn]["inherits"] = self.recipecache.inherits.get(fn, None)
+
+                # if we have extra caches, list all attributes they bring in
+                extra_info = []
+                for cache_class in self.caches_array:
+                    if type(cache_class) is type and issubclass(cache_class, bb.cache.RecipeInfoCommon) and hasattr(cache_class, 'cachefields'):
+                        cachefields = getattr(cache_class, 'cachefields', [])
+                        extra_info = extra_info + cachefields
+
+                # for all attributes stored, add them to the dependency tree
+                for ei in extra_info:
+                    depend_tree["pn"][pn][ei] = vars(self.recipecache)[ei][fn]
+
+
             for dep in rq.rqdata.runq_depends[task]:
                 depfn = taskdata.fn_index[rq.rqdata.runq_fnid[dep]]
                 deppn = self.recipecache.pkg_fn[depfn]
@@ -510,35 +605,30 @@ class BBCooker:
         depend_tree["rdepends-pkg"] = {}
         depend_tree["rrecs-pkg"] = {}
 
+        # if we have extra caches, list all attributes they bring in
+        extra_info = []
+        for cache_class in self.caches_array:
+            if type(cache_class) is type and issubclass(cache_class, bb.cache.RecipeInfoCommon) and hasattr(cache_class, 'cachefields'):
+                cachefields = getattr(cache_class, 'cachefields', [])
+                extra_info = extra_info + cachefields
+
         for task in xrange(len(tasks_fnid)):
             fnid = tasks_fnid[task]
             fn = taskdata.fn_index[fnid]
             pn = self.recipecache.pkg_fn[fn]
-            version  = "%s:%s-%s" % self.recipecache.pkg_pepvpr[fn]
+
-            summary = self.recipecache.summary[fn]
-            lic = self.recipecache.license[fn]
-            section = self.recipecache.section[fn]
-            description = self.recipecache.description[fn]
-            homepage = self.recipecache.homepage[fn]
-            bugtracker = self.recipecache.bugtracker[fn]
-            files_info = self.recipecache.files_info[fn]
-            rdepends = self.recipecache.rundeps[fn]
-            rrecs = self.recipecache.runrecs[fn]
-            prevision = self.recipecache.prevision[fn]
-            inherits = self.recipecache.inherits.get(fn, None)
             if pn not in depend_tree["pn"]:
                 depend_tree["pn"][pn] = {}
                 depend_tree["pn"][pn]["filename"] = fn
+                version  = "%s:%s-%s" % self.recipecache.pkg_pepvpr[fn]
                 depend_tree["pn"][pn]["version"] = version
-                depend_tree["pn"][pn]["summary"] = summary
+                rdepends = self.recipecache.rundeps[fn]
-                depend_tree["pn"][pn]["license"] = lic
+                rrecs = self.recipecache.runrecs[fn]
-                depend_tree["pn"][pn]["section"] = section
+                depend_tree["pn"][pn]["inherits"] = self.recipecache.inherits.get(fn, None)
-                depend_tree["pn"][pn]["description"] = description
+
-                depend_tree["pn"][pn]["inherits"] = inherits
+                # for all extra attributes stored, add them to the dependency tree
-                depend_tree["pn"][pn]["homepage"] = homepage
+                for ei in extra_info:
-                depend_tree["pn"][pn]["bugtracker"] = bugtracker
+                    depend_tree["pn"][pn][ei] = vars(self.recipecache)[ei][fn]
-                depend_tree["pn"][pn]["files_info"] = files_info
-                depend_tree["pn"][pn]["revision"] = prevision
 
             if fnid not in seen_fnids:
                 seen_fnids.append(fnid)
@@ -1001,7 +1091,6 @@ class BBCooker:
 
         self.buildSetVars()
 
-        self.recipecache = bb.cache.CacheData(self.caches_array)
         infos = bb.cache.Cache.parse(fn, self.collection.get_file_appends(fn), \
                                      self.data,
                                      self.caches_array)
@@ -1113,6 +1202,24 @@ class BBCooker:
 
         self.configuration.server_register_idlecallback(buildTargetsIdle, rq)
 
+
+    def getAllKeysWithFlags(self, flaglist):
+        dump = {}
+        for k in self.data.keys():
+            try:
+                v = self.data.getVar(k, True)
+                if not k.startswith("__") and not isinstance(v, bb.data_smart.DataSmart):
+                    dump[k] = {
+    'v' : v ,
+    'history' : self.data.varhistory.variable(k),
+                    }
+                    for d in flaglist:
+                        dump[k][d] = self.data.getVarFlag(k, d)
+            except Exception as e:
+                print(e)
+        return dump
+
+
     def generateNewImage(self, image, base_image, package_queue, timestamp, description):
         '''
         Create a new image with a "require"/"inherit" base_image statement
@@ -1192,6 +1299,9 @@ class BBCooker:
 
     def checkPackages(self, pkgs_to_build):
 
+        # Return a copy, don't modify the original
+        pkgs_to_build = pkgs_to_build[:]
+
         if len(pkgs_to_build) == 0:
             raise NothingToBuild
 

bitbake/lib/bb/cookerdata.py

@@ -127,7 +127,6 @@ class CookerConfiguration(object):
         self.dump_signatures = False
         self.dry_run = False
         self.tracking = False
-        self.extra_caches = []
 
         self.env = {}
 

bitbake/lib/bb/data_smart.py

@@ -40,7 +40,7 @@ logger = logging.getLogger("BitBake.Data")
 
 __setvar_keyword__ = ["_append", "_prepend", "_remove"]
 __setvar_regexp__ = re.compile('(?P<base>.*?)(?P<keyword>_append|_prepend|_remove)(_(?P<add>.*))?$')
-__expand_var_regexp__ = re.compile(r"\${[^{}@]+}")
+__expand_var_regexp__ = re.compile(r"\${[^{}@\n\t ]+}")
 __expand_python_regexp__ = re.compile(r"\${@.+?}")
 
 def infer_caller_details(loginfo, parent = False, varval = True):
@@ -96,10 +96,9 @@ class VariableParse:
                     raise Exception("variable %s references itself!" % self.varname)
             if key in self.d.expand_cache:
                 varparse = self.d.expand_cache[key]
-                self.references |= varparse.references
+                var = varparse.value
-                self.execs |= varparse.execs
+            else:
-                return varparse.value
+                var = self.d.getVar(key, True)
-            var = self.d.getVar(key, True)
             self.references.add(key)
             if var is not None:
                 return var
@@ -282,9 +281,13 @@ class VariableHistory(object):
                 lines.append(line)
         return lines
 
-    def del_var_history(self, var):
+    def del_var_history(self, var, f=None, line=None):
+        """If file f and line are not given, the entire history of var is deleted"""
         if var in self.variables:
-            self.variables[var] = []
+            if f and line:
+                self.variables[var] = [ x for x in self.variables[var] if x['file']!=f and x['line']!=line]
+            else:
+                self.variables[var] = []
 
 class DataSmart(MutableMapping):
     def __init__(self, special = COWDictBase.copy(), seen = COWDictBase.copy() ):

bitbake/lib/bb/event.py

@@ -589,6 +589,16 @@ class PackageInfo(Event):
         Event.__init__(self)
         self._pkginfolist = pkginfolist
 
+class MetadataEvent(Event):
+    """
+    Generic event that target for OE-Core classes
+    to report information during asynchrous execution
+    """
+    def __init__(self, eventtype, eventdata):
+        Event.__init__(self)
+        self.type = eventtype
+        self.data = eventdata
+
 class SanityCheck(Event):
     """
     Event to issue sanity check

bitbake/lib/bb/monitordisk.py

@@ -225,7 +225,7 @@ class diskMonitor:
                         self.preFreeS[k] = freeSpace
 
                     if action == "STOPTASKS" and not self.checked[k]:
-                        logger.error("No new tasks can be excuted since the disk space monitor action is \"STOPTASKS\"!")
+                        logger.error("No new tasks can be executed since the disk space monitor action is \"STOPTASKS\"!")
                         self.checked[k] = True
                         rq.finish_runqueue(False)
                         bb.event.fire(bb.event.DiskFull(dev, 'disk', freeSpace, path), self.configuration)
@@ -253,7 +253,7 @@ class diskMonitor:
                         self.preFreeI[k] = freeInode
 
                     if action  == "STOPTASKS" and not self.checked[k]:
-                        logger.error("No new tasks can be excuted since the disk space monitor action is \"STOPTASKS\"!")
+                        logger.error("No new tasks can be executed since the disk space monitor action is \"STOPTASKS\"!")
                         self.checked[k] = True
                         rq.finish_runqueue(False)
                         bb.event.fire(bb.event.DiskFull(dev, 'inode', freeInode, path), self.configuration)

bitbake/lib/bb/runqueue.py

@@ -217,6 +217,15 @@ class RunQueueData:
             ret.extend([nam])
         return ret
 
+    def get_task_name(self, task):
+        return self.runq_task[task]
+
+    def get_task_file(self, task):
+        return self.taskData.fn_index[self.runq_fnid[task]]
+
+    def get_task_hash(self, task):
+        return self.runq_hash[task]
+
     def get_user_idstring(self, task, task_name_suffix = ""):
         fn = self.taskData.fn_index[self.runq_fnid[task]]
         taskname = self.runq_task[task] + task_name_suffix
@@ -999,6 +1008,11 @@ class RunQueue:
             else:
                 self.state = runQueueSceneInit
 
+                # we are ready to run, see if any UI client needs the dependency info
+                if bb.cooker.CookerFeatures.SEND_DEPENDS_TREE in self.cooker.featureset:
+                    depgraph = self.cooker.buildDependTree(self, self.rqdata.taskData)
+                    bb.event.fire(bb.event.DepTreeGenerated(depgraph), self.cooker.data)
+
         if self.state is runQueueSceneInit:
             if self.cooker.configuration.dump_signatures:
                 self.dump_signatures()
@@ -1781,6 +1795,9 @@ class runQueueEvent(bb.event.Event):
     def __init__(self, task, stats, rq):
         self.taskid = task
         self.taskstring = rq.rqdata.get_user_idstring(task)
+        self.taskname = rq.rqdata.get_task_name(task)
+        self.taskfile = rq.rqdata.get_task_file(task)
+        self.taskhash = rq.rqdata.get_task_hash(task)
         self.stats = stats.copy()
         bb.event.Event.__init__(self)
 
@@ -1792,6 +1809,9 @@ class sceneQueueEvent(runQueueEvent):
         runQueueEvent.__init__(self, task, stats, rq)
         realtask = rq.rqdata.runq_setscene[task]
         self.taskstring = rq.rqdata.get_user_idstring(realtask, "_setscene")
+        self.taskname = rq.rqdata.get_task_name(realtask) + "_setscene"
+        self.taskfile = rq.rqdata.get_task_file(realtask)
+        self.taskhash = rq.rqdata.get_task_hash(task)
 
 class runQueueTaskStarted(runQueueEvent):
     """

bitbake/lib/bb/server/__init__.py

@@ -89,7 +89,7 @@ class BitBakeBaseServer(object):
     def detach(self):
         return
 
-    def establishConnection(self):
+    def establishConnection(self, featureset):
         raise   "Must redefine the %s.establishConnection()" % self.__class__.__name__
 
     def endSession(self):

bitbake/lib/bb/server/process.py

@@ -31,7 +31,7 @@ import sys
 import time
 import select
 from Queue import Empty
-from multiprocessing import Event, Process, util, Queue, Pipe, queues
+from multiprocessing import Event, Process, util, Queue, Pipe, queues, Manager
 
 from . import BitBakeBaseServer, BitBakeBaseServerConnection, BaseImplServer
 
@@ -78,12 +78,13 @@ class ProcessServer(Process, BaseImplServer):
     profile_filename = "profile.log"
     profile_processed_filename = "profile.log.processed"
 
-    def __init__(self, command_channel, event_queue):
+    def __init__(self, command_channel, event_queue, featurelist):
         BaseImplServer.__init__(self)
-        Process.__init__(self)
+        Process.__init__(self, args=(featurelist))
         self.command_channel = command_channel
         self.event_queue = event_queue
         self.event = EventAdapter(event_queue)
+        self.featurelist = featurelist
         self.quit = False
 
         self.keep_running = Event()
@@ -94,6 +95,14 @@ class ProcessServer(Process, BaseImplServer):
         for event in bb.event.ui_queue:
             self.event_queue.put(event)
         self.event_handle.value = bb.event.register_UIHhandler(self)
+
+        # process any feature changes based on what UI requested
+        original_featureset = list(self.cooker.featureset)
+        while len(self.featurelist)> 0:
+            self.cooker.featureset.setFeature(self.featurelist.pop())
+        if (original_featureset != list(self.cooker.featureset)):
+            self.cooker.reset()
+
         bb.cooker.server_main(self.cooker, self.main)
 
     def main(self):
@@ -198,13 +207,17 @@ class BitBakeServer(BitBakeBaseServer):
         #
         self.ui_channel, self.server_channel = Pipe()
         self.event_queue = ProcessEventQueue(0)
-        self.serverImpl = ProcessServer(self.server_channel, self.event_queue)
+        manager = Manager()
+        self.featurelist = manager.list()
+        self.serverImpl = ProcessServer(self.server_channel, self.event_queue, self.featurelist)
 
     def detach(self):
         self.serverImpl.start()
         return
 
-    def establishConnection(self):
+    def establishConnection(self, featureset):
+        for f in featureset:
+            self.featurelist.append(f)
         self.connection = BitBakeProcessServerConnection(self.serverImpl, self.ui_channel, self.event_queue)
         signal.signal(signal.SIGTERM, lambda i, s: self.connection.terminate())
         return self.connection

bitbake/lib/bb/server/xmlrpc.py

@@ -89,12 +89,23 @@ class BitBakeServerCommands():
         self.server = server
         self.has_client = False
 
-    def registerEventHandler(self, host, port):
+    def registerEventHandler(self, host, port, featureset = []):
         """
         Register a remote UI Event Handler
         """
         s, t = _create_server(host, port)
 
+        # we don't allow connections if the cooker is running
+        if (self.cooker.state in [bb.cooker.state.parsing, bb.cooker.state.running]):
+            return None
+
+        original_featureset = list(self.cooker.featureset)
+        for f in featureset:
+            self.cooker.featureset.setFeature(f)
+
+        if (original_featureset != list(self.cooker.featureset)):
+            self.cooker.reset()
+
         self.event_handle = bb.event.register_UIHhandler(s)
         return self.event_handle
 
@@ -263,11 +274,12 @@ class XMLRPCServer(SimpleXMLRPCServer, BaseImplServer):
         self.connection_token = token
 
 class BitBakeXMLRPCServerConnection(BitBakeBaseServerConnection):
-    def __init__(self, serverImpl, clientinfo=("localhost", 0), observer_only = False):
+    def __init__(self, serverImpl, clientinfo=("localhost", 0), observer_only = False, featureset = []):
         self.connection, self.transport = _create_server(serverImpl.host, serverImpl.port)
         self.clientinfo = clientinfo
         self.serverImpl = serverImpl
         self.observer_only = observer_only
+        self.featureset = featureset
 
     def connect(self):
         if not self.observer_only:
@@ -277,7 +289,8 @@ class BitBakeXMLRPCServerConnection(BitBakeBaseServerConnection):
         if token is None:
             return None
         self.transport.set_connection_token(token)
-        self.events = uievent.BBUIEventQueue(self.connection, self.clientinfo)
+
+        self.events = uievent.BBUIEventQueue(self.connection, self.clientinfo, self.featureset)
         for event in bb.event.ui_queue:
             self.events.queue_event(event)
         return self
@@ -301,14 +314,15 @@ class BitBakeXMLRPCServerConnection(BitBakeBaseServerConnection):
 
 class BitBakeServer(BitBakeBaseServer):
     def initServer(self, interface = ("localhost", 0)):
+        self.interface = interface
         self.serverImpl = XMLRPCServer(interface)
 
     def detach(self):
         daemonize.createDaemon(self.serverImpl.serve_forever, "bitbake-cookerdaemon.log")
         del self.cooker
 
-    def establishConnection(self):
+    def establishConnection(self, featureset):
-        self.connection = BitBakeXMLRPCServerConnection(self.serverImpl)
+        self.connection = BitBakeXMLRPCServerConnection(self.serverImpl, self.interface, False, featureset)
         return self.connection.connect()
 
     def set_connection_token(self, token):
@@ -318,12 +332,13 @@ class BitBakeXMLRPCClient(BitBakeBaseServer):
 
     def __init__(self, observer_only = False):
         self.observer_only = observer_only
+        # if we need extra caches, just tell the server to load them all
         pass
 
     def saveConnectionDetails(self, remote):
         self.remote = remote
 
-    def establishConnection(self):
+    def establishConnection(self, featureset):
         # The format of "remote" must be "server:port"
         try:
             [host, port] = self.remote.split(":")
@@ -339,9 +354,12 @@ class BitBakeXMLRPCClient(BitBakeBaseServer):
             s.close()
         except:
             return None
-        self.serverImpl = XMLRPCProxyServer(host, port)
+        try:
-        self.connection = BitBakeXMLRPCServerConnection(self.serverImpl, (ip, 0), self.observer_only)
+            self.serverImpl = XMLRPCProxyServer(host, port)
-        return self.connection.connect()
+            self.connection = BitBakeXMLRPCServerConnection(self.serverImpl, (ip, 0), self.observer_only, featureset)
+            return self.connection.connect()
+        except Exception as e:
+            bb.fatal("Could not connect to server at %s:%s (%s)" % (host, port, str(e)))
 
     def endSession(self):
         self.connection.removeClient()

bitbake/lib/bb/tests/data.py

@@ -27,21 +27,21 @@ import bb.data
 class DataExpansions(unittest.TestCase):
     def setUp(self):
         self.d = bb.data.init()
-        self.d["foo"] = "value of foo"
+        self.d["foo"] = "value_of_foo"
-        self.d["bar"] = "value of bar"
+        self.d["bar"] = "value_of_bar"
-        self.d["value of foo"] = "value of 'value of foo'"
+        self.d["value_of_foo"] = "value_of_'value_of_foo'"
 
     def test_one_var(self):
         val = self.d.expand("${foo}")
-        self.assertEqual(str(val), "value of foo")
+        self.assertEqual(str(val), "value_of_foo")
 
     def test_indirect_one_var(self):
         val = self.d.expand("${${foo}}")
-        self.assertEqual(str(val), "value of 'value of foo'")
+        self.assertEqual(str(val), "value_of_'value_of_foo'")
 
     def test_indirect_and_another(self):
         val = self.d.expand("${${foo}} ${bar}")
-        self.assertEqual(str(val), "value of 'value of foo' value of bar")
+        self.assertEqual(str(val), "value_of_'value_of_foo' value_of_bar")
 
     def test_python_snippet(self):
         val = self.d.expand("${@5*12}")
@@ -49,11 +49,11 @@ class DataExpansions(unittest.TestCase):
 
     def test_expand_in_python_snippet(self):
         val = self.d.expand("${@'boo ' + '${foo}'}")
-        self.assertEqual(str(val), "boo value of foo")
+        self.assertEqual(str(val), "boo value_of_foo")
 
     def test_python_snippet_getvar(self):
         val = self.d.expand("${@d.getVar('foo', True) + ' ${bar}'}")
-        self.assertEqual(str(val), "value of foo value of bar")
+        self.assertEqual(str(val), "value_of_foo value_of_bar")
 
     def test_python_snippet_syntax_error(self):
         self.d.setVar("FOO", "${@foo = 5}")
@@ -70,7 +70,7 @@ class DataExpansions(unittest.TestCase):
 
     def test_value_containing_value(self):
         val = self.d.expand("${@d.getVar('foo', True) + ' ${bar}'}")
-        self.assertEqual(str(val), "value of foo value of bar")
+        self.assertEqual(str(val), "value_of_foo value_of_bar")
 
     def test_reference_undefined_var(self):
         val = self.d.expand("${undefinedvar} meh")
@@ -109,7 +109,7 @@ class DataExpansions(unittest.TestCase):
 
     def test_rename(self):
         self.d.renameVar("foo", "newfoo")
-        self.assertEqual(self.d.getVar("newfoo"), "value of foo")
+        self.assertEqual(self.d.getVar("newfoo"), "value_of_foo")
         self.assertEqual(self.d.getVar("foo"), None)
 
     def test_deletion(self):
@@ -118,17 +118,17 @@ class DataExpansions(unittest.TestCase):
 
     def test_keys(self):
         keys = self.d.keys()
-        self.assertEqual(keys, ['value of foo', 'foo', 'bar'])
+        self.assertEqual(keys, ['value_of_foo', 'foo', 'bar'])
 
 class TestNestedExpansions(unittest.TestCase):
     def setUp(self):
         self.d = bb.data.init()
         self.d["foo"] = "foo"
         self.d["bar"] = "bar"
-        self.d["value of foobar"] = "187"
+        self.d["value_of_foobar"] = "187"
 
     def test_refs(self):
-        val = self.d.expand("${value of ${foo}${bar}}")
+        val = self.d.expand("${value_of_${foo}${bar}}")
         self.assertEqual(str(val), "187")
 
     #def test_python_refs(self):
@@ -154,11 +154,11 @@ class TestNestedExpansions(unittest.TestCase):
     #    self.assertEqual(str(val), str(depth + 1))
 
     def test_mixed(self):
-        val = self.d.expand("${value of ${@('${foo}'+'bar')[0:3]}${${@'BAR'.lower()}}}")
+        val = self.d.expand("${value_of_${@('${foo}'+'bar')[0:3]}${${@'BAR'.lower()}}}")
         self.assertEqual(str(val), "187")
 
     def test_runtime(self):
-        val = self.d.expand("${${@'value of' + ' f'+'o'+'o'+'b'+'a'+'r'}}")
+        val = self.d.expand("${${@'value_of' + '_f'+'o'+'o'+'b'+'a'+'r'}}")
         self.assertEqual(str(val), "187")
 
 class TestMemoize(unittest.TestCase):

bitbake/lib/bb/ui/buildinfohelper.py

@@ -0,0 +1,738 @@
+#
+# BitBake ToasterUI Implementation
+#
+# Copyright (C) 2013        Intel Corporation
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+import datetime
+import sys
+import bb
+import re
+import subprocess
+
+
+os.environ.setdefault("DJANGO_SETTINGS_MODULE", "toaster.toastermain.settings")
+
+import toaster.toastermain.settings as toaster_django_settings
+from toaster.orm.models import Build, Task, Recipe, Layer_Version, Layer, Target, LogMessage
+from toaster.orm.models import Variable, VariableHistory
+from toaster.orm.models import Target_Package, Build_Package, Build_File
+from toaster.orm.models import Task_Dependency, Build_Package_Dependency
+from toaster.orm.models import Target_Package_Dependency, Recipe_Dependency
+from bb.msg import BBLogFormatter as format
+
+class ORMWrapper(object):
+    """ This class creates the dictionaries needed to store information in the database
+        following the format defined by the Django models. It is also used to save this
+        information in the database.
+    """
+
+    def __init__(self):
+        pass
+
+
+    def create_build_object(self, build_info):
+
+        build = Build.objects.create(
+                                    machine=build_info['machine'],
+                                    image_fstypes=build_info['image_fstypes'],
+                                    distro=build_info['distro'],
+                                    distro_version=build_info['distro_version'],
+                                    started_on=build_info['started_on'],
+                                    completed_on=build_info['completed_on'],
+                                    cooker_log_path=build_info['cooker_log_path'],
+                                    build_name=build_info['build_name'],
+                                    bitbake_version=build_info['bitbake_version'])
+
+        return build
+
+    def create_target_objects(self, target_info):
+        targets = []
+        for tgt_name in target_info['targets']:
+            tgt_object = Target.objects.create( build = target_info['build'],
+                                    target = tgt_name,
+                                    is_image = False,
+                                    file_name = "",
+                                    file_size = 0);
+            targets.append(tgt_object)
+        return targets
+
+    def update_build_object(self, build, errors, warnings, taskfailures):
+
+        outcome = Build.SUCCEEDED
+        if errors or taskfailures:
+            outcome = Build.FAILED
+
+        build.completed_on = datetime.datetime.now()
+        build.errors_no = errors
+        build.warnings_no = warnings
+        build.outcome = outcome
+        build.save()
+
+
+    def get_update_task_object(self, task_information):
+        task_object, created = Task.objects.get_or_create(
+                                build=task_information['build'],
+                                recipe=task_information['recipe'],
+                                task_name=task_information['task_name'],
+                                )
+
+        for v in vars(task_object):
+            if v in task_information.keys():
+                vars(task_object)[v] = task_information[v]
+        # if we got covered by a setscene task, we're SSTATE
+        if task_object.outcome == Task.OUTCOME_COVERED and 1 == Task.objects.filter(task_executed=True, build = task_object.build, recipe = task_object.recipe, task_name=task_object.task_name+"_setscene").count():
+            task_object.outcome = Task.OUTCOME_SSTATE
+            outcome_task_setscene = Task.objects.get(task_executed=True, build = task_object.build,
+                                    recipe = task_object.recipe, task_name=task_object.task_name+"_setscene").outcome
+            if outcome_task_setscene == Task.OUTCOME_SUCCESS:
+                task_object.sstate_result = Task.SSTATE_RESTORED
+            elif outcome_task_setscene == Task.OUTCOME_FAILED:
+                task_object.sstate_result = Task.SSTATE_FAILED
+
+        # mark down duration if we have a start time
+        if 'start_time' in task_information.keys():
+            duration = datetime.datetime.now() - task_information['start_time']
+            task_object.elapsed_time = duration.total_seconds()
+
+        task_object.save()
+        return task_object
+
+
+    def get_update_recipe_object(self, recipe_information):
+
+        recipe_object, created = Recipe.objects.get_or_create(
+                                         layer_version=recipe_information['layer_version'],
+                                         file_path=recipe_information['file_path'])
+
+        for v in vars(recipe_object):
+            if v in recipe_information.keys():
+                vars(recipe_object)[v] = recipe_information[v]
+
+        recipe_object.save()
+
+        return recipe_object
+
+    def get_layer_version_object(self, layer_version_information):
+
+        layer_version_object = Layer_Version.objects.get_or_create(
+                                    layer = layer_version_information['layer'],
+                                    branch = layer_version_information['branch'],
+                                    commit = layer_version_information['commit'],
+                                    priority = layer_version_information['priority']
+                                    )
+
+        layer_version_object[0].save()
+
+        return layer_version_object[0]
+
+    def get_update_layer_object(self, layer_information):
+
+        layer_object = Layer.objects.get_or_create(
+                                name=layer_information['name'],
+                                local_path=layer_information['local_path'],
+                                layer_index_url=layer_information['layer_index_url'])
+        layer_object[0].save()
+
+        return layer_object[0]
+
+
+    def save_target_package_information(self, target_obj, packagedict, bldpkgs, recipes):
+        for p in packagedict:
+            packagedict[p]['object'] = Target_Package.objects.create( target = target_obj,
+                                        name = p,
+                                        size = packagedict[p]['size'])
+            if p in bldpkgs:
+                packagedict[p]['object'].version = bldpkgs[p]['version']
+                packagedict[p]['object'].recipe =  recipes[bldpkgs[p]['pn']]
+                packagedict[p]['object'].save()
+
+        for p in packagedict:
+            for (px,deptype) in packagedict[p]['depends']:
+                Target_Package_Dependency.objects.create( package = packagedict[p]['object'],
+                                        depends_on = packagedict[px]['object'],
+                                        dep_type = deptype);
+
+
+    def create_logmessage(self, log_information):
+        log_object = LogMessage.objects.create(
+                        build = log_information['build'],
+                        level = log_information['level'],
+                        message = log_information['message'])
+
+        for v in vars(log_object):
+            if v in log_information.keys():
+                vars(log_object)[v] = log_information[v]
+
+        return log_object.save()
+
+
+    def save_build_package_information(self, build_obj, package_info, recipes):
+        # create and save the object
+        bp_object = Build_Package.objects.create( build = build_obj,
+                                       recipe = recipes[package_info['PN']],
+                                       name = package_info['PKG'],
+                                       version = package_info['PKGV'],
+                                       revision = package_info['PKGR'],
+                                       summary = package_info['SUMMARY'],
+                                       description = package_info['DESCRIPTION'],
+                                       size = int(package_info['PKGSIZE']) * 1024,
+                                       section = package_info['SECTION'],
+                                       license = package_info['LICENSE'],
+                                       )
+        # save any attached file information
+        for path in package_info['FILES_INFO']:
+                fo = Build_File.objects.create( bpackage = bp_object,
+                                        path = path,
+                                        size = package_info['FILES_INFO'][path] )
+
+        # save soft dependency information
+        if 'RDEPENDS' in package_info and package_info['RDEPENDS']:
+            for p in bb.utils.explode_deps(package_info['RDEPENDS']):
+                Build_Package_Dependency.objects.get_or_create( package = bp_object,
+                    depends_on = p, dep_type = Build_Package_Dependency.TYPE_RDEPENDS)
+        if 'RPROVIDES' in package_info and package_info['RPROVIDES']:
+            for p in bb.utils.explode_deps(package_info['RPROVIDES']):
+                Build_Package_Dependency.objects.get_or_create( package = bp_object,
+                    depends_on = p, dep_type = Build_Package_Dependency.TYPE_RPROVIDES)
+        if 'RRECOMMENDS' in package_info and package_info['RRECOMMENDS']:
+            for p in bb.utils.explode_deps(package_info['RRECOMMENDS']):
+                Build_Package_Dependency.objects.get_or_create( package = bp_object,
+                    depends_on = p, dep_type = Build_Package_Dependency.TYPE_RRECOMMENDS)
+        if 'RSUGGESTS' in package_info and package_info['RSUGGESTS']:
+            for p in bb.utils.explode_deps(package_info['RSUGGESTS']):
+                Build_Package_Dependency.objects.get_or_create( package = bp_object,
+                    depends_on = p, dep_type = Build_Package_Dependency.TYPE_RSUGGESTS)
+        if 'RREPLACES' in package_info and package_info['RREPLACES']:
+            for p in bb.utils.explode_deps(package_info['RREPLACES']):
+                Build_Package_Dependency.objects.get_or_create( package = bp_object,
+                    depends_on = p, dep_type = Build_Package_Dependency.TYPE_RREPLACES)
+        if 'RCONFLICTS' in package_info and package_info['RCONFLICTS']:
+            for p in bb.utils.explode_deps(package_info['RCONFLICTS']):
+                Build_Package_Dependency.objects.get_or_create( package = bp_object,
+                    depends_on = p, dep_type = Build_Package_Dependency.TYPE_RCONFLICTS)
+
+        return bp_object
+
+    def save_build_variables(self, build_obj, vardump):
+        for k in vardump:
+            if not bool(vardump[k]['func']):
+                value = vardump[k]['v'];
+                if value is None:
+                    value = ''
+                desc = vardump[k]['doc'];
+                if desc is None:
+                    var_words = [word for word in k.split('_')]
+                    root_var = "_".join([word for word in var_words if word.isupper()])
+                    if root_var and root_var != k and root_var in vardump:
+                        desc = vardump[root_var]['doc']
+                if desc is None:
+                    desc = ''
+                variable_obj = Variable.objects.create( build = build_obj,
+                    variable_name = k,
+                    variable_value = value,
+                    description = desc)
+                for vh in vardump[k]['history']:
+                    VariableHistory.objects.create( variable = variable_obj,
+                            file_name = vh['file'],
+                            line_number = vh['line'],
+                            operation = vh['op'])
+
+class BuildInfoHelper(object):
+    """ This class gathers the build information from the server and sends it
+        towards the ORM wrapper for storing in the database
+        It is instantiated once per build
+        Keeps in memory all data that needs matching before writing it to the database
+    """
+
+    def __init__(self, server, has_build_history = False):
+        self._configure_django()
+        self.internal_state = {}
+        self.task_order = 0
+        self.server = server
+        self.orm_wrapper = ORMWrapper()
+        self.has_build_history = has_build_history
+        self.tmp_dir = self.server.runCommand(["getVariable", "TMPDIR"])[0]
+
+    def _configure_django(self):
+        # Add toaster to sys path for importing modules
+        sys.path.append(os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), 'toaster'))
+
+    ###################
+    ## methods to convert event/external info into objects that the ORM layer uses
+
+    def _get_layer_dict(self, layer_path):
+
+        layer_info = {}
+        layer_name = layer_path.split('/')[-1]
+        layer_url = 'http://layers.openembedded.org/layerindex/layer/{layer}/'
+        layer_url_name = self._get_url_map_name(layer_name)
+
+        layer_info['name'] = layer_name
+        layer_info['local_path'] = layer_path
+        layer_info['layer_index_url'] = layer_url.format(layer=layer_url_name)
+
+        return layer_info
+
+    def _get_url_map_name(self, layer_name):
+        """ Some layers have a different name on openembedded.org site,
+            this method returns the correct name to use in the URL
+        """
+
+        url_name = layer_name
+        url_mapping = {'meta': 'openembedded-core'}
+
+        for key in url_mapping.keys():
+            if key == layer_name:
+                url_name = url_mapping[key]
+
+        return url_name
+
+    def _get_layer_information(self):
+
+        layer_info = {}
+
+        return layer_info
+
+    def _get_layer_version_information(self, layer_object):
+
+        layer_version_info = {}
+        layer_version_info['build'] = self.internal_state['build']
+        layer_version_info['layer'] = layer_object
+        layer_version_info['branch'] = self._get_git_branch(layer_object.local_path)
+        layer_version_info['commit'] = self._get_git_revision(layer_object.local_path)
+        layer_version_info['priority'] = 0
+
+        return layer_version_info
+
+
+    def _get_git_branch(self, layer_path):
+        branch = subprocess.Popen("git symbolic-ref HEAD 2>/dev/null ", cwd=layer_path, shell=True, stdout=subprocess.PIPE).communicate()[0]
+        branch = branch.replace('refs/heads/', '').rstrip()
+        return branch
+
+    def _get_git_revision(self, layer_path):
+        revision = subprocess.Popen("git rev-parse HEAD 2>/dev/null ", cwd=layer_path, shell=True, stdout=subprocess.PIPE).communicate()[0].rstrip()
+        return revision
+
+
+    def _get_build_information(self):
+        build_info = {}
+        # Generate an identifier for each new build
+
+        build_info['machine'] = self.server.runCommand(["getVariable", "MACHINE"])[0]
+        build_info['distro'] = self.server.runCommand(["getVariable", "DISTRO"])[0]
+        build_info['distro_version'] = self.server.runCommand(["getVariable", "DISTRO_VERSION"])[0]
+        build_info['started_on'] = datetime.datetime.now()
+        build_info['completed_on'] = datetime.datetime.now()
+        build_info['image_fstypes'] = self._remove_redundant(self.server.runCommand(["getVariable", "IMAGE_FSTYPES"])[0] or "")
+        build_info['cooker_log_path'] = self.server.runCommand(["getVariable", "BB_CONSOLELOG"])[0]
+        build_info['build_name'] = self.server.runCommand(["getVariable", "BUILDNAME"])[0]
+        build_info['bitbake_version'] = self.server.runCommand(["getVariable", "BB_VERSION"])[0]
+
+        return build_info
+
+    def _get_task_information(self, event, recipe):
+
+
+        task_information = {}
+        task_information['build'] = self.internal_state['build']
+        task_information['outcome'] = Task.OUTCOME_NA
+        task_information['recipe'] = recipe
+        task_information['task_name'] = event.taskname
+        try:
+            # some tasks don't come with a hash. and that's ok
+            task_information['sstate_checksum'] = event.taskhash
+        except AttributeError:
+            pass
+        return task_information
+
+    def _get_layer_version_for_path(self, path):
+        def _slkey(layer_version):
+            return len(layer_version.layer.local_path)
+
+        # Heuristics: we always match recipe to the deepest layer path that
+        # we can match to the recipe file path
+        for bl in sorted(self.internal_state['layer_versions'], reverse=True, key=_slkey):
+            if (path.startswith(bl.layer.local_path)):
+                return bl
+
+        #TODO: if we get here, we didn't read layers correctly
+        assert False
+        return None
+
+    def _get_recipe_information_from_build_event(self, event):
+
+        layer_version_obj = self._get_layer_version_for_path(re.split(':', event.taskfile)[-1])
+
+        recipe_info = {}
+        recipe_info['layer_version'] = layer_version_obj
+        recipe_info['file_path'] = re.split(':', event.taskfile)[-1]
+
+        return recipe_info
+
+    def _get_task_build_stats(self, task_object):
+        bs_path = self._get_path_information(task_object)
+        for bp in bs_path:  # TODO: split for each target
+            task_build_stats = self._get_build_stats_from_file(bp, task_object.task_name)
+
+        return task_build_stats
+
+    def _get_path_information(self, task_object):
+        build_stats_format = "{tmpdir}/buildstats/{target}-{machine}/{buildname}/{package}/"
+        build_stats_path = []
+
+        for t in self.internal_state['targets']:
+            target = t.target
+            machine = self.internal_state['build'].machine
+            buildname = self.internal_state['build'].build_name
+            pe, pv = task_object.recipe.version.split(":",1)
+            if len(pe) > 0:
+                package = task_object.recipe.name + "-" + pe + "_" + pv
+            else:
+                package = task_object.recipe.name + "-" + pv
+
+            build_stats_path.append(build_stats_format.format(tmpdir=self.tmp_dir, target=target,
+                                                     machine=machine, buildname=buildname,
+                                                     package=package))
+
+        return build_stats_path
+
+    def _get_build_stats_from_file(self, bs_path, task_name):
+
+        task_bs_filename = str(bs_path) + str(task_name)
+        task_bs = open(task_bs_filename, 'r')
+
+        cpu_usage = 0
+        disk_io = 0
+        startio = ''
+        endio = ''
+
+        for line in task_bs.readlines():
+            if line.startswith('CPU usage: '):
+                cpu_usage = line[11:]
+            elif line.startswith('EndTimeIO: '):
+                endio = line[11:]
+            elif line.startswith('StartTimeIO: '):
+                startio = line[13:]
+
+        task_bs.close()
+
+        if startio and endio:
+            disk_io = int(endio.strip('\n ')) - int(startio.strip('\n '))
+
+        if cpu_usage:
+            cpu_usage = float(cpu_usage.strip('% \n'))
+
+        task_build_stats = {'cpu_usage': cpu_usage, 'disk_io': disk_io}
+
+        return task_build_stats
+
+    def _remove_redundant(self, string):
+        ret = []
+        for i in string.split():
+            if i not in ret:
+                ret.append(i)
+        return " ".join(ret)
+
+
+    ################################
+    ## external available methods to store information
+
+    def store_layer_info(self):
+        layers = self.server.runCommand(["getVariable", "BBLAYERS"])[0].strip().split(" ")
+        self.internal_state['layers'] = []
+        for layer_path in { l for l in layers if len(l) }:
+            layer_information = self._get_layer_dict(layer_path)
+            self.internal_state['layers'].append(self.orm_wrapper.get_update_layer_object(layer_information))
+
+    def store_started_build(self, event):
+
+        build_information = self._get_build_information()
+
+        build_obj = self.orm_wrapper.create_build_object(build_information)
+        self.internal_state['build'] = build_obj
+
+        # create target information
+        target_information = {}
+        target_information['targets'] = event.getPkgs()
+        target_information['build'] = build_obj
+
+        self.internal_state['targets'] = self.orm_wrapper.create_target_objects(target_information)
+
+        # Load layer information for the build
+        self.internal_state['layer_versions'] = []
+        for layer_object in self.internal_state['layers']:
+            layer_version_information = self._get_layer_version_information(layer_object)
+            self.internal_state['layer_versions'].append(self.orm_wrapper.get_layer_version_object(layer_version_information))
+
+        del self.internal_state['layers']
+        # Save build configuration
+        self.orm_wrapper.save_build_variables(build_obj, self.server.runCommand(["getAllKeysWithFlags", ["doc", "func"]])[0])
+
+
+    def update_build_information(self, event, errors, warnings, taskfailures):
+        if 'build' in self.internal_state:
+            self.orm_wrapper.update_build_object(self.internal_state['build'], errors, warnings, taskfailures)
+
+    def store_started_task(self, event):
+        identifier = event.taskfile + event.taskname
+
+        recipe_information = self._get_recipe_information_from_build_event(event)
+        recipe = self.orm_wrapper.get_update_recipe_object(recipe_information)
+
+        task_information = self._get_task_information(event, recipe)
+        task_information['outcome'] = Task.OUTCOME_NA
+
+        if isinstance(event, bb.runqueue.runQueueTaskSkipped):
+            task_information['task_executed'] = False
+            if event.reason == "covered":
+                task_information['outcome'] = Task.OUTCOME_COVERED
+            if event.reason == "existing":
+                task_information['outcome'] = Task.OUTCOME_EXISTING
+        else:
+            task_information['task_executed'] = True
+            if 'noexec' in vars(event) and event.noexec == True:
+                task_information['script_type'] = Task.CODING_NOEXEC
+
+        self.task_order += 1
+        task_information['order'] = self.task_order
+        task_obj = self.orm_wrapper.get_update_task_object(task_information)
+
+        self.internal_state[identifier] = {'start_time': datetime.datetime.now()}
+
+    def update_and_store_task(self, event):
+        identifier = event.taskfile + event.taskname
+        recipe_information = self._get_recipe_information_from_build_event(event)
+        recipe = self.orm_wrapper.get_update_recipe_object(recipe_information)
+        task_information = self._get_task_information(event,recipe)
+        try:
+            task_information['start_time'] = self.internal_state[identifier]['start_time']
+        except:
+            pass
+
+        if 'logfile' in vars(event):
+            task_information['logfile'] = event.logfile
+
+        if '_message' in vars(event):
+            task_information['message'] = event._message
+
+        if 'taskflags' in vars(event):
+            # with TaskStarted, we get even more information
+            if 'python' in event.taskflags.keys() and event.taskflags['python'] == '1':
+                task_information['script_type'] = Task.CODING_PYTHON
+            else:
+                task_information['script_type'] = Task.CODING_SHELL
+
+        if isinstance(event, (bb.runqueue.runQueueTaskCompleted, bb.runqueue.sceneQueueTaskCompleted)):
+            task_information['outcome'] = Task.OUTCOME_SUCCESS
+            task_build_stats = self._get_task_build_stats(self.orm_wrapper.get_update_task_object(task_information))
+            task_information['cpu_usage'] = task_build_stats['cpu_usage']
+            task_information['disk_io'] = task_build_stats['disk_io']
+            del self.internal_state[identifier]
+
+        if isinstance(event, (bb.runqueue.runQueueTaskFailed, bb.runCommand.sceneQueueTaskFailed)):
+            task_information['outcome'] = Task.OUTCOME_FAILED
+            del self.internal_state[identifier]
+
+        self.orm_wrapper.get_update_task_object(task_information)
+
+
+    def read_target_package_dep_data(self, event):
+        # for all targets
+        for target in self.internal_state['targets']:
+            # verify that we have something to read
+            if not target.is_image or not self.has_build_history:
+                print "not collecting package info ", target.is_image, self.has_build_history
+                break
+
+            # TODO this is a temporary replication of the code in buildhistory.bbclass
+            # This MUST be changed to query the actual BUILD_DIR_IMAGE in the target context when
+            # the capability will be implemented in Bitbake
+
+            MACHINE_ARCH, error = self.server.runCommand(['getVariable', 'MACHINE_ARCH'])
+            TCLIBC, error = self.server.runCommand(['getVariable', 'TCLIBC'])
+            BUILDHISTORY_DIR, error = self.server.runCommand(['getVariable', 'BUILDHISTORY_DIR'])
+            BUILDHISTORY_DIR_IMAGE = "%s/images/%s/%s/%s" % (BUILDHISTORY_DIR, MACHINE_ARCH, TCLIBC, target.target)
+
+            self.internal_state['packages'] = {}
+
+            with open("%s/installed-package-sizes.txt" % BUILDHISTORY_DIR_IMAGE, "r") as fin:
+                for line in fin:
+                    line = line.rstrip(";")
+                    psize, px = line.split("\t")
+                    punit, pname = px.split(" ")
+                    self.internal_state['packages'][pname.strip()] = {'size':int(psize)*1024, 'depends' : []}
+
+            with open("%s/depends.dot" % BUILDHISTORY_DIR_IMAGE, "r") as fin:
+                p = re.compile(r' -> ')
+                dot = re.compile(r'.*style=dotted')
+                for line in fin:
+                    line = line.rstrip(';')
+                    linesplit = p.split(line)
+                    if len(linesplit) == 2:
+                        pname = linesplit[0].rstrip('"').strip('"')
+                        dependsname = linesplit[1].split(" ")[0].strip().strip(";").strip('"').rstrip('"')
+                        deptype = Target_Package_Dependency.TYPE_DEPENDS
+                        if dot.match(line):
+                            deptype = Target_Package_Dependency.TYPE_RECOMMENDS
+                        if not pname in self.internal_state['packages']:
+                            self.internal_state['packages'][pname] = {'size': 0, 'depends' : []}
+                        if not dependsname in self.internal_state['packages']:
+                            self.internal_state['packages'][dependsname] = {'size': 0, 'depends' : []}
+                        self.internal_state['packages'][pname]['depends'].append((dependsname, deptype))
+
+            self.orm_wrapper.save_target_package_information(target,
+                        self.internal_state['packages'],
+                        self.internal_state['bldpkgs'], self.internal_state['recipes'])
+
+
+    def store_dependency_information(self, event):
+        # save layer version priorities
+        if 'layer-priorities' in event._depgraph.keys():
+            for lv in event._depgraph['layer-priorities']:
+                (name, path, regexp, priority) = lv
+                layer_version_obj = self._get_layer_version_for_path(path[1:]) # paths start with a ^
+                assert layer_version_obj is not None
+                layer_version_obj.priority = priority
+                layer_version_obj.save()
+
+        # save build time package information
+        self.internal_state['bldpkgs'] = {}
+        for pkg  in event._depgraph['packages']:
+            self.internal_state['bldpkgs'][pkg] = event._depgraph['packages'][pkg]
+
+        # save recipe information
+        self.internal_state['recipes'] = {}
+        for pn in event._depgraph['pn']:
+
+            file_name = re.split(':', event._depgraph['pn'][pn]['filename'])[-1]
+            layer_version_obj = self._get_layer_version_for_path(re.split(':', file_name)[-1])
+
+            assert layer_version_obj is not None
+
+            recipe_info = {}
+            recipe_info['name'] = pn
+            recipe_info['version'] = event._depgraph['pn'][pn]['version']
+            recipe_info['layer_version'] = layer_version_obj
+            recipe_info['summary'] = event._depgraph['pn'][pn]['summary']
+            recipe_info['license'] = event._depgraph['pn'][pn]['license']
+            recipe_info['description'] = event._depgraph['pn'][pn]['description']
+            recipe_info['section'] = event._depgraph['pn'][pn]['section']
+            recipe_info['licensing_info'] = 'Not Available'
+            recipe_info['homepage'] = event._depgraph['pn'][pn]['homepage']
+            recipe_info['bugtracker'] = event._depgraph['pn'][pn]['bugtracker']
+            recipe_info['file_path'] = file_name
+            recipe = self.orm_wrapper.get_update_recipe_object(recipe_info)
+            if 'inherits' in event._depgraph['pn'][pn].keys():
+                recipe.is_image = True in map(lambda x: x.endswith('image.bbclass'), event._depgraph['pn'][pn]['inherits'])
+            else:
+                recipe.is_image = False
+            if recipe.is_image:
+                for t in self.internal_state['targets']:
+                    if pn == t.target:
+                        t.is_image = True
+                        t.save()
+            self.internal_state['recipes'][pn] = recipe
+
+        # save recipe dependency
+        # buildtime
+        for recipe in event._depgraph['depends']:
+            try:
+                target = self.internal_state['recipes'][recipe]
+                for dep in event._depgraph['depends'][recipe]:
+                    dependency = self.internal_state['recipes'][dep]
+                    Recipe_Dependency.objects.get_or_create( recipe = target,
+                            depends_on = dependency, dep_type = Recipe_Dependency.TYPE_DEPENDS)
+            except KeyError:    # we'll not get recipes for key w/ values listed in ASSUME_PROVIDED
+                pass
+
+        # runtime
+        for recipe in event._depgraph['rdepends-pn']:
+            try:
+                target = self.internal_state['recipes'][recipe]
+                for dep in event._depgraph['rdepends-pn'][recipe]:
+                    dependency = self.internal_state['recipes'][dep]
+                    Recipe_Dependency.objects.get_or_create( recipe = target,
+                            depends_on = dependency, dep_type = Recipe_Dependency.TYPE_RDEPENDS)
+
+            except KeyError:    # we'll not get recipes for key w/ values listed in ASSUME_PROVIDED
+                pass
+
+        # save all task information
+        def _save_a_task(taskdesc):
+            spec = re.split(r'\.', taskdesc);
+            pn = ".".join(spec[0:-1])
+            taskname = spec[-1]
+            e = event
+            e.taskname = pn
+            recipe = self.internal_state['recipes'][pn]
+            task_info = self._get_task_information(e, recipe)
+            task_info['task_name'] = taskname
+            task_obj = self.orm_wrapper.get_update_task_object(task_info)
+            return task_obj
+
+        for taskdesc in event._depgraph['tdepends']:
+            target = _save_a_task(taskdesc)
+            for taskdesc1 in event._depgraph['tdepends'][taskdesc]:
+                dep = _save_a_task(taskdesc1)
+                Task_Dependency.objects.get_or_create( task = target, depends_on = dep )
+
+    def store_build_package_information(self, event):
+        package_info = event.data
+        self.orm_wrapper.save_build_package_information(self.internal_state['build'],
+                            package_info,
+                            self.internal_state['recipes'],
+                            )
+
+    def _store_log_information(self, level, text):
+        log_information = {}
+        log_information['build'] = self.internal_state['build']
+        log_information['level'] = level
+        log_information['message'] = text
+        self.orm_wrapper.create_logmessage(log_information)
+
+    def store_log_info(self, text):
+        self._store_log_information(LogMessage.INFO, text)
+
+    def store_log_warn(self, text):
+        self._store_log_information(LogMessage.WARNING, text)
+
+    def store_log_error(self, text):
+        self._store_log_information(LogMessage.ERROR, text)
+
+    def store_log_event(self, event):
+        # look up license files info from insane.bbclass
+        m = re.match("([^:]*): md5 checksum matched for ([^;]*)", event.msg)
+        if m:
+            (pn, fn) = m.groups()
+            self.internal_state['recipes'][pn].licensing_info = fn
+            self.internal_state['recipes'][pn].save()
+
+        if event.levelno < format.WARNING:
+            return
+        if not 'build' in self.internal_state:
+            return
+        log_information = {}
+        log_information['build'] = self.internal_state['build']
+        if event.levelno >= format.ERROR:
+            log_information['level'] = LogMessage.ERROR
+        elif event.levelno == format.WARNING:
+            log_information['level'] = LogMessage.WARNING
+        log_information['message'] = event.msg
+        log_information['pathname'] = event.pathname
+        log_information['lineno'] = event.lineno
+        self.orm_wrapper.create_logmessage(log_information)
+

bitbake/lib/bb/ui/crumbs/builder.py

@@ -31,6 +31,7 @@ import re
 import logging
 import sys
 import signal
+import time
 from bb.ui.crumbs.imageconfigurationpage import ImageConfigurationPage
 from bb.ui.crumbs.recipeselectionpage import RecipeSelectionPage
 from bb.ui.crumbs.packageselectionpage import PackageSelectionPage
@@ -197,7 +198,7 @@ class Configuration:
         handler.set_var_in_file("BBLAYERS", self.layers, "bblayers.conf")
         # local.conf
         if not defaults:
-            handler.set_var_in_file("MACHINE", self.curr_mach, "local.conf")
+            handler.early_assign_var_in_file("MACHINE", self.curr_mach, "local.conf")
         handler.set_var_in_file("DISTRO", self.curr_distro, "local.conf")
         handler.set_var_in_file("DL_DIR", self.dldir, "local.conf")
         handler.set_var_in_file("SSTATE_DIR", self.sstatedir, "local.conf")
@@ -217,7 +218,7 @@ class Configuration:
         handler.set_var_in_file("SDKMACHINE", self.curr_sdk_machine, "local.conf")
         handler.set_var_in_file("CONF_VERSION", self.conf_version, "local.conf")
         handler.set_var_in_file("LCONF_VERSION", self.lconf_version, "bblayers.conf")
-        handler.set_var_in_file("EXTRA_SETTING", self.extra_setting, "local.conf")
+        handler.set_extra_config(self.extra_setting)
         handler.set_var_in_file("TOOLCHAIN_BUILD", self.toolchain_build, "local.conf")
         handler.set_var_in_file("IMAGE_FSTYPES", self.image_fstypes, "local.conf")
         if not defaults:
@@ -1466,3 +1467,10 @@ class Builder(gtk.Window):
 
     def get_topdir(self):
         return self.handler.get_topdir()
+
+    def wait(self, delay):
+        time_start = time.time()
+        time_end = time_start + delay
+        while time_end > time.time():
+            while gtk.events_pending():
+                gtk.main_iteration()

bitbake/lib/bb/ui/crumbs/hig/advancedsettingsdialog.py

@@ -234,7 +234,10 @@ class AdvancedSettingsDialog (CrumbsDialog, SettingsUIHelper):
             article = ""
             if image_type.startswith(("a", "e", "i", "o", "u")):
                 article = "n"
-            self.image_types_checkbuttons[image_type].set_tooltip_text("Build a%s %s image" % (article, image_type))
+            if image_type == "live":
+                self.image_types_checkbuttons[image_type].set_tooltip_text("Build iso and hddimg images")
+            else:
+                self.image_types_checkbuttons[image_type].set_tooltip_text("Build a%s %s image" % (article, image_type))
             table.attach(self.image_types_checkbuttons[image_type], j - 1, j + 3, i, i + 1)
             if image_type in self.configuration.image_fstypes.split():
                 self.image_types_checkbuttons[image_type].set_active(True)

bitbake/lib/bb/ui/crumbs/hobeventhandler.py

@@ -21,6 +21,7 @@
 
 import gobject
 import logging
+import ast
 from bb.ui.crumbs.runningbuild import RunningBuild
 
 class HobHandler(gobject.GObject):
@@ -315,7 +316,7 @@ class HobHandler(gobject.GObject):
 
     def set_machine(self, machine):
         if machine:
-            self.set_var_in_file("MACHINE", machine, "local.conf")
+            self.early_assign_var_in_file("MACHINE", machine, "local.conf")
 
     def set_sdk_machine(self, sdk_machine):
         self.set_var_in_file("SDKMACHINE", sdk_machine, "local.conf")
@@ -357,7 +358,20 @@ class HobHandler(gobject.GObject):
     def set_incompatible_license(self, incompat_license):
         self.set_var_in_file("INCOMPATIBLE_LICENSE", incompat_license, "local.conf")
 
+    def set_extra_setting(self, extra_setting):
+        self.set_var_in_file("EXTRA_SETTING", extra_setting, "local.conf")
+
     def set_extra_config(self, extra_setting):
+        old_extra_setting = ast.literal_eval(self.runCommand(["getVariable", "EXTRA_SETTING"]) or "{}")
+        if extra_setting:
+            self.set_var_in_file("EXTRA_SETTING", extra_setting, "local.conf")
+        else:
+            self.remove_var_from_file("EXTRA_SETTING")
+
+        #remove not needed settings from conf
+        for key in old_extra_setting:
+            if key not in extra_setting:
+                self.remove_var_from_file(key)
         for key in extra_setting.keys():
             value = extra_setting[key]
             self.set_var_in_file(key, value, "local.conf")
@@ -472,6 +486,14 @@ class HobHandler(gobject.GObject):
         self.server.runCommand(["setVarFile", var, val, default_file, "set"])
         self.runCommand(["disableDataTracking"])
 
+    def early_assign_var_in_file(self, var, val, default_file=None):
+        self.runCommand(["enableDataTracking"])
+        self.server.runCommand(["setVarFile", var, val, default_file, "earlyAssign"])
+        self.runCommand(["disableDataTracking"])
+
+    def remove_var_from_file(self, var):
+        self.server.runCommand(["removeVarFile", var])
+
     def append_var_in_file(self, var, val, default_file=None):
         self.server.runCommand(["setVarFile", var, val, default_file, "append"])
 

bitbake/lib/bb/ui/crumbs/hoblistmodel.py

@@ -690,7 +690,7 @@ class RecipeListModel(gtk.ListStore):
             inherits = event_model["pn"][item]["inherits"]
             summary = event_model["pn"][item]["summary"]
             version = event_model["pn"][item]["version"]
-            revision = event_model["pn"][item]["revision"]
+            revision = event_model["pn"][item]["prevision"]
             homepage = event_model["pn"][item]["homepage"]
             bugtracker = event_model["pn"][item]["bugtracker"]
             filename = event_model["pn"][item]["filename"]

bitbake/lib/bb/ui/crumbs/imageconfigurationpage.py

@@ -300,7 +300,12 @@ class ImageConfigurationPage (HobPage):
     def view_warnings_button_clicked_cb(self, button):
         self.builder.show_warning_dialog()
 
+    def machine_combo_changed_idle_cb(self):
+        self.builder.window.set_cursor(None)
+
     def machine_combo_changed_cb(self, machine_combo):
+        self.builder.window.set_cursor(gtk.gdk.Cursor(gtk.gdk.WATCH))
+        self.builder.wait(0.1) #wait for combo and cursor to update
         self.stopping = False
         self.builder.parsing_warnings = []
         combo_item = machine_combo.get_active_text()
@@ -324,6 +329,8 @@ class ImageConfigurationPage (HobPage):
         # Do reparse recipes
         self.builder.populate_recipe_package_info_async()
 
+        glib.idle_add(self.machine_combo_changed_idle_cb)
+
     def update_machine_combo(self):
         self.disable_warnings_bar()
         all_machines = [self.__dummy_machine__] + self.builder.parameters.all_machines
@@ -527,7 +534,10 @@ class ImageConfigurationPage (HobPage):
         if not response:
             return
         if settings_changed:
+            self.builder.window.set_cursor(gtk.gdk.Cursor(gtk.gdk.WATCH))
+            self.builder.wait(0.1) #wait for adv_settings_dialog to terminate
             self.builder.reparse_post_adv_settings()
+            self.builder.window.set_cursor(None)
 
     def just_bake_button_clicked_cb(self, button):
         self.builder.parsing_warnings = []

bitbake/lib/bb/ui/crumbs/imagedetailspage.py

@@ -635,6 +635,14 @@ class ImageDetailsPage (HobPage):
         images_dir = topdir + "/recipes/images/"
         self.builder.ensure_dir(images_dir)
 
+        self.name_field_template = self.builder.image_configuration_page.custom_image_selected
+        if self.name_field_template:
+            image_path = self.builder.recipe_model.pn_path[self.name_field_template]
+            image_iter = self.builder.recipe_model.get_iter(image_path)
+            self.description_field_template = self.builder.recipe_model.get_value(image_iter, self.builder.recipe_model.COL_DESC)
+        else:
+            self.name_field_template = ""
+
         dialog = SaveImageDialog(images_dir, self.name_field_template, self.description_field_template,
                  "Save image recipe", self.builder, gtk.DIALOG_MODAL | gtk.DIALOG_DESTROY_WITH_PARENT)
         response = dialog.run()

bitbake/lib/bb/ui/hob.py

@@ -46,7 +46,7 @@ from bb.ui.crumbs.hoblistmodel import RecipeListModel, PackageListModel
 from bb.ui.crumbs.hobeventhandler import HobHandler
 from bb.ui.crumbs.builder import Builder
 
-extraCaches = ['bb.cache_extra:HobRecipeInfo']
+featureSet = [bb.cooker.CookerFeatures.HOB_EXTRA_CACHES]
 
 def event_handle_idle_func(eventHandler, hobHandler):
     # Consume as many messages as we can in the time available to us

bitbake/lib/bb/ui/knotty.py

@@ -472,8 +472,12 @@ def main(server, eventHandler, params, tf = TerminalFilter):
                              event.taskid, event.taskstring, event.exitcode)
                 continue
 
+            if isinstance(event, bb.event.DepTreeGenerated):
+                continue
+
             # ignore
             if isinstance(event, (bb.event.BuildBase,
+                                  bb.event.MetadataEvent,
                                   bb.event.StampUpdate,
                                   bb.event.ConfigParsed,
                                   bb.event.RecipeParsed,

bitbake/lib/bb/ui/toasterui.py

@@ -0,0 +1,272 @@
+#
+# BitBake ToasterUI Implementation
+# based on (No)TTY UI Implementation by Richard Purdie
+#
+# Handling output to TTYs or files (no TTY)
+#
+# Copyright (C) 2006-2012 Richard Purdie
+# Copyright (C) 2013      Intel Corporation
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+from __future__ import division
+try:
+    import bb
+except RuntimeError as exc:
+    sys.exit(str(exc))
+
+from bb.ui import uihelper
+from bb.ui.buildinfohelper import BuildInfoHelper
+
+import bb.msg
+import copy
+import fcntl
+import logging
+import os
+import progressbar
+import signal
+import struct
+import sys
+import time
+import xmlrpclib
+
+featureSet = [bb.cooker.CookerFeatures.HOB_EXTRA_CACHES, bb.cooker.CookerFeatures.SEND_DEPENDS_TREE, bb.cooker.CookerFeatures.BASEDATASTORE_TRACKING]
+
+logger = logging.getLogger("BitBake")
+interactive = sys.stdout.isatty()
+
+
+
+def _log_settings_from_server(server):
+    # Get values of variables which control our output
+    includelogs, error = server.runCommand(["getVariable", "BBINCLUDELOGS"])
+    if error:
+        logger.error("Unable to get the value of BBINCLUDELOGS variable: %s" % error)
+        raise BaseException(error)
+    loglines, error = server.runCommand(["getVariable", "BBINCLUDELOGS_LINES"])
+    if error:
+        logger.error("Unable to get the value of BBINCLUDELOGS_LINES variable: %s" % error)
+        raise BaseException(error)
+    return includelogs, loglines
+
+def main(server, eventHandler, params ):
+
+    includelogs, loglines = _log_settings_from_server(server)
+
+    # verify and warn
+    build_history_enabled = True
+    inheritlist, error = server.runCommand(["getVariable", "INHERIT"])
+    if not "buildhistory" in inheritlist.split(" "):
+        logger.warn("buildhistory is not enabled. Please enable INHERIT += \"buildhistory\" to see image details.")
+        build_history_enabled = False
+
+    helper = uihelper.BBUIHelper()
+
+    console = logging.StreamHandler(sys.stdout)
+    format_str = "%(levelname)s: %(message)s"
+    format = bb.msg.BBLogFormatter(format_str)
+    bb.msg.addDefaultlogFilter(console)
+    console.setFormatter(format)
+    logger.addHandler(console)
+
+    if not params.observe_only:
+        logger.error("ToasterUI can only work in observer mode")
+        return
+
+
+    main.shutdown = 0
+    interrupted = False
+    return_value = 0
+    errors = 0
+    warnings = 0
+    taskfailures = []
+
+    buildinfohelper = BuildInfoHelper(server, build_history_enabled)
+    buildinfohelper.store_layer_info()
+
+
+    while True:
+        try:
+            event = eventHandler.waitEvent(0.25)
+
+            if event is None:
+                if main.shutdown > 0:
+                    break
+                continue
+
+            helper.eventHandler(event)
+
+            if isinstance(event, bb.event.BuildStarted):
+                buildinfohelper.store_started_build(event)
+
+            if isinstance(event, (bb.build.TaskStarted, bb.build.TaskSucceeded, bb.build.TaskFailedSilent)):
+                buildinfohelper.update_and_store_task(event)
+                continue
+
+            if isinstance(event, bb.event.LogExecTTY):
+                logger.warn(event.msg)
+                continue
+
+            if isinstance(event, logging.LogRecord):
+                buildinfohelper.store_log_event(event)
+                if event.levelno >= format.ERROR:
+                    errors = errors + 1
+                    return_value = 1
+                elif event.levelno == format.WARNING:
+                    warnings = warnings + 1
+                # For "normal" logging conditions, don't show note logs from tasks
+                # but do show them if the user has changed the default log level to
+                # include verbose/debug messages
+                if event.taskpid != 0 and event.levelno <= format.NOTE:
+                    continue
+
+                logger.handle(event)
+                continue
+
+            if isinstance(event, bb.build.TaskFailed):
+                buildinfohelper.update_and_store_task(event)
+                return_value = 1
+                logfile = event.logfile
+                if logfile and os.path.exists(logfile):
+                    bb.error("Logfile of failure stored in: %s" % logfile)
+                continue
+
+            # these events are unprocessed now, but may be used in the future to log
+            # timing and error informations from the parsing phase in Toaster
+            if isinstance(event, bb.event.ParseStarted):
+                continue
+            if isinstance(event, bb.event.ParseProgress):
+                continue
+            if isinstance(event, bb.event.ParseCompleted):
+                continue
+            if isinstance(event, bb.event.CacheLoadStarted):
+                continue
+            if isinstance(event, bb.event.CacheLoadProgress):
+                continue
+            if isinstance(event, bb.event.CacheLoadCompleted):
+                continue
+            if isinstance(event, bb.event.MultipleProviders):
+                continue
+            if isinstance(event, bb.event.NoProvider):
+                return_value = 1
+                errors = errors + 1
+                if event._runtime:
+                    r = "R"
+                else:
+                    r = ""
+
+                if event._dependees:
+                    text = "Nothing %sPROVIDES '%s' (but %s %sDEPENDS on or otherwise requires it)" % (r, event._item, ", ".join(event._dependees), r)
+                else:
+                    text = "Nothing %sPROVIDES '%s'" % (r, event._item)
+
+                logger.error(text)
+                if event._reasons:
+                    for reason in event._reasons:
+                        logger.error("%s", reason)
+                        text += reason
+                buildinfohelper.store_log_error(text)
+                continue
+
+            if isinstance(event, bb.event.ConfigParsed):
+                continue
+            if isinstance(event, bb.event.RecipeParsed):
+                continue
+
+            # end of saved events
+
+            if isinstance(event, (bb.runqueue.sceneQueueTaskStarted, bb.runqueue.runQueueTaskStarted, bb.runqueue.runQueueTaskSkipped)):
+                buildinfohelper.store_started_task(event)
+                continue
+
+            if isinstance(event, bb.runqueue.runQueueTaskCompleted):
+                buildinfohelper.update_and_store_task(event)
+                continue
+
+            if isinstance(event, bb.runqueue.runQueueTaskFailed):
+                buildinfohelper.update_and_store_task(event)
+                taskfailures.append(event.taskstring)
+                logger.error("Task %s (%s) failed with exit code '%s'",
+                             event.taskid, event.taskstring, event.exitcode)
+                continue
+
+            if isinstance(event, (bb.runqueue.sceneQueueTaskCompleted, bb.runqueue.sceneQueueTaskFailed)):
+                buildinfohelper.update_and_store_task(event)
+                continue
+
+
+            if isinstance(event, (bb.event.TreeDataPreparationStarted, bb.event.TreeDataPreparationCompleted)):
+                continue
+
+            if isinstance(event, (bb.event.BuildCompleted)):
+                buildinfohelper.read_target_package_dep_data(event)
+                buildinfohelper.update_build_information(event, errors, warnings, taskfailures)
+                continue
+
+            if isinstance(event, (bb.command.CommandCompleted,
+                                  bb.command.CommandFailed,
+                                  bb.command.CommandExit)):
+
+                buildinfohelper.update_build_information(event, errors, warnings, taskfailures)
+
+                # we start a new build info
+                errors = 0
+                warnings = 0
+                taskfailures = []
+                buildinfohelper = BuildInfoHelper(server, build_history_enabled)
+                buildinfohelper.store_layer_info()
+                continue
+
+            if isinstance(event, bb.event.MetadataEvent):
+                if event.type == "SinglePackageInfo":
+                    buildinfohelper.store_build_package_information(event)
+                continue
+
+            # ignore
+            if isinstance(event, (bb.event.BuildBase,
+                                  bb.event.StampUpdate,
+                                  bb.event.RecipePreFinalise,
+                                  bb.runqueue.runQueueEvent,
+                                  bb.runqueue.runQueueExitWait,
+                                  bb.event.OperationProgress,
+                                  bb.command.CommandFailed,
+                                  bb.command.CommandExit,
+                                  bb.command.CommandCompleted,
+                                  bb.cooker.CookerExit)):
+                continue
+
+            if isinstance(event, bb.event.DepTreeGenerated):
+                buildinfohelper.store_dependency_information(event)
+                continue
+
+            logger.error("Unknown event: %s", event)
+
+        except EnvironmentError as ioerror:
+            # ignore interrupted io
+            if ioerror.args[0] == 4:
+                pass
+        except KeyboardInterrupt:
+            main.shutdown = 1
+            pass
+        except Exception as e:
+            logger.error(e)
+            import traceback
+            traceback.print_exc()
+            pass
+
+    if interrupted:
+        if return_value == 0:
+            return_value = 1
+
+    return return_value

bitbake/lib/bb/ui/uievent.py

@@ -28,7 +28,7 @@ import socket, threading, pickle
 from SimpleXMLRPCServer import SimpleXMLRPCServer, SimpleXMLRPCRequestHandler
 
 class BBUIEventQueue:
-    def __init__(self, BBServer, clientinfo=("localhost, 0")):
+    def __init__(self, BBServer, clientinfo=("localhost, 0"), featureset=[]):
 
         self.eventQueue = []
         self.eventQueueLock = threading.Lock()
@@ -44,7 +44,10 @@ class BBUIEventQueue:
         server.register_function( self.send_event, "event.sendpickle" )
         server.socket.settimeout(1)
 
-        self.EventHandle = self.BBServer.registerEventHandler(self.host, self.port)
+        self.EventHandle = self.BBServer.registerEventHandler(self.host, self.port, featureset)
+
+        if (self.EventHandle == None):
+            bb.fatal("Could not register UI event handler")
 
         self.server = server
 

bitbake/lib/bb/utils.py

@@ -724,7 +724,7 @@ def copyfile(src, dest, newmtime = None, sstat = None):
         if not sstat:
             sstat = os.lstat(src)
     except Exception as e:
-        print("copyfile: Stating source file failed...", e)
+        logger.warn("copyfile: stat of %s failed (%s)" % (src, e))
         return False
 
     destexists = 1
@@ -751,7 +751,7 @@ def copyfile(src, dest, newmtime = None, sstat = None):
             #os.lchown(dest,sstat[stat.ST_UID],sstat[stat.ST_GID])
             return os.lstat(dest)
         except Exception as e:
-            print("copyfile: failed to properly create symlink:", dest, "->", target, e)
+            logger.warn("copyfile: failed to create symlink %s to %s (%s)" % (dest, target, e))
             return False
 
     if stat.S_ISREG(sstat[stat.ST_MODE]):
@@ -766,7 +766,7 @@ def copyfile(src, dest, newmtime = None, sstat = None):
             shutil.copyfile(src, dest + "#new")
             os.rename(dest + "#new", dest)
         except Exception as e:
-            print('copyfile: copy', src, '->', dest, 'failed.', e)
+            logger.warn("copyfile: copy %s to %s failed (%s)" % (src, dest, e))
             return False
         finally:
             if srcchown:
@@ -777,13 +777,13 @@ def copyfile(src, dest, newmtime = None, sstat = None):
         #we don't yet handle special, so we need to fall back to /bin/mv
         a = getstatusoutput("/bin/cp -f " + "'" + src + "' '" + dest + "'")
         if a[0] != 0:
-            print("copyfile: Failed to copy special file:" + src + "' to '" + dest + "'", a)
+            logger.warn("copyfile: failed to copy special file %s to %s (%s)" % (src, dest, a))
             return False # failure
     try:
         os.lchown(dest, sstat[stat.ST_UID], sstat[stat.ST_GID])
         os.chmod(dest, stat.S_IMODE(sstat[stat.ST_MODE])) # Sticky is reset on chown
     except Exception as e:
-        print("copyfile: Failed to chown/chmod/unlink", dest, e)
+        logger.warn("copyfile: failed to chown/chmod %s (%s)" % (dest, e))
         return False
 
     if newmtime:

bitbake/lib/toaster/bldviewer/api.py

@@ -0,0 +1,37 @@
+#
+# BitBake Toaster Implementation
+#
+# Copyright (C) 2013        Intel Corporation
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+from django.conf.urls import patterns, include, url
+
+
+urlpatterns = patterns('bldviewer.views',
+        url(r'^builds$', 'model_explorer',  {'model_name':'build'}, name='builds'),
+        url(r'^targets$', 'model_explorer',  {'model_name':'target'}, name='targets'),
+        url(r'^tasks$', 'model_explorer', {'model_name':'task'}, name='task'),
+        url(r'^task_dependencies$', 'model_explorer',  {'model_name':'task_dependency'}, name='task_dependencies'),
+        url(r'^packages$', 'model_explorer',  {'model_name':'build_package'}, name='build_packages'),
+        url(r'^package_dependencies$', 'model_explorer',  {'model_name':'build_package_dependency'}, name='build_package_dependencies'),
+        url(r'^target_packages$', 'model_explorer',  {'model_name':'target_package'}, name='target_packages'),
+        url(r'^package_files$', 'model_explorer',  {'model_name':'build_file'}, name='build_files'),
+        url(r'^layers$', 'model_explorer', {'model_name':'layer'}, name='layer'),
+        url(r'^layerversions$', 'model_explorer', {'model_name':'layerversion'}, name='layerversion'),
+        url(r'^recipes$', 'model_explorer', {'model_name':'recipe'}, name='recipe'),
+        url(r'^recipe_dependencies$', 'model_explorer',  {'model_name':'recipe_dependency'}, name='recipe_dependencies'),
+        url(r'^variables$', 'model_explorer',  {'model_name':'variable'}, name='variables'),
+        url(r'^logmessages$', 'model_explorer',  {'model_name':'logmessage'}, name='logmessages'),
+)

bitbake/lib/toaster/bldviewer/templates/base.html

@@ -0,0 +1,30 @@
+<!DOCTYPE html>
+{% load static %}
+<html>
+	<head>
+		<title>Toaster Simple Explorer</title>
+<script src="{% static 'js/jquery-2.0.3.js' %}">
+</script>
+<script src="{% static 'js/bootstrap.js' %}">
+</script>
+<link href="{% static 'css/bootstrap.css' %}" rel="stylesheet" type="text/css">
+	</head>
+
+<body style="height: 100%">
+<div style="width:100%; height: 100%; position:absolute">
+<div style="width: 100%; height: 3em" class="nav">
+    <ul class="nav nav-tabs">
+        <li><a href="{% url all-builds %}">All Builds</a></li>
+        <li><a href="{% url all-layers %}">All Layers</a></li>
+    </ul>
+</div>
+
+<div style="overflow-y:scroll; width: 100%; position: absolute; top: 3em; bottom:70px ">
+{% block pagecontent %}
+{% endblock %}
+</div>
+<div class="navbar" style="position: absolute; bottom: 0; width:100%"><br/>About Toaster | Yocto Project </div>
+</div>
+</body>
+</html>
+

bitbake/lib/toaster/bldviewer/templates/basebuildpage.html

@@ -0,0 +1,17 @@
+{% extends "basetable.html" %}
+
+{% block pagename %}
+<ul class="nav nav-tabs" style="display: inline-block">
+  <li><a>Build {{build.target_set.all|join:"&nbsp;"}} at {{build.started_on}} : </a></li>
+  <li><a href="{% url task build.id %}"> Tasks </a></li>
+  <li><a href="{% url bpackage build.id %}"> Build Packages </a></li>
+    {% for t in build.target_set.all %}
+        {% if t.is_image %}
+  <li><a href="{% url tpackage build.id t.pk %}"> Packages for {{t.target}} </a> </li>
+        {% endif %}
+    {% endfor %}
+  <li><a href="{% url configuration build.id %}"> Configuration </a> </li>
+</ul>
+     <h1>Toaster - Build {% block pagetitle %} {% endblock %}</h1>
+{% endblock %}
+

bitbake/lib/toaster/bldviewer/templates/basetable.html

@@ -0,0 +1,46 @@
+{% extends "base.html" %}
+
+{% block pagecontent %}
+<script>
+function showhideTableColumn(i, sh) {
+    if (sh)
+        $('td:nth-child('+i+'),th:nth-child('+i+')').show();
+    else
+        $('td:nth-child('+i+'),th:nth-child('+i+')').hide();
+}
+
+
+function filterTableRows(test) {
+    if (test.length > 0) {
+        var r = test.split(/[ ,]+/).map(function (e) { return new RegExp(e, 'i') });
+        $('tr.data').map( function (i, el) {
+            (! r.map(function (j) { return j.test($(el).html())}).reduce(function (c, p) { return c && p;} )) ? $(el).hide() : $(el).show();
+        });
+    } else
+    {
+        $('tr.data').show();
+    }
+}
+</script>
+<div style="margin-bottom: 0.5em">
+
+    {% block pagename %}
+    {% endblock %}
+    <div align="left" style="display:inline-block; width: 40%; margin-left: 2em"> Search: <input type="search" id="filterstring" style="width: 80%" onkeyup="filterTableRows($('#filterstring').val())" autocomplete="off">
+    </div>
+    {% if hideshowcols %}
+    <div align="right" style="display: inline-block; width: 40%">Show/Hide columns:
+    {% for i in hideshowcols %}
+        <span>{{i.name}} <input type="checkbox" id="ct{{i.name}}" onchange="showhideTableColumn({{i.order}}, $('#ct{{i.name}}').is(':checked'))" checked autocomplete="off"></span> |
+    {% endfor %}
+    </div>
+    {% endif %}
+</div>
+<div>
+    <table class="table table-striped table-condensed" style="width:95%">
+{% block pagetable %}
+{% endblock %}
+    </table>
+</div>
+
+{% endblock %}

bitbake/lib/toaster/bldviewer/templates/bfile.html

@@ -0,0 +1,24 @@
+{% extends "basebuildpage.html" %}
+
+{% block pagetitle %}Files for package {{files.0.bpackage.name}} {% endblock %}
+{% block pagetable %}
+    {% if not files %}
+        <p>No files were recorded for this package!</p>
+    {% else %}
+
+            <tr>
+            <th>Name</th>
+            <th>Size (Bytes)</th>
+            </tr>
+
+            {% for file in files %}
+
+            <tr class="data">
+            <td>{{file.path}}</td>
+            <td>{{file.size}}</td>
+
+            {% endfor %}
+
+    {% endif %}
+
+{% endblock %}

bitbake/lib/toaster/bldviewer/templates/bpackage.html

@@ -0,0 +1,44 @@
+{% extends "basebuildpage.html" %}
+
+{% block pagetitle %}Packages{% endblock %}
+{% block pagetable %}
+    {% if not packages %}
+        <p>No packages were recorded for this target!</p>
+    {% else %}
+
+            <tr>
+            <th>Name</th>
+            <th>Version</th>
+            <th>Recipe</th>
+            <th>Summary</th>
+            <th>Section</th>
+            <th>Description</th>
+            <th>Size on host disk (Bytes)</th>
+            <th>License</th>
+            <th>Dependencies List (all)</th>
+            </tr>
+
+            {% for package in packages %}
+
+            <tr class="data">
+                <td><a name="#{{package.name}}" href="{% url bfile build.pk package.pk %}">{{package.name}} ({{package.filelist_bpackage.count}} files)</a></td>
+                <td>{{package.version}}-{{package.revision}}</td>
+                <td><a href="{% url layer_versions_recipes package.recipe.layer_version_id %}#{{package.recipe.name}}">{{package.recipe.name}}</a>{{package.package_name}}</a></td>
+
+            <td>{{package.summary}}</td>
+            <td>{{package.section}}</td>
+            <td>{{package.description}}</td>
+            <td>{{package.size}}</td>
+            <td>{{package.license}}</td>
+            <td>
+        <div style="height: 3em; overflow:auto">
+            {% for bpd in package.bpackage_dependencies_package.all %}
+                {{bpd.dep_type}}: {{bpd.depends_on}} <br/>
+            {% endfor %}
+        </div>
+            </td>
+            {% endfor %}
+
+    {% endif %}
+
+{% endblock %}

bitbake/lib/toaster/bldviewer/templates/build.html

@@ -0,0 +1,43 @@
+{% extends "basetable.html" %}
+
+{% block pagename %}
+    <h1>Toaster - Builds</h1>
+{% endblock %}
+
+{% block pagetable %}
+
+    {% load projecttags %}
+        <tr>
+            <th>Outcome</th>
+            <th>Started On</th>
+            <th>Completed On</th>
+            <th>Target</th>
+            <th>Machine</th>
+            <th>Time</th>
+            <th>Errors</th>
+            <th>Warnings</th>
+            <th>Output</th>
+            <th>Log</th>
+            <th>Bitbake Version</th>
+            <th>Build Name</th>
+        </tr>
+        {% for build in builds %}
+        <tr class="data">
+            <td><a href="{% url configuration build.id %}">{{build.get_outcome_display}}</a></td>
+            <td>{{build.started_on}}</td>
+            <td>{{build.completed_on}}</td>
+            <td>{% for t in build.target_set.all %}<a href="{% url tpackage build.id t.id %}">{{t.target}}</a>{% if t.is_image %} (Img){% endif %}<br/>{% endfor %}</td>
+            <td>{{build.machine}}</td>
+            <td>{% time_difference build.started_on build.completed_on %}</td>
+            <td>{{build.errors_no}}:{% if  build.errors_no %}{% for error in logs %}{% if error.build == build %}{% if error.level == 2 %}<p>{{error.message}}</p>{% endif %}{% endif %}{% endfor %}{% else %}None{% endif %}</td>
+            <td>{{build.warnings_no}}:{% if  build.warnings_no %}{% for warning in logs %}{% if warning.build == build %}{% if warning.level == 1 %}<p>{{warning.message}}</p>{% endif %}{% endif %}{% endfor %}{% else %}None{% endif %}</td>
+            <td>{% if build.outcome == 0 %}{% for t in build.target_set.all %}{% if t.is_image %}{{build.image_fstypes}}{% endif %}{% endfor %}{% endif %}</td>
+            <td>{{build.cooker_log_path}}</td>
+            <td>{{build.bitbake_version}}</td>
+            <td>{{build.build_name}}</td>
+        </tr>
+
+        {% endfor %}
+{% endblock %}
+
+

bitbake/lib/toaster/bldviewer/templates/configuration.html

@@ -0,0 +1,22 @@
+{% extends "basebuildpage.html" %}
+
+{% block pagetitle %}Configuration{% endblock %}
+{% block pagetable %}
+
+            <tr>
+            <th>Name</th>
+            <th>Description</th>
+            <th>Definition history</th>
+            <th>Value</th>
+            </tr>
+
+            {% for variable in configuration %}
+
+            <tr class="data">
+                <td>{{variable.variable_name}}</td>
+                <td>{% if variable.description %}{{variable.description}}{% endif %}</td>
+                <td>{% for vh in variable.variablehistory_set.all %}{{vh.operation}} in {{vh.file_name}}:{{vh.line_number}}<br/>{%endfor%}</td>
+                <td>{{variable.variable_value}}</td>
+            {% endfor %}
+
+{% endblock %}

bitbake/lib/toaster/bldviewer/templates/layer.html

@@ -0,0 +1,34 @@
+{% extends "basetable.html" %}
+
+{% block pagename %}
+    <h1>Toaster - Layers</h1>
+{% endblock %}
+
+{% block pagetable %}
+    {% load projecttags %}
+
+        <tr>
+            <th>Name</th>
+            <th>Local Path</th>
+            <th>Layer Index URL</th>
+            <th>Known Versions</th>
+        </tr>
+
+        {% for layer in layers %}
+
+        <tr class="data">
+            <td>{{layer.name}}</td>
+            <td>{{layer.local_path}}</td>
+            <td><a href='{{layer.layer_index_url}}'>{{layer.layer_index_url}}</a></td>
+            <td><table>
+            {% for lv in layer.versions %}
+                <tr><td>
+        <a href="{% url layer_versions_recipes lv.id %}">({{lv.priority}}){{lv.branch}}:{{lv.commit}} ({{lv.count}} recipes)</a>
+                </td></tr>
+            {% endfor %}
+            </table></td>
+        </tr>
+
+        {% endfor %}
+
+{% endblock %}

bitbake/lib/toaster/bldviewer/templates/package.html

@@ -0,0 +1,36 @@
+{% extends "basebuildpage.html" %}
+
+{% block pagetable %}
+    {% if not packages %}
+        <p>No packages were recorded for this target!</p>
+    {% else %}
+
+            <tr>
+            <th>Name</th>
+            <th>Version</th>
+            <th>Size (Bytes)</th>
+            <th>Recipe</th>
+            <th>Depends on</th>
+            </tr>
+
+            {% for package in packages %}
+
+            <tr class="data">
+                <td><a name="#{{package.name}}">{{package.name}}</a></td>
+                <td>{{package.version}}</td>
+                <td>{{package.size}}</td>
+                <td>{%if package.recipe %}<a name="{{package.recipe.name}}.{{package.package_name}}">
+                <a href="{% url layer_versions_recipes package.recipe.layer_version_id %}#{{package.recipe.name}}">{{package.recipe.name}}</a>{{package.package_name}}</a>{%endif%}</td>
+                <td>
+            <div style="height: 4em; overflow:auto">
+                    {% for d in package.tpackage_dependencies_package.all %}
+                    <a href="#{{d.name}}">{{d.depends_on.name}}</a><br/>
+                    {% endfor %}
+            </div>
+                </td>
+
+            {% endfor %}
+
+    {% endif %}
+
+{% endblock %}

bitbake/lib/toaster/bldviewer/templates/recipe.html

@@ -0,0 +1,53 @@
+{% extends "basetable.html" %}
+
+{% block pagename %}
+<ul class="nav nav-tabs" style="display: inline-block">
+  <li><a>Layer  {{layer_version.layer.name}}&nbsp;:&nbsp;{{layer_version.branch}}&nbsp;:&nbsp;{{layer_version.commit}}&nbsp;:&nbsp;{{layer_version.priority}}</a></li>
+</ul>
+    <h1>Toaster - Recipes for a Layer</h1>
+{% endblock %}
+
+{% block pagetable %}
+    {% load projecttags %}
+
+        <tr>
+        </tr>
+            <th>Name</th>
+            <th>Version</th>
+            <th>Summary</th>
+            <th>Description</th>
+            <th>Section</th>
+            <th>License</th>
+            <th>License file</th>
+            <th>Homepage</th>
+            <th>Bugtracker</th>
+            <th>Author</th>
+            <th>File_path</th>
+            <th style="width: 30em">Recipe Dependency</th>
+
+
+        {% for recipe in recipes %}
+
+        <tr class="data">
+            <td><a name="{{recipe.name}}">{{recipe.name}}</a></td>
+            <td>{{recipe.version}}</td>
+            <td>{{recipe.summary}}</td>
+            <td>{{recipe.description}}</td>
+            <td>{{recipe.section}}</td>
+            <td>{{recipe.license}}</td>
+            <td>{{recipe.licensing_info}}</td>
+            <td>{{recipe.homepage}}</td>
+            <td>{{recipe.bugtracker}}</td>
+            <td>{{recipe.file_path}}</td>
+            <td>
+        <div style="height: 5em; overflow:auto">
+            {% for rr in recipe.r_dependencies_recipe.all %}
+                <a href="#{{rr.depends_on.name}}">{{rr.depends_on.name}}</a><br/>
+            {% endfor %}
+        </div>
+            </td>
+        </tr>
+
+        {% endfor %}
+
+{% endblock %}

bitbake/lib/toaster/bldviewer/templates/task.html

@@ -0,0 +1,63 @@
+{% extends "basebuildpage.html" %}
+
+{% block pagetitle %}Tasks{% endblock %}
+{% block pagetable %}
+    {% if not tasks %}
+        <p>No tasks were executed in this build!</p>
+    {% else %}
+
+            <tr>
+            <th>Order</th>
+            <th>Task</th>
+            <th>Recipe Version</th>
+            <th>Task Type</th>
+            <th>Checksum</th>
+            <th>Outcome</th>
+            <th>Message</th>
+            <th>Logfile</th>
+            <th>Time</th>
+            <th>CPU usage</th>
+            <th>Disk I/O</th>
+            <th>Script type</th>
+            <th>File path</th>
+            <th>Depends</th>
+            </tr>
+
+            {% for task in tasks %}
+
+            <tr class="data">
+                <td>{{task.order}}</td>
+                <td><a name="{{task.recipe.name}}.{{task.task_name}}">
+                <a href="{% url layer_versions_recipes task.recipe.layer_version_id %}#{{task.recipe.name}}">{{task.recipe.name}}</a>.{{task.task_name}}</a></td>
+                <td>{{task.recipe.version}}</td>
+
+                {% if task.task_executed %}
+                <td>Executed</td>
+                {% else %}
+                <td>Prebuilt</td>
+                {% endif %}
+
+                <td>{{task.sstate_checksum}}</td>
+                <td>{{task.get_outcome_display}}{% if task.provider %}</br>(by <a href="#{{task.provider.recipe.name}}.{{task.provider.task_name}}">{{task.provider.recipe.name}}.{{task.provider.task_name}}</a>){% endif %}</td>
+                <td><p>{{task.message}}</td>
+                <td><a target="_fileview" href="file:///{{task.logfile}}">{{task.logfile}}</a></td>
+                <td>{{task.elapsed_time}}</td>
+                <td>{{task.cpu_usage}}</td>
+                <td>{{task.disk_io}}</td>
+                <td>{{task.get_script_type_display}}</td>
+                <td><a target="_fileview" href="file:///{{task.recipe.file_path}}">{{task.recipe.file_path}}</a></td>
+                <td>
+            <div style="height: 3em; overflow:auto">
+                {% for tt in task.task_dependencies_task.all %}
+                    <a href="#{{tt.depends_on.recipe.name}}.{{tt.depends_on.task_name}}">
+                    {{tt.depends_on.recipe.name}}.{{tt.depends_on.task_name}}</a><br/>
+                {% endfor %}
+            </div>
+                </td>
+                </tr>
+
+            {% endfor %}
+
+    {% endif %}
+
+{% endblock %}

bitbake/lib/toaster/bldviewer/templatetags/projecttags.py

@@ -0,0 +1,26 @@
+#
+# BitBake Toaster Implementation
+#
+# Copyright (C) 2013        Intel Corporation
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+from datetime import datetime
+from django import template
+
+register = template.Library()
+
+@register.simple_tag
+def time_difference(start_time, end_time):
+    return end_time - start_time

bitbake/lib/toaster/bldviewer/urls.py

@@ -0,0 +1,32 @@
+#
+# BitBake Toaster Implementation
+#
+# Copyright (C) 2013        Intel Corporation
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+from django.conf.urls import patterns, include, url
+from django.views.generic.simple import redirect_to
+
+urlpatterns = patterns('bldviewer.views',
+        url(r'^builds/$', 'build', name='all-builds'),
+        url(r'^build/(?P<build_id>\d+)/task/$', 'task', name='task'),
+        url(r'^build/(?P<build_id>\d+)/packages/$', 'bpackage', name='bpackage'),
+        url(r'^build/(?P<build_id>\d+)/package/(?P<package_id>\d+)/files/$', 'bfile', name='bfile'),
+        url(r'^build/(?P<build_id>\d+)/target/(?P<target_id>\d+)/packages/$', 'tpackage', name='tpackage'),
+        url(r'^build/(?P<build_id>\d+)/configuration/$', 'configuration', name='configuration'),
+        url(r'^layers/$', 'layer', name='all-layers'),
+        url(r'^layerversions/(?P<layerversion_id>\d+)/recipes/.*$', 'layer_versions_recipes', name='layer_versions_recipes'),
+        url(r'^$', redirect_to, {'url': 'builds/'}),
+)

bitbake/lib/toaster/bldviewer/views.py

@@ -0,0 +1,260 @@
+#
+# BitBake Toaster Implementation
+#
+# Copyright (C) 2013        Intel Corporation
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+import operator
+
+from django.db.models import Q
+from django.shortcuts import render
+from orm.models import Build, Target, Task, Layer, Layer_Version, Recipe, Target_Package, LogMessage, Variable
+from orm.models import Task_Dependency, Recipe_Dependency, Build_Package, Build_File, Build_Package_Dependency
+from django.views.decorators.cache import cache_control
+
+@cache_control(no_store=True)
+def build(request):
+    template = 'build.html'
+    build_info = Build.objects.all()
+
+    logs = LogMessage.objects.all()
+
+    context = {'builds': build_info, 'logs': logs ,
+        'hideshowcols' : [
+                {'name': 'Output', 'order':10},
+                {'name': 'Log', 'order':11},
+            ]}
+
+    return render(request, template, context)
+
+
+def _find_task_revdep(task):
+    tp = []
+    for p in Task_Dependency.objects.filter(depends_on=task):
+        tp.append(p.task);
+    return tp
+
+def _find_task_provider(task):
+    task_revdeps = _find_task_revdep(task)
+    for tr in task_revdeps:
+        if tr.outcome != Task.OUTCOME_COVERED:
+            return tr
+    for tr in task_revdeps:
+        trc = _find_task_provider(tr)
+        if trc is not None:
+            return trc
+    return None
+
+def task(request, build_id):
+    template = 'task.html'
+
+    tasks = Task.objects.filter(build=build_id)
+
+    for t in tasks:
+        if t.outcome == Task.OUTCOME_COVERED:
+            t.provider = _find_task_provider(t)
+
+    context = {'build': Build.objects.filter(pk=build_id)[0], 'tasks': tasks}
+
+    return render(request, template, context)
+
+def configuration(request, build_id):
+    template = 'configuration.html'
+    variables = Variable.objects.filter(build=build_id)
+    context = {'build': Build.objects.filter(pk=build_id)[0], 'configuration' : variables}
+    return render(request, template, context)
+
+def bpackage(request, build_id):
+    template = 'bpackage.html'
+    packages = Build_Package.objects.filter(build = build_id)
+    context = {'build': Build.objects.filter(pk=build_id)[0], 'packages' : packages}
+    return render(request, template, context)
+
+def bfile(request, build_id, package_id):
+    template = 'bfile.html'
+    files = Build_File.objects.filter(bpackage = package_id)
+    context = {'build': Build.objects.filter(pk=build_id)[0], 'files' : files}
+    return render(request, template, context)
+
+def tpackage(request, build_id, target_id):
+    template = 'package.html'
+
+    packages = Target_Package.objects.filter(target=target_id)
+
+    context = {'build' : Build.objects.filter(pk=build_id)[0],'packages': packages}
+
+    return render(request, template, context)
+
+def layer(request):
+    template = 'layer.html'
+    layer_info = Layer.objects.all()
+
+    for li in layer_info:
+        li.versions = Layer_Version.objects.filter(layer = li)
+        for liv in li.versions:
+            liv.count = Recipe.objects.filter(layer_version__id = liv.id).count()
+
+    context = {'layers': layer_info}
+
+    return render(request, template, context)
+
+
+def layer_versions_recipes(request, layerversion_id):
+    template = 'recipe.html'
+    recipes = Recipe.objects.filter(layer_version__id = layerversion_id)
+
+    context = {'recipes': recipes,
+            'layer_version' : Layer_Version.objects.filter( id = layerversion_id )[0]
+    }
+
+    return render(request, template, context)
+
+#### API
+
+import json
+from django.core import serializers
+from django.http import HttpResponse, HttpResponseBadRequest
+
+
+def model_explorer(request, model_name):
+
+    DESCENDING = 'desc'
+    response_data = {}
+    model_mapping = {
+        'build': Build,
+        'target': Target,
+        'target_package': Target_Package,
+        'task': Task,
+        'task_dependency': Task_Dependency,
+        'package': Build_Package,
+        'layer': Layer,
+        'layerversion': Layer_Version,
+        'recipe': Recipe,
+        'recipe_dependency': Recipe_Dependency,
+        'build_package': Build_Package,
+        'build_package_dependency': Build_Package_Dependency,
+        'build_file': Build_File,
+        'variable': Variable,
+        'logmessage': LogMessage,
+        }
+
+    if model_name not in model_mapping.keys():
+        return HttpResponseBadRequest()
+
+    model = model_mapping[model_name]
+
+    try:
+        limit = int(request.GET.get('limit', 0))
+    except ValueError:
+        limit = 0
+
+    try:
+        offset = int(request.GET.get('offset', 0))
+    except ValueError:
+        offset = 0
+
+    ordering_string, invalid = _validate_input(request.GET.get('orderby', ''),
+                                               model)
+    if invalid:
+        return HttpResponseBadRequest()
+
+    filter_string, invalid = _validate_input(request.GET.get('filter', ''),
+                                             model)
+    if invalid:
+        return HttpResponseBadRequest()
+
+    search_term = request.GET.get('search', '')
+
+    if filter_string:
+        filter_terms = _get_filtering_terms(filter_string)
+        try:
+            queryset = model.objects.filter(**filter_terms)
+        except ValueError:
+            queryset = []
+    else:
+        queryset = model.objects.all()
+
+    if search_term:
+        queryset = _get_search_results(search_term, queryset, model)
+
+    if ordering_string and queryset:
+        column, order = ordering_string.split(':')
+        if order.lower() == DESCENDING:
+            queryset = queryset.order_by('-' + column)
+        else:
+            queryset = queryset.order_by(column)
+
+    if offset and limit:
+        queryset = queryset[offset:(offset+limit)]
+    elif offset:
+        queryset = queryset[offset:]
+    elif limit:
+        queryset = queryset[:limit]
+
+    if queryset:
+        response_data['count'] = queryset.count()
+    else:
+        response_data['count'] = 0
+
+    response_data['list'] = serializers.serialize('json', queryset)
+
+    return HttpResponse(json.dumps(response_data),
+                        content_type='application/json')
+
+def _get_filtering_terms(filter_string):
+
+    search_terms = filter_string.split(":")
+    keys = search_terms[0].split(',')
+    values = search_terms[1].split(',')
+
+    return dict(zip(keys, values))
+
+def _validate_input(input, model):
+
+    invalid = 0
+
+    if input:
+        input_list = input.split(":")
+
+        # Check we have only one colon
+        if len(input_list) != 2:
+            invalid = 1
+            return None, invalid
+
+        # Check we have an equal number of terms both sides of the colon
+        if len(input_list[0].split(',')) != len(input_list[1].split(',')):
+            invalid = 1
+            return None, invalid
+
+        # Check we are looking for a valid field
+        valid_fields = model._meta.get_all_field_names()
+        for field in input_list[0].split(','):
+            if field not in valid_fields:
+                invalid = 1
+                return None, invalid
+
+    return input, invalid
+
+def _get_search_results(search_term, queryset, model):
+    search_objects = []
+    for st in search_term.split(" "):
+        q_map = map(lambda x: Q(**{x+'__icontains': st}),
+                model.search_allowed_fields)
+
+        search_objects.append(reduce(operator.or_, q_map))
+    search_object = reduce(operator.and_, search_objects)
+    queryset = queryset.filter(search_object)
+
+    return queryset

bitbake/lib/toaster/manage.py

@@ -0,0 +1,10 @@
+#!/usr/bin/env python
+import os
+import sys
+
+if __name__ == "__main__":
+    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "toastermain.settings")
+
+    from django.core.management import execute_from_command_line
+
+    execute_from_command_line(sys.argv)

bitbake/lib/toaster/orm/models.py

@@ -0,0 +1,265 @@
+#
+# BitBake Toaster Implementation
+#
+# Copyright (C) 2013        Intel Corporation
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+from django.db import models
+from django.utils.encoding import python_2_unicode_compatible
+
+
+class Build(models.Model):
+    SUCCEEDED = 0
+    FAILED = 1
+    IN_PROGRESS = 2
+
+    BUILD_OUTCOME = (
+        (SUCCEEDED, 'Succeeded'),
+        (FAILED, 'Failed'),
+        (IN_PROGRESS, 'In Progress'),
+    )
+
+    search_allowed_fields = ['machine',
+                             'cooker_log_path']
+
+    machine = models.CharField(max_length=100)
+    image_fstypes = models.CharField(max_length=100)
+    distro = models.CharField(max_length=100)
+    distro_version = models.CharField(max_length=100)
+    started_on = models.DateTimeField()
+    completed_on = models.DateTimeField()
+    outcome = models.IntegerField(choices=BUILD_OUTCOME, default=IN_PROGRESS)
+    errors_no = models.IntegerField(default=0)
+    warnings_no = models.IntegerField(default=0)
+    cooker_log_path = models.CharField(max_length=500)
+    build_name = models.CharField(max_length=100)
+    bitbake_version = models.CharField(max_length=50)
+
+@python_2_unicode_compatible
+class Target(models.Model):
+    search_allowed_fields = ['target', 'image_fstypes', 'file_name']
+    build = models.ForeignKey(Build)
+    target = models.CharField(max_length=100)
+    is_image = models.BooleanField(default = False)
+    file_name = models.CharField(max_length=100)
+    file_size = models.IntegerField()
+
+    def __str__(self):
+        return self.target
+
+
+class Task(models.Model):
+
+    SSTATE_NA = 0
+    SSTATE_MISS = 1
+    SSTATE_FAILED = 2
+    SSTATE_RESTORED = 3
+
+    SSTATE_RESULT = (
+        (SSTATE_NA, 'Not Applicable'), # For rest of tasks, but they still need checking.
+        (SSTATE_MISS, 'Missing'), # it is a miss
+        (SSTATE_FAILED, 'Failed'), # there was a pkg, but the script failed
+        (SSTATE_RESTORED, 'Restored'), # succesfully restored
+    )
+
+    CODING_NA = 0
+    CODING_NOEXEC = 1
+    CODING_PYTHON = 2
+    CODING_SHELL = 3
+
+    TASK_CODING = (
+        (CODING_NA, 'N/A'),
+        (CODING_NOEXEC, 'NoExec'),
+        (CODING_PYTHON, 'Python'),
+        (CODING_SHELL, 'Shell'),
+    )
+
+    OUTCOME_SUCCESS = 0
+    OUTCOME_COVERED = 1
+    OUTCOME_SSTATE = 2
+    OUTCOME_EXISTING = 3
+    OUTCOME_FAILED = 4
+    OUTCOME_NA = 5
+
+    TASK_OUTCOME = (
+        (OUTCOME_SUCCESS, 'Succeeded'),
+        (OUTCOME_COVERED, 'Covered'),
+        (OUTCOME_SSTATE, 'Sstate'),
+        (OUTCOME_EXISTING, 'Existing'),
+        (OUTCOME_FAILED, 'Failed'),
+        (OUTCOME_NA, 'Not Available'),
+    )
+
+    build = models.ForeignKey(Build, related_name='task_build')
+    order = models.IntegerField(null=True)
+    task_executed = models.BooleanField(default=False) # True means Executed, False means Prebuilt
+    outcome = models.IntegerField(choices=TASK_OUTCOME, default=OUTCOME_NA)
+    sstate_checksum = models.CharField(max_length=100, blank=True)
+    path_to_sstate_obj = models.FilePathField(max_length=500, blank=True)
+    recipe = models.ForeignKey('Recipe', related_name='build_recipe')
+    task_name = models.CharField(max_length=100)
+    source_url = models.FilePathField(max_length=255, blank=True)
+    work_directory = models.FilePathField(max_length=255, blank=True)
+    script_type = models.IntegerField(choices=TASK_CODING, default=CODING_NA)
+    line_number = models.IntegerField(default=0)
+    disk_io = models.IntegerField(null=True)
+    cpu_usage = models.DecimalField(max_digits=6, decimal_places=2, null=True)
+    elapsed_time = models.CharField(max_length=50, default=0)
+    sstate_result = models.IntegerField(choices=SSTATE_RESULT, default=SSTATE_NA)
+    message = models.CharField(max_length=240)
+    logfile = models.FilePathField(max_length=255, blank=True)
+
+    class Meta:
+        ordering = ('order', 'recipe' ,)
+
+
+class Task_Dependency(models.Model):
+    task = models.ForeignKey(Task, related_name='task_dependencies_task')
+    depends_on = models.ForeignKey(Task, related_name='task_dependencies_depends')
+
+
+class Build_Package(models.Model):
+    build = models.ForeignKey('Build')
+    recipe = models.ForeignKey('Recipe', null=True)
+    name = models.CharField(max_length=100)
+    version = models.CharField(max_length=100, blank=True)
+    revision = models.CharField(max_length=32, blank=True)
+    summary = models.CharField(max_length=200, blank=True)
+    description = models.CharField(max_length=200, blank=True)
+    size = models.IntegerField(default=0)
+    section = models.CharField(max_length=80, blank=True)
+    license = models.CharField(max_length=80, blank=True)
+
+class Build_Package_Dependency(models.Model):
+    TYPE_RDEPENDS = 0
+    TYPE_RPROVIDES = 1
+    TYPE_RRECOMMENDS = 2
+    TYPE_RSUGGESTS = 3
+    TYPE_RREPLACES = 4
+    TYPE_RCONFLICTS = 5
+    DEPENDS_TYPE = (
+        (TYPE_RDEPENDS, "rdepends"),
+        (TYPE_RPROVIDES, "rprovides"),
+        (TYPE_RRECOMMENDS, "rrecommends"),
+        (TYPE_RSUGGESTS, "rsuggests"),
+        (TYPE_RREPLACES, "rreplaces"),
+        (TYPE_RCONFLICTS, "rconflicts"),
+    )
+    package = models.ForeignKey(Build_Package, related_name='bpackage_dependencies_package')
+    depends_on = models.CharField(max_length=100)   # soft dependency
+    dep_type = models.IntegerField(choices=DEPENDS_TYPE)
+
+
+class Target_Package(models.Model):
+    target = models.ForeignKey('Target')
+    recipe = models.ForeignKey('Recipe', null=True)
+    name = models.CharField(max_length=100)
+    version = models.CharField(max_length=100, blank=True)
+    size = models.IntegerField()
+
+
+class Target_Package_Dependency(models.Model):
+    TYPE_DEPENDS = 0
+    TYPE_RDEPENDS = 1
+    TYPE_RECOMMENDS = 2
+
+    DEPENDS_TYPE = (
+        (TYPE_DEPENDS, "depends"),
+        (TYPE_RDEPENDS, "rdepends"),
+        (TYPE_RECOMMENDS, "recommends"),
+    )
+    package = models.ForeignKey(Target_Package, related_name='tpackage_dependencies_package')
+    depends_on = models.ForeignKey(Target_Package, related_name='tpackage_dependencies_depends')
+    dep_type = models.IntegerField(choices=DEPENDS_TYPE)
+
+
+class Build_File(models.Model):
+    bpackage = models.ForeignKey(Build_Package, related_name='filelist_bpackage')
+    path = models.FilePathField(max_length=255, blank=True)
+    size = models.IntegerField()
+
+class Target_File(models.Model):
+    tpackage = models.ForeignKey(Target_Package, related_name='filelist_tpackage')
+    path = models.FilePathField(max_length=255, blank=True)
+    size = models.IntegerField()
+
+
+class Recipe(models.Model):
+    name = models.CharField(max_length=100, blank=True)
+    version = models.CharField(max_length=100, blank=True)
+    layer_version = models.ForeignKey('Layer_Version', related_name='recipe_layer_version')
+    summary = models.CharField(max_length=100, blank=True)
+    description = models.CharField(max_length=100, blank=True)
+    section = models.CharField(max_length=100, blank=True)
+    license = models.CharField(max_length=200, blank=True)
+    licensing_info = models.TextField(blank=True)
+    homepage = models.URLField(blank=True)
+    bugtracker = models.URLField(blank=True)
+    file_path = models.FilePathField(max_length=255)
+
+
+class Recipe_Dependency(models.Model):
+    TYPE_DEPENDS = 0
+    TYPE_RDEPENDS = 1
+
+    DEPENDS_TYPE = (
+        (TYPE_DEPENDS, "depends"),
+        (TYPE_RDEPENDS, "rdepends"),
+    )
+    recipe = models.ForeignKey(Recipe, related_name='r_dependencies_recipe')
+    depends_on = models.ForeignKey(Recipe, related_name='r_dependencies_depends')
+    dep_type = models.IntegerField(choices=DEPENDS_TYPE)
+
+class Layer(models.Model):
+    name = models.CharField(max_length=100)
+    local_path = models.FilePathField(max_length=255)
+    layer_index_url = models.URLField()
+
+
+class Layer_Version(models.Model):
+    layer = models.ForeignKey(Layer, related_name='layer_version_layer')
+    branch = models.CharField(max_length=50)
+    commit = models.CharField(max_length=100)
+    priority = models.IntegerField()
+
+
+class Variable(models.Model):
+    build = models.ForeignKey(Build, related_name='variable_build')
+    variable_name = models.CharField(max_length=100)
+    variable_value = models.TextField(blank=True)
+    changed = models.BooleanField(default=False)
+    human_readable_name = models.CharField(max_length=200)
+    description = models.TextField(blank=True)
+
+class VariableHistory(models.Model):
+    variable = models.ForeignKey(Variable)
+    file_name = models.FilePathField(max_length=255)
+    line_number = models.IntegerField(null=True)
+    operation = models.CharField(max_length=16)
+
+class LogMessage(models.Model):
+    INFO = 0
+    WARNING = 1
+    ERROR = 2
+
+    LOG_LEVEL = ( (INFO, "info"),
+            (WARNING, "warn"),
+            (ERROR, "error") )
+
+    build = models.ForeignKey(Build)
+    level = models.IntegerField(choices=LOG_LEVEL, default=INFO)
+    message=models.CharField(max_length=240)
+    pathname = models.FilePathField(max_length=255, blank=True)
+    lineno = models.IntegerField(null=True)

bitbake/lib/toaster/toastergui/templates/index.html

@@ -0,0 +1,13 @@
+<html>
+<head>
+	<title>GUI Page</title>
+</head>
+<body>
+{% load staticfiles %}
+
+<img src="{% static "/static/images/yocto.jpg" %}" alt="Yocto"/>
+
+This is your basic index page!
+</body>
+
+</html>

bitbake/lib/toaster/toastergui/urls.py

@@ -0,0 +1,27 @@
+#
+# BitBake Toaster Implementation
+#
+# Copyright (C) 2013        Intel Corporation
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+import os
+
+from django.conf import settings
+from django.conf.urls import patterns, include, url
+
+
+urlpatterns = patterns('toastergui.views',
+        url(r'^$', 'guihome', name='guihome'),
+)

bitbake/lib/toaster/toastergui/views.py

@@ -0,0 +1,26 @@
+#
+# BitBake Toaster Implementation
+#
+# Copyright (C) 2013        Intel Corporation
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+from django.shortcuts import render
+from orm.models import Build, Task
+
+
+def guihome(request):
+    template = 'index.html'
+
+    return render(request, template)

bitbake/lib/toaster/toastermain/settings.py

@@ -0,0 +1,191 @@
+#
+# BitBake Toaster Implementation
+#
+# Copyright (C) 2013        Intel Corporation
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+# Django settings for Toaster project.
+
+DEBUG = True
+TEMPLATE_DEBUG = DEBUG
+
+ADMINS = (
+    # ('Your Name', 'your_email@example.com'),
+)
+
+MANAGERS = ADMINS
+
+DATABASES = {
+    'default': {
+        'ENGINE': 'django.db.backends.sqlite3', # Add 'postgresql_psycopg2', 'mysql', 'sqlite3' or 'oracle'.
+        'NAME': 'toaster.sqlite',                      # Or path to database file if using sqlite3.
+        'USER': '',
+        'PASSWORD': '',
+        'HOST': '127.0.0.1',                      # Empty for localhost through domain sockets or '127.0.0.1' for localhost through TCP.
+        'PORT': '3306',                      # Set to empty string for default.
+    }
+}
+
+# Hosts/domain names that are valid for this site; required if DEBUG is False
+# See https://docs.djangoproject.com/en/1.5/ref/settings/#allowed-hosts
+ALLOWED_HOSTS = []
+
+# Local time zone for this installation. Choices can be found here:
+# http://en.wikipedia.org/wiki/List_of_tz_zones_by_name
+# although not all choices may be available on all operating systems.
+# In a Windows environment this must be set to your system time zone.
+
+# Always use local computer's time zone
+import time
+TIME_ZONE = time.tzname[0]
+
+# Language code for this installation. All choices can be found here:
+# http://www.i18nguy.com/unicode/language-identifiers.html
+LANGUAGE_CODE = 'en-us'
+
+SITE_ID = 1
+
+# If you set this to False, Django will make some optimizations so as not
+# to load the internationalization machinery.
+USE_I18N = True
+
+# If you set this to False, Django will not format dates, numbers and
+# calendars according to the current locale.
+USE_L10N = True
+
+# If you set this to False, Django will not use timezone-aware datetimes.
+USE_TZ = True
+
+# Absolute filesystem path to the directory that will hold user-uploaded files.
+# Example: "/var/www/example.com/media/"
+MEDIA_ROOT = ''
+
+# URL that handles the media served from MEDIA_ROOT. Make sure to use a
+# trailing slash.
+# Examples: "http://example.com/media/", "http://media.example.com/"
+MEDIA_URL = ''
+
+# Absolute path to the directory static files should be collected to.
+# Don't put anything in this directory yourself; store your static files
+# in apps' "static/" subdirectories and in STATICFILES_DIRS.
+# Example: "/var/www/example.com/static/"
+STATIC_ROOT = ''
+
+# URL prefix for static files.
+# Example: "http://example.com/static/", "http://static.example.com/"
+STATIC_URL = '/static/'
+
+# Additional locations of static files
+STATICFILES_DIRS = (
+    # Put strings here, like "/home/html/static" or "C:/www/django/static".
+    # Always use forward slashes, even on Windows.
+    # Don't forget to use absolute paths, not relative paths.
+)
+
+# List of finder classes that know how to find static files in
+# various locations.
+STATICFILES_FINDERS = (
+    'django.contrib.staticfiles.finders.FileSystemFinder',
+    'django.contrib.staticfiles.finders.AppDirectoriesFinder',
+#    'django.contrib.staticfiles.finders.DefaultStorageFinder',
+)
+
+# Make this unique, and don't share it with anybody.
+SECRET_KEY = 'NOT_SUITABLE_FOR_HOSTED_DEPLOYMENT'
+
+# List of callables that know how to import templates from various sources.
+TEMPLATE_LOADERS = (
+    'django.template.loaders.filesystem.Loader',
+    'django.template.loaders.app_directories.Loader',
+#     'django.template.loaders.eggs.Loader',
+)
+
+MIDDLEWARE_CLASSES = (
+    'django.middleware.common.CommonMiddleware',
+    'django.contrib.sessions.middleware.SessionMiddleware',
+    'django.middleware.csrf.CsrfViewMiddleware',
+    'django.contrib.auth.middleware.AuthenticationMiddleware',
+    'django.contrib.messages.middleware.MessageMiddleware',
+    # Uncomment the next line for simple clickjacking protection:
+    # 'django.middleware.clickjacking.XFrameOptionsMiddleware',
+)
+
+ROOT_URLCONF = 'toastermain.urls'
+
+# Python dotted path to the WSGI application used by Django's runserver.
+WSGI_APPLICATION = 'toastermain.wsgi.application'
+
+TEMPLATE_DIRS = (
+    # Put strings here, like "/home/html/django_templates" or "C:/www/django/templates".
+    # Always use forward slashes, even on Windows.
+    # Don't forget to use absolute paths, not relative paths.
+)
+
+INSTALLED_APPS = (
+    #'django.contrib.auth',
+    #'django.contrib.contenttypes',
+    #'django.contrib.sessions',
+    #'django.contrib.sites',
+    #'django.contrib.messages',
+    'django.contrib.staticfiles',
+    # Uncomment the next line to enable the admin:
+    # 'django.contrib.admin',
+    # Uncomment the next line to enable admin documentation:
+    # 'django.contrib.admindocs',
+    'orm',
+    'toastermain',
+    'bldviewer',
+    'toastergui',
+)
+
+# A sample logging configuration. The only tangible logging
+# performed by this configuration is to send an email to
+# the site admins on every HTTP 500 error when DEBUG=False.
+# See http://docs.djangoproject.com/en/dev/topics/logging for
+# more details on how to customize your logging configuration.
+LOGGING = {
+    'version': 1,
+    'disable_existing_loggers': False,
+    'filters': {
+        'require_debug_false': {
+            '()': 'django.utils.log.RequireDebugFalse'
+        }
+    },
+    'handlers': {
+        'mail_admins': {
+            'level': 'ERROR',
+            'filters': ['require_debug_false'],
+            'class': 'django.utils.log.AdminEmailHandler'
+        }
+    },
+    'loggers': {
+        'django.request': {
+            'handlers': ['mail_admins'],
+            'level': 'ERROR',
+            'propagate': True,
+        },
+    }
+}
+
+# If we're using sqlite, we need to tweak the performance a bit
+from django.db.backends.signals import connection_created
+def activate_synchronous_off(sender, connection, **kwargs):
+    if connection.vendor == 'sqlite':
+        cursor = connection.cursor()
+        cursor.execute('PRAGMA synchronous = 0;')
+connection_created.connect(activate_synchronous_off)
+#
+
+

bitbake/lib/toaster/toastermain/urls.py

@@ -0,0 +1,41 @@
+#
+# BitBake Toaster Implementation
+#
+# Copyright (C) 2013        Intel Corporation
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+from django.conf.urls import patterns, include, url
+from django.views.generic.simple import redirect_to
+from django.views.decorators.cache import never_cache
+
+
+# Uncomment the next two lines to enable the admin:
+# from django.contrib import admin
+# admin.autodiscover()
+
+urlpatterns = patterns('',
+    url(r'^simple/', include('bldviewer.urls')),
+    url(r'^api/1.0/', include('bldviewer.api')),
+    url(r'^gui/', include('toastergui.urls')),
+    url(r'^$', never_cache(redirect_to), {'url': '/simple/'}),
+    # Examples:
+    # url(r'^toaster/', include('toaster.foo.urls')),
+
+    # Uncomment the admin/doc line below to enable admin documentation:
+    # url(r'^admin/doc/', include('django.contrib.admindocs.urls')),
+
+    # Uncomment the next line to enable the admin:
+    # url(r'^admin/', include(admin.site.urls)),
+)

bitbake/lib/toaster/toastermain/wsgi.py

@@ -0,0 +1,32 @@
+"""
+WSGI config for Toaster project.
+
+This module contains the WSGI application used by Django's development server
+and any production WSGI deployments. It should expose a module-level variable
+named ``application``. Django's ``runserver`` and ``runfcgi`` commands discover
+this application via the ``WSGI_APPLICATION`` setting.
+
+Usually you will have the standard Django WSGI application here, but it also
+might make sense to replace the whole Django WSGI application with a custom one
+that later delegates to the Django one. For example, you could introduce WSGI
+middleware here, or combine a Django application with an application of another
+framework.
+
+"""
+import os
+
+# We defer to a DJANGO_SETTINGS_MODULE already in the environment. This breaks
+# if running multiple sites in the same mod_wsgi process. To fix this, use
+# mod_wsgi daemon mode with each site in its own daemon process, or use
+# os.environ["DJANGO_SETTINGS_MODULE"] = "Toaster.settings"
+os.environ.setdefault("DJANGO_SETTINGS_MODULE", "toastermain.settings")
+
+# This application object is used by any WSGI server configured to use this
+# file. This includes Django's development server, if the WSGI_APPLICATION
+# setting points here.
+from django.core.wsgi import get_wsgi_application
+application = get_wsgi_application()
+
+# Apply WSGI middleware here.
+# from helloworld.wsgi import HelloWorldApplication
+# application = HelloWorldApplication(application)

documentation/Makefile

@@ -200,7 +200,8 @@ TARFILES = mega-manual.html mega-style.css figures/yocto-environment.png figures
 	figures/yocto-environment-ref.png figures/user-configuration.png figures/source-input.png \
 	figures/package-feeds.png figures/layer-input.png figures/images.png figures/sdk.png \
 	figures/source-fetching.png figures/patching.png figures/configuration-compile-autoreconf.png \
-	figures/analysis-for-package-splitting.png
+	figures/analysis-for-package-splitting.png figures/image-generation.png \
+	figures/sdk-generation.png
 	endif
 
 MANUALS = $(DOC)/$(DOC).html
@@ -219,7 +220,8 @@ TARFILES = ref-manual.html ref-style.css figures/poky-title.png \
 	figures/user-configuration.png figures/yocto-environment-ref.png \
 	figures/images.png figures/sdk.png figures/source-fetching.png \
 	figures/patching.png figures/configuration-compile-autoreconf.png \
-	figures/analysis-for-package-splitting.png
+	figures/analysis-for-package-splitting.png figures/image-generation.png \
+	figures/sdk-generation.png
 MANUALS = $(DOC)/$(DOC).html $(DOC)/$(DOC).pdf $(DOC)/eclipse
 FIGURES = figures
 STYLESHEET = $(DOC)/*.css
@@ -381,8 +383,8 @@ publish:
             echo " "; \
             echo "******** Publishing "$(DOC)".html"; \
             echo " "; \
-            scp -r $(MANUALS) $(STYLESHEET) www.yoctoproject.org:/srv/www/www.yoctoproject.org-docs/$(VER)/$(DOC); \
+            scp -r $(MANUALS) $(STYLESHEET) docs.yp:/var/www/www.yoctoproject.org-docs/$(VER)/$(DOC); \
-            cd $(DOC); scp -r $(FIGURES) www.yoctoproject.org:/srv/www/www.yoctoproject.org-docs/$(VER)/$(DOC); \
+            cd $(DOC); scp -r $(FIGURES) docs.yp:/var/www/www.yoctoproject.org-docs/$(VER)/$(DOC); \
 	else \
           echo " "; \
           echo $(DOC)".html missing.  Generate the file first then try again."; \

documentation/adt-manual/adt-manual.xml

@@ -63,7 +63,7 @@
             </revision>
             <revision>
                 <revnumber>1.5</revnumber>
-                <date>Sometime in 2013</date>
+                <date>October 2013</date>
                 <revremark>Released with the Yocto Project 1.5 Release.</revremark>
             </revision>
        </revhistory>

documentation/adt-manual/adt-prepare.xml

@@ -268,7 +268,7 @@
                             target, go into the <filename>x86_64</filename>
                             folder and download the following installer:
                             <literallayout class='monospaced'>
-     poky-eglibc-x86_64-core-image-sato-i586.sh
+     poky-eglibc-x86_64-core-image-sato-i586-&DISTRO;.sh
                             </literallayout></para></listitem>
                         <listitem><para>Build your own toolchain installer.
                             For cases where you cannot use an installer
@@ -287,7 +287,7 @@
                     The example assumes the toolchain installer is located
                     in <filename>~/Downloads/</filename>.
                     <literallayout class='monospaced'>
-     $ ~/Downloads/poky-eglibc-x86_64-core-image-sato-i586.sh
+     $ ~/Downloads/poky-eglibc-x86_64-core-image-sato-i586-&DISTRO;.sh
                     </literallayout>
                     <note>
                         If you do not have write permissions for the directory
@@ -376,10 +376,11 @@
     </para>
 
     <para>
-        Be sure to run the environment setup script that matches the architecture for
+        Be sure to run the environment setup script that matches the
-        which you are developing.
+        architecture for which you are developing.
-        Environment setup scripts begin with the string "<filename>environment-setup</filename>"
+        Environment setup scripts begin with the string
-        and include as part of their name the architecture.
+        "<filename>environment-setup</filename>" and include as part of their
+        name the architecture.
         For example, the toolchain environment setup script for a 64-bit
         IA-based architecture installed in the default installation directory
         would be the following:
@@ -542,7 +543,7 @@
 
     <para>
         A final method of building the toolchain installer exists that has
-        significant advantages over the previous two methods.
+        significant advantages over the previous method.
         This method results in a toolchain installer that contains the sysroot
         that matches your target root filesystem.
         To build this installer, use the

documentation/bsp-guide/bsp-guide.xml

@@ -75,7 +75,7 @@
             </revision>
             <revision>
                 <revnumber>1.5</revnumber>
-                <date>Sometime in 2013</date>
+                <date>October 2013</date>
                 <revremark>Released with the Yocto Project 1.5 Release.</revremark>
             </revision>
         </revhistory>

documentation/bsp-guide/bsp.xml

@@ -183,17 +183,15 @@
      meta-crownbay/recipes-graphics/xorg-xserver/xserver-xf86-config/
      meta-crownbay/recipes-graphics/xorg-xserver/xserver-xf86-config/crownbay/
      meta-crownbay/recipes-graphics/xorg-xserver/xserver-xf86-config/crownbay/xorg.conf
-     meta-crownbay/recipes-graphics/xorg-xserver/xserver-xf86-config/crownbay-noemgd/
-     meta-crownbay/recipes-graphics/xorg-xserver/xserver-xf86-config/crownbay-noemgd/xorg.conf
      meta-crownbay/recipes-kernel/
      meta-crownbay/recipes-kernel/linux/
-     meta-crownbay/recipes-kernel/linux/linux-yocto_3.2.bbappend
      meta-crownbay/recipes-kernel/linux/linux-yocto_3.4.bbappend
      meta-crownbay/recipes-kernel/linux/linux-yocto_3.8.bbappend
+     meta-crownbay/recipes-kernel/linux/linux-yocto_3.10.bbappend
      meta-crownbay/recipes-kernel/linux/linux-yocto-dev.bbappend
-     meta-crownbay/recipes-kernel/linux/linux-yocto-rt_3.2.bbappend
      meta-crownbay/recipes-kernel/linux/linux-yocto-rt_3.4.bbappend
      meta-crownbay/recipes-kernel/linux/linux-yocto-rt_3.8.bbappend
+     meta-crownbay/recipes-kernel/linux/linux-yocto-rt_3.10.bbappend
                 </literallayout>
             </para>
 
@@ -376,15 +374,6 @@
             <para>
                 Each BSP Layer requires at least one machine file.
                 However, you can supply more than one file.
-                For example, in the Crown Bay BSP shown earlier in this section, the
-                <filename>conf/machine</filename> directory contains two configuration files:
-                <filename>crownbay.conf</filename> and <filename>crownbay-noemgd.conf</filename>.
-                The <filename>crownbay.conf</filename> file is used for the Crown Bay BSP
-                that supports the <trademark class='registered'>Intel</trademark> Embedded
-                Media and Graphics Driver (<trademark class='registered'>Intel</trademark>
-                EMGD), while the <filename>crownbay-noemgd</filename> file is used for the
-                Crown Bay BSP that supports Video Electronics Standards Association (VESA)
-                graphics only.
             </para>
 
             <para>
@@ -428,15 +417,13 @@
                 This optional directory contains miscellaneous recipe files for the BSP.
                 Most notably would be the formfactor files.
                 For example, in the Crown Bay BSP there is the
-                <filename>formfactor_0.0.bbappend</filename> file, which is an append file used
+                <filename>formfactor_0.0.bbappend</filename> file, which is an
-                to augment the recipe that starts the build.
+                append file used to augment the recipe that starts the build.
-                Furthermore, there are machine-specific settings used during the build that are
+                Furthermore, there are machine-specific settings used during the
-                defined by the <filename>machconfig</filename> files.
+                build that are defined by the <filename>machconfig</filename>.
-                In the Crown Bay example, two <filename>machconfig</filename> files exist:
+                In the Crown Bay example, two <filename>machconfig</filename> files
-                one that supports the
+                exist: one that supports the Intel® Embedded Media and Graphics
-                <trademark class='registered'>Intel</trademark> Embedded
+                Driver (Intel® EMGD) and one that does not:
-                Media and Graphics Driver (<trademark class='registered'>Intel</trademark>
-                EMGD) and one that does not:
                 <literallayout class='monospaced'>
      meta-crownbay/recipes-bsp/formfactor/formfactor/crownbay/machconfig
      meta-crownbay/recipes-bsp/formfactor/formfactor/crownbay-noemgd/machconfig
@@ -467,15 +454,12 @@
                 This optional directory contains recipes for the BSP if it has
                 special requirements for graphics support.
                 All files that are needed for the BSP to support a display are kept here.
-                For example, the Crown Bay BSP contains two versions of the
+                For example, the Crown Bay BSP's <filename>xorg.conf</filename> file
-                <filename>xorg.conf</filename> file.
+                detects the graphics support needed (i.e. the Intel® Embedded Media
-                The version in <filename>crownbay</filename> builds a BSP that supports the
+                Graphics Driver (EMGD) or the Video Electronics Standards Association
-                <trademark class='registered'>Intel</trademark> Embedded Media Graphics Driver (EMGD),
+                (VESA) graphics):
-                while the version in <filename>crownbay-noemgd</filename> builds
-                a BSP that supports Video Electronics Standards Association (VESA) graphics only:
                 <literallayout class='monospaced'>
      meta-crownbay/recipes-graphics/xorg-xserver/xserver-xf86-config_0.1.bbappend
-     meta-crownbay/recipes-graphics/xorg-xserver/xserver-xf86-config/crownbay/xorg.conf
      meta-crownbay/recipes-graphics/xorg-xserver/xserver-xf86-config/crownbay-noemgd/xorg.conf
                 </literallayout>
             </para>
@@ -502,28 +486,28 @@
                 the <filename>meta-&lt;bsp_name&gt;/recipes-kernel/linux</filename> directory).
             </para>
             <para>
-                Suppose you are using the <filename>linux-yocto_3.8.bb</filename> recipe to build
+                Suppose you are using the <filename>linux-yocto_3.10.bb</filename> recipe to build
                 the kernel.
                 In other words, you have selected the kernel in your
                 <filename>&lt;bsp_name&gt;.conf</filename> file by adding these types
                 of statements:
                 <literallayout class='monospaced'>
      PREFERRED_PROVIDER_virtual/kernel ?= "linux-yocto"
-     PREFERRED_VERSION_linux-yocto ?= "3.8%"
+     PREFERRED_VERSION_linux-yocto ?= "3.10%"
                 </literallayout>
                 <note>
                     When the preferred provider is assumed by default, the
                     <filename>PREFERRED_PROVIDER</filename> statement does not appear in the
                     <filename>&lt;bsp_name&gt;.conf</filename> file.
                 </note>
-                You would use the <filename>linux-yocto_3.8.bbappend</filename> file to append
+                You would use the <filename>linux-yocto_3.10.bbappend</filename> file to append
                 specific BSP settings to the kernel, thus configuring the kernel for your particular BSP.
             </para>
             <para>
                 As an example, look at the existing Crown Bay BSP.
                 The append file used is:
                 <literallayout class='monospaced'>
-     meta-crownbay/recipes-kernel/linux/linux-yocto_3.8.bbappend
+     meta-crownbay/recipes-kernel/linux/linux-yocto_3.10.bbappend
                 </literallayout>
                 The following listing shows the file.
                 Be aware that the actual commit ID strings in this example listing might be different
@@ -532,46 +516,18 @@
                 <literallayout class='monospaced'>
      FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
 
-     COMPATIBLE_MACHINE_crownbay = "crownbay"
-     KMACHINE_crownbay = "crownbay"
-     KBRANCH_crownbay = "standard/crownbay"
-     KERNEL_FEATURES_crownbay_append = " features/drm-emgd/drm-emgd-1.16 cfg/vesafb"
-
      COMPATIBLE_MACHINE_crownbay-noemgd = "crownbay-noemgd"
      KMACHINE_crownbay-noemgd = "crownbay"
      KBRANCH_crownbay-noemgd = "standard/crownbay"
-     KERNEL_FEATURES_crownbay-noemgd_append = " cfg/vesafb"
+     KERNEL_FEATURES_append_crownbay-noemgd = " cfg/vesafb"
 
-     LINUX_VERSION = "3.8.4"
+     LINUX_VERSION = "3.10.11"
 
-     SRCREV_meta_crownbay = "2a6d36e75ca0a121570a389d7bab76ec240cbfda"
+     SRCREV_meta_crownbay-noemgd = "285f93bf942e8f6fa678ffc6cc53696ed5400718"
-     SRCREV_machine_crownbay = "47aed0c17c1c55988198ad39f86ae88894c8e0a4"
+     SRCREV_machine_crownbay-noemgd = "702040ac7c7ec66a29b4d147665ccdd0ff015577"
-     SRCREV_emgd_crownbay = "c780732f175ff0ec866fac2130175876b519b576"
-
-     SRCREV_meta_crownbay-noemgd = "2a6d36e75ca0a121570a389d7bab76ec240cbfda"
-     SRCREV_machine_crownbay-noemgd = "47aed0c17c1c55988198ad39f86ae88894c8e0a4"
-
-     SRC_URI_crownbay = "git://git.yoctoproject.org/linux-yocto-3.8.git;protocol=git;nocheckout=1;branch=${KBRANCH},${KMETA},emgd-1.16;name=machine,meta,emgd"
                 </literallayout>
-                This append file contains statements used to support the Crown Bay BSP for both
+                This append file contains statements used to support the Crown Bay BSP.
-                <trademark class='registered'>Intel</trademark> EMGD and the VESA graphics.
+                The file defines <filename>crownbay</filename> as the
-                The build process, in this case, recognizes and uses only the statements that
-                apply to the defined machine name - <filename>crownbay</filename> in this case.
-                So, the applicable statements in the <filename>linux-yocto_3.8.bbappend</filename>
-                file are follows:
-                <literallayout class='monospaced'>
-     FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
-
-     COMPATIBLE_MACHINE_crownbay = "crownbay"
-     KMACHINE_crownbay = "crownbay"
-     KBRANCH_crownbay = "standard/crownbay"
-     KERNEL_FEATURES_crownbay_append = " features/drm-emgd/drm-emgd-1.16 cfg/vesafb"
-
-     SRCREV_meta_crownbay = "2a6d36e75ca0a121570a389d7bab76ec240cbfda"
-     SRCREV_machine_crownbay = "47aed0c17c1c55988198ad39f86ae88894c8e0a4"
-     SRCREV_emgd_crownbay = "c780732f175ff0ec866fac2130175876b519b576"
-                </literallayout>
-                The append file defines <filename>crownbay</filename> as the
                 <ulink url='&YOCTO_DOCS_REF_URL;#var-COMPATIBLE_MACHINE'><filename>COMPATIBLE_MACHINE</filename></ulink>
                 and uses the
                 <ulink url='&YOCTO_DOCS_REF_URL;#var-KMACHINE'><filename>KMACHINE</filename></ulink> variable to
@@ -588,12 +544,6 @@
                 <ulink url='&YOCTO_DOCS_DEV_URL;#source-directory'>Source Directory</ulink> Git
                 repository and the <filename>meta</filename> Git repository branches to identify the
                 exact kernel needed to build the Crown Bay BSP.
-                <note>
-                    For <filename>crownbay</filename>, a specific commit is also needed to point
-                    to the branch that supports EMGD graphics.
-                    At a minimum, every BSP points to the
-                    <filename>machine</filename> and <filename>meta</filename> commits.
-                </note>
             </para>
 
             <para>
@@ -1281,31 +1231,35 @@
                     Following is the complete example:
                     <literallayout class='monospaced'>
      $ yocto-bsp create myarm qemu
+     Checking basic git connectivity...
+     Done.
+
      Which qemu architecture would you like to use? [default: i386]
-	     1) i386    (32-bit)
+     	     1) i386    (32-bit)
 	     2) x86_64  (64-bit)
 	     3) ARM     (32-bit)
 	     4) PowerPC (32-bit)
 	     5) MIPS    (32-bit)
      3
-     Would you like to use the default (3.8) kernel? (y/n) [default: y]
+     Would you like to use the default (3.10) kernel? (y/n) [default: y] y
      Do you need a new machine branch for this BSP (the alternative is to re-use an existing branch)? [y/n] [default: y]
-     Getting branches from remote repo git://git.yoctoproject.org/linux-yocto-3.8.git...
+     Getting branches from remote repo git://git.yoctoproject.org/linux-yocto-3.10.git...
      Please choose a machine branch to base your new BSP branch on: [default: standard/base]
-             1) standard/arm-versatile-926ejs
+	     1) standard/arm-versatile-926ejs
-             2) standard/base
+	     2) standard/base
-             3) standard/beagleboard
+	     3) standard/beagleboard
-             4) standard/ck
+	     4) standard/ck
-             5) standard/crownbay
+	     5) standard/crownbay
-             6) standard/edf
+	     6) standard/edf
-             7) standard/emenlow
+	     7) standard/emenlow
-             8) standard/fri2
+	     8) standard/fri2
-             9) standard/fsl-mpc8315e-rdb
+	     9) standard/fsl-mpc8315e-rdb
-             10) standard/mti-malta32
+	     10) standard/minnow
-             11) standard/mti-malta64
+	     11) standard/mti-malta32
-             12) standard/qemuppc
+	     12) standard/mti-malta64
-             13) standard/routerstationpro
+	     13) standard/qemuppc
-             14) standard/sys940x
+	     14) standard/routerstationpro
+	     15) standard/sys940x
      1
      Would you like SMP support? (y/n) [default: y]
      Does your BSP have a touchscreen? (y/n) [default: n]
@@ -1320,7 +1274,7 @@
                             In the example, we use the ARM architecture.
                             </para></listitem>
                         <listitem><para>The script then prompts you for the kernel.
-                            The default 3.8 kernel is acceptable.
+                            The default 3.10 kernel is acceptable.
                             So, the example accepts the default.
                             If you enter 'n', the script prompts you to further enter the kernel
                             you do want to use (e.g. 3.2, 3.2_preempt-rt, and so forth.).</para></listitem>

documentation/dev-manual/dev-manual-intro.xml

@@ -24,17 +24,6 @@
             set up your host development system and build an image, which you
             find in the
             <ulink url='&YOCTO_DOCS_QS_URL;'>Yocto Project Quick Start</ulink>.
-            <note>
-                By default, using the Yocto Project creates a Poky distribution.
-                However, you can create your own distribution by providing key
-                <link linkend='metadata'>Metadata</link>.
-                A good example is Angstrom, which has had a distribution
-                based on the Yocto Project since its inception.
-                Other examples include  commercial distributions like
-                Wind River Linux, Mentor Embedded Linux, and ENEA Linux.
-                See the "<link linkend='creating-your-own-distribution'>Creating Your Own Distribution</link>"
-                section for more information.
-            </note>
         </para>
 
         <para>
@@ -43,13 +32,25 @@
             reconfigure the kernel, and develop an application using the
             popular <trademark class='trade'>Eclipse</trademark> IDE.
         </para>
+
+        <note>
+            By default, using the Yocto Project creates a Poky distribution.
+            However, you can create your own distribution by providing key
+            <link linkend='metadata'>Metadata</link>.
+            A good example is Angstrom, which has had a distribution
+            based on the Yocto Project since its inception.
+            Other examples include  commercial distributions like
+            Wind River Linux, Mentor Embedded Linux, and ENEA Linux.
+            See the "<link linkend='creating-your-own-distribution'>Creating Your Own Distribution</link>"
+            section for more information.
+        </note>
     </section>
 
     <section id='what-this-manual-provides'>
         <title>What This Manual Provides</title>
 
         <para>
-            The following list describes what you can get from this guide:
+            The following list describes what you can get from this manual:
             <itemizedlist>
                 <listitem><para>Information that lets you get set
                     up to develop using the Yocto Project.</para></listitem>
@@ -75,17 +76,17 @@
         <para>
             This manual will not give you the following:
             <itemizedlist>
-                <listitem><para>Step-by-step instructions if those instructions exist in other Yocto
+                <listitem><para><emphasis>Step-by-step instructions when those instructions exist in other Yocto
-                    Project documentation.
+                    Project documentation:</emphasis>
                     For example, the Yocto Project Application Developer's Guide contains detailed
                     instructions on how to run the
-                    <ulink url='&YOCTO_DOCS_ADT_URL;#installing-the-adt'>Installing the ADT and Toolchains</ulink>,
+                    <ulink url='&YOCTO_DOCS_ADT_URL;#installing-the-adt'>ADT Installer</ulink>,
                     which is used to set up a cross-development environment.</para></listitem>
-                <listitem><para>Reference material.
+                <listitem><para><emphasis>Reference material:</emphasis>
                     This type of material resides in an appropriate reference manual.
                     For example, system variables are documented in the
                     <ulink url='&YOCTO_DOCS_REF_URL;'>Yocto Project Reference Manual</ulink>.</para></listitem>
-                <listitem><para>Detailed public information that is not specific to the Yocto Project.
+                <listitem><para><emphasis>Detailed public information that is not specific to the Yocto Project:</emphasis>
                     For example, exhaustive information on how to use Git is covered better through the
                     Internet than in this manual.</para></listitem>
             </itemizedlist>
@@ -109,7 +110,8 @@
                     with the Yocto Project and quickly begin building an image.</para></listitem>
                 <listitem><para><emphasis>
                     <ulink url='&YOCTO_DOCS_REF_URL;'>Yocto Project Reference Manual</ulink>:</emphasis> This manual is a reference
-                    guide to the OpenEmbedded build system known as "Poky."
+                    guide to the OpenEmbedded build system, which is based on BitBake.
+                    The build system is sometimes referred to as "Poky".
                     </para></listitem>
                 <listitem><para><emphasis>
                     <ulink url='&YOCTO_DOCS_ADT_URL;'>Yocto Project Application Developer's Guide</ulink>:</emphasis>

documentation/dev-manual/dev-manual-newbie.xml

@@ -254,9 +254,9 @@
                 <listitem><para><ulink url='http://sitaramc.github.com/gitolite/master-toc.html'>The <filename>gitolite</filename> master index</ulink>:
                     All topics for <filename>gitolite</filename>.
                     </para></listitem>
-                <listitem><para><ulink url='http://hjemli.net/git/cgit/tree/README'><filename>cgit</filename> index</ulink>:
+                <listitem><para><ulink url='https://git.wiki.kernel.org/index.php/Interfaces,_frontends,_and_tools'>Interfaces, frontends, and tools</ulink>:
-                    A <filename>README</filename> file on how to create a
+                    Documentation on how to create interfaces and frontends
-                    fast web interface for Git.</para></listitem>
+                    for Git.</para></listitem>
             </itemizedlist>
         </note>
     </section>
@@ -275,9 +275,8 @@
         </para>
 
         <para>
-            See "<ulink url='http://autobuilder.yoctoproject.org:8010/'>Welcome to the buildbot for the Yocto Project</ulink>"
+            See "<ulink url='http://autobuilder.yoctoproject.org'>Yocto Project Autobuilder</ulink>"
-            for the Yocto Project's reference implementation that uses
+            for more information and links to buildbot.
-            buildbot.
             The Yocto Project team has found this implementation
             works well in this role.
             A public example of this is the Yocto Project
@@ -394,7 +393,7 @@
                     For some guidance on mailing lists to use, see the list in the
                     "<link linkend='how-to-submit-a-change'>How to Submit a Change</link>"
                     section.
-                    For a description of the available mailing lists, see
+                    For a description of the available mailing lists, see the
                     "<ulink url='&YOCTO_DOCS_REF_URL;#resources-mailinglist'>Mailing Lists</ulink>"
                     section in the Yocto Project Reference Manual.
                     </para></listitem>
@@ -615,7 +614,7 @@
                 "<ulink url='&YOCTO_DOCS_BSP_URL;#bsp-layers'>BSP Layers</ulink>"
                 section in the Yocto Project Board Support Packages (BSP)
                 Developer's Guide.</para></listitem>
-            <listitem><para id='meta-toochain'><emphasis>Meta-Toolchain:</emphasis>
+            <listitem><para id='meta-toolchain'><emphasis>Meta-Toolchain:</emphasis>
                 A term sometimes used for
                 <link linkend='cross-development-toolchain'>Cross-Development Toolchain</link>.
                 </para></listitem>
@@ -689,7 +688,7 @@
                 For example, downloading and unpacking <filename>&YOCTO_POKY_TARBALL;</filename>
                 results in a Source Directory whose top-level folder is named
                 <filename>&YOCTO_POKY;</filename>.
-                If you create a local copy of the Git repository, then you can name the repository
+                If you create a local copy of the Git repository, you can name the repository
                 anything you like.
                 Throughout much of the documentation, <filename>poky</filename> is used as the name of
                 the top-level folder of the local copy of the poky Git repository.
@@ -865,8 +864,8 @@
             It is important to understand that Git tracks content change and not files.
             Git uses "branches" to organize different development efforts.
             For example, the <filename>poky</filename> repository has
-            <filename>bernard</filename>,
+            <filename>denzil</filename>, <filename>danny</filename>,
-            <filename>edison</filename>, <filename>denzil</filename>, <filename>danny</filename>
+            <filename>dylan</filename>, <filename>dora</filename>,
             and <filename>master</filename> branches among others.
             You can see all the branches by going to
             <ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/'></ulink> and
@@ -1233,6 +1232,10 @@
                 occurred.</para></listitem>
             <listitem><para>Be sure to indicate the Severity of the bug.
                 Severity communicates how the bug impacted your work.</para></listitem>
+            <listitem><para>Select the appropriate "Documentation change" item
+                for the bug.
+                Fixing a bug may or may not affect the Yocto Project
+                documentation.</para></listitem>
             <listitem><para>Provide a brief summary of the issue.
                 Try to limit your summary to just a line or two and be sure to capture the
                 essence of the issue.</para></listitem>

documentation/dev-manual/dev-manual-start.xml

@@ -32,7 +32,7 @@
 
     <para>
         You can use the OpenEmbedded build system, which uses
-        BitBake to develop complete Linux
+        BitBake, to develop complete Linux
         images and associated user-space applications for architectures based
         on ARM, MIPS, PowerPC, x86 and x86-64.
         <note>
@@ -122,11 +122,11 @@
                         <literallayout class='monospaced'>
      $ git clone git://git.yoctoproject.org/poky
      Cloning into 'poky'...
-     remote: Counting objects: 183981, done.
+     remote: Counting objects: 203728, done.
-     remote: Compressing objects: 100% (47428/47428), done.
+     remote: Compressing objects: 100% (52371/52371), done.
-     remote: Total 183981 (delta 132271), reused 183703 (delta 132044)
+     remote: Total 203728 (delta 147444), reused 202891 (delta 146614)
-     Receiving objects: 100% (183981/183981), 89.71 MiB | 2.93 MiB/s, done.
+     Receiving objects: 100% (203728/203728), 95.54 MiB | 308 KiB/s, done.
-     Resolving deltas: 100% (132271/132271), done.
+     Resolving deltas: 100% (147444/147444), done.
                         </literallayout></para>
                         <para>For another example of how to set up your own local Git repositories, see this
                         <ulink url='&YOCTO_WIKI_URL;/wiki/Transcript:_from_git_checkout_to_meta-intel_BSP'>
@@ -145,27 +145,28 @@
                 For simplicity, it is recommended that you create these structures outside of the
                 Source Directory (usually <filename>poky</filename>).</para>
                 <para>As an example, the following transcript shows how to create the bare clone
-                of the <filename>linux-yocto-3.8</filename> kernel and then create a copy of
+                of the <filename>linux-yocto-3.10</filename> kernel and then create a copy of
                 that clone.
                 <note>When you have a local Yocto Project kernel Git repository, you can
                 reference that repository rather than the upstream Git repository as
                 part of the <filename>clone</filename> command.
                 Doing so can speed up the process.</note></para>
                 <para>In the following example, the bare clone is named
-                <filename>linux-yocto-3.8.git</filename>, while the
+                <filename>linux-yocto-3.10.git</filename>, while the
-                copy is named <filename>my-linux-yocto-3.8-work</filename>:
+                copy is named <filename>my-linux-yocto-3.10-work</filename>:
                 <literallayout class='monospaced'>
-     $ git clone --bare git://git.yoctoproject.org/linux-yocto-3.8 linux-yocto-3.8.git
+     $ git clone --bare git://git.yoctoproject.org/linux-yocto-3.10 linux-yocto-3.10.git
-     Cloning into bare repository 'linux-yocto-3.8.git'...
+     Cloning into bare repository 'linux-yocto-3.10.git'...
-     remote: Counting objects: 2847090, done.
+     remote: Counting objects: 3364487, done.
-     remote: Compressing objects: 100% (454675/454675), done.
+     remote: Compressing objects: 100% (507178/507178), done.
-     remote: Total 2847090 (delta 2386170), reused 2825793 (delta 2364886)
+     remote: Total 3364487 (delta 2827715), reused 3364481 (delta 2827709)
-     Receiving objects: 100% (2847090/2847090), 603.19 MiB | 3.54 MiB/s, done.
+     Receiving objects: 100% (3364487/3364487), 722.95 MiB | 423 KiB/s, done.
-     Resolving deltas: 100% (2386170/2386170), done.                </literallayout></para>
+     Resolving deltas: 100% (2827715/2827715), done.
+                </literallayout></para>
                 <para>Now create a clone of the bare clone just created:
                 <literallayout class='monospaced'>
-     $ git clone linux-yocto-3.8.git my-linux-yocto-3.8-work
+     $ git clone linux-yocto-3.10.git my-linux-yocto-3.10-work
-     Cloning into 'my-linux-yocto-3.8-work'...
+     Cloning into 'my-linux-yocto-3.10-work'...
      done.
                 </literallayout></para></listitem>
             <listitem id='meta-yocto-kernel-extras-repo'><para><emphasis>
@@ -189,11 +190,12 @@
      $ cd ~/poky
      $ git clone git://git.yoctoproject.org/meta-yocto-kernel-extras meta-yocto-kernel-extras
      Cloning into 'meta-yocto-kernel-extras'...
-     remote: Counting objects: 690, done.
+     remote: Counting objects: 727, done.
-     remote: Compressing objects: 100% (431/431), done.
+     remote: Compressing objects: 100% (452/452), done.
-     remote: Total 690 (delta 238), reused 690 (delta 238)
+     remote: Total 727 (delta 260), reused 719 (delta 252)
-     Receiving objects: 100% (690/690), 532.60 KiB, done.
+     Receiving objects: 100% (727/727), 536.36 KiB | 102 KiB/s, done.
-     Resolving deltas: 100% (238/238), done.                </literallayout></para></listitem>
+     Resolving deltas: 100% (260/260), done.
+               </literallayout></para></listitem>
            <listitem><para id='supported-board-support-packages-(bsps)'><emphasis>Supported Board
                 Support Packages (BSPs):</emphasis>
                 The Yocto Project provides a layer called <filename>meta-intel</filename> and
@@ -226,8 +228,8 @@
                         BSP tarball from the same "Downloads" page of the
                         <ulink url='&YOCTO_HOME_URL;'>Yocto Project Website</ulink>
                         to get the Yocto Project release.
-                        Once on the "Download" page, look for "BSP" under the
+                        Once on the "Download" page, look to the right of the
-                        "Type" heading.</para>
+                        page and scroll down to find the BSP tarballs.</para>
                         <para>Once you have the tarball, just extract it into a directory of your choice.
                         Again, this method just produces a snapshot of the BSP layer in the form
                         of a hierarchical directory structure.</para></listitem>
@@ -246,16 +248,17 @@
      $ cd ~/poky
      $ git clone git://git.yoctoproject.org/meta-intel.git
      Cloning into 'meta-intel'...
-     remote: Counting objects: 6264, done.
+     remote: Counting objects: 7366, done.
-     remote: Compressing objects: 100% (2135/2135), done.
+     remote: Compressing objects: 100% (2491/2491), done.
-     remote: Total 6264 (delta 3321), reused 6235 (delta 3293)
+     remote: Total 7366 (delta 3997), reused 7299 (delta 3930)
-     Receiving objects: 100% (6264/6264), 2.17 MiB | 2.63 MiB/s, done.
+     Receiving objects: 100% (7366/7366), 2.31 MiB | 95 KiB/s, done.
-     Resolving deltas: 100% (3321/3321), done.
+     Resolving deltas: 100% (3997/3997), done.
                         </literallayout></para>
                         <para>The same
-                        <ulink url='&YOCTO_WIKI_URL;/wiki/Transcript:_from_git_checkout_to_meta-intel_BSP'>
+                        <ulink url='&YOCTO_WIKI_URL;/wiki/Transcript:_from_git_checkout_to_meta-intel_BSP'>wiki page</ulink>
-                        wiki page</ulink> referenced earlier covers how to
+                        referenced earlier covers how to
-                        set up the <filename>meta-intel</filename> Git repository.</para></listitem>
+                        set up the <filename>meta-intel</filename> Git repository.
+                        </para></listitem>
                 </itemizedlist></para></listitem>
             <listitem><para><emphasis>Eclipse Yocto Plug-in:</emphasis>  If you are developing
                 applications using the Eclipse Integrated Development Environment (IDE),

documentation/dev-manual/dev-manual.xml

@@ -53,7 +53,7 @@
             </revision>
             <revision>
                 <revnumber>1.5</revnumber>
-                <date>Sometime in 2013</date>
+                <date>October 2013</date>
                 <revremark>Released with the Yocto Project 1.5 Release.</revremark>
             </revision>
         </revhistory>

documentation/kernel-dev/kernel-dev-advanced.xml

@@ -296,6 +296,20 @@
      $ git commit --allow-empty -m "Create orphan meta branch"
             </literallayout>
         </para>
+
+        <para>
+            If you modify the Metadata in the linux-yocto 
+            <filename>meta</filename> branch, you must not forget to update 
+            the 
+            <ulink url='&YOCTO_DOCS_REF_URL;#var-SRCREV'><filename>SRCREV</filename></ulink>
+            statements in the kernel's recipe.
+            In particular, you need to update the 
+            <filename>SRCREV_meta</filename> variable to match the commit in 
+            the <filename>KMETA</filename> branch you wish to use.
+            Changing the data in these branches and not updating the 
+            <filename>SRCREV</filename> statements to match will cause the 
+            build to fetch an older commit.
+        </para>
     </section>
 </section>
 

documentation/kernel-dev/kernel-dev.xml

@@ -38,7 +38,7 @@
             </revision>
             <revision>
                 <revnumber>1.5</revnumber>
-                <date>Sometime in 2013</date>
+                <date>October 2013</date>
                 <revremark>Released with the Yocto Project 1.5 Release.</revremark>
             </revision>
         </revhistory>

documentation/poky.ent

@@ -1,6 +1,6 @@
 <!ENTITY DISTRO "1.5">
 <!ENTITY DISTRO_COMPRESSED "15">
-<!ENTITY DISTRO_NAME "tbd">
+<!ENTITY DISTRO_NAME "dora">
 <!ENTITY YOCTO_DOC_VERSION "1.5">
 <!ENTITY POKYVERSION "10.0.0">
 <!ENTITY POKYVERSION_COMPRESSED "1000">
@@ -26,6 +26,7 @@
 <!ENTITY ECLIPSE_UPDATES_URL "&ECLIPSE_DL_URL;/tm/updates/3.3">
 <!ENTITY ECLIPSE_INDIGO_URL "&ECLIPSE_DL_URL;/releases/indigo">
 <!ENTITY ECLIPSE_JUNO_URL "&ECLIPSE_DL_URL;/releases/juno">
+<!ENTITY ECLIPSE_KEPLER_URL "&ECLIPSE_DL_URL;/releases/kepler">
 <!ENTITY ECLIPSE_INDIGO_CDT_URL "&ECLIPSE_DL_URL;tools/cdt/releases/indigo">
 <!ENTITY YOCTO_DOCS_URL "&YOCTO_HOME_URL;/docs">
 <!ENTITY YOCTO_SOURCES_URL "&YOCTO_HOME_URL;/sources/">
@@ -35,7 +36,7 @@
 <!ENTITY YOCTO_RELEASE_DL_URL "&YOCTO_DL_URL;/releases/yocto/yocto-&DISTRO;">
 <!ENTITY YOCTO_TOOLCHAIN_DL_URL "&YOCTO_RELEASE_DL_URL;/toolchain/">
 <!ENTITY YOCTO_ECLIPSE_DL_URL "&YOCTO_RELEASE_DL_URL;/eclipse-plugin/indigo;">
-<!ENTITY YOCTO_ADTINSTALLER_DL_URL "&YOCTO_RELEASE_DL_URL;/adt_installer">
+<!ENTITY YOCTO_ADTINSTALLER_DL_URL "&YOCTO_RELEASE_DL_URL;/adt-installer">
 <!ENTITY YOCTO_POKY_DL_URL "&YOCTO_RELEASE_DL_URL;/&YOCTO_POKY;.tar.bz2">
 <!ENTITY YOCTO_MACHINES_DL_URL "&YOCTO_RELEASE_DL_URL;/machines">
 <!ENTITY YOCTO_QEMU_DL_URL "&YOCTO_MACHINES_DL_URL;/qemu">
@@ -56,8 +57,8 @@
 <!ENTITY UBUNTU_HOST_PACKAGES_ESSENTIAL "gawk wget git-core diffstat unzip texinfo \
      build-essential chrpath">
 <!ENTITY FEDORA_HOST_PACKAGES_ESSENTIAL "gawk make wget tar bzip2 gzip python unzip perl patch \
-     diffutils diffstat git cpp gcc gcc-c++ eglibc-devel texinfo chrpath \
+     diffutils diffstat git cpp gcc gcc-c++ glibc-devel texinfo chrpath \
-     ccache">
+     ccache perl-Data-Dumper perl-Text-ParseWords">
 <!ENTITY OPENSUSE_HOST_PACKAGES_ESSENTIAL "python gcc gcc-c++ git chrpath make wget python-xml \
      diffstat texinfo python-curses patch">
 <!ENTITY CENTOS_HOST_PACKAGES_ESSENTIAL "gawk make wget tar bzip2 gzip python unzip perl patch \

documentation/profile-manual/profile-manual-usage.xml

@@ -3094,7 +3094,7 @@
                 If you already have an LTTng trace on a remote target and
                 would like to view it in Eclipse on the host, you can easily
                 copy it from the target to the host and import it into
-                Eclipse to view it using the LTTng Eclipse plugin already
+                Eclipse to view it using the LTTng Eclipse plug-in already
                 bundled in the Eclipse (Juno SR1 or greater).
             </para>
 
@@ -3172,7 +3172,7 @@
 
             <para>
                 You can access extensive help information on how to use
-                the LTTng plugin to search and analyze captured traces via
+                the LTTng plug-in to search and analyze captured traces via
                 the Eclipse help system:
                 <literallayout class='monospaced'>
      Help | Help Contents | LTTng Plug-in User Guide

documentation/profile-manual/profile-manual.xml

@@ -38,7 +38,7 @@
             </revision>
             <revision>
                 <revnumber>1.5</revnumber>
-                <date>Sometime in 2013</date>
+                <date>October 2013</date>
                 <revremark>Released with the Yocto Project 1.5 Release.</revremark>
             </revision>
         </revhistory>

documentation/ref-manual/closer-look.xml

@@ -491,6 +491,14 @@
             <link linkend='var-DL_DIR'><filename>DL_DIR</filename></link>
             variable.
             This area is a cache that can hold previously downloaded source.
+            You can also instruct the OpenEmbedded build system to create
+            tarballs from Git repositories, which is not the default behavior,
+            and store them in the <filename>DL_DIR</filename> by using the
+            <link linkend='var-BB_GENERATE_MIRROR_TARBALLS'><filename>BB_GENERATE_MIRROR_TARBALLS</filename></link>
+            variable.
+        </para>
+
+        <para>
             Judicious use of a <filename>DL_DIR</filename> directory can
             save the build system a trip across the Internet when looking
             for files.
@@ -566,6 +574,15 @@
                 fetcher module.
             </para>
 
+            <note>
+                For information on how to have the OpenEmbedded build system
+                generate tarballs for Git repositories and place them in the
+                <link linkend='var-DL_DIR'><filename>DL_DIR</filename></link>
+                directory, see the
+                <link linkend='var-BB_GENERATE_MIRROR_TARBALLS'><filename>BB_GENERATE_MIRROR_TARBALLS</filename></link>
+                variable.
+            </note>
+
             <para>
                 When fetching a repository, BitBake uses the
                 <link linkend='var-SRCREV'><filename>SRCREV</filename></link>
@@ -601,6 +618,58 @@
         </section>
     </section>
 
+    <section id="package-feeds-dev-environment">
+        <title>Package Feeds</title>
+
+        <para>
+            When the OpenEmbedded build system generates an image or an SDK,
+            it gets the packages from a package feed area located in the
+            <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>.
+            The
+            <link linkend='general-yocto-environment-figure'>general Yocto Project Development Environment figure</link>
+            shows this package feeds area in the upper-right corner.
+        </para>
+
+        <para>
+            This section looks a little closer into the package feeds area used
+            by the build system.
+            Here is a more detailed look at the area:
+            <imagedata fileref="figures/package-feeds.png" align="center" width="7in" depth="6in" />
+        </para>
+
+        <para>
+            Package feeds are an intermediary step in the build process.
+            BitBake generates packages whose type is defined by the
+            <link linkend='var-PACKAGE_CLASSES'><filename>PACKAGE_CLASSES</filename></link>
+            variable.
+            Before placing the packages into package feeds,
+            the build process validates them with generated output quality
+            assurance checks through the
+            <link linkend='ref-classes-insane'><filename>insane.bbclass</filename></link>
+            class.
+        </para>
+
+        <para>
+            The package feed area resides in
+            <filename>tmp/deploy</filename> of the Build Directory.
+            Folders are created that correspond to the package type
+            (IPK, DEB, or RPM) created.
+            Further organization is derived through the value of the
+            <link linkend='var-PACKAGE_ARCH'><filename>PACKAGE_ARCH</filename></link>
+            variable for each package.
+            For example, packages can exist for the i586 or qemux86
+            architectures.
+            The package files themselves reside within the appropriate
+            architecture folder.
+        </para>
+
+        <para>
+            BitBake uses the <filename>do_package_write_*</filename> task to
+            place generated packages into the package holding area (e.g.
+            <filename>do_package_write_ipk</filename> for IPK packages).
+        </para>
+    </section>
+
     <section id='bitbake-dev-environment'>
         <title>BitBake</title>
 
@@ -618,7 +687,7 @@
             <para>
                 The first stages of building a recipe are to fetch and unpack
                 the source code:
-                <imagedata fileref="figures/source-fetching.png" align="center" width="6.5in" depth="4in" />
+                <imagedata fileref="figures/source-fetching.png" align="center" width="6.5in" depth="5in" />
             </para>
 
             <para>
@@ -830,58 +899,151 @@
                 </note>
             </para>
         </section>
-    </section>
 
-    <section id="package-feeds-dev-environment">
+        <section id='image-generation-dev-environment'>
-        <title>Package Feeds</title>
+            <title>Image Generation</title>
 
-        <para>
+            <para>
-            When the OpenEmbedded build system generates an image or an SDK,
+                Once packages are split and stored in the Package Feeds area,
-            it gets the packages from a package feed area located in the
+                the OpenEmbedded build system uses BitBake to generate the
-            <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>.
+                root filesystem image:
-            The 
+                <imagedata fileref="figures/image-generation.png" align="center" width="6in" depth="7in" />
-            <link linkend='general-yocto-environment-figure'>general Yocto Project Development Environment figure</link>
+            </para>
-            shows this package feeds area in the upper-right corner.
-        </para>
 
-        <para>
+            <para>
-            This section looks a little closer into the package feeds area used
+                The image generation process consists of several stages and
-            by the build system.
+                depends on many variables.
-            Here is a more detailed look at the area:
+                The <filename>do_rootfs</filename> task uses these key variables
-            <imagedata fileref="figures/package-feeds.png" align="center" width="7in" depth="6in" />
+                to help create the list of packages to actually install:
-        </para>
+                <itemizedlist>
+                    <listitem><para><link linkend='var-IMAGE_INSTALL'><filename>IMAGE_INSTALL</filename></link>:
+                        Lists out the base set of packages to install from
+                        the Package Feeds area.</para></listitem>
+                    <listitem><para><link linkend='var-PACKAGE_EXCLUDE'><filename>PACKAGE_EXCLUDE</filename></link>:
+                        Specifies packages that should not be installed.
+                        </para></listitem>
+                    <listitem><para><link linkend='var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></link>:
+                        Specifies features to include in the image.
+                        Most of these features map to additional packages for
+                        installation.</para></listitem>
+                    <listitem><para><link linkend='var-PACKAGE_CLASSES'><filename>PACKAGE_CLASSES</filename></link>:
+                        Specifies the package backend to use and consequently
+                        helps determine where to locate packages within the
+                        Package Feeds area.</para></listitem>
+                    <listitem><para><link linkend='var-IMAGE_LINGUAS'><filename>IMAGE_LINGUAS</filename></link>:
+                        Determines the language(s) for which additional
+                        language support packages are installed.
+                        </para></listitem>
+                </itemizedlist>
+            </para>
 
-        <para>
+            <para>
-            Package feeds are an intermediary step in the build process.
+                Package installation is under control of the package manager
-            BitBake generates packages whose type is defined by the
+                (e.g. smart/rpm, opkg, or apt/dpkg) regardless of whether or
-            <link linkend='var-PACKAGE_CLASSES'><filename>PACKAGE_CLASSES</filename></link>
+                not package management is enabled for the target.
-            variable.
+                At the end of the process, if package management is not
-            Before placing the packages into package feeds,
+                enabled for the target, the package manager's data files
-            the build process validates them with generated output quality
+                are deleted from the root filesystem.
-            assurance checks through the
+            </para>
-            <link linkend='ref-classes-insane'><filename>insane.bbclass</filename></link>
-            class.
-        </para>
 
-        <para>
+            <para>
-            The package feed area resides in
+                During image generation, the build system attempts to run
-            <filename>tmp/deploy</filename> of the Build Directory.
+                all post installation scripts.
-            Folders are created that correspond to the package type
+                Any that fail to run on the build host are run on the
-            (IPK, DEB, or RPM) created.
+                target when the target system is first booted.
-            Further organization is derived through the value of the
+                If you are using a
-            <link linkend='var-PACKAGE_ARCH'><filename>PACKAGE_ARCH</filename></link>
+                <ulink url='&YOCTO_DOCS_DEV_URL;#creating-a-read-only-root-filesystem'>read-only root filesystem</ulink>,
-            variable for each package.
+                all the post installation scripts must succeed during the
-            For example, packages can exist for the i586 or qemux86
+                package installation phase since the root filesystem cannot be
-            architectures.
+                written into.
-            The package files themselves reside within the appropriate
+            </para>
-            architecture folder.
-        </para>
 
-        <para>
+            <para>
-            BitBake uses the <filename>do_package_write_*</filename> task to
+                During Optimization, optimizing processes are run across
-            place generated packages into the package holding area (e.g.
+                the image.
-            <filename>do_package_write_ipk</filename> for IPK packages).
+                These processes include <filename>mklibs</filename> and
-        </para>
+                <filename>prelink</filename>.
+                The <filename>mklibs</filename> process optimizes the size
+                of the libraries.
+                A <filename>prelink</filename> process optimizes the dynamic
+                linking of shared libraries to reduce start up time of
+                executables.
+            </para>
+
+            <para>
+                Part of the image generation process includes compressing the
+                root filesystem image.
+                Compression is accomplished through several optimization
+                routines designed to reduce the overall size of the image.
+            </para>
+
+            <para>
+                After the root filesystem has been constructed, the image
+                generation process turns everything into an image file or
+                a set of image files.
+                The formats used for the root filesystem depend on the
+                <link linkend='var-IMAGE_FSTYPES'><filename>IMAGE_FSTYPES</filename></link>
+                variable.
+            </para>
+
+            <note>
+                The entire image generation process is run under Pseudo.
+                Running under Pseudo ensures that the files in the root
+                filesystem have correct ownership.
+            </note>
+        </section>
+
+        <section id='sdk-generation-dev-environment'>
+            <title>SDK Generation</title>
+
+            <para>
+                The OpenEmbedded build system uses BitBake to generate the
+                Software Development Kit (SDK) installer script:
+                <imagedata fileref="figures/sdk-generation.png" align="center" width="6in" depth="7in" />
+            </para>
+
+            <note>
+                For more information on the cross-development toolchain
+                generation, see the
+                "<link linkend='cross-development-toolchain-generation'>Cross-Development Toolchain Generation</link>"
+                section.
+            </note>
+
+            <para>
+                Like image generation, the SDK script process consists of
+                several stages and depends on many variables.
+                The <filename>do_populate_sdk</filename> task uses these
+                key variables to help create the list of packages to actually
+                install.
+                For information on the variables listed in the figure, see the
+                "<link linkend='sdk-dev-environment'>Application Development SDK</link>"
+                section.
+            </para>
+
+            <para>
+                The <filename>do_populate_sdk</filename> task handles two
+                parts: a target part and a host part.
+                The target part is the part built for the target hardware and
+                includes libraries and headers.
+                The host part is the part of the SDK that runs on the
+                <link linkend='var-SDKMACHINE'><filename>SDKMACHINE</filename></link>.
+            </para>
+
+            <para>
+                Once both parts are constructed, the
+                <filename>do_populate_sdk</filename> task performs some cleanup
+                on both parts.
+                After the cleanup, the task creates a cross-development
+                environment setup script and any configuration files that
+                might be needed.
+            </para>
+
+            <para>
+                The final output of the task is the Cross-development
+                toolchain installation script (<filename>.sh</filename> file),
+                which includes the environment setup script.
+            </para>
+        </section>
     </section>
 
     <section id='images-dev-environment'>
@@ -907,13 +1069,17 @@
         <para>
             Images are written out to the
             <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>
-            inside the <filename>deploy/images</filename> folder as shown
+            inside the <filename>deploy/images/&lt;machine&gt;/</filename>
-            in the figure.
+            folder as shown in the figure.
             This folder contains any files expected to be loaded on the
             target device.
             The
             <link linkend='var-DEPLOY_DIR'><filename>DEPLOY_DIR</filename></link>
-            variable points to the <filename>deploy</filename> directory.
+            variable points to the <filename>deploy</filename> directory,
+            while the
+            <link linkend='var-DEPLOY_DIR_IMAGE'><filename>DEPLOY_DIR_IMAGE</filename></link>
+            variable points to the appropriate directory containing images for
+            the current configuration.
             <itemizedlist>
                 <listitem><para><filename>&lt;kernel-image&gt;</filename>:
                     A kernel binary file.
@@ -922,8 +1088,9 @@
                     kernel image file.
                     Depending on that variable, the file could begin with
                     a variety of naming strings.
-                    The <filename>deploy/images</filename> directory can
+                    The <filename>deploy/images/&lt;machine&gt;</filename>
-                    contain multiple image files.</para></listitem>
+                    directory can contain multiple image files for the
+                    machine.</para></listitem>
                 <listitem><para><filename>&lt;root-filesystem-image&gt;</filename>:
                     Root filesystems for the target device (e.g.
                     <filename>*.ext3</filename> or <filename>*.bz2</filename>
@@ -931,25 +1098,27 @@
                     The <link linkend='var-IMAGE_FSTYPES'><filename>IMAGE_FSTYPES</filename></link>
                     variable setting determines the root filesystem image
                     type.
-                    The <filename>deploy/images</filename> directory can
+                    The <filename>deploy/images/&lt;machine&gt;</filename>
-                    contain multiple root filesystems.</para></listitem>
+                    directory can contain multiple root filesystems for the
+                    machine.</para></listitem>
                 <listitem><para><filename>&lt;kernel-modules&gt;</filename>:
                     Tarballs that contain all the modules built for the kernel.
                     Kernel module tarballs exist for legacy purposes and
                     can be suppressed by setting the
                     <link linkend='var-MODULE_TARBALL_DEPLOY'><filename>MODULE_TARBALL_DEPLOY</filename></link>
                     variable to "0".
-                    The <filename>deploy/images</filename> directory can
+                    The <filename>deploy/images/&lt;machine&gt;</filename>
-                    contain multiple kernel module tarballs.
+                    directory can contain multiple kernel module tarballs
-                    </para></listitem>
+                    for the machine.</para></listitem>
                 <listitem><para><filename>&lt;bootloaders&gt;</filename>:
                     Bootloaders supporting the image, if applicable to the
                     target machine.
-                    The <filename>deploy/images</filename> directory can
+                    The <filename>deploy/images/&lt;machine&gt;</filename>
-                    contain multiple bootloaders.
+                    directory can contain multiple bootloaders for the
-                    </para></listitem>
+                    machine.</para></listitem>
                 <listitem><para><filename>&lt;symlinks&gt;</filename>:
-                    The <filename>deploy/images</filename> folder contains
+                    The <filename>deploy/images/&lt;machine&gt;</filename>
+                    folder contains
                     a symbolic link that points to the most recently built file
                     for each machine.
                     These links might be useful for external scripts that

documentation/ref-manual/introduction.xml

@@ -133,17 +133,26 @@
                 </para>
             </note>
             <itemizedlist>
-                <listitem><para>Ubuntu 10.04</para></listitem>
+<!--                <listitem><para>Ubuntu 10.04</para></listitem>
-                <listitem><para>Ubuntu 11.10</para></listitem>
+                <listitem><para>Ubuntu 11.10</para></listitem> -->
                 <listitem><para>Ubuntu 12.04 (LTS)</para></listitem>
                 <listitem><para>Ubuntu 12.10</para></listitem>
                 <listitem><para>Ubuntu 13.04</para></listitem>
-                <listitem><para>Fedora release 17 (Beefy Miracle)</para></listitem>
+<!--                <listitem><para>Fedora 16 (Verne)</para></listitem>
+                <listitem><para>Fedora 17 (Spherical)</para></listitem> -->
                 <listitem><para>Fedora release 18 (Spherical Cow)</para></listitem>
-                <listitem><para>CentOS release 6.3 (Final)</para></listitem>
+                <listitem><para>Fedora release 19 (Schrödinger's Cat)</para></listitem>
-                <listitem><para>CentOS release 6.4 (Final)</para></listitem>
+<!--                <listitem><para>CentOS release 5.6 (Final)</para></listitem>
-                <listitem><para>Debian GNU/Linux 6.0 (Squeeze)</para></listitem>
+                <listitem><para>CentOS release 5.7 (Final)</para></listitem>
+                <listitem><para>CentOS release 5.8 (Final)</para></listitem>
+                <listitem><para>CentOS release 6.3 (Final)</para></listitem> -->
+                <listitem><para>CentOS release 6.4</para></listitem>
+<!--                <listitem><para>Debian GNU/Linux 6.0 (Squeeze)</para></listitem> -->
+                <listitem><para>Debian GNU/Linux 6.0.7 (Squeeze)</para></listitem>
                 <listitem><para>Debian GNU/Linux 7.0 (Wheezy)</para></listitem>
+                <listitem><para>Debian GNU/Linux 7.1 (Wheezy)</para></listitem>
+<!--                <listitem><para>openSUSE 11.4</para></listitem>
+                <listitem><para>openSUSE 12.1</para></listitem> -->
                 <listitem><para>openSUSE 12.2</para></listitem>
                 <listitem><para>openSUSE 12.3</para></listitem>
             </itemizedlist>

documentation/ref-manual/migration.xml

@@ -580,6 +580,509 @@
         </para>
     </section>
 </section>
+
+<section id='moving-to-the-yocto-project-1.5-release'>
+    <title>Moving to the Yocto Project 1.5 Release</title>
+
+    <para>
+        This section provides migration information for moving to the
+        Yocto Project 1.5 Release from the prior release.
+    </para>
+
+    <section id='migration-1.5-host-dependency-changes'>
+        <title>Host Dependency Changes</title>
+
+        <para>
+            The OpenEmbedded build system now has some additional requirements
+            on the host system:
+            <itemizedlist>
+                <listitem><para>Python 2.7.3+</para></listitem>
+                <listitem><para>Tar 1.24+</para></listitem>
+                <listitem><para>Git 1.7.5+</para></listitem>
+                <listitem><para>Patched version of Make if you are using
+                    3.82.
+                    Most distributions that provide Make 3.82 use the patched
+                    version.</para></listitem>
+            </itemizedlist>
+            If the Linux distribution you are using on your build host
+            does not provide packages for these, you can install and use
+            the Buildtools tarball, which provides an SDK-like environment
+            containing them.
+        </para>
+
+        <para>
+            For more information on this requirement, see the
+            "<link linkend='required-git-tar-and-python-versions'>Required Git, tar, and Python Versions</link>"
+            section.
+        </para>
+    </section>
+
+    <section id='migration-1.5-atom-pc-bsp'>
+        <title><filename>atom-pc</filename> Board Support Package (BSP)</title>
+
+        <para>
+            The <filename>atom-pc</filename> hardware reference BSP has been
+            replaced by a <filename>genericx86</filename> BSP.
+            This BSP is not necessarily guaranteed to work on all x86
+            hardware, but it will run on a wider range of systems than the
+            <filename>atom-pc</filename> did.
+            <note>
+                Additionally, a <filename>genericx86-64</filename> BSP has been
+                added for 64-bit systems.
+            </note>
+        </para>
+    </section>
+
+    <section id='migration-1.5-bitbake'>
+        <title>BitBake</title>
+
+        <para>
+            The following changes have been made that relate to BitBake:
+            <itemizedlist>
+                <listitem><para>
+                    BitBake now supports a <filename>_remove</filename>
+                    operator.
+                    The addition of this operator means you will have to
+                    rename any items in recipe space (functions, variables)
+                    whose names currently contain
+                    <filename>_remove_</filename> or end with
+                    <filename>_remove</filename> to avoid unexpected behavior.
+                    </para></listitem>
+                <listitem><para>
+                    BitBake's global method pool has been removed.
+                    This method is not particularly useful and led to clashes
+                    between recipes containing functions that had the
+                    same name.</para></listitem>
+                <listitem><para>
+                    The "none" server backend has been removed.
+                    The "process" server backend has been serving well as the
+                    default for a long time now.</para></listitem>
+                <listitem><para>
+                    The <filename>bitbake-runtask</filename> script has been
+                    removed.</para></listitem>
+                <listitem><para>
+                    <filename>${</filename><link linkend='var-P'><filename>P</filename></link><filename>}</filename>
+                    and
+                    <filename>${</filename><link linkend='var-PF'><filename>PF</filename></link><filename>}</filename>
+                    are no longer added to
+                    <link linkend='var-PROVIDES'><filename>PROVIDES</filename></link>
+                    by default in <filename>bitbake.conf</filename>.
+                    These version-specific <filename>PROVIDES</filename>
+                    items were seldom used.
+                    Attempting to use them could result in two versions being
+                    built simultaneously rather than just one version due to
+                    the way BitBake resolves dependencies.</para></listitem>
+            </itemizedlist>
+        </para>
+    </section>
+
+    <section id='migration-1.5-qa-warnings'>
+        <title>QA Warnings</title>
+
+        <para>
+            The following changes have been made to the package QA checks:
+            <itemizedlist>
+                <listitem><para>
+                    If you have customized
+                    <link linkend='var-ERROR_QA'><filename>ERROR_QA</filename></link>
+                    or <link linkend='var-WARN_QA'><filename>WARN_QA</filename></link>
+                    values in your configuration, check that they contain all of
+                    the issues that you wish to be reported.
+                    Previous Yocto Project versions contained a bug that meant
+                    that any item not mentioned in <filename>ERROR_QA</filename>
+                    or <filename>WARN_QA</filename> would be treated as a
+                    warning.
+                    Consequently, several important items were not already in
+                    the default value of <filename>WARN_QA</filename>.
+                    All of the possible QA checks are now documented in the
+                    "<link linkend='ref-classes-insane'><filename>insane.bbclass</filename></link>"
+                    section.</para></listitem>
+                <listitem><para>
+                    An additional QA check has been added to check if
+                    <filename>/usr/share/info/dir</filename> is being installed.
+                    Your recipe should delete this file within
+                    <filename>do_install</filename> if "make install" is
+                    installing it.</para></listitem>
+                <listitem><para>
+                    If you are using the buildhistory class, the check for the
+                    package version going backwards is now controlled using a
+                    standard QA check.
+                    Thus, if you have customized your
+                    <filename>ERROR_QA</filename> or
+                    <filename>WARN_QA</filename> values and still wish to have
+                    this check performed, you should add
+                    "version-going-backwards" to your value for one or the
+                    other variables depending on how you wish it to be handled.
+                    See the documented QA checks in the
+                    "<link linkend='ref-classes-insane'><filename>insane.bbclass</filename></link>"
+                    section.
+                    </para></listitem>
+            </itemizedlist>
+        </para>
+    </section>
+
+    <section id='migration-1.5-directory-layout-changes'>
+        <title>Directory Layout Changes</title>
+
+        <para>
+            The following directory changes exist:
+            <itemizedlist>
+                <listitem><para>
+                    Output SDK installer files are now named to include the
+                    image name and tuning architecture through the
+                    <link linkend='var-SDK_NAME'><filename>SDK_NAME</filename></link>
+                    variable.</para></listitem>
+                <listitem><para>
+                    Images and related files are now installed into a directory
+                    that is specific to the machine, instead of a parent
+                    directory containing output files for multiple machines.
+                    The
+                    <link linkend='var-DEPLOY_DIR_IMAGE'><filename>DEPLOY_DIR_IMAGE</filename></link>
+                    variable continues to point to the directory containing
+                    images for the current
+                    <link linkend='var-MACHINE'><filename>MACHINE</filename></link>
+                    and should be used anywhere there is a need to refer to
+                    this directory.
+                    The <filename>runqemu</filename> script now uses this
+                    variable to find images and kernel binaries and will use
+                    BitBake to determine the directory.
+                    Alternatively, you can set the
+                    <filename>DEPLOY_DIR_IMAGE</filename> variable in the
+                    external environment.</para></listitem>
+                <listitem><para>
+                    When buildhistory is enabled, its output is now written
+                    under the
+                    <ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>
+                    rather than
+                    <link linkend='var-TMPDIR'><filename>TMPDIR</filename></link>.
+                    Doing so makes it a easier to delete
+                    <filename>TMPDIR</filename> and preserve the build history.
+                    Additionally, data for produced SDKs is now split by
+                    <link linkend='var-IMAGE_NAME'><filename>IMAGE_NAME</filename></link>.
+                    </para></listitem>
+                <listitem><para>
+                    The <filename>pkgdata</filename> directory produced as
+                    part of the packaging process has been collapsed into a
+                    single machine-specific directory.
+                    This directory is located under
+                    <filename>sysroots</filename> and uses a machine-specific
+                    name (i.e.
+                    <filename>tmp/sysroots/&lt;machine&gt;/pkgdata</filename>).
+                    </para></listitem>
+            </itemizedlist>
+        </para>
+    </section>
+
+    <section id='migration-1.5-shortened-git-srcrev-values'>
+        <title>Shortened Git <filename>SRCREV</filename> Values</title>
+
+        <para>
+            BitBake will now shorten revisions from Git repositories from the
+            normal 40 characters down to 10 characters within
+            <link linkend='var-SRCPV'><filename>SRCPV</filename></link>
+            for improved usability in path and file names.
+            This change should be safe within contexts where these revisions
+            are used because the chances of spatially close collisions
+            is very low.
+            Distant collisions are not a major issue in the way
+            the values are used.
+        </para>
+    </section>
+
+    <section id='migration-1.5-image-features'>
+        <title><filename>IMAGE_FEATURES</filename></title>
+
+        <para>
+            The following changes have been made that relate to
+            <link linkend='var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></link>:
+            <itemizedlist>
+                <listitem><para>
+                    The value of
+                    <link linkend='var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></link>
+                    is now validated to ensure invalid feature items are not
+                    added.
+                    Some users mistakenly add package names to this variable
+                    instead of using
+                    <link linkend='var-IMAGE_INSTALL'><filename>IMAGE_INSTALL</filename></link>
+                    in order to have the package added to the image, which does
+                    not work.
+                    This change is intended to catch those kinds of situations.
+                    Valid <filename>IMAGE_FEATURES</filename> are drawn from
+                    <link linkend='var-PACKAGE_GROUP'><filename>PACKAGE_GROUP</filename></link>
+                    definitions,
+                    <link linkend='var-COMPLEMENTARY_GLOB'><filename>COMPLEMENTARY_GLOB</filename></link>
+                    and a new "validitems" varflag on
+                    <filename>IMAGE_FEATURES</filename>.
+                    The "validitems" varflag change allows additional features
+                    to be added if they are not provided using the previous
+                    two mechanisms.
+                    </para></listitem>
+                <listitem><para>
+                    The previously deprecated "apps-console-core"
+                    <filename>IMAGE_FEATURES</filename> item is no longer
+                    supported.
+                    Add "splash" to <filename>IMAGE_FEATURES</filename> if you
+                    wish to have the splash screen enabled, since this is
+                    all that apps-console-core was doing.</para></listitem>
+            </itemizedlist>
+        </para>
+    </section>
+
+    <section id='migration-1.5-run'>
+        <title><filename>/run</filename></title>
+
+        <para>
+            The <filename>/run</filename> directory from the Filesystem
+            Hierarchy Standard 3.0 has been introduced.
+            You can find some of the implications for this change
+            <ulink url='http://cgit.openembedded.org/openembedded-core/commit/?id=0e326280a15b0f2c4ef2ef4ec441f63f55b75873'>here</ulink>.
+            The change also means that recipes that install files to
+            <filename>/var/run</filename> must be changed.
+            You can find a guide on how to make these changes
+            <ulink url='http://permalink.gmane.org/gmane.comp.handhelds.openembedded/58530'>here</ulink>.
+        </para>
+    </section>
+
+    <section id='migration-1.5-removal-of-package-manager-database-within-image-recipes'>
+        <title>Removal of Package Manager Database Within Image Recipes</title>
+
+        <para>
+            The image <filename>core-image-minimal</filename> no longer adds
+            <filename>remove_packaging_data_files</filename> to
+            <link linkend='var-ROOTFS_POSTPROCESS_COMMAND'><filename>ROOTFS_POSTPROCESS_COMMAND</filename></link>.
+            This addition is now handled automatically when "package-management"
+            is not in
+            <link linkend='var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></link>.
+            If you have custom image recipes that make this addition,
+            you should remove the lines, as they are not needed and might
+            interfere with correct operation of postinstall scripts.
+        </para>
+    </section>
+
+    <section id='migration-1.5-images-now-rebuild-only-on-changes-instead-of-every-time'>
+        <title>Images Now Rebuild Only on Changes Instead of Every Time</title>
+
+        <para>
+            The <filename>do_rootfs</filename> and other related image
+            construction tasks are no longer marked as "nostamp".
+            Consequently, they will only be re-executed when their inputs have
+            changed.
+            Previous versions of the OpenEmbedded build system always rebuilt
+            the image when requested rather when necessary.
+        </para>
+    </section>
+
+    <section id='migration-1.5-task-recipes'>
+        <title>Task Recipes</title>
+
+        <para>
+            The previously deprecated <filename>task.bbclass</filename> has
+            now been dropped.
+            For recipes that previously inherited from this task, you should
+            rename them from <filename>task-*</filename> to
+            <filename>packagegroup-*</filename> and inherit packagegroup
+            instead.
+        </para>
+
+        <para>
+            For more information, see the
+            "<link linkend='ref-classes-packagegroup'>Package Groups - <filename>packagegroup.bbclass</filename></link>"
+            section.
+        </para>
+    </section>
+
+    <section id='migration-1.5-busybox'>
+        <title>BusyBox</title>
+
+        <para>
+            By default, we now split BusyBox into two binaries:
+            one that is suid root for those components that need it, and
+            another for the rest of the components.
+            Splitting BusyBox allows for optimization that eliminates the
+            <filename>tinylogin</filename> recipe as recommended by upstream.
+            You can disable this split by setting
+            <link linkend='var-BUSYBOX_SPLIT_SUID'><filename>BUSYBOX_SPLIT_SUID</filename></link>
+            to "0".
+        </para>
+    </section>
+
+    <section id='migration-1.5-automated-image-testing'>
+        <title>Automated Image Testing</title>
+
+        <para>
+            A new automated image testing framework has been added
+            through the
+            <link linkend='ref-classes-testimage'><filename>testimage*.bbclass</filename></link>
+            class.
+            This framework replaces the older
+            <filename>imagetest-qemu</filename> framework.
+        </para>
+
+        <para>
+            You can learn more about performing automated image tests in the
+            "<ulink url='&YOCTO_DOCS_DEV_URL;#performing-automated-runtime-testing'>Performing Automated Runtime Testing</ulink>"
+            section.
+        </para>
+    </section>
+
+    <section id='migration-1.5-build-history'>
+        <title>Build History</title>
+
+        <para>
+            Following are changes to Build History:
+            <itemizedlist>
+                <listitem><para>
+                     Installed package sizes:
+                     <filename>installed-package-sizes.txt</filename> for an
+                     image now records the size of the files installed by each
+                     package instead of the size of each compressed package
+                     archive file.</para></listitem>
+                <listitem><para>
+                    The dependency graphs (<filename>depends*.dot</filename>)
+                    now use the actual package names instead of replacing
+                    dashes, dots and plus signs with underscores.
+                    </para></listitem>
+                <listitem><para>
+                    The <filename>buildhistory-diff</filename> and
+                    <filename>buildhistory-collect-srcrevs</filename>
+                    utilities have improved command-line handling.
+                    Use the <filename>&dash;&dash;help</filename> option for
+                    each utility for more information on the new syntax.
+                    </para></listitem>
+            </itemizedlist>
+            For more information on Build History, see the
+            "<link linkend='maintaining-build-output-quality'>Maintaining Build Output Quality</link>"
+            section.
+        </para>
+    </section>
+
+    <section id='migration-1.5-udev'>
+        <title><filename>udev</filename></title>
+
+        <para>
+            Following are changes to <filename>udev</filename>:
+            <itemizedlist>
+                <listitem><para>
+                    <filename>udev</filename> no longer brings in
+                    <filename>udev-extraconf</filename> automatically
+                    through
+                    <link linkend='var-RRECOMMENDS'><filename>RRECOMMENDS</filename></link>,
+                    since this was originally intended to be optional.
+                    If you need the extra rules, then add
+                    <filename>udev-extraconf</filename> to your image.
+                    </para></listitem>
+                <listitem><para>
+                    <filename>udev</filename> no longer brings in
+                    <filename>pciutils-ids</filename> or
+                    <filename>usbutils-ids</filename> through
+                    <filename>RRECOMMENDS</filename>.
+                    These are not needed by <filename>udev</filename> itself
+                    and removing them saves around 350KB.
+                    </para></listitem>
+            </itemizedlist>
+        </para>
+    </section>
+
+    <section id='removed-renamed-recipes'>
+        <title>Removed and Renamed Recipes</title>
+
+        <itemizedlist>
+            <listitem><para>
+                The <filename>linux-yocto</filename> 3.2 kernel has been
+                removed.</para></listitem>
+            <listitem><para>
+                <filename>libtool-nativesdk</filename> has been renamed to
+                <filename>nativesdk-libtool</filename>.</para></listitem>
+            <listitem><para>
+                <filename>tinylogin</filename> has been removed.
+                It has been replaced by a suid portion of Busybox.
+                See the
+                "<link linkend='migration-1.5-busybox'>BusyBox</link>" section
+                for more information.</para></listitem>
+            <listitem><para>
+                <filename>external-python-tarball</filename> has been renamed
+                to <filename>buildtools-tarball</filename>.
+                </para></listitem>
+            <listitem><para>
+                <filename>web-webkit</filename> has been removed.
+                It has been functionally replaced by
+                <filename>midori</filename>.</para></listitem>
+            <listitem><para>
+                <filename>imake</filename> has been removed.
+                It is no longer needed by any other recipe.
+                </para></listitem>
+            <listitem><para>
+                <filename>transfig-native</filename> has been removed.
+                It is no longer needed by any other recipe.
+                </para></listitem>
+            <listitem><para>
+                <filename>anjuta-remote-run</filename> has been removed.
+                Anjuta IDE integration has not been officially supported for
+                several releases.</para></listitem>
+        </itemizedlist>
+    </section>
+
+    <section id='migration-1.5-other-changes'>
+        <title>Other Changes</title>
+
+        <para>
+            Following is a list of short entries describing other changes:
+            <itemizedlist>
+                <listitem><para>
+                    <filename>run-postinsts</filename>: Make this generic.
+                    </para></listitem>
+                <listitem><para>
+                    <filename>base-files</filename>: Remove the unnecessary
+                    <filename>/media/xxx</filename> directories.
+                    </para></listitem>
+                <listitem><para>
+                    <filename>alsa-state</filename>: Provide an empty
+                    <filename>asound.conf</filename> by default.
+                    </para></listitem>
+                <listitem><para>
+                    <filename>classes/image</filename>: Ensure
+                    <link linkend='var-BAD_RECOMMENDATIONS'><filename>BAD_RECOMMENDATIONS</filename></link>
+                    supports pre-renamed package names.</para></listitem>
+                <listitem><para>
+                    <filename>classes/rootfs_rpm</filename>: Implement
+                    <link linkend='var-BAD_RECOMMENDATIONS'><filename>BAD_RECOMMENDATIONS</filename></link>
+                    for RPM.</para></listitem>
+                <listitem><para>
+                    <filename>systemd</filename>: Remove
+                    <filename>systemd_unitdir</filename> if
+                    <filename>systemd</filename> is not in
+                    <link linkend='var-DISTRO_FEATURES'><filename>DISTRO_FEATURES</filename></link>.
+                    </para></listitem>
+                <listitem><para>
+                    <filename>systemd</filename>: Remove
+                    <filename>init.d</filename> dir if
+                    <filename>systemd</filename> unit file is present and
+                    <filename>sysvinit</filename> is not a distro feature.
+                    </para></listitem>
+                <listitem><para>
+                    <filename>libpam</filename>: Deny all services for the
+                    <filename>OTHER</filename> entries.
+                    </para></listitem>
+                <listitem><para>
+                    <filename>image.bbclass</filename>: Move
+                    <filename>runtime_mapping_rename</filename> to avoid
+                    conflict with <filename>multilib</filename>.
+                    See
+                    <ulink url='https://bugzilla.yoctoproject.org/show_bug.cgi?id=4993'><filename>YOCTO #4993</filename></ulink>
+                    in Bugzilla for more information.
+                    </para></listitem>
+                <listitem><para>
+                    <filename>linux-dtb</filename>: Use kernel build system
+                    to generate the <filename>dtb</filename> files.
+                    </para></listitem>
+                <listitem><para>
+                    <filename>kern-tools</filename>: Switch from guilt to
+                    new <filename>kgit-s2q</filename> tool.
+                    </para></listitem>
+            </itemizedlist>
+        </para>
+    </section>
+</section>
 </chapter>
 <!--
 vim: expandtab tw=80 ts=4