# BitBake User Manual

## Introduction

fundamentally, BitBake is a generic task execution engine that allows shell and Python tasks to be run efficiently and in parallel while working within complex inter-task dependency constraints. One of BitBake's main users, OpenEmbedded, takes this core and builds embedded Linux software stacks using a task-oriented approach.

Conceptually, BitBake is similar to GNU Make in some regards but has significant differences:

- BitBake executes tasks according to provided metadata that builds up the tasks. Metadata is stored in recipe (`.bb`), configuration (`.conf`), and class (`.bbclass`) files and provides BitBake with instructions on what tasks to run and the dependencies between those tasks.

- BitBake includes a fetcher library for obtaining source code from various places such as source control systems or websites.

- The instructions for each unit to be built (e.g. a piece of software) are known as recipe files and contain all the information about the unit (dependencies, source file locations, checksums, description and so on).

- BitBake includes a client/server abstraction and can be used from a command line or used as a service over XMLRPC and has several different user interfaces.

## History and Goals

BitBake was originally a part of the OpenEmbedded project. It was inspired by the Portage package management system used by the Gentoo Linux distribution. On December 7, 2004, OpenEmbedded project team member, Chris Larson split the project into two distinct pieces:

- BitBake, a generic task executor

- OpenEmbedded, a metadata set utilized by BitBake.

Today, BitBake is the primary basis of the OpenEmbedded project, which is being used to build and maintain a number of projects and embedded Linux distributions such as the Angstrom Distribution and the Yocto Project.

Prior to BitBake, no other build tool adequately met the needs of an aspiring embedded Linux distribution. All of the build systems used by traditional desktop Linux distributions lacked important functionality, and none of the ad-hoc buildroot systems, prevalent in the embedded space, were scalable or maintainable.

Some important original goals for BitBake were:

- Handle cross-compilation.

- Handle inter-package dependencies (build time on target architecture, build time on native architecture, and runtime).

- Support running any number of tasks within a given package, including, but not limited to, fetching upstream sources, unpacking them, patching them, configuring them, etc.

- Must be Linux distribution agnostic (both build and target).

- Must be architecture agnostic

- Must support multiple build and target operating systems (including Cygwin, the BSDs, etc).

- Must be able to be self contained, rather than tightly integrated into the build machine's root filesystem.

- There must be a way to handle conditional metadata (on target architecture, operating system, distribution, machine).

- It must be easy for the person using the tools to supply their own local metadata and packages to operate against.

- Must make it easy to collaborate between multiple projects using BitBake for their builds.

- Should provide an inheritance mechanism to share common metadata between many packages.

Over time it has become apparent that some further requirements were necessary:

- Handle variants of a base recipe (native, sdk, multilib).

- Able to split metadata into layers and allow layers to override each other.

- Allow representation of a given set of input variables to a task as a checksum. Based on that checksum, allow acceleration of builds with prebuilt components.

BitBake satisfies all the original requirements and many more with extensions being made to the basic functionality to reflect the additional requirements. Flexibility and power have always been the priorities. It is highly extensible, supporting embedded Python code and execution of any arbitrary tasks.

## Concepts

BitBake is a program written in the Python language. At the highest level, BitBake interprets metadata, decides what tasks are required to run, and executes those tasks. Similar to GNU Make, BitBake controls how software is built. GNU Make does this using "makefiles". BitBake uses "recipes". BitBake extends the capabilities of a simple tool like GNU make by allowing for much more complex tasks to be completed, such as assembling entire embedded Linux distributions. Several concepts must be understood to be able to leverage the power of the tool.

### Recipes

A BitBake Recipe, denoted by the file extension `.bb` is the most basic metadata file. These recipe files tell BitBake the following:

- Descriptive information about the package

- The version of the recipe

- When dependencies exist

- Where the source code resides

- Whether the source code requires any patches

- How to compile the source code

- Where on the target machine to install the package being compiled

Within the context of BitBake, or any project utilizing BitBake as it's build system, files with the `.bb` extension are referred to as recipes. The term "package" is also commonly used to describe recipes. However, since the same word is used to describe packaged output from a project, it is best to maintain a single descriptive term, "recipes".

### Configuration Files

Configuration files, denoted by the `.conf` extension define various configuration variables that govern the project build process. These files fall into several areas that define machine configuration options, distribution configuration options, compiler tuning options, general common configuration options and user configuration options. The main configuration file is the sample `bitbake.conf` file, located within the bitbake source tree `/conf` directory.

### Classes

Class files, denoted by the `.bbclass` extension contain information that is useful to share between metadata files. The BitBake source tree comes with one class metadata file currently, called `base.bbclass` and it is found in the `/classes` directory. The `base.bbclass` is special in that any new classes that a developer adds to a project is required to inherit it automatically. This class contains definitions for standard basic tasks such as fetching, unpacking, configuring (empty by default), compiling (runs any Makefile present), installing (empty by default) and packaging (empty by default). These classes are often overridden or extended by other classes added during the project development process.

## Obtaining BitBake

There are several ways to obtain BitBake. These include installing using your Linux distribution's package management system (not recommended), downloading a snapshot from the BitBake source code repository, or using Git to clone the BitBake source code repository. The recommended method for daily BitBake use is to download a stable release from the BitBake source code repository. Using your distribution's version as provided in the package management system is generally not recommended as in most cases, such as with the Ubuntu and Fedora distributions, the version provided is several releases behind the repository snapshot version and is missing important bug fixes and enhancements. Similarly, daily use of the latest clone of the Git repository is not recommended as it can be unstable. However, the Git repository clone will provide the User with the absolute latest version of BitBake.

## Downloading a Snapshot from the BitBake Source Tree

The recommended method for obtaining and using BitBake on a daily basis is to download the most recent stable snapshot from the Git source code repository as follows:

         $ wget http://git.openembedded.org/bitbake/snapshot/bitbake-1.17.0.tar.gz
         $ tar zxpvf bitbake-1.17.0.tar.gz

After extraction of the tarball using the tar utility, you will have a directory entitled `bitbake-1.17.0`.

## Cloning the BitBake Git Repository

To obtain the latest BitBake source code from the BitBake Git repository:

         $ git clone git://git.openembedded.org/bitbake

This will clone the BitBake Git repository into a directory called `bitbake`. Alternatively, you can designate a directory after the git clone command if you'd prefer to call the new directory something other than `bitbake`. For example:

         $ git clone git://git.openembedded.org/bitbake bbdev

This would clone the Git repository into a local directory called `bbdev`. Please note that although this method of obtaining the source code will provide the absolute latest version, it is under active development and may not be as stable as a released snapshot.

## Summary

At this point you should have a general idea of the concepts that BitBake was built on and how the source code is organized. You should have a working version of BitBake installed and understand how to setup your environment.