diff --git a/documentation/dev-manual/dev-manual-newbie.xml b/documentation/dev-manual/dev-manual-newbie.xml
index 5ca2c25b52..68a2f4a211 100644
--- a/documentation/dev-manual/dev-manual-newbie.xml
+++ b/documentation/dev-manual/dev-manual-newbie.xml
@@ -5,46 +5,527 @@
Working with Open Source Code
-
- Introduction
+
+ This chapter presents information for users new or unfamiliar with working in an open source environment.
+ Working in this type of environment is different than working in a closed, proprietary environment.
+ The chapter also presents information specific to the Yocto Project environment.
+ It specifically addresses licensing issues, code repositories, the open-source distributed version control
+ system Git, and best practices within Yocto Project.
+
+
+
+ If you are a seasoned open source developer and are familiar with Git, you might just be interested
+ in the Yocto Project specific information in this chapter.
+
+
+
+ Open Source Philosophy
+
- The Yocto Project presents the kernel as a fully patched, history-clean Git
- repository.
- The Git tree represents the selected features, board support,
- and configurations extensively tested by Yocto Project.
- The Yocto Project kernel allows the end user to leverage community
- best practices to seamlessly manage the development, build and debug cycles.
+ Open source philosophy is characterized by software development directed by peer production,
+ bartering, and collaboration through a concerned community of developers.
+ Contrast this to the more standard centralized development models used by commercial software
+ companies where a finite set of developers produce a product for sale using a defined set
+ of procedures that ultimately result in an end-product whose architecture and source material
+ are closed to the public.
+
- This manual describes the Yocto Project kernel by providing information
- on its history, organization, benefits, and use.
- The manual consists of two sections:
-
- Concepts - Describes concepts behind the kernel.
- You will understand how the kernel is organized and why it is organized in
- the way it is. You will understand the benefits of the kernel's organization
- and the mechanisms used to work with the kernel and how to apply it in your
- design process.
- Using the Kernel - Describes best practices and "how-to" information
- that lets you put the kernel to practical use. Some examples are "How to Build a
- Project Specific Tree", "How to Examine Changes in a Branch", and "Saving Kernel
- Modifications."
-
+ Open source projects conceptually have differing concurrent agendas, approaches, and production.
+ These facets of the development process can come from anyone in the public (community) that has a
+ stake in the software project.
+ The open source environment contains new copyright, licensing, domain, and consumer issues
+ that differ from the more traditional development environment.
+ In an open source environment the end-product, source material, and documentation are
+ all available to the public at no cost.
+
- For more information on the kernel, see the following links:
-
-
-
-
-
-
- You can find more information on Yocto Project by visiting the website at
- .
-
+ A benchmark example of an open source project is the Linux Kernel, which was initially conceived
+ and created by Finnish computer science student Linus Torvalds in 1991.
+ Conversely, a good example of a non-open source project is the Windows family of operating
+ systems developed by Microsoft Corporation.
+
+
+
+ Wikipedia has a good historical description of the Open Source Philosophy
+ here.
+
+
+
+ You can also find helpful information on how to participate in the Linux Community
+ here.
+
+ Yocto Project Repositories
+
+
+ The Yocto Project team maintains complete source repositories that allow further development
+ of Yocto Project, its tools, Board Support Packages, and so forth.
+ As a developer who uses Yocto Project, however, you need only to be able to access your
+ kernel or application source code and any layers (modifications) on which you might be working.
+
+
+
+ For any supported release of Yocto Project you can go to the Yocto Project website’s
+ download page and get a
+ .bz2 tarball of the release.
+ You can also go to this site to download any supported BSP tarballs.
+
+
+
+ After obtaining the code, you can unpack the tarballs and have a working Git repository
+ from which you can develop.
+ Or, you can take steps to create local repositories of Yocto Project source code and metadata on
+ your development system.
+ See the information
+ here
+ for information on how to set up these local Git repositories.
+
+
+
+ Should you be interested in locations of complete Yocto Project development code, there are
+ two areas where this code is maintained:
+
+ Source Repositories:
+ This area contains IDE Plugins, Matchbox, Poky, Poky Support, Tools, Yocto Linux Kernel, and Yocto
+ Metadata Layers.
+ Index of /downloads:
+ This area contains an index of the Eclipse-plugin, miscellaneous support, poky, pseudo, and
+ all released versions of Yocto Project.
+ [WRITER NOTE: link will be http://downloads.yoctoproject.org.]
+
+
+
+
+
+ Licensing
+
+
+ Because open source projects are open to the public they have different licensing structures in place.
+ License evolution for both Open Source and Free Software has an interesting history.
+ If you are interested in the history you can find basic information here:
+
+ Open source license history
+
+ Free software license
+ history
+
+
+
+
+ In general, Yocto Project is broadly licensed under the Massachusetts Institute of Technology
+ (MIT) License.
+ MIT licensing permits the reuse of software within proprietary software as long as the
+ license is distributed with that software.
+ MIT is also compatible with the GNU General Public License (GPL).
+ Patches to the Yocto Project follow the up-stream licensing scheme.
+
+
+
+ You can find information on the MIT License here.
+ You can find information on the GNU GPL here.
+
+
+
+ When you build an image using Yocto Project the build process uses a known list of licenses to
+ ensure compliance.
+ Once the build completes the list of all licenses found and used during the build are
+ kept in the resulting build directory at
+ <build_directory>/tmp/deploy/images/licenses.
+ If a module requires a license that is not in the base list then the build process
+ generates a warning during the build.
+ It is up to the developer to resolve potential licensing issues.
+
+
+
+ The base list of licenses used by the build process is a combination of the Software Package
+ Data Exchange (SPDX) list and the Open Source Initiative (OSI) projects.
+ SPDX Group is a working group of the Linux Foundation
+ that maintains a specification
+ for a standard format for communicating the components, licenses, and copyrights
+ associated with a software package.
+ OSI is a corporation dedicated to the Open Source
+ Definition and the effort for reviewing
+ and approving licenses that are OSD-conformant.
+
+
+
+ You can find a list of the combined SPDX and OSI licenses that the Yocto Project uses
+ here.
+ The wiki page discusses the license infrastructure used by the Yocto Project.
+
+
+
+
+ Git
+
+
+ The Yocto Project uses Git, which is a free, open source distributed version control.
+ Git supports distributed development, non-linear development, can handle large projects,
+ cryptographic authentication of history, and toolkit design.
+ It is best that you know how to work with Git if you are going to use Yocto Project for development.
+
+
+
+ Git has an extensive set of commands that lets you manage and collaborate changes over the life
+ of a project.
+ Conveniently though, you can manage with a small set of basic operations and workflows
+ once you understand the basic philosophy behind Git.
+ You do not have to be an expert in Git to be functional.
+ A good place to look for instruction on a minimal set of Git commands is
+ here.
+ If you need to download Git you can do so
+ here.
+
+
+
+ Git works by using branching techniques that track content change (not files)
+ within a project (e.g. a new feature or updated documentation).
+ Creating a tree-like structure based on project divergence allows for excellent historical
+ information over the life of a project.
+ This methodology also allows for an environment in which you can do lots of
+ experimentation on your project as you develop changes or new features.
+ For example, you can create a “branch”, experiment with some feature, and then
+ if you like the feature you incorporate the branch into the tree.
+ If you don’t, you cut the branch off by deleting it.
+
+
+
+ If you don’t know much about Git it is strongly suggested that you educate
+ yourself by visiting the links previously mentioned.
+
+
+
+ The following list briefly describes some basic Git operations as a way to get started.
+ As with any set of commands, this list (in most cases) simply shows the base command and
+ omits the many arguments they support.
+ See the Git documentation for complete descriptions and strategies on how to use these commands:
+
+ git init – Initializes an empty Git repository.
+ You cannot use Git commands unless you have a .git repository.
+ git clone – Creates a clone of a repository.
+ During collaboration this command allows you to create a local repository that is on
+ equal footing with a fellow developer’s repository.
+ git add – Adds updated file contents to the index that
+ Git uses to track changes.
+ All files that have changed must be added before they can be committed.
+ git commit – Creates a “commit” that documents the changes you made.
+ Commits are used for historical purposes, for determining if a maintainer of a project
+ will allow the change, and for ultimately pushing the change from your local Git repository
+ into the project’s upstream (or master) repository.
+ git status – Reports any modified files that possibly need
+ added and committed.
+ git checkout <branch-name> - Changes
+ your working branch. This command is analogous to “cd”.
+ git checkout –b <working-branch> - Creates
+ a working branch on your local machine where you can isolate work.
+ It is a good idea to use local branches when adding specific features or changes.
+ This way if you don’t like what you have done you can easily get rid of the work.
+ git branch – Reports existing branches and tells you which
+ branch in which you are currently working.
+ git pull – Retrieves information from an upstream Git
+ repository and places it in your local Git repository.
+ You use this command to make sure you are synchronized with the upstream repository
+ from which the project’s maintainer uses to pull changes into the master repository.
+ git push – Sends all your local changes you have committed
+ to an upstream Git repository.
+ The maintainer of the project draws from these repositories when adding your changes to the
+ project’s master repository.
+ git merge – Combines or adds changes from one local branch of
+ your repository with another branch.
+ When you create a local Git repository the default branch is named “master”.
+ A typical workflow is to create a temporary branch for isolated work, make and commit your
+ changes, switch to the master branch, merge the changes in the temporary branch with the
+ master branch, and then delete the temporary branch
+ git cherry-pick – Choose and apply specific commits from
+ one branch into another branch.
+ There are times when you might not be able to merge all the changes in one branch with
+ another but need to pick out certain ones.
+ gitk – Provides a GUI view of the branches and changes in
+ your local Git repository.
+ This command is a good way to see where things have diverged in your local repository.
+ git log – Reports a history of your changes to the
+ repository.
+
+
+
+
+
+ Workflows
+
+
+ This section provides some overview on workflows using Git.
+ In particular, the information covers basic practices that describe roles and actions in a
+ collaborative development environment.
+ Again, if you are familiar with this type of development environment you might want to just skip the section.
+
+
+
+ Following are the definitions for some terms used in the Yocto Project.
+ [WRITER NOTE: I need to move this list of definitions somewhere useful.]
+
+ Image - An image is a collection of recipes created with
+ Bitbake (baked). Images run on specific hardware and use cases.
+ Recipe - A set of instructions for building packages.
+ A recipe describes where you get the source and which patches to apply.
+ Recipes describe dependencies for libraries or for other recipes and they
+ also contain configuration and compilation options.
+ Recipes also let you ‘install’ customizations.
+ Recipes contain the logical unit of execution, the software/images to build and
+ use the .bb file extension.
+ BitBake - The task executor and scheduler used by Yocto Project
+ to build images.
+ For more information on BitBake, see the
+ BitBake documentation.
+ Package - A collection of ‘baked’ recipes.
+ You ‘bake’ something by running it through Bitbake.
+ Layer - A logical collection of recipes representing the core,
+ a BSP, or an application stack.
+ Metadata - Information for a build that is generally
+ architecture-independent.
+ This information includes Task definitions in recipes, classes, and configuration
+ information.
+ Configuration File: Configuration information in the
+ .conf files provides global definition of variables.
+ The build/conf/local.conf configuration file defines local user-defined variables.
+ The distro/poky.conf configuration file defines Yocto ‘distro’ configuration
+ variables.
+ The machine/beagleboard.conf configuration file defines machine-specific variables.
+ Configuration files end with a .conf filename extension.
+ Classes - Files that encapsulate and inherit logic.
+ Class files end with the .bbclass filename extension.
+ Tasks - Arbitrary groups of software used to contain Recipes.
+ You simply use Tasks to hold recipes that when build usually accomplished a single task.
+ For example, a task could contain the recipes for a company’s proprietary or value-add software.
+ Or the task could contain the recipes that enable graphics.
+ A task is really just another recipe.
+ Because task files are recipes, they end with the .bb filename
+ extension.
+ Common OE-Core - A core set of metadata originating
+ with OpenEmbedded (OE) that is shared between OE and the Yocto Project.
+
+
+
+
+ A master Git repository exists that contains the project.
+ Usually a key individual is responsible for this repository.
+ It is the “upstream” repository where the final builds of the project occur.
+ The maintainer is responsible for allowing changes in from other developers and for
+ organizing the branch structure of the repository to reflect release strategies and so forth.
+
+
+
+ The maintainer of the project also owns a contribution repository usually known as a “contrib” area.
+ The contrib area temporarily holds changes to the project that have been submitted or committed
+ by the development team.
+ The maintainer determines if the changes are qualified to be moved into the master repository.
+
+
+
+ Developers create and maintain cloned repositories of the upstream master repository.
+ These repositories are local to their development platforms and are used to develop changes.
+ When a developer is satisfied with a particular feature or change they “push” the changes
+ up to the contrib repository.
+ Developers are responsible for keeping their local repository up-to-date with the master
+ repository.
+ They are also responsible for straightening out any conflicts that might arise within files
+ that are being worked on simultaneously by more than one person.
+ All this work is done locally on the developer’s machine before anything is pushed upstream
+ and examined at the maintainer’s level.
+
+
+
+ A somewhat formal method exists by which developers commit changes and push them into the
+ contrib area and subsequently request that the maintainer include them into the master repository.
+ This process is called “submitting a patch” or “submitting a change.”
+
+
+
+ To summarize the environment: we have a single point of entry for changes into the project’s
+ master repository, which is controlled by the project’s maintainer.
+ And, we have a set of developers who independently develop, test, and submit changes
+ upstream for the maintainer to examine.
+ The maintainer then chooses which changes are going to become permanently a part of the project.
+
+
+
+ [WRITER NOTE: Would like a figure here for Git workflow]
+
+
+
+ While each development environment is unique, there are some best practices or methods
+ that help development run smoothly.
+ The following list describes some of these practices.
+ For more detailed information about these strategies see
+ Git Workflows.
+
+ Small Changes - It is best to keep your changes you commit
+ small as compared to bundling many disparate changes into a single commit.
+ This practice not only keeps things manageable but also allows the maintainer
+ to more easily include or refuse changes.
+ Use Branches Liberally - It is very easy to create, use, and
+ delete local branches in your working Git repository.
+ You can name these branches anything you like.
+ It is helpful to give them names associated with the particular feature or change
+ on which you are working.
+ Once you are done with a feature or change you simply discard the branch.
+ Merge Changes - The Git merge command allows you to take the
+ changes from one branch and fold them into another branch.
+ This process is especially helpful when more than a single developer might be working
+ on different parts of the same feature.
+ Merging changes also automatically identifies any collisions or “conflicts”
+ that might happen resulting from the same lines of code be altered by two different
+ developers.
+ Manage Branches - Because branches are easy to use, you should
+ use a system where branches indicate varying levels of code readiness.
+ For example, you can have a “work” branch to develop in, a “test” branch where the code or
+ change is tested, a “stage” branch where changes are ready to be committed, and so forth.
+ As your project develops, you can merge code across the branches to reflect ever-increasing
+ stable states of the development.
+ Use Push and Pull - The push-pull workflow is based on the
+ concept of developers “pushing” local commits upstream to the remote repository, which is
+ usually a contribution repository.
+ It is also based on the developers “pulling” known states of the project down into their
+ local development repositories.
+ This workflow easily allows you to pull changes submitted by other developers from the
+ upstream repository into your work area ensuring that you have the most recent software
+ on which to develop.
+ Patch Workflow - This workflow allows you to notify the
+ maintainer through an email that you have a change (or patch) you would like considered
+ for the master repository.
+ To send this type of change you format the patch and then send the email using the Git commands
+ git format-patch and git send-email.
+ You can find information on how to submit later in this chapter.
+
+
+
+
+
+ Tracking Bugs
+
+
+ The Yocto Project uses Bugzilla to track bugs.
+ This bug-tracking application works well for group development because it tracks bugs and code
+ changes, can be used to communicate changes and problems with developers, can be used to
+ submit and review patches, and can be used to manage quality assurance.
+ You can find a good overview of Bugzilla here.
+
+
+
+ Sometimes it is helpful to submit, investigate, or track a bug against the Yocto Project itself.
+ While normally this is a process relevant only to Yocto Project developers, you can find information
+ for Bugzilla configuration and bug tracking procedures specific to the Yocto Project
+ here.
+
+
+
+ The Yocto Project uses its own version of the Bugzilla application.
+ You can find the home page here.
+ You need to use this implementation of Bugzilla when logging a defect against anything released
+ by the Yocto Project team.
+
+
+
+ Here are some things to remember when dealing with bugs against the Yocto Project:
+
+ The Yocto Project follows a naming bug-naming convention:
+ [YOCTO <number>], where <number> is the
+ assigned defect ID used in Bugzilla.
+ So, for example, a valid way to refer to a defect when creating a commit comment
+ would be [YOCTO 1011].
+ This convention becomes important if you are submitting patches against the Yocto Project
+ code itself (see the next section “How to Submit a Change”).
+ Defects for Yocto Project fall into one of four classifications: Yocto Projects,
+ Infrastructure, Poky, and Yocto Metadata Layers.
+
+
+
+
+
+ How to Submit a Change
+
+
+ During the development process it is necessary to submit your changes to the maintainer
+ of the project.
+ Furthermore, in a collaborative environment it is necessary to have some sort of standard
+ or method through which you submit changes.
+ Otherwise, things would get quite chaotic.
+
+
+
+ Sometimes you might find it necessary to submit a change or patch to the Yocto Project.
+ If so, you must follow certain procedures.
+ In particular, the headers in patches and the commit messages must follow a certain standard.
+ The general process is the same as described earlier in this section.
+ For complete details on how to create proper commit messages and patch headers see
+ [WRITER NOTE: I need the link to Mark's wiki page here that describes the process.]
+
+
+
+ Following are general instructions for both pushing changes upstream and for submitting changes as patches.
+
+
+
+ Pushing a Change Upstream
+
+
+ The basic flow for pushing a change to an upstream contrib repository is as follows:
+
+ Make your changes in your local repository.
+ Stage your commit (or change) by using the git add
+ command.
+ Commit the change by using the git commit
+ command and push it to an upstream contrib repository.
+ Be sure to provide a commit message that follows the project’s commit standards.
+ Notify the maintainer that you have pushed a change.
+
+ You can find detailed information on how to push a change upstream
+
+ here.
+
+
+
+
+ Submitting a Patch
+
+
+ If you have a just a few changes you can commit them and then submit them as an email to the maintainer.
+ Here is the general procedure:
+
+ Make your changes in your local repository.
+ Stage your commit (or change) by using the git add
+ command.
+ Commit the change by using the git commit command.
+ Be sure to provide a commit message that follows the project’s commit standards.
+ Format the commit by using the git-format-patch
+ command.
+ This step produces a numbered series of files in the current directory – one for
+ each commit.
+ Import the files into your mail client by using the
+ git-send-email command.
+ Send the email by hand to the maintainer.
+
+ Be aware that there could be protocols and standards that you need to follow for your particular
+ project.
+ You can find detailed information on the general process
+
+ here.
+
+
+
+
+
+
+
+
+
+
+
+
+