mirror of
https://git.yoctoproject.org/poky
synced 2026-01-29 21:08:42 +01:00
ae06e04cd2
Creation involved removing the overview-manual and replacing it with the getting-started manual. All links to the string "&YOCTO_DOCS_OVERVIEW_URL" had to be replaced with "&YOCTO_DOCS_GS_URL" across the entire YP manual set. I renamed files used to create the manual with prefixes suited for the new manual name, which is "Getting Started With Yocto Project". The style sheet for the new manual needed updating to display the new .PNG image for the title page. The mega-manual file had to be updated to include the files. The mega-manual.sed file had to be updated to include the new manual and not use the overview manual. (From yocto-docs rev: 6c7abf9192390121000f577d6c98f259d290d15d) Signed-off-by: Scott Rifenbark <srifenbark@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
269 lines
14 KiB
HTML
269 lines
14 KiB
HTML
<html>
|
||
<head>
|
||
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
|
||
<title>3.3.3. Shared State</title>
|
||
<link rel="stylesheet" type="text/css" href="../book.css">
|
||
<meta name="generator" content="DocBook XSL Stylesheets V1.76.1">
|
||
<link rel="home" href="index.html" title="Getting Started With Yocto Project">
|
||
<link rel="up" href="shared-state-cache.html" title="3.3. Shared State Cache">
|
||
<link rel="prev" href="overview-checksums.html" title="3.3.2. Checksums (Signatures)">
|
||
<link rel="next" href="tips-and-tricks.html" title="3.3.4. Tips and Tricks">
|
||
</head>
|
||
<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="section" title="3.3.3. Shared State">
|
||
<div class="titlepage"><div><div><h3 class="title">
|
||
<a name="shared-state"></a>3.3.3. Shared State</h3></div></div></div>
|
||
<p>
|
||
Checksums and dependencies, as discussed in the previous
|
||
section, solve half the problem of supporting a shared state.
|
||
The other part of the problem is being able to use checksum
|
||
information during the build and being able to reuse or rebuild
|
||
specific components.
|
||
</p>
|
||
<p>
|
||
The
|
||
<a class="link" href="../ref-manual/ref-classes-sstate.html" target="_self"><code class="filename">sstate</code></a>
|
||
class is a relatively generic implementation of how to
|
||
"capture" a snapshot of a given task.
|
||
The idea is that the build process does not care about the
|
||
source of a task's output.
|
||
Output could be freshly built or it could be downloaded and
|
||
unpacked from somewhere - the build process does not need to
|
||
worry about its origin.
|
||
</p>
|
||
<p>
|
||
There are two types of output, one is just about creating a
|
||
directory in
|
||
<a class="link" href="../ref-manual/var-WORKDIR.html" target="_self"><code class="filename">WORKDIR</code></a>.
|
||
A good example is the output of either
|
||
<a class="link" href="../ref-manual/ref-tasks-install.html" target="_self"><code class="filename">do_install</code></a>
|
||
or
|
||
<a class="link" href="../ref-manual/ref-tasks-package.html" target="_self"><code class="filename">do_package</code></a>.
|
||
The other type of output occurs when a set of data is merged
|
||
into a shared directory tree such as the sysroot.
|
||
</p>
|
||
<p>
|
||
The Yocto Project team has tried to keep the details of the
|
||
implementation hidden in <code class="filename">sstate</code> class.
|
||
From a user's perspective, adding shared state wrapping to a task
|
||
is as simple as this
|
||
<a class="link" href="../ref-manual/ref-tasks-deploy.html" target="_self"><code class="filename">do_deploy</code></a>
|
||
example taken from the
|
||
<a class="link" href="../ref-manual/ref-classes-deploy.html" target="_self"><code class="filename">deploy</code></a>
|
||
class:
|
||
</p>
|
||
<pre class="literallayout">
|
||
DEPLOYDIR = "${WORKDIR}/deploy-${PN}"
|
||
SSTATETASKS += "do_deploy"
|
||
do_deploy[sstate-inputdirs] = "${DEPLOYDIR}"
|
||
do_deploy[sstate-outputdirs] = "${DEPLOY_DIR_IMAGE}"
|
||
|
||
python do_deploy_setscene () {
|
||
sstate_setscene(d)
|
||
}
|
||
addtask do_deploy_setscene
|
||
do_deploy[dirs] = "${DEPLOYDIR} ${B}"
|
||
</pre>
|
||
<p>
|
||
The following list explains the previous example:
|
||
</p>
|
||
<div class="itemizedlist"><ul class="itemizedlist" type="disc">
|
||
<li class="listitem"><p>
|
||
Adding "do_deploy" to <code class="filename">SSTATETASKS</code>
|
||
adds some required sstate-related processing, which is
|
||
implemented in the
|
||
<a class="link" href="../ref-manual/ref-classes-sstate.html" target="_self"><code class="filename">sstate</code></a>
|
||
class, to before and after the
|
||
<a class="link" href="../ref-manual/ref-tasks-deploy.html" target="_self"><code class="filename">do_deploy</code></a>
|
||
task.
|
||
</p></li>
|
||
<li class="listitem"><p>
|
||
The
|
||
<code class="filename">do_deploy[sstate-inputdirs] = "${DEPLOYDIR}"</code>
|
||
declares that <code class="filename">do_deploy</code> places its
|
||
output in <code class="filename">${DEPLOYDIR}</code> when run
|
||
normally (i.e. when not using the sstate cache).
|
||
This output becomes the input to the shared state cache.
|
||
</p></li>
|
||
<li class="listitem">
|
||
<p>
|
||
The
|
||
<code class="filename">do_deploy[sstate-outputdirs] = "${DEPLOY_DIR_IMAGE}"</code>
|
||
line causes the contents of the shared state cache to be
|
||
copied to <code class="filename">${DEPLOY_DIR_IMAGE}</code>.
|
||
</p>
|
||
<div class="note" title="Note" style="margin-left: 0.5in; margin-right: 0.5in;">
|
||
<h3 class="title">Note</h3>
|
||
If <code class="filename">do_deploy</code> is not already in
|
||
the shared state cache or if its input checksum
|
||
(signature) has changed from when the output was
|
||
cached, the task will be run to populate the shared
|
||
state cache, after which the contents of the shared
|
||
state cache is copied to
|
||
<code class="filename">${DEPLOY_DIR_IMAGE}</code>.
|
||
If <code class="filename">do_deploy</code> is in the shared
|
||
state cache and its signature indicates that the
|
||
cached output is still valid (i.e. if no
|
||
relevant task inputs have changed), then the
|
||
contents of the shared state cache will be copied
|
||
directly to
|
||
<code class="filename">${DEPLOY_DIR_IMAGE}</code> by the
|
||
<code class="filename">do_deploy_setscene</code> task
|
||
instead, skipping the
|
||
<code class="filename">do_deploy</code> task.
|
||
</div>
|
||
<p>
|
||
</p>
|
||
</li>
|
||
<li class="listitem">
|
||
<p>
|
||
The following task definition is glue logic needed to
|
||
make the previous settings effective:
|
||
</p>
|
||
<pre class="literallayout">
|
||
python do_deploy_setscene () {
|
||
sstate_setscene(d)
|
||
}
|
||
addtask do_deploy_setscene
|
||
</pre>
|
||
<p>
|
||
<code class="filename">sstate_setscene()</code> takes the flags
|
||
above as input and accelerates the
|
||
<code class="filename">do_deploy</code> task through the
|
||
shared state cache if possible.
|
||
If the task was accelerated,
|
||
<code class="filename">sstate_setscene()</code> returns True.
|
||
Otherwise, it returns False, and the normal
|
||
<code class="filename">do_deploy</code> task runs.
|
||
For more information, see the
|
||
"<a class="link" href="../bitbake-user-manual/setscene.html" target="_self">setscene</a>"
|
||
section in the BitBake User Manual.
|
||
</p>
|
||
</li>
|
||
<li class="listitem">
|
||
<p>
|
||
The <code class="filename">do_deploy[dirs] = "${DEPLOYDIR} ${B}"</code>
|
||
line creates <code class="filename">${DEPLOYDIR}</code> and
|
||
<code class="filename">${B}</code> before the
|
||
<code class="filename">do_deploy</code> task runs, and also sets
|
||
the current working directory of
|
||
<code class="filename">do_deploy</code> to
|
||
<code class="filename">${B}</code>.
|
||
For more information, see the
|
||
"<a class="link" href="../bitbake-user-manual/variable-flags.html" target="_self">Variable Flags</a>"
|
||
section in the BitBake User Manual.
|
||
</p>
|
||
<div class="note" title="Note" style="margin-left: 0.5in; margin-right: 0.5in;">
|
||
<h3 class="title">Note</h3>
|
||
In cases where
|
||
<code class="filename">sstate-inputdirs</code> and
|
||
<code class="filename">sstate-outputdirs</code> would be the
|
||
same, you can use
|
||
<code class="filename">sstate-plaindirs</code>.
|
||
For example, to preserve the
|
||
<code class="filename">${PKGD}</code> and
|
||
<code class="filename">${PKGDEST}</code> output from the
|
||
<a class="link" href="../ref-manual/ref-tasks-package.html" target="_self"><code class="filename">do_package</code></a>
|
||
task, use the following:
|
||
<pre class="literallayout">
|
||
do_package[sstate-plaindirs] = "${PKGD} ${PKGDEST}"
|
||
</pre>
|
||
</div>
|
||
<p>
|
||
</p>
|
||
</li>
|
||
<li class="listitem">
|
||
<p>
|
||
<code class="filename">sstate-inputdirs</code> and
|
||
<code class="filename">sstate-outputdirs</code> can also be used
|
||
with multiple directories.
|
||
For example, the following declares
|
||
<code class="filename">PKGDESTWORK</code> and
|
||
<code class="filename">SHLIBWORK</code> as shared state
|
||
input directories, which populates the shared state
|
||
cache, and <code class="filename">PKGDATA_DIR</code> and
|
||
<code class="filename">SHLIBSDIR</code> as the corresponding
|
||
shared state output directories:
|
||
</p>
|
||
<pre class="literallayout">
|
||
do_package[sstate-inputdirs] = "${PKGDESTWORK} ${SHLIBSWORKDIR}"
|
||
do_package[sstate-outputdirs] = "${PKGDATA_DIR} ${SHLIBSDIR}"
|
||
</pre>
|
||
<p>
|
||
</p>
|
||
</li>
|
||
<li class="listitem">
|
||
<p>
|
||
These methods also include the ability to take a
|
||
lockfile when manipulating shared state directory
|
||
structures, for cases where file additions or removals
|
||
are sensitive:
|
||
</p>
|
||
<pre class="literallayout">
|
||
do_package[sstate-lockfile] = "${PACKAGELOCK}"
|
||
</pre>
|
||
<p>
|
||
</p>
|
||
</li>
|
||
</ul></div>
|
||
<p>
|
||
</p>
|
||
<p>
|
||
Behind the scenes, the shared state code works by looking in
|
||
<a class="link" href="../ref-manual/var-SSTATE_DIR.html" target="_self"><code class="filename">SSTATE_DIR</code></a>
|
||
and
|
||
<a class="link" href="../ref-manual/var-SSTATE_MIRRORS.html" target="_self"><code class="filename">SSTATE_MIRRORS</code></a>
|
||
for shared state files.
|
||
Here is an example:
|
||
</p>
|
||
<pre class="literallayout">
|
||
SSTATE_MIRRORS ?= "\
|
||
file://.* http://someserver.tld/share/sstate/PATH;downloadfilename=PATH \n \
|
||
file://.* file:///some/local/dir/sstate/PATH"
|
||
</pre>
|
||
<p>
|
||
</p>
|
||
<div class="note" title="Note" style="margin-left: 0.5in; margin-right: 0.5in;">
|
||
<h3 class="title">Note</h3>
|
||
The shared state directory
|
||
(<code class="filename">SSTATE_DIR</code>) is organized into
|
||
two-character subdirectories, where the subdirectory
|
||
names are based on the first two characters of the hash.
|
||
If the shared state directory structure for a mirror has the
|
||
same structure as <code class="filename">SSTATE_DIR</code>, you must
|
||
specify "PATH" as part of the URI to enable the build system
|
||
to map to the appropriate subdirectory.
|
||
</div>
|
||
<p>
|
||
</p>
|
||
<p>
|
||
The shared state package validity can be detected just by
|
||
looking at the filename since the filename contains the task
|
||
checksum (or signature) as described earlier in this section.
|
||
If a valid shared state package is found, the build process
|
||
downloads it and uses it to accelerate the task.
|
||
</p>
|
||
<p>
|
||
The build processes use the <code class="filename">*_setscene</code>
|
||
tasks for the task acceleration phase.
|
||
BitBake goes through this phase before the main execution
|
||
code and tries to accelerate any tasks for which it can find
|
||
shared state packages.
|
||
If a shared state package for a task is available, the
|
||
shared state package is used.
|
||
This means the task and any tasks on which it is dependent
|
||
are not executed.
|
||
</p>
|
||
<p>
|
||
As a real world example, the aim is when building an IPK-based
|
||
image, only the
|
||
<a class="link" href="../ref-manual/ref-tasks-package_write_ipk.html" target="_self"><code class="filename">do_package_write_ipk</code></a>
|
||
tasks would have their shared state packages fetched and
|
||
extracted.
|
||
Since the sysroot is not used, it would never get extracted.
|
||
This is another reason why a task-based approach is preferred
|
||
over a recipe-based approach, which would have to install the
|
||
output from every task.
|
||
</p>
|
||
</div></body>
|
||
</html>
|