Commit Graph

70 Commits

Author SHA1 Message Date
Scott Garman
62d538fbe6 make exception handling syntax consistent
Update exception handling syntax to use the modern style:
except ExcType as localvar

(Bitbake rev: dbf5f42b06bef81749b13aa99945cc1292a6676d)

Signed-off-by: Scott Garman <scott.a.garman@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-15 11:13:13 +01:00
Richard Purdie
e5f3cc34df bitbake/fetch2: When replacing URLs in mirror handling mask out empty entries
The symptom of this problem is something like a cvs url which specifies
a username where the username is then passed through to something like
an http mirror.

This patch fixes things by ensuring empty entries are preserved in the
new URL.

(Bitbake rev: c1d978d7bd1ac8eb1e2d50029ab2384be9f72fb4)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-13 15:10:07 +01:00
Chris Larson
754d1c6983 persist_data: implement comparison, same as dict
(Bitbake rev: 1190406c526c7bb7cf415867be83e0403812a7dd)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-05-06 18:05:11 +01:00
Chris Larson
7fc020aa15 persist_data: raise KeyError on missing elements
(Bitbake rev: a4f62433845c29f98c6a9746d5d2847bf9506ea5)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-05-06 15:49:17 +01:00
Chris Larson
88dbb0523c Initial work on getting bitbake working under pypy
- use os.chmod, not os.fchmod, as the latter is missing under pypy
- rearrange our imports a bit
- don't die if sqlite3 is missing shared cache support

(Bitbake rev: f229824dc9c453adf6067500e2bf6761536e4f2f)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-05-06 15:42:03 +01:00
Richard Purdie
c4371138f7 bitbake/fetch2: Fix the problems introduced by the git fetcher AUTOREV fix
The ordering constrains on the urldata_init functions are not straight
forward. To avoid further problems, create a helper function to setup
the source revisions which the init functions can all at the appropriate
point.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-04-20 02:13:26 +01:00
Richard Purdie
2d2d07275c bitbake/fetch2/git: Fix a bug where AUTOREV and the git fetcher interact badly
Fix a bug where ud.branches were being referenced before it was set by
the git fetcher when using AUTOREV. To do this some ordering needed
to be changed. This fixes errors like:

ERROR: Error parsing /recipes-kernel/linux/rt-tests_git.bb: Failure expanding variable
SRCPV, expression was ${@bb.fetch2.get_srcrev(d)} which triggered exception
AttributeError: 'FetchData' object has no attribute 'branches'

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-04-19 23:48:41 +01:00
Richard Purdie
a778fb3115 bitbake/fetch2: Allow local file:// urls to be found on mirrors
With the current implementation, file:// urls as used by sstate don't access the
mirror code, breaking sstate mirror support. This change enables the usual
mirror handling. To do this, we remove the localfile special case, using the basename
paramemter instead. We also ensure the downloads directory is checked for files.

The drawback of this change is that file urls containing "*" globing require special
casing in the core.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-03-08 11:23:34 -08:00
Khem Raj
54e10127c4 fetch, fetch2: Get rid of DeprecationWarning notice
* This patch fixes a cosmetic issue currently we get with master

WARNING: /home/kraj/work/bitbake/lib/bb/fetch2/__init__.py:733:
DeprecationWarning: Call to deprecated function bb.mkdirhier: Please use bb.utils.mkdirhier instead.  bb.mkdirhier("%s/%s" % (rootdir, destdir))

(Bitbake rev: 36fe59ce314c295d239b76de34c8714def2c32d5)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-03-03 22:51:33 +00:00
Richard Purdie
707132c60a bitbake/fetch2: Ensure SRCREV_pn-PN is checked for a revision when the SRC_URI is unnamed
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-23 11:47:50 +00:00
Richard Purdie
3ef2d39dbf bitbake/fecth2: Ensure BB_FETCH_PREMIRRORONLY being set as false is handled correctly
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-16 12:35:35 +00:00
Richard Purdie
b9a97e07b4 bitbake/fetch2: Revert part of the unpack change until the issues with it are resolved
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-13 00:05:06 +00:00
Mark Hatle
3faa635fd4 fetch2: unpack revision
Revise the unpack function to have a way to disable the unpack.  This is
based on the work from "Andreas Oberritter <obi@opendreambox.org>", see
http://cgit.openembedded.net/cgit.cgi/openembedded/commit/?id=2bdfe8519eda8067845019a699acdf19a21ba380

In addition, the to_boolean function comes from the work of
"Chris Larson <chris_larson@mentor.com>", see
http://cgit.openembedded.net/cgit.cgi/openembedded/commit/?id=900cc29b603691eb3a077cb660545ead3715ed54

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
2011-02-12 00:30:29 +00:00
Richard Purdie
029f19bd3d bitbake/fetch2: Add explict network access exception and handling to give users usable error messages
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-11 22:08:21 +00:00
Richard Purdie
4a226460d5 bitbake/fetch2: Fix message about md5 checksums when sha256 is incorrect
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-11 21:59:23 +00:00
Richard Purdie
132ec8d3b7 bitbake/fetch2: Fix patch merge error
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-11 16:24:26 +00:00
Richard Purdie
a72d49478e bitbake/fetch2: Add some debug output so its clear when PREMIRRORS, upstream and MIRRORS are being used
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-11 16:16:51 +00:00
Richard Purdie
20afb21a37 bitbake/fetch2: Ensure network access status is reset for each loop iteration in download()
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-11 16:15:36 +00:00
Richard Purdie
ea70c4362f bitbake/fetch2: Ensure failed fetch attempts are logged in the debug logs
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-11 12:14:20 +00:00
Richard Purdie
7f30131faa bitbake/fetch2: Correctly handle git mirror tarball fetching
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-11 12:06:44 +00:00
Richard Purdie
20ab1cb0ca bitbake/fetch2: Pass over malformatted (empty) mirror url lines
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-11 12:04:59 +00:00
Saul Wold
14dea89521 fetch2: Correct the clean() mechanism for the fetcher2 code
This create a clean() method in each of the fetcher modules
and correctly cleans the .done stamp file and lock files

Signed-off-by: Saul Wold <sgw@linux.intel.com>
2011-02-10 23:49:25 +00:00
Richard Purdie
29ed2c6e9a bitbake/fetch2: Fix negated if check for BB_FETCH_PREMIRRORONLY
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-09 01:23:56 +00:00
Mark Hatle
906285ff00 fetch2: Add SRPM knowledge
Enable the fetcher to be able to unpack and SRPM.  By default the system will
unpack the contents of the SRPM into the WORKDIR.

A new syntax "unpack=file" was developed for the SRC_URI, to allow for a
recipe to extract a specific file within an SRPM.  An unpack operation will
then be executed on the extracted file.

In order to apply extracted patches (or unpack files not specified with
unpack), you must specify the path using WORKDIR, i.e.:

file://${WORKDIR}/mypatch.patch

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
2011-02-08 18:01:35 +00:00
Richard Purdie
aa45760702 bitbake/fetch2: Ignore UnboundLocalError in exception handler in try_mirror
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-08 17:30:45 +00:00
Richard Purdie
ee1a9c0476 bitbake/fetch2: Move symlink handling into try_mirror where is belongs instead of the main download function
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-08 14:02:16 +00:00
Richard Purdie
c4e66a7fe0 bitbake/fetch2: When using BB_FETCH_PREMIRRORONLY, set BB_NO_NETWORK after premirrors as there could be data processing needed by the real fetcher
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-08 14:02:16 +00:00
Richard Purdie
91eeb9b0d3 bitbake/fetch2: Ensure we only remove files, not directories when fetch failures occur
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-08 14:02:16 +00:00
Richard Purdie
0713fb6b99 bitbake/fetch2: Ensure original ud is preserved in try_mirror
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-08 14:02:16 +00:00
Saul Wold
1544aa8ab4 fetch2: add try/finally to ensure lockfile is unlocked on failure
Signed-off-by: Saul Wold <sgw@linux.intel.com>
2011-02-07 17:29:46 -08:00
Richard Purdie
500c66337c bitbake/fetch2:Fetch Make using the fn based cache optional
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 21:13:13 +00:00
Richard Purdie
d4b4b48add bitbake/fetch2: Fix pickling issues with fetcher exceptions
See the problems in http://bugs.python.org/issue1692335, need to set self.args
correctly.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 21:13:13 +00:00
Richard Purdie
94faffdaf6 bitbake/fetch2: Update mirror processing to ensure we look for mirror tarballs
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 19:48:21 +00:00
Richard Purdie
37624b9745 bitbake/fetch2: Update forcefetch and mirror handling to clean up, simplfy and bug fix the code
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 16:52:42 +00:00
Richard Purdie
ded83ffc3b bitbake/fetch2: Match Fetcher log domain to that in bb.msg
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 14:50:27 +00:00
Richard Purdie
7202a77134 bitbake/fetch2: Use True instead of integer values
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:37 +00:00
Richard Purdie
8f2abf4a9f bitbake/fetch2: Drop name/override ordering backwards compatibility code
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:37 +00:00
Richard Purdie
ca7adf7529 bitbake/fetch2: Define a new interface to the fetcher code though the 'Fetch' class
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:37 +00:00
Richard Purdie
08a9fef4fa bitbake/fetch2: Fix typo
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:37 +00:00
Richard Purdie
806d1c25f8 bitbake/fetch2: Drop legacy CVSDATE support
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:36 +00:00
Richard Purdie
f60d9d6bd4 bitbake/fetch2: Move getSRCDate to FetchData class where is more appropriate
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:36 +00:00
Richard Purdie
831c6c1d3c bitbake/fetch2: Make srcrev_internal_helper a normal function, doesn't belong in the FetchMethod class
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:36 +00:00
Richard Purdie
ab0dd13974 bitbake/fetch2: Rename Fetch class to FetchMethod
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:36 +00:00
Richard Purdie
74b71864fe bitbake/fetch2: Drop old md5 handling code
Drop some old md5 functions since we have improved functionality now which includes
sha256 checksum support. This stops each download being md5 checksumed twice.

Also change ".md5" stamp extentions to ".done" to better describe its use as a
download complete marker file and no longer write the md5 sum to the files.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:36 +00:00
Richard Purdie
d08397ba4d bitbake/fetch2: Rewrite and improve exception handling, reusing core functions for common operations where possible
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:36 +00:00
Richard Purdie
f6eefb3ca3 bitbake/fetch2: Have all fetcher exceptions derived from a common BBFetchException class
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:35 +00:00
Richard Purdie
8daab5b951 bitbake/fetch2: Drop unused parameter for localpath() and update comments
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:35 +00:00
Richard Purdie
7e4fbfc1c1 bitbake/fetch2: Simplfy setup_srcrevs
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:35 +00:00
Richard Purdie
49a022d25d bitbake/fetch2: Simplify localpath variable handling FetchData init
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:35 +00:00
Richard Purdie
5a52cb80d6 bitbake/fetch2: Define a sane localpath function and remove code duplication
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:35 +00:00