Commit Graph

57181 Commits

Author SHA1 Message Date
Scott Branden via Openembedded-core
e4abc4dbf3 cryptodev: update to Jan.23, 2020
* Remove backported patches:
  0001-Fix-module-loading-with-Linux-v5.0-rc5.patch

Update to latest cryptodev dated Jan.23, 2020.
Fixes build on kernel v5.5+

(From OE-Core rev: 877d3ff61fbd55baa5e5fd00bd8a58b31e214cc9)

Signed-off-by: Scott Branden <scott.branden@broadcom.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:02 +00:00
Jaewon Lee
ed4bdd0f91 sstate.bbclass: fix issue while handling long sstate filenames
When moving to python3, divison using '/' now returns float instead of
an integer. In upstream commit b8025e9720
sstate filenames longer than the limit are changed to just include
necessary info + 3 fields just for information. The space left over
after the necessary info is divided into 3 for each of the fields.
Using '//' instead to do the division to solve the following error
message:

avail = (254 - len(hash + "_" + taskname + extension) -
len(components[0]) - len(components[1]) - len(components[5]) -
len(components[6]) - 7) / 3
    >        components[2] = components[2][:avail]
             components[3] = components[3][:avail]
TypeError: slice indices must be integers or None or have an __index__
method

(From OE-Core rev: 2acfee61a062c6520a413b2a797544d968bb0c76)

Signed-off-by: Jaewon Lee <jaewon.lee@xilinx.com>
Signed-off-by: Mark Hatle <mark.hatle@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:02 +00:00
Mark Hatle
38a754214a package.bbclass: Support stripping and debug copy of static libraries
By default, we won't copy and strip static libraries.  However, this
functionality can be useful in some cases where people are doing
development on the target, and don't generally want the larger debug
capable static libraries.  To enable the new functionality set:

    PACKAGE_DEBUG_STATIC_SPLIT = '1'

Add a new function splitstaticdebuginfo.  Thus function will copy the
unmodified static library into the specific debug directory location.
By keeping an unmodified version, it is possible for a user trying
to debug something to use -L /usr/lib/.debug-static and their existing
build commands to switch from stripped to full debug versions.

The PACKAGE_DEBUG_SPLIT_STYLE will select between two different
approaches, /usr/lib/debug-static or <path>/.debug-static.

Additionally you can now choose to strip static libraries to conserve
space.  If either 'PACKAGE_DEBUG_STATIC_SPLIT' or 'PACKAGE_STRIP_STATIC'
is set to 1, the static library will be stripped.  (This is not on by
default, as it could make diagnosing static library usage difficult in
some cases.)

Add to insane.bbclass a skip to the staticdev warning for the specific
-dbg package versions.

(From OE-Core rev: 17fa66c8199d73f0b59b2b3e609075933bf1e74b)

Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:02 +00:00
Mark Hatle
dae03ec7e8 package.bbclass: Allow INHIBIT_PACKAGE_STRIP_FILES to skip .ko and static libs
Change the order of the skip processing to happen before any .ko and static
library processing.  This will allow these types of files to be individually
skipped if necessary.

(From OE-Core rev: c50fcd54ffe60b63d042d05e6cf538a593dc410f)

Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:02 +00:00
Mark Hatle
6a543159a3 package.bbclass: Fix debug source processing for static libraries
Format of the sources list is the [ (file, [source, ...]), ... ] before
this change, the static libraries were processed but the items were
included incorrectly causing no sources for static libraries to be
included.

(From OE-Core rev: fa356b23c2f4599681693bba50d36659b07a8125)

Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:02 +00:00
Alexander Kanavin
0e48ca10aa chkconfig: remove the recipe
chkconfig is one of the options for virtual/update-alternatives,
however opkg-utils have been used as the default for a very long time,
while chkconfig isn't anymore tested in any way, and is stuck
at a very old version due to newer versions requiring selinux.

[YOCTO #11264]

(From OE-Core rev: 61efc1e287326f52810a439ccde996f45ef89733)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:02 +00:00
Trevor Gamblin
9afefb02b0 qemurunner.py: add try/except for pid handling race
In some instances, attempts to remove the qemu pidfile within the
stop() method fail despite the os.path.exists() call immediately
before implying that the file is present. Add a try/except block
to log a warning if this occurs, rather than failing outright,
since the process simply appears to be exiting at an inconvenient
time.

[YOCTO #13675]

(From OE-Core rev: eadb899e23b18eb9eaff145c3bf5b20fb417c3e8)

Signed-off-by: Trevor Gamblin <trevor.gamblin@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:02 +00:00
Khem Raj
504b206e37 oeqa: Use cpio 2.13 as testcase
cpio 2.12 was released in 2015 and might have used older autotools
which could result in errors like

https://bugzilla.yoctoproject.org/show_bug.cgi?id=13779

Bumping to 2.13 will help in matching the tool versions
A good change on top would be to run

aclocal -I .; autoheader; autoconf; automake --add-missing -c

before running configure step perhaps

[YOCTO #13779]

(From OE-Core rev: 84eb1dc4fe8a11cd2d05b703070a6fb6de05b873)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:02 +00:00
Bruce Ashfield
6113de76c8 kern-tools: fix merge_config when LD contains parameters
To ensure that the kernel linker is used when allno/mod/yes config
merge_config steps were executed, the call to make was tweaked to
explicitly pass LD.

But since the variable wasn't quoted, any parameters to LD (like
the sysroot) were mistakenly passed to make, and hence could trigger
an error on some architectures.

We also tweak the logging to hightlight errors like this in the
future and avoid losing it in the noise of merge configs sometimes
overly verbose output.

(From OE-Core rev: a60c4c116efecd7a6ee5a11b1d366bb00b9d23ce)

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:02 +00:00
Gavin Li
cc71bb7b73 kernel-yocto: fix defconfig detection in find_sccs()
The current code would cause a file like "config.bin" to added to the
config sources list. I am sure the intention was to add any files with
defconfig in its name and not the other way around.

(From OE-Core rev: 06577d49c50b7e9f1f40fe5b52ec88d1bdc1430c)

Signed-off-by: Gavin Li <gavinli@thegavinli.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:02 +00:00
Martin Jansa
7ba241a697 kernel-yocto.bbclass: do_kernel_configme: don't use +errexit, merge_config_build.log and fail when /.config wasn't created
* for whatever reason, instead of silently continuing to build default kernel config

(From OE-Core rev: ec281a948a23365bd4536139dac077fb1a5eae6d)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:02 +00:00
Martin Jansa
040ec2d8ab kernel-yocto.bbclass: set KERNEL_LD also for merge_config.sh
* when ld-is-gold is used, merge_config.sh silently fails and doesn't generate anything
  useful in .config (and also include directory isn't created):

  tmp-glibc-bfd-qemux86-do_kernel_configme/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build:
  total 164K
  drwxr-xr-x 4 martin martin 4.0K Feb  5 02:27 .
  drwxrwxr-x 7 martin martin 4.0K Feb  5 02:27 ..
  -rw-rw-r-- 1 martin martin 133K Feb  5 02:28 .config
  -rw-rw-r-- 1 martin martin   39 Feb  5 02:27 .gitignore
  drwxr-xr-x 4 martin martin 4.0K Feb  5 02:27 include
  -rw-rw-r-- 1 martin martin  201 Feb  5 02:27 Makefile
  drwxrwxr-x 4 martin martin 4.0K Feb  5 02:27 scripts
  lrwxrwxrwx 1 martin martin   61 Feb  5 02:27 source -> /OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source

  tmp-glibc-gold-qemux86-do_kernel_configme/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build:
  total 28K
  drwxr-xr-x 3 martin martin 4.0K Feb  5 02:26 .
  drwxrwxr-x 7 martin martin 4.0K Feb  5 02:26 ..
  -rw-rw-r-- 1 martin martin   74 Feb  5 02:26 .config
  -rw-rw-r-- 1 martin martin   39 Feb  5 02:26 .gitignore
  -rw-rw-r-- 1 martin martin  201 Feb  5 02:26 Makefile
  drwxrwxr-x 4 martin martin 4.0K Feb  5 02:26 scripts
  lrwxrwxrwx 1 martin martin   61 Feb  5 02:26 source -> /OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source

  $ cat tmp-glibc-gold-qemux86-do_kernel_configme/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build/.config
  CONFIG_LOCALVERSION="-yocto-standard"

* and because the failure is silent, it will happily start building
  kernel with default defconfig, which even builds OK for qemux86-64
  but on qemux86 I've noticed this issue, because incorrectly
  configured kernel build fails with:

  $ tail -n 20 tmp-glibc-gold-qemux86/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/temp/log.do_compile
    LD       /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build/tools/objtool/libsubcmd-in.o
    AR       /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build/tools/objtool/libsubcmd.a
    HOSTCC  scripts/mod/mk_elfconfig
    CC      scripts/mod/devicetable-offsets.s
    CC      scripts/mod/empty.o
  cc1: error: code model 'kernel' not supported in the 32 bit mode
  cc1: sorry, unimplemented: 64-bit mode not compiled in
  make[2]: *** [/OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source/scripts/Makefile.build:99: scripts/mod/devicetable-offsets.s] Error 1
  make[2]: *** Waiting for unfinished jobs....
  cc1: error: code model 'kernel' not supported in the 32 bit mode
  cc1: sorry, unimplemented: 64-bit mode not compiled in
  make[2]: *** [/OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source/scripts/Makefile.build:266: scripts/mod/empty.o] Error 1
  make[1]: *** [/OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source/Makefile:1111: prepare0] Error 2
  make[1]: *** Waiting for unfinished jobs....
    LD       /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build/tools/objtool/objtool-in.o
  /OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source/tools/objtool
    LINK     /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build/tools/objtool/objtool
  make: *** [/OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source/Makefile:179: sub-make] Error 2
  WARNING: exit code 1 from a shell command.

* the issue happens in log.do_kernel_configme, but the log is completely useless:

  cat tmp-glibc-gold-qemux86-do_kernel_configme/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/temp/log.do_kernel_configme
  DEBUG: Executing python function extend_recipe_sysroot
  NOTE: Direct dependencies are ['virtual:native:/OE/build/oe-core/openembedded-core/meta/recipes-devtools/bison/bison_3.5.bb:do_populate_sysroot', '/OE/build/oe-core/openembedded-core/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb:do_populate_sysroot', 'virtual:native:/OE/build/oe-core/openembedded-core/meta/recipes-extended/bc/bc_1.07.1.bb:do_populate_sysroot', '/OE/build/oe-core/openembedded-core/meta/recipes-devtools/binutils/binutils-cross_2.33.bb:do_populate_sysroot', '/OE/build/oe-core/openembedded-core/meta/recipes-devtools/quilt/quilt-native_0.66.bb:do_populate_sysroot', '/OE/build/oe-core/openembedded-core/meta/recipes-devtools/gcc/gcc-cross_9.2.bb:do_populate_sysroot']
  NOTE: Installed into sysroot: ['bison-native', 'bc-native', 'binutils-cross-i686', 'gcc-cross-i686', 'autoconf-native', 'xz-native', 'flex-native', 'gnu-config-native', 'automake-native', 'libtool-native', 'texinfo-dummy-native', 'gettext-minimal-native', 'readline-native', 'zlib-native', 'gmp-native', 'libmpc-native', 'mpfr-native', 'linux-libc-headers', 'm4-native', 'ncurses-native', 'pkgconfig-native']
  NOTE: Skipping as already exists in sysroot: ['kern-tools-native', 'quilt-native']
  DEBUG: sed -e 's:^[^/]*/:/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/recipe-sysroot-native/:g' /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/bison-native/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/gcc-cross-i686/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/autoconf-native/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/gnu-config-native/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/automake-native/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/libtool-native/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/gmp-native/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/ncurses-native/fixmepath /OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/pkgconfig-native/fixmepath | xargs sed -i -e 's:FIXMESTAGINGDIRTARGET:/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/recipe-sysroot:g; s:FIXMESTAGINGDIRHOST:/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/recipe-sysroot-native:g' -e 's:FIXME_PSEUDO_SYSROOT:/OE/build/oe-core/tmp-glibc/sysroots-components/x86_64/pseudo-native:g' -e 's:FIXME_HOSTTOOLS_DIR:/OE/build/oe-core/tmp-glibc/hosttools:g' -e 's:FIXME_PKGDATA_DIR:/OE/build/oe-core/tmp-glibc/pkgdata/qemux86:g' -e 's:FIXME_PSEUDO_LOCALSTATEDIR:/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/pseudo/:g' -e 's:FIXME_LOGFIFO:/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/temp/fifo.10476:g'
  DEBUG: Python function extend_recipe_sysroot finished
  DEBUG: Executing shell function do_kernel_configme
  DEBUG: Shell function do_kernel_configme finished

  ]because merge_config.sh is redirected to work-shared/qemux86/kernel-source/.kernel-meta/cfg/merge_config_build.log
  and bbfatal_log called in do_kernel_configme only when merge_config.sh fails which it should in this case
  but doesn't.

  The merge_config_build.log shows 5.4 specific error:
    scripts/Kconfig.include:39:  gold linker 'i686-oe-linux-ld' not supported
  but then it happily continues reporting what isn't applied in not created
  .config file

  --- tmp-glibc-bfd-qemux86-do_kernel_configme/work-shared/qemux86/kernel-source/.kernel-meta/cfg/merge_config_build.log  2020-02-05 02:28:01.455520207 +0100
  +++ tmp-glibc-gold-qemux86-do_kernel_configme/work-shared/qemux86/kernel-source/.kernel-meta/cfg/merge_config_build.log 2020-02-05 02:26:30.656470909 +0100
  @@ -171,586 +171,5113 @@
     HOSTCC  scripts/kconfig/symbol.o
     HOSTLD  scripts/kconfig/conf
   scripts/kconfig/conf  --alldefconfig Kconfig
  -./.tmp.config.jDjIEYi9Yq:2208:warning: unexpected data:  #
  -./.tmp.config.jDjIEYi9Yq:2209:warning: unexpected data:  # Generic Driver Options
  -./.tmp.config.jDjIEYi9Yq:2210:warning: unexpected data:  #
  -./.tmp.config.jDjIEYi9Yq:3102:warning: symbol value 'm' invalid for SAMPLE_SECCOMP
  -./.tmp.config.jDjIEYi9Yq:3119:warning: symbol value 'm' invalid for NF_CT_PROTO_GRE
  -./.tmp.config.jDjIEYi9Yq:3120:warning: symbol value 'm' invalid for NF_CT_PROTO_SCTP
  -./.tmp.config.jDjIEYi9Yq:3121:warning: symbol value 'm' invalid for NF_CT_PROTO_UDPLITE
  -#
  -# configuration written to .config
  -#
  +scripts/Kconfig.include:39:  gold linker 'i686-oe-linux-ld' not supported
  +make[2]: *** [/OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source/scripts/kconfig/Makefile:73: alldefconfig] Error 1
  +make[1]: *** [/OE/build/oe-core/tmp-glibc/work-shared/qemux86/kernel-source/Makefile:567: alldefconfig] Error 2
   make[1]: Leaving directory '/OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build'
  +make: *** [Makefile:179: sub-make] Error 2
  +grep: /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build/.config: No such file or directory
  +Value requested for CONFIG_LOCALVERSION not in final .config
  +Requested value:  CONFIG_LOCALVERSION=""
  +Actual value:
  +
  +grep: /OE/build/oe-core/tmp-glibc/work/qemux86-oe-linux/linux-yocto/5.4.15-r0/linux-qemux86-standard-build/.config: No such file or directory
  +Value requested for CONFIG_LOCALVERSION_AUTO not in final .config
  +Requested value:  # CONFIG_LOCALVERSION_AUTO is not set
  +Actual value:

* I don't know why merge_config.sh uses separate more difficult to find merge_config_build.log
  instead of leaving the output to end in log.do_kernel_configme, I'll send it as separate
  commit so that it can be discussed there.

(From OE-Core rev: 36fe2efc91fec5a52374a2dc4814a69076a2d28e)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:02 +00:00
Paul Barker
83c1977475 kernel-yocto: Only override CONFIG_LOCALVERSION if LINUX_VERSION_EXTENSION is set
CONFIG_LOCALVERSION may already be set in a defconfig or config fragment
and this should not be unconditionally overridden.

(From OE-Core rev: 613c43f910cbdf5e03ef3d5bbf92de69082a30c9)

Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:02 +00:00
Paul Barker
f88946d48d kernel-yocto: Drop setting of unused variables in do_kernel_metadata
The machine_branch and machine_srcrev variables were set but not used in
do_kernel_metadata.

(From OE-Core rev: ee358e03b1ad6eb0624cee0306fef807de0116a6)

Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:01 +00:00
Paul Barker
de2ef01204 kernel-yocto: Move defaults and tasks from linux-yocto.inc into bbclass
This allows the kernel-yocto bbclass to be inherited in a recipe without
needing to include linux-yocto.inc. The bbclass should stand on its own
and linux-yocto.inc does a few things which may not be desired in other
kernel recipes (such as modifying KERNEL_FEATURES).

The LINUX_VERSION_EXTENSION default is not moved as other kernel recipes
may not want this setting in place.

(From OE-Core rev: 48d8f97975c0b167c0f4f32145813f00b8c8a7a8)

Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:01 +00:00
Martin Jansa
caac8938fe kern-tools-native: use more common S value and oe_runmake
(From OE-Core rev: 83a552d0aea7f789b3a5a7ff96f3d3a5e346cc2c)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:01 +00:00
Bruce Ashfield
2f7f4ca89d kern-tools: integrate merge_config fix for gold linker
Bumping the SRCREV to pickup the following fix:

   Author: Martin Jansa <Martin.Jansa@gmail.com>
   Date:   Wed Feb 5 03:26:57 2020 +0100

       merge_config.sh: pass LD variable from shell environment to make

       * since 5.4 kernel Kconfig will fail immediately when it detects
         that LD points to gold linker:

         scripts/Kconfig.include:39:  gold linker 'i686-oe-linux-ld' not supported

       * in OE we already pass bfd linker in KERNEL_LD variable to merge_config.sh
         but we need to pass it also into the make call here

       Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>

(From OE-Core rev: fa964575f9e00d8530563e61075992c5b69df137)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:01 +00:00
Bruce Ashfield
36491a9220 kern-tools: tweak symbol_why to be python safe
Updating the SRCREV to pickup tweaks to symbol_why.py to be
python3 safe:

  - we explicitly call /usr/bin/env python3
  - we full specifiy our symbols
  - do not assume that 'None' can be converted to a string

(From OE-Core rev: eaa83453fe206567253257fcefdbf6feb6d53d72)

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:01 +00:00
Richard Purdie
459e92aa93 ncurses: Fix reproducibility issue
The build was deciding whether to rename manpages based upon the presence of
/etc/debian_version. Be explicit about the configuration instead and
ensure determinism.

[YOCTO #13781]

(From OE-Core rev: 09c8a28893e7ca94a44232d802e1cb02a8f34b87)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:01 +00:00
Richard Purdie
ad4f1d59e0 libevdev: Fix determinism issue
We need to sort python dict output to be deterministic and generate consistent
header files.

(From OE-Core rev: 75e4cedb986379db2e8a897df52ee1363f9a9a80)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:01 +00:00
Richard Purdie
0f2fdd4abf sysvinit: Fix Reproducibility issue
With a sequence like:

bitbake sysvinit
bitbake sysvinit -c clean
bitbake sysvinit -c package_write_ipk -f

then the resulting package has two files with group "root/70" rather
than "root/shutdown". The issue is that of do_package is a setscene
task, base-passwd isn't present. This patch fixes that dependency
but there may be other cases of this problem around.

[YOCTO #13776]

(From OE-Core rev: 0227e929021263c51d2e7db36224000fecb01f1c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:01 +00:00
Ross Burton
1b463c9e74 sudo: specify where target tools are
sudo uses AC_PATH_PROG to find target paths, which means at best
potential host-contamination (and reproducible issues) and at worst it
thinks sh is at /your/build/path/hosttools/sh.

Solve this by explicitly passing the correct paths to configure.

(From OE-Core rev: 61650dd8498a093f3bfa93202c9cd2e9a7fb7834)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:01 +00:00
Richard Purdie
468575f973 libgcrypt: Fix determinism issue
The build was injection git information from the wrong git tree, stop this
to allow reproducible builds.

(From OE-Core rev: 506b36b6d86b3454fcc3cb85f6229cbe8d14f5b5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:01 +00:00
Richard Purdie
c0726d98e6 libinput: Fix determinism issue
The build was injection git information from the wrong git tree, stop this
to allow reproducible builds.

(From OE-Core rev: c3f6a6113f562ecdb13386c3ff52adb7973980a4)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:01 +00:00
Richard Purdie
d10da5f6e6 iputils: Fix build determinism
The suid/setcap code depends on whether setcap is on the host system or not
with suid as a fallback. Disable this functionality to be deterministic.

(From OE-Core rev: 8b00ec484fb851c301f13145e17707c0167feab1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:01 +00:00
Richard Purdie
49e4e1de7d openssl: Fix reproducibility issue
There was a build architecture leaking into the target ptest which
could vary depending upon host. Remove it as its cosmetic.

[YOCTO #13770]

(From OE-Core rev: 37db519eedb7eb5cd4f14d05f30f5d580aa7458d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:01 +00:00
Richard Purdie
2ac1632f18 perl: Fix various reproducibile build issues
Add a patch which handles the following issues:

a) Remove the \n from configure_attr.sh since it gets quoted differently depending on
   whether the shell is bash or dash which can cause the test result to be incorrect.
   Reported upstream: https://github.com/arsv/perl-cross/issues/87

b) Sort the order of the module lists from configure_mods.sh since otherwise
   the result isn't the same leading to makefile differences.
   Reported upstream: https://github.com/arsv/perl-cross/issues/88

c) Sort the Encode::Byte byte_t.fnm file output (and the makefile depends whilst
   there for good measure)
   This needs to go to upstream perl (not done)

d) Use bash for perl-cross configure since otherwise trnl gets set to "\n" with bash
   and "" with dash
   Reported upstream: https://github.com/arsv/perl-cross/issues/87

(From OE-Core rev: 482fd0d99f989b5a72a25bdf402fb2f219420b5d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:01 +00:00
Alexander Kanavin
155d79d526 perl: do not install files that contain build host specific data
This was breaking reproducibility, and the files aren't needed on
target.

[YOCTO #13772]

(From OE-Core rev: 2e0f30c4680221c693495e3a0327378d502a518b)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 13:20:01 +00:00
Frazer Clews
6ca91a91af bitbake: cooker/toaster: replaced deprecated method warn() with warning()
Removed the deprecated methods as it will only cause problems later on,
and since warn() just calls warning(), it shouldn't change anything

(Bitbake rev: a194f275235f22411cb2368f06a44f61ceb6a0f3)

Signed-off-by: Frazer Clews <frazer.clews@codethink.co.uk>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-06 22:49:29 +00:00
Mark Hatle
cdcacf26ec microblaze: Adjust Linux items from microblazeeb to microblaze
Due to recent changes to the tune, in order to match config.guess, the name
of the big-endian microblaze architecture was changes to 'microblaze'.

(From OE-Core rev: 6f6a6bbac684ead3fe6d070d61f17c2f611a2c87)

Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-06 12:16:34 +00:00
Mark Hatle
d6b0154a4c microblaze tune: cleanup +=
Various += were used, refactor these to be either = or .= depending on
usuage.

CONFLICTS should be '=', as no leading space is required and they are not
amending any other conflict settings.

The TUNE_CCARGS should be .= so that if the feature does not define a CCARG
blank spaces are not added to the CFLAGS.  This is consistent to how the arm
tuning is implemented.

(From OE-Core rev: 78c38857486d3107ecd95d0ceefabcf5152c3928)

Signed-off-by: Mark Hatle <mark.hatle@xilinx.com>
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-06 12:16:34 +00:00
Mark Hatle
83e58868b5 microblaze tune: Enable 64-bit
64-bit is not yet available in Linux, but some non-Linux uses exist.

(From OE-Core rev: 389691be34b79da1fafa8df2e6369e7771406cc2)

Signed-off-by: Mark Hatle <mark.hatle@xilinx.com>
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-06 12:16:34 +00:00
Mark Hatle
e45c382033 microblaze tune: Allow no version to be set
We want to allow no version to be configured.  This should use the GCC default
which is the latest defined version, currently 11.0.

(From OE-Core rev: 0d1551dcc169f2d8dbfbe01b4f1f0ae3ce4770ed)

Signed-off-by: Mark Hatle <mark.hatle@xilinx.com>
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-06 12:16:34 +00:00
Mark Hatle
7f150a2514 microblaze tune: change microblazeeb to microblaze
Using microblazeeb breaks a number of autoconf recipes, including newlib
components.  'microblaze' is defined as the big-endian version, while
microblazeel is defined as the little-endian version.

config.sub: 2018-07-03

...
        | maxq | mb | microblaze | microblazeel | mcore | mep | metag \
...
        | microblaze-* | microblazeel-* \
...
        microblaze*)
                basic_machine=microblaze-xilinx
...

(From OE-Core rev: c052b0c984b28d64527a66ea8e2936ca28b9406f)

Signed-off-by: Mark Hatle <mark.hatle@xilinx.com>
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-06 12:16:34 +00:00
Ross Burton
2aac5700b8 gtk+3: sort resources for reproducible binaries
The list of resources is gathered with $(wildcard) in Make, which isn't
sorted. If this order changes then the generated libraries will differ.

(From OE-Core rev: f3675be6be29426688187a135221431a0941d007)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-06 12:16:34 +00:00
Richard Purdie
934d21e032 libidn2: Fix reproducibility issue
The previous tweak for reproducibility didn't handle the duplicate
whitepace left behind, fix this.

[YOCTO #13771]

(From OE-Core rev: 0392fcbdc85180581ce7392212808ebb822cc2e8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-06 12:16:34 +00:00
Richard Purdie
829476febc tar: Fix build determinism, disable rsh
rsh is insecure and obsolete but tar will enable support if the binary is
on the host system. Some systems point it at ssh. Lets explictly disable it
for now unless someone actually needs/uses this at which point it could
become a packageconfig.

(From OE-Core rev: d14a4b0db92a9a7d1ff72a2e0faca7f1a23a0b68)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-06 12:16:34 +00:00
Richard Purdie
dd41dcbaa3 xserver-xorg: Fix reproducibility issue
The host kernel version was being encoded into the Xorg binary. Set
an appropriate configure option to avoid this and be deterministic.

(From OE-Core rev: 39e25b045231385c1e2a442fde2c5d4ee07640a8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-06 12:16:34 +00:00
Richard Purdie
500206534f patch: Extend to native/nativesdk and depend upon
There is a bug in patch 2.7.3 and earlier where index lines
in patches can change file modes when they shouldn't:
http://git.savannah.gnu.org/cgit/patch.git/patch/?id=82b800c9552a088a241457948219d25ce0a407a4

This leaks into debug sources in particular (e.g. tcp-wrappers where
source files are read-only). Add the dependency to target recipes
to avoid this problem until we can rely on 2.7.4 or later.

We could try and remove all index lines from patch files but it will be a
losing battle. We could try and identify all the recipes which change
modes on files in patches but again, its a losing battle.

Instead, compromise and have patch-native as a dependency
for target recipes. We use patch-replacement-native since patch-native
is in ASSUME_PROVIDED.

Also add nativesdk-patch to buildtools-tarball.

[YOCTO #13777]

(From OE-Core rev: 5ed0840c93804488cd1c1aba6cb382b2434714a5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-06 12:16:34 +00:00
Richard Purdie
8e2afe5dd3 mc: Fix manpage date indeterminism
The man page date can vary depending upon the host perl, e.g. in Russian
some versions print 'июня', others 'Июнь' or Polish 'czerwca' or 'czerwiec'.
Rather than depend upon perl-native to fix this, just remove the date from
the manpages.

(From OE-Core rev: 5553c20f9fa4f35bf711b6b9d5717dcf4bfefafa)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-06 12:16:34 +00:00
Richard Purdie
291a7414db mc: Set zipinfo presence determinstically
This value was floating causing differences in generated files.
Set it determinstically.

(From OE-Core rev: 11d7a9e37c1d3fc21396a98fefc9d34c0b9e784b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-06 12:16:34 +00:00
Richard Purdie
7c201279cb oeqa/reproducible: Improve test output and ensure deb+ipk compared
Adding newline characters between the packages in the failure output
massively improves readability.

Also ensure to output ipk failures when there are deb failures by
calling self.fail() at the end, else sometimes only partial differences
are returned.

(From OE-Core rev: 6e2e0480852177db75a6108d77c99c92c4e9950f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-06 12:16:34 +00:00
Richard Purdie
09c6a41b75 opkg-utils: Fix reproducibility issues in opkg-build
There is a sorting problem with opkg-build where the ipk generated is depending
upon the order of files on disk. The reason is the --sort option to tar only
influences the orders of files tar reads, not those passed by the -T option.

Add in a sort call to resolve this issue. To ensure consistent sorting we
also need to force to a specific locale (C) else the results are still not
deterministic.

(From OE-Core rev: a9b8287984c63420e10329a69f7ac5125f1687f8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-06 12:16:34 +00:00
Anuj Mittal
73333dd4b8 sqlite3: upgrade 3.30.1 -> 3.31.1
(From OE-Core rev: 621ea68239763ce8740731e745c5002c956d4c67)

Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-05 12:00:34 +00:00
Anuj Mittal
0ca68e153d libxml2: fix CVE-2020-7595
(From OE-Core rev: f2f7aa9a495774fe5a2e3947584cb3503bd1eaf1)

Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-05 12:00:34 +00:00
Anuj Mittal
8046030dd4 ncurses: add CVE_VERSION
Include the version number in a format that is comparable to what we get
from NVD.

(From OE-Core rev: fdb2a95d5e0265de1172940b6dc71fc7d602e8d1)

Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-05 12:00:34 +00:00
Anuj Mittal
b5b12cec76 x264: upgrade to latest revision
(From OE-Core rev: f7cc44c98792f7c2f479a9bd86d6ed9c96373c00)

Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-05 12:00:34 +00:00
Taras Kondratiuk via Openembedded-core
e03c56c30b gcc-9.2: fix bug #91102 'aarch64 ICE on Linux kernel with -Os'
Linux kernel compilation for aarch64 triggers ICE if
CONFIG_CC_OPTIMIZE_FOR_SIZE=y.

The rootcause is GCC bug #91102 'aarch64 ICE on Linux kernel with -Os'.
Apply the fix to 9.2.

(From OE-Core rev: 14f34d32bfdaa752f5043e62750d2e7b92c4b419)

Signed-off-by: Taras Kondratiuk <takondra@cisco.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-05 12:00:34 +00:00
Khem Raj
210daa5ea7 webkitgtk: Prefer -pthread over -lpthread
-pthread is compiler driver option which links in needed dependencies
for pthreads along with -lpthread, this aids in fixing build failures
on rv64

(From OE-Core rev: 94446ea1bfcb175af6fcaf5969a3a4507fdbf470)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-05 12:00:34 +00:00
Trevor Gamblin
b486903ccc logrotate.py: improve oeqa test implementation
See bug https://bugzilla.yoctoproject.org/show_bug.cgi?id=13632

Autobuilder tests occasionally fail, reporting that a new logfile
could not be created. While this failure did occur multiple times, it
could not be manually reproduced. However, there are issues with the
implementation of the logrotate.py script that can be fixed. These
changes will help make the failures clearer, should they continue to
occur.

Previously, the test_2_logrotate test would, after running the
logrotate tool, use "ls -al $HOME/logrotate_dir | wc -l" to count
the number of files in the rotation directory and determine if the
rotation was successful. The test to see if there are at least three
files is problematic, because depending on the version of ls used, it
may report the target value of 3 even when there are only hidden files
in the directory, potentially reporting a pass for the test when it
should actually fail. An example with coreutils:

root@qemux86-64:~# ls -al emptydir/
total 2
drwxr-xr-x 2 root root 1024 Jan 14 19:50 .
drwx------ 3 root root 1024 Jan 14 19:50 ..
root@qemux86-64:~#

Where "total" is the number of blocks used. Compare with busybox ls:

root@qemux86-64:~# ls -al emptydir/
drwxr-xr-x    2 root     root          1024 Jan 14 19:54 .
drwx------    3 root     root          1024 Jan 14 19:54 ..
root@qemux86-64:~#

Instead of using ls to verify that a certain number of files exists
in $HOME/logrotate_dir, the tests have been changed to rotate two
specific logs: the log for wtmp and a new logrotate_testfile created
during the second test. Both tests check that the logs are correctly
rotated into $HOME/logrotate_dir by using find and grep on the
expected filename (e.g. "wtmp" when rotated becomes "wtmp.1", so we
check to see that wtmp.1 is present in $HOME/logrotate_dir). In
addition, should the test fail, the output from logrotate -vf is
included in the test log to aid debugging. It has also been seen that
in some cases, the logrotate test fails because the /var/log/wtmp file
is not yet present. Since the objective of the test is to check the
logrotate functionality and not the presence of certain log files,
test_logrotate_wtmp uses the touch command to help ensure that the
file is present before the call to logrotate is issued.

Finally, note that while the autobuilder failures that this patch
addresses were only seen during core-image-full-cmdline tests, these
changes were successfully tested on core-image-minimal and
core-image-sato with the manual addition of logrotate and openssh-sshd
to the images.

(From OE-Core rev: b9392403e96c6d880de844b44da2c7300fcf411c)

Signed-off-by: Trevor Gamblin <trevor.gamblin@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-05 12:00:34 +00:00