We only build one gcc-cross per architecture and having target specific
flags means gcc-cross would rebuild. These flags are really for the
on target case, so they should be set in gcc-target only.
(From OE-Core rev: 851937dde81de2a9ef54c5f19a78fb12fb82afd4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>