Hosting binary sysroots for BuildStream builds
by Sam Thursfield
Hello,
BuildStream doesn't support the "bootstrap mode" that Morph and YBD use.
This is for the best as building stuff outside a sandbox can lead to
madness[1], but it means that Baserock needs to build and provide
sysroot binaries that we can then use to "seed" our reference system builds.
Currently we use the freedesktop SDK binaries, but this sucks for a few
reasons including that they are huge (1GB+ when we really only need
<300MB of stuff), and they aren't available for many platforms (x86_32,
x86_64 and armv8l64 only I think).
Producing our own binaries is easy enough to do; our
build-essential.morph / gnu-toolchain.bst stack already does everything
we need.
What I'm wondering is how best to host them for posterity. For now I've
been pulling the artifacts from https://ostree.baserock.org/cache/
directly, but this isn't good because things in a cache can be deleted,
and if our reference system builds depend on the existence of these
binaries then they really really shouldn't be deleted ever.
Here's a proposal, which I welcome comments on and will put into
practice next week if nobody disagrees with it.
* We add a bootstrap/stage3-sysroot.bst element, which is just
gnu-toolchain.bst with some splitting rules to remove debug symbols,
docs and locale.
* We set up a new ostree repo at http://ostree.baserock.org/releases
* We push builds of stage3-sysroot.bst for each platform, using the
following naming convention for the refs:
stage3-sysroot/x86_32
stage3-sysroot/x86_64
stage3-sysroot/armv8l64
* In base.bst we put the ref *and* the sha256 hash, so a given commit
of definitions will always pull the same binaries forever, even if a new
version is pushed to the ref `stage3-sysroot/x86_32`
And that's it. We could perhaps push new releases there every time
something changes in gnu-toolchain.bst; it doesn't actually matter
though because we do a whole 3-stage bootstrap from these binaries
anyway so they will very rarely need to change (I'd guess yearly updates
at most?).
I'm not actually sure how to do the push, the `bst push` command will
use the buildstream cache ID as the ref which isn't helpful here, so
that might need some modification to allow pushing to a different ref name.
And that's it. For those who haven't spent the past couple of months
thinking about baserock and buildstream this email might not fully make
sense, feel free to ask questions (although i might not get to them
until next week).
Thanks
Sam
1. for example, I once lost a week debugging random crashes that turned
out to be caused by an ABI break in GLIBC on armv7 -- the stage1-gcc
binary would work on systems with the newer GLIBC but fail on systems
with older GLIBC, but Baserock's cache key didn't include anything about
the host so there was no way to avoid the breakage...
--
Sam Thursfield, Codethink Ltd.
Office telephone: +44 161 236 5575
5 years, 4 months
- 1
- 2
- ← Newer
- 1
- Older →