Proposal to use BuildStream
by Tristan Van Berkom
Hi all !
This is my formal proposal that Baserock start to use BuildStream for its
building purposes.
Some of you may have heard of this project, the rumors are true, BuildStream
is actually a thing :)
First some obligatory links:
o Announcement Blog: https://blogs.gnome.org/tvb/2017/02/06/introducing-buildstream-2/
o Project repository: https://gitlab.com/BuildStream/buildstream/
o Documentation: https://buildstream.gitlab.io/buildstream/
o Migration path: https://gitlab.com/BuildStream/defs2bst/
o IRC: irc.gnome.org #buildstream
o Project Roadmap: https://wiki.gnome.org/Projects/BuildStream/Roadmap
There is a ton of information in the above that I wont repeat here, but for an outline
of what BuildStream is in general, please refer to the announcement blog post (first link above).
If you want to give it a test run, here's what to do:
o Install ostree with introspection for python3, and install bubblewrap
o git clone git@gitlab.com:BuildStream/buildstream.git
o cd buildstream
o pip install --user -e .
o cd ..
o git clone git@gitlab.com:BuildStream/buildstream-tests.git
o cd buildstream-test
o git checkout build-gnome
o bst build gnome/gnome-system-image.bst
o bst checkout gnome/gnome-system-image.bst ~/output
o qemu-system-x86_64 -m size=1024 ~/output/sda.img
(VirtualBox is a bit nicer UX though)
o login as root and type `systemctl start gdm`
Some of the advantages you get right now over using YBD include:
o No host tools ever used in the build environment, this means
much better repeatability, systems can be built on any linux
system where BuildStream can be installed.
o Similar to the above, no host tools used even for deployment
of bootable images. This means no issues with upgrading syslinux
and the like which we have experienced before with Baserock.
o Better task scheduling and parallelism in the build. Fetching
git repositories is parallelized and depth sorted (sources you
are likely to need first are downloaded first).
o Nice UI with good process management from the scheduler. Interactive
mode lets you drop into a shell and debug a failed build while
ongoing builds are temporarily suspended, options to continue
processing ongoing builds or terminate them etc.
o Built in branch tracking features, `bst track element.bst` allows
BuildStream to update an elements commit sha based on it's tracking
branch and rewrite the project inline.
o You may have flashbacks from last years "Axing the Stratum"[0] email
thread which made some waves back then. Pretty much all that was
discussed in that thread is implemented in BuildStream from the
start.
Disadvantages you have right now using BuildStream instead of YBD:
o Artifact cache sharing needs to be implemented, it's next on
our roadmap and coming soon to a build server near you.
Longer term, BuildStream is geared much more to flexibility and
convenience of the user, which is to say the developer. Not only is
it suitable to run long tasks on a build server but it should be
usable for a regular developer on their desktop or even laptop.
When I say usable here, I mean:
o Allow overriding a source repository with a local build source
directory, allowing tight developer loops without having to
jump through many hoops just to get your printf() debugging
statement into a library on the system you want to test in
a VM.
o Dual cache key calculation modes are coming too, which means
in a weak cache key calculation scenario, you should be able
to change a line in glibc and redeploy a VM for testing
immediately, without rebuilding the 500 modules which depend
on it (intended for developers and testing, strictly
deterministic builds are the default).
Hopefully this will spark some interesting discussion and debate, BuildStream
was written with Baserock in mind as a potential use case so I am very hopeful
that you will consider my proposal.
Best Regards,
-Tristan
[0]: Axing the Stratum
https://listmaster.pepperfish.net/pipermail/baserock-dev-baserock.org/201...
6 years, 2 months
- 3
- 8
- ← Newer
- 1
- Older →