One of my goals with the GNOME baserock system is to achieve some
continuous integration setup for the GNOME system, hopefully generating
some automated deployment of the latest bleeding edge GNOME image and
making these images available for people to easily test the latest
To this end, I'd like to get the opinion of those working on CIAT and
try to assess how far we are from achieving this in some form.
Below, I've listed some of the steps I imagine will be involved in the
CI process, I would be grateful if those working on CIAT could both
clarify the status of how far we are with these and also correct me and
explain how this is intended to work from your own point of view.
Pulling latest git revisions from upstreams
From what I understand, there is already a system in place to check
upstreams for a given "chunk" and modify a local definitions checkout
as to point to the latest version.
a.) Is this already solved ?
b.) Does it only work for master, or is there some way - perhaps based
on a chunk's 'unpetrify-ref'; to consume the latest revision of
the active branch desired for the given chunk ?
We want the images to actually boot and work, so I think 'b' is
critical, currently definitions has master branches here and stable
branches there, a sort of melting pot that needs some cleanup but at
least with the integration work I've been doing; we will be starting
with a baseline that is known to successfully boot in to GDM, launch
gnome-shell and have applications and desktop applets that generally
Handling git submodules
Currently we have a hand full of downstream deltas in the trove to
modify the .gitmodules files, obviously this is not practical and might
be a stumbling block for CI.
In any case, downstream deltas need to be considered irregardless of the
Do we plan on addressing the submodule situation for the purpose of CI,
is there already an approach for this in place ?
I could imagine a couple of approaches... as the lorries repository
already serves as a database to identify and swap a submodule for an
existing clone in the trove this seems quite solvable.
a.) Possibly we patch the submodules after checkout in ybd's
build phase directly
b.) Possibly we invoke git in a special way (not exactly sure
how)... so that we can override the method to clone a submodule
on git's behalf; fetching the matching trove module instead.
Applying our downstream deltas
Asides from submodules, there will surely always be some cases where a
downstream patch is required in order to achieve a fully functional
One which I encountered recently while integrating libpwquality:
Currently we build with --disable-nls and avoid the patch, but this
disables the translations; resulting in untranslated strings such as
"This password is too weak", which can be embarrassing when booting up a
GNOME image for the first time, after choosing your preferred language
and proceeding to set a password for the new user in gnome-initial-setup
), which I'm trying to
integrate for my next milestone (almost there).
Considering that the CI must operate on a separate repository (or
branch) of the definitions repository, and that it is not an option to
in a way that breaks our upstream mainline
definitions builds, I am curious as to how we will handle this.
A.) Will the CI use an entire duplicate trove (expensive) ?
B.) Will the CI be creating some special branches in
repositories for the purpose of trying and
testing a build ?
In either case, the goal is of course to rebase our downstream patches
against the head of an upstream repository *without* effecting the
results of a regular build (CI builds should remain encapsulated and
running them should not compromise our production builds).
Consuming definitions changes from CIAT and upgrading definitions
Once CIAT has successfully built and tested a given build, this build
should be marked somehow as ready/passed so that baserock maintainers
have some way of integrating the results into our upstream production
I think the idea thus far is to have CIAT submit patches to the
definitions repository over gerrit.
Anyone care to clarify and elaborate on what exactly the plan is for
this on the mailing list ?
Configuration(s) of CIAT's CI instances
One thing I think we're going to need from day one in order to produce
booting and well running GNOME images, is a way to configure our GNOME
CI instance so that we can select which "stratum" are eligible for CI.
In other words, it would also be fun to try to build every single thing
from master every time - but I would prefer to start with a more
conservative approach and only pull new chunk revisions for specific
stratum (i.e. lets build master of all gnome.morph modules against a
stable system and have more useful images as a result).
NOTE: I did try for example, to upgrade the X server and all X libraries
in one shot when upgrading libXi a couple weeks ago, however I was
unable to get the new X server to start, I'm not sure why; but I would
rather start with automatically generating something that mostly always
works for the purpose of GNOME image testing.
In the current setup thus far, is there any mechanism in place to
configure a CI instance to pull new revisions of modules selectively,
such as "only update gnome.morph chunks" ?
If not, can we please have this ?
Once this is all up and running, we will want to provide an easy way for
people in general to download a "GNOME nightly" image to try it out on
any given day.
Do we have a plan on how publishing these will work ?
Just have the CI instance make the successfully built images available
in a shared directory somewhere the public can download from ?