Hi all, One of my goals with the GNOME baserock system is to achieve some continuous integration setup for the GNOME system, hopefully generating some automated deployment of the latest bleeding edge GNOME image and making these images available for people to easily test the latest GNOME. To this end, I'd like to get the opinion of those working on CIAT and try to assess how far we are from achieving this in some form. Below, I've listed some of the steps I imagine will be involved in the CI process, I would be grateful if those working on CIAT could both clarify the status of how far we are with these and also correct me and explain how this is intended to work from your own point of view. Best Regards, -Tristan Pulling latest git revisions from upstreams ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ upstreams for a given "chunk" and modify a local definitions checkout so as to point to the latest version. a.) Is this already solved ? b.) Does it only work for master, or is there some way - perhaps based on a chunk's 'unpetrify-ref'; to consume the latest revision of the active branch desired for the given chunk ? We want the images to actually boot and work, so I think 'b' is critical, currently definitions has master branches here and stable branches there, a sort of melting pot that needs some cleanup but at least with the integration work I've been doing; we will be starting with a baseline that is known to successfully boot in to GDM, launch gnome-shell and have applications and desktop applets that generally work. Handling git submodules ~~~~~~~~~~~~~~~~~~~~~~~ Currently we have a hand full of downstream deltas in the trove to modify the .gitmodules files, obviously this is not practical and might be a stumbling block for CI. In any case, downstream deltas need to be considered irregardless of the submodule situation. Do we plan on addressing the submodule situation for the purpose of CI, is there already an approach for this in place ? I could imagine a couple of approaches... as the lorries repository already serves as a database to identify and swap a submodule for an existing clone in the trove this seems quite solvable. a.) Possibly we patch the submodules after checkout in ybd's build phase directly b.) Possibly we invoke git in a special way (not exactly sure how)... so that we can override the method to clone a submodule on git's behalf; fetching the matching trove module instead. Applying our downstream deltas ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Asides from submodules, there will surely always be some cases where a downstream patch is required in order to achieve a fully functional build. One which I encountered recently while integrating libpwquality: https://git.gnome.org/browse/jhbuild/tree/patches/libpwquality.libintl-li... Currently we build with --disable-nls and avoid the patch, but this disables the translations; resulting in untranslated strings such as "This password is too weak", which can be embarrassing when booting up a GNOME image for the first time, after choosing your preferred language and proceeding to set a password for the new user in gnome-initial-setup (https://wiki.gnome.org/Design/OS/InitialSetup), which I'm trying to integrate for my next milestone (almost there). Considering that the CI must operate on a separate repository (or branch) of the definitions repository, and that it is not an option to change git.baserock.org/delta in a way that breaks our upstream mainline definitions builds, I am curious as to how we will handle this. A.) Will the CI use an entire duplicate trove (expensive) ? B.) Will the CI be creating some special branches in git.baserock.org/delta repositories for the purpose of trying and testing a build ? In either case, the goal is of course to rebase our downstream patches against the head of an upstream repository *without* effecting the results of a regular build (CI builds should remain encapsulated and running them should not compromise our production builds). Consuming definitions changes from CIAT and upgrading definitions ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Once CIAT has successfully built and tested a given build, this build should be marked somehow as ready/passed so that baserock maintainers have some way of integrating the results into our upstream production definitions repository/branches. I think the idea thus far is to have CIAT submit patches to the definitions repository over gerrit. Anyone care to clarify and elaborate on what exactly the plan is for this on the mailing list ? Configuration(s) of CIAT's CI instances ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ One thing I think we're going to need from day one in order to produce booting and well running GNOME images, is a way to configure our GNOME CI instance so that we can select which "stratum" are eligible for CI. In other words, it would also be fun to try to build every single thing from master every time - but I would prefer to start with a more conservative approach and only pull new chunk revisions for specific stratum (i.e. lets build master of all gnome.morph modules against a stable system and have more useful images as a result). NOTE: I did try for example, to upgrade the X server and all X libraries in one shot when upgrading libXi a couple weeks ago, however I was unable to get the new X server to start, I'm not sure why; but I would rather start with automatically generating something that mostly always works for the purpose of GNOME image testing. In the current setup thus far, is there any mechanism in place to configure a CI instance to pull new revisions of modules selectively, such as "only update gnome.morph chunks" ? If not, can we please have this ? Publishing ~~~~~~~~~~ Once this is all up and running, we will want to provide an easy way for people in general to download a "GNOME nightly" image to try it out on any given day. Do we have a plan on how publishing these will work ? Just have the CI instance make the successfully built images available in a shared directory somewhere the public can download from ?