These patches attempt to make it possible to upgrade the
git.baserock.org Trove instance. gbo is different from other Troves in
that it's TROVE_ID (which determines things like the prefix for local
repositories, and local branches) and TROVE_HOSTNAME (the public
address by which the Trove is used) are different. These changes make
it possible to specify them separately, and also drop the setting of
the system hostname. Further, they add a morphology to deploying the
I have not yet actually done the upgrade to git.baserock.org, but I
have done them to a local, slightly futzed copy of it, and things seem
to work OK with these patches. There may be more to fix when we
attempt an upgrade of the real system.
Lars Wirzenius (4):
Stop setting system hostname to TROVE_ID
Add set-hostname to trove-system's config extensions
Allow TROVE_HOSTNAME to be set separately from TROVE_ID
Add a cluster morph for upgrading git.baserock.org
gbo-upgrade.morph | 25 +++++++++++++++++++++++++
trove-system-x86_64.morph | 1 +
trove.configure | 11 +++++++----
3 files changed, 33 insertions(+), 4 deletions(-)
create mode 100644 gbo-upgrade.morph
As discussed in other threads, this delivers GitLab server, shell and CI as
normal Baserock chunks. It is not ideal - bundler is still required to install
dependencies at first boot - but it's an improvement on the current solution.
My branch of this is at
Paul Sherwood (3):
add bundler v1.6.2
baserock has /bin/kill, not /usr/bin/kill
gitlab chunks and stratum
databases.morph | 3 ++-
.../systemd-units/gitlab-ci-runner.service | 4 ++--
gitlab-server.morph | 1 +
.../systemd-units/gitlab-ci-unicorn.service | 4 ++--
.../systemd-units/gitlab-unicorn.service | 4 ++--
gitlab-server/usr/share/gitlab-setup | 8 ++++----
gitlab.morph | 21 +++++++++++++++++++++
ruby.morph | 8 +++++++-
8 files changed, 41 insertions(+), 12 deletions(-)
create mode 100644 gitlab.morph
I remember JPohlmann helped me to frab this a long time ago, but I
forgot how, until google helped me out today...
For anyone interested, the trick in a devel machine would be to run
and from there install a couple of missing modules
Then 'normal' git send-email instructions should work, once you've
configured your server etc.
Incidentally the default perl locale config in Baserock throws a lot of
warnings... anyone know if that is easily fixable?
Starting a new thread, so this doesn't get tangled in the review of
the gitlab backup automation patches.
On Tue, May 20, 2014 at 05:50:59PM +0100, Paul Sherwood wrote:
> I'm working on this - I have a (mostly) working gitlab stratum  and
> morph files for the gitlab components 
> Am I on the right track?
Without actually understanding how the GitLab installation happens...
hopefully yes. Do your git repositories contain all the gem
dependencies of GitLab? If not, how are those installed?
As for the actual morphologies:
* Always quote (with "") any use of $DESTDIR, since it may contain
shell special characters. (It hopefully never will, but we should be
defensive in our use of shell scripting.)
* It's better to use the install command to create directories and
install files than mkdir and cp. mkdir and cp obey the umask, and
install doesn't; for software installation, not obeying the umask is
better. Also, it's better to use "$PREFIX" instead of "/usr", in
case we need to change the prefix at any point.
However, you need to install a directory hierarchy, and the install
command isn't great for that. Thus, you should make sure you fix up
the permission bits after copying.
mkdir -p "$DESTDIR/$PREFIX/gitlab-ce"
cp -r * "$DESTDIR/$PREFIX/gitlab-ce"
chmod -R a+rX,g+w "$DESTDIR/$PREFIX/gitlab-ce"
As part of the work on upgrades, I have been looking at pain points of
morph deploy. One of these has been the inability to only deploy part
of a cluster morphology. If I have a cluster morphology containing a
number of systems and I only want to deploy one of them, I need to create
a new cluster morph or edit the one I already have. This is inconvenient.
This patch series addresses this problem by letting you run
morph deploy CLUSTER DEPLOYMENT...
to deploy one or more systems in the cluster. The old behaviour to deploy
all systems in the cluster is still present.
Adam Coldrick (4):
Don't validate when loading all morphologies
Validate cluster morphologies
Allow the user to specify deployments in a cluster
Update the help text for morph deploy
morphlib/morphloader.py | 18 +++++++++++++++++-
morphlib/plugins/deploy_plugin.py | 33 ++++++++++++++++++++++-----------
morphlib/sysbranchdir.py | 4 +++-
3 files changed, 42 insertions(+), 13 deletions(-)
This adds some checking of the vm path to the kvm check extension.
Richard Ipsum (1):
Check vmpath before deploying
morphlib/exts/kvm.check | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)
Previously, it would take on the order of 20 kiloseconds to generate
the dynamic page that lists all jobs on git.baserock.org (about 87000
jobs). It now takes about 40 seconds, on my laptop using the g.b.o
Lorry Controller database, and should be significantly faster than
that on g.b.o. 40 seconds is still slow, but it'll work; 20
kiloseconds times out the HTTP request.
Lars Wirzenius (1):
Make listjobs faster by getting all info in one query
lorrycontroller/listjobs.py | 5 ++---
lorrycontroller/statedb.py | 20 ++++++++++++++++++++
2 files changed, 22 insertions(+), 3 deletions(-)
I am currently working on being able to deploy only part of a cluster
morphology with morph deploy. All the examples of this functionality I
have seen have been of the form
morph deploy CLUSTER SYSTEM
However, it strikes me that there are a number of use-cases where that
doesn't help For example, a distbuild cluster morphology may be as
- morph: distbuild-system-x86_64
- morph: distbuild-system-x86_32
- morph: distbuild-system-armv7lhf-highbank
(1): We want to deploy a full x86_64 distbuild network
In this case, we want to deploy all the deployments available to the
distbuild-system-x86_64 system, so the above method will work.
morph deploy distbuild distbuild-system-x86_64
(2): We want to deploy just an x86_64 distbuild controller
In this case, we want to deploy only one of the deployments of a
particular system, so the above method won't work. We could instead use
the deployment name rather than the system name.
morph deploy distbuild x86_64-controller
However, when we want to deploy a whole distbuild network with this
method, we will need to do something like the following.
morph deploy distbuild x86_64-controller x86_64-node-1
If there are a lot of deployments then this will be long (especially if
they are not sensibly named to allow the use of something like
(3): We want to deploy just an x86_64 distbuild controller, but the name
is not unique
In this case, imagine all the controllers are just called `controller`.
In this case the method in (2) is ambiguous. We could make it an error
to have non-unique deployment names, but that seems like it duplicates
information to me (the architecture is defined in both the systems and
the deployments in the above cluster). We could then use both the system
name and the deployment name to clarify this.
morph deploy distbuild distbuild-system-x86_64:controller
However, it strikes me that this could get long. However, it would solve
the issue of deploying a whole network in (1).
morph deploy distbuild distbuild-system-x86_64:*
(4): We want to deploy controllers of all architectures
The approach described in (3) could work very nicely here, so that we
will only need to do something like
morph deploy distbuild *:controller
if we name the deployments sensibly.
What do people think is the correct approach regarding this syntax?
Personally I think we definitely can't get by with only using the system
name, since cases like distbuild will be completely unchanged in that
However with errors and sensible naming requirements we could work with
the method described in (2) to get something which suits all situations,
although I think (3) would be a solution that works with the least
amount of friction (wrt. naming things properly).
The publically visible, static HTML status page for Lorry Controller
currently has links and buttons that don't work, since they try to
access a non-public URL. This patch removes the links and buttons to
Lars Wirzenius (1):
Render static HTML page without links
lorrycontroller/status.py | 5 ++++-
templates/status.tpl | 15 ++++++++++++++-
2 files changed, 18 insertions(+), 2 deletions(-)