[PATCH 0/4] Trove: fix upgrades of git.baserock.org
by Lars Wirzenius
repo: git://git.baserock.org/baserock/baserock/definitions
branch: baserock/liw/gbo-deployments
commit: f5d2a0da229d63f38b11bd4bff951ab28217575f
land: master
card: S11154
These patches attempt to make it possible to upgrade the
git.baserock.org Trove instance. gbo is different from other Troves in
that it's TROVE_ID (which determines things like the prefix for local
repositories, and local branches) and TROVE_HOSTNAME (the public
address by which the Trove is used) are different. These changes make
it possible to specify them separately, and also drop the setting of
the system hostname. Further, they add a morphology to deploying the
upgrades.
I have not yet actually done the upgrade to git.baserock.org, but I
have done them to a local, slightly futzed copy of it, and things seem
to work OK with these patches. There may be more to fix when we
attempt an upgrade of the real system.
Lars Wirzenius (4):
Stop setting system hostname to TROVE_ID
Add set-hostname to trove-system's config extensions
Allow TROVE_HOSTNAME to be set separately from TROVE_ID
Add a cluster morph for upgrading git.baserock.org
gbo-upgrade.morph | 25 +++++++++++++++++++++++++
trove-system-x86_64.morph | 1 +
trove.configure | 11 +++++++----
3 files changed, 33 insertions(+), 4 deletions(-)
create mode 100644 gbo-upgrade.morph
--
1.9.1
9 years, 6 months
[PATCH 0/3] GitLab as chunks
by Paul Sherwood
As discussed in other threads, this delivers GitLab server, shell and CI as
normal Baserock chunks. It is not ideal - bundler is still required to install
dependencies at first boot - but it's an improvement on the current solution.
My branch of this is at
http://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/definitions.gi...
Paul Sherwood (3):
add bundler v1.6.2
baserock has /bin/kill, not /usr/bin/kill
gitlab chunks and stratum
databases.morph | 3 ++-
.../systemd-units/gitlab-ci-runner.service | 4 ++--
gitlab-server.morph | 1 +
.../systemd-units/gitlab-ci-unicorn.service | 4 ++--
.../systemd-units/gitlab-unicorn.service | 4 ++--
gitlab-server/usr/share/gitlab-setup | 8 ++++----
gitlab.morph | 21 +++++++++++++++++++++
ruby.morph | 8 +++++++-
8 files changed, 41 insertions(+), 12 deletions(-)
create mode 100644 gitlab.morph
--
1.8.4
9 years, 6 months
sending patches to this list from inside baserock..
by Paul Sherwood
Hi
I remember JPohlmann helped me to frab this a long time ago, but I
forgot how, until google helped me out today...
For anyone interested, the trick in a devel machine would be to run
cpan
and from there install a couple of missing modules
install MIME::Base64
install Authen::SASL
exit
Then 'normal' git send-email instructions should work, once you've
configured your server etc.
Incidentally the default perl locale config in Baserock throws a lot of
warnings... anyone know if that is easily fixable?
br
Paul
9 years, 6 months
GitLab chunks
by Lars Wirzenius
Starting a new thread, so this doesn't get tangled in the review of
the gitlab backup automation patches.
On Tue, May 20, 2014 at 05:50:59PM +0100, Paul Sherwood wrote:
> I'm working on this - I have a (mostly) working gitlab stratum [1] and
> morph files for the gitlab components [2][3][4]
>
> Am I on the right track?
Without actually understanding how the GitLab installation happens...
hopefully yes. Do your git repositories contain all the gem
dependencies of GitLab? If not, how are those installed?
As for the actual morphologies:
* Always quote (with "") any use of $DESTDIR, since it may contain
shell special characters. (It hopefully never will, but we should be
defensive in our use of shell scripting.)
* It's better to use the install command to create directories and
install files than mkdir and cp. mkdir and cp obey the umask, and
install doesn't; for software installation, not obeying the umask is
better. Also, it's better to use "$PREFIX" instead of "/usr", in
case we need to change the prefix at any point.
However, you need to install a directory hierarchy, and the install
command isn't great for that. Thus, you should make sure you fix up
the permission bits after copying.
Thus:
mkdir -p "$DESTDIR/$PREFIX/gitlab-ce"
cp -r * "$DESTDIR/$PREFIX/gitlab-ce"
chmod -R a+rX,g+w "$DESTDIR/$PREFIX/gitlab-ce"
--
http://www.codethink.co.uk/ http://wiki.baserock.org/ http://www.baserock.com/
9 years, 6 months
[PATCH 0/4] Allow morph deploy to deploy individual systems from a cluster
by Adam Coldrick
Hi all,
As part of the work on upgrades, I have been looking at pain points of
morph deploy. One of these has been the inability to only deploy part
of a cluster morphology. If I have a cluster morphology containing a
number of systems and I only want to deploy one of them, I need to create
a new cluster morph or edit the one I already have. This is inconvenient.
This patch series addresses this problem by letting you run
morph deploy CLUSTER DEPLOYMENT...
to deploy one or more systems in the cluster. The old behaviour to deploy
all systems in the cluster is still present.
Repo: git://git.baserock.org/baserock/baserock/morph.git
Ref: baserock/adamcoldrick/deploy-specific-systems
SHA1: fa9c15d129834cc765cdec10397cdc119652181b
Target: master
Adam Coldrick (4):
Don't validate when loading all morphologies
Validate cluster morphologies
Allow the user to specify deployments in a cluster
Update the help text for morph deploy
morphlib/morphloader.py | 18 +++++++++++++++++-
morphlib/plugins/deploy_plugin.py | 33 ++++++++++++++++++++++-----------
morphlib/sysbranchdir.py | 4 +++-
3 files changed, 42 insertions(+), 13 deletions(-)
--
1.9.1
9 years, 6 months
[PATCH] Check vmpath before deploying
by Richard Ipsum
repo: git://git.baserock.org/baserock/baserock/morph
ref: baserock/richardipsum/improve_kvm_deploy
head: 986a45489a26fb971f5f5030824f6224b6b5ff2a
Hey,
This adds some checking of the vm path to the kvm check extension.
Richard Ipsum (1):
Check vmpath before deploying
morphlib/exts/kvm.check | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)
--
1.7.10.4
9 years, 6 months
[PATCH] lorry-controller: speed up /1.0/list-jobs-html query
by Lars Wirzenius
repo: git://git.baserock.org/baserock/baserock/lorry-controller
branch: baserock/liw/lc-list-all-jobs-optimisation
commit: f106bbe5c5aee8109521fa60387d54ac88e5f46d
land: master
card: 11152
Previously, it would take on the order of 20 kiloseconds to generate
the dynamic page that lists all jobs on git.baserock.org (about 87000
jobs). It now takes about 40 seconds, on my laptop using the g.b.o
Lorry Controller database, and should be significantly faster than
that on g.b.o. 40 seconds is still slow, but it'll work; 20
kiloseconds times out the HTTP request.
Lars Wirzenius (1):
Make listjobs faster by getting all info in one query
lorrycontroller/listjobs.py | 5 ++---
lorrycontroller/statedb.py | 20 ++++++++++++++++++++
2 files changed, 22 insertions(+), 3 deletions(-)
--
1.9.1
9 years, 6 months
[PATCH] Run checks earlier for ssh-rsync upgrades
by Sam Thursfield
Repository: git://git.baserock.org/baserock/baserock/morph
Ref: sam/ssh-rsync-check
Sha1: 7d849ca1db73c1544073acb6b7c88a33ec88be8d
It's better to raise errors as soon as possible.
Sam Thursfield (1):
deploy: Do sanity checks earlier in ssh-rsync (upgrade) extension
morphlib/exts/ssh-rsync.check | 24 ++++++++++++++++++++++++
morphlib/exts/ssh-rsync.write | 26 --------------------------
2 files changed, 24 insertions(+), 26 deletions(-)
--
1.8.5.3
9 years, 6 months
[RFC] Command line syntax of morph deploy
by Adam Coldrick
Hi all,
I am currently working on being able to deploy only part of a cluster
morphology with morph deploy. All the examples of this functionality I
have seen have been of the form
morph deploy CLUSTER SYSTEM
However, it strikes me that there are a number of use-cases where that
doesn't help For example, a distbuild cluster morphology may be as
follows:
name: distbuild
type: cluster
systems:
- morph: distbuild-system-x86_64
deploy-defaults:
...
deploy:
x86_64-controller:
...
x86_64-node-1:
...
x86_64-node-2:
...
- morph: distbuild-system-x86_32
deploy-defaults:
...
deploy:
x86_32-controller:
...
x86_32-node-1:
...
x86_32-node-2:
...
- morph: distbuild-system-armv7lhf-highbank
deploy-defaults:
...
deploy:
armv7lhf-controller:
...
armv7lhf-node-1:
...
armv7lhf-node-2:
...
Use cases
=========
(1): We want to deploy a full x86_64 distbuild network
------------------------------------------------------
In this case, we want to deploy all the deployments available to the
distbuild-system-x86_64 system, so the above method will work.
morph deploy distbuild distbuild-system-x86_64
(2): We want to deploy just an x86_64 distbuild controller
----------------------------------------------------------
In this case, we want to deploy only one of the deployments of a
particular system, so the above method won't work. We could instead use
the deployment name rather than the system name.
morph deploy distbuild x86_64-controller
However, when we want to deploy a whole distbuild network with this
method, we will need to do something like the following.
morph deploy distbuild x86_64-controller x86_64-node-1
x86_64-node-2
If there are a lot of deployments then this will be long (especially if
they are not sensibly named to allow the use of something like
`x86_64-*`).
(3): We want to deploy just an x86_64 distbuild controller, but the name
is not unique
------------------------------------------------------------------------
In this case, imagine all the controllers are just called `controller`.
In this case the method in (2) is ambiguous. We could make it an error
to have non-unique deployment names, but that seems like it duplicates
information to me (the architecture is defined in both the systems and
the deployments in the above cluster). We could then use both the system
name and the deployment name to clarify this.
morph deploy distbuild distbuild-system-x86_64:controller
However, it strikes me that this could get long. However, it would solve
the issue of deploying a whole network in (1).
morph deploy distbuild distbuild-system-x86_64:*
(4): We want to deploy controllers of all architectures
-------------------------------------------------------
The approach described in (3) could work very nicely here, so that we
will only need to do something like
morph deploy distbuild *:controller
if we name the deployments sensibly.
-----
What do people think is the correct approach regarding this syntax?
Personally I think we definitely can't get by with only using the system
name, since cases like distbuild will be completely unchanged in that
situation.
However with errors and sensible naming requirements we could work with
the method described in (2) to get something which suits all situations,
although I think (3) would be a solution that works with the least
amount of friction (wrt. naming things properly).
-Adam
9 years, 6 months
[PATCH] lorry-controller: remove broken links, buttons from static HTML status page
by Lars Wirzenius
repo: git://git.baserock.org/baserock/baserock/lorry-controller
branch: baserock/liw/lc-static-html-without-links
commit: f2ea06f8a0b0d9ba20756c169373d6db8569e370
land: master
card:
The publically visible, static HTML status page for Lorry Controller
currently has links and buttons that don't work, since they try to
access a non-public URL. This patch removes the links and buttons to
avoid confusion.
Lars Wirzenius (1):
Render static HTML page without links
lorrycontroller/status.py | 5 ++++-
templates/status.tpl | 15 ++++++++++++++-
2 files changed, 18 insertions(+), 2 deletions(-)
--
1.9.1
9 years, 6 months
