4:19 p.m.
Hi all!
After my first RFC email about Trove configuration [0], many things (problems
and limitations) has happened in my development which have made ​​me change my
model.
Following my previous model
---------------------------
In my last model I explained my desire to remove all the configuration done in
the trove.configure extension. That would be possible creating a new systemd
unit to run before trove-early-setup, to configure the system in the same way
trove.configure did.
This new unit that I called trove-early-configure, would read the configuration
needed from a file (e.g. /baserock/trove/trove.conf), and then configure the
trove to leave everything ready for trove-early-setup. Some of this
configuration implied the modification of the trove-early-setup unit.
After this , I came to some conclusions:
1 - Create a different unit is undesirable. Is not clear what have to be done
in the different units.
2 - Performing substitutions on installed unit files in running systems is not
a good idea.
3- Modify files in /usr or in /baserock is not a good decision, I should treat
everything outside /etc and /var as read-only
This problems seemed easy to fix, so I started working in how to change the
model to make it fit with the previous conclusions
Le unit: One unit to configure Trove
------------------------------------
After all the comments I had in the email, I decided that the configuration
needed to configure a Trove should be all together in one script, being
executed from one unit only once. Doing this I would fix the problem mentioned
in the conclusion No 1.
Also I made an effort to modify the trove-early-setup script to include all the
configuration needed in the Trove (that is, the configuration previously done
in trove.configure and trove-early-setup). Now the script would read the
configuration from /etc/trove/trove.conf, which means now it doesn't need to
perform substitutions in other units, avoiding the problem explained in the
conclusion No 2.
And, to finish the implementation of this model and fix the problem described
in the conclusion No 3, I moved the files I was modifying from /usr to /etc.
- Some files were from gitano, and upgrading gitano to the latest version
allowed me to move its skeleton files to /etc.
- Others files are scripts for migrations located in /home/shares. It must be
easy to move them to /etc.
But WAIT, doesn't it mean that we were already installing more than one unit to
configure a Trove? And also, will the upgrades work with this model?
Migrations and "why my model is wrong?"
---------------------------------------
I have to say I forgot to think about the migrations scripts, and about how
would my model handle them. So after this thought came to my mind, I decided to
check what would do my model if I try to upgrade an old Trove.
If, for example, I try to upgrade a Trove with the old lorry controller, the
configuration needed for the new lorry controller which are now generated in
trove.configure extension, will be generated (in my model) in trove-early-setup
(first boot), but this script won't be called, because the Trove is only
configured once, so I should put this specific configuration in a migration script.
* But, shouldn't I do the same (create a migration script and unit) for all the
changes we have done in trove.configure since it was created?
Yes I should.
* And doesn't it mean that we will have to create a new syetemd unit and a new
script for every further changes in the trove configuration?
Yes it does.
* And is it possible that we will end up with some migration scripts depending
in other migration scripts making the creation of new migration scripts complex?
Yes it is.
This conclusions made me realize that my new model make things more complex
than before. So I decided to rethink the model.
Rethinking the model
--------------------
If we don't want to end up with a migration script for every change we do in
the configuration needed for Trove, we have some options:
1 - Following the model I've described in "Le Unit:...", if I move all the
configuration to the trove-early-setup script, and I want to add also the
migrations scripts into it, then trove-early-setup must run every boot, and be
clever enough to configure just the things that it needs to configure.
2 - Divide all the configuration needed in scripts, every script makes an
indivisible part of the configuration. A single systemd unit can call them, and
they have to run on the right order, and without breaking things if they run
more than once.
3 - Implement a Baserock configuration manager or integrate an existing one.
In my ideal world of configuration, we should be able to:
- Specify all the configuration in one place.
- Add more configuration scripts whenever needed and be sure that the new
script will run after a fresh deployment and after an upgrade.
Declarative configuration
-------------------------
When I was in the "Rethinking the model" phase, somebody suggested me to take a
look at "Declarative configuration". And I found it pretty interesting. If we
integrate a configuration manager in Baserock we can change the Trove
configuration model to a simpler way.
After researching about one concrete declarative configuration management tool
(Ansible [1]). I can say that using it we can:
- create simple pieces of configuration (scripts) to configure the systems.
- create dependencies between the pieces of configuration.
- it has facilities to create idempotent scripts.
This is really close to my "ideal world", if is not exactly what I was looking
for from the beginning. I would like to hear opinions about this, and see if
including Ansible as a configuration management tool in some of our systems
makes sense. Meanwhile I will create a Trove with Ansible integrated and
working as a proof of concept.
Pedro.
[0]:
http://listmaster.pepperfish.net/pipermail/baserock-dev-baserock.org/2014...
[1]: http://docs.ansible.com/
7:01 p.m.
New subject: [PATCH 0/5] Use the trove-setup implementation.
Repo: baserock:baserock/definitions
Branch: baserock/pedroalvarez/ansible-trove-rebased
Sha1: 28f10a622fb4d31f090cf5b895e8b525c1560eb1
Landing: master
The new trove-setup implementation requires Ansible, and Ansible requires
shadow for one of the modules that trove-setup uses.
This patch series adds shadow to the core stratum, an Ansible stratum, and
changes trove.configure to fit with the new trove-setup implementation.
It also updates gitano and lorry-controller, needed also for the new
implementation of trove-setup.
Pedro Alvarez (5):
Add shadow to core stratum
Add ansible stratum
Add ansible to the trove
Use new version of trove-setup, gitano and lorry-controller.
New trove.configure
ansible.morph | 29 +++++
core.morph | 10 ++
trove-backup.configure | 5 ----------
trove-system-x86_64.morph | 2 +-
trove.configure | 256 +++++++--------------------------------------
trove.morph | 6 +-
6 files changed, 83 insertions(+), 275 deletions(-)
create mode 100644 ansible.morph
delete mode 100755 trove-backup.configure
--
1.7.10.4
7:01 p.m.
New subject: [PATCH 1/5] Add shadow to core stratum
---
core.morph | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/core.morph b/core.morph
index cbe3348..16d874a 100644
--- a/core.morph
+++ b/core.morph
@@ -240,3 +240,13 @@ chunks:
unpetrify-ref: baserock/morph
build-depends:
- python-setuptools
+- name: shadow
+ repo: upstream:shadow
+ ref: 4f5000a45963c2cc2a403ad23e459f20296b29c2
+ unpetrify-ref: baserock/4.2
+ build-depends:
+ - autoconf
+ - automake
+ - gettext
+ - libtool
+ - bison
--
1.7.10.4
7:01 p.m.
New subject: [PATCH 2/5] Add ansible stratum
---
ansible.morph | 29 +++++++++++++++++++++++++++++
1 file changed, 29 insertions(+)
create mode 100644 ansible.morph
diff --git a/ansible.morph b/ansible.morph
new file mode 100644
index 0000000..00f0475
--- /dev/null
+++ b/ansible.morph
@@ -0,0 +1,29 @@
+name: ansible
+kind: stratum
+description: A stratum with ansible and its dependencies
+build-depends:
+- morph: core
+chunks:
+- name: paramiko
+ repo: upstream:paramiko
+ ref: 951faed80b017e553a27c4cb98f210df44341f8f
+ unpetrify-ref: baserock/morph
+ build-depends: []
+- name: markupsafe
+ repo: upstream:markupsafe
+ ref: 58cde05bdcb0a53d87213b4a5bb605937f178171
+ unpetrify-ref: baserock/morph
+ build-depends: []
+- name: jinja2
+ repo: upstream:jinja2
+ ref: 91fa138077d9ed5cf73a7903479077498e695492
+ unpetrify-ref: baserock/morph
+ build-depends:
+ - markupsafe
+- name: ansible
+ repo: upstream:ansible
+ ref: aa56db7e28d4fe256471043b05120c2f41a840e5
+ unpetrify-ref: baserock/morph
+ build-depends:
+ - paramiko
+ - jinja2
--
1.7.10.4
7:01 p.m.
New subject: [PATCH 3/5] Add ansible to the trove
---
trove-system-x86_64.morph | 1 +
1 file changed, 1 insertion(+)
diff --git a/trove-system-x86_64.morph b/trove-system-x86_64.morph
index d599d4b..0ad1153 100644
--- a/trove-system-x86_64.morph
+++ b/trove-system-x86_64.morph
@@ -18,3 +18,4 @@ strata:
- morph: tools
- morph: trove
- morph: nfs
+- morph: ansible
--
1.7.10.4
7:01 p.m.
New subject: [PATCH 4/5] Use new version of trove-setup, gitano and lorry-controller.
trove-setup:
- Use ansible to configure the trove.
lorry-controller:
- Changes needed for the new trove-setup. Now lorry-controller
does't enable its units when installing.
gitano:
- Updated to a new version needed for the new trove-setup.
With this new version is possible to change the path
of the skeleton of gitano.
update gitano to change paths of skel
---
trove.morph | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/trove.morph b/trove.morph
index cd35092..a77c22e 100644
--- a/trove.morph
+++ b/trove.morph
@@ -65,7 +65,7 @@ chunks:
- lua
- name: gitano
repo: upstream:gitano/gitano
- ref: d5a76a5caf51d12c811317ac6e376942b7633770
+ ref: dadb8086d7b47510a4cc23541778fe5a91c71e96
unpetrify-ref: baserock/morph
build-depends:
- lua
@@ -178,12 +178,12 @@ chunks:
- hg-fast-export
- name: trove-setup
repo: baserock:baserock/trove-setup
- ref: eafba37e2bfc3897e3e7f65f2ce087fbee358f43
+ ref: 92aa3046237778273271fbe64fd9ca7b58b0a533
unpetrify-ref: master
build-depends: []
- name: lorry-controller
repo: baserock:baserock/lorry-controller
- ref: 33403e1ca0b33fc12e626de2752d56bcd65dd913
+ ref: 0a2c76765d044ab85d3c988a76b04ce14f5f2b3b
unpetrify-ref: master
build-depends: []
- name: lighttpd
--
1.7.10.4
7:01 p.m.
New subject: [PATCH 5/5] New trove.configure
---
trove-backup.configure | 55 ----------
trove-system-x86_64.morph | 1 -
trove.configure | 256 +++++++--------------------------------------
3 files changed, 40 insertions(+), 272 deletions(-)
delete mode 100755 trove-backup.configure
diff --git a/trove-backup.configure b/trove-backup.configure
deleted file mode 100755
index 59d9072..0000000
--- a/trove-backup.configure
+++ /dev/null
@@ -1,55 +0,0 @@
-#!/bin/sh
-#
-# Copyright (C) 2013 Codethink Limited
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; version 2 of the License.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-#
-#
-# This is a "morph deploy" configuration extension to set up the Trove with a
-# backup user that can be accessed with rsync.
-# It takes one environment variable:
-#
-# TROVE_BACKUP_KEYS - a space-separated list of paths to SSH keys.
-
-set -e
-
-ROOT="$1"
-BACKUP_HOME=/root/backup-user-home
-
-##########################################################################
-
-if [ -n "$TROVE_BACKUP_KEYS" ]; then
- cat >"$1/etc/rsyncd.conf" <<EOF
-numeric ids = yes
-uid = 0
-gid = 0
-read only = yes
-
-[etc]
-path = /etc
-comment = System configuration
-
-[home]
-path = /home
-comment = Home directories
-EOF
-
- echo "backup:x:0:0::$BACKUP_HOME:/bin/sh" >>"$1/etc/passwd"
- mkdir -p "$1/$BACKUP_HOME/.ssh"
-
- touch "$1/$BACKUP_HOME/.ssh/authorized_keys"
- for key in $TROVE_BACKUP_KEYS; do
- cat "$key" >> "$1/$BACKUP_HOME/.ssh/authorized_keys"
- done
-fi
diff --git a/trove-system-x86_64.morph b/trove-system-x86_64.morph
index 0ad1153..e65b054 100644
--- a/trove-system-x86_64.morph
+++ b/trove-system-x86_64.morph
@@ -3,7 +3,6 @@ configuration-extensions:
- set-hostname
- trove
- nfsboot-server
-- trove-backup
- fstab
- simple-network
- install-files
diff --git a/trove.configure b/trove.configure
index 840f15f..c5d25df 100755
--- a/trove.configure
+++ b/trove.configure
@@ -41,230 +41,54 @@ set -e
ROOT="$1"
-if [ -z "$TROVE_HOSTNAME" ]
-then
- export TROVE_HOSTNAME="$TROVE_ID"
-fi
-
-##########################################################################
-# Configuration in /etc, which we need to do on all deployments.
-##########################################################################
-
-##########################################################################
-lua_escape()
-{
- echo -n "$1" | perl -pe 's/([-+\(\).%*?^$\[\]])/%$1/g'
-}
+TROVE_DATA="$ROOT/etc/trove"
+install -d "$TROVE_DATA"
+
+install -d "$TROVE_DATA"
+install -m 0600 "$LORRY_SSH_KEY" "$TROVE_DATA/lorry.key"
+install -m 0644 "${LORRY_SSH_KEY}.pub" "$TROVE_DATA/lorry.key.pub"
+install -m 0644 "$TROVE_ADMIN_SSH_PUBKEY"
"$TROVE_DATA/admin.key.pub"
+install -m 0644 "$WORKER_SSH_PUBKEY" "$TROVE_DATA/worker.key.pub"
+install -m 0644 "$MASON_SSH_PUBKEY" "$TROVE_DATA/mason.key.pub"
+
+cat <<EOF > "$TROVE_DATA/trove.conf"
+TROVE_ID: $TROVE_ID
+MASON_ID: $MASON_ID
+TROVE_COMPANY: $TROVE_COMPANY
+UPSTREAM_TROVE: $UPSTREAM_TROVE
+TROVE_ADMIN_USER: $TROVE_ADMIN_USER
+TROVE_ADMIN_EMAIL: $TROVE_ADMIN_EMAIL
+TROVE_ADMIN_NAME: $TROVE_ADMIN_NAME
+
+LORRY_SSH_KEY: /etc/trove/lorry.key
+LORRY_SSH_PUBKEY: /etc/trove/lorry.key.pub
+TROVE_ADMIN_SSH_PUBKEY: /etc/trove/admin.key.pub
+WORKER_SSH_PUBKEY: /etc/trove/worker.key.pub
+MASON_SSH_PUBKEY: /etc/trove/mason.key.pub
-echo "Creating /etc/trove-setup.sed"
-
-cat <<EOF > "$ROOT"/etc/trove-setup.sed
-s/##TROVE_HOSTNAME##/$TROVE_HOSTNAME/g
-s/##MASON_HOST##/$MASON_ID/g
-s/##MASON_PORT##/18755/g
-s/##TROVE_TITLE##/$TROVE_ID/g
-s/##TROVE_COMPANY##/$TROVE_COMPANY/g
-s/##TROVE_LOG_PREFIX##/$TROVE_ID/g
-s/##ESC_PERSONAL_PREFIX##/people/g
-s/##PREFIX##/$TROVE_ID/g
-s/##UPSTREAM_TROVE##/$UPSTREAM_TROVE/g
-## The same prefix as above, only lua-pattern-escaped
-s/##ESC_PREFIX##/$(lua_escape "$TROVE_ID")/g
EOF
-##########################################################################
-
-echo "Performing substitutions in /etc"
-
-sed -f "$ROOT"/etc/trove-setup.sed -i \
- "$ROOT"/etc/cgitrc \
- "$ROOT"/etc/gitano-setup.clod \
- "$ROOT"/etc/lorry.conf \
- "$ROOT"/usr/share/gitano/skel/gitano-admin/*/*.lace \
- "$ROOT"/usr/share/gitano/skel/gitano-admin/*/*.lua \
- "$ROOT"/usr/share/gitano/skel/gitano-admin/users/*/user.conf \
- "$ROOT"/usr/share/trove-setup/releases-repo-migration.sh \
- "$ROOT"/usr/share/trove-setup/releases-repo-README \
- "$ROOT"/usr/lib/systemd/system/releases-repo-migration.service
-
-##########################################################################
-
-# trove-early-setup needs "localhost" to be defined, and there's no
-# guarantee it's going to be in DNS, or that external networking is
-# up when trove-early-setup runs. We work around this by creating
-# /etc/hosts with the right line.
-echo "Add localhost to /etc/hosts"
-cat <<EOF >> "$ROOT/etc/hosts"
-127.0.0.1 localhost
-EOF
-
-##########################################################################
-
-# create a symlink in /var/www/htdocs to what will be the rsync area of
-# the releases repository
-echo "Symlink rsync releases in htdocs"
-ln -s "/home/git/repos/$TROVE_ID/site/releases.git/rsync" \
- "$ROOT/var/www/htdocs/releases"
-
-
-##########################################################################
-
-echo "Create Lorry Controller config"
-install -d "$ROOT/etc/lorry-controller"
-cat <<EOF > "$ROOT/etc/lorry-controller/webapp.conf"
-[config]
-log = /home/lorry/webapp.log
-log-max = 100M
-log-keep = 10
-log-level = debug
-statedb = /home/lorry/webapp.db
-configuration-directory = /home/lorry/confgit
-status-html = /home/lorry/lc-status.html
-wsgi = yes
-debug-port = 12765
-templates = /usr/share/lorry-controller/templates
-confgit-url = ssh://git@localhost/$TROVE_ID/local-config/lorries
-EOF
-
-
-echo "Create MINION config"
-cat <<EOF > "$ROOT/etc/lorry-controller/minion.conf"
-[config]
-log = syslog
-log-level = debug
-webapp-host = localhost
-webapp-port = 12765
-webapp-timeout = 3600
-EOF
-
-
-echo "Set up Lorry Controller MINIONs"
-UNITS="$ROOT/usr/lib/systemd/system"
-seq "${LORRY_CONTROLLER_MINIONS:-4}" |
-while read i
-do
- ln -s "../lorry-controller-minion@.service" \
- "$UNITS/multi-user.target.wants/lorry-controller-minion(a)$i.service"
-done
-
-
-
-##########################################################################
-# Configuration of trove-early-setup
-#
-# We configure trove-early-setup so that it runs at first boot of an initial
-# deployment, to do the parts of Trove system setup that require running
-# commands from the deployed system.
-##########################################################################
+# Optional fields
-if [ "$UPGRADE" == "yes" ]; then
- echo "Not configuring trove-early-setup because this is an upgrade."
- exit 0
+if [ -n "$HOSTNAME" ]
+then
+ echo "HOSTNAME: $HOSTNAME" >> "$TROVE_DATA/trove.conf"
fi
-echo "Create /var/lib/trove-setup"
-install -d -o 0 -g 0 -m 0755 "$ROOT/var/lib/trove-setup"
-
-echo "Create /etc/trove-setup.needed"
-touch "$ROOT/etc/trove-setup.needed"
-chown 0:0 "$ROOT/etc/trove-setup.needed"
-chmod 0600 "$ROOT/etc/trove-setup.needed"
-
-##########################################################################
-
-# Put the lorry ssh keys onto the system. The trove-early-setup unit will
-# put them into the right place for the lorry user upon first boot.
-# We can't do that right now, because the lorry user won't exist until
-# trove-early-setup has run.
-echo "Copy Lorry ssh key to system"
-install -m 0600 "$LORRY_SSH_KEY"
"$ROOT/var/lib/trove-setup/lorry.key"
-install -m 0644 "${LORRY_SSH_KEY}.pub" \
- "$ROOT/var/lib/trove-setup/lorry.key.pub"
-
-##########################################################################
-
-echo "Copy admin's ssh public key to system"
-install -m 0644 "$TROVE_ADMIN_SSH_PUBKEY" \
- "$ROOT/var/lib/trove-setup/admin.key.pub"
-
-##########################################################################
-
-echo "Copy worker's ssh public key to system"
-install -m 0644 "$WORKER_SSH_PUBKEY" \
- "$ROOT/var/lib/trove-setup/worker.key.pub"
-
-##########################################################################
-
-echo "Copy mason's ssh public key to system"
-install -m 0644 "$MASON_SSH_PUBKEY" \
- "$ROOT/var/lib/trove-setup/mason.key.pub"
-
-##########################################################################
-
-if [ "x$MASON_DEFAULT_CI_HOSTS_FILE" = x ]; then
- echo "No default Mason hosts provided, using '[]'"
- printf '[\n]\n' >"$ROOT/var/lib/trove-setup/hosts.json.txt"
-else
- echo "Copy default Mason host configuration to the System"
- install -m 0644 "$MASON_DEFAULT_CI_HOSTS_FILE" \
- "$ROOT/var/lib/trove-setup/hosts.json.txt"
+if [ -n "$TROVE_HOSTNAME" ]
+then
+ echo "TROVE_HOSTNAME: $TROVE_HOSTNAME" >>
"$TROVE_DATA/trove.conf"
fi
-if [ "x$MASON_DEFAULT_CI_SYSTEMS_FILE" = x ]; then
- echo "No default Mason systems provided, using '[]'"
- printf '[\n]\n' >"$ROOT/var/lib/trove-setup/systems.json.txt"
-else
- echo "Copy default Mason system configuration to the System"
- install -m 0644 "$MASON_DEFAULT_CI_SYSTEMS_FILE" \
- "$ROOT/var/lib/trove-setup/systems.json.txt"
+if [ -n "$LORRY_CONTROLLER_MINIONS" ]
+then
+ echo "LORRY_CONTROLLER_MINIONS: $LORRY_CONTROLLER_MINIONS" >>
"$TROVE_DATA/trove.conf"
fi
-##########################################################################
-
-echo "Create trove-early-setup unit file"
-cat <<EOF > "$ROOT/etc/systemd/system/trove-early-setup.service"
-[Unit]
-Description=Run trove-early-setup (once)
-Requires=network.target
-After=network.target
-Requires=opensshd.service
-After=opensshd.service
-
-# If there's a shared /var subvolume, it must be mounted before this
-# unit runs.
-Requires=local-fs.target
-After=local-fs.target
-
-ConditionPathExists=/etc/trove-setup.needed
-
-# These must wait until we have created the required users on first boot.
-# We reboot the machine after this unit completes so these lines are not
-# strictly required, but it's nice to have a dependency graph that is true.
-Before=lighttpd.service
-Before=git-daemon.service
-
-
-[Service]
-Type=oneshot
-ExecStart=/bin/sh -c 'ssh-keyscan localhost $UPSTREAM_TROVE>
/etc/ssh/ssh_known_hosts'
-ExecStart=/usr/bin/trove-early-setup
-ExecStart=/usr/bin/install -m 0600 -o lorry -g lorry /var/lib/trove-setup/lorry.key
/home/lorry/.ssh/id_rsa
-ExecStart=/usr/bin/install -m 0644 -o lorry -g lorry /var/lib/trove-setup/lorry.key.pub
/home/lorry/.ssh/id_rsa.pub
-ExecStart=/bin/su git -c 'ssh git@localhost as lorry sshkey add configured <
/var/lib/trove-setup/lorry.key.pub'
-ExecStart=/bin/su git -c 'ssh git@localhost user add $TROVE_ADMIN_USER
$TROVE_ADMIN_EMAIL $TROVE_ADMIN_NAME'
-ExecStart=/bin/su git -c 'ssh git@localhost group adduser trove-admin
$TROVE_ADMIN_USER'
-ExecStart=/bin/su git -c 'ssh git@localhost as $TROVE_ADMIN_USER sshkey add default
< /var/lib/trove-setup/admin.key.pub'
-ExecStart=/bin/su git -c 'ssh git@localhost as distbuild sshkey add default <
/var/lib/trove-setup/worker.key.pub'
-ExecStart=/bin/su git -c 'ssh git@localhost as mason sshkey add default <
/var/lib/trove-setup/mason.key.pub'
-ExecStart=/bin/mkdir -p /var/run/lighttpd/
-ExecStart=/bin/chown cache:cache /var/run/lighttpd/
-ExecStart=/bin/rm /etc/trove-setup.needed
-ExecStart=/sbin/reboot
-Restart=no
-EOF
-
-##########################################################################
-
-ln -s "/etc/systemd/system/trove-early-setup.service" \
- "$ROOT/etc/systemd/system/multi-user.target.wants/trove-early-setup.service"
+if [ -n "$TROVE_BACKUP_KEYS" ]
+then
+ mkdir -p "$TROVE_DATA/backup-keys"
+ cp $TROVE_BACKUP_KEYS "$TROVE_DATA/backup-keys"
+ echo "TROVE_BACKUP_KEYS: /etc/trove/backup-keys/*" >>
"$TROVE_DATA/trove.conf"
+fi
--
1.7.10.4
11:20 a.m.
New subject: [PATCH 5/5] New trove.configure
On Tue, Jul 01, 2014 at 06:01:48PM +0100, Pedro Alvarez wrote:
+if [ -n "$TROVE_BACKUP_KEYS" ]
+then
+ mkdir -p "$TROVE_DATA/backup-keys"
+ cp $TROVE_BACKUP_KEYS "$TROVE_DATA/backup-keys"
+ echo "TROVE_BACKUP_KEYS: /etc/trove/backup-keys/*" >>
"$TROVE_DATA/trove.conf"
+fi
Quote $TROVE_BACKUP_KEYS please.
There's an argument for requiring us to also do `cp --
"$TROVE_BACKUP_KEYS" "$TROVE_DATA/backup-keys"`, using `--` to ensure
"$TROVE_BACKUP_KEYS" is not interpreted as an option, but since there's
no variable expansion providing further arguments, this will end up with
the command failing, rather than doing something unexpected, so we can
get away with not adding the `--` until we need it.
4:11 p.m.
New subject: [PATCH 5/5] New trove.configure
On 02/07/14 10:20, Richard Maw wrote:
On Tue, Jul 01, 2014 at 06:01:48PM +0100, Pedro Alvarez wrote:
>
> +if [ -n "$TROVE_BACKUP_KEYS" ]
> +then
> + mkdir -p "$TROVE_DATA/backup-keys"
> + cp $TROVE_BACKUP_KEYS "$TROVE_DATA/backup-keys"
> + echo "TROVE_BACKUP_KEYS: /etc/trove/backup-keys/*" >>
"$TROVE_DATA/trove.conf"
> +fi
Quote $TROVE_BACKUP_KEYS please.
There's an argument for requiring us to also do `cp --
"$TROVE_BACKUP_KEYS" "$TROVE_DATA/backup-keys"`, using `--` to
ensure
"$TROVE_BACKUP_KEYS" is not interpreted as an option, but since there's
no variable expansion providing further arguments, this will end up with
the command failing, rather than doing something unexpected, so we can
get away with not adding the `--` until we need it.
From trove-backup.configure:
"TROVE_BACKUP_KEYS - a space-separated list of paths to SSH keys."
My intention running `cp $TROVE_BACKUP_KEYS` is to copy all of the files listed
on the "space-separated list". I assumed that all the files listed there are
whitespace safe, do you think assuming that is correct in this case?
I'm with you on using `--`, I will change that. I will also add the
documentation that was written on trove-backup.configure to the new
trove.configure (which includes now the backups configuration).
Pedro.
5:24 p.m.
New subject: [PATCH 5/5] New trove.configure
On Wed, Jul 02, 2014 at 03:11:52PM +0100, Pedro Alvarez wrote:
On 02/07/14 10:20, Richard Maw wrote:
> On Tue, Jul 01, 2014 at 06:01:48PM +0100, Pedro Alvarez wrote:
>>
>> +if [ -n "$TROVE_BACKUP_KEYS" ]
>> +then
>> + mkdir -p "$TROVE_DATA/backup-keys"
>> + cp $TROVE_BACKUP_KEYS "$TROVE_DATA/backup-keys"
>> + echo "TROVE_BACKUP_KEYS: /etc/trove/backup-keys/*" >>
"$TROVE_DATA/trove.conf"
>> +fi
>
> Quote $TROVE_BACKUP_KEYS please.
>
> There's an argument for requiring us to also do `cp --
> "$TROVE_BACKUP_KEYS" "$TROVE_DATA/backup-keys"`, using `--` to
ensure
> "$TROVE_BACKUP_KEYS" is not interpreted as an option, but since
there's
> no variable expansion providing further arguments, this will end up with
> the command failing, rather than doing something unexpected, so we can
> get away with not adding the `--` until we need it.
>
>From trove-backup.configure:
"TROVE_BACKUP_KEYS - a space-separated list of paths to SSH keys."
I missed that detail.
My intention running `cp $TROVE_BACKUP_KEYS` is to copy all of the
files listed
on the "space-separated list". I assumed that all the files listed there are
whitespace safe, do you think assuming that is correct in this case?
It'll do. Extensions are run relative to the definitions repository,
so we can avoid spaces.
I'm with you on using `--`, I will change that. I will also add
the
documentation that was written on trove-backup.configure to the new
trove.configure (which includes now the backups configuration).
7:01 p.m.
New subject: [PATCH 0/9] Change how trove-setup configures the trove.
Repo: baserock:baserock/trove-setup
Branch: baserock/pedroalvarez/ansible-clean
Sha1: 92aa3046237778273271fbe64fd9ca7b58b0a533
Landing: master
This patch series changes the way the trove is configured.
Now trove-setup uses ansible to manage all the configuration needed
and also the migration scripts. The trove will be configured at boot time
and the new solution includes all the configuration previously done in
trove.configure.
That means now is easier to deploy generic troves and configure them
later.
The patch series includes all the changes necessary to use Ansible.
Pedro Alvarez (9):
Remove old scripts and units
Move gitano skeleton to /usr/share/trove-setup/
Move template files from /etc to shares/trove-setup/etc
Change placeholders to jinja placeholders
Do not enable the units when installing
Add new resources needed to configure the lorry-controller
Add Ansible scripts
Install Ansible scripts and create a unit to run them
Update skel path of gitano
Makefile | 12 +-
ansible/hosts | 1 +
ansible/roles/trove-setup/tasks/backups.yml | 17 +++
ansible/roles/trove-setup/tasks/cache-setup.yml | 12 ++
ansible/roles/trove-setup/tasks/check.yml | 78 ++++++++++++
ansible/roles/trove-setup/tasks/git.yml | 8 ++
.../roles/trove-setup/tasks/gitano-admin-setup.yml | 39 ++++++
.../roles/trove-setup/tasks/gitano-lorry-setup.yml | 18 +++
.../roles/trove-setup/tasks/gitano-mason-setup.yml | 16 +++
ansible/roles/trove-setup/tasks/gitano-setup.yml | 48 +++++++
.../trove-setup/tasks/gitano-worker-setup.yml | 18 +++
ansible/roles/trove-setup/tasks/hostname.yml | 26 ++++
.../roles/trove-setup/tasks/known-hosts-setup.yml | 7 ++
ansible/roles/trove-setup/tasks/lighttpd.yml | 51 ++++++++
.../trove-setup/tasks/lorry-controller-setup.yml | 104 +++++++++++++++
ansible/roles/trove-setup/tasks/lorry-setup.yml | 20 +++
ansible/roles/trove-setup/tasks/main.yml | 19 +++
ansible/roles/trove-setup/tasks/minions.yml | 15 +++
ansible/roles/trove-setup/tasks/nfs-setup.yml | 7 ++
ansible/roles/trove-setup/tasks/releases.yml | 26 ++++
ansible/roles/trove-setup/tasks/site-groups.yml | 88 +++++++++++++
ansible/roles/trove-setup/tasks/users.yml | 38 ++++++
ansible/trove-setup.yml | 6 +
bins/trove-early-setup | 124 ------------------
etc/cgitrc | 26 ----
etc/gitano-setup.clod | 18 ---
etc/lorry.conf | 11 --
gitano-admin/global-hooks/post-receive.lua | 105 ----------------
gitano-admin/groups/local-config-admins.conf | 1 -
gitano-admin/groups/local-config-managers.conf | 3 -
gitano-admin/groups/local-config-readers.conf | 5 -
gitano-admin/groups/local-config-writers.conf | 3 -
gitano-admin/groups/trove-admin.conf | 1 -
gitano-admin/groups/workers.conf | 4 -
gitano-admin/rules/adminchecks.lace | 25 ----
gitano-admin/rules/aschecks.lace | 30 -----
gitano-admin/rules/core.lace | 47 -------
gitano-admin/rules/createrepo.lace | 23 ----
gitano-admin/rules/defines.lace | 106 ----------------
gitano-admin/rules/destroyrepo.lace | 20 ---
gitano-admin/rules/other-project.lace | 25 ----
gitano-admin/rules/project.lace | 38 ------
gitano-admin/rules/remoteconfigchecks.lace | 20 ---
gitano-admin/rules/renamerepo.lace | 19 ---
gitano-admin/rules/selfchecks.lace | 15 ---
gitano-admin/rules/siteadmin.lace | 32 -----
gitano-admin/rules/trove-project.lace | 29 -----
gitano-admin/users/distbuild/user.conf | 2 -
gitano-admin/users/lorry/user.conf | 2 -
gitano-admin/users/mason/user.conf | 2 -
.../remove-lorry-controller-from-lorry-crontab | 22 ----
share/README.lorry-controller | 2 +-
share/etc/cgitrc | 26 ++++
share/etc/gitano-setup.clod | 19 +++
share/etc/lorry-controller/minion.conf | 6 +
share/etc/lorry-controller/webapp.conf | 12 ++
share/etc/lorry.conf | 11 ++
.../gitano-admin/global-hooks/post-receive.lua | 105 ++++++++++++++++
.../gitano-admin/groups/local-config-admins.conf | 1 +
.../gitano-admin/groups/local-config-managers.conf | 3 +
.../gitano-admin/groups/local-config-readers.conf | 5 +
.../gitano-admin/groups/local-config-writers.conf | 3 +
.../skel/gitano-admin/groups/trove-admin.conf | 1 +
share/gitano/skel/gitano-admin/groups/workers.conf | 4 +
.../skel/gitano-admin/rules/adminchecks.lace | 25 ++++
share/gitano/skel/gitano-admin/rules/aschecks.lace | 30 +++++
share/gitano/skel/gitano-admin/rules/core.lace | 47 +++++++
.../gitano/skel/gitano-admin/rules/createrepo.lace | 23 ++++
share/gitano/skel/gitano-admin/rules/defines.lace | 106 ++++++++++++++++
.../skel/gitano-admin/rules/destroyrepo.lace | 20 +++
.../skel/gitano-admin/rules/other-project.lace | 25 ++++
share/gitano/skel/gitano-admin/rules/project.lace | 38 ++++++
.../gitano-admin/rules/remoteconfigchecks.lace | 20 +++
.../gitano/skel/gitano-admin/rules/renamerepo.lace | 19 +++
.../gitano/skel/gitano-admin/rules/selfchecks.lace | 15 +++
.../gitano/skel/gitano-admin/rules/siteadmin.lace | 32 +++++
.../skel/gitano-admin/rules/trove-project.lace | 29 +++++
.../skel/gitano-admin/users/distbuild/user.conf | 2 +
.../gitano/skel/gitano-admin/users/lorry/user.conf | 2 +
.../gitano/skel/gitano-admin/users/mason/user.conf | 2 +
share/lorry-controller.conf | 8 +-
share/releases-repo-README | 4 +-
share/releases-repo-migration.sh | 132 --------------------
units/drop-lorry-controller-cronjob.service | 13 --
units/releases-repo-migration.service | 14 ---
units/trove-setup.service | 16 +++
86 files changed, 1319 insertions(+), 933 deletions(-)
create mode 100644 ansible/hosts
create mode 100644 ansible/roles/trove-setup/tasks/backups.yml
create mode 100644 ansible/roles/trove-setup/tasks/cache-setup.yml
create mode 100644 ansible/roles/trove-setup/tasks/check.yml
create mode 100644 ansible/roles/trove-setup/tasks/git.yml
create mode 100644 ansible/roles/trove-setup/tasks/gitano-admin-setup.yml
create mode 100644 ansible/roles/trove-setup/tasks/gitano-lorry-setup.yml
create mode 100644 ansible/roles/trove-setup/tasks/gitano-mason-setup.yml
create mode 100644 ansible/roles/trove-setup/tasks/gitano-setup.yml
create mode 100644 ansible/roles/trove-setup/tasks/gitano-worker-setup.yml
create mode 100644 ansible/roles/trove-setup/tasks/hostname.yml
create mode 100644 ansible/roles/trove-setup/tasks/known-hosts-setup.yml
create mode 100644 ansible/roles/trove-setup/tasks/lighttpd.yml
create mode 100644 ansible/roles/trove-setup/tasks/lorry-controller-setup.yml
create mode 100644 ansible/roles/trove-setup/tasks/lorry-setup.yml
create mode 100644 ansible/roles/trove-setup/tasks/main.yml
create mode 100644 ansible/roles/trove-setup/tasks/minions.yml
create mode 100644 ansible/roles/trove-setup/tasks/nfs-setup.yml
create mode 100644 ansible/roles/trove-setup/tasks/releases.yml
create mode 100644 ansible/roles/trove-setup/tasks/site-groups.yml
create mode 100644 ansible/roles/trove-setup/tasks/users.yml
create mode 100644 ansible/trove-setup.yml
delete mode 100755 bins/trove-early-setup
delete mode 100644 etc/cgitrc
delete mode 100644 etc/gitano-setup.clod
delete mode 100644 etc/lorry.conf
delete mode 100644 gitano-admin/global-hooks/post-receive.lua
delete mode 100644 gitano-admin/groups/local-config-admins.conf
delete mode 100644 gitano-admin/groups/local-config-managers.conf
delete mode 100644 gitano-admin/groups/local-config-readers.conf
delete mode 100644 gitano-admin/groups/local-config-writers.conf
delete mode 100644 gitano-admin/groups/trove-admin.conf
delete mode 100644 gitano-admin/groups/workers.conf
delete mode 100644 gitano-admin/rules/adminchecks.lace
delete mode 100644 gitano-admin/rules/aschecks.lace
delete mode 100644 gitano-admin/rules/core.lace
delete mode 100644 gitano-admin/rules/createrepo.lace
delete mode 100644 gitano-admin/rules/defines.lace
delete mode 100644 gitano-admin/rules/destroyrepo.lace
delete mode 100644 gitano-admin/rules/other-project.lace
delete mode 100644 gitano-admin/rules/project.lace
delete mode 100644 gitano-admin/rules/remoteconfigchecks.lace
delete mode 100644 gitano-admin/rules/renamerepo.lace
delete mode 100644 gitano-admin/rules/selfchecks.lace
delete mode 100644 gitano-admin/rules/siteadmin.lace
delete mode 100644 gitano-admin/rules/trove-project.lace
delete mode 100644 gitano-admin/users/distbuild/user.conf
delete mode 100644 gitano-admin/users/lorry/user.conf
delete mode 100644 gitano-admin/users/mason/user.conf
delete mode 100755 libexecs/remove-lorry-controller-from-lorry-crontab
create mode 100644 share/etc/cgitrc
create mode 100644 share/etc/gitano-setup.clod
create mode 100644 share/etc/lorry-controller/minion.conf
create mode 100644 share/etc/lorry-controller/webapp.conf
create mode 100644 share/etc/lorry.conf
create mode 100644 share/gitano/skel/gitano-admin/global-hooks/post-receive.lua
create mode 100644 share/gitano/skel/gitano-admin/groups/local-config-admins.conf
create mode 100644 share/gitano/skel/gitano-admin/groups/local-config-managers.conf
create mode 100644 share/gitano/skel/gitano-admin/groups/local-config-readers.conf
create mode 100644 share/gitano/skel/gitano-admin/groups/local-config-writers.conf
create mode 100644 share/gitano/skel/gitano-admin/groups/trove-admin.conf
create mode 100644 share/gitano/skel/gitano-admin/groups/workers.conf
create mode 100644 share/gitano/skel/gitano-admin/rules/adminchecks.lace
create mode 100644 share/gitano/skel/gitano-admin/rules/aschecks.lace
create mode 100644 share/gitano/skel/gitano-admin/rules/core.lace
create mode 100644 share/gitano/skel/gitano-admin/rules/createrepo.lace
create mode 100644 share/gitano/skel/gitano-admin/rules/defines.lace
create mode 100644 share/gitano/skel/gitano-admin/rules/destroyrepo.lace
create mode 100644 share/gitano/skel/gitano-admin/rules/other-project.lace
create mode 100644 share/gitano/skel/gitano-admin/rules/project.lace
create mode 100644 share/gitano/skel/gitano-admin/rules/remoteconfigchecks.lace
create mode 100644 share/gitano/skel/gitano-admin/rules/renamerepo.lace
create mode 100644 share/gitano/skel/gitano-admin/rules/selfchecks.lace
create mode 100644 share/gitano/skel/gitano-admin/rules/siteadmin.lace
create mode 100644 share/gitano/skel/gitano-admin/rules/trove-project.lace
create mode 100644 share/gitano/skel/gitano-admin/users/distbuild/user.conf
create mode 100644 share/gitano/skel/gitano-admin/users/lorry/user.conf
create mode 100644 share/gitano/skel/gitano-admin/users/mason/user.conf
delete mode 100755 share/releases-repo-migration.sh
delete mode 100644 units/drop-lorry-controller-cronjob.service
delete mode 100644 units/releases-repo-migration.service
create mode 100644 units/trove-setup.service
--
1.7.10.4
7:01 p.m.
New subject: [PATCH 1/9] Remove old scripts and units
---
Makefile | 4 -
bins/trove-early-setup | 124 ------------------
.../remove-lorry-controller-from-lorry-crontab | 22 ----
share/releases-repo-migration.sh | 132 --------------------
units/drop-lorry-controller-cronjob.service | 13 --
units/releases-repo-migration.service | 14 ---
6 files changed, 309 deletions(-)
delete mode 100755 bins/trove-early-setup
delete mode 100755 libexecs/remove-lorry-controller-from-lorry-crontab
delete mode 100755 share/releases-repo-migration.sh
delete mode 100644 units/drop-lorry-controller-cronjob.service
delete mode 100644 units/releases-repo-migration.service
diff --git a/Makefile b/Makefile
index 3a64344..ba0b752 100644
--- a/Makefile
+++ b/Makefile
@@ -14,10 +14,6 @@ install:
ln -s /home/lorry/tarballs "${DESTDIR}/var/www/htdocs/tarballs"
ln -s /home/lorry/lc-status.html "${DESTDIR}/var/www/htdocs/lc-status.html"
ln -s /usr/share/lorry-controller/static/
"${DESTDIR}/var/www/htdocs/lc-static"
- mkdir -p "${DESTDIR}/usr/bin"
- cp bins/* "${DESTDIR}/usr/bin/"
- mkdir -p "${DESTDIR}/usr/libexec"
- cp libexecs/* "${DESTDIR}/usr/libexec/"
mkdir -p "${DESTDIR}/usr/share/trove-setup"
cp -r share/* "${DESTDIR}/usr/share/trove-setup/"
diff --git a/bins/trove-early-setup b/bins/trove-early-setup
deleted file mode 100755
index 5ce2d7a..0000000
--- a/bins/trove-early-setup
+++ /dev/null
@@ -1,124 +0,0 @@
-#!/usr/bin/make -f
-#
-# Copyright (C) 2013 Codethink Limited
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; version 2 of the License.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-#
-# -*- Make -*-
-
-
-all: gitano-configured lorry-configured cache-configured mason-configured nfs-configured
cert-generated
-
-USERS := git lorry cache mason
-
-# $1 == username to make
-define make_user_rules
-
-/home/$1/.created:
- adduser -g "Trove $1 service" -s /bin/bash -D $1
- su -c 'mkdir .ssh; chmod 750 .ssh' - $1
- su -c 'ssh-keygen -t rsa -N "" -q -f .ssh/id_rsa' - $1
- (cat /etc/ssh/ssh_host_*_key.pub | cut -d\ -f1,2 | \
- sed -e's/^/'$(shell hostname)',localhost /' > \
- /home/$1/.ssh/known_hosts)
- chown $1:$1 /home/$1/.ssh/known_hosts
- chmod 600 /home/$1/.ssh/known_hosts
- touch $$@
-
-ALL_USER_TARGETS := $$(ALL_USER_TARGETS) /home/$1/.created
-
-endef
-
-$(eval $(foreach USER,$(USERS),$(call make_user_rules,$(USER))))
-
-/home/git/.git-setup: $(ALL_USER_TARGETS)
- su -c 'git config --global user.name "Trove Git Controller"' - git
- su -c 'git config --global user.email "git@trove"' - git
- touch $@
-
-/home/git/.gitano-setup: /home/git/.git-setup
- su -c 'gitano-setup /etc/gitano-setup.clod' - git
- passwd -u git
- touch $@
-
-/home/git/.gitano-lorry-setup: /home/git/.gitano-setup
- cp /home/lorry/.ssh/id_rsa.pub /tmp/lorry.pub
- su -c 'ssh git@localhost as lorry sshkey add trove < /tmp/lorry.pub' - git
- rm /tmp/lorry.pub
- touch $@
-
-.PHONY: gitano-configured
-gitano-configured: /home/git/.gitano-setup
-
-/home/lorry/.lorry-setup: $(ALL_USER_TARGETS)
- su -c 'mkdir /home/lorry/bundles /home/lorry/tarballs' - lorry
- touch $@
-
-/home/lorry/.lorry-controller-setup: /home/lorry/.lorry-setup
/home/git/.gitano-lorry-setup
- PREFIX=$$(echo "##PREFIX##" | sed -f /etc/trove-setup.sed); \
- su -c "ssh localhost create $${PREFIX}/local-config/lorries" - git; \
- su -c "git clone ssh://localhost/$${PREFIX}/local-config/lorries.git
/tmp/lorries" - git; \
- su -c "sed -f /etc/trove-setup.sed <
/usr/share/trove-setup/lorry-controller.conf > /tmp/lorries/lorry-controller.conf"
- git
- su -c "sed -f /etc/trove-setup.sed <
/usr/share/trove-setup/README.lorry-controller > /tmp/lorries/README" - git
- su -c "mkdir /tmp/lorries/open-source-lorries" - git
- su -c "cp /usr/share/trove-setup/open-source-lorries/README
/tmp/lorries/open-source-lorries/README" - git
- su -c "mkdir /tmp/lorries/closed-source-lorries" - git
- su -c "cp /usr/share/trove-setup/closed-source-lorries/README
/tmp/lorries/closed-source-lorries/README" - git
- su -c "cd /tmp/lorries; git add README lorry-controller.conf
open-source-lorries/README closed-source-lorries/README; git commit -m 'Initial
configuration'; git push origin master" - git
- su -c "rm -rf /tmp/lorries" - git
- touch $@
-
-.PHONY: lorry-configured
-lorry-configured: /home/lorry/.lorry-setup /home/lorry/.lorry-controller-setup
-
-/home/cache/.cache-setup: $(ALL_USER_TARGETS)
- su -c 'mkdir /home/cache/artifacts' - cache
- su -c 'mkdir /home/cache/ccache' - cache
- echo '/home/cache/ccache
*(rw,all_squash,no_subtree_check,anonuid=1002,anongid=1002)' > /etc/exports.cache
- touch $@
-
-.PHONY: cache-configured
-cache-configured: /home/cache/.cache-setup
-
-/home/git/.mason-setup: /home/git/.gitano-setup $(ALL_USER_TARGETS)
- PREFIX=$$(echo "##PREFIX##" | sed -f /etc/trove-setup.sed); \
- su -c "ssh localhost create $${PREFIX}/local-config/mason" - git; \
- su -c "git clone ssh://localhost/$${PREFIX}/local-config/mason.git
/tmp/mason-config" - git
- su -c "mkdir /tmp/mason-config/ci1" - git
- su -c "cp /var/lib/trove-setup/hosts.json.txt /tmp/mason-config/ci1" - git
- su -c "cp /var/lib/trove-setup/systems.json.txt /tmp/mason-config/ci1" - git
- su -c "cd /tmp/mason-config; git add ci1; git commit -m 'Set initial Mason
config'; git push origin master" - git
- su -c "rm -fr /tmp/mason-config" - git
- su -c 'mkdir /home/mason/jobs' - mason
- echo '/home/mason/jobs
*(rw,all_squash,no_subtree_check,anonuid=1003,anongid=1003)' > /etc/exports.mason
- touch $@
-
-.PHONY: mason-configured
-mason-configured: /home/git/.mason-setup
-
-/etc/exports: /home/cache/.cache-setup /home/git/.mason-setup
- cat /etc/exports.cache /etc/exports.mason >/etc/exports
-
-.PHONY: nfs-configured
-nfs-configured: /etc/exports
-
-/home/git/.cert-generated:
- mkdir -p /etc/lighttpd/certs
- echo -ne '\n\n\n\n\n\n\n' | openssl req -new -x509 \
- -keyout /etc/lighttpd/certs/lighttpd.pem \
- -out /etc/lighttpd/certs/lighttpd.pem -days 36525 -nodes
- touch $@
-
-.PHONY: cert-generated
-cert-generated: /home/git/.cert-generated
diff --git a/libexecs/remove-lorry-controller-from-lorry-crontab
b/libexecs/remove-lorry-controller-from-lorry-crontab
deleted file mode 100755
index 8fc6cf3..0000000
--- a/libexecs/remove-lorry-controller-from-lorry-crontab
+++ /dev/null
@@ -1,22 +0,0 @@
-#!/bin/sh
-#
-# Trove used to run a version of Lorry Controller that wasn't a
-# daemon, but instead was invoked once a minute from a crontab owned
-# by the lorry user. When we upgrade to a version of Lorry Controller
-# that does run as a daemon, we need to disable the cronjob. This
-# script does that.
-#
-# The lorry user crontab may contain other jobs, so we can't just
-# willy-nilly delete the whole crontab. Instead, we remove the
-# specific line. The line looks like this:
-#
-# */1 * * * * flock -x -n /home/lorry/lorry-controller-area/lockfile
-# -c lorry-controller --work-area=/home/lorry/lorry-controller-area
-# --log=syslog --log-level=info --html-file=/home/lorry/lc-status.html
-#
-# Except, of course, all on one line.
-
-
-crontab -l |
-grep -v -e '-c lorry-controller' |
-crontab -
diff --git a/share/releases-repo-migration.sh b/share/releases-repo-migration.sh
deleted file mode 100755
index 654da0c..0000000
--- a/share/releases-repo-migration.sh
+++ /dev/null
@@ -1,132 +0,0 @@
-#!/bin/bash
-
-function create_readers_group()
-{
- set +e
- (
- set -e
- ssh localhost group add site-readers \
- 'Users with read access to the site project'
- )
- local ret="$?"
- if [ "$ret" != 0 ]; then
- token=$(ssh localhost group del site-readers 2>&1 | tail -1 | \
- cut -d' ' -f 2)
- ssh localhost group del site-readers $token
- fi
- return $ret
-}
-
-function create_writers_group()
-{
- set +e
- (
- set -e
- ssh localhost group add site-writers \
- 'Users with write access to the site project'
- create_readers_group
- )
- local ret="$?"
- if [ "$ret" != 0 ]; then
- token=$(ssh localhost group del site-writers 2>&1 | tail -1 | \
- cut -d' ' -f 2)
- ssh localhost group del site-writers $token
- fi
- return $ret
-}
-
-function create_admins_group()
-{
- set +e
- (
- set -e
- ssh localhost group add site-admins \
- 'Users with admin access to the site project'
- create_writers_group
- )
- local ret="$?"
- if [ "$ret" != 0 ]; then
- token=$(ssh localhost group del site-admins 2>&1 | tail -1 | \
- cut -d' ' -f 2)
- ssh localhost group del site-admins $token
- fi
- return $ret
-}
-
-function create_managers_group()
-{
- set +e
- (
- set -e
- ssh localhost group add site-managers \
- 'Users with manager access to the site project'
- create_admins_group
- )
- local ret="$?"
- if [ "$ret" != 0 ]; then
- token=$(ssh localhost group del site-managers 2>&1 | tail -1 | \
- cut -d' ' -f 2)
- ssh localhost group del site-managers $token
- fi
- return $ret
-}
-
-function link_groups()
-{
- set -e
- ssh localhost group addgroup site-admins site-managers
- ssh localhost group addgroup site-writers site-admins
- ssh localhost group addgroup site-readers site-writers
-}
-
-function delete_groups()
-{
- token=$(ssh localhost group del site-managers 2>&1 | tail -1 | \
- cut -d' ' -f 2)
- ssh localhost group del site-managers $token
- token=$(ssh localhost group del site-admins 2>&1 | tail -1 | \
- cut -d' ' -f 2)
- ssh localhost group del site-admins $token
- token=$(ssh localhost group del site-writers 2>&1 | tail -1 | \
- cut -d' ' -f 2)
- ssh localhost group del site-writers $token
- token=$(ssh localhost group del site-readers 2>&1 | tail -1 | \
- cut -d' ' -f 2)
- ssh localhost group del site-readers $token
-}
-
-function create_groups()
-{
- # call managers_group which calls admin_group and so on...
- create_managers_group
- set +e
- (
- set -e
- link_groups
- )
- local ret="$?"
- if [ "$ret" != 0 ]; then
- delete_groups
- fi
-}
-
-site_groups=$(ssh localhost group list | grep -cE "site-[[:alnum:]]+")
-if [ "$site_groups" == 0 ]; then
- create_groups
-fi
-ssh localhost create "##PREFIX##/site/releases"
-description="This is a special repository for distributing release binaries
-over HTTP. Visit http://##PREFIX##/releases/ to browse content."
-ssh localhost config "##PREFIX##/site/releases" \
- set project.description "$description"
-
-# add a readme to the repository
-repo=$(mktemp -d)
-git clone ssh://localhost/##PREFIX##/site/releases $repo
-cp /usr/share/trove-setup/releases-repo-README $repo/README
-cd $repo
-git add $repo/README
-git commit -m 'Add README'
-git push origin master
-cd -
-rm -Rf $repo
diff --git a/units/drop-lorry-controller-cronjob.service
b/units/drop-lorry-controller-cronjob.service
deleted file mode 100644
index 8cad21f..0000000
--- a/units/drop-lorry-controller-cronjob.service
+++ /dev/null
@@ -1,13 +0,0 @@
-[Unit]
-Description=Drop lorry-controller from lorry's crontab
-After=basic.target
-ConditionPathExists=!/etc/lorry-controller/lorry-controller-removed-from-crontab
-
-[Service]
-Type=oneshot
-Restart=no
-ExecStart=/usr/libexec/remove-lorry-controller-from-lorry-crontab
-ExecStartPost=/bin/touch /etc/lorry-controller/lorry-controller-removed-from-crontab
-User=lorry
-Group=lorry
-PermissionsStartOnly=true
diff --git a/units/releases-repo-migration.service
b/units/releases-repo-migration.service
deleted file mode 100644
index 1e161fb..0000000
--- a/units/releases-repo-migration.service
+++ /dev/null
@@ -1,14 +0,0 @@
-[Unit]
-Description=Create the ##PREFIX##/site/releases repository
-ConditionPathExists=!/home/git/repos/##PREFIX##/site/releases.git
-Requires=network.target
-After=network.target
-Requires=opensshd.service
-After=opensshd.service
-Requires=trove-early-setup.service
-After=trove-early-setup.service
-
-[Service]
-User=git
-ExecStart=/usr/share/trove-setup/releases-repo-migration.sh
-Restart=no
--
1.7.10.4
7:01 p.m.
New subject: [PATCH 2/9] Move gitano skeleton to /usr/share/trove-setup/
---
Makefile | 2 -
gitano-admin/global-hooks/post-receive.lua | 105 -------------------
gitano-admin/groups/local-config-admins.conf | 1 -
gitano-admin/groups/local-config-managers.conf | 3 -
gitano-admin/groups/local-config-readers.conf | 5 -
gitano-admin/groups/local-config-writers.conf | 3 -
gitano-admin/groups/trove-admin.conf | 1 -
gitano-admin/groups/workers.conf | 4 -
gitano-admin/rules/adminchecks.lace | 25 -----
gitano-admin/rules/aschecks.lace | 30 ------
gitano-admin/rules/core.lace | 47 ---------
gitano-admin/rules/createrepo.lace | 23 -----
gitano-admin/rules/defines.lace | 106 --------------------
gitano-admin/rules/destroyrepo.lace | 20 ----
gitano-admin/rules/other-project.lace | 25 -----
gitano-admin/rules/project.lace | 38 -------
gitano-admin/rules/remoteconfigchecks.lace | 20 ----
gitano-admin/rules/renamerepo.lace | 19 ----
gitano-admin/rules/selfchecks.lace | 15 ---
gitano-admin/rules/siteadmin.lace | 32 ------
gitano-admin/rules/trove-project.lace | 29 ------
gitano-admin/users/distbuild/user.conf | 2 -
gitano-admin/users/lorry/user.conf | 2 -
gitano-admin/users/mason/user.conf | 2 -
.../gitano-admin/global-hooks/post-receive.lua | 105 +++++++++++++++++++
.../gitano-admin/groups/local-config-admins.conf | 1 +
.../gitano-admin/groups/local-config-managers.conf | 3 +
.../gitano-admin/groups/local-config-readers.conf | 5 +
.../gitano-admin/groups/local-config-writers.conf | 3 +
.../skel/gitano-admin/groups/trove-admin.conf | 1 +
share/gitano/skel/gitano-admin/groups/workers.conf | 4 +
.../skel/gitano-admin/rules/adminchecks.lace | 25 +++++
share/gitano/skel/gitano-admin/rules/aschecks.lace | 30 ++++++
share/gitano/skel/gitano-admin/rules/core.lace | 47 +++++++++
.../gitano/skel/gitano-admin/rules/createrepo.lace | 23 +++++
share/gitano/skel/gitano-admin/rules/defines.lace | 106 ++++++++++++++++++++
.../skel/gitano-admin/rules/destroyrepo.lace | 20 ++++
.../skel/gitano-admin/rules/other-project.lace | 25 +++++
share/gitano/skel/gitano-admin/rules/project.lace | 38 +++++++
.../gitano-admin/rules/remoteconfigchecks.lace | 20 ++++
.../gitano/skel/gitano-admin/rules/renamerepo.lace | 19 ++++
.../gitano/skel/gitano-admin/rules/selfchecks.lace | 15 +++
.../gitano/skel/gitano-admin/rules/siteadmin.lace | 32 ++++++
.../skel/gitano-admin/rules/trove-project.lace | 29 ++++++
.../skel/gitano-admin/users/distbuild/user.conf | 2 +
.../gitano/skel/gitano-admin/users/lorry/user.conf | 2 +
.../gitano/skel/gitano-admin/users/mason/user.conf | 2 +
47 files changed, 557 insertions(+), 559 deletions(-)
delete mode 100644 gitano-admin/global-hooks/post-receive.lua
delete mode 100644 gitano-admin/groups/local-config-admins.conf
delete mode 100644 gitano-admin/groups/local-config-managers.conf
delete mode 100644 gitano-admin/groups/local-config-readers.conf
delete mode 100644 gitano-admin/groups/local-config-writers.conf
delete mode 100644 gitano-admin/groups/trove-admin.conf
delete mode 100644 gitano-admin/groups/workers.conf
delete mode 100644 gitano-admin/rules/adminchecks.lace
delete mode 100644 gitano-admin/rules/aschecks.lace
delete mode 100644 gitano-admin/rules/core.lace
delete mode 100644 gitano-admin/rules/createrepo.lace
delete mode 100644 gitano-admin/rules/defines.lace
delete mode 100644 gitano-admin/rules/destroyrepo.lace
delete mode 100644 gitano-admin/rules/other-project.lace
delete mode 100644 gitano-admin/rules/project.lace
delete mode 100644 gitano-admin/rules/remoteconfigchecks.lace
delete mode 100644 gitano-admin/rules/renamerepo.lace
delete mode 100644 gitano-admin/rules/selfchecks.lace
delete mode 100644 gitano-admin/rules/siteadmin.lace
delete mode 100644 gitano-admin/rules/trove-project.lace
delete mode 100644 gitano-admin/users/distbuild/user.conf
delete mode 100644 gitano-admin/users/lorry/user.conf
delete mode 100644 gitano-admin/users/mason/user.conf
create mode 100644 share/gitano/skel/gitano-admin/global-hooks/post-receive.lua
create mode 100644 share/gitano/skel/gitano-admin/groups/local-config-admins.conf
create mode 100644 share/gitano/skel/gitano-admin/groups/local-config-managers.conf
create mode 100644 share/gitano/skel/gitano-admin/groups/local-config-readers.conf
create mode 100644 share/gitano/skel/gitano-admin/groups/local-config-writers.conf
create mode 100644 share/gitano/skel/gitano-admin/groups/trove-admin.conf
create mode 100644 share/gitano/skel/gitano-admin/groups/workers.conf
create mode 100644 share/gitano/skel/gitano-admin/rules/adminchecks.lace
create mode 100644 share/gitano/skel/gitano-admin/rules/aschecks.lace
create mode 100644 share/gitano/skel/gitano-admin/rules/core.lace
create mode 100644 share/gitano/skel/gitano-admin/rules/createrepo.lace
create mode 100644 share/gitano/skel/gitano-admin/rules/defines.lace
create mode 100644 share/gitano/skel/gitano-admin/rules/destroyrepo.lace
create mode 100644 share/gitano/skel/gitano-admin/rules/other-project.lace
create mode 100644 share/gitano/skel/gitano-admin/rules/project.lace
create mode 100644 share/gitano/skel/gitano-admin/rules/remoteconfigchecks.lace
create mode 100644 share/gitano/skel/gitano-admin/rules/renamerepo.lace
create mode 100644 share/gitano/skel/gitano-admin/rules/selfchecks.lace
create mode 100644 share/gitano/skel/gitano-admin/rules/siteadmin.lace
create mode 100644 share/gitano/skel/gitano-admin/rules/trove-project.lace
create mode 100644 share/gitano/skel/gitano-admin/users/distbuild/user.conf
create mode 100644 share/gitano/skel/gitano-admin/users/lorry/user.conf
create mode 100644 share/gitano/skel/gitano-admin/users/mason/user.conf
diff --git a/Makefile b/Makefile
index ba0b752..0559468 100644
--- a/Makefile
+++ b/Makefile
@@ -1,6 +1,4 @@
install:
- mkdir -p "${DESTDIR}/usr/share/gitano/skel"
- cp -a gitano-admin "${DESTDIR}/usr/share/gitano/skel"
mkdir -p "${DESTDIR}/usr/lib/systemd/system/multi-user.target.wants"
cp units/* "${DESTDIR}/usr/lib/systemd/system"
for I in $$(cd units; ls); do \
diff --git a/gitano-admin/global-hooks/post-receive.lua
b/gitano-admin/global-hooks/post-receive.lua
deleted file mode 100644
index d1b3864..0000000
--- a/gitano-admin/global-hooks/post-receive.lua
+++ /dev/null
@@ -1,105 +0,0 @@
--- mason-notify.post-receive.lua
---
--- Global post-receive hook which notifies Mason of any and all refs updates
--- (except refs/gitano/*) which happen.
---
--- It notifies Mason *before* passing the updates on to the project hook.
---
--- Copyright 2012 Codethink Limited
---
--- This is a part of Trove and re-use is limited to Baserock systems only.
---
-
-local project_hook, repo, updates = ...
-
-local EMPTY_SHA = ("0"):rep(40)
-
-local masonhost = "##MASON_HOST##:##MASON_PORT##"
-local basepath = "/1.0"
-local urlbases = {
- "git://##TROVE_HOSTNAME##/",
- "ssh://git@##TROVE_HOSTNAME##/",
-}
-
-local notify_mason = false
-
-for ref in pairs(updates) do
- if not ref:match("^refs/gitano/") then
- notify_mason = true
- end
-end
-
-if notify_mason and repo.name ~= "gitano-admin" then
- -- Build the report...
- local masoninfo, indent_level = {}, 0
- local function _(...)
- masoninfo[#masoninfo+1] = (" "):rep(indent_level) ..
table.concat({...})
- end
- local function indent()
- indent_level = indent_level + 1
- end
- local function dedent()
- indent_level = indent_level - 1
- end
- _ "{" indent()
-
- _ '"urls": [' indent()
-
- for i = 1, #urlbases do
- local comma = (i==#urlbases) and "" or ","
- _(("%q,"):format(urlbases[i] .. repo.name))
- _(("%q%s"):format(urlbases[i] .. repo.name .. ".git", comma))
- end
-
- dedent() _ "],"
-
- _ '"changes": [' indent()
-
- local toreport = {}
- for ref, info in pairs(updates) do
- if not ref:match("^refs/gitano") then
- local action
- if info.oldsha == EMPTY_SHA then
- action = "create"
- elseif info.newsha == EMPTY_SHA then
- action = "delete"
- else
- action = "update"
- end
- toreport[#toreport+1] = {
- ('"ref": %q,'):format(ref),
- ('"action": %q,'):format(action),
- ('"old": %q,'):format(info.oldsha),
- ('"new": %q'):format(info.newsha)
- }
- end
- end
- for i = 1, #toreport do
- local comma = (i==#toreport) and "" or ","
- _ "{" indent()
- for __, ent in ipairs(toreport[i]) do
- _(ent)
- end
- dedent() _("}", comma)
- end
- dedent() _ "]"
-
- dedent() _ "}"
-
- -- And finalise the JSON object
- _("")
- masoninfo = table.concat(masoninfo, "\n")
- log.state("Notifying Mason of changes...")
-
- local code, msg, headers, content =
- http.post(masonhost, basepath, "application/json", masoninfo)
- if code ~= "200" then
- log.state("Notification failed somehow")
- end
- for line in content:gmatch("([^\r\n]*)\r?\n") do
- log.state("Mason: " .. line)
- end
-end
-
--- Finally, chain to the project hook
-return project_hook(repo, updates)
diff --git a/gitano-admin/groups/local-config-admins.conf
b/gitano-admin/groups/local-config-admins.conf
deleted file mode 100644
index 435a297..0000000
--- a/gitano-admin/groups/local-config-admins.conf
+++ /dev/null
@@ -1 +0,0 @@
-description "Users who are permitted to administer the local-config project"
diff --git a/gitano-admin/groups/local-config-managers.conf
b/gitano-admin/groups/local-config-managers.conf
deleted file mode 100644
index 711be8f..0000000
--- a/gitano-admin/groups/local-config-managers.conf
+++ /dev/null
@@ -1,3 +0,0 @@
-description "Users who are permitted to manage the local-config project"
-
-subgroups["*"] "local-config-admins"
diff --git a/gitano-admin/groups/local-config-readers.conf
b/gitano-admin/groups/local-config-readers.conf
deleted file mode 100644
index 63e6bb3..0000000
--- a/gitano-admin/groups/local-config-readers.conf
+++ /dev/null
@@ -1,5 +0,0 @@
-description "Users who are permitted to read from the local-config project"
-
-members["*"] "lorry"
-
-subgroups["*"] "local-config-writers"
diff --git a/gitano-admin/groups/local-config-writers.conf
b/gitano-admin/groups/local-config-writers.conf
deleted file mode 100644
index 9bbff24..0000000
--- a/gitano-admin/groups/local-config-writers.conf
+++ /dev/null
@@ -1,3 +0,0 @@
-description "Users who are permitted to write to the local-config project"
-
-subgroups["*"] "local-config-managers"
diff --git a/gitano-admin/groups/trove-admin.conf b/gitano-admin/groups/trove-admin.conf
deleted file mode 100644
index e912653..0000000
--- a/gitano-admin/groups/trove-admin.conf
+++ /dev/null
@@ -1 +0,0 @@
-description "Trove-local administration"
diff --git a/gitano-admin/groups/workers.conf b/gitano-admin/groups/workers.conf
deleted file mode 100644
index 5586538..0000000
--- a/gitano-admin/groups/workers.conf
+++ /dev/null
@@ -1,4 +0,0 @@
-description "Workers who have read-access to everything"
-
-members["*"] "distbuild"
-members["*"] "mason"
diff --git a/gitano-admin/rules/adminchecks.lace b/gitano-admin/rules/adminchecks.lace
deleted file mode 100644
index ffe99a0..0000000
--- a/gitano-admin/rules/adminchecks.lace
+++ /dev/null
@@ -1,25 +0,0 @@
-# _____
-# |_ _| __ _____ _____
-# | || '__/ _ \ \ / / _ \
-# | || | | (_) \ V / __/
-# |_||_| \___/ \_/ \___|
-#
-# Copyright 2012 Codethink Limited
-#
-# Core project administration rules
-
-# Called with ref known to be refs/gitano/admin
-
-# Administrators already got to do anything, so this is for non-admins
-
-# Non-admin members may not delete the admin ref
-deny "Non-administrators may not delete the admin ref" op_deleteref
-
-# Otherwise, the project's owner is allowed to alter the admin tree
-allow "Project owner may alter the admin ref" is_owner repo_is_personal
-
-# Project admins may alter admin refs
-allow "Project admins may alter the admin ref of project repos"
repo_is_local_project project_admin
-
-# Any other opportunities for altering the admin ref must be provided
-# by the project's rules
diff --git a/gitano-admin/rules/aschecks.lace b/gitano-admin/rules/aschecks.lace
deleted file mode 100644
index fc76440..0000000
--- a/gitano-admin/rules/aschecks.lace
+++ /dev/null
@@ -1,30 +0,0 @@
-# _____
-# |_ _| __ _____ _____
-# | || '__/ _ \ \ / / _ \
-# | || | | (_) \ V / __/
-# |_||_| \___/ \_/ \___|
-#
-# Copyright 2012 Codethink Limited
-#
-# Rules for when we're running as another user.
-
-# Only 'deny' things which are not allowed. If you 'allow' then it will
allow
-# the actual operation, not just fail to deny the fact that it's 'as'
someone
-# else.
-
-define as_is_admin as_group gitano-admin
-
-# trove-admin members are permitted to run sshkey and whoami on behalf
-# of others in order to check users and grant access, providing the target
-# user is not part of the gitano-admin group.
-
-define as_is_trove_admin as_group trove-admin
-define as_trove_admin_ok allof as_is_trove_admin !is_admin op_self
-
-# You are permitted to do things 'as' others if and only if the caller is
-# either a member of the administration group, or else meets the above
-# requirements.
-define as_is_ok anyof as_is_admin as_trove_admin_ok
-
-# Explicitly deny any impersonation operation which does not meet the above.
-deny "You may not run things as another user unless you are an admin"
!as_is_ok
diff --git a/gitano-admin/rules/core.lace b/gitano-admin/rules/core.lace
deleted file mode 100644
index dab7cfb..0000000
--- a/gitano-admin/rules/core.lace
+++ /dev/null
@@ -1,47 +0,0 @@
-# _____
-# |_ _| __ _____ _____
-# | || '__/ _ \ \ / / _ \
-# | || | | (_) \ V / __/
-# |_||_| \___/ \_/ \___|
-#
-# Copyright 2012 Codethink Limited
-#
-# Core ruleset definitions for Trove.
-
-default deny "Trove ruleset failed to define result. Access denied."
-
-include global:defines
-
-# The users in the administration group (gitano-admin) may do anything
-# they choose (providing they're not being impersonated). By default
-# Only the user created as part of trove-setup has this level of access.
-allow "Administrators can do anything" is_admin !if_asanother
-
-# Now let's decide if we can use 'as'
-include global:aschecks if_asanother
-
-# Operations which are against 'self' get checked next
-include global:selfchecks
-
-# Administration operations (users, groups) next
-include global:siteadmin op_is_admin
-
-# Site-defined rules for repository creation
-include global:createrepo op_createrepo
-
-# Site-defined rules for repository renaming
-include global:renamerepo op_renamerepo
-
-# Site-defined rules for repository destruction
-include global:destroyrepo op_destroyrepo
-
-# Site-defined rules for project repositories, including admin of them
-include global:project
-
-# Now the project rules themselves
-include main
-
-# If you're running your access control somewhat more openly than most, You can
-# now uncomment the following and allow git:// access to *everything* which is
-# not the admin repository
-# allow "Anonymous access is okay" op_read !is_admin_repo
diff --git a/gitano-admin/rules/createrepo.lace b/gitano-admin/rules/createrepo.lace
deleted file mode 100644
index bf4683e..0000000
--- a/gitano-admin/rules/createrepo.lace
+++ /dev/null
@@ -1,23 +0,0 @@
-# _____
-# |_ _| __ _____ _____
-# | || '__/ _ \ \ / / _ \
-# | || | | (_) \ V / __/
-# |_||_| \___/ \_/ \___|
-#
-# Copyright 2012 Codethink Limited
-#
-# Rules related to creating repositories
-
-# Administrators have already been permitted whatever they like
-# so this is for site-wide non-admins.
-
-##PEOPLE_COMMENT##allow "Personal repo creation is okay" repo_is_personal
-
-# Allow people in *-admins to create repositories under <foo>
-allow "Project admins may make project repositories" repo_is_local_project
project_admin
-
-# Allow lorry to create repositories anywhere but the local project root
-allow "Lorry may create lorryable repos" is_lorry lorryable_repo
-
-# Otherwise the default is that non-admins can't create repositories
-deny "Repository creation is not permitted."
diff --git a/gitano-admin/rules/defines.lace b/gitano-admin/rules/defines.lace
deleted file mode 100644
index 380948a..0000000
--- a/gitano-admin/rules/defines.lace
+++ /dev/null
@@ -1,106 +0,0 @@
-# _____
-# |_ _| __ _____ _____
-# | || '__/ _ \ \ / / _ \
-# | || | | (_) \ V / __/
-# |_||_| \___/ \_/ \___|
-#
-# Copyright 2012,2013 Codethink Limited
-#
-# Core definitions for access control
-
-# Gitano provided definitions first
-
-# User/group related
-define is_admin group gitano-admin
-define is_owner owner ${user}
-define is_anonymous user gitano/anonymous
-
-define if_asanother as_user ~.
-
-# Self-related operations
-define op_whoami operation whoami
-define op_sshkey operation sshkey
-define op_passwd operation passwd
-define op_self anyof op_whoami op_sshkey op_passwd
-
-# Admin-related operations
-
-## Users
-define op_useradd operation useradd
-define op_userdel operation userdel
-define op_userlist operation userlist
-define op_useremail operation useremail
-define op_username operation username
-define op_user anyof op_userlist op_useradd op_userdel op_useremail op_username
-
-## Groups
-define op_grouplist operation grouplist
-define op_groupshow operation groupshow
-define op_groupadd operation groupadd
-define op_groupdel operation groupdel
-define op_groupadduser operation groupadduser
-define op_groupdeluser operation groupdeluser
-define op_groupaddgroup operation groupaddgroup
-define op_groupdelgroup operation groupdelgroup
-define op_groupdescription operation groupdescription
-define op_group anyof op_grouplist op_groupshow op_groupadd op_groupdel op_groupadduser
op_groupdeluser op_groupaddgroup op_groupdelgroup op_groupdescription
-
-## Aggregation of admin ops
-define op_is_admin anyof op_user op_group
-
-# Primary repository-related operations
-define op_read operation read
-define op_write operation write
-define op_createrepo operation createrepo
-define op_renamerepo operation renamerepo
-define op_destroyrepo operation destroyrepo
-
-# Remote configuration operations
-define op_config_show operation config_show
-define op_config_set operation config_set
-define op_config_del operation config_del
-define op_is_config anyof op_config_show op_config_set op_config_del
-
-# Reference update related operations
-define op_createref operation createref
-define op_deleteref operation deleteref
-define op_fastforward operation updaterefff
-define op_forcedupdate operation updaterefnonff
-
-# Combinator operations
-define op_is_basic anyof op_read op_write
-define op_is_update anyof op_fastforward op_forcedupdate
-define op_is_normal anyof op_fastforward op_createref op_deleteref
-
-# Administration
-define is_admin_repo repository gitano-admin
-define is_gitano_ref ref ~^refs/gitano/
-define is_admin_ref ref refs/gitano/admin
-
-#
-#
-# Trove definitions after here
-#
-#
-
-define repo_is_personal repository ~^##ESC_PERSONAL_PREFIX##/${user}/
-define ref_is_personal ref ~^refs/heads/##ESC_PREFIX##/${user}/
-define repo_is_local_project repository ~^##ESC_PREFIX##/[^/]+/
-
-define project_reader group ${repository/2}-readers
-define project_writer group ${repository/2}-writers
-define project_admin group ${repository/2}-admins
-define project_manager group ${repository/2}-managers
-
-define master_ref ref ~^refs/heads/master$
-
-define op_is_reffy anyof op_is_normal op_forcedupdate
-
-define trove_site_admin group trove-admin
-define target_group_gitano_admin targetgroup gitano-admin
-
-define is_lorry user lorry
-define is_local_ref ref ~^refs/heads/##ESC_PREFIX##/
-define lorryable_repo allof !repo_is_local_project !repo_is_personal !is_admin_repo
-
-define is_worker group workers
diff --git a/gitano-admin/rules/destroyrepo.lace b/gitano-admin/rules/destroyrepo.lace
deleted file mode 100644
index 6e6b446..0000000
--- a/gitano-admin/rules/destroyrepo.lace
+++ /dev/null
@@ -1,20 +0,0 @@
-# _____
-# |_ _| __ _____ _____
-# | || '__/ _ \ \ / / _ \
-# | || | | (_) \ V / __/
-# |_||_| \___/ \_/ \___|
-#
-# Copyright 2012 Codethink Limited
-#
-# Rules related to the destroying of repositories
-
-# Owners may destroy personal repositories
-allow "You may destroy your own repositories" is_owner repo_is_personal
-
-# Project admins may destroy repos inside their projects
-allow "Project admins may destroy project repos" repo_is_local_project
project_admin
-
-# Allow lorry to destroy repositories anywhere but the local project root
-allow "Lorry may destroy lorryable repos" is_lorry lorryable_repo
-
-deny "You may not destroy repositories you do not own"
diff --git a/gitano-admin/rules/other-project.lace
b/gitano-admin/rules/other-project.lace
deleted file mode 100644
index 7bc80cc..0000000
--- a/gitano-admin/rules/other-project.lace
+++ /dev/null
@@ -1,25 +0,0 @@
-# _____
-# |_ _| __ _____ _____
-# | || '__/ _ \ \ / / _ \
-# | || | | (_) \ V / __/
-# |_||_| \___/ \_/ \___|
-#
-# Copyright 2012,2013 Codethink Limited
-#
-# Rules for any repository not under ##PREFIX##
-
-# This is, by default, /baserock/ and /delta/
-
-# There are two classes of accessors here. Lorry and Others
-allow "Anyone may read here" op_read
-allow "Anyone may write here" op_write !is_anonymous
-
-# Lorry can do anything reffy which is not inside the local refs
-allow "Lorry may touch everything but refs/heads/##PREFIX##" op_is_reffy
is_lorry !is_local_ref
-
-# Noone can rewind/rebase outside of their personal refs
-deny "Non-personal branches may not be rewound/rebased" op_forcedupdate
!is_lorry !ref_is_personal
-
-# Everyone else can do reffy things inside refs/heads/##PREFIX##
-allow "Project writers may alter any refs" op_is_reffy !is_lorry is_local_ref
-
diff --git a/gitano-admin/rules/project.lace b/gitano-admin/rules/project.lace
deleted file mode 100644
index aa5e1e2..0000000
--- a/gitano-admin/rules/project.lace
+++ /dev/null
@@ -1,38 +0,0 @@
-# _____
-# |_ _| __ _____ _____
-# | || '__/ _ \ \ / / _ \
-# | || | | (_) \ V / __/
-# |_||_| \___/ \_/ \___|
-#
-# Copyright 2012 Codethink Limited
-#
-# Core project administration rules
-
-# Admins already got allowed, so this is for non-admin users only
-allow "Owners can always read and write" op_is_basic is_owner repo_is_personal
-
-# Any non-gitano-admin repo is readable to the lorry user and the worker group
-allow "Lorry may read" op_read is_lorry lorryable_repo
-allow "Workers may read" op_read !is_admin_repo is_worker
-
-# Force /baserock and /delta to always be anon-readable which means git:// will
-# work. This is part of the core ruleset for Baserock because /baserock/ and
-# /delta/ are always open source.
-define is_baserock_repo repository ~^baserock/
-define is_delta_repo repository ~^delta/
-define is_opensource_repo anyof is_baserock_repo is_delta_repo
-
-allow "Anonymous access always allowed" op_read !is_admin_repo
is_opensource_repo
-
-# Project remote-configuration rules (set-head etc)
-include global:remoteconfigchecks op_is_config
-
-# Okay, if we're altering the admin ref, in we go
-include global:adminchecks is_admin_ref
-
-# Now we're into branch operations.
-# Owners of personal repositories can do any reffy operation
-allow "Owners can create refs" op_is_reffy is_owner repo_is_personal
-
-include global:trove-project repo_is_local_project
-include global:other-project lorryable_repo
diff --git a/gitano-admin/rules/remoteconfigchecks.lace
b/gitano-admin/rules/remoteconfigchecks.lace
deleted file mode 100644
index 6f88f5f..0000000
--- a/gitano-admin/rules/remoteconfigchecks.lace
+++ /dev/null
@@ -1,20 +0,0 @@
-# _____
-# |_ _| __ _____ _____
-# | || '__/ _ \ \ / / _ \
-# | || | | (_) \ V / __/
-# |_||_| \___/ \_/ \___|
-#
-# Copyright 2012 Codethink Limited
-#
-# Remote config checks
-
-# Owners may do any remote admin operation they choose
-allow "Owners may remote-admin their repositories" is_owner repo_is_personal
-
-# *-admins may remote-admin their project's repositories
-allow "Project admins may admin project repos" repo_is_local_project
project_admin
-
-# lorry may remote-admin lorryable repositories
-allow "Lorry may admin lorry repos" is_lorry lorryable_repo
-
-deny "You may not configure this repository remotely"
diff --git a/gitano-admin/rules/renamerepo.lace b/gitano-admin/rules/renamerepo.lace
deleted file mode 100644
index e4a51be..0000000
--- a/gitano-admin/rules/renamerepo.lace
+++ /dev/null
@@ -1,19 +0,0 @@
-# _____
-# |_ _| __ _____ _____
-# | || '__/ _ \ \ / / _ \
-# | || | | (_) \ V / __/
-# |_||_| \___/ \_/ \___|
-#
-# Copyright 2012 Codethink Limited
-#
-# Rules related to renaming repositories
-
-# Owners may rename their own repositories
-allow "Owners may rename repositories" op_renamerepo repo_is_personal is_owner
-
-# Project admins may rename repos provided they're admin of source *and* target
-# Since the rename operation checks 'create' for the target, we can just
-# check the source here
-allow "Admins may rename project repositories" op_renamerepo
repo_is_local_project project_admin
-
-deny "You may not rename a repository you do not own"
diff --git a/gitano-admin/rules/selfchecks.lace b/gitano-admin/rules/selfchecks.lace
deleted file mode 100644
index 83ef778..0000000
--- a/gitano-admin/rules/selfchecks.lace
+++ /dev/null
@@ -1,15 +0,0 @@
-# _____
-# |_ _| __ _____ _____
-# | || '__/ _ \ \ / / _ \
-# | || | | (_) \ V / __/
-# |_||_| \___/ \_/ \___|
-#
-# Copyright 2012 Codethink Limited
-#
-# Checks against self-like operations.
-
-allow "You may ask who you are" op_whoami
-
-allow "You may manage your own ssh keys" op_sshkey
-
-allow "You may change your own password" op_passwd
diff --git a/gitano-admin/rules/siteadmin.lace b/gitano-admin/rules/siteadmin.lace
deleted file mode 100644
index 06c71bb..0000000
--- a/gitano-admin/rules/siteadmin.lace
+++ /dev/null
@@ -1,32 +0,0 @@
-# _____
-# |_ _| __ _____ _____
-# | || '__/ _ \ \ / / _ \
-# | || | | (_) \ V / __/
-# |_||_| \___/ \_/ \___|
-#
-# Copyright 2012 Codethink Limited
-#
-# Site administration rules
-
-# You must explicitly allow site administration here for anyone who
-# has the rights to do site admin but isn't an administrator.
-
-# trove_site_admin is a predicate which matches members of the trove-admin
-# group (The site-wide user/group administration group which is not the full
-# administration group)
-allow "Trove Site Admins can manage users" trove_site_admin op_user
-allow "Trove Site Admins can manage groups other than gitano-admin"
trove_site_admin op_group !target_group_gitano_admin
-
-# XXX-managers members are permitted to edit XXX-* groups
-define trove_may_admin_target_group group ${targetgroup/prefix}-managers
-define target_group_has_hyphen targetgroup ~%-
-allow "Trove project managers can manage the groups for their projects"
op_group target_group_has_hyphen trove_may_admin_target_group
-
-# Anyone is permitted to look at the people in trove-admin and *-managers
-define trove_target_group_is_trove_admin targetgroup trove-admin
-define trove_target_group_is_project_managers targetgroup ~^.+-managers$
-define trove_show_target_ok anyof trove_target_group_is_trove_admin
trove_target_group_is_project_managers
-allow "Anyone may see admin groups" op_groupshow trove_show_target_ok
-
-# Otherwise we always deny site administration
-deny "You may not perform site administration"
diff --git a/gitano-admin/rules/trove-project.lace
b/gitano-admin/rules/trove-project.lace
deleted file mode 100644
index 383ba98..0000000
--- a/gitano-admin/rules/trove-project.lace
+++ /dev/null
@@ -1,29 +0,0 @@
-# _____
-# |_ _| __ _____ _____
-# | || '__/ _ \ \ / / _ \
-# | || | | (_) \ V / __/
-# |_||_| \___/ \_/ \___|
-#
-# Copyright 2012,2013 Codethink Limited
-#
-# Rules for ##PREFIX##/... repositories
-
-# Reading the repository
-allow "Project readers may read" op_read project_reader
-deny "This repository is not for you" op_read
-
-# Basic writes to the repo
-allow "Project writers may write" op_write project_writer
-deny "This repository is not for you" op_write
-
-# Ref based rules for the repo
-deny "Non-personal branches may not be rewound/rebased" op_forcedupdate
!ref_is_personal
-
-## Master
-allow "Master may be created" op_createref master_ref
-allow "Master may be altered" op_is_update master_ref
-deny "Master may not be deleted" op_deleteref master_ref
-
-## Anything else.
-allow "Project writers may alter any refs" op_is_reffy !master_ref
project_writer
-
diff --git a/gitano-admin/users/distbuild/user.conf
b/gitano-admin/users/distbuild/user.conf
deleted file mode 100644
index 62ac3f5..0000000
--- a/gitano-admin/users/distbuild/user.conf
+++ /dev/null
@@ -1,2 +0,0 @@
-email_address "distbuild@##TROVE_HOSTNAME##"
-real_name "Baserock Distributed Build Service"
diff --git a/gitano-admin/users/lorry/user.conf b/gitano-admin/users/lorry/user.conf
deleted file mode 100644
index f21fac7..0000000
--- a/gitano-admin/users/lorry/user.conf
+++ /dev/null
@@ -1,2 +0,0 @@
-email_address "lorry@##TROVE_HOSTNAME##"
-real_name "Source Code Lorry Service"
diff --git a/gitano-admin/users/mason/user.conf b/gitano-admin/users/mason/user.conf
deleted file mode 100644
index 639de4e..0000000
--- a/gitano-admin/users/mason/user.conf
+++ /dev/null
@@ -1,2 +0,0 @@
-email_address "mason@##TROVE_HOSTNAME##"
-real_name "Baserock Continuous Integration Service"
diff --git a/share/gitano/skel/gitano-admin/global-hooks/post-receive.lua
b/share/gitano/skel/gitano-admin/global-hooks/post-receive.lua
new file mode 100644
index 0000000..d1b3864
--- /dev/null
+++ b/share/gitano/skel/gitano-admin/global-hooks/post-receive.lua
@@ -0,0 +1,105 @@
+-- mason-notify.post-receive.lua
+--
+-- Global post-receive hook which notifies Mason of any and all refs updates
+-- (except refs/gitano/*) which happen.
+--
+-- It notifies Mason *before* passing the updates on to the project hook.
+--
+-- Copyright 2012 Codethink Limited
+--
+-- This is a part of Trove and re-use is limited to Baserock systems only.
+--
+
+local project_hook, repo, updates = ...
+
+local EMPTY_SHA = ("0"):rep(40)
+
+local masonhost = "##MASON_HOST##:##MASON_PORT##"
+local basepath = "/1.0"
+local urlbases = {
+ "git://##TROVE_HOSTNAME##/",
+ "ssh://git@##TROVE_HOSTNAME##/",
+}
+
+local notify_mason = false
+
+for ref in pairs(updates) do
+ if not ref:match("^refs/gitano/") then
+ notify_mason = true
+ end
+end
+
+if notify_mason and repo.name ~= "gitano-admin" then
+ -- Build the report...
+ local masoninfo, indent_level = {}, 0
+ local function _(...)
+ masoninfo[#masoninfo+1] = (" "):rep(indent_level) ..
table.concat({...})
+ end
+ local function indent()
+ indent_level = indent_level + 1
+ end
+ local function dedent()
+ indent_level = indent_level - 1
+ end
+ _ "{" indent()
+
+ _ '"urls": [' indent()
+
+ for i = 1, #urlbases do
+ local comma = (i==#urlbases) and "" or ","
+ _(("%q,"):format(urlbases[i] .. repo.name))
+ _(("%q%s"):format(urlbases[i] .. repo.name .. ".git", comma))
+ end
+
+ dedent() _ "],"
+
+ _ '"changes": [' indent()
+
+ local toreport = {}
+ for ref, info in pairs(updates) do
+ if not ref:match("^refs/gitano") then
+ local action
+ if info.oldsha == EMPTY_SHA then
+ action = "create"
+ elseif info.newsha == EMPTY_SHA then
+ action = "delete"
+ else
+ action = "update"
+ end
+ toreport[#toreport+1] = {
+ ('"ref": %q,'):format(ref),
+ ('"action": %q,'):format(action),
+ ('"old": %q,'):format(info.oldsha),
+ ('"new": %q'):format(info.newsha)
+ }
+ end
+ end
+ for i = 1, #toreport do
+ local comma = (i==#toreport) and "" or ","
+ _ "{" indent()
+ for __, ent in ipairs(toreport[i]) do
+ _(ent)
+ end
+ dedent() _("}", comma)
+ end
+ dedent() _ "]"
+
+ dedent() _ "}"
+
+ -- And finalise the JSON object
+ _("")
+ masoninfo = table.concat(masoninfo, "\n")
+ log.state("Notifying Mason of changes...")
+
+ local code, msg, headers, content =
+ http.post(masonhost, basepath, "application/json", masoninfo)
+ if code ~= "200" then
+ log.state("Notification failed somehow")
+ end
+ for line in content:gmatch("([^\r\n]*)\r?\n") do
+ log.state("Mason: " .. line)
+ end
+end
+
+-- Finally, chain to the project hook
+return project_hook(repo, updates)
diff --git a/share/gitano/skel/gitano-admin/groups/local-config-admins.conf
b/share/gitano/skel/gitano-admin/groups/local-config-admins.conf
new file mode 100644
index 0000000..435a297
--- /dev/null
+++ b/share/gitano/skel/gitano-admin/groups/local-config-admins.conf
@@ -0,0 +1 @@
+description "Users who are permitted to administer the local-config project"
diff --git a/share/gitano/skel/gitano-admin/groups/local-config-managers.conf
b/share/gitano/skel/gitano-admin/groups/local-config-managers.conf
new file mode 100644
index 0000000..711be8f
--- /dev/null
+++ b/share/gitano/skel/gitano-admin/groups/local-config-managers.conf
@@ -0,0 +1,3 @@
+description "Users who are permitted to manage the local-config project"
+
+subgroups["*"] "local-config-admins"
diff --git a/share/gitano/skel/gitano-admin/groups/local-config-readers.conf
b/share/gitano/skel/gitano-admin/groups/local-config-readers.conf
new file mode 100644
index 0000000..63e6bb3
--- /dev/null
+++ b/share/gitano/skel/gitano-admin/groups/local-config-readers.conf
@@ -0,0 +1,5 @@
+description "Users who are permitted to read from the local-config project"
+
+members["*"] "lorry"
+
+subgroups["*"] "local-config-writers"
diff --git a/share/gitano/skel/gitano-admin/groups/local-config-writers.conf
b/share/gitano/skel/gitano-admin/groups/local-config-writers.conf
new file mode 100644
index 0000000..9bbff24
--- /dev/null
+++ b/share/gitano/skel/gitano-admin/groups/local-config-writers.conf
@@ -0,0 +1,3 @@
+description "Users who are permitted to write to the local-config project"
+
+subgroups["*"] "local-config-managers"
diff --git a/share/gitano/skel/gitano-admin/groups/trove-admin.conf
b/share/gitano/skel/gitano-admin/groups/trove-admin.conf
new file mode 100644
index 0000000..e912653
--- /dev/null
+++ b/share/gitano/skel/gitano-admin/groups/trove-admin.conf
@@ -0,0 +1 @@
+description "Trove-local administration"
diff --git a/share/gitano/skel/gitano-admin/groups/workers.conf
b/share/gitano/skel/gitano-admin/groups/workers.conf
new file mode 100644
index 0000000..5586538
--- /dev/null
+++ b/share/gitano/skel/gitano-admin/groups/workers.conf
@@ -0,0 +1,4 @@
+description "Workers who have read-access to everything"
+
+members["*"] "distbuild"
+members["*"] "mason"
diff --git a/share/gitano/skel/gitano-admin/rules/adminchecks.lace
b/share/gitano/skel/gitano-admin/rules/adminchecks.lace
new file mode 100644
index 0000000..ffe99a0
--- /dev/null
+++ b/share/gitano/skel/gitano-admin/rules/adminchecks.lace
@@ -0,0 +1,25 @@
+# _____
+# |_ _| __ _____ _____
+# | || '__/ _ \ \ / / _ \
+# | || | | (_) \ V / __/
+# |_||_| \___/ \_/ \___|
+#
+# Copyright 2012 Codethink Limited
+#
+# Core project administration rules
+
+# Called with ref known to be refs/gitano/admin
+
+# Administrators already got to do anything, so this is for non-admins
+
+# Non-admin members may not delete the admin ref
+deny "Non-administrators may not delete the admin ref" op_deleteref
+
+# Otherwise, the project's owner is allowed to alter the admin tree
+allow "Project owner may alter the admin ref" is_owner repo_is_personal
+
+# Project admins may alter admin refs
+allow "Project admins may alter the admin ref of project repos"
repo_is_local_project project_admin
+
+# Any other opportunities for altering the admin ref must be provided
+# by the project's rules
diff --git a/share/gitano/skel/gitano-admin/rules/aschecks.lace
b/share/gitano/skel/gitano-admin/rules/aschecks.lace
new file mode 100644
index 0000000..fc76440
--- /dev/null
+++ b/share/gitano/skel/gitano-admin/rules/aschecks.lace
@@ -0,0 +1,30 @@
+# _____
+# |_ _| __ _____ _____
+# | || '__/ _ \ \ / / _ \
+# | || | | (_) \ V / __/
+# |_||_| \___/ \_/ \___|
+#
+# Copyright 2012 Codethink Limited
+#
+# Rules for when we're running as another user.
+
+# Only 'deny' things which are not allowed. If you 'allow' then it will
allow
+# the actual operation, not just fail to deny the fact that it's 'as'
someone
+# else.
+
+define as_is_admin as_group gitano-admin
+
+# trove-admin members are permitted to run sshkey and whoami on behalf
+# of others in order to check users and grant access, providing the target
+# user is not part of the gitano-admin group.
+
+define as_is_trove_admin as_group trove-admin
+define as_trove_admin_ok allof as_is_trove_admin !is_admin op_self
+
+# You are permitted to do things 'as' others if and only if the caller is
+# either a member of the administration group, or else meets the above
+# requirements.
+define as_is_ok anyof as_is_admin as_trove_admin_ok
+
+# Explicitly deny any impersonation operation which does not meet the above.
+deny "You may not run things as another user unless you are an admin"
!as_is_ok
diff --git a/share/gitano/skel/gitano-admin/rules/core.lace
b/share/gitano/skel/gitano-admin/rules/core.lace
new file mode 100644
index 0000000..dab7cfb
--- /dev/null
+++ b/share/gitano/skel/gitano-admin/rules/core.lace
@@ -0,0 +1,47 @@
+# _____
+# |_ _| __ _____ _____
+# | || '__/ _ \ \ / / _ \
+# | || | | (_) \ V / __/
+# |_||_| \___/ \_/ \___|
+#
+# Copyright 2012 Codethink Limited
+#
+# Core ruleset definitions for Trove.
+
+default deny "Trove ruleset failed to define result. Access denied."
+
+include global:defines
+
+# The users in the administration group (gitano-admin) may do anything
+# they choose (providing they're not being impersonated). By default
+# Only the user created as part of trove-setup has this level of access.
+allow "Administrators can do anything" is_admin !if_asanother
+
+# Now let's decide if we can use 'as'
+include global:aschecks if_asanother
+
+# Operations which are against 'self' get checked next
+include global:selfchecks
+
+# Administration operations (users, groups) next
+include global:siteadmin op_is_admin
+
+# Site-defined rules for repository creation
+include global:createrepo op_createrepo
+
+# Site-defined rules for repository renaming
+include global:renamerepo op_renamerepo
+
+# Site-defined rules for repository destruction
+include global:destroyrepo op_destroyrepo
+
+# Site-defined rules for project repositories, including admin of them
+include global:project
+
+# Now the project rules themselves
+include main
+
+# If you're running your access control somewhat more openly than most, You can
+# now uncomment the following and allow git:// access to *everything* which is
+# not the admin repository
+# allow "Anonymous access is okay" op_read !is_admin_repo
diff --git a/share/gitano/skel/gitano-admin/rules/createrepo.lace
b/share/gitano/skel/gitano-admin/rules/createrepo.lace
new file mode 100644
index 0000000..bf4683e
--- /dev/null
+++ b/share/gitano/skel/gitano-admin/rules/createrepo.lace
@@ -0,0 +1,23 @@
+# _____
+# |_ _| __ _____ _____
+# | || '__/ _ \ \ / / _ \
+# | || | | (_) \ V / __/
+# |_||_| \___/ \_/ \___|
+#
+# Copyright 2012 Codethink Limited
+#
+# Rules related to creating repositories
+
+# Administrators have already been permitted whatever they like
+# so this is for site-wide non-admins.
+
+##PEOPLE_COMMENT##allow "Personal repo creation is okay" repo_is_personal
+
+# Allow people in *-admins to create repositories under <foo>
+allow "Project admins may make project repositories" repo_is_local_project
project_admin
+
+# Allow lorry to create repositories anywhere but the local project root
+allow "Lorry may create lorryable repos" is_lorry lorryable_repo
+
+# Otherwise the default is that non-admins can't create repositories
+deny "Repository creation is not permitted."
diff --git a/share/gitano/skel/gitano-admin/rules/defines.lace
b/share/gitano/skel/gitano-admin/rules/defines.lace
new file mode 100644
index 0000000..380948a
--- /dev/null
+++ b/share/gitano/skel/gitano-admin/rules/defines.lace
@@ -0,0 +1,106 @@
+# _____
+# |_ _| __ _____ _____
+# | || '__/ _ \ \ / / _ \
+# | || | | (_) \ V / __/
+# |_||_| \___/ \_/ \___|
+#
+# Copyright 2012,2013 Codethink Limited
+#
+# Core definitions for access control
+
+# Gitano provided definitions first
+
+# User/group related
+define is_admin group gitano-admin
+define is_owner owner ${user}
+define is_anonymous user gitano/anonymous
+
+define if_asanother as_user ~.
+
+# Self-related operations
+define op_whoami operation whoami
+define op_sshkey operation sshkey
+define op_passwd operation passwd
+define op_self anyof op_whoami op_sshkey op_passwd
+
+# Admin-related operations
+
+## Users
+define op_useradd operation useradd
+define op_userdel operation userdel
+define op_userlist operation userlist
+define op_useremail operation useremail
+define op_username operation username
+define op_user anyof op_userlist op_useradd op_userdel op_useremail op_username
+
+## Groups
+define op_grouplist operation grouplist
+define op_groupshow operation groupshow
+define op_groupadd operation groupadd
+define op_groupdel operation groupdel
+define op_groupadduser operation groupadduser
+define op_groupdeluser operation groupdeluser
+define op_groupaddgroup operation groupaddgroup
+define op_groupdelgroup operation groupdelgroup
+define op_groupdescription operation groupdescription
+define op_group anyof op_grouplist op_groupshow op_groupadd op_groupdel op_groupadduser
op_groupdeluser op_groupaddgroup op_groupdelgroup op_groupdescription
+
+## Aggregation of admin ops
+define op_is_admin anyof op_user op_group
+
+# Primary repository-related operations
+define op_read operation read
+define op_write operation write
+define op_createrepo operation createrepo
+define op_renamerepo operation renamerepo
+define op_destroyrepo operation destroyrepo
+
+# Remote configuration operations
+define op_config_show operation config_show
+define op_config_set operation config_set
+define op_config_del operation config_del
+define op_is_config anyof op_config_show op_config_set op_config_del
+
+# Reference update related operations
+define op_createref operation createref
+define op_deleteref operation deleteref
+define op_fastforward operation updaterefff
+define op_forcedupdate operation updaterefnonff
+
+# Combinator operations
+define op_is_basic anyof op_read op_write
+define op_is_update anyof op_fastforward op_forcedupdate
+define op_is_normal anyof op_fastforward op_createref op_deleteref
+
+# Administration
+define is_admin_repo repository gitano-admin
+define is_gitano_ref ref ~^refs/gitano/
+define is_admin_ref ref refs/gitano/admin
+
+#
+#
+# Trove definitions after here
+#
+#
+
+define repo_is_personal repository ~^##ESC_PERSONAL_PREFIX##/${user}/
+define ref_is_personal ref ~^refs/heads/##ESC_PREFIX##/${user}/
+define repo_is_local_project repository ~^##ESC_PREFIX##/[^/]+/
+
+define project_reader group ${repository/2}-readers
+define project_writer group ${repository/2}-writers
+define project_admin group ${repository/2}-admins
+define project_manager group ${repository/2}-managers
+
+define master_ref ref ~^refs/heads/master$
+
+define op_is_reffy anyof op_is_normal op_forcedupdate
+
+define trove_site_admin group trove-admin
+define target_group_gitano_admin targetgroup gitano-admin
+
+define is_lorry user lorry
+define is_local_ref ref ~^refs/heads/##ESC_PREFIX##/
+define lorryable_repo allof !repo_is_local_project !repo_is_personal !is_admin_repo
+
+define is_worker group workers
diff --git a/share/gitano/skel/gitano-admin/rules/destroyrepo.lace
b/share/gitano/skel/gitano-admin/rules/destroyrepo.lace
new file mode 100644
index 0000000..6e6b446
--- /dev/null
+++ b/share/gitano/skel/gitano-admin/rules/destroyrepo.lace
@@ -0,0 +1,20 @@
+# _____
+# |_ _| __ _____ _____
+# | || '__/ _ \ \ / / _ \
+# | || | | (_) \ V / __/
+# |_||_| \___/ \_/ \___|
+#
+# Copyright 2012 Codethink Limited
+#
+# Rules related to the destroying of repositories
+
+# Owners may destroy personal repositories
+allow "You may destroy your own repositories" is_owner repo_is_personal
+
+# Project admins may destroy repos inside their projects
+allow "Project admins may destroy project repos" repo_is_local_project
project_admin
+
+# Allow lorry to destroy repositories anywhere but the local project root
+allow "Lorry may destroy lorryable repos" is_lorry lorryable_repo
+
+deny "You may not destroy repositories you do not own"
diff --git a/share/gitano/skel/gitano-admin/rules/other-project.lace
b/share/gitano/skel/gitano-admin/rules/other-project.lace
new file mode 100644
index 0000000..7bc80cc
--- /dev/null
+++ b/share/gitano/skel/gitano-admin/rules/other-project.lace
@@ -0,0 +1,25 @@
+# _____
+# |_ _| __ _____ _____
+# | || '__/ _ \ \ / / _ \
+# | || | | (_) \ V / __/
+# |_||_| \___/ \_/ \___|
+#
+# Copyright 2012,2013 Codethink Limited
+#
+# Rules for any repository not under ##PREFIX##
+
+# This is, by default, /baserock/ and /delta/
+
+# There are two classes of accessors here. Lorry and Others
+allow "Anyone may read here" op_read
+allow "Anyone may write here" op_write !is_anonymous
+
+# Lorry can do anything reffy which is not inside the local refs
+allow "Lorry may touch everything but refs/heads/##PREFIX##" op_is_reffy
is_lorry !is_local_ref
+
+# Noone can rewind/rebase outside of their personal refs
+deny "Non-personal branches may not be rewound/rebased" op_forcedupdate
!is_lorry !ref_is_personal
+
+# Everyone else can do reffy things inside refs/heads/##PREFIX##
+allow "Project writers may alter any refs" op_is_reffy !is_lorry is_local_ref
+
diff --git a/share/gitano/skel/gitano-admin/rules/project.lace
b/share/gitano/skel/gitano-admin/rules/project.lace
new file mode 100644
index 0000000..aa5e1e2
--- /dev/null
+++ b/share/gitano/skel/gitano-admin/rules/project.lace
@@ -0,0 +1,38 @@
+# _____
+# |_ _| __ _____ _____
+# | || '__/ _ \ \ / / _ \
+# | || | | (_) \ V / __/
+# |_||_| \___/ \_/ \___|
+#
+# Copyright 2012 Codethink Limited
+#
+# Core project administration rules
+
+# Admins already got allowed, so this is for non-admin users only
+allow "Owners can always read and write" op_is_basic is_owner repo_is_personal
+
+# Any non-gitano-admin repo is readable to the lorry user and the worker group
+allow "Lorry may read" op_read is_lorry lorryable_repo
+allow "Workers may read" op_read !is_admin_repo is_worker
+
+# Force /baserock and /delta to always be anon-readable which means git:// will
+# work. This is part of the core ruleset for Baserock because /baserock/ and
+# /delta/ are always open source.
+define is_baserock_repo repository ~^baserock/
+define is_delta_repo repository ~^delta/
+define is_opensource_repo anyof is_baserock_repo is_delta_repo
+
+allow "Anonymous access always allowed" op_read !is_admin_repo
is_opensource_repo
+
+# Project remote-configuration rules (set-head etc)
+include global:remoteconfigchecks op_is_config
+
+# Okay, if we're altering the admin ref, in we go
+include global:adminchecks is_admin_ref
+
+# Now we're into branch operations.
+# Owners of personal repositories can do any reffy operation
+allow "Owners can create refs" op_is_reffy is_owner repo_is_personal
+
+include global:trove-project repo_is_local_project
+include global:other-project lorryable_repo
diff --git a/share/gitano/skel/gitano-admin/rules/remoteconfigchecks.lace
b/share/gitano/skel/gitano-admin/rules/remoteconfigchecks.lace
new file mode 100644
index 0000000..6f88f5f
--- /dev/null
+++ b/share/gitano/skel/gitano-admin/rules/remoteconfigchecks.lace
@@ -0,0 +1,20 @@
+# _____
+# |_ _| __ _____ _____
+# | || '__/ _ \ \ / / _ \
+# | || | | (_) \ V / __/
+# |_||_| \___/ \_/ \___|
+#
+# Copyright 2012 Codethink Limited
+#
+# Remote config checks
+
+# Owners may do any remote admin operation they choose
+allow "Owners may remote-admin their repositories" is_owner repo_is_personal
+
+# *-admins may remote-admin their project's repositories
+allow "Project admins may admin project repos" repo_is_local_project
project_admin
+
+# lorry may remote-admin lorryable repositories
+allow "Lorry may admin lorry repos" is_lorry lorryable_repo
+
+deny "You may not configure this repository remotely"
diff --git a/share/gitano/skel/gitano-admin/rules/renamerepo.lace
b/share/gitano/skel/gitano-admin/rules/renamerepo.lace
new file mode 100644
index 0000000..e4a51be
--- /dev/null
+++ b/share/gitano/skel/gitano-admin/rules/renamerepo.lace
@@ -0,0 +1,19 @@
+# _____
+# |_ _| __ _____ _____
+# | || '__/ _ \ \ / / _ \
+# | || | | (_) \ V / __/
+# |_||_| \___/ \_/ \___|
+#
+# Copyright 2012 Codethink Limited
+#
+# Rules related to renaming repositories
+
+# Owners may rename their own repositories
+allow "Owners may rename repositories" op_renamerepo repo_is_personal is_owner
+
+# Project admins may rename repos provided they're admin of source *and* target
+# Since the rename operation checks 'create' for the target, we can just
+# check the source here
+allow "Admins may rename project repositories" op_renamerepo
repo_is_local_project project_admin
+
+deny "You may not rename a repository you do not own"
diff --git a/share/gitano/skel/gitano-admin/rules/selfchecks.lace
b/share/gitano/skel/gitano-admin/rules/selfchecks.lace
new file mode 100644
index 0000000..83ef778
--- /dev/null
+++ b/share/gitano/skel/gitano-admin/rules/selfchecks.lace
@@ -0,0 +1,15 @@
+# _____
+# |_ _| __ _____ _____
+# | || '__/ _ \ \ / / _ \
+# | || | | (_) \ V / __/
+# |_||_| \___/ \_/ \___|
+#
+# Copyright 2012 Codethink Limited
+#
+# Checks against self-like operations.
+
+allow "You may ask who you are" op_whoami
+
+allow "You may manage your own ssh keys" op_sshkey
+
+allow "You may change your own password" op_passwd
diff --git a/share/gitano/skel/gitano-admin/rules/siteadmin.lace
b/share/gitano/skel/gitano-admin/rules/siteadmin.lace
new file mode 100644
index 0000000..06c71bb
--- /dev/null
+++ b/share/gitano/skel/gitano-admin/rules/siteadmin.lace
@@ -0,0 +1,32 @@
+# _____
+# |_ _| __ _____ _____
+# | || '__/ _ \ \ / / _ \
+# | || | | (_) \ V / __/
+# |_||_| \___/ \_/ \___|
+#
+# Copyright 2012 Codethink Limited
+#
+# Site administration rules
+
+# You must explicitly allow site administration here for anyone who
+# has the rights to do site admin but isn't an administrator.
+
+# trove_site_admin is a predicate which matches members of the trove-admin
+# group (The site-wide user/group administration group which is not the full
+# administration group)
+allow "Trove Site Admins can manage users" trove_site_admin op_user
+allow "Trove Site Admins can manage groups other than gitano-admin"
trove_site_admin op_group !target_group_gitano_admin
+
+# XXX-managers members are permitted to edit XXX-* groups
+define trove_may_admin_target_group group ${targetgroup/prefix}-managers
+define target_group_has_hyphen targetgroup ~%-
+allow "Trove project managers can manage the groups for their projects"
op_group target_group_has_hyphen trove_may_admin_target_group
+
+# Anyone is permitted to look at the people in trove-admin and *-managers
+define trove_target_group_is_trove_admin targetgroup trove-admin
+define trove_target_group_is_project_managers targetgroup ~^.+-managers$
+define trove_show_target_ok anyof trove_target_group_is_trove_admin
trove_target_group_is_project_managers
+allow "Anyone may see admin groups" op_groupshow trove_show_target_ok
+
+# Otherwise we always deny site administration
+deny "You may not perform site administration"
diff --git a/share/gitano/skel/gitano-admin/rules/trove-project.lace
b/share/gitano/skel/gitano-admin/rules/trove-project.lace
new file mode 100644
index 0000000..383ba98
--- /dev/null
+++ b/share/gitano/skel/gitano-admin/rules/trove-project.lace
@@ -0,0 +1,29 @@
+# _____
+# |_ _| __ _____ _____
+# | || '__/ _ \ \ / / _ \
+# | || | | (_) \ V / __/
+# |_||_| \___/ \_/ \___|
+#
+# Copyright 2012,2013 Codethink Limited
+#
+# Rules for ##PREFIX##/... repositories
+
+# Reading the repository
+allow "Project readers may read" op_read project_reader
+deny "This repository is not for you" op_read
+
+# Basic writes to the repo
+allow "Project writers may write" op_write project_writer
+deny "This repository is not for you" op_write
+
+# Ref based rules for the repo
+deny "Non-personal branches may not be rewound/rebased" op_forcedupdate
!ref_is_personal
+
+## Master
+allow "Master may be created" op_createref master_ref
+allow "Master may be altered" op_is_update master_ref
+deny "Master may not be deleted" op_deleteref master_ref
+
+## Anything else.
+allow "Project writers may alter any refs" op_is_reffy !master_ref
project_writer
+
diff --git a/share/gitano/skel/gitano-admin/users/distbuild/user.conf
b/share/gitano/skel/gitano-admin/users/distbuild/user.conf
new file mode 100644
index 0000000..62ac3f5
--- /dev/null
+++ b/share/gitano/skel/gitano-admin/users/distbuild/user.conf
@@ -0,0 +1,2 @@
+email_address "distbuild@##TROVE_HOSTNAME##"
+real_name "Baserock Distributed Build Service"
diff --git a/share/gitano/skel/gitano-admin/users/lorry/user.conf
b/share/gitano/skel/gitano-admin/users/lorry/user.conf
new file mode 100644
index 0000000..f21fac7
--- /dev/null
+++ b/share/gitano/skel/gitano-admin/users/lorry/user.conf
@@ -0,0 +1,2 @@
+email_address "lorry@##TROVE_HOSTNAME##"
+real_name "Source Code Lorry Service"
diff --git a/share/gitano/skel/gitano-admin/users/mason/user.conf
b/share/gitano/skel/gitano-admin/users/mason/user.conf
new file mode 100644
index 0000000..639de4e
--- /dev/null
+++ b/share/gitano/skel/gitano-admin/users/mason/user.conf
@@ -0,0 +1,2 @@
+email_address "mason@##TROVE_HOSTNAME##"
+real_name "Baserock Continuous Integration Service"
--
1.7.10.4
7:01 p.m.
New subject: [PATCH 3/9] Move template files from /etc to shares/trove-setup/etc
---
etc/cgitrc | 26 --------------------------
etc/gitano-setup.clod | 18 ------------------
etc/lorry.conf | 11 -----------
share/etc/cgitrc | 26 ++++++++++++++++++++++++++
share/etc/gitano-setup.clod | 18 ++++++++++++++++++
share/etc/lorry.conf | 11 +++++++++++
6 files changed, 55 insertions(+), 55 deletions(-)
delete mode 100644 etc/cgitrc
delete mode 100644 etc/gitano-setup.clod
delete mode 100644 etc/lorry.conf
create mode 100644 share/etc/cgitrc
create mode 100644 share/etc/gitano-setup.clod
create mode 100644 share/etc/lorry.conf
diff --git a/etc/cgitrc b/etc/cgitrc
deleted file mode 100644
index c526e17..0000000
--- a/etc/cgitrc
+++ /dev/null
@@ -1,26 +0,0 @@
-clone-prefix=git://##TROVE_HOSTNAME## http://##TROVE_HOSTNAME##/git
https://##TROVE_HOSTNAME##/git ssh://git@##TROVE_HOSTNAME##
-strict-export=git-daemon-export-ok
-
-css=/cgit/cgit.css
-logo=/trove.png
-
-head-include=/etc/cgit-trove-head.inc
-footer=/etc/cgit-trove-footer.inc
-
-enable-index-links=1
-root-title=##TROVE_TITLE## Git Repositories
-root-desc=Baserock Trove -- For ##TROVE_COMPANY##
-snapshots=tar.gz
-enable-commit-graph=1
-enable-log-filecount=1
-enable-log-linecount=1
-
-mimetype.gif=image/gif
-mimetype.html=text/html
-mimetype.jpg=image/jpeg
-mimetype.jpeg=image/jpeg
-mimetype.pdf=application/pdf
-mimetype.png=image/png
-mimetype.svg=image/svg+xml
-
-scan-path=/home/git/repos/
diff --git a/etc/gitano-setup.clod b/etc/gitano-setup.clod
deleted file mode 100644
index b63aeb6..0000000
--- a/etc/gitano-setup.clod
+++ /dev/null
@@ -1,18 +0,0 @@
--- Configuration for gitano-setup
-
-paths.home "/home/git"
-paths.ssh "/home/git/.ssh"
-paths.pubkey "/home/git/.ssh/id_rsa.pub"
-paths.repos "/home/git/repos"
-
-admin.username "trove"
-admin.realname "Trove Instance Administrator"
-admin.email "trove@trove-instance"
-admin.keyname "trove"
-
-site.name "##TROVE_TITLE## for ##TROVE_COMPANY##"
-log.prefix "##TROVE_LOG_PREFIX##"
-
-use.htpasswd "yes"
-
-setup.batch = true
diff --git a/etc/lorry.conf b/etc/lorry.conf
deleted file mode 100644
index 16552cb..0000000
--- a/etc/lorry.conf
+++ /dev/null
@@ -1,11 +0,0 @@
-[config]
-mirror-base-url-push = ssh://git@localhost
-mirror-base-url-fetch = git://##TROVE_HOSTNAME##
-bundle = never
-bundle-dest = /home/lorry/bundles
-tarball = always
-tarball-dest = /home/lorry/tarballs
-working-area = /home/lorry/working-area
-verbose = yes
-log = /dev/stdout
-log-level = debug
diff --git a/share/etc/cgitrc b/share/etc/cgitrc
new file mode 100644
index 0000000..c526e17
--- /dev/null
+++ b/share/etc/cgitrc
@@ -0,0 +1,26 @@
+clone-prefix=git://##TROVE_HOSTNAME## http://##TROVE_HOSTNAME##/git
https://##TROVE_HOSTNAME##/git ssh://git@##TROVE_HOSTNAME##
+strict-export=git-daemon-export-ok
+
+css=/cgit/cgit.css
+logo=/trove.png
+
+head-include=/etc/cgit-trove-head.inc
+footer=/etc/cgit-trove-footer.inc
+
+enable-index-links=1
+root-title=##TROVE_TITLE## Git Repositories
+root-desc=Baserock Trove -- For ##TROVE_COMPANY##
+snapshots=tar.gz
+enable-commit-graph=1
+enable-log-filecount=1
+enable-log-linecount=1
+
+mimetype.gif=image/gif
+mimetype.html=text/html
+mimetype.jpg=image/jpeg
+mimetype.jpeg=image/jpeg
+mimetype.pdf=application/pdf
+mimetype.png=image/png
+mimetype.svg=image/svg+xml
+
+scan-path=/home/git/repos/
diff --git a/share/etc/gitano-setup.clod b/share/etc/gitano-setup.clod
new file mode 100644
index 0000000..b63aeb6
--- /dev/null
+++ b/share/etc/gitano-setup.clod
@@ -0,0 +1,18 @@
+-- Configuration for gitano-setup
+
+paths.home "/home/git"
+paths.ssh "/home/git/.ssh"
+paths.pubkey "/home/git/.ssh/id_rsa.pub"
+paths.repos "/home/git/repos"
+
+admin.username "trove"
+admin.realname "Trove Instance Administrator"
+admin.email "trove@trove-instance"
+admin.keyname "trove"
+
+site.name "##TROVE_TITLE## for ##TROVE_COMPANY##"
+log.prefix "##TROVE_LOG_PREFIX##"
+
+use.htpasswd "yes"
+
+setup.batch = true
diff --git a/share/etc/lorry.conf b/share/etc/lorry.conf
new file mode 100644
index 0000000..16552cb
--- /dev/null
+++ b/share/etc/lorry.conf
@@ -0,0 +1,11 @@
+[config]
+mirror-base-url-push = ssh://git@localhost
+mirror-base-url-fetch = git://##TROVE_HOSTNAME##
+bundle = never
+bundle-dest = /home/lorry/bundles
+tarball = always
+tarball-dest = /home/lorry/tarballs
+working-area = /home/lorry/working-area
+verbose = yes
+log = /dev/stdout
+log-level = debug
--
1.7.10.4
7:01 p.m.
New subject: [PATCH 4/9] Change placeholders to jinja placeholders
---
share/README.lorry-controller | 2 +-
share/etc/cgitrc | 6 +++---
share/etc/gitano-setup.clod | 4 ++--
share/etc/lorry.conf | 2 +-
share/gitano/skel/gitano-admin/global-hooks/post-receive.lua | 6 +++---
share/gitano/skel/gitano-admin/rules/createrepo.lace | 2 +-
share/gitano/skel/gitano-admin/rules/defines.lace | 8 ++++----
share/gitano/skel/gitano-admin/rules/other-project.lace | 6 +++---
share/gitano/skel/gitano-admin/rules/trove-project.lace | 2 +-
share/gitano/skel/gitano-admin/users/distbuild/user.conf | 2 +-
share/gitano/skel/gitano-admin/users/lorry/user.conf | 2 +-
share/gitano/skel/gitano-admin/users/mason/user.conf | 2 +-
share/lorry-controller.conf | 8 ++++----
share/releases-repo-README | 4 ++--
14 files changed, 28 insertions(+), 28 deletions(-)
diff --git a/share/README.lorry-controller b/share/README.lorry-controller
index 1c70617..3bd0a90 100644
--- a/share/README.lorry-controller
+++ b/share/README.lorry-controller
@@ -14,5 +14,5 @@ scenarios regarding adding external software to your Trove before
attempting to
add any additional configuration to this repository.
Remember, the Lorry tool is not permitted to manage repositories inside your
-prefix which is ##PREFIX##.
+prefix which is {{ TROVE_ID }}.
diff --git a/share/etc/cgitrc b/share/etc/cgitrc
index c526e17..28540dd 100644
--- a/share/etc/cgitrc
+++ b/share/etc/cgitrc
@@ -1,4 +1,4 @@
-clone-prefix=git://##TROVE_HOSTNAME## http://##TROVE_HOSTNAME##/git
https://##TROVE_HOSTNAME##/git ssh://git@##TROVE_HOSTNAME##
+clone-prefix=git://{{ TROVE_HOSTNAME }} http://{{ TROVE_HOSTNAME }}/git https://{{
TROVE_HOSTNAME }}/git ssh://git@{{ TROVE_HOSTNAME }}
strict-export=git-daemon-export-ok
css=/cgit/cgit.css
@@ -8,8 +8,8 @@ head-include=/etc/cgit-trove-head.inc
footer=/etc/cgit-trove-footer.inc
enable-index-links=1
-root-title=##TROVE_TITLE## Git Repositories
-root-desc=Baserock Trove -- For ##TROVE_COMPANY##
+root-title={{ TROVE_ID }} Git Repositories
+root-desc=Baserock Trove -- For {{ TROVE_COMPANY }}
snapshots=tar.gz
enable-commit-graph=1
enable-log-filecount=1
diff --git a/share/etc/gitano-setup.clod b/share/etc/gitano-setup.clod
index b63aeb6..6139c4e 100644
--- a/share/etc/gitano-setup.clod
+++ b/share/etc/gitano-setup.clod
@@ -10,8 +10,8 @@ admin.realname "Trove Instance Administrator"
admin.email "trove@trove-instance"
admin.keyname "trove"
-site.name "##TROVE_TITLE## for ##TROVE_COMPANY##"
-log.prefix "##TROVE_LOG_PREFIX##"
+site.name "{{ TROVE_ID }} for {{ TROVE_COMPANY }}"
+log.prefix "{{ TROVE_ID }}"
use.htpasswd "yes"
diff --git a/share/etc/lorry.conf b/share/etc/lorry.conf
index 16552cb..cc94e8d 100644
--- a/share/etc/lorry.conf
+++ b/share/etc/lorry.conf
@@ -1,6 +1,6 @@
[config]
mirror-base-url-push = ssh://git@localhost
-mirror-base-url-fetch = git://##TROVE_HOSTNAME##
+mirror-base-url-fetch = git://{{ TROVE_HOSTNAME }}
bundle = never
bundle-dest = /home/lorry/bundles
tarball = always
diff --git a/share/gitano/skel/gitano-admin/global-hooks/post-receive.lua
b/share/gitano/skel/gitano-admin/global-hooks/post-receive.lua
index d1b3864..c7ab051 100644
--- a/share/gitano/skel/gitano-admin/global-hooks/post-receive.lua
+++ b/share/gitano/skel/gitano-admin/global-hooks/post-receive.lua
@@ -14,11 +14,11 @@ local project_hook, repo, updates = ...
local EMPTY_SHA = ("0"):rep(40)
-local masonhost = "##MASON_HOST##:##MASON_PORT##"
+local masonhost = "{{ MASON_ID }}:{{ MASON_PORT }}"
local basepath = "/1.0"
local urlbases = {
- "git://##TROVE_HOSTNAME##/",
- "ssh://git@##TROVE_HOSTNAME##/",
+ "git://{{ TROVE_HOSTNAME }}/",
+ "ssh://git@{{ TROVE_HOSTNAME }}/",
}
local notify_mason = false
diff --git a/share/gitano/skel/gitano-admin/rules/createrepo.lace
b/share/gitano/skel/gitano-admin/rules/createrepo.lace
index bf4683e..a07a744 100644
--- a/share/gitano/skel/gitano-admin/rules/createrepo.lace
+++ b/share/gitano/skel/gitano-admin/rules/createrepo.lace
@@ -11,7 +11,7 @@
# Administrators have already been permitted whatever they like
# so this is for site-wide non-admins.
-##PEOPLE_COMMENT##allow "Personal repo creation is okay" repo_is_personal
+{{ PEOPLE_COMMENT }}allow "Personal repo creation is okay" repo_is_personal
# Allow people in *-admins to create repositories under <foo>
allow "Project admins may make project repositories" repo_is_local_project
project_admin
diff --git a/share/gitano/skel/gitano-admin/rules/defines.lace
b/share/gitano/skel/gitano-admin/rules/defines.lace
index 380948a..466ac6f 100644
--- a/share/gitano/skel/gitano-admin/rules/defines.lace
+++ b/share/gitano/skel/gitano-admin/rules/defines.lace
@@ -83,9 +83,9 @@ define is_admin_ref ref refs/gitano/admin
#
#
-define repo_is_personal repository ~^##ESC_PERSONAL_PREFIX##/${user}/
-define ref_is_personal ref ~^refs/heads/##ESC_PREFIX##/${user}/
-define repo_is_local_project repository ~^##ESC_PREFIX##/[^/]+/
+define repo_is_personal repository ~^{{ ESC_PERSONAL_PREFIX }}/${user}/
+define ref_is_personal ref ~^refs/heads/{{ ESC_PREFIX }}/${user}/
+define repo_is_local_project repository ~^{{ ESC_PREFIX }}/[^/]+/
define project_reader group ${repository/2}-readers
define project_writer group ${repository/2}-writers
@@ -100,7 +100,7 @@ define trove_site_admin group trove-admin
define target_group_gitano_admin targetgroup gitano-admin
define is_lorry user lorry
-define is_local_ref ref ~^refs/heads/##ESC_PREFIX##/
+define is_local_ref ref ~^refs/heads/{{ ESC_PREFIX }}/
define lorryable_repo allof !repo_is_local_project !repo_is_personal !is_admin_repo
define is_worker group workers
diff --git a/share/gitano/skel/gitano-admin/rules/other-project.lace
b/share/gitano/skel/gitano-admin/rules/other-project.lace
index 7bc80cc..e5f05be 100644
--- a/share/gitano/skel/gitano-admin/rules/other-project.lace
+++ b/share/gitano/skel/gitano-admin/rules/other-project.lace
@@ -6,7 +6,7 @@
#
# Copyright 2012,2013 Codethink Limited
#
-# Rules for any repository not under ##PREFIX##
+# Rules for any repository not under {{ TROVE_ID }}
# This is, by default, /baserock/ and /delta/
@@ -15,11 +15,11 @@ allow "Anyone may read here" op_read
allow "Anyone may write here" op_write !is_anonymous
# Lorry can do anything reffy which is not inside the local refs
-allow "Lorry may touch everything but refs/heads/##PREFIX##" op_is_reffy
is_lorry !is_local_ref
+allow "Lorry may touch everything but refs/heads/{{ TROVE_ID }}" op_is_reffy
is_lorry !is_local_ref
# Noone can rewind/rebase outside of their personal refs
deny "Non-personal branches may not be rewound/rebased" op_forcedupdate
!is_lorry !ref_is_personal
-# Everyone else can do reffy things inside refs/heads/##PREFIX##
+# Everyone else can do reffy things inside refs/heads/{{ TROVE_ID }}
allow "Project writers may alter any refs" op_is_reffy !is_lorry is_local_ref
diff --git a/share/gitano/skel/gitano-admin/rules/trove-project.lace
b/share/gitano/skel/gitano-admin/rules/trove-project.lace
index 383ba98..c13b307 100644
--- a/share/gitano/skel/gitano-admin/rules/trove-project.lace
+++ b/share/gitano/skel/gitano-admin/rules/trove-project.lace
@@ -6,7 +6,7 @@
#
# Copyright 2012,2013 Codethink Limited
#
-# Rules for ##PREFIX##/... repositories
+# Rules for {{ TROVE_ID }}/... repositories
# Reading the repository
allow "Project readers may read" op_read project_reader
diff --git a/share/gitano/skel/gitano-admin/users/distbuild/user.conf
b/share/gitano/skel/gitano-admin/users/distbuild/user.conf
index 62ac3f5..6954826 100644
--- a/share/gitano/skel/gitano-admin/users/distbuild/user.conf
+++ b/share/gitano/skel/gitano-admin/users/distbuild/user.conf
@@ -1,2 +1,2 @@
-email_address "distbuild@##TROVE_HOSTNAME##"
+email_address "distbuild@{{ TROVE_HOSTNAME }}"
real_name "Baserock Distributed Build Service"
diff --git a/share/gitano/skel/gitano-admin/users/lorry/user.conf
b/share/gitano/skel/gitano-admin/users/lorry/user.conf
index f21fac7..d00b635 100644
--- a/share/gitano/skel/gitano-admin/users/lorry/user.conf
+++ b/share/gitano/skel/gitano-admin/users/lorry/user.conf
@@ -1,2 +1,2 @@
-email_address "lorry@##TROVE_HOSTNAME##"
+email_address "lorry@{{ TROVE_HOSTNAME }}"
real_name "Source Code Lorry Service"
diff --git a/share/gitano/skel/gitano-admin/users/mason/user.conf
b/share/gitano/skel/gitano-admin/users/mason/user.conf
index 639de4e..3139295 100644
--- a/share/gitano/skel/gitano-admin/users/mason/user.conf
+++ b/share/gitano/skel/gitano-admin/users/mason/user.conf
@@ -1,2 +1,2 @@
-email_address "mason@##TROVE_HOSTNAME##"
+email_address "mason@{{ TROVE_HOSTNAME }}"
real_name "Baserock Continuous Integration Service"
diff --git a/share/lorry-controller.conf b/share/lorry-controller.conf
index bdbbbd5..0c90cc4 100644
--- a/share/lorry-controller.conf
+++ b/share/lorry-controller.conf
@@ -1,9 +1,9 @@
[
{
"type": "trove",
- "uuid": "##PREFIX##/initial",
+ "uuid": "{{ TROVE_ID }}/initial",
"serial": 1,
- "trovehost": "##UPSTREAM_TROVE##",
+ "trovehost": "{{ UPSTREAM_TROVE }}",
"protocol": "ssh",
"ls-interval": "4H",
"interval": "2H",
@@ -21,7 +21,7 @@
},
{
"type": "lorries",
- "uuid": "##PREFIX##/open-source-lorries",
+ "uuid": "{{ TROVE_ID }}/open-source-lorries",
"serial": 1,
"interval": "6H",
"create": "always",
@@ -35,7 +35,7 @@
},
{
"type": "lorries",
- "uuid": "##PREFIX##/closed-source-lorries",
+ "uuid": "{{ TROVE_ID }}/closed-source-lorries",
"serial": 1,
"interval": "6H",
"create": "always",
diff --git a/share/releases-repo-README b/share/releases-repo-README
index d3f872b..69ee875 100644
--- a/share/releases-repo-README
+++ b/share/releases-repo-README
@@ -2,10 +2,10 @@ site/releases repository
------------------------
This is a special repository for distributing release binaries over HTTP.
-Visit http://##PREFIX##/releases/ to browse content.
+Visit http://{{ TROVE_ID }}/releases/ to browse content.
To add a release to this repository, you need to be a member of the
Gitano group site-writers. With the correct permissions, you can push
releases to the repository by doing:
- rsync $RELEASE git@##PREFIX##:##PREFIX##/site/releases
+ rsync $RELEASE git@{{ TROVE_HOSTNAME }}:{{ TROVE_ID }}/site/releases
--
1.7.10.4
7:01 p.m.
New subject: [PATCH 5/9] Do not enable the units when installing
---
Makefile | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/Makefile b/Makefile
index 0559468..52e3106 100644
--- a/Makefile
+++ b/Makefile
@@ -1,9 +1,7 @@
install:
mkdir -p "${DESTDIR}/usr/lib/systemd/system/multi-user.target.wants"
cp units/* "${DESTDIR}/usr/lib/systemd/system"
- for I in $$(cd units; ls); do \
- ln -sf ../$$I
"${DESTDIR}/usr/lib/systemd/system/multi-user.target.wants/$$I"; \
- done
+ ln -sf ../trove-setup.service
"${DESTDIR}/usr/lib/systemd/system/multi-user.target.wants/trove-setup.service"
cp -r etc "${DESTDIR}"
mkdir -p "${DESTDIR}/var/www/htdocs"
cp http-assets/* "${DESTDIR}/var/www/htdocs"
--
1.7.10.4
11:43 a.m.
New subject: [PATCH 5/9] Do not enable the units when installing
I think this could do with a commit message explaining why. It might
make it clearer how the new system works.
As I understand it, Ansible now starts all the Trove-related systemd
units manually once it's created / verified the configuration, so we
don't need to link them into one of the targets that is started on boot
automatically anymore. This is pretty good because it means we don't
need to reboot after configuring the Trove, too.
On 01/07/14 18:01, Pedro Alvarez wrote:
---
Makefile | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/Makefile b/Makefile
index 0559468..52e3106 100644
--- a/Makefile
+++ b/Makefile
@@ -1,9 +1,7 @@
install:
mkdir -p "${DESTDIR}/usr/lib/systemd/system/multi-user.target.wants"
cp units/* "${DESTDIR}/usr/lib/systemd/system"
- for I in $$(cd units; ls); do \
- ln -sf ../$$I
"${DESTDIR}/usr/lib/systemd/system/multi-user.target.wants/$$I"; \
- done
+ ln -sf ../trove-setup.service
"${DESTDIR}/usr/lib/systemd/system/multi-user.target.wants/trove-setup.service"
cp -r etc "${DESTDIR}"
mkdir -p "${DESTDIR}/var/www/htdocs"
cp http-assets/* "${DESTDIR}/var/www/htdocs"
--
Sam Thursfield, Codethink Ltd.
Office telephone: +44 161 236 5575
3:12 p.m.
New subject: [PATCH 5/9] Do not enable the units when installing
On Wed, Jul 02, 2014 at 10:43:45AM +0100, Sam Thursfield wrote:
I think this could do with a commit message explaining why. It might
make it clearer how the new system works.
As I understand it, Ansible now starts all the Trove-related systemd
units manually once it's created / verified the configuration, so we
don't need to link them into one of the targets that is started on
boot automatically anymore. This is pretty good because it means we
don't need to reboot after configuring the Trove, too.
More precisely, they aren't eligible to be started until they are
configured, and Ansible handles both the initial start, and configuring
it to start automatically on next boot.
3:25 p.m.
New subject: [PATCH 5/9] Do not enable the units when installing
On 02/07/14 10:43, Sam Thursfield wrote:
I think this could do with a commit message explaining why. It might
make it
clearer how the new system works.
As I understand it, Ansible now starts all the Trove-related systemd units
manually once it's created / verified the configuration, so we don't need to
link them into one of the targets that is started on boot automatically
anymore. This is pretty good because it means we don't need to reboot after
configuring the Trove, too.
You are absolutely right with your understanding, and I agree that a better
commit message would be nice. I will add an explanation of why I do this in the
commit.
Thanks,
Pedro.
7:01 p.m.
New subject: [PATCH 6/9] Add new resources needed to configure the lorry-controller
They where generated in trove.configure before.
---
share/etc/lorry-controller/minion.conf | 6 ++++++
share/etc/lorry-controller/webapp.conf | 12 ++++++++++++
2 files changed, 18 insertions(+)
create mode 100644 share/etc/lorry-controller/minion.conf
create mode 100644 share/etc/lorry-controller/webapp.conf
diff --git a/share/etc/lorry-controller/minion.conf
b/share/etc/lorry-controller/minion.conf
new file mode 100644
index 0000000..99abdba
--- /dev/null
+++ b/share/etc/lorry-controller/minion.conf
@@ -0,0 +1,6 @@
+[config]
+log = syslog
+log-level = debug
+webapp-host = localhost
+webapp-port = 12765
+webapp-timeout = 3600
diff --git a/share/etc/lorry-controller/webapp.conf
b/share/etc/lorry-controller/webapp.conf
new file mode 100644
index 0000000..2e9df0d
--- /dev/null
+++ b/share/etc/lorry-controller/webapp.conf
@@ -0,0 +1,12 @@
+[config]
+log = /home/lorry/webapp.log
+log-max = 100M
+log-keep = 10
+log-level = debug
+statedb = /home/lorry/webapp.db
+configuration-directory = /home/lorry/confgit
+status-html = /home/lorry/lc-status.html
+wsgi = yes
+debug-port = 12765
+templates = /usr/share/lorry-controller/templates
+confgit-url = ssh://git@localhost/{{ TROVE_ID }}/local-config/lorries
--
1.7.10.4
7:02 p.m.
New subject: [PATCH 7/9] Add Ansible scripts
---
ansible/hosts | 1 +
ansible/roles/trove-setup/tasks/backups.yml | 17 ++++
ansible/roles/trove-setup/tasks/cache-setup.yml | 12 +++
ansible/roles/trove-setup/tasks/check.yml | 78 +++++++++++++++
ansible/roles/trove-setup/tasks/git.yml | 8 ++
.../roles/trove-setup/tasks/gitano-admin-setup.yml | 39 ++++++++
.../roles/trove-setup/tasks/gitano-lorry-setup.yml | 18 ++++
.../roles/trove-setup/tasks/gitano-mason-setup.yml | 16 +++
ansible/roles/trove-setup/tasks/gitano-setup.yml | 48 +++++++++
.../trove-setup/tasks/gitano-worker-setup.yml | 18 ++++
ansible/roles/trove-setup/tasks/hostname.yml | 26 +++++
.../roles/trove-setup/tasks/known-hosts-setup.yml | 7 ++
ansible/roles/trove-setup/tasks/lighttpd.yml | 51 ++++++++++
.../trove-setup/tasks/lorry-controller-setup.yml | 104 ++++++++++++++++++++
ansible/roles/trove-setup/tasks/lorry-setup.yml | 20 ++++
ansible/roles/trove-setup/tasks/main.yml | 19 ++++
ansible/roles/trove-setup/tasks/minions.yml | 15 +++
ansible/roles/trove-setup/tasks/nfs-setup.yml | 7 ++
ansible/roles/trove-setup/tasks/releases.yml | 26 +++++
ansible/roles/trove-setup/tasks/site-groups.yml | 88 +++++++++++++++++
ansible/roles/trove-setup/tasks/users.yml | 38 +++++++
ansible/trove-setup.yml | 6 ++
22 files changed, 662 insertions(+)
create mode 100644 ansible/hosts
create mode 100644 ansible/roles/trove-setup/tasks/backups.yml
create mode 100644 ansible/roles/trove-setup/tasks/cache-setup.yml
create mode 100644 ansible/roles/trove-setup/tasks/check.yml
create mode 100644 ansible/roles/trove-setup/tasks/git.yml
create mode 100644 ansible/roles/trove-setup/tasks/gitano-admin-setup.yml
create mode 100644 ansible/roles/trove-setup/tasks/gitano-lorry-setup.yml
create mode 100644 ansible/roles/trove-setup/tasks/gitano-mason-setup.yml
create mode 100644 ansible/roles/trove-setup/tasks/gitano-setup.yml
create mode 100644 ansible/roles/trove-setup/tasks/gitano-worker-setup.yml
create mode 100644 ansible/roles/trove-setup/tasks/hostname.yml
create mode 100644 ansible/roles/trove-setup/tasks/known-hosts-setup.yml
create mode 100644 ansible/roles/trove-setup/tasks/lighttpd.yml
create mode 100644 ansible/roles/trove-setup/tasks/lorry-controller-setup.yml
create mode 100644 ansible/roles/trove-setup/tasks/lorry-setup.yml
create mode 100644 ansible/roles/trove-setup/tasks/main.yml
create mode 100644 ansible/roles/trove-setup/tasks/minions.yml
create mode 100644 ansible/roles/trove-setup/tasks/nfs-setup.yml
create mode 100644 ansible/roles/trove-setup/tasks/releases.yml
create mode 100644 ansible/roles/trove-setup/tasks/site-groups.yml
create mode 100644 ansible/roles/trove-setup/tasks/users.yml
create mode 100644 ansible/trove-setup.yml
diff --git a/ansible/hosts b/ansible/hosts
new file mode 100644
index 0000000..5b97818
--- /dev/null
+++ b/ansible/hosts
@@ -0,0 +1 @@
+localhost ansible_connection=local
diff --git a/ansible/roles/trove-setup/tasks/backups.yml
b/ansible/roles/trove-setup/tasks/backups.yml
new file mode 100644
index 0000000..a15e4c4
--- /dev/null
+++ b/ansible/roles/trove-setup/tasks/backups.yml
@@ -0,0 +1,17 @@
+# Depends on:
+# - check.yml
+---
+- name: Create the backups user if TROVE_BACKUP_KEYS is defined
+ user: name=backup comment="Backup user" shell=/bin/sh
home=/root/backup-user-home group=root uid=0 non_unique=yes
+ when: TROVE_BACKUP_KEYS is defined
+
+- name: Creates the .ssh directory to the backups user if TROVE_BACKUP_KEYS is defined
+ file: path=/root/backup-user-home/.ssh state=directory
+ when: TROVE_BACKUP_KEYS is defined
+
+- name: Copy the TROVE_BACKUP_KEYS if defined to authorized_keys of the backup user
+ shell: |
+ cat {{ TROVE_BACKUP_KEYS }} >>
/root/backup-user-home/.ssh/authorized_keys
+ creates=/root/backup-user-home/.ssh/authorized_keys
+ when: TROVE_BACKUP_KEYS is defined
+
diff --git a/ansible/roles/trove-setup/tasks/cache-setup.yml
b/ansible/roles/trove-setup/tasks/cache-setup.yml
new file mode 100644
index 0000000..ea6ffd7
--- /dev/null
+++ b/ansible/roles/trove-setup/tasks/cache-setup.yml
@@ -0,0 +1,12 @@
+# Depends on:
+# - users.yml
+---
+- name: Create artifacts and ccache folder for the cache user
+ file: path=/home/cache/{{ item }} state=directory owner=cache group=cache
+ with_items:
+ - artifacts
+ - ccache
+- name: Create /etc/exports.cache
+ shell: |
+ echo '/home/cache/ccache
*(rw,all_squash,no_subtree_check,anonuid=1002,anongid=1002)' > /etc/exports.cache
+ creates=/etc/exports.cache
diff --git a/ansible/roles/trove-setup/tasks/check.yml
b/ansible/roles/trove-setup/tasks/check.yml
new file mode 100644
index 0000000..8827ea5
--- /dev/null
+++ b/ansible/roles/trove-setup/tasks/check.yml
@@ -0,0 +1,78 @@
+---
+- fail: msg='TROVE_ID is mandatory'
+ when: TROVE_ID is not defined
+
+- fail: msg='TROVE_COMPANY is mandatory'
+ when: TROVE_COMPANY is not defined
+
+- fail: msg='TROVE_ADMIN_USER is mandatory'
+ when: TROVE_ADMIN_USER is not defined
+
+- fail: msg='TROVE_ADMIN_EMAIL is mandatory'
+ when: TROVE_ADMIN_EMAIL is not defined
+
+- fail: msg='TROVE_ADMIN_NAME is mandatory'
+ when: TROVE_ADMIN_NAME is not defined
+
+- fail: msg='LORRY_SSH_KEY is mandatory'
+ when: LORRY_SSH_KEY is not defined
+
+- fail: msg='LORRY_SSH_PUBKEY is mandatory'
+ when: LORRY_SSH_PUBKEY is not defined
+
+- fail: msg='TROVE_ADMIN_SSH_PUBKEY is mandatory'
+ when: TROVE_ADMIN_SSH_PUBKEY is not defined
+
+- fail: msg='WORKER_SSH_PUBKEY is mandatory'
+ when: WORKER_SSH_PUBKEY is not defined
+
+- fail: msg='MASON_SSH_PUBKEY is mandatory'
+ when: MASON_SSH_PUBKEY is not defined
+
+- fail: msg='UPSTREAM_TROVE is mandatory'
+ when: UPSTREAM_TROVE is not defined
+
+- set_fact: TROVE_HOSTNAME={{ TROVE_ID }}
+ when: TROVE_HOSTNAME is not defined
+
+- set_fact: LORRY_CONTROLLER_MINIONS=4
+ when: LORRY_CONTROLLER_MINIONS is not defined
+
+- set_fact: MASON_ID=''
+ when: MASON_ID is not defined
+
+- set_fact: MASON_PORT='18755'
+ when: MASON_PORT is not defined
+
+- name: Calculate ESC_PREFIX
+ shell: echo -n "{{ TROVE_ID }}" | perl -pe
's/([-+\(\).%*?^$\[\]])/%$1/g'
+ register: var_esc_prefix
+ changed_when: False
+
+- set_fact: ESC_PREFIX={{ var_esc_prefix.stdout }}
+
+- set_fact: ESC_PERSONAL_PREFIX='people'
+
+- set_fact: PEOPLE_COMMENT='#'
+
+- name: Check if the ssh keys are valid
+ shell: ssh-keygen -l -f {{ item }}
+ with_items:
+ - '{{ TROVE_ADMIN_SSH_PUBKEY }}'
+ - '{{ LORRY_SSH_PUBKEY }}'
+ - '{{ MASON_SSH_PUBKEY }}'
+ - '{{ WORKER_SSH_PUBKEY }}'
+ changed_when: False
+
+- name: Check if the ssh keys are unique
+ shell: |
+ cat {{ TROVE_ADMIN_SSH_PUBKEY }} \
+ {{ LORRY_SSH_PUBKEY }} \
+ {{ MASON_SSH_PUBKEY }} \
+ {{ WORKER_SSH_PUBKEY }} \
+ | cut -d ' ' -f 1,2 | sort -u | wc -l
+ changed_when: False
+ register: number_ssh_keys
+
+- fail: msg="The ssh keys MUST be different"
+ when: number_ssh_keys.stdout != '4'
diff --git a/ansible/roles/trove-setup/tasks/git.yml
b/ansible/roles/trove-setup/tasks/git.yml
new file mode 100644
index 0000000..c083541
--- /dev/null
+++ b/ansible/roles/trove-setup/tasks/git.yml
@@ -0,0 +1,8 @@
+# Depends on:
+# - users.yml
+---
+- name: Configure Git user.name and usr.email
+ shell: |
+ su -c 'git config --global user.name "Trove Git Controller"' -
git
+ su -c 'git config --global user.email "git@trove"' - git
+ creates=/home/git/.gitconfig
diff --git a/ansible/roles/trove-setup/tasks/gitano-admin-setup.yml
b/ansible/roles/trove-setup/tasks/gitano-admin-setup.yml
new file mode 100644
index 0000000..5c9f13a
--- /dev/null
+++ b/ansible/roles/trove-setup/tasks/gitano-admin-setup.yml
@@ -0,0 +1,39 @@
+# Depends on:
+# - gitano-setup.yml
+---
+- name: Check if the admin user is configured in gitano (This task can fail)
+ shell: su -c 'ssh git@localhost user' - git | grep '^{{ TROVE_ADMIN_USER
}}:'
+ register: gitano_admin_user
+ changed_when: False
+ ignore_errors: True
+# If the admin user doesn't exist
+- name: Create the admin user
+ shell: su git -c 'ssh git@localhost user add {{ TROVE_ADMIN_USER }} {{
TROVE_ADMIN_EMAIL }} {{ TROVE_ADMIN_NAME }}'
+ when: gitano_admin_user|failed
+
+- name: Check if admin user is in trove-admin group in gitano (This task can fail)
+ shell: su -c 'ssh git@localhost as {{ TROVE_ADMIN_USER }} whoami' - git | grep
'trove-admin. Trove-local administration'
+ register: gitano_admin_group
+ changed_when: False
+ ignore_errors: True
+# If the admin user is not in the trove-admin group
+- name: Add the admin user to the trove-admin group in gitano
+ shell: su -c 'ssh git@localhost group adduser trove-admin {{ TROVE_ADMIN_USER
}}' - git
+ when: gitano_admin_group|failed
+
+- name: Check if admin user has a sshkey configured in gitano (This task can fail)
+ shell: su -c 'ssh git@localhost as {{ TROVE_ADMIN_USER }} sshkey' - git 2>
/dev/stdout | grep WARNING
+ register: gitano_admin_key
+ changed_when: False
+ ignore_errors: True
+# If admin user doesn't have an sshkey configured
+- name: Create /home/git/keys/ to store sshkeys
+ file: path=/home/git/keys state=directory owner=git group=git
+ when: gitano_admin_key|success
+- name: Copy the TROVE_ADMIN_SSH_PUBKEY to /home/git/keys/admin.key.pub
+ copy: src={{ TROVE_ADMIN_SSH_PUBKEY }} dest=/home/git/keys/admin.key.pub mode=0644
+ when: gitano_admin_key|success
+
+- name: Add /home/git/keys/admin.key.pub ssh key to the admin user in gitano.
+ shell: su -c 'ssh git@localhost as {{ TROVE_ADMIN_USER }} sshkey add default <
/home/git/keys/admin.key.pub' - git
+ when: gitano_admin_key|success
diff --git a/ansible/roles/trove-setup/tasks/gitano-lorry-setup.yml
b/ansible/roles/trove-setup/tasks/gitano-lorry-setup.yml
new file mode 100644
index 0000000..b2681dd
--- /dev/null
+++ b/ansible/roles/trove-setup/tasks/gitano-lorry-setup.yml
@@ -0,0 +1,18 @@
+# Depends on:
+# - gitano-setup.yml
+---
+- name: Check if lorry has a sshkey configured in gitano (This task can fail)
+ shell: su -c 'ssh git@localhost as lorry sshkey' - git 2> /dev/stdout | grep
WARNING
+ register: gitano_lorry_key
+ changed_when: False
+ ignore_errors: True
+# If lorry user doesn't have an sshkey configured
+- name: Create /home/git/keys folder to store ssh keys
+ file: path=/home/git/keys state=directory owner=git group=git
+ when: gitano_lorry_key|success
+- name: Copy LORRY_SSH_PUBKEY to /home/git/keys/lorry.key.pub
+ copy: src={{ LORRY_SSH_PUBKEY }} dest=/home/git/keys/lorry.key.pub mode=0644
+ when: gitano_lorry_key|success
+- name: Add to the gitano lorry user the /home/git/keys/lorry.key.pub
+ shell: su -c 'ssh git@localhost as lorry sshkey add trove <
/home/git/keys/lorry.key.pub' - git
+ when: gitano_lorry_key|success
diff --git a/ansible/roles/trove-setup/tasks/gitano-mason-setup.yml
b/ansible/roles/trove-setup/tasks/gitano-mason-setup.yml
new file mode 100644
index 0000000..edfd873
--- /dev/null
+++ b/ansible/roles/trove-setup/tasks/gitano-mason-setup.yml
@@ -0,0 +1,16 @@
+# Depends on:
+# - gitano-setup.yml
+---
+- name: Check if mason has a sshkey configured in gitano (This task can fail)
+ shell: su -c 'ssh git@localhost as mason sshkey' - git 2> /dev/stdout | grep
WARNING
+ register: gitano_mason_key
+ changed_when: False
+ ignore_errors: True
+
+# If distbuild user doesn't have an sshkey configured
+- file: path=/home/git/keys state=directory owner=git group=git
+ when: gitano_mason_key|success
+- copy: src={{ MASON_SSH_PUBKEY }} dest=/home/git/keys/worker.key.pub mode=0644
+ when: gitano_mason_key|success
+- shell: su -c 'ssh git@localhost as mason sshkey add trove <
/home/git/keys/mason.key.pub' - git
+ when: gitano_mason_key|success
diff --git a/ansible/roles/trove-setup/tasks/gitano-setup.yml
b/ansible/roles/trove-setup/tasks/gitano-setup.yml
new file mode 100644
index 0000000..a7b7903
--- /dev/null
+++ b/ansible/roles/trove-setup/tasks/gitano-setup.yml
@@ -0,0 +1,48 @@
+# Depends on:
+# - git.yml
+---
+# Before configuring Gitano, it's necessary to modify the placeholders
+# of the skeleton template of Gitano with the values of /etc/trove/trove.conf.
+# Ansible does not provide an efficient way to do this. Its template module
+# is not able to run recursively over directories, and is not able to create
+# the directories needed.
+#
+# The solution implemented consists in create the directories first and then
+# using the template module in all the files. This could be possible to
+# implement using the 'with_lines' option combinated with the 'find'
command.
+#
+# Create the directories
+- name: Create the directories needed for the Gitano skeleton.
+ file: path=/etc/{{ item }} state=directory
+ with_lines:
+ - (cd /usr/share/trove-setup && find gitano -type d)
+# Copy all the files to the right place and fill the templates whenever possible
+- name: Create the Gitano skeleton using the templates
+ template: src=/usr/share/trove-setup/{{ item }} dest=/etc/{{ item }}
+ with_lines:
+ - (cd /usr/share/trove-setup && find gitano -type f)
+
+# Configure gitano
+- name: Configure Gitano with /etc/gitano-setup.clod
+ command: |
+ /bin/su -c 'gitano-setup /etc/gitano-setup.clod' - git
+ creates=/home/git/repos/gitano-admin.git
+
+- name: Unlock the password of the git user (This task can fail)
+ shell: busybox passwd -u git
+ register: passwd_result
+ changed_when: passwd_result|success
+ ignore_errors: True
+
+# Now that /home/git/repos exists, we can enable the git-daemon service
+- name: Enable the git-daemon.service
+ shell: |
+ ln -s "/usr/lib/systemd/system/git-daemon.service" \
+ "/etc/systemd/system/multi-user.target.wants/git-daemon.service"
+ creates=/etc/systemd/system/multi-user.target.wants/git-daemon.service
+ register: git_daemon_service
+
+# Now we can start the service without rebooting the system
+- name: Restart git-daemon.service
+ service: name=git-daemon state=restarted
+ when: git_daemon_service|changed
diff --git a/ansible/roles/trove-setup/tasks/gitano-worker-setup.yml
b/ansible/roles/trove-setup/tasks/gitano-worker-setup.yml
new file mode 100644
index 0000000..a44adc0
--- /dev/null
+++ b/ansible/roles/trove-setup/tasks/gitano-worker-setup.yml
@@ -0,0 +1,18 @@
+# Depends on:
+# - gitano-setup.yml
+---
+- name: Check if worker has a sshkey configured in gitano (This task can fail)
+ shell: su -c 'ssh git@localhost as distbuild sshkey' - git 2> /dev/stdout |
grep WARNING
+ register: gitano_worker_key
+ changed_when: False
+ ignore_errors: True
+# If distbuild user doesn't have an sshkey configured
+- name: Create /home/git/keys/ to store ssh keys
+ file: path=/home/git/keys state=directory owner=git group=git
+ when: gitano_worker_key|success
+- name: Copy WORKER_SSH_PUBKEY to /home/git/keys/worker.key.pub
+ copy: src={{ WORKER_SSH_PUBKEY }} dest=/home/git/keys/worker.key.pub mode=0644
+ when: gitano_worker_key|success
+- name: Add /home/git/keys/worker.key.pub to the distbuild user in Gitano
+ shell: su -c 'ssh git@localhost as distbuild sshkey add trove <
/home/git/keys/worker.key.pub' - git
+ when: gitano_worker_key|success
diff --git a/ansible/roles/trove-setup/tasks/hostname.yml
b/ansible/roles/trove-setup/tasks/hostname.yml
new file mode 100644
index 0000000..513bd89
--- /dev/null
+++ b/ansible/roles/trove-setup/tasks/hostname.yml
@@ -0,0 +1,26 @@
+# Depends on:
+# - check.yml
+---
+- name: Check the /etc/hostname and compare it with HOSTNAME (This task can fail)
+ shell: if [ "$(cat /etc/hostname)" != "{{ HOSTNAME }}" ]; then exit
1; fi
+ register: hostname_file
+ ignore_errors: True
+ changed_when: False
+ when: HOSTNAME is defined
+
+# If /etc/hostname doesn't match with HOSTNAME
+- name: Rewrite /etc/hostname with HOSTNAME
+ shell: echo {{ HOSTNAME }} > /etc/hostname
+ when: hostname_file|failed
+
+- name: Check the actual hostname with `hostname` and compare it with HOSTNAME (This task
can fail)
+ shell: if [ "$(hostname)" != "{{ HOSTNAME }}" ]; then exit 1; fi
+ register: actual_hostname
+ ignore_errors: True
+ changed_when: False
+ when: HOSTNAME is defined
+
+# If `hostname` doesn't match with HOSTNAME
+- name: Change the hostname to HOSTNAME
+ shell: hostname "{{ HOSTNAME }}"
+ when: actual_hostname|failed
diff --git a/ansible/roles/trove-setup/tasks/known-hosts-setup.yml
b/ansible/roles/trove-setup/tasks/known-hosts-setup.yml
new file mode 100644
index 0000000..9f91ffa
--- /dev/null
+++ b/ansible/roles/trove-setup/tasks/known-hosts-setup.yml
@@ -0,0 +1,7 @@
+# Depends on:
+# - check.yml
+---
+- name: Add localhost and UPSTREAM_TROVE to /etc/ssh/ssh_known_hosts
+ shell: |
+ /bin/sh -c "ssh-keyscan localhost {{ UPSTREAM_TROVE }} >
/etc/ssh/ssh_known_hosts"
+ creates=/etc/ssh/ssh_known_hosts
diff --git a/ansible/roles/trove-setup/tasks/lighttpd.yml
b/ansible/roles/trove-setup/tasks/lighttpd.yml
new file mode 100644
index 0000000..105e307
--- /dev/null
+++ b/ansible/roles/trove-setup/tasks/lighttpd.yml
@@ -0,0 +1,51 @@
+---
+- name: Create /etc/lighttpd/certs directory
+ file: path=/etc/lighttpd/certs state=directory
+- name: Create certificates for lighttpd in /etc/lighttpd/certs/lighttpd.pem
+ shell: |
+ yes '' | openssl req -new -x509 \
+ -keyout /etc/lighttpd/certs/lighttpd.pem \
+ -out /etc/lighttpd/certs/lighttpd.pem -days 36525 -nodes
+ creates=/etc/lighttpd/certs/lighttpd.pem
+ register: lighttpd_certs
+- name: Create /var/run/lighttpd for cache user
+ file: path=/var/run/lighttpd state=directory owner=cache group=cache
+ register: lighttpd_folder
+
+# Now that the lighttpd certificates and the /var/run/lighttpd exist, we can
+# enable the lighttpd-git service
+- name: Enable lighttpd-git service
+ shell: |
+ ln -s "/usr/lib/systemd/system/lighttpd-git.service" \
+ "/etc/systemd/system/multi-user.target.wants/lighttpd-git.service"
+ creates=/etc/systemd/system/multi-user.target.wants/lighttpd-git.service
+ register: lighttpd_git_service
+
+# Now we can start the service without rebooting the system
+- name: Restart the lighttpd-git service
+ service: name=lighttpd-git state=restarted
+ when: lighttpd_git_service|changed
+
+# Once the service lighttpd-git is running it's possible to do the same
+# with the following services:
+# - lighttpd-morph-cache
+# - lighttpd-lorry-controller-webapp
+- name: Enable lighttpd-morph-cache service
+ shell: |
+ ln -s "/usr/lib/systemd/system/lighttpd-morph-cache.service" \
+
"/etc/systemd/system/multi-user.target.wants/lighttpd-morph-cache.service"
+
creates=/etc/systemd/system/multi-user.target.wants/lighttpd-morph-cache.service
+ register: lighttpd_morph_cache_service
+- name: Restart the lighttpd-morph-cache service
+ service: name=lighttpd-morph-cache state=restarted
+ when: lighttpd_morph_cache_service|changed
+
+- name: Enable the lighttpd-lorry-controller-webapp service
+ shell: |
+ ln -s
"/usr/lib/systemd/system/lighttpd-lorry-controller-webapp.service" \
+
"/etc/systemd/system/multi-user.target.wants/lighttpd-lorry-controller-webapp.service"
+
creates=/etc/systemd/system/multi-user.target.wants/lighttpd-lorry-controller-webapp.service
+ register: lighttpd_lorry_controller_webapp_service
+- name: Restart the lighttpd-lorry-controller-webapp service
+ service: name=lighttpd-lorry-controller-webapp state=restarted
+ when: lighttpd_lorry_controller_webapp_service|changed
diff --git a/ansible/roles/trove-setup/tasks/lorry-controller-setup.yml
b/ansible/roles/trove-setup/tasks/lorry-controller-setup.yml
new file mode 100644
index 0000000..0e179d1
--- /dev/null
+++ b/ansible/roles/trove-setup/tasks/lorry-controller-setup.yml
@@ -0,0 +1,104 @@
+# Depends on:
+# - gitano-setup.yml
+# - lighttpd.yml
+---
+- name: Create the TROVE_ID/local-config/lorries repository
+ command: |
+ /bin/su -c "ssh localhost create {{ TROVE_ID
}}/local-config/lorries" - git
+ creates=/home/git/repos/{{ TROVE_ID }}/local-config/lorries.git
+- name: Create a temporary folder to copy templates
+ file: path=/tmp/lorries-tmp state=directory
+- name: Create the configuration files of lorry-controller using templates
+ template: src=/usr/share/trove-setup/{{ item }} dest=/tmp/lorries-tmp/{{ item }}
+ with_items:
+ - lorry-controller.conf
+ - README.lorry-controller
+- name: Configure the lorry-controller
+ shell: |
+ su -c "git clone ssh://localhost/{{ TROVE_ID }}/local-config/lorries.git
/tmp/lorries" - git
+ su -c "cp /tmp/lorries-tmp/lorry-controller.conf
/tmp/lorries/lorry-controller.conf" - git
+ su -c "cp /tmp/lorries-tmp/README.lorry-controller
/tmp/lorries/README" - git
+ su -c "mkdir /tmp/lorries/open-source-lorries" - git
+ su -c "cp /usr/share/trove-setup/open-source-lorries/README
/tmp/lorries/open-source-lorries/README" - git
+ su -c "mkdir /tmp/lorries/closed-source-lorries" - git
+ su -c "cp /usr/share/trove-setup/closed-source-lorries/README
/tmp/lorries/closed-source-lorries/README" - git
+ su -c "cd /tmp/lorries; git add README lorry-controller.conf
open-source-lorries/README closed-source-lorries/README; git commit -m 'Initial
configuration'; git push origin master" - git
+ su -c "rm -rf /tmp/lorries" - git
+ creates=/home/git/repos/{{ TROVE_ID
}}/local-config/lorries.git/refs/heads/master
+
+# Migration: Remove the old lorry-controller cronjob if exists
+- name: Look for lorry-controller old cronjob (This task can fail)
+ shell: su -c 'crontab -l | grep -e "-c lorry-controller"' - lorry
+ register: lorry_controller_cronjob
+ changed_when: False
+ ignore_errors: True
+
+- name: Remove the old lorry-controller cronjob
+ shell: su -c '/usr/libexec/remove-lorry-controller-from-lorry-crontab' - lorry
+ when: lorry_controller_cronjob|success
+
+
+# Now that the lorry-controller is configured we can enable the following
+# services and timers, and also start them
+# - lorry-controller-status
+# - lorry-controller-readconf
+# - lorry-controller-ls-troves
+- name: Enable lorry-controller-status service
+ shell: |
+ ln -s "/usr/lib/systemd/system/lorry-controller-status.service" \
+
"/etc/systemd/system/multi-user.target.wants/lorry-controller-status.service"
+
creates=/etc/systemd/system/multi-user.target.wants/lorry-controller-status.service
+ register: lorry_controller_status_service
+- name: Start lorry-controller-status service
+ service: name=lorry-controller-status.service state=restarted
+ when: lorry_controller_status_service|changed
+
+- name: Enable lorry-controller-readconf service
+ shell: |
+ ln -s "/usr/lib/systemd/system/lorry-controller-readconf.service" \
+
"/etc/systemd/system/multi-user.target.wants/lorry-controller-readconf.service"
+
creates=/etc/systemd/system/multi-user.target.wants/lorry-controller-readconf.service
+ register: lorry_controller_readconf_service
+- name: Start lorry-controller-readconf service
+ service: name=lorry-controller-readconf.service state=restarted
+ when: lorry_controller_readconf_service|changed
+
+- name: Enable lorry-controller-ls-troves service
+ shell: |
+ ln -s "/usr/lib/systemd/system/lorry-controller-ls-troves.service" \
+
"/etc/systemd/system/multi-user.target.wants/lorry-controller-ls-troves.service"
+
creates=/etc/systemd/system/multi-user.target.wants/lorry-controller-ls-troves.service
+ register: lorry_controller_ls_troves_service
+- name: Start lorry-controller-ls-troves service
+ service: name=lorry-controller-ls-troves.service state=restarted
+ when: lorry_controller_ls_troves_service|changed
+
+- name: Enable lorry-controller-status timer
+ shell: |
+ ln -s "/usr/lib/systemd/system/lorry-controller-status.timer" \
+
"/etc/systemd/system/multi-user.target.wants/lorry-controller-status.timer"
+
creates=/etc/systemd/system/multi-user.target.wants/lorry-controller-status.timer
+ register: lorry_controller_status_timer
+- name: Start lorry-controller-status timer
+ service: name=lorry-controller-status.timer state=restarted
+ when: lorry_controller_status_timer|changed
+
+- name: Enable lorry-controller-readconf timer
+ shell: |
+ ln -s "/usr/lib/systemd/system/lorry-controller-readconf.timer" \
+
"/etc/systemd/system/multi-user.target.wants/lorry-controller-readconf.timer"
+
creates=/etc/systemd/system/multi-user.target.wants/lorry-controller-readconf.timer
+ register: lorry_controller_readconf_timer
+- name: Start lorry-controller-readconf timer
+ service: name=lorry-controller-readconf.timer state=restarted
+ when: lorry_controller_readconf_timer|changed
+
+- name: Enable lorry-controller-ls-troves timer
+ shell: |
+ ln -s "/usr/lib/systemd/system/lorry-controller-ls-troves.timer" \
+
"/etc/systemd/system/multi-user.target.wants/lorry-controller-ls-troves.timer"
+
creates=/etc/systemd/system/multi-user.target.wants/lorry-controller-ls-troves.timer
+ register: lorry_controller_ls_troves_timer
+- name: Start lorry-controller-ls-troves timer
+ service: name=lorry-controller-ls-troves.timer state=restarted
+ when: lorry_controller_ls_troves_timer|changed
diff --git a/ansible/roles/trove-setup/tasks/lorry-setup.yml
b/ansible/roles/trove-setup/tasks/lorry-setup.yml
new file mode 100644
index 0000000..c50b49d
--- /dev/null
+++ b/ansible/roles/trove-setup/tasks/lorry-setup.yml
@@ -0,0 +1,20 @@
+# Depends on:
+# - users.yml
+---
+- name: Create bundles and tarballs folder for the lorry user
+ file: path=/home/lorry/{{ item }} state=directory owner=lorry group=lorry
+ with_items:
+ - bundles
+ - tarballs
+# Following the same strategy as explained in gitano-setup.yml, use
+# templates recursively over directories.
+# Create the directories needed to copy the files
+- name: Create directories needed in /etc for the lorry configuration
+ file: path=/etc/{{ item }} state=directory
+ with_lines:
+ - (cd /usr/share/trove-setup/etc && find -type d)
+# Copy all the files to the right place and fill the templates whenever possible
+- name: Add the configuration needed for lorry in /etc using templates
+ template: src=/usr/share/trove-setup/etc/{{ item }} dest=/etc/{{ item }}
+ with_lines:
+ - (cd /usr/share/trove-setup/etc && find -type f)
diff --git a/ansible/roles/trove-setup/tasks/main.yml
b/ansible/roles/trove-setup/tasks/main.yml
new file mode 100644
index 0000000..150b6f8
--- /dev/null
+++ b/ansible/roles/trove-setup/tasks/main.yml
@@ -0,0 +1,19 @@
+---
+- include: check.yml
+- include: hostname.yml
+- include: known-hosts-setup.yml
+- include: users.yml
+- include: cache-setup.yml
+- include: lighttpd.yml
+- include: lorry-setup.yml
+- include: git.yml
+- include: gitano-setup.yml
+- include: lorry-controller-setup.yml
+- include: minions.yml
+- include: site-groups.yml
+- include: releases.yml
+- include: nfs-setup.yml
+- include: gitano-worker-setup.yml
+- include: gitano-lorry-setup.yml
+- include: gitano-admin-setup.yml
+- include: backups.yml
diff --git a/ansible/roles/trove-setup/tasks/minions.yml
b/ansible/roles/trove-setup/tasks/minions.yml
new file mode 100644
index 0000000..22903d3
--- /dev/null
+++ b/ansible/roles/trove-setup/tasks/minions.yml
@@ -0,0 +1,15 @@
+# Depends on:
+# - lorry-controller-setup.yml
+---
+- name: Enable as many MINIONS as specified in LORRY_CONTROLLER_MINIONS
+ shell: |
+ ln -s "/usr/lib/systemd/system/lorry-controller-minion@.service" \
+ "/etc/systemd/system/multi-user.target.wants/lorry-controller-minion@{{
item }}.service"
+ creates=/etc/systemd/system/multi-user.target.wants/lorry-controller-minion@{{
item }}.service
+ with_sequence: count={{ LORRY_CONTROLLER_MINIONS }}
+ register: minions_creation
+
+- name: Start the all the MINIONS created (if any)
+ service: name=lorry-controller-minion@{{ item.item }} state=restarted
+ with_items: minions_creation.results
+ when: item|changed
diff --git a/ansible/roles/trove-setup/tasks/nfs-setup.yml
b/ansible/roles/trove-setup/tasks/nfs-setup.yml
new file mode 100644
index 0000000..2e125dc
--- /dev/null
+++ b/ansible/roles/trove-setup/tasks/nfs-setup.yml
@@ -0,0 +1,7 @@
+# Depends on:
+# - cache-setup.yml
+---
+- name: Configure nfs exports
+ shell: |
+ cat /etc/exports.cache >/etc/exports
+ creates=/etc/exports
diff --git a/ansible/roles/trove-setup/tasks/releases.yml
b/ansible/roles/trove-setup/tasks/releases.yml
new file mode 100644
index 0000000..322a46a
--- /dev/null
+++ b/ansible/roles/trove-setup/tasks/releases.yml
@@ -0,0 +1,26 @@
+# Depends on:
+# - site-groups.yml
+---
+- name: Create the releases repository
+ shell: |
+ su -c 'ssh localhost create {{ TROVE_ID }}/site/releases' - git
+ creates=/home/git/repos/{{ TROVE_ID }}/site/releases.git
+
+- name: Create temporary folder to copy templates
+ file: path=/tmp/releases-tmp/ state=directory
+- name: Create the files needed for the releases repository
+ template: src=/usr/share/trove-setup/releases-repo-README
dest=/tmp/releases-tmp/releases-repo-README
+
+- name: Configure the releases repository
+ shell: |
+ su -c "git clone ssh://localhost/{{ TROVE_ID }}/site/releases.git
/tmp/releases" - git
+ su -c "cp /tmp/releases-tmp/releases-repo-README /tmp/releases/README"
- git
+ su -c "cd /tmp/releases; git add README; git commit -m 'Add
README'; git push origin master" - git
+ su -c "rm -Rf /tmp/releases"
+ creates=/home/git/repos/{{ TROVE_ID }}/site/releases.git/refs/heads/master
+
+- name: Link the releases repository to enable the access throught browser
+ file: |
+ src=/home/git/repos/{{ TROVE_ID }}/site/releases.git/rsync
+ dest=/var/www/htdocs/releases state=link
+ force=yes
diff --git a/ansible/roles/trove-setup/tasks/site-groups.yml
b/ansible/roles/trove-setup/tasks/site-groups.yml
new file mode 100644
index 0000000..bb9da85
--- /dev/null
+++ b/ansible/roles/trove-setup/tasks/site-groups.yml
@@ -0,0 +1,88 @@
+# Depends on:
+# - gitano-setup.yml
+---
+# First of all check if the site groups are created.
+- name: Check for site groups (This task can fail)
+ shell: su -c 'ssh git@localhost group list' - git | grep '^{{ item.name
}}:'
+ changed_when: False
+ ignore_errors: True
+ with_items:
+ - { name: 'site-readers', description: 'Users with read access to the site
project' }
+ - { name: 'site-writers', description: 'Users with write access to the site
project' }
+ - { name: 'site-admins', description: 'Users with admin access to the site
project' }
+ - { name: 'site-managers', description: 'Users with manager access to the
site project' }
+ register: gitano_groups
+# Iterate over the results of the previous check, and create the sites needed.
+# In this task we are using the list of results of the previous task
+# - item is the result of the execution of one of the elements of
+# the list of the previous task.
+# - item.item is the item of the previous task being executed when
+# the result (stored in item) was taken.
+#
+# For example, the task: (From
http://docs.ansible.com/playbooks_loops.html#using-register-with-a-loop)
+#
+# - shell: echo "{{ item }}"
+# with_items:
+# - one
+# - two
+# register: echo
+#
+# Would register in the variable "echo":
+#
+# {
+# "changed": true,
+# "msg": "All items completed",
+# "results": [
+# {
+# "changed": true,
+# "cmd": "echo \"one\" ",
+# "delta": "0:00:00.003110",
+# "end": "2013-12-19 12:00:05.187153",
+# "invocation": {
+# "module_args": "echo \"one\"",
+# "module_name": "shell"
+# },
+# "item": "one",
+# "rc": 0,
+# "start": "2013-12-19 12:00:05.184043",
+# "stderr": "",
+# "stdout": "one"
+# },
+# {
+# "changed": true,
+# "cmd": "echo \"two\" ",
+# "delta": "0:00:00.002920",
+# "end": "2013-12-19 12:00:05.245502",
+# "invocation": {
+# "module_args": "echo \"two\"",
+# "module_name": "shell"
+# },
+# "item": "two",
+# "rc": 0,
+# "start": "2013-12-19 12:00:05.242582",
+# "stderr": "",
+# "stdout": "two"
+# }
+# ]
+# }
+
+- name: Create the site groups needed.
+ shell: su -c 'ssh git@localhost group add {{ item.item.name }} {{
item.item.description }}' - git
+ when: item|failed
+ with_items: gitano_groups.results
+
+# When the groups are created, check if they are linked.
+- name: Check for linked groups (This task can fail)
+ shell: su -c 'ssh git@localhost group show {{ item.name }}' - git | grep '^
\[] {{ item.super_group }}'
+ changed_when: False
+ ignore_errors: True
+ with_items:
+ - { name: 'site-readers', super_group: 'site-writers' }
+ - { name: 'site-writers', super_group: 'site-admins' }
+ - { name: 'site-admins', super_group: 'site-managers' }
+ register: gitano_linked_groups
+
+# Link the groups that weren't linked following the same strategy as for the groups
+- shell: su -c 'ssh git@localhost group addgroup {{ item.item.name }} {{
item.item.super_group }}' - git
+ when: item|failed
+ with_items: gitano_linked_groups.results
diff --git a/ansible/roles/trove-setup/tasks/users.yml
b/ansible/roles/trove-setup/tasks/users.yml
new file mode 100644
index 0000000..65e7755
--- /dev/null
+++ b/ansible/roles/trove-setup/tasks/users.yml
@@ -0,0 +1,38 @@
+# Depends on:
+# - check.yml
+---
+- name: Create the lorry user without generating sshkeys.
+ user: name=lorry comment="Trove lorry service" shell=/bin/bash
+- name: Create the /home/lorry/.ssh folder
+ file: path=/home/lorry/.ssh state=directory owner=lorry group=lorry mode=0700
+
+- name: Create users (git, cache, mason) and ssh keys for them.
+ user: name={{ item }} comment="Trove {{ item }} service" shell=/bin/bash
generate_ssh_key=yes
+ with_items:
+ - git
+ - cache
+ - mason
+- name: Create known_hosts for all the users
+ shell: |
+ cat /etc/ssh/ssh_host_*_key.pub | cut -d\ -f1,2 | \
+ sed -e's/^/{{ TROVE_HOSTNAME }},localhost /' > \
+ /home/{{ item }}/.ssh/known_hosts
+ chown {{ item }}:{{ item }} /home/{{ item }}/.ssh/known_hosts
+ chmod 600 /home/{{ item }}/.ssh/known_hosts
+ creates=/home/{{ item }}/.ssh/known_hosts
+ with_items:
+ - git
+ - cache
+ - mason
+ - lorry
+
+- name: Copy the lorry ssh private key
+ copy: |
+ src={{ LORRY_SSH_KEY }}
+ dest=/home/lorry/.ssh/id_rsa
+ owner=lorry group=lorry mode=600
+- name: Copy the lorry ssh public key
+ copy: |
+ src={{ LORRY_SSH_PUBKEY }}
+ dest=/home/lorry/.ssh/id_rsa.pub
+ owner=lorry group=lorry mode=644
diff --git a/ansible/trove-setup.yml b/ansible/trove-setup.yml
new file mode 100644
index 0000000..0ab7f0e
--- /dev/null
+++ b/ansible/trove-setup.yml
@@ -0,0 +1,6 @@
+---
+- hosts: localhost
+ vars_files:
+ - "/etc/trove/trove.conf"
+ roles:
+ - trove-setup
--
1.7.10.4
3:10 p.m.
New subject: [PATCH 7/9] Add Ansible scripts
On Tue, Jul 01, 2014 at 06:02:00PM +0100, Pedro Alvarez wrote:
diff --git a/ansible/roles/trove-setup/tasks/backups.yml
b/ansible/roles/trove-setup/tasks/backups.yml
new file mode 100644
index 0000000..a15e4c4
--- /dev/null
+++ b/ansible/roles/trove-setup/tasks/backups.yml
@@ -0,0 +1,17 @@
+# Depends on:
+# - check.yml
+---
+- name: Create the backups user if TROVE_BACKUP_KEYS is defined
+ user: name=backup comment="Backup user" shell=/bin/sh
home=/root/backup-user-home group=root uid=0 non_unique=yes
+ when: TROVE_BACKUP_KEYS is defined
+
+- name: Creates the .ssh directory to the backups user if TROVE_BACKUP_KEYS is defined
+ file: path=/root/backup-user-home/.ssh state=directory
+ when: TROVE_BACKUP_KEYS is defined
+
+- name: Copy the TROVE_BACKUP_KEYS if defined to authorized_keys of the backup user
+ shell: |
+ cat {{ TROVE_BACKUP_KEYS }} >>
/root/backup-user-home/.ssh/authorized_keys
Does ansible convert the `{{ TROVE_BACKUP_KEYS }}` interpolation into
a shell escaped string?
I'm not familiar with how Ansible/Jinja2 do interpolations, but I've
seen this be done incorrectly in other contexts.
I also saw you do an interpolation as `echo -n "{{ TROVE_ID }}"`
later, which could do unexpected things if TROVE_ID had shell special
characters in, and would do odd things indeed if TROVE_ID had a " in it.
I'd recommend not doing any quoting of your own of the {{ }}
interpolations, and instead using the quote filter.
From my limited understanding, this requires instead of using
`{{ TROVE_ID }}`, using `{{ TROVE_ID|quote }}`.
diff --git a/ansible/roles/trove-setup/tasks/gitano-admin-setup.yml
b/ansible/roles/trove-setup/tasks/gitano-admin-setup.yml
new file mode 100644
index 0000000..5c9f13a
--- /dev/null
+++ b/ansible/roles/trove-setup/tasks/gitano-admin-setup.yml
@@ -0,0 +1,39 @@
+# Depends on:
+# - gitano-setup.yml
+---
+- name: Check if the admin user is configured in gitano (This task can fail)
+ shell: su -c 'ssh git@localhost user' - git | grep '^{{ TROVE_ADMIN_USER
}}:'
Is this meant to be run as root rather than the git user?
+ register: gitano_admin_user
+ changed_when: False
+ ignore_errors: True
+# If the admin user doesn't exist
+- name: Create the admin user
+ shell: su git -c 'ssh git@localhost user add {{ TROVE_ADMIN_USER }} {{
TROVE_ADMIN_EMAIL }} {{ TROVE_ADMIN_NAME }}'
I think you can get away without prefixing the address with git@ if you're logged in
as git.
Also, I don't like the inconsistency of sometimes being `su git -c
"$command"`, but other times being `su -c "$command" - git`.
+ when: gitano_admin_user|failed
+
+- name: Check if admin user is in trove-admin group in gitano (This task can fail)
+ shell: su -c 'ssh git@localhost as {{ TROVE_ADMIN_USER }} whoami' - git | grep
'trove-admin. Trove-local administration'
+ register: gitano_admin_group
+ changed_when: False
+ ignore_errors: True
+# If the admin user is not in the trove-admin group
+- name: Add the admin user to the trove-admin group in gitano
+ shell: su -c 'ssh git@localhost group adduser trove-admin {{ TROVE_ADMIN_USER
}}' - git
+ when: gitano_admin_group|failed
+
+- name: Check if admin user has a sshkey configured in gitano (This task can fail)
+ shell: su -c 'ssh git@localhost as {{ TROVE_ADMIN_USER }} sshkey' - git 2>
/dev/stdout | grep WARNING
`2>&1` is more idiomatic shell than `2> /dev/stdout`, plus has the
advantage of working when /dev is not managed by udev.
+ register: gitano_admin_key
+ changed_when: False
+ ignore_errors: True
+# If admin user doesn't have an sshkey configured
+- name: Create /home/git/keys/ to store sshkeys
+ file: path=/home/git/keys state=directory owner=git group=git
+ when: gitano_admin_key|success
+- name: Copy the TROVE_ADMIN_SSH_PUBKEY to /home/git/keys/admin.key.pub
+ copy: src={{ TROVE_ADMIN_SSH_PUBKEY }} dest=/home/git/keys/admin.key.pub mode=0644
+ when: gitano_admin_key|success
+
+- name: Add /home/git/keys/admin.key.pub ssh key to the admin user in gitano.
+ shell: su -c 'ssh git@localhost as {{ TROVE_ADMIN_USER }} sshkey add default <
/home/git/keys/admin.key.pub' - git
+ when: gitano_admin_key|success
diff --git a/ansible/roles/trove-setup/tasks/gitano-lorry-setup.yml
b/ansible/roles/trove-setup/tasks/gitano-lorry-setup.yml
new file mode 100644
index 0000000..b2681dd
--- /dev/null
+++ b/ansible/roles/trove-setup/tasks/gitano-lorry-setup.yml
@@ -0,0 +1,18 @@
+# Depends on:
+# - gitano-setup.yml
+---
+- name: Check if lorry has a sshkey configured in gitano (This task can fail)
+ shell: su -c 'ssh git@localhost as lorry sshkey' - git 2> /dev/stdout |
grep WARNING
See the previous comment about /dev/stdout
+ register: gitano_lorry_key
+ changed_when: False
+ ignore_errors: True
+# If lorry user doesn't have an sshkey configured
+- name: Create /home/git/keys folder to store ssh keys
+ file: path=/home/git/keys state=directory owner=git group=git
+ when: gitano_lorry_key|success
+- name: Copy LORRY_SSH_PUBKEY to /home/git/keys/lorry.key.pub
+ copy: src={{ LORRY_SSH_PUBKEY }} dest=/home/git/keys/lorry.key.pub mode=0644
+ when: gitano_lorry_key|success
+- name: Add to the gitano lorry user the /home/git/keys/lorry.key.pub
+ shell: su -c 'ssh git@localhost as lorry sshkey add trove <
/home/git/keys/lorry.key.pub' - git
+ when: gitano_lorry_key|success
diff --git a/ansible/roles/trove-setup/tasks/gitano-mason-setup.yml
b/ansible/roles/trove-setup/tasks/gitano-mason-setup.yml
new file mode 100644
index 0000000..edfd873
--- /dev/null
+++ b/ansible/roles/trove-setup/tasks/gitano-mason-setup.yml
@@ -0,0 +1,16 @@
+# Depends on:
+# - gitano-setup.yml
+---
+- name: Check if mason has a sshkey configured in gitano (This task can fail)
+ shell: su -c 'ssh git@localhost as mason sshkey' - git 2> /dev/stdout |
grep WARNING
See the previous comment about /dev/stdout
+ register: gitano_mason_key
+ changed_when: False
+ ignore_errors: True
+
+# If distbuild user doesn't have an sshkey configured
+- file: path=/home/git/keys state=directory owner=git group=git
+ when: gitano_mason_key|success
+- copy: src={{ MASON_SSH_PUBKEY }} dest=/home/git/keys/worker.key.pub mode=0644
+ when: gitano_mason_key|success
+- shell: su -c 'ssh git@localhost as mason sshkey add trove <
/home/git/keys/mason.key.pub' - git
+ when: gitano_mason_key|success
diff --git a/ansible/roles/trove-setup/tasks/gitano-setup.yml
b/ansible/roles/trove-setup/tasks/gitano-setup.yml
new file mode 100644
index 0000000..a7b7903
--- /dev/null
+++ b/ansible/roles/trove-setup/tasks/gitano-setup.yml
@@ -0,0 +1,48 @@
+# Depends on:
+# - git.yml
+---
+# Before configuring Gitano, it's necessary to modify the placeholders
+# of the skeleton template of Gitano with the values of /etc/trove/trove.conf.
+# Ansible does not provide an efficient way to do this. Its template module
+# is not able to run recursively over directories, and is not able to create
+# the directories needed.
+#
+# The solution implemented consists in create the directories first and then
+# using the template module in all the files. This could be possible to
+# implement using the 'with_lines' option combinated with the 'find'
command.
+#
+# Create the directories
+- name: Create the directories needed for the Gitano skeleton.
+ file: path=/etc/{{ item }} state=directory
+ with_lines:
+ - (cd /usr/share/trove-setup && find gitano -type d)
+# Copy all the files to the right place and fill the templates whenever possible
+- name: Create the Gitano skeleton using the templates
+ template: src=/usr/share/trove-setup/{{ item }} dest=/etc/{{ item }}
+ with_lines:
+ - (cd /usr/share/trove-setup && find gitano -type f)
I'm a little annoyed there's nothing like with_lines that takes NUL
terminated output, since this would have issues if we had files with
newlines in their names, however this is just my usual pickiness about
whitespace unsafety, and we are in control of what goes into trove-setup,
so we can avoid this issue.
+
+# Configure gitano
+- name: Configure Gitano with /etc/gitano-setup.clod
+ command: |
+ /bin/su -c 'gitano-setup /etc/gitano-setup.clod' - git
+ creates=/home/git/repos/gitano-admin.git
+
+- name: Unlock the password of the git user (This task can fail)
+ shell: busybox passwd -u git
+ register: passwd_result
+ changed_when: passwd_result|success
+ ignore_errors: True
+
+# Now that /home/git/repos exists, we can enable the git-daemon service
+- name: Enable the git-daemon.service
+ shell: |
+ ln -s "/usr/lib/systemd/system/git-daemon.service" \
+ "/etc/systemd/system/multi-user.target.wants/git-daemon.service"
+ creates=/etc/systemd/system/multi-user.target.wants/git-daemon.service
+ register: git_daemon_service
+
+# Now we can start the service without rebooting the system
+- name: Restart git-daemon.service
+ service: name=git-daemon state=restarted
+ when: git_daemon_service|changed
diff --git a/ansible/roles/trove-setup/tasks/gitano-worker-setup.yml
b/ansible/roles/trove-setup/tasks/gitano-worker-setup.yml
new file mode 100644
index 0000000..a44adc0
--- /dev/null
+++ b/ansible/roles/trove-setup/tasks/gitano-worker-setup.yml
@@ -0,0 +1,18 @@
+# Depends on:
+# - gitano-setup.yml
+---
+- name: Check if worker has a sshkey configured in gitano (This task can fail)
+ shell: su -c 'ssh git@localhost as distbuild sshkey' - git 2> /dev/stdout |
grep WARNING
See previous comments about /dev/stdout.
+ register: gitano_worker_key
+ changed_when: False
+ ignore_errors: True
+# If distbuild user doesn't have an sshkey configured
+- name: Create /home/git/keys/ to store ssh keys
+ file: path=/home/git/keys state=directory owner=git group=git
+ when: gitano_worker_key|success
+- name: Copy WORKER_SSH_PUBKEY to /home/git/keys/worker.key.pub
+ copy: src={{ WORKER_SSH_PUBKEY }} dest=/home/git/keys/worker.key.pub mode=0644
+ when: gitano_worker_key|success
+- name: Add /home/git/keys/worker.key.pub to the distbuild user in Gitano
+ shell: su -c 'ssh git@localhost as distbuild sshkey add trove <
/home/git/keys/worker.key.pub' - git
+ when: gitano_worker_key|success
diff --git a/ansible/roles/trove-setup/tasks/hostname.yml
b/ansible/roles/trove-setup/tasks/hostname.yml
new file mode 100644
index 0000000..513bd89
--- /dev/null
+++ b/ansible/roles/trove-setup/tasks/hostname.yml
@@ -0,0 +1,26 @@
+# Depends on:
+# - check.yml
+---
+- name: Check the /etc/hostname and compare it with HOSTNAME (This task can fail)
+ shell: if [ "$(cat /etc/hostname)" != "{{ HOSTNAME }}" ]; then
exit 1; fi
I don't understand Ansible's shell command semantics, but unless
Ansible adds extra commands at the end, couldn't this just be
`[ "$(cat /etc/hostname)" != {{ HOSTNAME|quote }} ]`?
+ register: hostname_file
+ ignore_errors: True
+ changed_when: False
+ when: HOSTNAME is defined
I take it this was to work around Ansible not having a generic way to set the hostname?
+# If /etc/hostname doesn't match with HOSTNAME
+- name: Rewrite /etc/hostname with HOSTNAME
+ shell: echo {{ HOSTNAME }} > /etc/hostname
+ when: hostname_file|failed
+
+- name: Check the actual hostname with `hostname` and compare it with HOSTNAME (This
task can fail)
+ shell: if [ "$(hostname)" != "{{ HOSTNAME }}" ]; then exit 1; fi
+ register: actual_hostname
+ ignore_errors: True
+ changed_when: False
+ when: HOSTNAME is defined
+
+# If `hostname` doesn't match with HOSTNAME
+- name: Change the hostname to HOSTNAME
+ shell: hostname "{{ HOSTNAME }}"
+ when: actual_hostname|failed
I tend to prefer `hostname -F /etc/hostname`, but I can see this
introduces a dependency on the file being corrected first.
diff --git a/ansible/roles/trove-setup/tasks/known-hosts-setup.yml
b/ansible/roles/trove-setup/tasks/known-hosts-setup.yml
new file mode 100644
index 0000000..9f91ffa
--- /dev/null
+++ b/ansible/roles/trove-setup/tasks/known-hosts-setup.yml
@@ -0,0 +1,7 @@
+# Depends on:
+# - check.yml
+---
+- name: Add localhost and UPSTREAM_TROVE to /etc/ssh/ssh_known_hosts
+ shell: |
+ /bin/sh -c "ssh-keyscan localhost {{ UPSTREAM_TROVE }} >
/etc/ssh/ssh_known_hosts"
Is there any particular reason you can't just run `ssh-keyscan localhost
{{ UPSTREAM_TROVE }}` here? Why does it need to be invoked as
`/bin/sh -c`, if you're already saying it has to be run in a shell?
+ creates=/etc/ssh/ssh_known_hosts
diff --git a/ansible/roles/trove-setup/tasks/lighttpd.yml
b/ansible/roles/trove-setup/tasks/lighttpd.yml
new file mode 100644
index 0000000..105e307
--- /dev/null
+++ b/ansible/roles/trove-setup/tasks/lighttpd.yml
@@ -0,0 +1,51 @@
+---
+- name: Create /etc/lighttpd/certs directory
+ file: path=/etc/lighttpd/certs state=directory
+- name: Create certificates for lighttpd in /etc/lighttpd/certs/lighttpd.pem
+ shell: |
+ yes '' | openssl req -new -x509 \
+ -keyout /etc/lighttpd/certs/lighttpd.pem \
+ -out /etc/lighttpd/certs/lighttpd.pem -days 36525 -nodes
+ creates=/etc/lighttpd/certs/lighttpd.pem
+ register: lighttpd_certs
+- name: Create /var/run/lighttpd for cache user
+ file: path=/var/run/lighttpd state=directory owner=cache group=cache
+ register: lighttpd_folder
+
+# Now that the lighttpd certificates and the /var/run/lighttpd exist, we can
+# enable the lighttpd-git service
+- name: Enable lighttpd-git service
+ shell: |
+ ln -s "/usr/lib/systemd/system/lighttpd-git.service" \
+ "/etc/systemd/system/multi-user.target.wants/lighttpd-git.service"
+ creates=/etc/systemd/system/multi-user.target.wants/lighttpd-git.service
+ register: lighttpd_git_service
+
+# Now we can start the service without rebooting the system
+- name: Restart the lighttpd-git service
+ service: name=lighttpd-git state=restarted
+ when: lighttpd_git_service|changed
+
+# Once the service lighttpd-git is running it's possible to do the same
+# with the following services:
+# - lighttpd-morph-cache
+# - lighttpd-lorry-controller-webapp
+- name: Enable lighttpd-morph-cache service
+ shell: |
+ ln -s "/usr/lib/systemd/system/lighttpd-morph-cache.service" \
+
"/etc/systemd/system/multi-user.target.wants/lighttpd-morph-cache.service"
+
creates=/etc/systemd/system/multi-user.target.wants/lighttpd-morph-cache.service
+ register: lighttpd_morph_cache_service
+- name: Restart the lighttpd-morph-cache service
+ service: name=lighttpd-morph-cache state=restarted
+ when: lighttpd_morph_cache_service|changed
+
+- name: Enable the lighttpd-lorry-controller-webapp service
+ shell: |
+ ln -s
"/usr/lib/systemd/system/lighttpd-lorry-controller-webapp.service" \
+
"/etc/systemd/system/multi-user.target.wants/lighttpd-lorry-controller-webapp.service"
+
creates=/etc/systemd/system/multi-user.target.wants/lighttpd-lorry-controller-webapp.service
+ register: lighttpd_lorry_controller_webapp_service
+- name: Restart the lighttpd-lorry-controller-webapp service
+ service: name=lighttpd-lorry-controller-webapp state=restarted
+ when: lighttpd_lorry_controller_webapp_service|changed
diff --git
a/ansible/roles/trove-setup/tasks/lorry-controller-setup.yml
b/ansible/roles/trove-setup/tasks/lorry-controller-setup.yml
new file mode 100644
index 0000000..0e179d1
--- /dev/null
+++ b/ansible/roles/trove-setup/tasks/lorry-controller-setup.yml
@@ -0,0 +1,104 @@
+# Depends on:
+# - gitano-setup.yml
+# - lighttpd.yml
+---
+- name: Create the TROVE_ID/local-config/lorries repository
+ command: |
+ /bin/su -c "ssh localhost create {{ TROVE_ID
}}/local-config/lorries" - git
+ creates=/home/git/repos/{{ TROVE_ID }}/local-config/lorries.git
+- name: Create a temporary folder to copy templates
+ file: path=/tmp/lorries-tmp state=directory
I don't like tempdirs with fixed names, could you get it to use `mktemp
-d`, or does it have some built-in commands for making a tempdir?
+- name: Create the configuration files of lorry-controller using
templates
+ template: src=/usr/share/trove-setup/{{ item }} dest=/tmp/lorries-tmp/{{ item }}
+ with_items:
+ - lorry-controller.conf
+ - README.lorry-controller
+- name: Configure the lorry-controller
+ shell: |
+ su -c "git clone ssh://localhost/{{ TROVE_ID }}/local-config/lorries.git
/tmp/lorries" - git
See previous comment about tempdirs with fixed names.
+ su -c "cp /tmp/lorries-tmp/lorry-controller.conf
/tmp/lorries/lorry-controller.conf" - git
+ su -c "cp /tmp/lorries-tmp/README.lorry-controller
/tmp/lorries/README" - git
+ su -c "mkdir /tmp/lorries/open-source-lorries" - git
+ su -c "cp /usr/share/trove-setup/open-source-lorries/README
/tmp/lorries/open-source-lorries/README" - git
+ su -c "mkdir /tmp/lorries/closed-source-lorries" - git
+ su -c "cp /usr/share/trove-setup/closed-source-lorries/README
/tmp/lorries/closed-source-lorries/README" - git
+ su -c "cd /tmp/lorries; git add README lorry-controller.conf
open-source-lorries/README closed-source-lorries/README; git commit -m 'Initial
configuration'; git push origin master" - git
+ su -c "rm -rf /tmp/lorries" - git
+ creates=/home/git/repos/{{ TROVE_ID
}}/local-config/lorries.git/refs/heads/master
+
+# Migration: Remove the old lorry-controller cronjob if exists
+- name: Look for lorry-controller old cronjob (This task can fail)
+ shell: su -c 'crontab -l | grep -e "-c lorry-controller"' - lorry
+ register: lorry_controller_cronjob
+ changed_when: False
+ ignore_errors: True
+
+- name: Remove the old lorry-controller cronjob
+ shell: su -c '/usr/libexec/remove-lorry-controller-from-lorry-crontab' -
lorry
+ when: lorry_controller_cronjob|success
+
+
+# Now that the lorry-controller is configured we can enable the following
+# services and timers, and also start them
+# - lorry-controller-status
+# - lorry-controller-readconf
+# - lorry-controller-ls-troves
+- name: Enable lorry-controller-status service
+ shell: |
+ ln -s "/usr/lib/systemd/system/lorry-controller-status.service" \
+
"/etc/systemd/system/multi-user.target.wants/lorry-controller-status.service"
+
creates=/etc/systemd/system/multi-user.target.wants/lorry-controller-status.service
+ register: lorry_controller_status_service
Is there anything to disable these services when their config is
incorrect, or am I misunderstanding what this is for?
+- name: Start lorry-controller-status service
+ service: name=lorry-controller-status.service state=restarted
+ when: lorry_controller_status_service|changed
+
+- name: Enable lorry-controller-readconf service
+ shell: |
+ ln -s "/usr/lib/systemd/system/lorry-controller-readconf.service" \
+
"/etc/systemd/system/multi-user.target.wants/lorry-controller-readconf.service"
+
creates=/etc/systemd/system/multi-user.target.wants/lorry-controller-readconf.service
+ register: lorry_controller_readconf_service
+- name: Start lorry-controller-readconf service
+ service: name=lorry-controller-readconf.service state=restarted
+ when: lorry_controller_readconf_service|changed
+
+- name: Enable lorry-controller-ls-troves service
+ shell: |
+ ln -s "/usr/lib/systemd/system/lorry-controller-ls-troves.service" \
+
"/etc/systemd/system/multi-user.target.wants/lorry-controller-ls-troves.service"
+
creates=/etc/systemd/system/multi-user.target.wants/lorry-controller-ls-troves.service
+ register: lorry_controller_ls_troves_service
+- name: Start lorry-controller-ls-troves service
+ service: name=lorry-controller-ls-troves.service state=restarted
+ when: lorry_controller_ls_troves_service|changed
+
+- name: Enable lorry-controller-status timer
+ shell: |
+ ln -s "/usr/lib/systemd/system/lorry-controller-status.timer" \
+
"/etc/systemd/system/multi-user.target.wants/lorry-controller-status.timer"
+
creates=/etc/systemd/system/multi-user.target.wants/lorry-controller-status.timer
+ register: lorry_controller_status_timer
+- name: Start lorry-controller-status timer
+ service: name=lorry-controller-status.timer state=restarted
+ when: lorry_controller_status_timer|changed
+
+- name: Enable lorry-controller-readconf timer
+ shell: |
+ ln -s "/usr/lib/systemd/system/lorry-controller-readconf.timer" \
+
"/etc/systemd/system/multi-user.target.wants/lorry-controller-readconf.timer"
+
creates=/etc/systemd/system/multi-user.target.wants/lorry-controller-readconf.timer
+ register: lorry_controller_readconf_timer
+- name: Start lorry-controller-readconf timer
+ service: name=lorry-controller-readconf.timer state=restarted
+ when: lorry_controller_readconf_timer|changed
+
+- name: Enable lorry-controller-ls-troves timer
+ shell: |
+ ln -s "/usr/lib/systemd/system/lorry-controller-ls-troves.timer" \
+
"/etc/systemd/system/multi-user.target.wants/lorry-controller-ls-troves.timer"
+
creates=/etc/systemd/system/multi-user.target.wants/lorry-controller-ls-troves.timer
+ register: lorry_controller_ls_troves_timer
+- name: Start lorry-controller-ls-troves timer
+ service: name=lorry-controller-ls-troves.timer state=restarted
+ when: lorry_controller_ls_troves_timer|changed
diff --git a/ansible/roles/trove-setup/tasks/cache-setup.yml
b/ansible/roles/trove-setup/tasks/cache-setup.yml
new file mode 100644
index 0000000..ea6ffd7
--- /dev/null
+++ b/ansible/roles/trove-setup/tasks/cache-setup.yml
@@ -0,0 +1,12 @@
+# Depends on:
+# - users.yml
+---
+- name: Create artifacts and ccache folder for the cache user
+ file: path=/home/cache/{{ item }} state=directory owner=cache group=cache
+ with_items:
+ - artifacts
+ - ccache
+- name: Create /etc/exports.cache
+ shell: |
+ echo '/home/cache/ccache
*(rw,all_squash,no_subtree_check,anonuid=1002,anongid=1002)' > /etc/exports.cache
+ creates=/etc/exports.cache
diff --git a/ansible/roles/trove-setup/tasks/nfs-setup.yml
b/ansible/roles/trove-setup/tasks/nfs-setup.yml
new file mode 100644
index 0000000..2e125dc
--- /dev/null
+++ b/ansible/roles/trove-setup/tasks/nfs-setup.yml
@@ -0,0 +1,7 @@
+# Depends on:
+# - cache-setup.yml
+---
+- name: Configure nfs exports
+ shell: |
+ cat /etc/exports.cache >/etc/exports
+ creates=/etc/exports
I think it may be possible to use the /etc/exports.d directory, instead
of combining multiple files to create /etc/exports.
diff --git a/ansible/roles/trove-setup/tasks/releases.yml
b/ansible/roles/trove-setup/tasks/releases.yml
new file mode 100644
index 0000000..322a46a
--- /dev/null
+++ b/ansible/roles/trove-setup/tasks/releases.yml
@@ -0,0 +1,26 @@
+# Depends on:
+# - site-groups.yml
+---
+- name: Create the releases repository
+ shell: |
+ su -c 'ssh localhost create {{ TROVE_ID }}/site/releases' - git
+ creates=/home/git/repos/{{ TROVE_ID }}/site/releases.git
+
+- name: Create temporary folder to copy templates
+ file: path=/tmp/releases-tmp/ state=directory
See previous comments about tempdirs with fixed names.
diff --git a/ansible/roles/trove-setup/tasks/users.yml
b/ansible/roles/trove-setup/tasks/users.yml
new file mode 100644
index 0000000..65e7755
--- /dev/null
+++ b/ansible/roles/trove-setup/tasks/users.yml
@@ -0,0 +1,38 @@
+# Depends on:
+# - check.yml
+---
+- name: Create the lorry user without generating sshkeys.
+ user: name=lorry comment="Trove lorry service" shell=/bin/bash
+- name: Create the /home/lorry/.ssh folder
+ file: path=/home/lorry/.ssh state=directory owner=lorry group=lorry mode=0700
+
+- name: Create users (git, cache, mason) and ssh keys for them.
+ user: name={{ item }} comment="Trove {{ item }} service" shell=/bin/bash
generate_ssh_key=yes
+ with_items:
+ - git
+ - cache
+ - mason
+- name: Create known_hosts for all the users
+ shell: |
+ cat /etc/ssh/ssh_host_*_key.pub | cut -d\ -f1,2 | \
+ sed -e's/^/{{ TROVE_HOSTNAME }},localhost /' > \
I don't think Ansible has any built-in filters that would let us
interpolate values into a sed expression safely.
+ /home/{{ item }}/.ssh/known_hosts
+ chown {{ item }}:{{ item }} /home/{{ item }}/.ssh/known_hosts
+ chmod 600 /home/{{ item }}/.ssh/known_hosts
+ creates=/home/{{ item }}/.ssh/known_hosts
+ with_items:
+ - git
+ - cache
+ - mason
+ - lorry
+
+- name: Copy the lorry ssh private key
+ copy: |
+ src={{ LORRY_SSH_KEY }}
+ dest=/home/lorry/.ssh/id_rsa
+ owner=lorry group=lorry mode=600
+- name: Copy the lorry ssh public key
+ copy: |
+ src={{ LORRY_SSH_PUBKEY }}
+ dest=/home/lorry/.ssh/id_rsa.pub
+ owner=lorry group=lorry mode=644
diff --git a/ansible/trove-setup.yml b/ansible/trove-setup.yml
new file mode 100644
index 0000000..0ab7f0e
--- /dev/null
+++ b/ansible/trove-setup.yml
@@ -0,0 +1,6 @@
+---
+- hosts: localhost
+ vars_files:
+ - "/etc/trove/trove.conf"
+ roles:
+ - trove-setup
7:02 p.m.
New subject: [PATCH 8/9] Install Ansible scripts and create a unit to run them
---
Makefile | 2 ++
units/trove-setup.service | 16 ++++++++++++++++
2 files changed, 18 insertions(+)
create mode 100644 units/trove-setup.service
diff --git a/Makefile b/Makefile
index 52e3106..134436b 100644
--- a/Makefile
+++ b/Makefile
@@ -1,4 +1,6 @@
install:
+ mkdir -p "${DESTDIR}/usr/lib/trove-setup/ansible"
+ cp -r ansible/* "${DESTDIR}/usr/lib/trove-setup/ansible"
mkdir -p "${DESTDIR}/usr/lib/systemd/system/multi-user.target.wants"
cp units/* "${DESTDIR}/usr/lib/systemd/system"
ln -sf ../trove-setup.service
"${DESTDIR}/usr/lib/systemd/system/multi-user.target.wants/trove-setup.service"
diff --git a/units/trove-setup.service b/units/trove-setup.service
new file mode 100644
index 0000000..3b923a2
--- /dev/null
+++ b/units/trove-setup.service
@@ -0,0 +1,16 @@
+[Unit]
+Description=Run trove-setup Ansible scripts
+Requires=network.target
+After=network.target
+Requires=opensshd.service
+After=opensshd.service
+
+# If there's a shared /var subvolume, it must be mounted before this
+# unit runs.
+Requires=local-fs.target
+After=local-fs.target
+
+ConditionPathExists=/etc/trove/trove.conf
+
+[Service]
+ExecStart=/usr/bin/ansible-playbook -v -i /usr/lib/trove-setup/ansible/hosts
/usr/lib/trove-setup/ansible/trove-setup.yml
--
1.7.10.4
7:02 p.m.
New subject: [PATCH 9/9] Update skel path of gitano
---
share/etc/gitano-setup.clod | 1 +
1 file changed, 1 insertion(+)
diff --git a/share/etc/gitano-setup.clod b/share/etc/gitano-setup.clod
index 6139c4e..511479f 100644
--- a/share/etc/gitano-setup.clod
+++ b/share/etc/gitano-setup.clod
@@ -4,6 +4,7 @@ paths.home "/home/git"
paths.ssh "/home/git/.ssh"
paths.pubkey "/home/git/.ssh/id_rsa.pub"
paths.repos "/home/git/repos"
+paths.skel "/etc/gitano/skel/gitano-admin"
admin.username "trove"
admin.realname "Trove Instance Administrator"
--
1.7.10.4
1:18 p.m.
New subject: [PATCH 0/9] Change how trove-setup configures the trove.
On 01/07/2014 18:01, Pedro Alvarez wrote:
Repo: baserock:baserock/trove-setup
Branch: baserock/pedroalvarez/ansible-clean
Sha1: 92aa3046237778273271fbe64fd9ca7b58b0a533
Landing: master
This patch series changes the way the trove is configured.
Now trove-setup uses ansible to manage all the configuration needed
and also the migration scripts. The trove will be configured at boot time
and the new solution includes all the configuration previously done in
trove.configure.
That means now is easier to deploy generic troves and configure them
later.
The patch series includes all the changes necessary to use Ansible.
I've built and deployed a working trove using pedro's branch.
There are a few ansible-playbook failed messages in journalctl - I'd be
happier to see them tidied up if possible, but +1 from me on the series
anyway
1:24 p.m.
New subject: [PATCH 0/9] Change how trove-setup configures the trove.
On 02/07/2014 12:18, Paul Sherwood wrote:
On 01/07/2014 18:01, Pedro Alvarez wrote:
> Repo: baserock:baserock/trove-setup
> Branch: baserock/pedroalvarez/ansible-clean
Actually, i tested Pedro's current branch,
http://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/definitions.gi...
the ansible-clean one is a couple of weeks old
> Sha1: 92aa3046237778273271fbe64fd9ca7b58b0a533
> Landing: master
>
> This patch series changes the way the trove is configured.
>
> Now trove-setup uses ansible to manage all the configuration needed
> and also the migration scripts. The trove will be configured at boot time
> and the new solution includes all the configuration previously done in
> trove.configure.
>
> That means now is easier to deploy generic troves and configure them
> later.
>
> The patch series includes all the changes necessary to use Ansible.
I've built and deployed a working trove using pedro's branch.
There are a few ansible-playbook failed messages in journalctl - I'd be
happier to see them tidied up if possible, but +1 from me on the series
anyway
_______________________________________________
baserock-dev mailing list
baserock-dev(a)baserock.org
http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/baserock-dev-ba...
--
Paul Sherwood Codethink Ltd.
Tel: +44 788 798 4900 302 Ducie House, Ducie Street,
http://www.codethink.co.uk/ Manchester, M1 2JW, United Kingdom.
Codethink provides advanced software design, development, integration &
test services: from embedded systems to high performance apps to cloud.
3:18 p.m.
New subject: [PATCH 0/9] Change how trove-setup configures the trove.
On 02/07/14 12:24, Paul Sherwood wrote:
On 02/07/2014 12:18, Paul Sherwood wrote:
> On 01/07/2014 18:01, Pedro Alvarez wrote:
>> Repo: baserock:baserock/trove-setup
>> Branch: baserock/pedroalvarez/ansible-clean
Actually, i tested Pedro's current branch,
http://git.baserock.org/cgi-bin/cgit.cgi/baserock/baserock/definitions.gi...
the ansible-clean one is a couple of weeks old
That's ok, the baserock/pedroalvarez/ansible-trove-rebased if the branch of
definitions.git, and the one I sent for review.
This patch is for the changes in trove-setup.git, and the branch where I made
the changes is baserock/pedroalvarez/ansible-clean, and is the one I'm using in
definitions.git.
Every patch has the right branch name in their cover letter, but I know this
can cause confusion to people reviewing. Sorry for being inconsistent with
naming my branches.
I promise you I will be more consistent from now on :)
Pedro-clean-and-rebased
7:02 p.m.
New subject: [PATCH 0/2] Don't enable the lorry-controller services by default
Repo: baserock:baserock/lorry-controller
Branch: baserock/pedroalvarez/ansible
Sha1: 0a2c76765d044ab85d3c988a76b04ce14f5f2b3b
Landing: master
The lorry-controller services will be enabled when the
trove is configured (trove-setup).
Pedro Alvarez (2):
Do not enable the services and timers
Remove dependency of lighttpd-lorry-controller-webapp
lorry-controller.morph | 4 ----
units/lorry-controller-ls-troves.service | 1 -
units/lorry-controller-ls-troves.timer | 1 -
units/lorry-controller-readconf.service | 1 -
units/lorry-controller-readconf.timer | 1 -
units/lorry-controller-status.service | 1 -
units/lorry-controller-status.timer | 1 -
7 files changed, 10 deletions(-)
--
1.7.10.4
7:02 p.m.
New subject: [PATCH 1/2] Do not enable the services and timers
trove-setup will enable them on the first boot.
---
lorry-controller.morph | 4 ----
1 file changed, 4 deletions(-)
diff --git a/lorry-controller.morph b/lorry-controller.morph
index 9f1623a..a16f9d9 100644
--- a/lorry-controller.morph
+++ b/lorry-controller.morph
@@ -8,7 +8,3 @@ post-install-commands:
TGT="$DESTDIR/usr/lib/systemd/system"
install -d "$TGT/multi-user.target.wants"
install -m 0644 units/*.service units/*.timer "$TGT/."
- cd "$TGT/multi-user.target.wants"
- # Ignore template units for symlinking. The template units will be
- # instantiated at deploy time by trove.configure.
- ln -s ../*[^(a)].service ../*[^(a)].timer .
--
1.7.10.4
7:02 p.m.
New subject: [PATCH 2/2] Remove dependency of lighttpd-lorry-controller-webapp
---
units/lorry-controller-ls-troves.service | 1 -
units/lorry-controller-ls-troves.timer | 1 -
units/lorry-controller-readconf.service | 1 -
units/lorry-controller-readconf.timer | 1 -
units/lorry-controller-status.service | 1 -
units/lorry-controller-status.timer | 1 -
6 files changed, 6 deletions(-)
diff --git a/units/lorry-controller-ls-troves.service
b/units/lorry-controller-ls-troves.service
index fe97811..55ecb6c 100644
--- a/units/lorry-controller-ls-troves.service
+++ b/units/lorry-controller-ls-troves.service
@@ -1,6 +1,5 @@
[Unit]
Description=Lorry Controller ls-troves
-After=lighttpd-lorry-controller-webapp.service
[Service]
ExecStart=/usr/bin/curl -o /dev/null -X POST --data ""
http://localhost:12765/1.0/ls-troves
diff --git a/units/lorry-controller-ls-troves.timer
b/units/lorry-controller-ls-troves.timer
index dbd157d..7d8275a 100644
--- a/units/lorry-controller-ls-troves.timer
+++ b/units/lorry-controller-ls-troves.timer
@@ -1,6 +1,5 @@
[Unit]
Description=Lorry Controller ls-troves
-After=lighttpd-lorry-controller-webapp.service
[Timer]
OnUnitInactiveSec=60
diff --git a/units/lorry-controller-readconf.service
b/units/lorry-controller-readconf.service
index 1f73b46..f9a4de9 100644
--- a/units/lorry-controller-readconf.service
+++ b/units/lorry-controller-readconf.service
@@ -1,6 +1,5 @@
[Unit]
Description=Lorry Controller read config at startup
-After=lighttpd-lorry-controller-webapp.service
[Service]
ExecStart=/usr/bin/curl -o /dev/null -X POST --data ""
http://localhost:12765/1.0/read-configuration
diff --git a/units/lorry-controller-readconf.timer
b/units/lorry-controller-readconf.timer
index 7e4f04e..c0da559 100644
--- a/units/lorry-controller-readconf.timer
+++ b/units/lorry-controller-readconf.timer
@@ -1,6 +1,5 @@
[Unit]
Description=Lorry Controller read config at startup
-After=lighttpd-lorry-controller-webapp.service
[Timer]
OnUnitInactiveSec=60
diff --git a/units/lorry-controller-status.service
b/units/lorry-controller-status.service
index 381677b..a259661 100644
--- a/units/lorry-controller-status.service
+++ b/units/lorry-controller-status.service
@@ -1,6 +1,5 @@
[Unit]
Description=Lorry Controller Status update
-After=lighttpd-lorry-controller-webapp.service
[Service]
ExecStart=/usr/bin/curl -o /dev/null http://localhost:12765/1.0/status
diff --git a/units/lorry-controller-status.timer b/units/lorry-controller-status.timer
index 1528b8c..0b10487 100644
--- a/units/lorry-controller-status.timer
+++ b/units/lorry-controller-status.timer
@@ -1,6 +1,5 @@
[Unit]
Description=Lorry Controller Status update
-After=lighttpd-lorry-controller-webapp.service
[Timer]
OnUnitInactiveSec=60
--
1.7.10.4
4:36 p.m.
Since the majority of the work is in the ansible scripts, either
their application, or their translation from the old method, pretty
much all of my concerns are covered in my mail with Message-Id:
20140702130959.GM4546@aegir
There's some stuff about using 2>/dev/stdout instead of 2>&1, inconsistent
use of `su git -c 'ssh git@localhost ...` vs `su -c 'ssh git@localhost
...' - git`, sometimes not switching user to git at all, and it being
redundant to specify the username again in the ssh command.
However, my biggest nit-pick is about proper interpolation of
configuration variables into shell scripts, and most interpolations
aren't doing this right.
Having learned more about Jinja2, I prefer it to our ad-hoc mish-mash,
as the filters mean we can do interpolation right.
Shell interpolations that were previously `command "foo ##OPTION##"`
should now be `command "foo "{{ OPTION|quote }}`.
However, I also saw `command | grep ^##OPTION##`, which needs to be
regular-expression escaped too, so if we had a `regex-escape` filter,
it would become `command | grep ^{{ OPTION|regex-escape|quote }}`
I also saw `sed -e's/^/{{ TROVE_HOSTNAME }},localhost /'`, which
would need to become
sed -e 's/^/'{{ TROVE_HOSTNAME|sed-escape|quote }}',localhost /'`
We could also do with one for interpolations into the Lace skeleton files,
but we're used to manually escaping those.
`sed-escape` and `regex-escape` of course don't
exist. http://docs.ansible.com/developing_plugins.html#filter-plugins
suggests we can add our own filters with plugins.
`sed-escape` could potentially be replaced with the
`regex_replace('([/&])', '\\\\\\1')` filter, since we only need the
replacement to be escaped, but I don't think it will be so easy for the
regular expressions.
If this becomes properly escaped like this, you get a +2 and my gratitude.
I think it would be better to fix this now, rather than later when
we're confused why it's failing when we put a space in a field, but if
this turns into a massive job, you can have a reluctant +1 to merge
without the escaping fixes.
5:11 p.m.
On Thu, Jul 03, 2014 at 03:36:37PM +0100, Richard Maw wrote:
`sed-escape` could potentially be replaced with the
`regex_replace('([/&])', '\\\\\\1')` filter, since we only need the
replacement to be escaped, but I don't think it will be so easy for the
regular expressions.
Having looked at what python does for its re.escape method,
`regex_replace('(\\W)', '\\\\\\1')` ought to work for anything but
embedded NULs, which would break everything anyway.
3445
days inactive
3481
days old
31 comments
4 participants
participants (4)
-
Paul Sherwood -
Pedro Alvarez -
Richard Maw -
Sam Thursfield