5:15 a.m.
Repo: git://git.baserock.org/baserock/baserock/trove-setup.git
Ref: baserock/adamcoldrick/binary-delivery-v4
SHA1: 2b24bfe7f1cbcb460a882e24940f8f5fead5a8a6
Target: master
Hi all,
Version 4 of this series fixes the small issues with the last series.
This patch adds to the lighttpd config to allow http://<trove>/releases
to show a directory listing of the rsync space of a releases repository.
Until something is pushed to this rsync space, the url will instead give
a 404 error.
It also includes a script to create a site project, and create the
required releases repository in it, along with a systemd unit that will
run this script on boot if the repository does not already exist.
Finally, the makefile for trove-setup is changed to install the script
in the right place.
Adam Coldrick (3):
Enable directory listing in <trovehost>/releases
Add a script to create the releases repository
Add a systemd unit to run the repo creation script
etc/lighttpd/git-httpd.conf | 4 ++
share/releases-repo-migration.sh | 117 +++++++++++++++++++++++++++++++++
units/releases-repo-migration.service | 14 ++++
3 files changed, 135 insertions(+)
create mode 100755 share/releases-repo-migration.sh
create mode 100644 units/releases-repo-migration.service
--
1.7.10.4
5:15 a.m.
New subject: [PATCH 1/3] Enable directory listing in <trovehost>/releases
Enable lighttpd's directory listing for /releases. This is a link
to the <trove_id>/site/releases repository's rsync store.
---
etc/lighttpd/git-httpd.conf | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/etc/lighttpd/git-httpd.conf b/etc/lighttpd/git-httpd.conf
index 94e9c26..dea86de 100644
--- a/etc/lighttpd/git-httpd.conf
+++ b/etc/lighttpd/git-httpd.conf
@@ -35,6 +35,10 @@ mimetype.assign = (
".css" => "text/css"
)
+$HTTP["url"] =~ "^/releases(/|$)" {
+ server.dir-listing = "enable"
+}
+
$HTTP["url"] =~ ".*/gitano-command.cgi$" {
setenv.add-environment = (
"HOME" => "/home/git",
--
1.7.10.4
5:15 a.m.
New subject: [PATCH 2/3] Add a script to create the releases repository
Add a script that will create a `site` project containing the releases
repository when run. It will be run on boot by a systemd unit.
---
share/releases-repo-migration.sh | 117 ++++++++++++++++++++++++++++++++++++++
1 file changed, 117 insertions(+)
create mode 100755 share/releases-repo-migration.sh
diff --git a/share/releases-repo-migration.sh b/share/releases-repo-migration.sh
new file mode 100755
index 0000000..a382b0a
--- /dev/null
+++ b/share/releases-repo-migration.sh
@@ -0,0 +1,117 @@
+#!/bin/bash
+
+function create_readers_group()
+{
+ set +e
+ (
+ set -e
+ ssh localhost group add site-readers \
+ 'Users with read access to the site project'
+ )
+ local ret="$?"
+ if [ "$ret" != 0 ]; then
+ token=$(ssh localhost group del site-readers 2>&1 | tail -1 | \
+ cut -d' ' -f 2)
+ ssh localhost group del site-readers $token
+ fi
+ return $ret
+}
+
+function create_writers_group()
+{
+ set +e
+ (
+ set -e
+ ssh localhost group add site-writers \
+ 'Users with write access to the site project'
+ create_readers_group
+ )
+ local ret="$?"
+ if [ "$ret" != 0 ]; then
+ token=$(ssh localhost group del site-writers 2>&1 | tail -1 | \
+ cut -d' ' -f 2)
+ ssh localhost group del site-writers $token
+ fi
+ return $ret
+}
+
+function create_admins_group()
+{
+ set +e
+ (
+ set -e
+ ssh localhost group add site-admins \
+ 'Users with admin access to the site project'
+ create_writers_group
+ )
+ local ret="$?"
+ if [ "$ret" != 0 ]; then
+ token=$(ssh localhost group del site-admins 2>&1 | tail -1 | \
+ cut -d' ' -f 2)
+ ssh localhost group del site-admins $token
+ fi
+ return $ret
+}
+
+function create_managers_group()
+{
+ set +e
+ (
+ set -e
+ ssh localhost group add site-managers \
+ 'Users with manager access to the site project'
+ create_admins_group
+ )
+ local ret="$?"
+ if [ "$ret" != 0 ]; then
+ token=$(ssh localhost group del site-managers 2>&1 | tail -1 | \
+ cut -d' ' -f 2)
+ ssh localhost group del site-managers $token
+ fi
+ return $ret
+}
+
+function link_groups()
+{
+ set -e
+ ssh localhost group addgroup site-admins site-managers
+ ssh localhost group addgroup site-writers site-admins
+ ssh localhost group addgroup site-readers site-writers
+}
+
+function delete_groups()
+{
+ token=$(ssh localhost group del site-managers 2>&1 | tail -1 | \
+ cut -d' ' -f 2)
+ ssh localhost group del site-managers $token
+ token=$(ssh localhost group del site-admins 2>&1 | tail -1 | \
+ cut -d' ' -f 2)
+ ssh localhost group del site-admins $token
+ token=$(ssh localhost group del site-writers 2>&1 | tail -1 | \
+ cut -d' ' -f 2)
+ ssh localhost group del site-writers $token
+ token=$(ssh localhost group del site-readers 2>&1 | tail -1 | \
+ cut -d' ' -f 2)
+ ssh localhost group del site-readers $token
+}
+
+function create_groups()
+{
+ # call managers_group which calls admin_group and so on...
+ create_managers_group
+ set +e
+ (
+ set -e
+ link_groups
+ )
+ local ret="$?"
+ if [ "$ret" != 0 ]; then
+ delete_groups
+ fi
+}
+
+site_groups=$(ssh localhost group list | grep -cE "site-[[:alnum:]]+")
+if [ "$site_groups" == 0 ]; then
+ create_groups
+fi
+ssh localhost create "##PREFIX##/site/releases"
--
1.7.10.4
6:16 a.m.
New subject: [PATCH 2/3] Add a script to create the releases repository
On 27/03/14 10:15, Adam Coldrick wrote:
Add a script that will create a `site` project containing the
releases
repository when run. It will be run on boot by a systemd unit.
---
share/releases-repo-migration.sh | 117 ++++++++++++++++++++++++++++++++++++++
1 file changed, 117 insertions(+)
create mode 100755 share/releases-repo-migration.sh
diff --git a/share/releases-repo-migration.sh b/share/releases-repo-migration.sh
new file mode 100755
index 0000000..a382b0a
--- /dev/null
+++ b/share/releases-repo-migration.sh
@@ -0,0 +1,117 @@
+#!/bin/bash
+
+function create_readers_group()
+{
+ set +e
+ (
+ set -e
+ ssh localhost group add site-readers \
+ 'Users with read access to the site project'
+ )
+ local ret="$?"
+ if [ "$ret" != 0 ]; then
+ token=$(ssh localhost group del site-readers 2>&1 | tail -1 | \
+ cut -d' ' -f 2)
+ ssh localhost group del site-readers $token
+ fi
+ return $ret
+}
+
+function create_writers_group()
+{
+ set +e
+ (
+ set -e
+ ssh localhost group add site-writers \
+ 'Users with write access to the site project'
+ create_readers_group
+ )
+ local ret="$?"
+ if [ "$ret" != 0 ]; then
+ token=$(ssh localhost group del site-writers 2>&1 | tail -1 | \
+ cut -d' ' -f 2)
+ ssh localhost group del site-writers $token
+ fi
+ return $ret
+}
+
+function create_admins_group()
+{
+ set +e
+ (
+ set -e
+ ssh localhost group add site-admins \
+ 'Users with admin access to the site project'
+ create_writers_group
+ )
+ local ret="$?"
+ if [ "$ret" != 0 ]; then
+ token=$(ssh localhost group del site-admins 2>&1 | tail -1 | \
+ cut -d' ' -f 2)
+ ssh localhost group del site-admins $token
+ fi
+ return $ret
+}
+
+function create_managers_group()
+{
+ set +e
+ (
+ set -e
+ ssh localhost group add site-managers \
+ 'Users with manager access to the site project'
+ create_admins_group
+ )
+ local ret="$?"
+ if [ "$ret" != 0 ]; then
+ token=$(ssh localhost group del site-managers 2>&1 | tail -1 | \
+ cut -d' ' -f 2)
+ ssh localhost group del site-managers $token
+ fi
+ return $ret
+}
+
+function link_groups()
+{
+ set -e
+ ssh localhost group addgroup site-admins site-managers
+ ssh localhost group addgroup site-writers site-admins
+ ssh localhost group addgroup site-readers site-writers
+}
+
+function delete_groups()
+{
+ token=$(ssh localhost group del site-managers 2>&1 | tail -1 | \
+ cut -d' ' -f 2)
+ ssh localhost group del site-managers $token
+ token=$(ssh localhost group del site-admins 2>&1 | tail -1 | \
+ cut -d' ' -f 2)
+ ssh localhost group del site-admins $token
+ token=$(ssh localhost group del site-writers 2>&1 | tail -1 | \
+ cut -d' ' -f 2)
+ ssh localhost group del site-writers $token
+ token=$(ssh localhost group del site-readers 2>&1 | tail -1 | \
+ cut -d' ' -f 2)
+ ssh localhost group del site-readers $token
+}
+
+function create_groups()
+{
+ # call managers_group which calls admin_group and so on...
+ create_managers_group
+ set +e
+ (
+ set -e
+ link_groups
+ )
+ local ret="$?"
+ if [ "$ret" != 0 ]; then
+ delete_groups
+ fi
+}
+
+site_groups=$(ssh localhost group list | grep -cE "site-[[:alnum:]]+")
+if [ "$site_groups" == 0 ]; then
+ create_groups
+fi
+ssh localhost create "##PREFIX##/site/releases"
As discussed in IRC, it would be nice if this repo was initialised with
a quick README and description. Something like this I guess:
description="This is a special repository for distributing release
binaries over HTTP. Visit http://##PREFIX##/releases/ to browse content.".
ssh localhost config "##PREFIX##/site/releases" set description
"$description"
repo=$(mkdtemp)
git clone ssh://localhost/##PREFIX##/site/releases $repo
echo "$message" > $repo/README
GIT_DIR=$repo/.git git add $repo/README
GIT_DIR=$repo/.git git commit -m "Add README"
GIT_DIR=$repo/.git git push
rm -Rf $repo
--
Sam Thursfield, Codethink Ltd.
Office telephone: +44 161 236 5575
6:37 a.m.
New subject: [PATCH 2/3] Add a script to create the releases repository
On 27/03/14 11:16, Sam Thursfield wrote:
On 27/03/14 10:15, Adam Coldrick wrote:
> Add a script that will create a `site` project containing the releases
> repository when run. It will be run on boot by a systemd unit.
> ---
...
As discussed in IRC, it would be nice if this repo was initialised
with
a quick README and description. Something like this I guess:
description="This is a special repository for distributing release
binaries over HTTP. Visit http://##PREFIX##/releases/ to browse content.".
ssh localhost config "##PREFIX##/site/releases" set description
"$description"
That would be 'set project.description' I think :)
--
Sam Thursfield, Codethink Ltd.
Office telephone: +44 161 236 5575
6:41 a.m.
New subject: [PATCH 2/3] Add a script to create the releases repository
On Thu, Mar 27, 2014 at 11:37:31 +0000, Sam Thursfield wrote:
>description="This is a special repository for distributing
release
>binaries over HTTP. Visit http://##PREFIX##/releases/ to browse content.".
>
>ssh localhost config "##PREFIX##/site/releases" set description
>"$description"
That would be 'set project.description' I think :)
Yep. Also it won't be visible to users of the Trove in any useful way until
and unless we can organise authentication to the cgit so that we can do
screened access to it.
D.
--
Daniel Silverstone http://www.codethink.co.uk/
Lead Architect GPG 4096/R Key Id: 3CCE BABE 206C 3B69
5:15 a.m.
New subject: [PATCH 3/3] Add a systemd unit to run the repo creation script
Add a systemd unit to check for the existence of the releases
repository, and to run the script to create it if it doesn't exist.
---
units/releases-repo-migration.service | 14 ++++++++++++++
1 file changed, 14 insertions(+)
create mode 100644 units/releases-repo-migration.service
diff --git a/units/releases-repo-migration.service
b/units/releases-repo-migration.service
new file mode 100644
index 0000000..1e161fb
--- /dev/null
+++ b/units/releases-repo-migration.service
@@ -0,0 +1,14 @@
+[Unit]
+Description=Create the ##PREFIX##/site/releases repository
+ConditionPathExists=!/home/git/repos/##PREFIX##/site/releases.git
+Requires=network.target
+After=network.target
+Requires=opensshd.service
+After=opensshd.service
+Requires=trove-early-setup.service
+After=trove-early-setup.service
+
+[Service]
+User=git
+ExecStart=/usr/share/trove-setup/releases-repo-migration.sh
+Restart=no
--
1.7.10.4
5:23 a.m.
New subject: [PATCH 0/2] Perform substitutions in script and systemd
Repo: git://git.baserock.org/baserock/baserock/definitions.git
Ref: baserock/adamcoldrick/binary-delivery-v4
SHA1: 87c9352e9cfb1f75ab4f423e6960a9633cf78ff9
Target: master
Hi,
Version 4 of this temporarily adds a change to trove.morph so that
the system will build and deploy properly.
This patch is also required by the series for binary delivery.
It makes trove.configure perform the needed substitutions in the script
and systemd unit created in the rest of the patch series.
Adam Coldrick (2):
Make trove.configure perform required substitutions
Use the right branch to build
trove.configure | 11 ++++++++++-
trove.morph | 3 ++-
2 files changed, 12 insertions(+), 2 deletions(-)
--
1.7.10.4
5:23 a.m.
New subject: [PATCH 1/2] Make trove.configure perform required substitutions
Since we added releases-repo-migration.sh and a systemd unit to call it,
and both of these need to know the Trove ID, add them to the command that
performs substitutions in trove.configure.
---
trove.configure | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)
diff --git a/trove.configure b/trove.configure
index a1abae9..2f943ff 100755
--- a/trove.configure
+++ b/trove.configure
@@ -80,7 +80,9 @@ sed -f "$ROOT"/etc/trove-setup.sed -i \
"$ROOT"/etc/lorry.conf \
"$ROOT"/usr/share/gitano/skel/gitano-admin/*/*.lace \
"$ROOT"/usr/share/gitano/skel/gitano-admin/*/*.lua \
- "$ROOT"/usr/share/gitano/skel/gitano-admin/users/*/user.conf
+ "$ROOT"/usr/share/gitano/skel/gitano-admin/users/*/user.conf \
+ "$ROOT"/usr/share/trove-setup/releases-repo-migration.sh \
+ "$ROOT"/usr/lib/systemd/system/releases-repo-migration.service
##########################################################################
@@ -94,6 +96,13 @@ cat <<EOF >> "$ROOT/etc/hosts"
EOF
##########################################################################
+
+# create a symlink in /var/www/htdocs to what will be the rsync area of
+# the releases repository
+ln -s "/home/git/repos/$TROVE_ID/site/releases.git/rsync" \
+ "$ROOT/var/www/htdocs/releases"
+
+##########################################################################
# Configuration of trove-early-setup
#
# We configure trove-early-setup so that it runs at first boot of an initial
--
1.7.10.4
5:23 a.m.
New subject: [PATCH 2/2] Use the right branch to build
---
trove.morph | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/trove.morph b/trove.morph
index 4c8ecd1..2b8e60a 100644
--- a/trove.morph
+++ b/trove.morph
@@ -150,7 +150,8 @@ chunks:
- hg-fast-export
- name: trove-setup
repo: baserock:baserock/trove-setup
- ref: master
+ ref: baserock/adamcoldrick/binary-delivery-v4
+ unpetrify-ref: master
build-depends: []
- name: lorry-controller
repo: baserock:baserock/lorry-controller
--
1.7.10.4
6:22 a.m.
New subject: [PATCH 0/3 v4] Create and display a releases repo - binary delivery
On 27/03/14 10:15, Adam Coldrick wrote:
Repo: git://git.baserock.org/baserock/baserock/trove-setup.git
Ref: baserock/adamcoldrick/binary-delivery-v4
SHA1: 2b24bfe7f1cbcb460a882e24940f8f5fead5a8a6
Target: master
Hi all,
Version 4 of this series fixes the small issues with the last series.
This patch adds to the lighttpd config to allow http://<trove>/releases
to show a directory listing of the rsync space of a releases repository.
Until something is pushed to this rsync space, the url will instead give
a 404 error.
It also includes a script to create a site project, and create the
required releases repository in it, along with a systemd unit that will
run this script on boot if the repository does not already exist.
Finally, the makefile for trove-setup is changed to install the script
in the right place.
Adam Coldrick (3):
Enable directory listing in <trovehost>/releases
Add a script to create the releases repository
Add a systemd unit to run the repo creation script
I've tested deploying a Trove from 'master' and then deploying this
branch as an upgrade. It seems to work fine. You can do 'rsync $FILE
git@$TROVE:$TROVE/site/releases' and then see it in http://$TROVE/releases.
I'm a bit sad that when there are no releases we see a basic 404 error
page, instead of an explanation, but I don't think it's worth the time
to fix that right now.
It'd be nice if you could add a description to the releases.git repo but
again, it may not be worth the time right now.
It makes me quite sad that we are doing state transitions using systemd
units and shell scripts, it's going to become unmanageable very quickly.
I hope we get time to look at coming up with a better solution soon.
+1 to this branch from me, anyway.
Sam
--
Sam Thursfield, Codethink Ltd.
Office telephone: +44 161 236 5575
8:40 a.m.
On Thu, Mar 27, 2014 at 10:15:07AM +0000, Adam Coldrick wrote:
Repo: git://git.baserock.org/baserock/baserock/trove-setup.git
Ref: baserock/adamcoldrick/binary-delivery-v4
SHA1: 2b24bfe7f1cbcb460a882e24940f8f5fead5a8a6
Target: master
The shell scripts look ok.
I haven't tried it myself, but I trust Sam's testing.
I don't care so much about the README and description of the releases
repository, so you can have a +1 with or without it, but I trust other
people have good reasons for wanting it.
12:33 p.m.
On Thu, 2014-03-27 at 13:40 +0000, Richard Maw wrote:
On Thu, Mar 27, 2014 at 10:15:07AM +0000, Adam Coldrick wrote:
> Repo: git://git.baserock.org/baserock/baserock/trove-setup.git
> Ref: baserock/adamcoldrick/binary-delivery-v4
> SHA1: 2b24bfe7f1cbcb460a882e24940f8f5fead5a8a6
> Target: master
The shell scripts look ok.
I haven't tried it myself, but I trust Sam's testing.
I don't care so much about the README and description of the releases
repository, so you can have a +1 with or without it, but I trust other
people have good reasons for wanting it.
Thanks, I added and tested Sam's suggestions, then merged.
_______________________________________________
baserock-dev mailing list
baserock-dev(a)baserock.org
http://vlists.pepperfish.net/cgi-bin/mailman/listinfo/baserock-dev-basero...
3284
days inactive
3284
days old
12 comments
4 participants
participants (4)
-
Adam Coldrick -
Daniel Silverstone -
Richard Maw -
Sam Thursfield