7:46 a.m.
Hello all,
I'm currently trying to work out a sensible way of allowing updates to
an upstream trove's repositories to propagate down to any downstream
troves as they happen.
The idea behind this being that there is a team of developers working
from multiple (possibly cross time zone) sites on the same project, that
may not have the most reliable or speedy network. For the purposes of
speeding things up, we can provide a trove at each site to synchronise
repositories across the sites, allowing local development to continue,
and also (hopefully) making collaboration between sites easier.
To further this "story", a CI pipeline would be running at the
"master"
site. that we wouldn't want to cause a bottleneck in with work done at
the others sites, they would instead have their own CI setup.
Currently, the solution would be to provide troves at each site, with
one site designated as the "master" trove, which will lorry back
anything from the downstream troves in a separate namespace. Enjoy this
ASCII art:
master
/ \
/ \
slave1 slave2
The current issue with such an idea being that you would configure
lorry-controller to scan for updates, say, every 2 hours, or higher
depending on the number of repositories to ensure it isn't constantly
playing catch-up; lorry-controller can be quite resource intensive. This
would not be speedy in any sense of the word, since it could be 2 hours
or longer before the repository is updated from upstream.
The only way I believe something like this would currently be possible
is to hack gitano/bin/gitano-post-receive-hook to run `ssh root@trove
'curl -X POST -d "path=repo/path"
http://localhost:12765/1.0/move-to-top` for each site, which would work,
providing you've added the public key of the master trove to
authorized_keys, but this results in the upstream trove requiring
knowledge of a(ny) downstream troves, and regardless is a truly yucky hack.
My rather vague idea is to do some work on Gitano that allows you to
register some form of an event listener, with an address, port and
namespace (or list of repositories) you wish to receive updates from.
Then on specific git events (post-receive would be the plan for now),
would send, for example, "delta/pip master oldsha newsha" to any
registered listeners. Something would then be written for
lorry-controller which would listen for these events and act accordingly.
This definitely needs some ironing out, however it's possible this has
been overthought (or not thought out enough), and it could be simpler
and easier to just configure lorry-controller to constantly cycle
through its repositories.
Any ideas and/or feedback is welcome.
Ben