Federated CI
by Lars Wirzenius
(Also posted as https://blog.liw.fi/posts/2018/08/30/federated_ci/ if
that's easier to read.)
In the modern world, a lot of computing happens on other people's
computers. We use a lot of services provided by various parties. This
is a problem for user freedom and software freedom. For example, when
I use Twitter, the software runs on Twitter's servers, and it's
entirely proprietary. Even if it were free software, even if it were
using the Affero GPL license (AGPL), my freedom would be limited by
the fact that I can't change the software running on Twitter's
servers.
If I could, it would be a fairly large security problem. If I could,
then anyone could, and they might not be good people like I am.
If the software were free, instead of proprietary, I could run it on
my own server, or find someone else to run the software for me. This
would make me more free.
That still leaves the data. My calendars would still be on Twitter's
servers: all my tweets, direct messages, the lists of people I follow,
or who follow me. Probably other things as well.
For true freedom in this context, I would need to have a way to
migrate my data from Twitter to another service. For practical
freedom, the migration should not be excessively much work, or be
excessively expensive, not just possible in principle.
For Twitter specifically, there's free-er alternatives, such as
Mastodon.
For ick, my CI / CD engine, here is my current thinking: ick should
not be a centralised service. It should be possible to pick and choose
between instances of its various components: the controller, the
workers, the artifact store, and Qvisqve (authentication server).
Ditto for any additional components in the future.
Since users and the components need to have some trust in each other,
and there may be payment involved, this may need some co-ordination,
and it may not be possible to pick entirely freely. However, as a
thought experiment, let's consider a scenario.
Alice has a bunch of non-mainstream computers she doesn't use herself
much: Arm boards, RISCV boards, PowerPC Macs, Amigas, etc. All in good
working condition. She'd be happy to set them up as build workers, and
let people use them, for a small fee to cover her expenses.
Bettina has a bunch of servers with lots of storage space. She'd be
happy to let people use them as artifact stores, for a fee.
Cecilia has a bunch of really fast x86-64 machines, with lots of RAM
and very fast NVMe disks. She'd also be happy to rent them out as
build workers.
Dinah needs a CI system, but only has one small server, which would
work fine as a controller for her own projects, but is too slow to
comfortably do any actual building.
Eliza also needs a CI system, but wants to keep her projects separate
from Dinah's, so wants to have her own controller. (Eliza and Dinah
can't tolerate each other and do not trust each other.)
Fatima is trusted by everyone, except Eliza, and would be happy to run
a secure server with Qvisqve.
Georgina is like Fatima, except Eliza trusts her, and Dinah doesn't.
The setup would be like this:
* Alice and Cecilia run build workers. The workers trust both Fatima's
and Georgina's Qvisqves. All of their workers are registered with
both Qvisqves, and both Dinah's and Eliza's controllers.
* Bettina's artifact store also trusts both Qvisqves.
* Dinah creates an account on Fatima's Qvisqve. Eliza on Georgina's
Qvisqve. They each get an API token from the respective Qvisqve.
* When Dinah's project builds, her controller uses the API token to
get an identity token from Fatima's Qvisqve, and gives that to each
worker used in her builds. The worker checks the ID token, and then
accepts work from Dinah's controller. The worker reports the time
used to do the work to its billing system, and Alice or Cecilia uses
that information to bill Dinah.
* If a build needs to use an artifact store, the ID token is again
used to bill Dinah.
* For Eliza, the same thing happens, except with another Qvisqve, and
costs from he builds go to her, not Dinah.
This can be generalised to any number of ick components, which can be
used criss-cross. Each component needs to be configured as to which
Qvisqves it trusts.
I think this would be a nicer thing to have than the centralised
hosted ick I've been thinking about so far. Much more complicated, and
much more work, of course. But interesting.
There are some interesting and difficult questions about security to
solve. I don't want to start thinking about the details yet, I'll play
with the general idea first.
What do you think?
5 years, 3 months
- 1
- 0
- ← Newer
- 1
- Older →