6:13 a.m.
Hi all,
So some of us had an offline conversation about how we can reorganize
definitions to be more workable in the future. While we can agree on
what the current set of problems we have to solve pretty easily, it's
harder to agree on the solution, so lets try to hash it out here on the
list. I really hope we can at least agree on a solution by the end of
the week.
These are the two main problems I see with the current project layout,
these I want to fix by proposing a new layout with minimal or even no
changes to the tooling surrounding definitions.
I've included a TL;DR for each of the following points, for those who
just want to gloss over this email.
Clutter
~~~~~~~
TL;DR: Creating a system with baserock means that one has to work in a
git repository cluttered with a vast amount of irrelevant "stuff".
Currently if one wants to create a system with baserock using the
current practices, they will clone/branch/fork the upstream definitions
repository and start adding stuff.
Possibly the result of the work done will be merged back into upstream
definitions if desirable, pushing even more "stuff" into there for the
next person to carry.
This has gone on to a point where we actually have deadcode present in
the system, extensions which exist but are not used by any existing
system. It would not surprise me that we have orphaned strata lying
around, serving no greater purpose than to clutter the user's workspace
and confuse them as to what is useful and what is not.
Further, even if one considers that if something is somehow referenced
by a system, "it's not deadcode", this is also wrong; as much of the
existing morphs in the systems directory are long unmaintained and just
add dead weight which we need to carry. While it can be agrued that
there is "some value" that we have a given system "that builds", the
fact that a given system builds is of negligeable value unless that
system has a user who tests that system to actually work in some more
meaningful way.
Some examples of what I cannot do with the current structure:
a.) I cannot peruse the git history of my own project which
uses baserock without being exposed to a huge amount of
work unrelated to my own project.
b.) I cannot easily grep through my own definitions repository
to find a given string in an efficient way because I get
a lot of unrelated results.
But most irritating to me, is that I cannot have a clean workspace
which belongs to my project. I cannot work in an environment where my
coffee table is cluttered with all the stuff my roommate left on the
table: Definitions is currently that cluttered coffee table.
Changes are pushed/forced upon higher level projects/components
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
TL;DR: Changes in low level strata are forced upon thier consumers,
regularly causing breakage and making it impossible for projects
depending on low level strata to ensure their systems actually work as
expected.
This might be perceived as a result of the above point but it is one of
the more damaging aspects.
Currently, as baserock definitions are maintained in one huge directory
tree, changes in lower level components get forced upon higher level
branches, which attempt to reuse these lower level components. This is
just plain wrong.
In C programming, teams who provide stable API/ABI surfaces have the
discipline to push only API/ABI compatible changes to their consumer
base. Changes of a C library which break the API contract are offered
with a clear version bump and consumers of that library must choose and
adapt to the new API or at least recompile for an ABI break.
For C programs, this has been an exact science for a long time,
mistakes are occasionally made but the methodology is sound.
In the case of definitions, changes have been made on a regular basis
which pull in a newer version of something in a lower level strata,
without much care as to whether there has been any API break or change.
Then master happily moves along and the burden of testing and repairing
resulting breakage falls on the maintainers of the consuming systems.
This cannot continue, until we have an exact science and better
discipline to maintain a stable version/branch of a low level stratum
or collection of strata, in which we know that the collective API
surface will absolutely not change, we cannot allow lower level strata
to force changes onto it's consumers.
Instead, the consuming higher level strata and systems mush be at
liberty to develop against a specific version of say, "core" and
"foundation", these projects which depend on the "core/foundation"
group must be able to pull in fresh changes at will and test their
resulting systems accordingly.
So those two points above are basically problems IMO and those are what
I want to address in the immediate future, below is my own proposal for
fixing these issues.
Proposal to use recursive git submodules to address this issue
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Yes I said it, and I know there are plenty of reasons to hate a
submodule, obnoxious things as they are. However I have not yet been
able to come up with a better solution.
So here is how I would propose to address the issues: I would first
propose the following split of definitions, each shared git repo would
be comprised of one or more strata. Systems would be split out into
repositories holding their own system specific strata, system morphs,
deployments and extensions.
o definitions-gnu-linux
This would contain:
- build-essential
- core
- foundataion
- the minimal system bsps
Possibly the stray python bits, maybe some things like icu could
live here too.
These seem to be the most widely shared stratum and they are the
"root of all evil" so to speak.
It's unimportant if a consumer of this component/repo is going to
use all of the strata provided by it.
When I write a C program, I strictly do not leave any deadcode
lying around in my own repository, but I dont mind linking against
Qt even if I do not happen to use every function provided by that
library, this I think is a proper analogy.
definitions-gnu-linux would strictly not contain anything that is
not needed for building these base strata and systems. It would not
contain any deployments or extensions that are not strictly needed
to produce the systems in this repository.
o definitions-freedesktop
This would contain everything needed to build X and also
alternatively wayland, or X with wayland support.
This would also contain the graphics common stuff and gstreamer as
well.
This repository itself would include definitions-gnu-linux as a
submodule, which means that this module can remain in charge of
what version of definitions-gnu-linux it wants to use.
o definitions-gnome
This would contain the gnome stratum and whatever else is needed to
assemble a full GNOME system.
It would contain a deployment as well, and _only_ the extensions
that are required to deploy GNOME.
Of course definitions-gnome would also consume
definitions-freedesktop as a submodule much in the same way that
definitions-freedesktop consumes definitions-gnu-linux as a
submodule, allowing a recursive chain of submodules to be used to
compose a system, and allowing the GNOME project maintainer to
choose what version of the definitions-freedesktop API surface it
wants to consume.
o Other systems
The same would be done initially for systems that we maintain, like
the genivi demo platform, which probably wants to additionally pull
in a Qt definitions module; possibly living side by side or on top
of the definitions-freedesktop module.
Systems that we do not currently maintain but obnoxiously exist,
will be left to die in the current definitions repository until
which time an interested maintainer steps up to maintain that
system using the new layout.
Caveats:
As far as I can see the main caveat in the above proposal is that
sometimes we will want to reorganize things, for instance something
from GNOME might become desirable in the freedesktop module in order to
be reusable by another system; in this case we have some discontinuity
of history.
I dont think this horribly bad, analogy: it has happened before that
custom widgets (GtkOverlay and GtkInfoBar) have been developed as a
part of the gedit program, but were later considered desirable for
other applications and then merged into GTK+ instead.
The above is IMO a much better scenario than if all of the GNOME
Desktop were to be maintained in a single git repository.
Thoughts on this proposal ? Improvements on the given proposal ? Any
counter proposals to address the same issues in a different way ?
Cheers,
-Tristan
6:57 a.m.
Hi Tristan,
I broadly agree with your summation of these two problems, so will
focus on your proposed solutions...
On 2016-10-10 11:13, Tristan Van Berkom wrote:
Clutter
~~~~~~~
TL;DR: Creating a system with baserock means that one has to work in
a
git repository cluttered with a vast amount of irrelevant "stuff".
<snip>
Changes are pushed/forced upon higher level projects/components
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
TL;DR: Changes in low level strata are forced upon thier consumers,
regularly causing breakage and making it impossible for projects
depending on low level strata to ensure their systems actually work
as
expected.
<snip>
Proposal to use recursive git submodules to address this issue
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Yes I said it, and I know there are plenty of reasons to hate a
submodule, obnoxious things as they are. However I have not yet been
able to come up with a better solution.
Well, your basic argument is that keeping everything in one repo is too
much clutter, and I can't really disagree with that. There are only two
ways to solve this:
- somehow (ab)use git to keep the clutter in separate branches of one
repo
- or use multiple repos
I'm willing to accept that the first approach would be too much
trickery and lead to confusion, so we're down to what's the best way to
combine multiple repos in one body of work.
Git submodules are not the only one. I'm not a git wizard but I believe
that 'git subtree' is broadly similar but also less invasive. I suggest
we consider both approaches to determine what's best for our usecases.
So here is how I would propose to address the issues: I would first
propose the following split of definitions, each shared git repo
would
be comprised of one or more strata. Systems would be split out into
repositories holding their own system specific strata, system morphs,
deployments and extensions.
<snip>
I'm ok with the general principle - others can consider the details of
what goes in which subtree/submodule
Caveats:
As far as I can see the main caveat in the above proposal is that
sometimes we will want to reorganize things, for instance something
from GNOME might become desirable in the freedesktop module in order
to
be reusable by another system; in this case we have some
discontinuity
of history.
I dont think this horribly bad, analogy: it has happened before that
custom widgets (GtkOverlay and GtkInfoBar) have been developed as a
part of the gedit program, but were later considered desirable for
other applications and then merged into GTK+ instead.
The above is IMO a much better scenario than if all of the GNOME
Desktop were to be maintained in a single git repository.
I'd just note that whatever the grouping that we settle on, there'll be
cases where downstream users need only part of a subgroup - so afaict
they'll still have the clutter you've described! ???
Thoughts on this proposal ? Improvements on the given proposal ? Any
counter proposals to address the same issues in a different way ?
I'm ok in principle with your proposed separation concept in principle,
provide we justify the subtree vs submodule choice.
Just to note, though - one of the things we need to consider in any
change like this is super-long-term reproducibility. Assuming we proceed
down this path, how are users expected to follow?
- do we go with morph's approach of forcing users to upgrade their
definitions and tool at the same time?
- currently YBD aims to continue building old definitions versions, and
could be extended to support the scheme you're proposing (as well as
Daniel's V9 and V10 proposals) but at some point the complexity of
handling all the versions may not make sense.
- we 'lock' a stable version of YBD for all versions prior to (say) 11,
and then move on with a migration and new forward-only tooling?
8:01 a.m.
On Mon, 2016-10-10 at 11:57 +0100, Paul Sherwood wrote:
Hi Tristan,
I broadly agree with your summation of these two problems, so will
focus on your proposed solutions...
Great, comments below...
On 2016-10-10 11:13, Tristan Van Berkom wrote:
>
> Clutter
> ~~~~~~~
> TL;DR: Creating a system with baserock means that one has to work
> in
> a
> git repository cluttered with a vast amount of irrelevant "stuff".
<snip>
>
> Changes are pushed/forced upon higher level projects/components
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> TL;DR: Changes in low level strata are forced upon thier consumers,
> regularly causing breakage and making it impossible for projects
> depending on low level strata to ensure their systems actually
> work
> as
> expected.
<snip>
>
> Proposal to use recursive git submodules to address this issue
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> Yes I said it, and I know there are plenty of reasons to hate a
> submodule, obnoxious things as they are. However I have not yet
> been
> able to come up with a better solution.
Well, your basic argument is that keeping everything in one repo is
too
much clutter, and I can't really disagree with that. There are only
two
ways to solve this:
- somehow (ab)use git to keep the clutter in separate branches of
one
repo
- or use multiple repos
I'm willing to accept that the first approach would be too much
trickery and lead to confusion, so we're down to what's the best way
to
combine multiple repos in one body of work.
Git submodules are not the only one. I'm not a git wizard but I
believe
that 'git subtree' is broadly similar but also less invasive. I
suggest
we consider both approaches to determine what's best for our
usecases.
I'm reading some blogs about git subtrees right now, looks interesting
indeed.
>
> So here is how I would propose to address the issues: I would first
> propose the following split of definitions, each shared git repo
> would
> be comprised of one or more strata. Systems would be split out into
> repositories holding their own system specific strata, system
> morphs,
> deployments and extensions.
>
<snip>
I'm ok with the general principle - others can consider the details
of
what goes in which subtree/submodule
>
> Caveats:
>
> As far as I can see the main caveat in the above proposal is that
> sometimes we will want to reorganize things, for instance something
> from GNOME might become desirable in the freedesktop module in
> order
> to
> be reusable by another system; in this case we have some
> discontinuity
> of history.
>
> I dont think this horribly bad, analogy: it has happened before
> that
> custom widgets (GtkOverlay and GtkInfoBar) have been developed as a
> part of the gedit program, but were later considered desirable for
> other applications and then merged into GTK+ instead.
>
> The above is IMO a much better scenario than if all of the GNOME
> Desktop were to be maintained in a single git repository.
I'd just note that whatever the grouping that we settle on, there'll
be
cases where downstream users need only part of a subgroup - so
afaict
they'll still have the clutter you've described! ???
I think not, the analogy I made for this specific case was:
"When I write a C program, I strictly do not leave any deadcode
lying around in my own repository, but I dont mind linking against
Qt even if I do not happen to use every function provided by that
library"
I.e. the presence of freedesktop inside the GNOME project module should
be seen as an opaque dependency, not as a part of the GNOME workspace.
If I have a problem with the freedesktop module, I should treat that as
I would a dependency of a C program. I should fork that freedesktop
module to apply my fix and propose a patch for it upstream, once it's
accepted I can depend on a newer version of freedesktop.
However, this presents a technical difficulty which I had overlooked,
which is what directory structure would work.
With submodules, I had envisioned something like this:
./gnome/[strata,systems,clusters,extensions]
./gnome/systems
./gnome/extensions
./gnome/freedesktop <-- git submodule
./gnome/freedesktop/[strata,systems,clusters,extensions]
./gnome/freedesktop/gnu-linux <-- recursive git submodule
./gnome/freedesktop/gnu-linux/[strata,systems,clusters,extensions]
The problems with the above are:
a.) The modules I depend obnoxiously appear inside my own work
directory (although I find this much less disturbing than the
present situation: The only power I have over this submodule is
to decide what version I'm building, I might not even have
commit rights to the freedesktop upstream git).
b.) Definitions needs to support relative path references, because
the project root for building gnu-linux is different when
building from inside freedesktop, and likewise again different
when building inside gnome.
If technically possible, it would be nicer to have a way to setup a
work directory like this:
./work
./work/gnu-linux
./work/freedesktop
./work/gnome
However it should be the gnome git history which dictates what version of freedesktop is
used, and in turn the freedesktop git history which determines the version of gnu-linux,
so this part indeed requires a bit more thought.
>
> Thoughts on this proposal ? Improvements on the given proposal ?
> Any
> counter proposals to address the same issues in a different way ?
I'm ok in principle with your proposed separation concept in
principle,
provide we justify the subtree vs submodule choice.
Just to note, though - one of the things we need to consider in any
change like this is super-long-term reproducibility. Assuming we
proceed
down this path, how are users expected to follow?
- do we go with morph's approach of forcing users to upgrade their
definitions and tool at the same time?
Certainly not, or, only in the worst of cases moving forward.
- currently YBD aims to continue building old definitions versions,
and
could be extended to support the scheme you're proposing (as well as
Daniel's V9 and V10 proposals) but at some point the complexity of
handling all the versions may not make sense.
- we 'lock' a stable version of YBD for all versions prior to
(say)
11,
and then move on with a migration and new forward-only tooling?
I think this is somewhat orthogonal to the splitting up of definitions.
It's possible that YBD needs to support relative paths (if it does not
already) to support, for example, build-depends references to:
"../../gnu-linux/strata/foundation.morph"
So as is, I don't think this should really constitute an API change in
YBD, at least not one which would be backwards incompatible.
However to answer that question, with the new changes in V9 and V10 it
may be a desirable thing to drop back compat, if it happens to be an
ideal moment for that, and if there is no critical user base depending
on support for the older formats.
Practically speaking: What is the cost of migrating existing
definitions that are maintained and useful to someone, vs the cost of
maintaining a tool that is backwards compatible down to V1 ? And, is
that cost entirely a cost of our own ? Or is it a cost we impose on
others ?
If we do both refactor YBD and drop support for < V10 definitions for
some reason, then that refactor should come with preparation for being
forever backwards compatible in the future down to V10.
Cheers,
-Tristan
4:43 a.m.
Hi again,
On Mon, 2016-10-10 at 21:01 +0900, Tristan Van Berkom wrote:
On Mon, 2016-10-10 at 11:57 +0100, Paul Sherwood wrote:
>
> Hi Tristan,
> I broadly agree with your summation of these two problems, so will
> focus on your proposed solutions...
>
Great, comments below...
>
> On 2016-10-10 11:13, Tristan Van Berkom wrote:
> >
> >
> > Clutter
> > ~~~~~~~
> > TL;DR: Creating a system with baserock means that one has to work
> > in
> > a
> > git repository cluttered with a vast amount of irrelevant
> > "stuff".
> <snip>
>
> >
> >
> > Changes are pushed/forced upon higher level projects/components
> > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > TL;DR: Changes in low level strata are forced upon thier
> > consumers,
> > regularly causing breakage and making it impossible for projects
> > depending on low level strata to ensure their systems actually
> > work
> > as
> > expected.
> <snip>
>
> >
> >
> > Proposal to use recursive git submodules to address this issue
> > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > Yes I said it, and I know there are plenty of reasons to hate a
> > submodule, obnoxious things as they are. However I have not yet
> > been
> > able to come up with a better solution.
>
> Well, your basic argument is that keeping everything in one repo is
> too
> much clutter, and I can't really disagree with that. There are only
> two
> ways to solve this:
>
> - somehow (ab)use git to keep the clutter in separate branches of
> one
> repo
> - or use multiple repos
>
> I'm willing to accept that the first approach would be too much
> trickery and lead to confusion, so we're down to what's the best
> way
> to
> combine multiple repos in one body of work.
>
> Git submodules are not the only one. I'm not a git wizard but I
> believe
> that 'git subtree' is broadly similar but also less invasive. I
> suggest
> we consider both approaches to determine what's best for our
> usecases.
I'm reading some blogs about git subtrees right now, looks
interesting
indeed.
So I've briefly looked into what subtrees are today and I can
confidently say that it does not do what I want it to do.
At face value subtrees are attractive because they are easier to
consume from a user perspective (no need to git submodule init && git
submodule update). However this would end up getting messy very quickly
because of the way subtrees work.
While subtrees do record *some* information about the merge point of
where the subtree came from for future merges, it is in fact just this;
a merging algorithm, not a relation from one project to another
independent project.
So in essence, using a subtree for the freedesktop module to be a
consumer of the gnu-linux module would really result in the freedesktop
module containing the payload and history of the gnu-linux module.
This setup would have some bad implications:
a.) Leave much more of the same mess behind, as in depending modules
would actually contain the history of the merged subtree.
b.) A maintainer of the GNOME module could have the liberty of
modifying things within the freedesktop or gnu-linux modules,
as they would in effect be a part of the same history.
I dont think that the depending module maintainer should be
allowed to do this, they can only choose what version of the
module they depend on should be used, so a GNOME maintainer's
options, when requiring a change in freedesktop, should be
limited to:
- Fixing their problem in the freedesktop module and
depending on a new version (if they have permissions)
- Branching the freedesktop module and depending on the
new branch, while proposing the change to the freedesktop
module maintainers (if they have repo permissions).
- Forking the freedesktop module on gitlab or such while
proposing the change to the freedesktop maintainers
(this would be in a gitlab type of setup).
For these reasons, mostly to enforce that modules remain separate
entities and that changes to these modules are propagated in the
correct places, I think submodules are the right approach when compared
to subtrees.
Besides the above, there are 2 more issues with splitting up
definitions as far as I can see, one of them discussed in this previous
email:
Directory Layout
~~~~~~~~~~~~~~~~
Unfortunately, there is no way for a git module (or subtree for that
matter), to be setup so as to clone the said module into an adjacent
directory: A submodule must reside somewhere inside the project tree.
Which leaves only 2 options to manage the splitting of definitions into
separate modules:
1.) Modules are checked out in sub directories, so:
gnome
gnome/[strata,systems,etc]
gnome/freedesktop
gnome/freedesktop/[strata,systems,etc]
gnome/freedesktop/gnu-linux
gnome/freedesktop/gnu-linux/[strata,systems,etc]
This has the benefit that the gnome project history itself is
in charge of managing what freedesktop version it's building
against.
And the disadvantage that some scripting/tooling needs to adapt
to understand that each project has it's own 'root' and these
can be nested, so that YBD knows how to find morph files.
2.) Modules exist as independent entities and are referred to only
by end user or 'master' projects.
So in baserock we might have one master project such like:
baserock
baserock/genevi
baserock/genevi
baserock/gnome
baserock/freedesktop
baserock/gnu-linux
baserock/...other maintained modules...
This comes with the opposing pros and cons, the burden of matching
versions together falls on the master project, which I think is
quite wrong.
To ensure stability of the freedesktop module, it really should be
the freedesktop module which dictates what version of the base
gnu-linux module it builds against.
And of course the benefit is that things build a bit more
homogeneously, we have a more flat directory structure, etc.
I am of course inclined to push for (1), even though it means a bit of churn to how things
logically come together.
To address the pathing issues, so that YBD knows where to look for a morph file referred,
I would propose to use a different relative base
directory depending on the morph being parsed, and to identify that base path by walking
the submodule tree first.
So, when parsing:
gnome/freedesktop/gnu-linux/strata/core.morph
And encountering a build-depends on strata/build-essential.morph,
the base directory for that parse can be determined as:
gnome/freedesktop/gnu-linux
Determining this path should be easy enough using git tooling to
introspect from what submodule the parsed file belongs to.
The other problem is:
DEFAULTS file
~~~~~~~~~~~~~
For the defaults file, I would propose to only have the DEFAULTS file
present in the base gnu-linux submodule.
This is a tricky problem though, as it's conceivable that different modules would like
to have their own DEFAULTS.
In this case, should it ever arise, it would be important that YBD builds gnu-linux with
the DEFAULTS provided by gnu-linux, and that freedesktop be built with the gnu-linux
DEFAULTS unless freedesktop provides it's own DEFAULTS and so on. I.e. it would be
important to not merge the DEFAULTS found in nested modules, instead each modules chunk
should stay with it's own determined DEFAULTS.
All in all, this is a little more tricky than I originally suspected but that's always
the case. Still I think this is a largely important
move to make.
Best,
-Tristan
9:51 a.m.
On Wed, 2016-10-12 at 17:43 +0900, Tristan Van Berkom wrote:
Hi again,
Following up on my own email again.
Yesterday I had some discussion with Rob Taylor who suggested 'git
subrepo' as another alternative to submodules.
This approach builds upon subtrees and expands on it by recording the
remote origin from whence the subrepo came, but squashes/stores the
history of the remote directly into the parent repo.
The upside of using these subrepos would be less pain from submodules,
which I have to acknowledge is a real concern.
The downside is that:
a.) We would need some hooks in place to ensure that local commits
are not made to a subrepo area, altering the correct flow of
changes (i.e. changes to a subrepo should be submitted upstream
and later consumed by a rebase). Further, without such commit
hooks it would be possible to introduce commits which touch files
both inside and outside of the subrepo, rendering some patches
unmergeable.
b.) Submitting a patch for a child repo would be awkward and require
a separate checkout, especially since we are using gerrit for
patch submission and review.
If we were to ignore the above and treat child modules simply as a
merging algorithm (which they would be with subrepo), then we would
essentially 'just' be duplicating child repo data in parent repos,
which might not be all that bad in fact; only that would discourage
upstreaming of fixes.
Submodule or subrepo questions aside, the directory layout and module
separation would be the same, so I will make an attempt at a prototype
starting now and hope that I have something to show by Monday or
Tuesday, building gnome from a split up set of 'cleansed' definitions.
Cheers,
-Tristan
On Mon, 2016-10-10 at 21:01 +0900, Tristan Van Berkom wrote:
>
> On Mon, 2016-10-10 at 11:57 +0100, Paul Sherwood wrote:
> >
> >
> > Hi Tristan,
> > I broadly agree with your summation of these two problems, so
> > will
> > focus on your proposed solutions...
> >
>
> Great, comments below...
>
> >
> >
> > On 2016-10-10 11:13, Tristan Van Berkom wrote:
> > >
> > >
> > >
> > > Clutter
> > > ~~~~~~~
> > > TL;DR: Creating a system with baserock means that one has to
> > > work
> > > in
> > > a
> > > git repository cluttered with a vast amount of irrelevant
> > > "stuff".
> > <snip>
> >
> > >
> > >
> > >
> > > Changes are pushed/forced upon higher level projects/components
> > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > TL;DR: Changes in low level strata are forced upon thier
> > > consumers,
> > > regularly causing breakage and making it impossible for
> > > projects
> > > depending on low level strata to ensure their systems actually
> > > work
> > > as
> > > expected.
> > <snip>
> >
> > >
> > >
> > >
> > > Proposal to use recursive git submodules to address this issue
> > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > Yes I said it, and I know there are plenty of reasons to hate a
> > > submodule, obnoxious things as they are. However I have not yet
> > > been
> > > able to come up with a better solution.
> >
> > Well, your basic argument is that keeping everything in one repo
> > is
> > too
> > much clutter, and I can't really disagree with that. There are
> > only
> > two
> > ways to solve this:
> >
> > - somehow (ab)use git to keep the clutter in separate branches of
> > one
> > repo
> > - or use multiple repos
> >
> > I'm willing to accept that the first approach would be too much
> > trickery and lead to confusion, so we're down to what's the best
> > way
> > to
> > combine multiple repos in one body of work.
> >
> > Git submodules are not the only one. I'm not a git wizard but I
> > believe
> > that 'git subtree' is broadly similar but also less invasive. I
> > suggest
> > we consider both approaches to determine what's best for our
> > usecases.
>
> I'm reading some blogs about git subtrees right now, looks
> interesting
> indeed.
So I've briefly looked into what subtrees are today and I can
confidently say that it does not do what I want it to do.
At face value subtrees are attractive because they are easier to
consume from a user perspective (no need to git submodule init && git
submodule update). However this would end up getting messy very
quickly
because of the way subtrees work.
While subtrees do record *some* information about the merge point of
where the subtree came from for future merges, it is in fact just
this;
a merging algorithm, not a relation from one project to another
independent project.
So in essence, using a subtree for the freedesktop module to be a
consumer of the gnu-linux module would really result in the
freedesktop
module containing the payload and history of the gnu-linux module.
This setup would have some bad implications:
a.) Leave much more of the same mess behind, as in depending
modules
would actually contain the history of the merged subtree.
b.) A maintainer of the GNOME module could have the liberty of
modifying things within the freedesktop or gnu-linux modules,
as they would in effect be a part of the same history.
I dont think that the depending module maintainer should be
allowed to do this, they can only choose what version of the
module they depend on should be used, so a GNOME maintainer's
options, when requiring a change in freedesktop, should be
limited to:
- Fixing their problem in the freedesktop module and
depending on a new version (if they have permissions)
- Branching the freedesktop module and depending on the
new branch, while proposing the change to the freedesktop
module maintainers (if they have repo permissions).
- Forking the freedesktop module on gitlab or such while
proposing the change to the freedesktop maintainers
(this would be in a gitlab type of setup).
For these reasons, mostly to enforce that modules remain separate
entities and that changes to these modules are propagated in the
correct places, I think submodules are the right approach when
compared
to subtrees.
Besides the above, there are 2 more issues with splitting up
definitions as far as I can see, one of them discussed in this
previous
email:
Directory Layout
~~~~~~~~~~~~~~~~
Unfortunately, there is no way for a git module (or subtree for that
matter), to be setup so as to clone the said module into an adjacent
directory: A submodule must reside somewhere inside the project tree.
Which leaves only 2 options to manage the splitting of definitions
into
separate modules:
1.) Modules are checked out in sub directories, so:
gnome
gnome/[strata,systems,etc]
gnome/freedesktop
gnome/freedesktop/[strata,systems,etc]
gnome/freedesktop/gnu-linux
gnome/freedesktop/gnu-linux/[strata,systems,etc]
This has the benefit that the gnome project history itself is
in charge of managing what freedesktop version it's building
against.
And the disadvantage that some scripting/tooling needs to adapt
to understand that each project has it's own 'root' and these
can be nested, so that YBD knows how to find morph files.
2.) Modules exist as independent entities and are referred to only
by end user or 'master' projects.
So in baserock we might have one master project such like:
baserock
baserock/genevi
baserock/genevi
baserock/gnome
baserock/freedesktop
baserock/gnu-linux
baserock/...other maintained modules...
This comes with the opposing pros and cons, the burden of
matching
versions together falls on the master project, which I think is
quite wrong.
To ensure stability of the freedesktop module, it really should
be
the freedesktop module which dictates what version of the base
gnu-linux module it builds against.
And of course the benefit is that things build a bit more
homogeneously, we have a more flat directory structure, etc.
I am of course inclined to push for (1), even though it means a bit
of churn to how things logically come together.
To address the pathing issues, so that YBD knows where to look for a
morph file referred, I would propose to use a different relative base
directory depending on the morph being parsed, and to identify that
base path by walking the submodule tree first.
So, when parsing:
gnome/freedesktop/gnu-linux/strata/core.morph
And encountering a build-depends on strata/build-essential.morph,
the base directory for that parse can be determined as:
gnome/freedesktop/gnu-linux
Determining this path should be easy enough using git tooling to
introspect from what submodule the parsed file belongs to.
The other problem is:
DEFAULTS file
~~~~~~~~~~~~~
For the defaults file, I would propose to only have the DEFAULTS file
present in the base gnu-linux submodule.
This is a tricky problem though, as it's conceivable that different
modules would like to have their own DEFAULTS.
In this case, should it ever arise, it would be important that YBD
builds gnu-linux with the DEFAULTS provided by gnu-linux, and that
freedesktop be built with the gnu-linux DEFAULTS unless freedesktop
provides it's own DEFAULTS and so on. I.e. it would be important to
not merge the DEFAULTS found in nested modules, instead each modules
chunk should stay with it's own determined DEFAULTS.
All in all, this is a little more tricky than I originally suspected
but that's always the case. Still I think this is a largely important
move to make.
Best,
-Tristan
_______________________________________________
baserock-dev mailing list
baserock-dev(a)baserock.org
http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/baserock-de
v-baserock.org
1:43 a.m.
On Thu, 2016-10-13 at 22:51 +0900, Tristan Van Berkom wrote:
On Wed, 2016-10-12 at 17:43 +0900, Tristan Van Berkom wrote:
>
> Hi again,
>
Following up on my own email again.
[...]
And again once more.
Since I discovered I have similar requirements I need to meet, I'm
punting the module separation to a later time when I can solve both
problems at once.
The first step will be to still perform a cleanup job in definitions,
removing any unmaintained systems and any dead code in the tree.
The module separation will still be required, and even more so, but the
problems I need to solve involve having a definitions module depend on
something that is not a definitions module.
A few examples of the type of dependency I want to support:
Binary SDK sysroot
~~~~~~~~~~~~~~~~~~
Its been known to occur that a provider might deliver a set of
production packages in rpm form along with an SDK for development.
The intention being that a consumer might distribute the rpms on
their device/product/appliance and use the provided SDK to build
their own software which would run on an appliance with those rpms
installed.
No, this is not a development practice that we would want to
encourage, but it's a reality we should be prepared to deal with
and having tooling which permits building against such an SDK
is a good thing to have.
In this scenario it would be interesting to:
a.) Allow a definitions module to depend on a tarball/sha256sum
so that those definitions can be built against a given SDK
b.) Allow extensibility from our tooling so that such a user
building chunks against an SDK can deploy their chunks to
rpm packages
OSTree Repository & Commit ID (and Flatpak Runtimes/SDKs)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Flatpak is becoming 'a thing', and it will be interesting to be
able to build against their runtimes/SDKs
Whether we want to build a runtime or an app, we should be able
to have some definitions depend on a signed OSTree commit
and deploy to another output OSTree repository.
Set of whatever happens to be installed on the host
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
I doubt this will ever be a real use case, but the host itself is
another plausible base to build software on.
An example of this is GNOME's JHBuild, which has a sysdeps stage
ensuring that a given set of packages are installed on your
debian or fedora host. This doesnt work very well in practice
(as in, mostly only the fedora build hosts work, and only if
you have a host that has very recent software, like less than
5 years old).
Still, it's plausible to say "Build this definitions module
by depending on a package/version vector being already installed
on the host; without doing any chroot or such, just build it into
a prefix to run on my host".
Not sure if that last use case is worth supporting, but the other two
are clearly important and should be fairly straight forward.
That said, a hypothetical GNOME definitions module which depends on
a hypothetical freedesktop definitions module, should equally be
buildable against a freedesktop flatpak SDK, I see the splitting of
these modules as the same problem and as such I want to solve them in
the same way.
Cheers,
-Tristan
9:28 a.m.
Hi Tristan
On 10/10/16 11:13, Tristan Van Berkom wrote:
Clutter
~~~~~~~
TL;DR: Creating a system with baserock means that one has to work in a
git repository cluttered with a vast amount of irrelevant "stuff".
I like this stuff. I like having a vast amount of resources, assemblages
and build instructions to draw ideas from and form solutions from. I
like big tubs of lego bricks of different shapes and sizes and
capabilities. Terratorializing these into separate git repos seems to me
dogmatic. I'd rather simply examine how it appears after we've swept up
that which we have no interest in maintaining.
Changes are pushed/forced upon higher level projects/components
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
TL;DR: Changes in low level strata are forced upon thier consumers,
regularly causing breakage and making it impossible for projects
depending on low level strata to ensure their systems actually work as
expected.
This is theoretically plausible but I'm not sure I've ever actually had
this happen because I can judge the validity of any strata I consume
before I take them. I'm neither convinced there's any essential reason
the existing stratum needs to be *replaced* with the new version in tree.
Br,
Dan
9:02 a.m.
On Mon, 2016-10-10 at 14:28 +0100, Daniel Firth wrote:
Hi Tristan
On 10/10/16 11:13, Tristan Van Berkom wrote:
>
>
>
> Clutter
> ~~~~~~~
> TL;DR: Creating a system with baserock means that one has to work
> in a
> git repository cluttered with a vast amount of irrelevant "stuff".
I like this stuff. I like having a vast amount of resources,
assemblages
and build instructions to draw ideas from and form solutions from. I
like big tubs of lego bricks of different shapes and sizes and
capabilities. Terratorializing these into separate git repos seems to
me
dogmatic. I'd rather simply examine how it appears after we've swept
up
that which we have no interest in maintaining.
This is a fair point from the perspective of a person or group who
builds operating systems inside their own bubble. I.e. in a case where
the maintainership of a single project which builds many operating
systems is a single group.
This is currently the case at Baserock, but I dont agree that it should
be the case. I would instead dream of a world where we offer multiple
consumable components from Baserock and allow downstream consumers to
consume parts of it, without either:
o having to fork the entire definitions repository in their own
project, in which they might not be so inclined to manage their
own build instructions inside such a huge repository.
o having to rebase their work on definitions in order to pull
changes from baserock.
The proposed split would allow downstreams to:
o Consume only the parts they want to build their appliance
on top of
o View these components as opaque chunks separate from their
own body of work
o Propose patches/changes related only to these components
It would also facility a possible world where, should we become popular
enough, we could offload the maintenance of some systems to their
respective downstreams. It would be grand I think, if GNOME ended up
maintaining the GNOME definitions on git.gnome.org, and if the GENIVI
demo platform were maintained by some joint committee, possibly on some
other infrastructure.
>
> Changes are pushed/forced upon higher level projects/components
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> TL;DR: Changes in low level strata are forced upon thier consumers,
> regularly causing breakage and making it impossible for projects
> depending on low level strata to ensure their systems actually work
> as
> expected.
>
This is theoretically plausible but I'm not sure I've ever actually
had
this happen because I can judge the validity of any strata I consume
before I take them. I'm neither convinced there's any essential
reason
the existing stratum needs to be *replaced* with the new version in
tree.
These are very real problems, let me try to voice them:
In my brief few month stint last year building the GNOME Desktop with
Baserock, it has happened on numerous occasions that some change made
in foundation or core has broken the functionality of GNOME, whether or
not it has broken the actual *build* of GNOME I'm not sure I recall.
On one occasion, a change was made so that systemd would no longer
build --with-compat-libs, which is a configure switch that is expected
to go away after some time once all systemd library depending modules
have been updated to use the newer libsystemd.
In the above case:
a.) The maintainer of the GNOME build may not be even interested
in building the 'master' of everything GNOME, perhaps that
maintainer is intentionally targeting a specific and stable
version of GNOME components. Those stable components which have
not yet been updated to use libsystemd yet; but we are forced
to either backport a patch for things to build or not build
stable GNOME anymore ?
b.) The maintainer of the GNOME build may be building GNOME master
but some of the upstreams have not yet updated their builds
to use the new libsystemd.
In this case my work on GNOME must be temporarily derailed,
whilst I spend time patching a module to build against
libsystemd instead of the older libs, and probably try to
upstream that patch, before I can get back to my primary
focus: To build a GNOME system that works and deal with
actual integration issues ?
With the proposed splitup, I could maintain a stable 3.20 branch of the
GNOME system while depending on a stable (or old enough) version of the
freedesktop module's strata - if I am lucky enough, the maintainer of
the freedesktop module might even be so kind as to offer me a stable
branch with a yearly or 6 month release cycle which I could base my
work off of.
Best,
-Tristan
Br,
Dan
_______________________________________________
baserock-dev mailing list
baserock-dev(a)baserock.org
http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/baserock-de
v-baserock.org
10:39 a.m.
On 10/10/16 14:02, Tristan Van Berkom wrote:
The proposed split would allow downstreams to:
o Consume only the parts they want to build their appliance
on top of
o View these components as opaque chunks separate from their
own body of work
o Propose patches/changes related only to these components
It would also facility a possible world where, should we become popular
enough, we could offload the maintenance of some systems to their
respective downstreams. It would be grand I think, if GNOME ended up
maintaining the GNOME definitions on git.gnome.org, and if the GENIVI
demo platform were maintained by some joint committee, possibly on some
other infrastructure.
I'm definitely for this, but I'd probably need to see it in a prototype.
How might we handle the volume of aliases that could potentially be
required by a giant bucket of definitions sourced from places that don't
utilize downstream troves?
On one occasion, a change was made so that systemd would no longer
build --with-compat-libs, which is a configure switch that is expected
to go away after some time once all systemd library depending modules
have been updated to use the newer libsystemd.
I'm *very* hopeful that this, as well as the architecture topology that
you mentioned, and any other ways of contextually parameterising build
instructions can be solved in a unified way using coeffects. I hope to
put together a solid case for this at some point.
In the above case:
a.) The maintainer of the GNOME build may not be even interested
in building the 'master' of everything GNOME, perhaps that
maintainer is intentionally targeting a specific and stable
version of GNOME components. Those stable components which have
not yet been updated to use libsystemd yet; but we are forced
to either backport a patch for things to build or not build
stable GNOME anymore ?
b.) The maintainer of the GNOME build may be building GNOME master
but some of the upstreams have not yet updated their builds
to use the new libsystemd.
In this case my work on GNOME must be temporarily derailed,
whilst I spend time patching a module to build against
libsystemd instead of the older libs, and probably try to
upstream that patch, before I can get back to my primary
focus: To build a GNOME system that works and deal with
actual integration issues ?
This is indeed awkward, I am personally happy to maintain more versions
of assemblages that still carry relevance, as happens with the multiple
gstreamer/-0.10 strata currently. We no longer need to duplicate entire
stacks so this should be marginally easier.
Br,
Dan
5:14 a.m.
On Mon, 2016-10-10 at 15:39 +0100, Daniel Firth wrote:
On 10/10/16 14:02, Tristan Van Berkom wrote:
>
>
> The proposed split would allow downstreams to:
>
> o Consume only the parts they want to build their appliance
> on top of
>
> o View these components as opaque chunks separate from their
> own body of work
>
> o Propose patches/changes related only to these components
>
>
> It would also facility a possible world where, should we become
> popular
> enough, we could offload the maintenance of some systems to their
> respective downstreams. It would be grand I think, if GNOME ended
> up
> maintaining the GNOME definitions on git.gnome.org, and if the
> GENIVI
> demo platform were maintained by some joint committee, possibly on
> some
> other infrastructure.
I'm definitely for this, but I'd probably need to see it in a
prototype.
How might we handle the volume of aliases that could potentially be
required by a giant bucket of definitions sourced from places that
don't
utilize downstream troves?
This is not a terribly difficult problem to solve.
Currently we configure aliases in the ybd configuration and refer to
these same aliases in definitions, arguably the ybd configuration
should be allowed to override alias configurations; but the definitions
repositories/projects themselves should be providing the default value
for their respective 'upstream' troves.
Fortunately not only is it a fairly easy problem to solve, but solving
that problem is not an immediate blocker to converging towards this
model: We can solve that at a later date.
>
> On one occasion, a change was made so that systemd would no longer
> build --with-compat-libs, which is a configure switch that is
> expected
> to go away after some time once all systemd library depending
> modules
> have been updated to use the newer libsystemd.
I'm *very* hopeful that this, as well as the architecture topology
that
you mentioned, and any other ways of contextually parameterising
build
instructions can be solved in a unified way using coeffects. I hope
to
put together a solid case for this at some point.
I'm not sure what you mean here, the problem is that essentially as the
maintainer of the GNOME system I must depend on a reliable foundation
and core, one that is not subject to the whims of other maintainers
focusing on other projects.
The cause of this problem as I perceive it, is that with our current
development model we allow changes in shared components to be forced
upon their consumers.
It would be great if we could come up with an exact science to say that
"this change in foundation will not carry any ill effect at runtime or
build time to any of it's consumers", but:
a.) We are a long ways from achieving that
b.) Even if we said we could achieve that, the current approach is
still not a model I feel I could trust as an external system
developer who might want to consume foundation.
>
> In the above case:
>
> a.) The maintainer of the GNOME build may not be even interested
> in building the 'master' of everything GNOME, perhaps that
> maintainer is intentionally targeting a specific and stable
> version of GNOME components. Those stable components which
> have
> not yet been updated to use libsystemd yet; but we are
> forced
> to either backport a patch for things to build or not build
> stable GNOME anymore ?
>
> b.) The maintainer of the GNOME build may be building GNOME
> master
> but some of the upstreams have not yet updated their builds
> to use the new libsystemd.
>
> In this case my work on GNOME must be temporarily derailed,
> whilst I spend time patching a module to build against
> libsystemd instead of the older libs, and probably try to
> upstream that patch, before I can get back to my primary
> focus: To build a GNOME system that works and deal with
> actual integration issues ?
This is indeed awkward, I am personally happy to maintain more
versions
of assemblages that still carry relevance, as happens with the
multiple
gstreamer/-0.10 strata currently. We no longer need to duplicate
entire
stacks so this should be marginally easier.
So you would propose instead that we duplicate various strata and name
them by their API version (or in fact, we would need to duplicate _all_
strata by API version in order to be free of this situation, or at
least better off).
First, I would much rather avoid a vastly growing tree of definitions,
and second, even if we took this approach we would be left with the
same problem: As the maintainer of a given system that chooses to
consume stable gstreamer-0.10, another developer can *push* changes to
gstreamer-0.10 and force these onto the downstream system maintainer
who consumes it.
The flow of changes here is fundamentally backwards.
Cheers,
-Tristan
Br,
Dan
_______________________________________________
baserock-dev mailing list
baserock-dev(a)baserock.org
http://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/baserock-de
v-baserock.org
6:40 a.m.
On 11/10/16 10:14, Tristan Van Berkom wrote:
So you would propose instead that we duplicate various strata and name
them by their API version (or in fact, we would need to duplicate _all_
strata by API version in order to be free of this situation, or at
least better off).
So I went and found a passage that's a bit easier to digest and might
make this a bit clearer. Additionally here's a repost of my explanation
yesterday in IRC in case anyone missed it.
<locallycompact> In essence explicit build-depend by name are coeffects
that have zero expressive power. i.e every chunk provides statements
that say "I provide NAME", "I require NAME".
<locallycompact> that's how we are currently
<locallycompact> But something might say "I require GRAPHICS STACK by
way of SPECIFIC DETAIL"
<locallycompact> "otherwise I don't want it, I want something else"
<locallycompact> the graphics stack knows all its predecessive coeffects
and provides them as proof/history
<locallycompact> all of this can be calculated ahead of build
<locallycompact> that specific detail might be something as fundamental
as "x86_64" or other feature
<locallycompact> and because they don't need to be explicitly evaluated
when they're provided, the app can backwards cause all of the
coeffectual specific details until it hits a solution or many solutions
it likes
<locallycompact> then it passes those forward
<locallycompact> sometimes it may genuinely not matter (like the GTK,
X11), but it shouldn't be left up to GTK to decide that for the entire
stack, it just says "I am GTK by way of X11"
<locallycompact> so app can context on GTK or GTK[X11]
<locallycompact> meaning I want a proof GTK were to be actually built
with X11 if I'm going to use it
Tristan then asked what exactly do I mean by "proof/history" and I
didn't have a satisfactory answer so I'll try and give one under a
slightly different interpretation. Try this section on the resource
interpretation, then read the rest of this email.
https://en.wikipedia.org/wiki/Linear_logic#The_resource_interpretation
Ok, now imagine each chunk as a vending machine that gives you back
capabilities according to some rules. One chunk might say "Give me a C
compiler by way of arm and I will give you back an application OR give
me a C compiler by way of x86 and I will give you back an application
that provides a certain feature *AND* a supplimentary debug
application", for example.
The important thing to highlight is that these expressions are first
class resources themselves, allowing a chunk to say for example "Give me
an arm compiler and I will give you back 'a way to build GTK by way of
X11 on arm'".
From the point of view of the chunk, it recieves a bundle of these
expressions representing the capacities, requirements, and degrees of
freedom and unfreedom the system has left to build things. This is what
I mean by a "proof", or a "proof net", it represents the system's
compromising agreement among all of its components.
So let's say at one point in the deduction CORBA has managed to prove
that it is capable of being built at any version, given the information
it has gathered. It hasn't yet made a decision which version because the
entire agreement has yet to be solved. This fact can be interpreted in a
multitude of different ways. "I can provide CORBA at any version but you
must pick one" or "I can provide CORBA at absolutely every version under
different prefixes should you so want it". It should also be possible to
express such bizarrities that two chunks each want to build against one
of either two different libraries, but they can not both build against
the same library because of a compatibility quirk. (i.e, they each must
pick one for themselves)
There's some immediate holes in this interpretation, (we don't generally
want to call a resource like a C compiler "consumed", as it can be
reused indefinitely), which is what led me to try and fit the
affine/linear/relevant model (
https://en.wikipedia.org/wiki/Substructural_type_system#Different_substru...
) to "include-mode", which is potentially overkill, but it would allow
us to declare things like "Here's the definition for the rust compiler,
but only actually build it if there's a rust application in the system
otherwise don't waste the time."
Br, Dan
7:10 a.m.
On 10/13/16, Daniel Firth <dan.firth(a)codethink.co.uk> wrote:
On 11/10/16 10:14, Tristan Van Berkom wrote:
>
> So you would propose instead that we duplicate various strata and name
> them by their API version (or in fact, we would need to duplicate _all_
> strata by API version in order to be free of this situation, or at
> least better off).
So I went and found a passage that's a bit easier to digest and might
make this a bit clearer. Additionally here's a repost of my explanation
yesterday in IRC in case anyone missed it.
<locallycompact> In essence explicit build-depend by name are coeffects
that have zero expressive power. i.e every chunk provides statements
that say "I provide NAME", "I require NAME".
<locallycompact> that's how we are currently
<locallycompact> But something might say "I require GRAPHICS STACK by
way of SPECIFIC DETAIL"
<locallycompact> "otherwise I don't want it, I want something else"
<locallycompact> the graphics stack knows all its predecessive coeffects
and provides them as proof/history
<locallycompact> all of this can be calculated ahead of build
<locallycompact> that specific detail might be something as fundamental
as "x86_64" or other feature
<locallycompact> and because they don't need to be explicitly evaluated
when they're provided, the app can backwards cause all of the
coeffectual specific details until it hits a solution or many solutions
it likes
<locallycompact> then it passes those forward
<locallycompact> sometimes it may genuinely not matter (like the GTK,
X11), but it shouldn't be left up to GTK to decide that for the entire
stack, it just says "I am GTK by way of X11"
<locallycompact> so app can context on GTK or GTK[X11]
<locallycompact> meaning I want a proof GTK were to be actually built
with X11 if I'm going to use it
I'm wary of trying to make this all this dependency resolution happen
implicitly; in my experience that sort of thing is nice when it works,
but very hard to figure out when it doesn't.
I'm not saying this can't be done, but I am saying don't ignore the
strength of a simple, explicit model that is slightly more painful to
use in the general case, but significantly easier to debug when
somebody makes a mistake.
For example, part of the reason I don't buy into the Haskell dream is
that I can't figure out most of its error messages when I get
something wrong, maybe that's cause I'm dumb, but either way it means
its capabilities aren't useful to me and in practice I can get much
more done using simpler tools.
In terms of depending on specific capabilities, RPM packagers do this
a lot already. I've not found this well documented anywhere, and I
think it's implemented purely by convention, but you can say things
like "depend on this pkg-config file being present"[1], "depend on
this Perl module being present"[2], etc. Then in theory 2
implementations of a program or library are interchangable, although I
can't think of any examples of this being used in practice.
1. http://pkgs.fedoraproject.org/cgit/rpms/tracker.git/tree/tracker.spec#n49
2.
http://pkgs.fedoraproject.org/cgit/rpms/perl-Algorithm-NaiveBayes.git/tre...
7:34 a.m.
On 13/10/16 12:10, Sam Thursfield wrote:
I'm not saying this can't be done, but I am saying don't
ignore the
strength of a simple, explicit model that is slightly more painful to
use in the general case, but significantly easier to debug when
somebody makes a mistake.
The simple model is still a trivial consequence of the resource
interpretation where every chunk/assemblage provides only one thing
denoted by an exact string, and every chunk depends on a list of strings
multiplicatively conjoined. The syntax wouldn't even need to change
particularly. If you don't need the expressivity nothing should change
for you.
Br, Dan
12:59 p.m.
Proposal to use recursive git submodules to address this issue
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Yes I said it, and I know there are plenty of reasons to hate a
submodule, obnoxious things as they are. However I have not yet been
able to come up with a better solution.
I suppose an alternative worth considering would be android repo tool,
I've been using it briefly and it doesn't seem entirely deplorable
although I doubt I have been using its workflow as intended, seems at
least worth considering.
1:49 p.m.
On Mon, Oct 10, 2016 at 9:59 AM, Matt Rice <ratmice(a)gmail.com> wrote:
> Proposal to use recursive git submodules to address this issue
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> Yes I said it, and I know there are plenty of reasons to hate a
> submodule, obnoxious things as they are. However I have not yet been
> able to come up with a better solution.
I suppose an alternative worth considering would be android repo tool,
I've been using it briefly and it doesn't seem entirely deplorable
although I doubt I have been using its workflow as intended, seems at
least worth considering.
sorry I think i see why repo probably didn't make the list, it is
probably a pain to lock down a manifest to a particular reproducable
checkout or at least requires some manifest generating script to do
so, as typically they refer to branches or tags
the nice thing i suppose is you can have one repository with multiple
manifest files
for various revisions...
https://github.com/seL4/sel4test-manifest
2082
days inactive
2091
days old
14 comments
5 participants
participants (5)
-
Daniel Firth -
Matt Rice -
Paul Sherwood -
Sam Thursfield -
Tristan Van Berkom