[RFC] Declarative Definitions
paul.sherwood at codethink.co.uk
Fri Feb 6 12:43:10 GMT 2015
at yesterday's meetup there was a BoF on improving Definitions (which
we often have referred to as morph files, or morphologies)
My rough notes of the discussion  lead me to think the actions are
- document migration philosophy and tooling (expect frequent small
- get 'version: 0' in place, tooling to detect it
- define 'version: 1', small delta, .def,
I've taken a shot at the first of these steps at , and would
appreciate feedback. For those who would prefer to comment inline on
this list, the text of  follows at the end of this mail.
Subject to feedback, I'll be aiming to work up drafts of 'version: 0'
and 'version: 1' at  and  - others are welcome to contribute
things there before me or with me :-)
# Declarative Definitions
We want to establish a standard syntax for describing software
components and systems. Learning from the experiences we have had with
morph files, we have established some general principles:
- We create descriptions of software components, systems, and sets of
systems. We call the descriptions Definitions.
- Software tools can use Definitions to create and operate on these
components and systems. By adopting the Definitions approach we are
aiming for traceability, reproducibility and ultimately deterministic
behaviour of the Defined systems and components.
- As far as possible Definitions contain all the information necessary
for build, deployment, instantiation etc.
- Definitions are a a set of text files in a directory tree.Typically
the files are stored in a single Git repo
- Definitions describe Components. Each Definition file may describe
one or more Components.
- A Component may consist of
- a single software project e.g. systemd, or
- a set of software projects, e.g. GNOME, or
- an integrated software system, e.g. a web server appliance, or
- a set or cluster of software systems, e.g. a ci controller with
several worker nodes
- Components may be nested and contain other components. The
information which specifies a Component may be split across multiple
- We have currently chosen YAML as the format for representing data
relationships in Definitions.
- Most Definitions contain YAML, but some definitions contain shell
scripts or other instructions to be executed as part of a configuration,
build, installation, deployment, instantiation, upgrade or other process
which tools can apply to Components or systems.
We have evolved morph files through many states over several years,
without providing a mechanism for versioning as the syntax has evolved.
As of the [[meetup]] we have decided to adopt a versioning scheme,
broadly as follows:
- The Definitions directory tree contains a file called VERSION. If
this file is not present, tooling may assume [[definitions-V0]] or fail
- The first line of the VERSION file must contain `version: ` followed
by an integer. The integer uniquely defines the version of the
- Later lines of the VERSION file are expected to contain YAML
describing default information which tooling such as Morph may require.
This information could include, for example
- default PATH
- locations for git server and artifact server
- default ARCH target etc.
- By specifying defaults here we aim to reduce complexity, hidden logic
and implicit assumptions in tools such as morph. The overall objectives
for this are
- running baserock tools on a set of Definitions should give
predictable and reproducible results - if it works for Aananth on his
machine it should work for Brenda on hers too
- it should be possible for new tooling to instrument, analyse and
make operations on the information contained in Definitions, with
confidence that there are no hidden tricks performed by existing tools
(e.g. anything that has to be done differently for ARMv7 should be
explicitly declared in Definitions, not as conditional logic in the
build tool itself)
- We expect that the version number will rise relatively quickly, so
- tooling should expect and handle changes gracefully
- if something is expected to break between VERSION N and VERSION
N+1, a migration script must be provided. We see this as similar to ruby
on rails migrations, and hope to standardize this process over the first
few VERSION iterations
More information about the baserock-dev