this is a mail mainly to remind us that there is a problem involving the
new ccache support, URL escaping and system branch repos. The problem is
When you run "morph build <system>" in a system branch and the build
involves locally cloned repos with colons in the filename (e.g.
"upstream:linux"), morph will attempt to create a
directory and will try to bind-mount that directory into the
staging area using
mount --bind /src/cache/ccache/upstream:linux
There are two problems with this:
a) mainly, this doesn't work: the mount command will assume that
"/src/cache/ccache/upstream" is a socket family or someting and
will fail with a "bad address" error.
b) secondly, the ccache directory name is not "unique", i.e., we
are not using the same repo URL escaping mechanism as we are
using for the local and remote repo caches.
The way this currently works is that the LocalRepoCache is asked to
create a repo from a file:// URL, it will simply use the local path. If
it is asked to do the same for a non-file URL, it will cache the repo
etc. The Builder classes uses this to obtain the full local path of e.g.
the /src/workspace/your/system/branch/upstream:linux repo, then takes
the basename and creates /src/cache/ccache/upstream:linux for ccache.
A quick fix is to simply replace : with _ in Builder. A better fix would
probably be to use URL escaping based on the resolved original repo URL.
So not escape file:///src/workspace/your/branch/upstream:linux but
resolve it's morph.repository URL and use that.
Senior Software Developer
I've taken the quiet days around christmas as an excuse to do some spare
time hacking around Baserock. I had a couple of projects in mind and
decided to do something graphical again for a change: an installer for
Here is a very early screenshot of the intro page:
The current concept, plans and of course the code can be viewed here:
It does everything up to (including) downloading all files for a release
but it doesn't set up VMs yet. So, yes, the hard parts are still
missing. Anyhow, I am quite pleased so far. I even manged to re-use some
of my first ever open source code from 2006 (a threaded downloader with
batch processing, cancellation and signal support)! ;)
I'm planning on adding support for creating/updating VirtualBox machines
tomorrow (looking at the clock, what I mean is later today). libvirt,
QEMU and others might follow but for that I will most likely need
What do you think? Any other ideas on what the installer should or could
do and how it should or could do it?
Senior Software Developer
Looks good! One thing I'd have it do is to ask for the eth port the
user wants to use for nfs booting their boards, and setting up a bit
ridged network adapter in the vm for that port.
Extra bonus points for setting up the networking in the vm so there's a
static ip on that adapter - I guess an easy approach would be to have
eth0 be natted, eth1 bridged to a real wired host adaptor and
statically assigned a 10.x.x.x address. This should make morph deploy
nfs much easier..
Hope that makes sense!
On 23 December 2012 06:25:05 Jannis Pohlmann
> Hi everyone,
> I've taken the quiet days around christmas as an excuse to do some spare
> time hacking around Baserock. I had a couple of projects in mind and
> decided to do something graphical again for a change: an installer for
> Here is a very early screenshot of the intro page:
> The current concept, plans and of course the code can be viewed here:
> It does everything up to (including) downloading all files for a release
> but it doesn't set up VMs yet. So, yes, the hard parts are still
> missing. Anyhow, I am quite pleased so far. I even manged to re-use some
> of my first ever open source code from 2006 (a threaded downloader with
> batch processing, cancellation and signal support)! ;)
> I'm planning on adding support for creating/updating VirtualBox machines
> tomorrow (looking at the clock, what I mean is later today). libvirt,
> QEMU and others might follow but for that I will most likely need
> What do you think? Any other ideas on what the installer should or could
> do and how it should or could do it?
> - Jannis
> Jannis Pohlmann
> Senior Software Developer
> Codethink Limited
> baserock-dev mailing list
This patch fixes up 'morph edit' and 'morph branch' so that they don't
break when strata have build dependencies on each other. It also adds
a check while creating the build order to make sure that a stratum
cannot build-depend on a different version of a morphology to the
one that is actually in the system.
At this point I'm considering it not an error to build-depend on
something not in the final system, since you may not want to include
everything needed for building in the output. However, there's no
test for this yet.
We currently allow strata within a system to have the same morphology
name, and to given a different name within the system using 'alias'.
However, some code does not honour 'alias' in systems, including the
first patch in this mail. I believe we should disallow alias in
systems so that it is not possible for strata to have the same name
as other strata. Fully disallowing the field requires code, tests,
etc., but I don't think anybody actually uses the feature so it's not
Sam Thursfield (3):
Validate stratum build-depends against system contents
morph edit: Update build-depends that refer to the edited stratum
morph build: Update repo and ref in stratum build-depends fields
morphlib/buildcommand.py | 19 +++
morphlib/plugins/branch_and_merge_plugin.py | 160 +++++++++++---------
.../build-handles-stratum-build-depends.script | 41 +++++
tests.as-root/setup | 7 +
.../edit-updates-stratum-build-depends.script | 77 ++++++++++
.../edit-updates-stratum-build-depends.stdout | 48 ++++++
6 files changed, 279 insertions(+), 73 deletions(-)
create mode 100755 tests.as-root/build-handles-stratum-build-depends.script
create mode 100755 tests.branching/edit-updates-stratum-build-depends.script
create mode 100644 tests.branching/edit-updates-stratum-build-depends.stdout
Currently we are creating the build order for an artifact by computing
a dependency graph, and then doing a sort of topological sort to compute
a flat ordering of the component artifacts to build. This works, but
it could be replaced by a simple depth-first graph traversal. Doing this
would mean the code in Morph is somewhat simpler, which means it'll be
easier to change in the future, when we need to do that.
I'd like to suggest we do the following:
* BuildCommand.compute_build_order: return an artifact only
* BuildCommand.build_in_order and build_artifacts: remove
* morphlib.buildorder*.py: remove
* Add a new method BuildCommand.build_recursively, which takes an artifact
object, and recurses in depth-first order so that it builds leaves
first, then their parents, etc, by calling build_artifact. This will
guarantee things are built so that their dependencies are built first.
* BuildCommand.build: call self.build_recursively instead of build_artifacts
* morphlib/plugins/show_dependencies_plugin.py: replace use of BuildOrder
with BuildCommand.compute_build_order, and traversing the dependency
tree of the returned artifact
I thought I'd request opinions before making the code changes, since this
is somewhat invasive. However, the actual changes look like they'll be
This solves a bug Jonathan found yesterday where `morph build`
failed due to looking at incorrect refs. It was confused by the
existance of another branch with 'master' in its name in the same
repo. That's now fixed.
The final patch is for what must be an unrelated issue in the
test suite, but I assume it's uncontroversial.
Sam Thursfield (3):
Always use `git rev-parse --verify` to resolve refs
Add a test to catch any future use of git show-ref
morph branch: Check if the ref already exists before anything else
morphlib/cachedrepo.py | 38 +++++++++++++--------------
morphlib/cachedrepo_tests.py | 38 +++++++++++----------------
morphlib/git.py | 2 +-
morphlib/plugins/branch_and_merge_plugin.py | 15 +++++------
tests.branching/ambiguous-refs.script | 6 +++--
tests/ambiguous-refs.script | 32 ++++++++++++++++++++++
tests/update-gits-chunk.script | 4 +--
7 files changed, 79 insertions(+), 56 deletions(-)
create mode 100755 tests/ambiguous-refs.script
Note: To test this, you will need to build a ccache chunk and add it
to your staging-filler.
Jonathan Maw (1):
Add support for ccache
morphlib/app.py | 19 +++++--------------
morphlib/buildenvironment.py | 4 +---
morphlib/builder2.py | 24 ++++++++++++++++++++++++
3 files changed, 30 insertions(+), 17 deletions(-)