[PATCH 0/2] morph deploy rsync/remote-virtualbox
by Jannis Pohlmann
Repo: ssh://git@git.baserock.org/baserock/baserock/morph.git
Ref: jannispohlmann/deployment-plugin
SHA1: aa363025d0b699b4251ee80aeec2b863e57dd7ec
Jannis Pohlmann (2):
Add a "morph deploy" plugin with a "morph deploy rsync" command
Add "morph deploy remote-virtualbox" command to deployment plugin
morphlib/plugins/deployment_plugin.py | 220 ++++++++++++++++++++++++++++++++++
1 file changed, 220 insertions(+)
create mode 100644 morphlib/plugins/deployment_plugin.py
--
1.7.11.4
10 years, 4 months
Issues with ccache, URL escaping and local repos
by Jannis Pohlmann
Hi,
this is a mail mainly to remind us that there is a problem involving the
new ccache support, URL escaping and system branch repos. The problem is
this:
When you run "morph build <system>" in a system branch and the build
involves locally cloned repos with colons in the filename (e.g.
"upstream:linux"), morph will attempt to create a
/src/cache/ccache/upstream:linux
directory and will try to bind-mount that directory into the
staging area using
mount --bind /src/cache/ccache/upstream:linux
There are two problems with this:
a) mainly, this doesn't work: the mount command will assume that
"/src/cache/ccache/upstream" is a socket family or someting and
will fail with a "bad address" error.
b) secondly, the ccache directory name is not "unique", i.e., we
are not using the same repo URL escaping mechanism as we are
using for the local and remote repo caches.
The way this currently works is that the LocalRepoCache is asked to
create a repo from a file:// URL, it will simply use the local path. If
it is asked to do the same for a non-file URL, it will cache the repo
etc. The Builder classes uses this to obtain the full local path of e.g.
the /src/workspace/your/system/branch/upstream:linux repo, then takes
the basename and creates /src/cache/ccache/upstream:linux for ccache.
A quick fix is to simply replace : with _ in Builder. A better fix would
probably be to use URL escaping based on the resolved original repo URL.
So not escape file:///src/workspace/your/branch/upstream:linux but
resolve it's morph.repository URL and use that.
Opinions?
- Jannis
--
Jannis Pohlmann
Senior Software Developer
Codethink Limited
http://codethink.co.uk
10 years, 4 months
Working on a graphical installer
by Jannis Pohlmann
Hi everyone,
I've taken the quiet days around christmas as an excuse to do some spare
time hacking around Baserock. I had a couple of projects in mind and
decided to do something graphical again for a change: an installer for
Baserock.
Here is a very early screenshot of the intro page:
http://users.xfce.org/~jannis/screenshots/baserock-installer-2012-12-23-0...
The current concept, plans and of course the code can be viewed here:
https://github.com/jannis/baserock-installer
It does everything up to (including) downloading all files for a release
but it doesn't set up VMs yet. So, yes, the hard parts are still
missing. Anyhow, I am quite pleased so far. I even manged to re-use some
of my first ever open source code from 2006 (a threaded downloader with
batch processing, cancellation and signal support)! ;)
I'm planning on adding support for creating/updating VirtualBox machines
tomorrow (looking at the clock, what I mean is later today). libvirt,
QEMU and others might follow but for that I will most likely need
contributors.
What do you think? Any other ideas on what the installer should or could
do and how it should or could do it?
- Jannis
--
Jannis Pohlmann
Senior Software Developer
Codethink Limited
http://codethink.co.uk
10 years, 5 months
Re: Working on a graphical installer
by Rob Taylor
Looks good! One thing I'd have it do is to ask for the eth port the
user wants to use for nfs booting their boards, and setting up a bit
ridged network adapter in the vm for that port.
Extra bonus points for setting up the networking in the vm so there's a
static ip on that adapter - I guess an easy approach would be to have
eth0 be natted, eth1 bridged to a real wired host adaptor and
statically assigned a 10.x.x.x address. This should make morph deploy
nfs much easier..
Hope that makes sense!
Merry xmas
Rob
On 23 December 2012 06:25:05 Jannis Pohlmann
<jannis.pohlmann(a)codethink.co.uk> wrote:
> Hi everyone,
>
> I've taken the quiet days around christmas as an excuse to do some spare
> time hacking around Baserock. I had a couple of projects in mind and
> decided to do something graphical again for a change: an installer for
> Baserock.
>
> Here is a very early screenshot of the intro page:
>
> http://users.xfce.org/~jannis/screenshots/baserock-installer-2012-12-23-0...
>
> The current concept, plans and of course the code can be viewed here:
>
> https://github.com/jannis/baserock-installer
>
> It does everything up to (including) downloading all files for a release
> but it doesn't set up VMs yet. So, yes, the hard parts are still
> missing. Anyhow, I am quite pleased so far. I even manged to re-use some
> of my first ever open source code from 2006 (a threaded downloader with
> batch processing, cancellation and signal support)! ;)
>
> I'm planning on adding support for creating/updating VirtualBox machines
> tomorrow (looking at the clock, what I mean is later today). libvirt,
> QEMU and others might follow but for that I will most likely need
> contributors.
>
> What do you think? Any other ideas on what the installer should or could
> do and how it should or could do it?
>
> - Jannis
>
> --
> Jannis Pohlmann
> Senior Software Developer
> Codethink Limited
> http://codethink.co.uk
>
> _______________________________________________
> baserock-dev mailing list
> baserock-dev(a)baserock.org
> http://vlists.pepperfish.net/cgi-bin/mailman/listinfo/baserock-dev-basero...
10 years, 5 months
[PATCH 0/3] V1: handle stratum build depends better
by Sam Thursfield
Repository: git://git.baserock.org/baserock/baserock/morph
Branch: samthursfield/stratum-build-depends
Sha1: 0b1471e2ab934adecfa7b15bb6841bb6c551e20a
This patch fixes up 'morph edit' and 'morph branch' so that they don't
break when strata have build dependencies on each other. It also adds
a check while creating the build order to make sure that a stratum
cannot build-depend on a different version of a morphology to the
one that is actually in the system.
At this point I'm considering it not an error to build-depend on
something not in the final system, since you may not want to include
everything needed for building in the output. However, there's no
test for this yet.
We currently allow strata within a system to have the same morphology
name, and to given a different name within the system using 'alias'.
However, some code does not honour 'alias' in systems, including the
first patch in this mail. I believe we should disallow alias in
systems so that it is not possible for strata to have the same name
as other strata. Fully disallowing the field requires code, tests,
etc., but I don't think anybody actually uses the feature so it's not
a priority.
Sam Thursfield (3):
Validate stratum build-depends against system contents
morph edit: Update build-depends that refer to the edited stratum
morph build: Update repo and ref in stratum build-depends fields
morphlib/buildcommand.py | 19 +++
morphlib/plugins/branch_and_merge_plugin.py | 160 +++++++++++---------
.../build-handles-stratum-build-depends.script | 41 +++++
tests.as-root/setup | 7 +
.../edit-updates-stratum-build-depends.script | 77 ++++++++++
.../edit-updates-stratum-build-depends.stdout | 48 ++++++
6 files changed, 279 insertions(+), 73 deletions(-)
create mode 100755 tests.as-root/build-handles-stratum-build-depends.script
create mode 100755 tests.branching/edit-updates-stratum-build-depends.script
create mode 100644 tests.branching/edit-updates-stratum-build-depends.stdout
--
1.7.10.4
10 years, 5 months
[PATCH] Silence logging during unit tests
by Lars Wirzenius
repo: git://git.baserock.org/baserock/baserock/morph
branch: liw/silence-log-messages-during-unit-tests
commit: f352c4e7804f64a29670c23e50cff3c56a902496
Because logging output during unit tests is ugly.
Lars Wirzenius (1):
Silence log messages during unit tests
morphlib/remoteartifactcache.py | 8 ++++----
morphlib/remoteartifactcache_tests.py | 6 ++++--
2 files changed, 8 insertions(+), 6 deletions(-)
--
1.7.9.rc0.27.ge0b3e
10 years, 5 months
[PATCH 0/2] Add support for ccache (v2)
by Jannis Pohlmann
Repo: ssh://git@git.baserock.org/baserock/baserock/morph.git
Branch: jannispohlmann/add-ccache-support
SHA1: d0eee4db796f40ef479f603dc0402090f99d7dc5
I've added a patch to finish Jonathan's work. Please verify that
my modifications make sense. This branch has been added to
baserock/merge-queue and is currently piped through the test
suite.
Jannis Pohlmann (1):
Fix default setting for compiler cache dir; update tests
Jonathan Maw (1):
Add support for ccache
morphlib/app.py | 22 ++++++++------------
morphlib/buildenvironment.py | 4 +---
morphlib/buildenvironment_tests.py | 8 --------
morphlib/builder2.py | 24 ++++++++++++++++++++++
.../morph-tag-creates-commit-and-tag.stdout | 4 ++--
.../morph-tag-fails-if-tag-exists.stderr | 4 +---
.../morph-tag-tag-works-as-expected.stdout | 4 ++--
...rph-tag-works-with-multiple-morphs-repos.stdout | 4 ++--
8 files changed, 40 insertions(+), 34 deletions(-)
--
1.7.11.4
10 years, 5 months
Changing how build ordering is done in Morph
by Lars Wirzenius
Hi,
Currently we are creating the build order for an artifact by computing
a dependency graph, and then doing a sort of topological sort to compute
a flat ordering of the component artifacts to build. This works, but
it could be replaced by a simple depth-first graph traversal. Doing this
would mean the code in Morph is somewhat simpler, which means it'll be
easier to change in the future, when we need to do that.
I'd like to suggest we do the following:
* BuildCommand.compute_build_order: return an artifact only
* BuildCommand.build_in_order and build_artifacts: remove
* morphlib.buildorder*.py: remove
* Add a new method BuildCommand.build_recursively, which takes an artifact
object, and recurses in depth-first order so that it builds leaves
first, then their parents, etc, by calling build_artifact. This will
guarantee things are built so that their dependencies are built first.
* BuildCommand.build: call self.build_recursively instead of build_artifacts
* morphlib/plugins/show_dependencies_plugin.py: replace use of BuildOrder
with BuildCommand.compute_build_order, and traversing the dependency
tree of the returned artifact
I thought I'd request opinions before making the code changes, since this
is somewhat invasive. However, the actual changes look like they'll be
pretty small.
--
http://www.codethink.co.uk/ http://wiki.baserock.org/ http://www.baserock.com/
10 years, 5 months
[PATCH 0/3] Resolve ambiguous refs correctly
by Sam Thursfield
Repository: git://git.baserock.org/baserock/baserock/morph
Branch: samthursfield/ambiguous-refs
Sha1: bec9ecbbb7ab988488c77ea1fe995164ec0c073f
This solves a bug Jonathan found yesterday where `morph build`
failed due to looking at incorrect refs. It was confused by the
existance of another branch with 'master' in its name in the same
repo. That's now fixed.
The final patch is for what must be an unrelated issue in the
test suite, but I assume it's uncontroversial.
Sam Thursfield (3):
Always use `git rev-parse --verify` to resolve refs
Add a test to catch any future use of git show-ref
morph branch: Check if the ref already exists before anything else
morphlib/cachedrepo.py | 38 +++++++++++++--------------
morphlib/cachedrepo_tests.py | 38 +++++++++++----------------
morphlib/git.py | 2 +-
morphlib/plugins/branch_and_merge_plugin.py | 15 +++++------
tests.branching/ambiguous-refs.script | 6 +++--
tests/ambiguous-refs.script | 32 ++++++++++++++++++++++
tests/update-gits-chunk.script | 4 +--
7 files changed, 79 insertions(+), 56 deletions(-)
create mode 100755 tests/ambiguous-refs.script
--
1.7.10.4
10 years, 5 months
[PATCH] Add ccache to morph (v1)
by Jonathan Maw
head: 52cdb57f9f8fe6e590d0a24bef596edebb1b65bd
branch: jonathanmaw/ccache-patch
repo: ssh://git@trove.baserock.org/baserock/baserock/morph
Note: To test this, you will need to build a ccache chunk and add it
to your staging-filler.
Jonathan Maw (1):
Add support for ccache
morphlib/app.py | 19 +++++--------------
morphlib/buildenvironment.py | 4 +---
morphlib/builder2.py | 24 ++++++++++++++++++++++++
3 files changed, 30 insertions(+), 17 deletions(-)
--
1.7.11.7
10 years, 5 months
