[PATCH 0/8] Add branch-from-image command
by Richard Maw
Repo: git://git.baserock.org/baserock/baserock/morph
Ref: baserock/richardmaw/branch-from-image
Sha1: 348064b89c734928c5ec4d1bc5cb122c3106e192
This patch series includes some required cleanups.
Checkout and Branch share code now.
The RepoAliasResolver should be a little quicker.
The RepoAliasResolver can now give aliases from URLs.
branch-from-image is untested where the root repository
you specify on the command line is different to the one
the image was built from.
It does not work with images built without pushing, since
they have local repository paths.
Some branch and merge cleanups were possible, but the petrify
command still uses a different code path to tag.
Richard Maw (8):
Make metadata include the repo-alias.
Add a regression test for metadata including repo alias
repoaliasresolver: compile patterns on creation
RepoAliasResolver: Add aliases_from_url method
Make petrify more versatile than just making tags
branch and merge: combine checkout and branch logic
branch and merge: add branch-from-image command
Add a black-box test for branch-from-image
morphlib/builder2.py | 1 +
morphlib/builder2_tests.py | 3 +-
morphlib/plugins/branch_and_merge_plugin.py | 206 +++++++++++++++------
morphlib/repoaliasresolver.py | 75 ++++++--
morphlib/repoaliasresolver_tests.py | 48 +++++-
tests.as-root/branch-from-image-works.script | 57 ++++++
tests.as-root/branch-from-image-works.setup | 1 +
tests.as-root/branch-from-image-works.stdout | 1 +
tests.as-root/metadata-includes-repo-alias.script | 49 +++++
tests.as-root/metadata-includes-repo-alias.setup | 48 +++++
10 files changed, 413 insertions(+), 76 deletions(-)
create mode 100755 tests.as-root/branch-from-image-works.script
create mode 120000 tests.as-root/branch-from-image-works.setup
create mode 100644 tests.as-root/branch-from-image-works.stdout
create mode 100755 tests.as-root/metadata-includes-repo-alias.script
create mode 100755 tests.as-root/metadata-includes-repo-alias.setup
--
1.7.5.4
10 years, 3 months
Maui OS project
by Rob Taylor
Hi all,
Just dropping a note on Maui Os - http://www.maui-project.org
Looks like these guys have some similar aims wrt to updates and
application handling. Might be worth checking out and possibly
collaborating.
I will look deeper into this when I get a chance.
Rob
--
Rob Taylor, CTO, Codethink Ltd.
http://www.codethink.co.uk
Office: +44 161 236 5575
Cell: +44 7891 533856
10 years, 4 months
[PATCH] Morph: Soften dependency on pyyaml
by Lars Wirzenius
* repo: git://git.baserock.org/baserock/baserock/morph
* branch: liw/soft-pyyaml-dep
* commit: e06eafcbf0784271f51c0936852d1e97c0ccce8b
This makes Morph work even when the yaml library is not available.
The test suite passes, by virtue of parts of it being skipped, and the
modified Morph can build the base system. Obviously morphologies that
use YAML won't work, but we aren't using them yet, and shouldn't, until
there's been a Baserock release with the new Morph.
Lars Wirzenius (1):
Make yaml be an optional dependency
check | 9 +-
morphlib/__init__.py | 14 ++++
morphlib/morph2.py | 2 +-
morphlib/morph2_tests.py | 34 ++++----
morphlib/morphologyfactory.py | 3 +-
morphlib/yamlparse.py | 185 ++++++++++++++++++++++-------------------
morphlib/yamlparse_tests.py | 15 ++--
7 files changed, 151 insertions(+), 111 deletions(-)
--
1.7.10.4
10 years, 4 months
[PATCH] Disable distcc by default
by Sam Thursfield
Repository: git://git.baserock.org/baserock/baserock/morph
Branch: samthursfield/no-distcc
Sha1: 80212317e1375829db3e2bb467273518c573de7f
This will fix bootstrap, which is currently failing due to missing
distcc.
Sam Thursfield (1):
Disable distcc by default
morphlib/app.py | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
--
1.7.10.4
10 years, 4 months
[RFC] Morph branch-from-image
by Richard Maw
The basic behaviour of given an image, create a System Branch petrified
at the point where it was built is well specified, but I still have some
further questions about how it should work before I get too tied to an
implementation which may turn out to be wrong.
1. How should the baserock metadata be given.
a. Path to Disk image
branch-from-image mounts it to find the metadata.
b. Path to directory
branch-from-image globs for metadata in the directory
c. List of paths to metadata files
branch-from-image leaves finding metadata up to caller
2. How is this testable.
I was thinking build a system, move the system artifact somewhere,
delete the cache and check whether the system artifact produced has
the same cache key.
3. Does the caller specify the branch's root repository and name?
The branch the System was built from and its repository is stored
in the metadata, but if it doesn't exist any more, it will need a
fallback anyway.
Specifying gives it a nice symmetry with `morph branch` which
defaults to branching from master, but can be told a different
branch instead. With it given you're saying branch as this but from
this image instead of master.
4. Should it fail to create the branch if it already exists in the
workspace, or treat it like a petrify?
5. When should it give up trying to make the branch?
a. When the commit the System was built from does not exist?
b. When the branch the System was built from does not have a similar
commit?
This could be expensive to work out and probably not worth the
work.
c. When the branch does not exist at all?
At this point the morphologies can be regenerated from the
metadata, but it's going to be complicated.
10 years, 4 months
[PATCH 00/17] Add tool to generate manifestst from system artifacts
by Jonathan Maw
head: 5752564ba1eebc3638967b978aab7639ba4b1190
branch: jannispohlmann/artifact-inspection-plugin-rebase
repo: ssh://git@trove.baserock.org/baserock/baserock/morph
This series of patch provides a tool that reads a rootfs artifact to
find out the chunks that were used to create it, and generates a manifest
of: artifact name, version number (if correctly guessed), commit,
repository, ref, morphology.
Jannis Pohlmann (17):
Move MountableImage class into morphlib
Make all messages in MountableImage chatty
Add ExtractedTarball class and method to extract/mount an artifact
Add image analysis plugin
Add a 'morph content-manifest' command to the image analysis plugin
Also list the original refs in "morph content-manifest"
Rename image analysis plugin to image inspection plugin
Rename artifact inspection plugin to image inspection plugin
Squashme: Add artifact_inspection_plugin.py to without-test-modules
Get rid of tabs in the content manifest output
Squashme: Small spacing adjustments
Add version guessing for autotools projects
Add configure.in to the AutotoolsVersionGuesser
Add support for reading manifest meta data from /baserock
Add labels to the manifest output fields
Avoid unnecessary spaces in the manifest outpu
Add support for the old syntax of AM_INIT_AUTOMAKE
morphlib/__init__.py | 2 +
morphlib/bins.py | 27 ++
morphlib/extractedtarball.py | 61 +++++
morphlib/mountableimage.py | 81 ++++++
morphlib/plugins/artifact_inspection_plugin.py | 294 +++++++++++++++++++++
morphlib/plugins/trebuchet_plugin.py | 58 +---
.../run-in-artifact-propagates-exit-code.exit | 1 +
.../run-in-artifact-propagates-exit-code.script | 33 +++
.../run-in-artifact-propagates-exit-code.stderr | 3 +
...run-in-artifact-with-different-artifacts.script | 47 ++++
...run-in-artifact-with-different-artifacts.stderr | 1 +
...run-in-artifact-with-different-artifacts.stdout | 12 +
without-test-modules | 4 +
13 files changed, 568 insertions(+), 56 deletions(-)
create mode 100644 morphlib/extractedtarball.py
create mode 100644 morphlib/mountableimage.py
create mode 100644 morphlib/plugins/artifact_inspection_plugin.py
create mode 100644 tests.as-root/run-in-artifact-propagates-exit-code.exit
create mode 100755 tests.as-root/run-in-artifact-propagates-exit-code.script
create mode 100644 tests.as-root/run-in-artifact-propagates-exit-code.stderr
create mode 100755 tests.as-root/run-in-artifact-with-different-artifacts.script
create mode 100644 tests.as-root/run-in-artifact-with-different-artifacts.stderr
create mode 100644 tests.as-root/run-in-artifact-with-different-artifacts.stdout
--
1.7.11.7
10 years, 4 months
[PATCH 0/4] Allow Morphologies to be written in YAML
by Richard Maw
Repo: git://git.baserock.org/baserock/baserock/morph
Ref: baserock/richardmaw/yaml
Sha1: b83d5eca7f8036ff295f9f5f07a8112074bdf016
Richard Maw (4):
Move OrderedDict and json loading to morphlib.util
Parse as YAML if not valid JSON
Fix tests
Test that systems can build with YAML morphologies
morphlib/__init__.py | 3 +-
morphlib/morph2.py | 35 ++++---
morphlib/morph2_tests.py | 19 +++-
morphlib/morphologyfactory.py | 6 +-
morphlib/morphologyfactory_tests.py | 6 +-
morphlib/util.py | 10 ++
morphlib/yamlparse.py | 100 +++++++++++++++++
morphlib/yamlparse_tests.py | 69 ++++++++++++
.../{lib => builds-with-yaml-morphologies.script} | 22 +---
tests.as-root/builds-with-yaml-morphologies.setup | 117 ++++++++++++++++++++
tests.as-root/lib | 4 +-
11 files changed, 348 insertions(+), 43 deletions(-)
create mode 100644 morphlib/yamlparse.py
create mode 100644 morphlib/yamlparse_tests.py
copy tests.as-root/{lib => builds-with-yaml-morphologies.script} (55%)
mode change 100644 => 100755
create mode 100755 tests.as-root/builds-with-yaml-morphologies.setup
--
1.7.5.4
10 years, 4 months
Tests and build-essential
by Sam Thursfield
Hi
tl;dr: is it OK if some of Morph's black box tests only work when
run on a Baserock devel system, and require build-essential
artifacts to be present in the artifact cache?
Earlier we discussed the testing strategy for the next stage of the work
I'm doing, which is to make Morph always build in a staging chroot and
to provide the staging filler automatically, by compiling the
build-essential stratum with the host's tools. We agreed some simple
tests that can go in our temporary Jenkins instance.
After the meeting I realised I have another problem. Morph has a bunch
of tests that build simple chunks in various ways using the host's
tools. These work in the "normal" build mode, which uses the host's
tools instead of a staging area. Ideally, I want to get rid of "normal"
build mode except for when build-essential is being rebuilt, but this
leaves the build tests at sea without a paddle. I see the following options:
- change them to require use of a staging area, and therefore use tools
from build-essential instead of the host's tools. This will mean that
these tests can only be run on a working Baserock devel system which can
provide a built build-essential stratum.
- change them to use the host-tools mode that we'll use for the first
stage of building build-essential itself. I worry that this will be
quite ugly, but it would probably work well enough.
- keep 'normal' mode in Morph just for the tests. This means that we are
testing a code path that is only used for tests, which seems quite silly.
Does anyone have opinions on this? Officially we only support Morph on
Baserock and on Debian Squeeze already and the Squeeze support will no
longer be needed once we have the new bootstrap mechanism in place, so
this change wouldn't be as drastic as it might sound.
Thanks
Sam
10 years, 4 months
ANNOUNCEMENT: build-essential part one is now merged
by Sam Thursfield
Dear Baserock users,
The changes I described in my original patch mails[1][2] have now been
merged to master.
The changes are quite drastic and currently cause some regressions, so
I'll quickly summarise them here:
- 'foundation' and 'devel' no longer exist. Baserock now consists of
the following strata:
- build-essential: the minimum set of tools that can rebuild
itself (from tarballs)
- core: the remaining set of components necessary to produce
a Baserock system which can rebuild itself from source
- tools: auxiliary development tools and libraries
- bsp: device-specific chunks
- the following chunks are now built from tarballs instead of from git:
autoconf, binutils, busybox, ccache, eglibc, flex, gawk, gcc,
gdbm, gettext, libtool, linux, m4, make, pkg-config
- the -base system is much larger (around 700MB) and includes lots of
unnecessary stuff. This is a known regression which will be fixed
when proper stratum splitting is implemented in Morph.
Users who are building real-world systems should stick with the stable
water-bomb release for the time being.
Feel free to respond to this mail with any further questions, but please
read the initial patch mails first to see if your question was answered
there.
Sam
[1]
http://vlists.pepperfish.net/pipermail/baserock-dev-baserock.org/2013-Jan...
[2]
http://vlists.pepperfish.net/pipermail/baserock-dev-baserock.org/2013-Jan...
10 years, 4 months
[PATCH 0/3] Make morph track its git version
by Richard Maw
Repo: git://git.baserock.org/baserock/baserock/morph
Ref: baserock/richardmaw/morph-version
Sha1: 132951b02d1e4e628cfbafab2d35bcda9c1fc24e
This makes morph able to know its own version from git.
`morph --version` now outputs the output of `git describe` at the time
it was built, or of the checkout it is run from.
The morphology metadata also includes this information, plus the ref,
commit and tree it was built from.
Richard Maw (3):
Make morph get its version from git.
Include morph-version in artifact metadata
Add regression test for metadata including version
morphlib/__init__.py | 5 +-
morphlib/builder2.py | 7 +++
morphlib/gitversion.py | 57 ++++++++++++++++++++
setup.py | 53 ++++++++++++++++--
.../metadata-includes-morph-version.script | 52 ++++++++++++++++++
.../metadata-includes-morph-version.setup | 48 ++++++++++++++++
without-test-modules | 1 +
7 files changed, 216 insertions(+), 7 deletions(-)
create mode 100644 morphlib/gitversion.py
create mode 100755 tests.as-root/metadata-includes-morph-version.script
create mode 100755 tests.as-root/metadata-includes-morph-version.setup
--
1.7.5.4
10 years, 4 months
