Baserock 12 is released!
by Pedro Alvarez
Baserock 12 is released
=======================
The [Baserock team] and [Codethink Limited] are proud to announce
version 12 of the Baserock system.
Baserock is a toolset and development methodology for developing
embedded and appliance Linux systems.
What's new in this version?
===========================
Cmdtest
-------
We updated the version of the cmdtest component used in Baserock. This
brings in a newer version of the yarn testing tool which has allowed us
to improve the Morph test suites to be cleaner and thus better.
SSH configuration extension
---------------------------
The ssh configuration extension's functionality has been replaced by
(non-identical, but equivalent) functionality in the install-files
configuration extension. Following the principle of simpler-is-better
we have therefore removed the ssh configuration extension and we
encourage users to migrate to install-files.
As an example of how to use this extension, add the environment variable
INSTALL_FILES to the deploy stanza for a system in the cluster
morphology:
name: foo
kind: cluster
systems:
- morph: base-system-x86_64-generic
repo: baserock:baserock/morphs
ref: master
deploy:
my-raw-disk-image:
type: rawdisk
location: /src/tmp/testdev.img
DISK_SIZE: 4G
INSTALL_FILES: foo-dir/manifest
Then create the 'foo-dir' directory:
foo-dir/
root/
.ssh/
id_rsa
id_rsa.pub
Where id_rsa and id_rsa.pub are the ssh key pair the root user on the
deployed system should have. Then create foo-dir/manifest:
0040755 0 0 /root
0040700 0 0 /root/.ssh
0100600 0 0 /root/.ssh/id_rsa
0100600 0 0 /root/.ssh/id_rsa.pub
After this, install-files will put the files in the right places on
the deployed system, with the right permissions.
Refactoring morph subcommands
-----------------------------
Work began on refactoring the 'build' command, but the new code will
not be available in Baserock until the next cycle.
Minor changes to morph
----------------------
* A number of bugs have been fixed during the Baserock 12 cycle,
including:
* prevent 'edit' subcommand being used without a system as argument.
* prevent building uncommmitted systems.
* deploy a non-cluster morphology now displays an error message.
* prevent cross-building.
* morph can now build when checking out a tag.
* We have improved the morph scenario test suite by:
* adding new scenarios, including regression scenarios for fixed
bugs, and replacing some of the old cmdtest based suite with
scenarios.
* reworking some of the statement language to be more consistent,
* making the voice of the scenarios slightly more active.
Changes to morphologies
=======================
Reducing redundancy in build dependencies
-----------------------------------------
Since at this time, build-depending on a stratum implicitly
build-depends on the build-dependencies of that stratum, we have
simplified the build-depends sections of the standard stratum
morphologies. This should make it easier to understand the relationship
between strata involved in building systems.
Other changes
-------------
* pyfilesystem, kexec-tools have been added to the 'tools' stratum.
* 'six' has been moved from 'glanceclient' to the 'tools' stratum.
* boto has been added to 'tools' to finish the integration of
cloud-init in Baserock. There is a short guide about how to deploy to
OpenStack on the [developing with Baserock] wiki page.
New systems and strata
----------------------
We have added a ceph services system which provides the daemons
necessary for the Ceph object, block and file storage services.
These daemons are contained within a ceph-service stratum which can be
included into your systems if you want them to be able to serve Ceph
filesystems.
This new system is an example and is not directly supported at this
time. However please do try it out if Ceph is of interest to you, and
let us know how you get on.
How do I get started?
=====================
You can find a [quick start] guide on the [Baserock wiki] and also
a short guide on [developing with Baserock] which follows on from
the quick start guide and shows you how to get to the point of proving
you can build Baserock within Baserock.
>From that point on, your imagination is the limit. You can follow our
development in the [Git repositories] we publish.
How do I get in contact?
========================
The Baserock project has an [IRC channel] and [mailing list] for
developers to gather and discuss anything associated with
Baserock. It is strongly recommended that you use the IRC and
lists to contact the team for anything associated with the public
development of Baserock. We also have a mailing list for
[announcements] which will be notified of any new releases or big
developments in Baserock.
If you manage to find a bug in Baserock, we'd like to hear
from you. You can find our [bug reporting guidelines] on the
Baserock wiki and we will do our best to help.
We hope you enjoy experimenting with Baserock and look forward to
hearing about any cool things you do with our work.
[Codethink limited]: http://www.codethink.co.uk/
[Baserock wiki]: http://wiki.baserock.org/
[Baserock team]: http://wiki.baserock.org/team/
[Git repositories]: http://git.baserock.org/cgi-bin/cgit.cgi
[IRC Channel]: http://wiki.baserock.org/irc-channel/
[mailing list]: http://wiki.baserock.org/mailinglist/
[announcements]: http://wiki.baserock.org/mailinglist/
[download page]: http://wiki.baserock.org/download/
[quick start]: http://wiki.baserock.org/quick-start/
[developing with Baserock]: http://wiki.baserock.org/devel-with/
[bug reporting guidelines]: http://wiki.baserock.org/bug-reporting/
[Baserock GENIVI Baseline]:
http://projects.genivi.org/GENIVI_Baselines/genivi-baserock/
9 years, 8 months
[PATCH 00/15] Rewrite build and deploy commands
by Richard Maw
Repo: git://git.baserock.org/baserock/baserock/morph.git
Ref: baserock/richardmaw/S9475/refactor-build-cmd-v2
SHA1: 49156c594439bacc00c4658470b6848cfda21732
Land: origin/master
This cover letter contains a summary of the changes between the patches.
Any specific patches will have an appropriate diff attached, though many
decided to completely change the approach.
Richard Maw (15):
yarns: Correctly allow run_morph to output stderr on failure
* This is new, because I was irritated at having to use --snapshot to
see the failure message, and I failed to fix it properly the
first time
morphloader: Don't use ValueError exception
* Requested as an addendum to the optional strata patch
morphloader: Set default values for cluster morphs
* A couple of implementation details changed
morphloader: Require systems have at least one stratum
* Replaces "morphloader: Allow strata to be optional"
validation: Require there be non-bootstrap chunks in systems
* Making the tests require strata led to discovering that the yarns
had unbuildable systems. This raises a validation error for how they
were broken and fixes the systems in the yarns.
morphloader: use getattr for validate, set defaults
* Came about as part of the review for "morphloader: Set default values
for cluster morphs"
yarns: un-parameterise architecture in system morphologies
* This is "yarns: implement system morphologies without arch" renamed
for clarity
sysbranchdir: Move load_all_morphologies helper here
* This pretty much replaces "util: Move load_all_sysbranch_morphologies
helper here"
branchmanager: Allow deferred and optional cleanup on success.
* Changing the temporary_build_branch to a class required the branch
managers to be made more useful when being used out of a with
statement
morphlib: Add BuildBranch abstraction
* This replaces "util: Add temporary_build_branch context manager",
some fragments of code may match and it follows the same flow,
but a diff isn't very useful here
plugins: Add new build command
* Uses the new BuildBranch instead
plugins: Use new build command as default
* mostly unchanged
yarns: Change expected result of building uncommitted morphologies
* Clarified description of "yarns: Build with uncommitted now passes."
plugins: Remove old build command
plugins: Convert deploy to new classes.
* Uses the new BuildBranch instead
morphlib/__init__.py | 1 +
morphlib/branchmanager.py | 26 ++-
morphlib/branchmanager_tests.py | 99 +++++++++
morphlib/buildbranch.py | 256 +++++++++++++++++++++
morphlib/buildcommand.py | 46 +++-
morphlib/morphloader.py | 91 ++++++--
morphlib/morphloader_tests.py | 157 ++++++++++---
morphlib/plugins/branch_and_merge_new_plugin.py | 11 +-
morphlib/plugins/branch_and_merge_plugin.py | 282 ------------------------
morphlib/plugins/build_plugin.py | 92 ++++++++
morphlib/plugins/deploy_plugin.py | 155 ++++++-------
morphlib/sysbranchdir.py | 14 ++
morphlib/util.py | 1 -
without-test-modules | 2 +
yarns/architecture.yarn | 2 +-
yarns/building.yarn | 6 +-
yarns/implementations.yarn | 49 ++--
yarns/morph.shell-lib | 17 +-
yarns/regression.yarn | 50 ++++-
19 files changed, 876 insertions(+), 481 deletions(-)
create mode 100644 morphlib/buildbranch.py
--
1.8.5.rc2
9 years, 10 months
Interesting problem with commit_tree tests on my Fedora machine
by Rob Taylor
Hi all,
I get an interesting issue running check --full on my fedora 18 system:
FAILURE: test_commit_tree (gitdir_tests.GitDirectoryContentsTests)
Traceback (most recent call last):
File "morphlib/gitdir_tests.py", line 203, in test_commit_tree
self.assertEqual(expected, gd.get_commit_contents(commit).split('\n'))
AssertionError: Lists differ: ['tree 01d830ae9bae1a9970a9f14... !=
['tree 01d830ae9bae1a9970a9f14...
First differing element 2:
author Author Name <author@email> 683074800 +0000
author Author Name <author@email> 683071200 +0100
['tree 01d830ae9bae1a9970a9f1491c914ad177f8afce',
'parent a035486ceee1c15acd7212420b793b0b2c9cef32',
- 'author Author Name <author@email> 683074800 +0000',
? ^^ ^
+ 'author Author Name <author@email> 683071200 +0100',
? ^^ ^
- 'committer Committer Name <committer@email> 683074800 +0000',
? ^^ ^
+ 'committer Committer Name <committer@email> 683071200 +0100',
? ^^ ^
'',
'MESSAGE',
'']
I'm guessing what's happening here is that my locale is en_GB, and the
test runner is setting the time for the tests to a time in summer?
Rob
9 years, 10 months
[PATCH] Add support for ppc64 in build-essential
by Pedro Alvarez
Repo: ssh://git@git.baserock.org/delta/linux.git
Branch: baserock/pedroalvarez/power-port
Sha1: 3a5f735f2bea240ca8f83a7f7a4febef5d99f5a6
Card: 9715
Pedro Alvarez (1):
Add support for ppc64 in build-essential
morph-arch | 2 ++
1 file changed, 2 insertions(+)
--
1.7.10.4
9 years, 10 months
[PATCH] show which ref is expected for each stratum in a build
by Paul Sherwood
From 6653e3585df23310632e78f5a3ba767f42a4c0e3 Mon Sep 17 00:00:00 2001
From: Paul Sherwood <paul.sherwood(a)codethink.co.uk>
Date: Wed, 20 Nov 2013 23:56:09 +0000
Subject: [PATCH] show which ref is expected for each stratum when
preparing
build
This has helped me tracking down 'conflicting stratum' issues, and
seems a
useful thing to report during each build in any case.
---
morphlib/buildcommand.py | 3 +++
1 file changed, 3 insertions(+)
diff --git a/morphlib/buildcommand.py b/morphlib/buildcommand.py
index 7b7612a..8ad893a 100644
--- a/morphlib/buildcommand.py
+++ b/morphlib/buildcommand.py
@@ -158,6 +158,9 @@ class BuildCommand(object):
# and Ref specified.
if src.morphology['kind'] == 'stratum':
name = src.morphology['name']
+ ref = src.sha1[:7]
+ self.app.status(msg='Stratum [%(name)s] version is
%(ref)s',
+ name=name, ref=ref)
if name in stratum_names:
raise morphlib.Error(
"Conflicting versions of stratum '%s' appear
in the "
--
1.8.4
9 years, 10 months
[RFC 00/10] Rewrite build and deploy commands
by Richard Maw
Repo: git://git.baserock.org/baserock/baserock/morph.git
Ref: baserock/richardmaw/S9475/refactor-build-cmd
SHA1: 738751c667031a2a5c2fb3c17bc05e7b0f2c1519
This is an RFC patch series, since while it passes all the
tests, more should be converted to yarns and the design of
morphlib.temporary_build_branch is questionable.
Commit messages are marked with TODOs to discuss questionable design
decisions.
Richard Maw (10):
morphloader: Set default values for cluster morphs
morphloader: Allow strata to be optional in system
yarns: implement system morphologies without arch
WIP: util: Move load_all_sysbranch_morphologies helper here
WIP: util: Add temporary_build_branch context manager
WIP: Add build command
WIP: Use new build command as default
yarns: Build with uncommitted now passes.
WIP: Remove old build command
WIP: Convert deploy to new classes.
morphlib/morphloader.py | 13 +-
morphlib/plugins/branch_and_merge_new_plugin.py | 12 +-
morphlib/plugins/branch_and_merge_plugin.py | 282 ------------------------
morphlib/plugins/build_plugin.py | 71 ++++++
morphlib/plugins/deploy_plugin.py | 134 ++++-------
morphlib/util.py | 184 +++++++++++++++-
yarns/architecture.yarn | 2 +-
yarns/building.yarn | 6 +-
yarns/implementations.yarn | 22 +-
9 files changed, 333 insertions(+), 393 deletions(-)
--
1.8.5.rc2
9 years, 10 months
[PATCH 00/21] Lay foundation for refactoring morph build
by Richard Maw
Repo: git://git.baserock.org/baserock/baserock/morph.git
Ref: baserock/richardmaw/S9475/build-refactor-foundations
SHA1: 0cdf25983d36d991e5ec29df449dac96c62e1d8c
Land: origin/master
A summary of changes from the previous series is included below.
Richard Maw (21):
tests: Run style check before slower tests
* Fixed a whitespace issue
yarns: Also output error messages to terminal
Remove unused invent_new_branch method
tempdir: Allow use as a context manager
* Patch removed in favour of using pyfilesystem
GitDir: remove duplicate definition of update_remotes
GitDir: Allow config values ending in whitespace
GitDir: Provide more specific object access than cat-file
* cat-file is now 3 methods
1. get a file's contents, given a ref and path
2. get a file's contents given a SHA1
3. get a commit's contents given a SHA1
GitDir: Add resolve_ref_to_{commit,tree} methods
* rev_parse is now resolve_ref_to_{commit,tree}
MorphologySet: Export traverse_specs method
util: Add helper for splitting up huge iterables
* Documented a little better
GitDir: add store_blob method
* hash_blob -> store_blob
GitDir: Add GitIndex class
* Split into a separate module
gitindex: Add GitIndex.set_to_tree(treeish) method
* read_tree -> set_to_tree
gitindex: Add GitIndex.add_files_from_index_info()
* add_info -> add_files_from_index_info
gitindex: Add GitIndex.add_files_from_working_tree()
* add_files -> add_files_from_working_tree
gitindex: Add GitIndex.write_tree()
GitDir: Add GitDirectory.commit_tree() method
GitDir: Add methods for ref management
* Less silly exceptions
GitDir: Split out Remote object
GitDir: Add support for push urls in Remote
GitDir: Add remote.push(RefSpec...)
* Less silly exceptions
* New exception for invalid refspec
morphlib: Add branch context managers
* Hopefully documented better
check | 81 ++--
morphlib/__init__.py | 2 +
morphlib/branchmanager.py | 158 ++++++++
morphlib/branchmanager_tests.py | 331 ++++++++++++++++
morphlib/gitdir.py | 483 +++++++++++++++++++++---
morphlib/gitdir_tests.py | 315 +++++++++++++++-
morphlib/gitindex.py | 159 ++++++++
morphlib/gitindex_tests.py | 92 +++++
morphlib/morphset.py | 12 +-
morphlib/plugins/branch_and_merge_new_plugin.py | 18 +-
morphlib/sysbranchdir.py | 3 +-
morphlib/util.py | 10 +
morphlib/util_tests.py | 11 +
scripts/check-silliness | 63 ++++
yarns/morph.shell-lib | 3 +
15 files changed, 1598 insertions(+), 143 deletions(-)
create mode 100644 morphlib/branchmanager.py
create mode 100644 morphlib/branchmanager_tests.py
create mode 100644 morphlib/gitindex.py
create mode 100644 morphlib/gitindex_tests.py
create mode 100755 scripts/check-silliness
--
1.8.5.rc2
9 years, 10 months
[PATCH 0/9] Miscellaneous cleanups
by Richard Maw
Repo: git://git.baserock.org/baserock/baserock/morph.git
Ref: baserock/richardmaw/S9475/misc-cleanups
SHA1: 2ccd87cce576093149105379007c1c4929ca2f4b
Land: origin/master
This are the accrued minor fixes from the larger work to come.
Richard Maw (9):
tests: Run style check before slower tests
yarns: Also output error messages to terminal
Remove unused invent_new_branch method
tempdir: Allow use as a context manager
GitDir: remove duplicate definition of update_remotes
GitDir: Allow config values ending in whitespace
GitDir: Change cat_file api to be more generic
GitDir: Export rev_parse as public gitdir api
MorphologySet: Export traverse_specs method
check | 81 +++++++------------------
morphlib/gitdir.py | 22 +++----
morphlib/gitdir_tests.py | 7 +++
morphlib/morphset.py | 12 ++--
morphlib/plugins/branch_and_merge_new_plugin.py | 16 +----
morphlib/tempdir.py | 14 ++++-
morphlib/tempdir_tests.py | 9 ++-
scripts/check-silliness | 63 +++++++++++++++++++
yarns/morph.shell-lib | 3 +
9 files changed, 132 insertions(+), 95 deletions(-)
create mode 100755 scripts/check-silliness
--
1.8.5.rc2
9 years, 10 months
[PATCH 0/8] Add API for generating commits
by Richard Maw
Repo: git://git.baserock.org/baserock/baserock/morph.git
Ref: baserock/richardmaw/S9475/commit-api
SHA1: b10b57740a808b144350dfb73085d9a08286ead2
Base: baserock/richardmaw/S9475/misc-cleanups
Land: master
This is based off the misc-cleanups branch, so don't merge this one
before reviewing that one.
This adds an Index class for handling working tree operations, to avoid
having lots of methods in gitdir that take a path to an alternative index.
Richard Maw (8):
util: Add helper for splitting up huge iterables
GitDir: add hash_blob method
GitDir: Add Index inner class
GitDir: Add Index.read_tree(treeish) method
GitDir: Add Index.add_info method
GitDir: Add Index.add_paths() method
GitDir: Add Index.write_tree() method
GitDir: Add GitDirectory.commit_tree() method
morphlib/gitdir.py | 175 ++++++++++++++++++++----
morphlib/gitdir_tests.py | 104 +++++++++++++-
morphlib/plugins/branch_and_merge_new_plugin.py | 2 +-
morphlib/util.py | 10 ++
morphlib/util_tests.py | 11 ++
5 files changed, 273 insertions(+), 29 deletions(-)
--
1.8.5.rc2
9 years, 10 months
[PATCH 0/3] Socket creation program and scenarios
by Ben Brown
Repo: baserock:baserock/tbdiff
Ref: benbrown/S9482/convert-socket-tests
Sha1: 56a67f1b4e71d0853b56d295c6dc1ca1ba54150e
Card: S9482
Ben Brown (3):
Added C program which creates sockets
Socket creation program added to 'make'
Socket scenarios added
Makefile.am | 3 ++-
yarns/create-deploy.yarn | 47 +++++++++++++++++++++++++++++++++++++++++++---
yarns/sockbind.c | 37 ++++++++++++++++++++++++++++++++++++
3 files changed, 83 insertions(+), 4 deletions(-)
create mode 100644 yarns/sockbind.c
--
1.7.10.4
9 years, 10 months
