[PATCH] Mount staging mounts inside staging area when bootstrap building
by Richard Maw
Repo: git://git.baserock.org/baserock/baserock/morph.git
Ref: baserock/richardmaw/bugfix/stagingarea-mounts-inside-destdir-v2
SHA1: 11555718e14ab147bd5da8b093d49279ad9d3ee8
Land: origin/master
This was missed in the move to unify the logic for containerised staging
areas.
It didn't bite anyone until now because it usually harmlessly mounts a
new /dev/shm on top of the old one in an isolated mount namespace.
This caused problems recently with the error "ERROR: /dev/shm: File
exists", which we assumed to be from attempting to mkdir /dev/shm.
This is surprising, since it only attempts to mkdir if the directory
doesn't exist, but it can happen if /dev/shm is a symlink and the code
in question uses stat rather than lstat to check whether the path exists.
The actual bug is that it's attempting to mount in /dev/shm rather than
"$stagingarea/dev/shm" though, so this patch fixes that, rather than
just using `os.path.lexists` rather than `os.path.exists`.
Richard Maw (1):
stagingarea: Mount things inside dirname
morphlib/stagingarea.py | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
--
1.7.10.4
8 years, 5 months
[PATCH] Add python3 to core
by Paul Sherwood
repo: baserock:baserock/definitions
ref: baserock/ps/add-python3
There was some discussion on #baserock about attempting to make python3
the default python in baserock systems, but I think that's unlikely to
work out-of-the-box.
So this patch adds python3 (3.4.2), to co-exist with python2 (2.7.8)
and leaves /usr/bin/python as the 2.7.8 version. I've built, deployed
and verified that both pythons are functional....
---
diff --git a/strata/core.morph b/strata/core.morph
index a1fc9a1..50d9b67 100644
--- a/strata/core.morph
+++ b/strata/core.morph
@@ -114,8 +114,8 @@ chunks:
ref: 518937ab89be812ccd45e9b8c1ce4ad721d35ef6
unpetrify-ref: baserock/genivi/baseline
build-depends: []
-- name: cpython
- morph: strata/core/cpython.morph
+- name: python2
+ morph: strata/core/python2.morph
repo: upstream:cpython
ref: 0b92c75574adef1e1ccf1b635a3724a86b567cd2
unpetrify-ref: v2.7.8
@@ -125,6 +125,17 @@ chunks:
- sqlite3
- ncurses
- readline
+- name: python3
+ morph: strata/core/python3.morph
+ repo: upstream:cpython
+ ref: d1414cefaddc7b56caef75eea87226b1fce5ca7c
+ unpetrify-ref: v3.4.2
+ build-depends:
+ - openssl-new
+ - bzip2
+ - sqlite3
+ - ncurses
+ - readline
- name: libtool
morph: strata/core/libtool.morph
repo: upstream:libtool
@@ -157,7 +168,7 @@ chunks:
build-depends:
- autoconf
- automake
- - cpython
+ - python2
- libtool
- xz
- name: ca-certificates
@@ -167,7 +178,7 @@ chunks:
unpetrify-ref: baserock/debian/20140325
build-depends:
- automake
- - cpython
+ - python2
- name: curl
morph: strata/core/curl.morph
repo: upstream:curl
@@ -203,7 +214,7 @@ chunks:
unpetrify-ref: v2.1.3
build-depends:
- autoconf
- - cpython
+ - python2
- curl
- gettext
- libexpat
@@ -276,7 +287,7 @@ chunks:
ref: 0aa6a4de5931d02876428388678802db2371fd37
unpetrify-ref: baserock/master
build-depends:
- - cpython
+ - python2
- name: pyyaml
morph: strata/core/pyyaml.morph
repo: upstream:pyyaml
diff --git a/strata/core/cpython.morph b/strata/core/python2.morph
index 6c9e4b3..997fe6f 100644
--- a/strata/core/cpython.morph
+++ b/strata/core/python2.morph
@@ -1,4 +1,4 @@
-name: cpython
+name: python2
kind: chunk
build-system: autotools
configure-commands:
diff --git a/strata/core/cpython.morph b/strata/core/python3.morph
index 6c9e4b3..f02452d 100644
--- a/strata/core/cpython.morph
+++ b/strata/core/python3.morph
@@ -1,7 +1,7 @@
-name: cpython
+name: python3
kind: chunk
build-system: autotools
configure-commands:
- ./configure --prefix="$PREFIX" --enable-shared
post-install-commands:
-- test -x "$DESTDIR"/"$PREFIX"/bin/python2 || ln -s python2.7
"$DESTDIR"/"$PREFIX"/bin/python2
+- test -x "$DESTDIR"/"$PREFIX"/bin/python3 || ln -s python3.4
"$DESTDIR"/"$PREFIX"/bin/python3
--
cgit v0.9.0.3-67-gacbf
8 years, 5 months
[PATCH] lorries: Add npm packages needed by the npm importer
by Sam Thursfield
Repository: ssh://git@git.baserock.org/baserock/local-config/lorries
Ref: sam/npm-importer-lorries
Sha1: 631660db7b4db2dfa0e943cebdacd0ea6f445fc4
Sam Thursfield (1):
Add npm packages needed by the npm importer
open-source-lorries/npm-packages.lorry | 10 ++++++++++
1 file changed, 10 insertions(+)
create mode 100644 open-source-lorries/npm-packages.lorry
--
1.9.3
8 years, 5 months
Patch trackers and git servers
by Emmet Hikory
I've been advocating the use of gerrit as a patch tracker for some
time, and am gladdened to see more people supporting this idea, and the
work of the operations team to put test gerrit instances up in the
Baserock infrastructure. However, gerrit is a git server, and I am a
little worried that we may lose some of the advantages we have enjoyed
from the use of gitano for Baserock curated archives.
My understanding is that gitano stores all history and all ACLs in a
gitano-managed git repository, so that short of generating collisions in
the underlying datastore, one can be confident that one has a traceable
record of everything that has happened to the repositories managed by
gitano. I also believe that gitano has no means of tracking candidates
that ought be merged to arbitrary targets, or easily allowing
individuals to self-generate credentials allowing them to upload
candidates against a given branch.
From what I can tell, gerrit has the opposite feature set:
specifically a rich means to manage registration, candidate submission,
and review, all with a documented API that has inspired a diverse
ecosystem of support tooling to interact with these services.
Conversely, configuration, state, and access controls are stored
externally, so that one would need to maintain a complete transaction
log for all changes to the hosting server(s) to ensure that the state of
the repositories were accurate.
For archive curation, I believe that we should continue to recommend
the use of gitano as a central storage facility, and further that we
should continue to use gitano to manage the archive used to generate the
reference systems. While there is a knowledge horizon imposed by
mirroring, in that we cannot validate the accuracy of commits not
applied directly to the archive mirror, the use of lorry provides us
with some degree of comfort that we have accurately captured the commits
made upstream, and will not lose commits removed upstream, and that we
have some traceability to this process. If we change to a git server
that does not provide the same level of guarantee as gitano, there is a
much greater potential for rogue commits to be present, making it less
safe for organisations to mirror the Baserock archive, and that the
Baserock project cannot have as much confidence in systems deployed by
the operations team.
For patch tracking, we clearly want to use gerrit, as gitano has no
such features. In the case of repositories for which Baserock is
upstream (lorry, morph, definitions, etc.), a sensible model might be to
have the "upstream" repository be on gerrit, and lorry this into the
curated archive, as with any other upstream component. In the event
that the "upstream" archive is compromised or corrupted, the project can
confidently detect the issue and safely recover from the curated archive
mirror.
In the case of repositories for which Baserock is not upstream,
things are a little more complicated. Adding a new repository to gerrit
for a single review for a patch expecting to land upstream, and then
switching the lorry back later seems horridly complicated, as is using
any other means of causing an upstream mirror to exist. However, and
especially for repositories for which upstream does not use git, it may
be difficult to cause a given patch to appear somewhere suitable for
lorrying. In the presence of a patch tracker, the habit of review on a
mailing list may fade, and so there is no obvious way in which members
of the Baserock community might propose patches to non-Baserock
components in advance of them landing upstream.
Some of the automation used by the Baserock project adds more
potential complication to the set of available choices. To date,
cache.baserock.org has been populated for any commits that reach master
of the primary repository for definitions. In the event that we
separate the archive curation and patch tracking facilities as described
above, how long does it take for a given change to mirror, and from
which source should we be populating the cache? My personal preference
is to populate the cache pre-merge, so as to provide a more robust cache
to any humans validating results, but this may require unacceptable
storage capacities.
For pre-merge testing automation, the situation is a bit simpler.
In the case of a change in definitions, the automation can check out the
candidate, and then safely run against the curated archive (assuming a
sufficiently small mirroring time that the referenced commits have landed).
In the case of a change to other Baserock components (e.g. morph), this
can be modeled as a change to definitions, where the repo and ref for the
change component references the ref of the candidate in the gerrit repo,
and everything else references the curated archive, avoiding the need
to keep track of which repository to test externally.
For integration automation, if the automation is tracking potential
changes in the curated archive, this means that they are available in
the curated archive, so candidate definitions branches using these
references may be safely submitted to pre-merge testing automation
as described above.
I would be interested in the opinions of others as to how to model
this well, gaining the advantages of both gerrit and gitano to allow
both effective patch tracking and reliable and traceable curated archives.
I am especially interested in suggestions that allow Baserock users to
configure patch tracking for downstream repositories, as one of the
advantages of a curated archive is the ability to land branches that
differ from upstream in one or more ways. I would hope that such solutions
might also be usable by the Baserock project to allow the operations team
to land patches in various components to support the needs of the
Baserock infrastructure.
--
Emmet HIKORY
8 years, 5 months
Re: [PATCH 02/13] Switch to more recent release of the boost libraries
by Francisco Redondo Marchena
Hello Patrick,
forget about my previous email, this is the correct one, also I forgot
to forward to the mailing list the previous one.
See my comments below.
On 2014-12-18 14:05, Francisco Redondo Marchena wrote:
> On 2014-12-18 11:05, Patrick Darley wrote:
>> On 2014-12-18 09:00, Francisco Redondo Marchena wrote:
>>> On 2014-12-17 18:34, Patrick Darley wrote:
>>>> ---
>>>> strata/ceph-service.morph | 2 +-
>>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/strata/ceph-service.morph b/strata/ceph-service.morph
>>>> index 25d6712..473831f 100644
>>>> --- a/strata/ceph-service.morph
>>>> +++ b/strata/ceph-service.morph
>>>> @@ -48,7 +48,7 @@ chunks:
>>>> - name: boost
>>>> morph: strata/ceph-service/boost.morph
>>>> repo: upstream:boost-tarball
>>>> - ref: 026fc3d84a27e84a61d0f4c26f011a68271a0d6e
>>>> + ref: boost_1_56_0
>>>> unpetrify-ref: baserock/markdoffman/morph
>>>> build-depends: []
>>>> - name: ceph
>>>
>>> Altought the "ref" field can be used in the way that you have done,
>>> we prefer that it is pointing to a sha1 (commit), adding the
>>> branch/tag
>>> name for that commit in "unpetrify-ref". On the other hand
>>> unpetrify-ref
>>> is showing a wrong information,it is not failing because
>>> unpetrify-ref
>>> is only used as a information field.
>>
>> Ahh, so that's what unpetrify-ref is for!
>>
>>> The way that we usually write the chunks would be:
>>>
>>> - name: boost
>>> morph: strata/ceph-service/boost.morph
>>> repo: upstream:boost-tarball
>>> ref: ed232fdd34968697a68783b3195b1da4226915b5
>>> unpetrify-ref: boost_1_56_0
>>> build-depends: []
>>>
>>> Where "ref" is the head commit in the branch boost_1_56_0.
>>> (Note that you could use any commit in the branch).
>>
>> Is boost_1_56_0 the tag relating to a specific commit on the
>> master branch? I am still quite new to git so I am trying to
>> understand.
>>
boost_1_56_0 is the tag created when the tarball was lorried
and the ref 1c3648bf5b7d17fcd4fe9bc95802b16fd9eee304 is the head
commit
in that tag which in this case is not the same as master because it is
not the
last tarball lorried.
You can have a look at the info here:
http://git.baserock.org/cgi-bin/cgit.cgi/delta/boost-tarball.git/commit/?...
and
http://git.baserock.org/cgi-bin/cgit.cgi/delta/boost-tarball.git/
Fran
--
Francisco Redondo Marchena
Software Developer
Codethink Ltd
302 Ducie House
Ducie Street
Manchester
M1 2JW
UK
Codethink delivers cutting edge open source design, development and
integration services - from embedded stack software to advanced ux
http://codethink.co.uk
Office: +44 161 236 5575
8 years, 5 months
[PATCH 0/3] Patches For review of Zookeeper Work
by Mike Smith
Repo: Baserock:baserock/definitions
ref: baserock/mikesmith/zookeeper-for-review
SHA1: e2b24f9b20ca11d86c0680be91a6532162b34baa
Landing: master
ZooKeeper is required to be usable in baserock by a client. This patch
series imports zookeeper and its requirements into baserock and also
brings in a small program developed by myself for both server and client
sides of the zookeeper environment, that can be used to demonstrate this
functionality to the client.
The Application that I have written for the demonstration purposes is
currently hosted on github. patch series is currently out for review
that would lorry this program into the main git.baserock.org location,
this is under the Kanban ticket S11952.
If the patch referred to by S11952 is accepted then I would be happy to
fix the reference for the Applications location within this patch series
at merge, or submit a fresh series fixing this issue.
The Java chunk that is used in this series is not written by myself, but
by Francisco Marchena. This chunk is known to only work on X86_64
architecture. And that is a notable limitation to this version of the
zookeeper build. however if Java can be updated at a later point to work
on multiple architectures then the ZooKeeper and ANT chunks should then
also work in all architectural environments that Java works within, due
to the nature of Java.
Mike Smith (3):
Add chunks for Java, ANT and ZooKeeper
Add ZooKeeper demonstration application and strata.
Add ZooKeeper systems for x86_64 and example cluster to deploy them.
clusters/zookeeper.morph | 21 +++++++++++++
strata/zookeeper-client.morph | 36 ++++++++++++++++++++++
strata/zookeeper-server.morph | 36 ++++++++++++++++++++++
strata/zookeeper/java-ant.morph | 9 ++++++
strata/zookeeper/java-binary.morph | 11 +++++++
strata/zookeeper/zookeeper-client.morph | 17 ++++++++++
strata/zookeeper/zookeeper-server.morph | 30 ++++++++++++++++++
strata/zookeeper/zookeeper.morph | 14 +++++++++
systems/devel-system-x86_64-zookeeper-client.morph | 31 +++++++++++++++++++
systems/devel-system-x86_64-zookeeper-server.morph | 31 +++++++++++++++++++
10 files changed, 236 insertions(+)
create mode 100644 clusters/zookeeper.morph
create mode 100644 strata/zookeeper-client.morph
create mode 100644 strata/zookeeper-server.morph
create mode 100644 strata/zookeeper/java-ant.morph
create mode 100644 strata/zookeeper/java-binary.morph
create mode 100644 strata/zookeeper/zookeeper-client.morph
create mode 100644 strata/zookeeper/zookeeper-server.morph
create mode 100644 strata/zookeeper/zookeeper.morph
create mode 100644 systems/devel-system-x86_64-zookeeper-client.morph
create mode 100644 systems/devel-system-x86_64-zookeeper-server.morph
--
1.8.4
8 years, 5 months
Re: [PATCH 0/5] Document 'generic' write extension parameters
by Pete Fotheringham
Rewarked after initial review
Main changes are to DTB_PATH and BOOTLOADER_* params
Pete Fotheringham (5):
Document DTB_PATH write extension parameter
Document BOOTLOADER_INSTALL and BOOTLOADER_CONFIG_FORMAT write
extension parameters
Whitespace removal
Document KERNEL_ARGS write extension parameter
Add 'do not use' warnings to nfsboot write extension
morphlib/exts/kvm.write.help | 31 ++++++++++++++++++++++++++++++-
morphlib/exts/nfsboot.write | 13 +++++++++++--
morphlib/exts/nfsboot.write.help | 9 ++++++++-
morphlib/exts/rawdisk.write.help | 30 ++++++++++++++++++++++++++++++
morphlib/exts/virtualbox-ssh.write.help | 30 ++++++++++++++++++++++++++++++
morphlib/writeexts.py | 22 ++++++++++++++--------
6 files changed, 123 insertions(+), 12 deletions(-)
--
1.7.11.3
8 years, 5 months
[PATCH 0/4] morph: Error message improvements
by Sam Thursfield
Repository: git://git.baserock.org/baserock/baserock/morph
Ref: sam/improve-command-failure-errors
Sha1: 9449dbfe1bb1800dfb15de025b87e1846e25e74a
Sam Thursfield (4):
Give less scary error messages when a containerised command fails
Remove duplicate 'import' statement
Give a useful path to failed staging areas
Tiny optimisation of app.runcmd code path
morphlib/app.py | 26 ++++++++++++++-----------
morphlib/builder2.py | 17 ++++++++++++-----
morphlib/stagingarea.py | 50 +++++++++++++++++++++++++++++++------------------
morphlib/util.py | 17 ++++++++++++++++-
4 files changed, 75 insertions(+), 35 deletions(-)
--
2.1.3
8 years, 5 months
[PATCH] Lorrying in zookeeper project
by Mike Smith
Repository location :
git://git.baserock.org/baserock/local-config/lorries.git
Branch: baserock/krin/zookeeper-programs-add
reason for adding: This lorries in a program that was created be
myself, using c++ standard libarys and the zookeeper API to show and
demonstrate that zookeeper now functions in baserock as both server and
client.
From 936b988c6ddb95a81a9083011189e2298bf2390b Mon Sep 17 00:00:00 2001
Date: Wed, 10 Dec 2014 14:33:17 +0000
---
open-source-lorries/zookeeper-programs.lorry | 7 +++++++
1 file changed, 7 insertions(+)
create mode 100644 open-source-lorries/zookeeper-programs.lorry
diff --git a/open-source-lorries/zookeeper-programs.lorry
b/open-source-lorries/zookeeper-programs.lorry
new file mode 100644
index 0000000..196f8f1
--- /dev/null
+++ b/open-source-lorries/zookeeper-programs.lorry
@@ -0,0 +1,7 @@
+{
+ "zookeeper-programs":
+ {
+ "type": "git",
+ "url": "https://github.com/MichaelPSmith/ZookeeperExperiments"
+ }
+}
--
1.8.4
8 years, 5 months
[PATCH 0/5] Document 'generic' write extension parameters
by Pete Fotheringham
Pushed to github branch petefoth/generic-params1
https://github.com/petefoth/morph/tree/petefoth/generic-params1
Pete Fotheringham (5):
Document DTB_PATH write extension parameter
Document BOOTLOADER_INSTALL and BOOTLOADER_CONFIG_FORMAT write
extension parameters
Whitespace removal
Document KERNEL_ARGS write extension parameter
Add 'do not use' warnings to nfsboot write extension
morphlib/exts/kvm.write.help | 29 ++++++++++++++++++++++++++++-
morphlib/exts/nfsboot.write | 13 +++++++++++--
morphlib/exts/nfsboot.write.help | 9 ++++++++-
morphlib/exts/rawdisk.write.help | 28 ++++++++++++++++++++++++++++
morphlib/exts/virtualbox-ssh.write.help | 28 ++++++++++++++++++++++++++++
morphlib/writeexts.py | 22 ++++++++++++++--------
6 files changed, 117 insertions(+), 12 deletions(-)
--
1.7.11.3
8 years, 5 months
