This patch finishes work to speed up distbuild's build graph calculation. It
currently takes a long time due to serialising everything about the artifacts
in order to exactly reconstruct them. Since we can now calculate the build
graph cheaply (it takes about 1 second), we don't need to reconstruct these
artifacts exactly as the worker can calculate the build graph itself and
find what it needs to build given less information.
This allows us to reduce the amount of data which we serialise when calculating
the build graph, which in turn speeds up the "json.dump(yaml.dump(content))"
that we do to serialise artifacts.
V3 of this patch makes it apply cleanly and still work, after Sam's recently
merged branch changed some of the same code.
NOTE: This patch requires Richard Ipsum's caching fixes to see much
Adam Coldrick (3):
Don't serialise the entire build graph
Make distbuild use an ArtifactReference not an Artifact internally
Calculate the build graph in worker-build
distbuild/build_controller.py | 22 +--
distbuild/serialise.py | 273 ++++++++++++++--------------------
distbuild/serialise_tests.py | 79 ++--------
distbuild/worker_build_scheduler.py | 29 ++--
morphlib/plugins/distbuild_plugin.py | 25 +++-
5 files changed, 178 insertions(+), 250 deletions(-)
This patch updates the version of cmake in the core stratum from
v220.127.116.11 to v3.2.1. I have built and run an image based on this
version of cmake without problem.
strata/core.morph | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/strata/core.morph b/strata/core.morph
index 263eac6..84dcb72 100644
@@ -12,8 +12,8 @@ chunks:
- name: cmake
- ref: 0b3781e4c497566acaa504d4106d11c02a642501
- unpetrify-ref: v18.104.22.168
+ ref: 732d8a467ab8a295b1bbf580f865980f8a1a9b29
+ unpetrify-ref: v3.2.1
- name: gdbm
Mox3 is designed to work with python 3, and is tested with
python 3.2, 2.7 & 2.6
it is a requirement of tempest for openstack and is an
unofficial port version of the google mox framework.
Mike Smith (1):
added branch of mox designed to work with python3, ie. mox3
open-source-lorries/python-packages.lorry | 4 ++++
1 file changed, 4 insertions(+)
This is the second part of the partial build/deployment work I mentioned
in an RFC to this list a few weeks ago. The patches for partial building
are on gerrit.baserock.org, but this one is being sent to this list since
it depends on my OSTree work, which isn't on Gerrit at the moment.
This patch series allows the deployment of one or more chunks/strata from
a system by adding a 'partial-deploy-components' field to deployments in
cluster morphologies. This will only be used if the --partial option is
set, and can only deploy to either tarballs or a sysroot.
It should NOT be used in a production system, but simply as a developer
tool to speed up deployment of individual components.
NOTE: This patch depends on my OSTree work, so you'll need overlayfs
and OSTree in your system for it to work.
Adam Coldrick (2):
Allow the deployment of individual chunks/strata from systems
Add a test for partial deployment
morphlib/app.py | 4 +
morphlib/plugins/deploy_plugin.py | 158 ++++++++++++++++++++++++++++---------
yarns/deployment.yarn | 22 ++++++
yarns/implementations.yarn | 33 +++++++-
4 files changed, 179 insertions(+), 38 deletions(-)
On non-x86 systems, the linker may not be called ld-linux.so.* but
the older name of ld.so.*
When the linker is ld.so.1, for example, the install script of
stage2-glibc does not exit with an error (as basename/ls are in
subshells), but rather $loader is empty string.
This means that the symlink made is from /lib/lib to /tools/lib
rather than from /lib/ld.so.1 to /tools/lib/ld.so.1.
strata/build-essential/stage2-glibc.morph | 2 ++
1 file changed, 2 insertions(+)
diff --git a/strata/build-essential/stage2-glibc.morph b/strata/build-essential/stage2-glibc.morph
index ffa0970..64dbedd 100644
@@ -53,5 +53,7 @@ install-commands:
loader=$(basename $(ls "$DESTDIR$PREFIX"/lib/ld-linux*))
+ [ -z $loader ] && loader=$(basename $(ls "$DESTDIR$PREFIX"/lib/ld.so*))
+ [ -z $loader ] && ( echo "Bug in stage2-glibc ld.so symlinks" ; exit 1 )
ln -s "$PREFIX/lib/$loader" "$DESTDIR/lib/$loader"