[RFC] YBD - proof-of-concept work for improving Baserock workflow
by Paul Sherwood
Over the last month or so I've been exploring some possible improvements
for Baserock, in particular some simplifications to
- syntax for definitions
- code for parsing definitions
- code for overall build logic
- hashing algorithm for cache-keys
My approach has been to start from a clean base, and then bring in
ideas/code from morph where I needed them. I did consider trying to
implement these things in morph itself, but got scared off by the
codebases for morph and its tests.
I've ignored some real-world complications, and frankly was not
intending to publish yet - I wanted to get it actually building
something first.
But given the discussions on runtime-depends, I'm offering this now,
just in case the simplifications I'm proposing can help to steer us away
from further complications in morph itself.
So, that said:
https://github.com/devcurmudgeon/ybd
To try it out
- git clone it into (say) /src/workspace
- git clone definitions into a sibling directory
- in the definitions directory
../ybd/morph-converter.sh
../ybd/check.sh systems/genivi-baseline-x86_64-generic.def
(or, for example) ../ybd/check.sh strata/core.def
Notes
- it doesn't build, but it does
- clone git repos from gbo if necessary (which is slow...)
- walk the definitions tree in build order, very fast
- calculate cache-keys for all required artifacts, very fast
- warn on duplicate definitions
(eg we have several u-boot definitions)
- error on recursive definitions
(eg ruby stratum includes ruby chunk)
- report last upstream tag for a given definition
- it can be run against any system/stratum/chunk
- less than 1000 lines of code at this point
I believe that
- this cache calculation is easier to understand than what's in morph
(and even if there are factors i'm missing, it would still be better
imo to do it this way)
- this logic allows us to assemble only components that are specified
for inclusion ('contents' in this syntax) in output artifacts, hence
no need (I think) for runtime-dependencies
br
Paul
--
Paul Sherwood Codethink Ltd.
Tel: +44 788 798 4900 302 Ducie House, Ducie Street,
http://www.codethink.co.uk/ Manchester, M1 2JW, United Kingdom.
Codethink provides advanced software design, development, integration &
test services: from embedded systems to high performance apps to cloud.
7 years, 5 months
Policy: Merging documentation changes
by Pete Fotheringham
As the documentation changes I am working on are mostly in source files,
I am sending them for review in the same way as for patches which change
code. However, the normal patch review workflow (whereby changes don't
get merged until all comments have been satisfactorily addressed) seems
a little heavy handed, given that there is no chance of my changes
'breaking' existing code.
I would like to propose a change, whereby
1. 'Documentation only' changes can be merged *before* all comments have
been addressed / corrected - unless any reviewer feels the changes are
bad enough to warrant a -1;
2. Comments received will be addressed in the next patch set that is
sent for review
Or to express it another way, reviewers can give these changes +1 even
if they have identified changes that need to be made.
I believe that making this change would speed up the process of
improving the quality of the Baserock user documentation, without any
negative effects on the functionality of the software.
Thoughts?
Pete
--
Pete Fotheringham
+44 7740 351755
Codethink Ltd
http://codethink.co.uk
7 years, 5 months
Deploy to ISO - for use with VMware player
by McCall, Gavin (G.F.)
Are there instructions to create an iso with deploy ilo img?
I ask because running VMware, I get full OS integration features running - USB, drag and drop, shared folders etc.
Despite a number of trys, with Virtual Box, I only get errors for the above ;(
Gavin
7 years, 5 months
[PATCH] morph: Rename builder2 module to builder
by Sam Thursfield
Repository: git://git.baserock.org/baserock/baserock/morph
Ref: sam/rename-builder2
Sha1: fb7eb661b2b02bf049d24fa4a232f303b5058b1b
No functional changes.
Sam Thursfield (1):
Rename builder2 module to builder
morphlib/__init__.py | 2 +-
morphlib/buildcommand.py | 4 ++--
morphlib/{builder2.py => builder.py} | 0
morphlib/{builder2_tests.py => builder_tests.py} | 8 ++++----
morphlib/plugins/cross-bootstrap_plugin.py | 4 ++--
5 files changed, 9 insertions(+), 9 deletions(-)
rename morphlib/{builder2.py => builder.py} (100%)
rename morphlib/{builder2_tests.py => builder_tests.py} (96%)
--
1.9.3
7 years, 5 months
[PATCH 00/12] Add python support to import tool
by Richard Ipsum
repo: git://git.baserock.org/baserock/baserock/import
ref: baserock/richardipsum/python_v3
head: 65278fdf1ec80a784f9ada8d390ad063a459c97a
This series adds support for importing python packages from pypi,
please see README.python for more details.
Richard Ipsum (12):
Make stratum build-depends be set by importer
Make rubygems importer depend on strata/ruby.morph
Add python subcommand
Log when we receive a msg on stderr
Add common functions used by python extensions
Add python.to_lorry
Add tests for python.to_lorry
Add python.find_deps
Add tests for requirement conflict detection
Add python.to_chunk
Add README.python
Add TODO.python
README.python | 59 ++++
TODO.python | 92 +++++++
baserockimport/app.py | 23 +-
baserockimport/exts/importer_python_common.py | 87 ++++++
baserockimport/exts/python.find_deps | 352 ++++++++++++++++++++++++
baserockimport/exts/python.to_chunk | 33 +++
baserockimport/exts/python.to_lorry | 219 +++++++++++++++
baserockimport/exts/python_find_deps_tests.py | 362 +++++++++++++++++++++++++
baserockimport/exts/python_lorry_tests.py | 72 +++++
baserockimport/mainloop.py | 18 +-
10 files changed, 1309 insertions(+), 8 deletions(-)
create mode 100644 README.python
create mode 100644 TODO.python
create mode 100644 baserockimport/exts/importer_python_common.py
create mode 100755 baserockimport/exts/python.find_deps
create mode 100755 baserockimport/exts/python.to_chunk
create mode 100755 baserockimport/exts/python.to_lorry
create mode 100755 baserockimport/exts/python_find_deps_tests.py
create mode 100755 baserockimport/exts/python_lorry_tests.py
--
1.7.10.4
7 years, 6 months
[PATCH 0/6] Changes needed for pxeboot.write
by Pedro Alvarez
Repo: baserock:baserock/definitions
Branch: baserock/pedroalvarez/pxeboot-changes-needed
Sha1: d1f63f564a2623fac63f5ea95ee3a8252d38dc0f
Land: master
I found some problems when trying to use the pxeboot.write extension.
This patch tries to solve all of them.
Pedro Alvarez (6):
Add ipmitool to strata/tools.morph
pxeboot.check: Fix a couple of syntax errors in the code
pxeboot.check: Multiple complete_matches is not an error
pxeboot.check: Fail with a nice error if PXEBOOT_MODE is needed and
not set
pxeboot.check: Check if nfs-server is running when is needed
Add a cluster example to deploy into hardware
clusters/hardware-deployment.morph | 36 ++++++++++++++++++++++++++++++++++++
pxeboot.check | 27 +++++++++++++++++++++------
strata/tools.morph | 7 +++++++
3 files changed, 64 insertions(+), 6 deletions(-)
create mode 100644 clusters/hardware-deployment.morph
--
1.7.10.4
7 years, 6 months
[PATCH] Update to latest version of the Import Tool
by Sam Thursfield
Repository: git://git.baserock.org/baserock/baserock/definitions
Ref: sam/update-import-tool
Sha1: 41ae2b94c73dc7ae26b03795971fa6bc4d8ee907
This updates the Baserock Import tool to include various fixes I made
in the process of writing up the tutorial at
<http://wiki.baserock.org/guides/import-tool-rubygems/>
I've not sent a full patch series to the mailing list for this change.
I can if it is considered an important enough use of our resources to do a
detailed review, but since this commit can't break anything other than the
import tool itself, that may not be the best use of developer time!
If you want to quickly glance the code to check for spyware and trojans,
this will show you all the changes included in this update:
git diff edd18a77973d8cd12b09bc6e3ac06fecd72a7e45..eb1a6a511c85163fe3e7ede56a348206075d9af9
Sam Thursfield (1):
Update to latest version of the Baserock Import tool
strata/baserock-import.morph | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--
1.9.3
7 years, 6 months
[PATCH] Add npm support to import tool
by Zara Zaimeche
Repository: git@github.com:Zarathecat/baserockimport.git
Ref: zaranpm
Sha1: 1cb1af33ef285cd872c77e32df1b16f0f0390a29
Some of the dependencies needed are integrated in the branch
'baserock/sam/import-npm', but not all of them are there yet,
so in the meantime you need to pass this on the commandline, before the
baserock-import command, to run the
tool:
NODE_PATH=/usr/lib/node_modules/:/usr/lib/node_modules/npm/node_modules:/usr/lib/node_modules/npm/node_modules/request/node_modules/form-data/node_modules
(eventually it should match up with the env variable currently given in the readme)
Zara Zaimeche (1):
Add npm support to import tool
README.npm | 21 ++++++++
baserockimport/app.py | 18 +++++++
baserockimport/exts/npm.find_deps | 95 +++++++++++++++++++++++++++++++++++++
baserockimport/exts/npm.to_chunk | 59 +++++++++++++++++++++++
baserockimport/exts/npm.to_lorry | 42 ++++++++++++++++
5 files changed, 235 insertions(+)
create mode 100644 README.npm
create mode 100755 baserockimport/exts/npm.find_deps
create mode 100755 baserockimport/exts/npm.to_chunk
create mode 100755 baserockimport/exts/npm.to_lorry
--
1.7.10.4
7 years, 6 months
[PATCH 00/13] Upgrade ceph and its dependencies
by Patrick Darley
Repo: git@github.com:padrigali/definitions.git
Sha1: faad922fb616e6b255e67ded1521e17a9ccbe8d1
Ref: updateCephServiceSWrb3
Patrick Darley (13):
Remove hack supressing build errors from boost-tarball
Switch to more recent release of the boost libraries
Switch to most recent release of libaio
Switch to most recent release of keyutils
Switch to latest version of libunwind
Switch to latest release of gperftools
Switch to a more recent version of leveldb
Update install commands for leveldb
Update version of snappy used by ceph service
Switch to a patched version of ceph release v0.89
Ensure ceph configures to be run without libxfs
Install the systemd units for operating ceph nodes and change name of
the SysV script to accomodate this
Switch to cpython v2.7.9
strata/ceph-service.morph | 20 ++++++++++----------
strata/ceph-service/boost.morph | 4 ++--
strata/ceph-service/ceph.morph | 11 +++++++++--
strata/ceph-service/leveldb.morph | 5 ++++-
strata/core.morph | 4 ++--
5 files changed, 27 insertions(+), 17 deletions(-)
--
1.7.10.4
7 years, 6 months
[PATCH] Support armv8l, armv8b, aarch64, and aarch64b
by Emmet Hikory
git-send-email doesn't like me, so I've attached the output of
git-format-patch inline, so others may review this candidate patch
to morph. In testing, this allowed me past the two obvious errors
when attempting to build armv7lhf artifacts in aarch64 in a chroot
prepared with linux32 (putting the system in armv8l mode). I was not
able to complete a system build, but the problem appeared to be with
an incomplete chroot configuration, rather than with morph.
The change in util.py is structured to easily allow the addition
of support for the armv6_8 architecture proposed by Rob Taylor, but this
is not included, as I have no means to test at the moment (and suspect
the nomenclature may benefit from further discussion).
--
Emmet HIKORY
7 years, 6 months
