> To summarize, I asked Christoph off list (because it was
> late at night and I didn't want him to feel forgotten) to check the
> kernel startup output from running ./aboriginal-start --interactive
> from the aboriginal-controller wrapper scripts to further debug the
> problem, and he has also been trying to add support for ARMv7-a
> float to aboriginal.
meanwhile I compiled the ARMv5l image again from scratch and added the --interactive option to the Python script in YBD (sandbox.py) that calls aboriginal-start.
This time I saw the boot messages of the kernel and finally got a shell prompt. So this seems to be working.
Regarding Cortex-A15 I suspect that a device tree is needed to bring up this platform.
On Fri, 2016-05-27 at 16:41 +0000, Baumann, Christoph (C.) wrote:
> Hello Tristan,
Bringing this back on-list, sorry I have been multitasking a lot and
trying to catch up with various emails this morning.
To summarize, I asked Christoph off list (because it was excessively
late at night and I didn't want him to feel forgotten) to check the
kernel startup output from running ./aboriginal-start --interactive
from the aboriginal-controller wrapper scripts to further debug the
problem, and he has also been trying to add support for ARMv7-a hard
float to aboriginal. A separate thread on the aboriginal mailing list
has been started regarding ARMv7-a support for which Rob's reply is
> as the ARMv5 was only a test to see if things can work, I started
> getting a ARMv7-a (i.e. Cortex-A15) VM running.
> But so far the resulting image wouldn't boot at all. QEMU crashes
> even before trying to start the kernel: "PFLASH: Possible BUG - Write
> block confirm".
> The qemu command used is this:
> qemu-system-arm -m 1024M -M vexpress-a15 -serial stdio -cpu cortex-
> a15 -nographic -no-reboot -kernel linux -initrd rootfs.cpio.gz
> -append "mem=1024M panic=1 HOST=armv7lhf $KERNEL_EXTRA" $QEMU_EXTRA
> As kernel config I used arch/arm/configs/vexpress_defconfig.
So we are looking at 2 separate issues here, one is that the aboriginal
automation of ybd is not working on your setup. To test this we should
not get ahead of ourselves and stick with armv5l which we know works.
For this first issue, it would be best to start the aboriginal-start
wrapper script with the interactive flag, because without that we do
not connect anything to the kernel's boot terminal and we do not get
any useful output.
I should point out that the aboriginal-start differs from the dev-
environment.sh script mostly because of the qemu features that it uses.
Particularly, it adds the following qemu options:
Setup a virtfs for the 9p mount:
Setup 2 buffered serial ports:
Without having looked at the bootup messages from aboriginal-start in
interactive mode, we cant really know for sure but I think it's a good
bet that this will not work on your slightly older version (2.0.0) of
For ARMv7-A support, it's great to see that someone is looking at this,
supporting ARMv7-A and AArch64 is also important to us :)
I did also make a similar attempt as you did for ARMv7-A and obtained
similar results. I have a local IRC log of a conversation with Rob in
which he tries to explain some details about what efforts go into
supporting a new arch.
First, I think we're missing the compiler tuning options you provided
for your ARMv7-A attempt, and further, I think that we can draw
inspiration from other projects, including Baserock definitions for the
armv7lhf target in build-essential. The trick here is to build a cross
compiler that targets a very specific ARM feature-set. For ARMv7-A, we
are most probably interested in generating binary code which:
o Expects thumb2 features are present
o Uses vfpv3 (vector floating point v3)
o Has neon SIMD available
The last one is probably unimportant for testing purposes but really
should be chosen for production.
This needs to be expressed in the GCC_FLAGS in your new ARMv7-A target
file, as Rob explains in his reply on the Aboriginal mailing list, the
kernel build system will interrogate the compiler to some extent and
decide how to build itself, in addition to it's own config.
Getting it off the ground is a delicate dance of bringing all three
aspects in line:
o Compiler target tuning options
o Kernel configuration
o Qemu launch command
At the end of this mail, I tried to summarize comments from an IRC
conversation which should give you at least a vague idea of what your
time might look like should you endeavor to debug it and add support
for ARMv7-A, it's a bit messy sorry for that, it's just a summary of a
random IRC conversation but should still help to provide some insight.
Summary of IRC chat explaining about adding new targets:
o Build your compiler (needs a tuple and gcc/binutils configure
options) and libc (uClibc needed a config, musl generally needs a
o Get a hello world program that qemu-blah can run.
- Finding an existing hello world and single-stepping through
qemu's -s mode hooked to gdb "target remote" is sometimes
- objdump -d is, unfortunately, your friend.
o Once you've got _that_, skip the rest of userspace and build a
- Try to get console output.
- Pick a target board qemu can emulate that has a linux defconfig.
- Lists available boards - qemu-system-blah -M "?"
So do your "make ARCH=arm64 defconfig", then "mv .config walrus" and
"ARCH=arm64 ~/aboriginal/more/miniconfig.sh walrus" that'll grind for
like an hour.
And at the end you have a mini.config file. Then run it through that
filter, which yanks the baseconfig symbols out ("filter
baseconfigfile miniconfigfile" with output to stdout)
Then read through your symbols and look each one up in
"make ARCH=arm64 menuconfig"'s help.
Forward slash is your friend there (search for config symbol)
o Build that (yanking all =m symbols first becuase if it's a module,
it's not needed to boot the system.)
o Once you've got console output, try to get an initramfs going.
for armv7l you can just use an armv5l cpio file, it's backwards
But the kernel won't build without the right toolchain because it uses
inline assembly that cares. :P
Eventually you want to add just the symbols to enable board support,
including board-specific devices.
Use the other sources/targets as examples.
Apart from various cleanups, this version is mainly about
- artifact-version: 4 (include definitions repo: and ref: in metadata
for systems and strata)
- @rjek provided a fix for ybd failing to handle nginx compressed
- @jjardon added support for running ybd on Arch Linux
e334c50 Add definitions repo+ref to .meta, from artifact-version: 4
77ad4e4 Only report skipping once
2b50266 Delay sandbox creation - may help for #139
fde2f67 Fix #175 using Eli Bendersky's Counter class
52e802d Gah... fix 543b654
543b654 Add 'skipping compose' message
8aed0f8 Lose a couple of lines
e403cfb Reverse logic, lose a line
62fedc2 Fix comments in default ybd.conf
7a3cb92 Fallback to copy all files if metadata copy fails
63e3567 Shorten some lines by 'from app import...'
739631b Shorten lines by 'from app import...'
016398f Make 'contents' field dict, containing splits if any
d33e0c8 Artifact-version: 4, include repo: & ref: in meta
13ab7a0 Standardise .meta artifact|files => artifact|components
dc24b2f Cache-key only in .meta from artifact-version 2
2b3786f System cache-key varies with default-splits
25c58d2 Report actual 'kind' in splitting log
b43bb44 Clarify what the try/except is for
41e064a Lose a couple of lines
4304456 Rename element => product
a1c8bc3 Don't include anchors in parse output
f00e6cd Output internal format as yaml
b09eeb4 Revert "Attempt fix for #218"
94461f4 KBAS was using response.raw, instead of response.content
b2f0866 Missed another absolute /static
f34bd8a Use relative path for css
5ccb421 Exit 1 on calling with no params
38c2220 Merge pull request #219 from jjardon/jjardon/arch
155c828 install_dependencies.sh: Add support for Arch
b4e6cdd Attempt fix for #218
5c2c9a3 Exit if YAML parsing fails on any definition
4972e30 Check for installation success
8959629 Check that apt|yum|dnf is found
dace4d1 Disable splitting by default, because it's slower
6aa0ffb No need to exit if bootstrap artifacts missing
42a2cd7 Drop 'Cache key is' text
5300c64 State where surprise exceptions happen
5ed3cba Exit on missing morph file
3155ba9 Only support split systems from artifact-version 3
779d3e7 Force new cache-key for split systems
461d282 Create runtime systems by default
7c73938 Move checkboxes so they are easier to scan
f33c244 Fixes #163 Warn if any specified .morph file is missing
950d6aa Remove wrangler.py
> First off, how precisely are you encountering this error ?
> After building the aboriginal cross compiler and system images
> after build.sh completes)... is the above what happens when running
> ./dev-environment.sh from the system image directory ?
starting it with dev-environment.sh works. But using YBD ('/path/to/ybd/ybd.py strata/build-essential.morph armv5l')
it gets stuck as previously described.
As my intended target is armv7l(hf)/Cortex-A9 I also tried to build an image for this. The build succeeded, but the image doesn't boot even with dev-environment.sh.
> While almost everything in the build is very self contained, one
> that we dont build from scratch is qemu. What is the output of
> system-arm --version' for you ?
> I have been testing with QEMU 2.5.0 (Debian 1:2.5+dfsg-5).
I use "QEMU emulator version 2.0.0 (Debian 2.0.0+dfsg-2ubuntu1.24)" on Xubuntu 'trusty'.
Recently I tried the experimental approach to cross-compiling described here:
It seems that the images I built won't boot up.
The kernel is uncompressed, but not started:
"uncompressing linux... done booting the kernel"
This is the only message I see.
The build was done with this command:
ENABLE_GPLV3=1 CROSS_COMPILER_HOST=x86_64 SYSIMAGE_TYPE=ext2 ./build.sh armv5l
> We introduced this behavior in:
> In Morph, we use the name of a chunk for various things, and having
> different chunks with the same name was creating various issues.
> In your case it seems that your system:
> - Is building 'audio' and 'audio-bluetooth', and both of them
> 'alsa-lib' and 'alsa-utils'. This may be caused for various
> - Is building a chunk called 'u-boot', twice. We have faced this
> too, and decided to rename the one in the bsp. In your case this
> solution would be changing the name of
> 'strata/bsp-armv7lhf-jcihw/u-boot.morph' from 'u-boot' to something
> else like 'u-boot-jcihw'
> - Is building 'gstreamer' twice, and different versions, I don't
> why without looking at your definitions.
> - Is building 'dbus-glib' twice, using the same chunk morphology,
> different versions. This one is really weird, and I can't tell
> looking at the definitions either.
thanks for the hints. I managed to disentangle the morphology.
There were indeed some oddities that showed up now.
It turned out that I was just not patient enough. The migration succeeded in the end.
In the next step I wanted to build the migrated morphology with the latest 'morph'.
There I ran into strange exceptions/bachtraces. Tinkering a bit with the Python code it
turned out that VERSION and DEFAULTS were searched in some odd tmp-directory.
After I replaced the corresponding variable with 'os.getcwd()' in some places it worked.
But I assume that there should be a 'correct' way to get the build running?
I'm trying to run the migration scripts from http://git.baserock.org/cgit/baserock/baserock/spec.git against our definitions.
This works until 006-specify-build-system.py which fails fetching info from our trove:
"WARNING: Unexpected response from server http://durlach-trove:8080/ for repo git://durlach-trove/delta/libndp: 404 Client Error: Not Found"
What would the correct (complete) URL be for this check? Maybe my system is lacking some configuration?
I'm raising this here since I've had a couple of comments on my
lorry-controller series regarding performing migrations as necessary
when entering the statedb context manager.
I personally don't have enough experience with databases to know how
best to go about this. The current implementation checks whether a
column exists within a table, and adds it if not. From the comments
I've received it seems as though it would be preferable for migrations
to happen outside of statedb.py, through some script or otherwise (see
Since submitting the series I have noticed StateDB does in fact have a
'version' table, which I could make use of instead of figuring out what
migrations are needed by nonexistence of the thing I want to add
(certainly seems like a better idea, despite the possible copy/paste
error (column in version table is named 'running')), but would the code
for this then be part of some other Migrations class that statedb calls
to? Or would I instead create a lorry-controller-migrations-runner
script or similar with accompanying systemd units to be run before any
other lorry-controller services?
Advice on this would be much appreciated. I'm somewhat blocked on doing
more Lorry Controller + GitLab related work until this series is merged,
and I'd like to do this properly so that future migrations aren't made