May 2016 - baserock-dev - listmaster.pepperfish.net

by Baumann, Christoph (C.)

Hello Tristan, > To summarize, I asked Christoph off list (because it was > excessively > late at night and I didn't want him to feel forgotten) to check the > kernel startup output from running ./aboriginal-start --interactive > from the aboriginal-controller wrapper scripts to further debug the > problem, and he has also been trying to add support for ARMv7-a > hard > float to aboriginal. meanwhile I compiled the ARMv5l image again from scratch and added the --interactive option to the Python script in YBD (sandbox.py) that calls aboriginal-start. This time I saw the boot messages of the kernel and finally got a shell prompt. So this seems to be working. Regarding Cortex-A15 I suspect that a device tree is needed to bring up this platform. Regards, Christoph

7 years, 3 months

1
0
0 / 0

Re: AW: Failing to boot resulting image

by Tristan Van Berkom

On Fri, 2016-05-27 at 16:41 +0000, Baumann, Christoph (C.) wrote: > Hello Tristan, Bringing this back on-list, sorry I have been multitasking a lot and trying to catch up with various emails this morning. To summarize, I asked Christoph off list (because it was excessively late at night and I didn't want him to feel forgotten) to check the kernel startup output from running ./aboriginal-start --interactive from the aboriginal-controller wrapper scripts to further debug the problem, and he has also been trying to add support for ARMv7-a hard float to aboriginal. A separate thread on the aboriginal mailing list has been started regarding ARMv7-a support for which Rob's reply is also enlightening: http://lists.landley.net/pipermail/aboriginal-landley.net/2016-May/0025 75.html > as the ARMv5 was only a test to see if things can work, I started > getting a ARMv7-a (i.e. Cortex-A15) VM running. > But so far the resulting image wouldn't boot at all. QEMU crashes > even before trying to start the kernel: "PFLASH: Possible BUG - Write > block confirm". > The qemu command used is this: > qemu-system-arm -m 1024M -M vexpress-a15 -serial stdio -cpu cortex- > a15 -nographic -no-reboot -kernel linux -initrd rootfs.cpio.gz > -append "mem=1024M panic=1 HOST=armv7lhf $KERNEL_EXTRA" $QEMU_EXTRA > > As kernel config I used arch/arm/configs/vexpress_defconfig. So we are looking at 2 separate issues here, one is that the aboriginal automation of ybd is not working on your setup. To test this we should not get ahead of ourselves and stick with armv5l which we know works. For this first issue, it would be best to start the aboriginal-start wrapper script with the interactive flag, because without that we do not connect anything to the kernel's boot terminal and we do not get any useful output. I should point out that the aboriginal-start differs from the dev- environment.sh script mostly because of the qemu features that it uses. Particularly, it adds the following qemu options: Setup a virtfs for the 9p mount: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -virtfs local,path=${SHARE_DIR},security_model=mapped- file,mount_tag=share Setup 2 buffered serial ports: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -device virtio-serial -chardev pipe,path=${COMM_OUT},id=comm0 -device virtserialport,chardev=comm0,name=serial0 -chardev pipe,path=${COMM_IN},id=comm1 -device virtserialport,chardev=comm1,name=serial1 Without having looked at the bootup messages from aboriginal-start in interactive mode, we cant really know for sure but I think it's a good bet that this will not work on your slightly older version (2.0.0) of qemu. For ARMv7-A support, it's great to see that someone is looking at this, supporting ARMv7-A and AArch64 is also important to us :) I did also make a similar attempt as you did for ARMv7-A and obtained similar results. I have a local IRC log of a conversation with Rob in which he tries to explain some details about what efforts go into supporting a new arch. First, I think we're missing the compiler tuning options you provided for your ARMv7-A attempt, and further, I think that we can draw inspiration from other projects, including Baserock definitions for the armv7lhf target in build-essential. The trick here is to build a cross compiler that targets a very specific ARM feature-set. For ARMv7-A, we are most probably interested in generating binary code which: o Expects thumb2 features are present o Uses vfpv3 (vector floating point v3) o Has neon SIMD available The last one is probably unimportant for testing purposes but really should be chosen for production. This needs to be expressed in the GCC_FLAGS in your new ARMv7-A target file, as Rob explains in his reply on the Aboriginal mailing list, the kernel build system will interrogate the compiler to some extent and decide how to build itself, in addition to it's own config. Getting it off the ground is a delicate dance of bringing all three aspects in line: o Compiler target tuning options o Kernel configuration o Qemu launch command At the end of this mail, I tried to summarize comments from an IRC conversation which should give you at least a vague idea of what your time might look like should you endeavor to debug it and add support for ARMv7-A, it's a bit messy sorry for that, it's just a summary of a random IRC conversation but should still help to provide some insight. Best Regards -Tristan Summary of IRC chat explaining about adding new targets: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ o Build your compiler (needs a tuple and gcc/binutils configure options) and libc (uClibc needed a config, musl generally needs a target added). o Get a hello world program that qemu-blah can run. - Finding an existing hello world and single-stepping through qemu's -s mode hooked to gdb "target remote" is sometimes required. :P - objdump -d is, unfortunately, your friend. o Once you've got _that_, skip the rest of userspace and build a kernel. - Try to get console output. - Pick a target board qemu can emulate that has a linux defconfig. - Lists available boards - qemu-system-blah -M "?" So do your "make ARCH=arm64 defconfig", then "mv .config walrus" and "ARCH=arm64 ~/aboriginal/more/miniconfig.sh walrus" that'll grind for like an hour. And at the end you have a mini.config file. Then run it through that filter, which yanks the baseconfig symbols out ("filter baseconfigfile miniconfigfile" with output to stdout) Then read through your symbols and look each one up in "make ARCH=arm64 menuconfig"'s help. Forward slash is your friend there (search for config symbol) o Build that (yanking all =m symbols first becuase if it's a module, it's not needed to boot the system.) o Once you've got console output, try to get an initramfs going. for armv7l you can just use an armv5l cpio file, it's backwards compatible. But the kernel won't build without the right toolchain because it uses inline assembly that cares. :P Eventually you want to add just the symbols to enable board support, including board-specific devices. Use the other sources/targets as examples.

7 years, 3 months

1
0
0 / 0

YBD 16.22 is released

by Paul Sherwood

Apart from various cleanups, this version is mainly about - artifact-version: 4 (include definitions repo: and ref: in metadata for systems and strata) - @rjek provided a fix for ybd failing to handle nginx compressed downloads - @jjardon added support for running ybd on Arch Linux e334c50 Add definitions repo+ref to .meta, from artifact-version: 4 onwards 77ad4e4 Only report skipping once 2b50266 Delay sandbox creation - may help for #139 fde2f67 Fix #175 using Eli Bendersky's Counter class 52e802d Gah... fix 543b654 543b654 Add 'skipping compose' message 8aed0f8 Lose a couple of lines e403cfb Reverse logic, lose a line 62fedc2 Fix comments in default ybd.conf 7a3cb92 Fallback to copy all files if metadata copy fails 63e3567 Shorten some lines by 'from app import...' 739631b Shorten lines by 'from app import...' 016398f Make 'contents' field dict, containing splits if any d33e0c8 Artifact-version: 4, include repo: & ref: in meta 13ab7a0 Standardise .meta artifact|files => artifact|components dc24b2f Cache-key only in .meta from artifact-version 2 2b3786f System cache-key varies with default-splits 25c58d2 Report actual 'kind' in splitting log b43bb44 Clarify what the try/except is for 41e064a Lose a couple of lines 4304456 Rename element => product a1c8bc3 Don't include anchors in parse output f00e6cd Output internal format as yaml b09eeb4 Revert "Attempt fix for #218" 94461f4 KBAS was using response.raw, instead of response.content b2f0866 Missed another absolute /static f34bd8a Use relative path for css 5ccb421 Exit 1 on calling with no params 38c2220 Merge pull request #219 from jjardon/jjardon/arch 155c828 install_dependencies.sh: Add support for Arch b4e6cdd Attempt fix for #218 5c2c9a3 Exit if YAML parsing fails on any definition 4972e30 Check for installation success 8959629 Check that apt|yum|dnf is found dace4d1 Disable splitting by default, because it's slower 6aa0ffb No need to exit if bootstrap artifacts missing 42a2cd7 Drop 'Cache key is' text 5300c64 State where surprise exceptions happen 5ed3cba Exit on missing morph file 3155ba9 Only support split systems from artifact-version 3 0d2d04f Pep8 779d3e7 Force new cache-key for split systems 461d282 Create runtime systems by default 7c73938 Move checkboxes so they are easier to scan f33c244 Fixes #163 Warn if any specified .morph file is missing 950d6aa Remove wrangler.py

7 years, 4 months

1
0
0 / 0

Re: Failing to boot resulting image

by Baumann, Christoph (C.)

Hi Tristan, > First off, how precisely are you encountering this error ? > > After building the aboriginal cross compiler and system images > (i.e. > after build.sh completes)... is the above what happens when running > ./dev-environment.sh from the system image directory ? starting it with dev-environment.sh works. But using YBD ('/path/to/ybd/ybd.py strata/build-essential.morph armv5l') it gets stuck as previously described. As my intended target is armv7l(hf)/Cortex-A9 I also tried to build an image for this. The build succeeded, but the image doesn't boot even with dev-environment.sh. > While almost everything in the build is very self contained, one > thing > that we dont build from scratch is qemu. What is the output of > 'qemu- > system-arm --version' for you ? > > I have been testing with QEMU 2.5.0 (Debian 1:2.5+dfsg-5). I use "QEMU emulator version 2.0.0 (Debian 2.0.0+dfsg-2ubuntu1.24)" on Xubuntu 'trusty'. Regards, Christoph

7 years, 4 months

1
0
0 / 0

Failing to boot resulting image

by Baumann, Christoph (C.)

Hi! Recently I tried the experimental approach to cross-compiling described here: https://listmaster.pepperfish.net/pipermail/baserock-dev-baserock.org/201... It seems that the images I built won't boot up. The kernel is uncompressed, but not started: "uncompressing linux... done booting the kernel" This is the only message I see. The build was done with this command: ENABLE_GPLV3=1 CROSS_COMPILER_HOST=x86_64 SYSIMAGE_TYPE=ext2 ./build.sh armv5l Any ideas? Regards, Christoph

7 years, 4 months

2
1
0 / 0

Re: Multiple ... chunks detected

by Baumann, Christoph (C.)

Hi Pedro, > We introduced this behavior in: > > - > http://git.baserock.org/cgit/baserock/baserock/morph.git/commit/?id > =6655ef761d77c3234c3cf775bf5ee38bd3f43b0c > > In Morph, we use the name of a chunk for various things, and having > 2 > different chunks with the same name was creating various issues. > > > In your case it seems that your system: > > - Is building 'audio' and 'audio-bluetooth', and both of them > include > 'alsa-lib' and 'alsa-utils'. This may be caused for various > reasons. > > - Is building a chunk called 'u-boot', twice. We have faced this > error > too, and decided to rename the one in the bsp. In your case this > solution would be changing the name of > 'strata/bsp-armv7lhf-jcihw/u-boot.morph' from 'u-boot' to something > else like 'u-boot-jcihw' > > - Is building 'gstreamer' twice, and different versions, I don't > know > why without looking at your definitions. > > - Is building 'dbus-glib' twice, using the same chunk morphology, > but > different versions. This one is really weird, and I can't tell > without > looking at the definitions either. thanks for the hints. I managed to disentangle the morphology. There were indeed some oddities that showed up now. Regards, Christoph

7 years, 4 months

1
0
0 / 0

Multiple ... chunks detected

by Baumann, Christoph (C.)

Hi! After migrating the morphology deploying fails with this error: " ERROR: Multiple `u-boot' chunks detected: upstream:u-boot|fe57382d04b46c37f34cf8d3b3ad876554fd12bf|strata/tools/u-boot.morph|u-boot ssh://git@a2378-trove-1/jci/u-boot.git|fda52ce55b1bb5f3c90053e8e516628e76e5a2d3|strata/bsp-armv7lhf-jcihw/u-boot.morph|u-boot Multiple `alsa-utils' chunks detected: upstream:alsa-utils|f12208425051a7abf5ffc87ac2b9d2bd9e096fe1|strata/audio/alsa-utils.morph|alsa-utils upstream:alsa-utils|f12208425051a7abf5ffc87ac2b9d2bd9e096fe1|strata/audio-bluetooth/alsa-utils.morph|alsa-utils Multiple `gstreamer' chunks detected: upstream:gstreamer|0217fba607637860a475d0b5d48a3d4b249ca360|strata/gstreamer-armv7lhf-rcar-h2/gstreamer.morph|gstreamer upstream:gstreamer|3c6cd2bdb59b9c49013cf3dff87f4d867f9b99bc|gstreamer.morph|gstreamer Multiple `dbus-glib' chunks detected: upstream:dbus-glib|ee85b071fb51efb9f9a48d4a658cbefefd14544a|dbus-glib.morph|dbus-glib upstream:dbus-glib|397e8297d433547c9bf4150ddd2b9e0b4c39628c|dbus-glib.morph|dbus-glib Multiple `alsa-lib' chunks detected: upstream:alsa-lib|ab715503853be8d8f365fd157c3cb97ef7a89f7b|strata/audio-bluetooth/alsa-lib.morph|alsa-lib upstream:alsa-lib|ab715503853be8d8f365fd157c3cb97ef7a89f7b|strata/audio/alsa-lib.morph|alsa-lib " What is this ought to tell me? Mit freundlichen Grüßen / Kind regards ________________________________ Christoph Baumann Senior Software Engineer, Visteon Electronics Germany Location: An der Raumfabrik 33b, 76227 Karlsruhe, Germany Phone: +49 (0) 721 4766 1754 F: +49 (0)721 4766 1263 Email: cbaumann(a)visteon.com Follow Visteon: Visteon Electronics Germany GmbH, Visteonstr. 4 - 10, 50170 Kerpen, Germany Geschäftsführung/Management Board: Michael Scott Booth, Jens Tillner, Jürgen Dörr Registration: Cologne, HRB 78269, VAT-ID Nummer: DE 294 194 802

7 years, 4 months

2
1
0 / 0

Re: running migration scripts with own trove fails

by Baumann, Christoph (C.)

Hello! It turned out that I was just not patient enough. The migration succeeded in the end. In the next step I wanted to build the migrated morphology with the latest 'morph'. There I ran into strange exceptions/bachtraces. Tinkering a bit with the Python code it turned out that VERSION and DEFAULTS were searched in some odd tmp-directory. After I replaced the corresponding variable with 'os.getcwd()' in some places it worked. But I assume that there should be a 'correct' way to get the build running? Regards, Christoph

7 years, 4 months

1
0
0 / 0

running migration scripts with own trove fails

by Baumann, Christoph (C.)

Hi! I'm trying to run the migration scripts from http://git.baserock.org/cgit/baserock/baserock/spec.git against our definitions. This works until 006-specify-build-system.py which fails fetching info from our trove: "WARNING: Unexpected response from server http://durlach-trove:8080/ for repo git://durlach-trove/delta/libndp: 404 Client Error: Not Found" What would the correct (complete) URL be for this check? Maybe my system is lacking some configuration? Regards, Christoph

7 years, 4 months

1
0
0 / 0

Lorry Controller StateDB Migrations

by Ben Brown

Hello all, I'm raising this here since I've had a couple of comments on my lorry-controller series[1] regarding performing migrations as necessary when entering the statedb context manager. I personally don't have enough experience with databases to know how best to go about this. The current implementation checks whether a column exists within a table, and adds it if not. From the comments I've received it seems as though it would be preferable for migrations to happen outside of statedb.py, through some script or otherwise (see https://pypi.python.org/pypi/yoyo-migrations). Since submitting the series I have noticed StateDB does in fact have a 'version' table, which I could make use of instead of figuring out what migrations are needed by nonexistence of the thing I want to add (certainly seems like a better idea, despite the possible copy/paste error (column in version table is named 'running')), but would the code for this then be part of some other Migrations class that statedb calls to? Or would I instead create a lorry-controller-migrations-runner script or similar with accompanying systemd units to be run before any other lorry-controller services? Advice on this would be much appreciated. I'm somewhat blocked on doing more Lorry Controller + GitLab related work until this series is merged, and I'd like to do this properly so that future migrations aren't made more difficult. Regards, Ben [1] https://gerrit.baserock.org/#/c/2099/

7 years, 4 months

2
3
0 / 0

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

baserock-dev May 2016