March 2016 - baserock-dev - listmaster.pepperfish.net

YBD + Aboriginal status update and HOWTO

by Tristan Van Berkom

Hi all, Warning, this email will be a bit long... I've included a new and revised list of milestones towards getting the ybd-aboriginal-baserock thing into a mergable state. Also, I have a prototype so I've included instructions at the end of this email for anyone who might be interested in trying to reproduce this setup. As an additional note which doesn't fit anywhere else, I've discovered another, surmountable obstacle just yesterday, which is concerning building baserock with aboriginal as a regular user and dropping the requirement to run the build as root. What I have found is that some of our build instructions try to install things explicitly as root, instructions such as: install -m644 -o root -g root ... These instructions work when run in an emulator where the emulator has a filesystem image mounted and is installing in a chroot on that filesystem. These instructions however do not work when running in the emulator and installing onto a 9p virtfs mount (a mount of a shared host directory), this is because in the virtfs share scenario, the emulator can only really write to the target filesystem using the privileges it has. I think that essentially this can be worked around by: o Disallowing any install-commands which try to explicitly install things as the root user/group. o Performing some additional chown commands in the system-integration commands. This would be a practical approach because using a virtfs mount for the builds is very practical, as one need not create fixed size filesystem images to slave to each qemu worker, and then extract the content of those fs images in between builds. However, once all of the artifacts are prepared for a given system image, it can be trivial enough to create a fixed size, large filesystem image for the emulator to perform the system integration commands on. Revised status and milestones ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Integration with YBD ~~~~~~~~~~~~~~~~~~~~ Some progress here already, still things to do: o Currently one must specify the emulator & controller directories in the ybd.conf file, this needs to be smarter so that depending on the target arch ybd uses the correct emulator for that arch... somehow. o YBD is not shutting down the emulators when exiting yet. o Eventually we probably want to absorb parts of the aboriginal-controller scripts into YBD and upstream some of this to aboriginal startup scripts proper, but it's not a hard blocker. Build base system ~~~~~~~~~~~~~~~~~ Currently we have most definitions for build-essential ported, build essential at this moment builds glibc, I would have ported my gcc runtime library builds by now (which has been tested and works) except that it takes a very long time to build and validate this. Once this is done, need to get through a build of a complete base system. Run Deployment in the emulator ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This item will essentially let us have a more robust deployment technique which is not dependent on host tooling in any way. The basic idea at this stage is to create a special chroot environment inside the emulator where we stage all the artifacts necessary to run the build extensions in a given deployment, and run the deployment in that chroot. o need to figure out how to identify which chunks are required to be staged, this may be hard coded in a first pass before becoming a part of the specification. o need to figure how to run the extensions inside a staged build area in the emulator in order to deploy the image, probably this means staging the build extension scripts into the same chroot area. Avoiding the emulator for native builds ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Need to sort out how we can optionally use a simple chroot and avoid the emulator workers. Here we'll want to use the same aboriginal compiler and the same build-essential of course. Aboriginal already has a 'chroot-splice.sh' script which sets up a directory with it's compiler tooling suitable for chrooting into which we should be able use almost as is. This should be fairly simple, possibly we will want the option to keep using the emulator even for a native build only in the final system integration and deployment stages. Doing this in combination with fixing any issues we have with linux-user-chroot would most probably allow us to shed the root requirement even for native builds. Need to assert the version of the aboriginal image from the definitions ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ In the interest of perfect reproducibility, we dont want arbitrary versions of the compiler used in conjunction with definitions and ybd. So, we will need a way to have the definitions themselves assert the exact version of sources used in the aboriginal kit and use a very specific version. There are a few ways to skin this cat, from creating a versioning scheme for the aboriginal compiler builds to just committing compiled copies of the aboriginal kit (they are relocatable) and specifying an sha1sum of the kit we want to use. NOTE: A more complex approach would be to internalize the aboriginal build completely and integrate it as a part of build essential. Personally I would rather find a way to cooperate with upstream Aboriginal as much as possible and avoid duplicating it inside of Baserock. Implement important arches in upstream aboriginal ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This is not strictly a part of the exercise, but it should be noted that aboriginal has support for the following arches: armv4l, armv4tl, armv5l, armv6l, i486, i586, i686, x86_64, m68k, mips, mips64, mipsel, powerpc, powerpc-440fp, sh2eb, sh2elf, sh4, sparc A hand full of these have not been ported to build with musl libc in aboriginal and as such will not build with the new GCC 5.3 toolchain. Some of these build for musl but still need some minor tweaks to aboriginal sources. I have only tested arm and mips target arches. Most importantly, we're lacking armv7, armv7lhf and aarch64. HOWTO Build (for now) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ There are now a few moving parts, in order to try this out you are going to need the following components: Step 1, Getting the source ~~~~~~~~~~~~~~~~~~~~~~~~~~ Aboriginal fork ~~~~~~~~~~~~~~~ URL: https://github.com/gtristan/aboriginal Branch: extra-changes This fork contains all the parts of the work intended for upstreaming to aboriginal proper (port to gcc 5.3, some patches to the kernel config), some things from the controller (below) may be added to this downstream over time as well. Aboriginal Controller ~~~~~~~~~~~~~~~~~~~~~ URL: https://github.com/gtristan/aboriginal-controller Branch: master This is a wrapper for the aboriginal launch scripts, it allows running the qemu environment in the background as a build slave and sending some commands to it over an IPC. YBD Fork ~~~~~~~~ URL: https://github.com/gtristan/ybd/ Branch: aboriginal My downstream ybd work branch which contains the experimental support for now. Definitions wip branch ~~~~~~~~~~~~~~~~~~~~~~ URL: https://git.baserock.org/git/baserock/baserock/definitions.git Branch: baserock/tristan/wip/aboriginal Step 2, Building stuff ~~~~~~~~~~~~~~~~~~~~~~ First, remember to take note of the branches I pointed out in the previous section. First build aboriginal ~~~~~~~~~~~~~~~~~~~~~~ You will have to choose 2 things at this point: - HOST: The arch you are building on - TARGET: The arch you are building for These are file names in ${ABORIGINAL}/sources/targets, but note that not all targets are supported with the newer gcc 5.3 toolchain we require. You can safely use intel arches, armv5l (and probably all the supported arm variants) and mips. To build, choose your host and target and run: cd /path/to/aboriginal ENABLE_GPLV3=1 \ CROSS_COMPILER_HOST=${HOST} \ SYSIMAGE_TYPE=ext2 \ ./build.sh ${TARGET} A reasonable invocation is: ENABLE_GPLV3=1 \ CROSS_COMPILER_HOST=x86_64 \ SYSIMAGE_TYPE=ext2 \ ./build.sh armv5l This will take anywhere between 30min (modern x86_64 quad core cpu with plenty of SSD) to 4hours to complete (reported for older dual-core x86 laptop and regular HDD). Now build the controller ~~~~~~~~~~~~~~~~~~~~~~~~ To build the controller, you need to have mksquashfs installed. To run the controller, you will need socat installed. To build, just: cd /path/to/aboriginal-controller make And your done. Configure YBD ~~~~~~~~~~~~~ For now ybd is hooked up to the emulator in a very nasty way, it needs to know both the path to the emulator (aboriginal system image) and it needs to know the path to the controller. Edit the file /path/to/ybd/ybd/config/ybd.conf and add the following: aboriginal-controller: '/path/to/aboriginal-controller' aboriginal-system: '/path/to/aboriginal/build/system-image-armv5l' To run multiple emulators in parallel, you should remember to also add: instances: 2 NOTE: The aboriginal system image and cross compilers dont really have to stay in the aboriginal build directory, but the cross compiler must be *beside* the system image wherever it is located. Build build-essential using YBD / Aboriginal ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Now you can run YBD in the regular way, assuming you have the correct definitions branch checked out: cd /path/to/definitions /path/to/ybd/ybd.py strata/build-essential.morph armv5l The first time you run this, startup will take at least a minute, this is because the first time you startup an emulator it will create a 1GB swap file. Subsequent YBD invocations once the swap is built will just pause for the time it takes to boot up the emulators. Logging is copied to the regular place after each build command is run, however if you are running a long lived build command and want to observe the output, you can look in the staging directory which should be somewhere like: ~/ybd/tmp/<tmpdir>/build.log I did have a 'tail -f build.log > realoutput.log' running on the host system at one point, but this was just expensive for nothing so I removed that.

7 years, 1 month

4
14
0 / 0

The process of updating definitions format

by Paul Sherwood

IIUC the process for improving the definitions format now requires a series of activities, across up to four repos. - update the spec repo - and new text (if any) to describe the improvement - update existing text (if any) that needs to change to correspond with the improvement - remove existing text (if any) that no longer applies - provide a migration to be applied to definitions, to - effect the change - update the version number in VERSION - update the changelog with a description of the change - update definitions - submit the result of applying the migration to gerrit - cause the migration to be applied to master - cause the resulting change to be tested - (optionally) update morph - make morph support the new version - optionally make morph refuse to support previous versions - run morph's tests - fix any tests which break for the new version - add new tests to deal with the new version - test the changes to morph and its tests - get the changes to morph and its tests reviewed via gerrit - once - (optionally) update ybd - make ybd support the new version - test that ybd builds the new version - test that ybd still builds old versions - submit a pull request Any ideas to simplify/improve this, or is this the best we can do?

7 years, 1 month

5
24
0 / 0

Baserock 16.13 release notes

by Pedro Alvarez

Hi all! I was doing another release today, and I thought that doing a Baserock release too wouldn't hurt, so here it is. Take this email as a release announcement, or as a release notes review, that will depend on how good they are and if nobody wants any changes: >From <http://wiki.baserock.org/releases/baserock-16.13/> # Baserock 16.13 is released! ## Definitions compatibility The Baserock reference systems are defined using version 7 of the definitions format. In 15.47, version 6 was used. The baserock-16.13 tag of definitions can be built with Morph from 15.47. The version of Morph in Baserock 16.13 can build definitions versions 7 and 8 For more information about these versions, see here: <http://wiki.baserock.org/definitions> and <https://docs.baserock.org> ## Changes since 15.47 This release is full of changes! * A lot of new components added and upgraded to the GNOME systems, and trust me, when I say "a lot", I mean it. * Some security related upgrades: - OpenSSL to 1.0.1s - Upgraded Git to 2.8.0-rc2 - Cgit to v.0.12 for security issues (TODO) - Upgraded to a recent version of glibc 2.22 branch * Added fail2ban to Trove systems, see how you can configure it here: <http://wiki.baserock.org/guides/configure-fail2ban/> * Added a generic IVI system and cluster for `x86_64` and Jetson. * Other various upgrades: - Upgraded Qt to 5.6 - Libcal to 2.0.0 - xserver to 1.18.2 - Systemd to v229 - Fuse to 2.9.4 - curl to 7.47.1 - connman 1.31 - alsa-lib and alsa-utils v1.0.29 - bluez to 5.37 - libxml2 to v2.9.3 - libcrypt to 1.5.5 - mesa to 1.11.2 - libdrm 2.4.67 - xdg-app-common to 0.4.13 - weston/wayland to 1.9.0 - dbus to 1.10.6 - gstreamer components to 1.6.3 - freetype to 2.6.2 - ... * Other bugfixes and improvements: see the Git logs of [definitions.git](http://git.baserock.org/cgit/baserock/baserock/definitio... and [morph.git](http://git.baserock.org/cgit/baserock/baserock/morph.git/log/?... for a full list. Thanks to everyone who has provided code, documentation, sponsorship or feedback for this Baserock release! # How do I get started? Start with the following page: <http://wiki.baserock.org/quick-start/> Those who are up to speed with Baserock already can make use of the 'cache.baserock.org' cache server, with the `artifact-cache-server = http://cache.baserock.org:8080/` option. All artifacts necessary to upgrade a 'build' system for `x86_32`, `x86_64` and `armv7lhf` are present in this cache. # How do I get in contact? - IRC: freenode #baserock - Public mailing list: baserock-dev(a)baserock.org - See also: <http://wiki.baserock.org/mailinglist/> If you find a bug in Baserock, we'd like to hear from you using one of the above methods. The Baserock project welcomes new participants! We hope you enjoy experimenting with Baserock and look forward to hearing about any cool things you do with Baserock. -- Pedro

7 years, 2 months

2
1
0 / 0

YBD 16.13 is released

by Paul Sherwood

The main change here is we can set config values as environment variables, as well as in conf files eg export `YBD_base=/src` is equivalent to putting `base: /src` in a config file This makes it easier to pass config to (for example) Travis and GitLab ci. NB I don't expect there to be a 16.14 release 354ad99 Merge branch 'master' of github.com:devcurmudgeon/ybd 31e02b2 Spot case where config is empty c4f4a77 Merge pull request #201 from devcurmudgeon/write-a-result 72799e1 Write cache-key to ybd.result 32dcaa7 Update .gitlab-ci to use the new install.sh 787cbce Re-spell GitLab f4444da Fix install.sh to handle all the dependencies c9fbf5d Tidyup badges 709618a Merge pull request #200 from devcurmudgeon/allow-environment-variables bf8dba5 Export YBD_mode=keys-only for initial travis run 24f8daa Add support for taking ybd config from YBD_* environent variables 7dddfb2 Add cache-keys run to .travis.yml c84b653 Merge pull request #199 from devcurmudgeon/fix-travis d4340b0 Fix travis install

7 years, 2 months

1
0
0 / 0

YBD 16.12 is released

by Paul Sherwood

Not much to see here: added a .travis.yml added build badges for Github and GitLab moved RetryException and lockfile code into app.py a symlink fix from Tristan 79bd032 Code should be tested and pass pep8 9e820f6 Fix ybd.conf in .travis.yml 1fa741a Only log sandbox dir remove if log-verbose b362dbd Move lockfile into app.py bcd34a2 Move RetryException into app.py be82feb Revert "Don't remove sandbox dir if an exception happens" d6d8cb5 Add badges to readme 5acac18 First pass at .travis.yml 47b08bd Pep8 39ca168 Don't remove sandbox dir if an exception happens 8f598a3 Merge pull request #196 from gtristan/master e387d40 utils.py: Dont ever set the timestamp of symlink targets. 646ada1 Add bootstrap-commands to defaults.conf

7 years, 2 months

1
0
0 / 0

Aboriginal YBD: Status, Roadmap, Etc...

by Tristan Van Berkom

Hi, So here is the current state of affairs for Aboriginal YBD. First some of the things I've been doing this week: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ o Fixing the virtfs.py code to handle hardlinks properly Some builds could not execute pkg-config, because that is a hardlink to ${target-triple}-pkg-config, discovered this and fixed the virtfs to handle hardlinks in tarballs properly. o Mounting /tmp in the sandbox on the emulator. The emulator uses a chroot sandbox into a directory that is a 9p virtfs share with the host. This is a cool little fs type but does not support mmap among a few other things. After making the transition to real coreutils from busybox, problems were revealed with configure scripts which do: cat > confdefs.h <<_ACEOF ... stuff ... _ACEOF These fail because with real bash and real cat from coreutils, bash fails to feed stdin with this semantic, unless it has a better filesystem at /tmp o Trying out virtio networking It had been a while I had been trying off and on to get qemu to startup with virtio-net working. After debugging the kernel, I found that CONFIG_NET_VIRTIO was being ignored, unless CONFIG_NET_CORE was first specified. This allows us to now use the virtio-net driver for the emulator to communicate with distccd, however, unfortunately there is not a very significant performance gain from this. o Improving aboriginal's ccwrap compiler frontend. After having built foundation stratum, I ran a build of the GNOME stratum just to see how far it would get. This was failing on configure checks against cpp (testing things such as whether cpp defines 'unix' or not). The aboriginal cpp was just a bash script which invokes `gcc -E "$@"`, this is however incorrect because cpp is specified to accept input from stdin as well as files. Fixed this by wrapping cpp properly in the same way that the ccwrap frontend wraps the cc and c++ compilers. o Improving the IPC Instead of using socket backends to the virtio serial ports, we discovered (thanks Rob Taylor) the 'pipe' backend, this allows us to back the guest's virtio serial ports with fifos on the host. The result is that communication is suddenly buffered, which makes things easier... BUT... the stuff is still really buggy, sometimes qemu's virtio serial ports just stop responding for some reason. I have a feeling we have to just bite the bullet and use the regular network with a bit more fancy code than trying to get it working with bash scripts over the serial port. This can probably be done with a ~100 line C program on the guest which connects to a server running on the host, so that the host runs a socket server and the guest connects to it much in the same way that distcc is working. The C program on the guest could be compiled the first time the guest boots up, and on the host we can write the server side with pythonic stuff. Summary of moving parts ~~~~~~~~~~~~~~~~~~~~~~~ It's worth saying something about the current peices involved, since I have a feeling that I might have less time to hack on this for the coming weeks, so this is a courtesy for the army of hackers who will be furthering the project while I'm not available to do it myself ;-) Aboriginal YBD ~~~~~~~~~~~~~~ https://github.com/gtristan/ybd/tree/aboriginal The 'aboriginal' branch in my downstream YBD repo. I've kept this branch relatively clean and squashed regularly, most of the changes revolve around sandbox.py and the addition of virtfs.py. Downstream Aboriginal Linux ~~~~~~~~~~~~~~~~~~~~~~~~~~~ https://github.com/gtristan/aboriginal/tree/extra-changes The 'extra-changes' branch of my downstream Aboriginal Linux repository. This branch is based on the 'dual-toolchain' branch which has a pending pull request upstream. Rob Landley is overworked, give him a bit of time, he'll get around to merging the work, then we can tickle him a bit more with the extra patches we need. Anything I've added to this repository is intended for eventual upstreaming into aboriginal proper, the extra-changes branch has some important changes to the compiler wrapper and kernel configs which we need. The aboriginal controller ~~~~~~~~~~~~~~~~~~~~~~~~~ https://github.com/gtristan/aboriginal-controller This should eventually be collapsed into Aboriginal YBD, it was originally written as a hand full of bash scripts so that I could test the setup of slaving the aboriginal guests without having to jump into integration with YBD right away. Currently Aboriginal YBD uses this controller repository to spawn the build slaves and tell them to build. Revised TODO list / Roadmap ~~~~~~~~~~~~~~~~~~~~~~~~~~~ In order of priority (I think): o Close the loop and perform a deployment with the armv5l arch Yeah we built a lot of stuff, but before continuing and building everything under the sun, lets close the loop and ensure that what we've built is deployable and runs on an actual armv5l machine. To perform a deployment, there are 2 missing links: Building a "system" ~~~~~~~~~~~~~~~~~~~ We currently cannot build a "system" because that requires extensions to run, in order to run the system-integration commands for each chunk. For this, we'll need an addition to the formalism, the system needs to be allowed to declare a stratum on which it depends to run those extensions. Of course, this is a different kind of dependency, in order to build a minimal system, we still need to have tooling staged, like python, in order to run those extension thingies. Performing a deployment ~~~~~~~~~~~~~~~~~~~~~~~ To do the cluster thingies, we need the same thing, we need to be able to declare what stratum provides all the tooling we need to perform a deployment. Once we've identified the stratum we need to have staged in order to perform a deployment, we'll need to stage that in the virtfs using virtfs.py as usual, and then we'll have to do a nested staging of the actual data to deploy. We'll also have to copy over the extension python scripts we need to run On disk this will probably look like: ~/ybd/tmp/<tmpdir>/ <-- staging of the deploy-depends ~/ybd/tmp/<tmpdir>/cluster.build/ <-- staging of the system ~/ybd/tmp/<tmpdir>/extensions/ <-- copied python extension files ~/ybd/tmp/<tmpdir>/cluster.inst/ <-- Where the image will end up With this setup, we'll be able to tell the emulator to jump in, sandbox / chroot itself into the shared <tmpdir>, and tell the emulator to run the extensions as perscribed by the definitions, creating the final disk image in /cluster.inst/ o Create a new / rock solid IPC As I described above, probably the best solution here is a simple C program using posix only APIs to act as a socket client on the guest and connect to a python-written socket server on the host. Other options a possibility, current solution not viable. o Finish off YBD integration details YBD currently needs to be setup to know exactly what emulator to use for it's build, this need tighter integration with the aboriginal images. Maybe the solution is to have prebuilt binaries for the system image and cross compilers for every supported arch somewhere on the trove, so that YBD can decide that "If I'm gonna build armv5l, I better just download the cross-compiler and system image for it". Further, we should also (as I pointed out above) remove the aboriginal controller repository from the list of moving parts, and get this all into Aboriginal YBD proper, implementing parts of it in python. The parts of this repository which cannot be converted into python are basically just some bash scripts which need to compose a squashfs that will be mounted readonly in a running aboriginal image, that part is where the aboriginal image boots up and acts as a build slave. o Along the same lines as the above, we need proper revisioning of the software going into the aboriginal builds. Currently the aboriginal builds use a downloads.sh which describes tarballs and sha1sums for each tarball... we should probably have a way to override those for build of the aboriginal image itself. This point is mostly about control on exactly what software goes into the build of an aboriginal cross compiler and image, and ensuring that we always have the tarballs required sitting in the trove and ready to reproduce the whole shabang 10 years later. o Implement additional / important arches in upstream aboriginal This is tricky stuff, but the nice thing is once it works it just continues to work. The trick here is to find the right gcc configure options to match with the right kernel configuration and the right invocations of qemu, to get that all working in symphony, so that we have a supporting guest (build slave) for a given arch. I think I've covered most of the bases here... Have a good weekend all ! Cheers, -Tristan

7 years, 2 months

2
3
0 / 0

[PATCH] Fix gall build with recent git upgrade

by Pedro Alvarez

Repo: upstream:gitano/gall Branch: baserock/pedroalvarez/git2.8.0-rc2-fix ref: ed869f92851d75115213eb12c4d97bcde2210d59 Landing: baserock/morph This patch is a workaround provided from Gitano upstream so that we can build 'gall' with recent Git Pedro Alvarez (1): Fix build with recent git (2.8.0-rc2) lib/gall/tree.lua | 5 +++-- test/test-gall.tree.lua | 2 +- 2 files changed, 4 insertions(+), 3 deletions(-) -- 2.1.4

7 years, 2 months

3
5
0 / 0

Security advisory for all Trove users

by Sam Thursfield

Hello Today we learned of a major security hole in Git, which could allow remote code-execution in the Trove server appliance. The last release of the Baserock reference systems contained Git 2.3.0, which is vulnerable. We have merged a patch to upgrade to Git 2.8.0-rc2, which is not vulnerable, and we are in the process of testing and deploying this update to our Trove instances. https://gerrit.baserock.org/2000/ https://gerrit.baserock.org/2001/ We think that there are two ways to compromise the 'lorry' or 'git' user accounts on a Trove: 1. a hostile push to a repo hosted on the Trove 2. compromising a public repo that the Trove pulls from For (1), the attacker must have SSH or HTTPS write access. If only trusted people have write access, and you didn't do anything to enable anonymous pushes, then it's nothing to worry about. For (2), it's not impossible that some open source project could turn hostile in this way. To avoid any risk, you can disable the 'lorry controller' service on a Trove until you deploy an update: systemctl stop lighttpd-lorry-controller-webapp.service If you want to avoid lots of harmless errors in the journal, disable services with 'lorry-controller' in their name. Updating a Trove doesn't require updating any infrastructure that uses it, such as a distbuild network. Please ask if you have any questions! Sam -- Sam Thursfield, Codethink Ltd. Office telephone: +44 161 236 5575

7 years, 2 months

1
0
0 / 0

Tarball lorry files and Trove behaviour on series of commits

by Stankov, Dimitar Georgiev (D.)

Hi human beings! I have some SQODs. Q-1. Imagine the following: - we have to mirror a software in Trove; - it is published as tarballs only; - all versions are there in form beautiful-software-0.1.tar.gz, beautiful-software-0.2.tar.gz, ..., beautiful-software-8.3.tar.gz; - the version we need right now is somewhere in between; - preferred is to have all the history in beautiful-software.git, instead of our version as initial snapshot. The tricky (and boring) part of the work is to update the lorry file, then commit, then push, then curl-->move-to-top, then wait Trove job, then update the file, and so on (imagine 30+ times). And here comes the SQOD - What happens if: 1. I make all subsequent changes to the lorry file and commit locally in a form one commit per version, e.g., such a diff: - "url": "http://dnl.everything.sam/beautiful-software-2.4.tar.gz" + "url": "http://dnl.everything.sam/beautiful-software-2.5.tar.gz" 2. Then push the branch once 3. Then curl-->move-to-top once 4. Then wait to see what happens What happens during step-4? Are we fetching each version or only the last? (of course, I can try that but prefer not to dirtify our precious when possible) Q-2: Is it a bad approach to create local git repository of the above software, to host it somewhere in our nerwork and to create a git-lorry file? I did that once for a software hosted at inaccessible host. It can be well automated and works but now we have to care about one more extra repository. Are there other troubles? Q-3: (not much important) Is there a command to get latest N-number of fetch-jobs of a lorry? E.g., (trove-1)# lotty-get-jobs "delta/coreutils" latest=3 http://trove-1:12765/1.0/job-html/1906912 http://trove-1:12765/1.0/job-html/1905543 http://trove-1:12765/1.0/job-html/1807291 Thanks for reading, D. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Dimitar Stankov Software Engineer Visteon Corporation 90 Tsarigradsko Shose Blvd., Capital Fort, Sofia, Bulgaria Phone: +359 29 30 36 22 (! pending update...)

7 years, 2 months

2
1
0 / 0

The infrastructure.git repo

by Sam Thursfield

Hello For a while we've been treating the 'master' branch of infrastructure.git as a description of how all our services at *.baserock.org are deployed. Then Dan sent a couple of patches to add things to the infrastructure repo that aren't going to be deployed at baserock.org (or at least, not immediately). - https://gerrit.baserock.org/1961 - https://gerrit.baserock.org/1962 Apparently the idea of these is that they are examples, which someone deploying project infrastructure can customise when deploying an actual system. This seems quite a good idea, but I think it would be messy to mix this up with the definitions of the live services running at baserock.org. So I propose: we move that into a branch named 'baserock/live', and remove the baserock.org-specific stuff from the 'master' branch. I sent a tentative branch for that here: <https://gerrit.baserock.org/1991/> And pushed 'baserock/live' to infrastructure.git: <http://git.baserock.org/cgit/baserock/baserock/infrastructure.git/log/?h=...> anyone have opinions on this? Don't worry if not :-) Sam -- Sam Thursfield, Codethink Ltd. Office telephone: +44 161 236 5575

7 years, 2 months

2
3
0 / 0

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

baserock-dev March 2016