May 2016 - baserock-dev - listmaster.pepperfish.net

by Paul Sherwood

OOPS: we're in week number 19, not 20. Never mind :-) Mainly this week has been about re-writing how ybd installs contents and dependencies into sandboxes. - splitting.py now uses ybd's sanitised definitions data model, rather than leftover information from morphs. The key commit is af40460. - ybd now shows a checkbox for whether the artifact already exists or not. 32325cf Show more details on sandbox error bb09db1 Re-write install_dependencies too af40460 Fix ybd representation of splits 0ecd262 Show checkbox for cache-keys already found c6d3922 Rename get_metafile, and lose some lines b03b41d Lose some lines importing app functions by name 6460b10 Remove install_contents e5381b7 Re-write install_contents as add_contents 84630ae Move assembly function into compose 32b972a Pep8 4f67f22 Tidyup manifest handling 26fb0e9 Lose a few lines by adding 'verbose' flag 927643f Remove leading space for 'omit' mode c7297b8 Add results of splitting test 4892798 Allow current cache.py to run with original splitting e278b36 Lose a couple of lines 7a0d2f0 Shorten a couple of lines

6 years, 1 month

1
0
0 / 0

Re: Problems setting up a trove

by Baumann, Christoph (C.)

Hi! > -----Ursprüngliche Nachricht----- > Von: Baumann, Christoph (C.) > Gesendet: Donnerstag, 28. April 2016 16:30 > An: Baserock Mailing List (baserock-dev(a)baserock.org) <baserock- > dev(a)baserock.org> > Betreff: Problems setting up a trove > > Hello, > > currently I'm struggling with setting up a trove. > The building in Baserock (16.13) and also deploying it to > VirtualBox worked. > After some struggle with the correct network settings I was able to > log in and noticed that systemd failed to bring up trove-setup: > " > [...] > " > > After calling trove-setup in the login shell (root) the setup > started and after a while reported success. Then the mirroring of > the upstream trove that was configured in the build process > started. > Also the web interface came up. But there seems to be an issue with > git. The web interface reports "Repository seems to be empty" for > any repository I click on. And also 'git clone' fails with a > similar error. > But: if I use the ssh ID of the lorry user the checkout succeeds. > This also works to checkout the trove-config. But it fails to push > changes. > Further the git user has /bin/bash as login shell and no commands > for git-shell are available. I suspect that the problem above was also caused by the PATH setup (see https://listmaster.pepperfish.net/pipermail/baserock-dev-baserock.org/201...). This would also explain why the problem could be solved by switching to SSH connection. Regards, Christoph

6 years, 2 months

1
0
0 / 0

Re: git service of a trove

by Baumann, Christoph (C.)

Hello, > -----Ursprüngliche Nachricht----- > Von: Baumann, Christoph (C.) > Gesendet: Mittwoch, 4. Mai 2016 09:16 > An: Baserock Mailing List (baserock-dev(a)baserock.org) <baserock- > dev(a)baserock.org> > Betreff: git service of a trove > > Hello, > > as I wrote in earlier mails I'm working on setting up a mirroring > trove. > Meanwhile the lorry service downloaded approx. 75 GB. The git web > interface shows all the repos. > But when trying to check out a repo or click it in the web > interface it is reported as empty. > Could it be that lorry only pushes to the git if it has all the > data from all the upstream repos? > Or is there some other issue? In the webapp.log I noticed some > error messages from git stating that the attempt of creating a repo > failed because it already existed. I found the issue. The python program /usr/bin/lorry wants to push the repos as tar archives to the git service. But as PATH doesn't seem to be set properly tar isn't found (is in /bin) and thus the push process fails. I added a symlink to tar (actually to busybox) in /usr/bin and everything starts to work! Regards, Christoph

6 years, 2 months

3
2
0 / 0

git service of a trove

by Baumann, Christoph (C.)

Hello, as I wrote in earlier mails I'm working on setting up a mirroring trove. Meanwhile the lorry service downloaded approx. 75 GB. The git web interface shows all the repos. But when trying to check out a repo or click it in the web interface it is reported as empty. Could it be that lorry only pushes to the git if it has all the data from all the upstream repos? Or is there some other issue? In the webapp.log I noticed some error messages from git stating that the attempt of creating a repo failed because it already existed. Regards, Christoph

6 years, 2 months

1
0
0 / 0

YBD 16.18 is released

by Paul Sherwood

Key changes - artifact-version is now 3 (we now only hash applicable build system in each key, rather than all of them) - if max-jobs and instances not specified, ybd will set them to parallelise with no instance getting more than 9 cores - @nuxeh fixed a long-standing bug in subsystem deployment #215 - fixed #206, where absence/presence of .git could break git clones on switching protocols in aliases 9fb0a8e Cleanups in definitions.py 5e04a62 Rename definition => item 0a076c7 Rename definitions => data f7b7b9e Tidyups in splitting.py, this => component 1c9e2c3 Move get_meta functions into splitting.py ec36a33 http repo urls must end in .git e6eff37 Add splitting result from rdale's original branch 54fc057 And now fix max-jobs/instances properly 580b7da Set instances, max-jobs if user doesn't 80fadf2 Remove some unused imports/variables c8cc2d0 Code improvement from discussion on #baserock 40162ba Fix ugly indent and lose a line 92d0b42 Merge pull request #215 from nuxeh/edcragg/fix-subsystem-deployment 8a76f05 Fix subsystem deployment 0ff0e98 Revert "Parse put into yaml instead of json" 621d946 Parse put into yaml instead of json 3f722ea Draft of new test appraoches for parsing and splitting bad3653 Allow result-file to be set in config f202922 Add new 'parse-only' and 'log-timings' modes e4671d3 Improve config comment ae5bbfd First-pass at manifest creation 95fd50e Catch exceptions on run_sandbox_with_redirection 1220e80 Now get_repo_url won't break for repo=None bd04031 Maybe fix for #209 groff copy fail e09ab43 Gah... 'this' was not specified 16f0dea Merge branch 'devel' 14a2d92 artifact-version: 3 78a8a20 Log sandbox dir 4d8b5a2 Show ybd version at top of log 689dffa Exit if cache-key fails for stratum content chunk 0cb5278 Prettify setup config for easier reading 258fe7c Capitalise 'splitting' messages bdebac9 Keep going if stratum reproduction fails 6bacc95 Use os.stat(x).st_atime, no need to touch files for access time eafb0fe Fixes #207 - exit if we can't process metafile 82e7159 Improvements to concourse pipeline generator ea12b8c If mode is no-build, no need to do the hashing de329fc Some readme fixes, including adding link to c.b.o 4846017 Some readme fixes, including adding link to c.b.o 2058035 Fix spelling mistake and pep8 566c62e Fixes #206 - tested on ci.morph 18691e8 Stripping .git on (say) http://foo.bar/bip.git breaks clone 6e8f787 28792b fix for #205 was incompatible with new bottle versions 4cf58e6 Restrict upload filenames 7359cf3 [ci skip] Various README fixes 0d13769 Add comments to kbas config, set default port to 8000 28792b9 Fixes #205, but how come it was working in general? 22b7b3b Add check-overlaps: ignore, warn, exit 934fd7e Amend cache-key algorithm - per build-system

6 years, 2 months

1
0
0 / 0

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

baserock-dev May 2016