Big merge and branch renaming and mess
by Lars Wirzenius
I have today merged a lot of stuff into a bunch of repositories. This
work consists of a change in the way morph builds stuff, namely, that it
unpacks the chunks it builds so later chunks can use them as build
dependencies. morph also now has a bootstrap mode, and its repo contains
a baserock-bootstrap script which can bootstrap a rudimentary native
toolchain for Baserock, based on Linux From Scratch 7.0.
I've also renamed a lot of branches, which may cause trouble for people
who merge/pull from gitorious. My apologies: I only realized afterwards
that this is going to be a bit of a mess. Please flame me if you are
inconvenienced.
I've also renamed the core stratum to foundation, since several people
asked be the difference between base (a system) and core (a stratum),
and I'm hoping this will help.
In the future, I hope I can work in smaller jumps so that the merges are
not quite so large.
We should also decide on a branch naming and merging policy at some
point, possibly soon.
11 years, 5 months
Baserock FAQ; btrfs speed
by Lars Wirzenius
Since nothing happens unless it's reported on the mailing list...
There's now an FAQ (with a single question) on the Baserock wiki:
http://wiki.baserock.org/faq/
Also, someone suggested that btrfs might be too slow for Baserock. I ran
some tests using the bonnie++ tool, on a laptop, against a spinning
disks (i.e., not on flash memory; didn't have a sufficiently large
SD card available).
fs type block write % of ext 4 block read % of ext4
ext2 44909 85 68865 116
ext3 44571 84 68782 116
ext4 52787 100 59518 100
btrfs 53118 101 54253 91
The results are satisfactory, I think.
11 years, 6 months
Debian eglibc patches
by Rob Taylor
Hi all,
I've put together a (pretty stupid) script for importing the debian eglibc
patches into our baserock repo.
It could probably do with quite a bit of extra stuff if we use it
long-term, eg.
- output results sensibly
- sensible branch handling
- checking for already applied patches.
It can be found in https://gitorious.org/baserock/baserock-debian-importer
Lars, should I push the results of this tool into a branch on
baserock-morphs/eglibc.git ?
HTH,
Rob
11 years, 6 months
Rsync and tbdiff
by Richard Maw
Trebuchet diff is the program used to make and apply patches.
The patches are transmitted to the target device and then applied by the
device to update itself.
It was pointed out that rsync has a batch mode option that allows the
same behaviour, however it has been decided that it is not suitable.
The default mode for patch creation uses the "quick check" algorithm,
which uses the heuristic of if a file has the same modification time and
size then it does not need to be updated.
However in use, it was found that this, while accurate for most files,
was not correct for all files.
Binary formats often keep the same file size and some files do not have
their modification times set correctly.
In some cases the timestamps were not identical, but had precision of
less than a second, which rsync interpreted as being the same.
To be sure that files need to be patched they should be read. This is
slow as it has to traverse an entire filesystem, but patches are only
created once, so they are allowed to be reasonably slow.
One option is to use --ignore-times, which forces all files to be
transmitted, however this makes the patches larger and the patch
creation take a long time. I had a job running all night and not finish,
while tbdiff did it in less than 10 minutes.
Another option is to use checksums with --checksum. This will read the
files and calculate if they need to be altered by comparing the
checksums. This slows down the patch creation, but to an acceptable
level.
However when deploying the patch it took 12 times longer than tbdiff.
This is presumably because it has to checksum every file that needs to
be altered. This sounds like a good idea, as it means a bad version
can't be created if the files have been corrupted, however patches are
applied to subvolumes the user shouldn't have access to, and if disk
failure is happening there's not too much that can be done anyway.
We need to be able to assume the files are correct for an acceptable
speed.
Fast deployment is important to baserock, so rsync is not suitable.
A summary of these points and some tables of the relative times are on
the wiki at http://wiki.baserock.org/Trebuchet/tbdiff
If somebody knows of a way to make rsync skip checksumming in batch
deployment then please, let us know so we can have the features of
rsync, but the speed of tbdiff.
11 years, 6 months
Debian eglibc patches
by robtaylor
Hi all,
I've put together a (pretty stupid) script for importing the debian eglibc
patches into our baserock repo.
It could probably do with quite a bit of extra stuff if we use it
long-term, eg.
- output results sensibly
- sensible branch handling
- checking for already applied patches.
It can be found in https://gitorious.org/baserock/baserock-debian-importer
Lars, should I push the results of this tool into a branch on
baserock-morphs/eglibc.git ?
HTH,
Rob
11 years, 6 months
