State of Obnam development (29c3 edition)

Josh Triplett josh at joshtriplett.org
Tue Jan 1 21:10:53 GMT 2013


On Tue, Jan 01, 2013 at 06:47:23PM +0000, Lars Wirzenius wrote:
> On Tue, Jan 01, 2013 at 10:41:34AM -0800, Josh Triplett wrote:
> > I wonder: would subchunking still provide a significant performance
> > difference if obnam ran sftp round-trips asynchronously and pipelined,
> > to avoid the latency?
> 
> I expect so. If and when such sftp access is implemented, we can run
> benchmarks and if it makes sense to do so, drop subchunks.
> 
> (Network lag is not, of course, the only reason to have subchunks.)

True; the overhead of forking gpg seems like the other likely candidate,
which would go away if some sensible OpenPGP library existed to do it
in-process.

> > On a different note: does this implementation of subchunking break the
> > ability to deduplicate small files?
> 
> At the moment, yes. The impact is quite small, however, in all
> the cases I've looked at. Small files only de-duplicate if they're
> identical anyway, and there's often not large numbers of them.

More concerned about deduplication across generations, here; unless
obnam always selects the same set of small files to combine into a
single chunk, this would break deduplication of the same small files in
repeated backups.

- Josh Triplett




More information about the obnam-support mailing list