344
edits
Changes
Version support for datastore/Progress (view source)
Revision as of 15:23, 9 June 2009
, 15:23, 9 June 2009final benchmark run
it's comparing very well with the other systems: It's on second place both for total (unweighted) runtime and
it's comparing very well with the other systems: It's on second place both for total (unweighted) runtime and
final repository size, giving perfect balance between those two goals.
final repository size, giving perfect balance between those two goals.
=== 2009-06-08 ===
Finished the planned enhancements to the benchmark. After two runs (with the timings differing by about 20% on average) of about 13h each
(with only the 100 of the Project Gutenberg files) there are a few loosers, but no clear winner. Monotone comes close but doesn't seem
to scale well with branches (both size and runtime) and also has the highest memory consumption.
The results emphasize need to use an abstraction layer so we can switch between different VCSs (depending on what resource is
scarcest) or even to a homegrown backend later. For the prototype git seems to be a good candidate as it combines average
resource consumption with a rich API, including low-level access.
==== Benchmark results ====
[[Image:Vcs-benchmark-run-2-total.png|thumb|Summary of the final run of the VCS benchmark]]
Darcs is fast, but needs over four times the original size to store the data.
[[Image:Vcs-benchmark-run-2-op-vs-size.png|thumb|Repository size summary]]
[[Image:Vcs-benchmark-run-2-op-vs-time.png|thumb|Runtime summary]]
[[Image:Vcs-benchmark-run-2-op-vs-size-monotone.png|thumb|Repository sizes for monotone]]
[[Image:Vcs-benchmark-run-2-op-vs-time-monotone.png|thumb|Timings for monotone]]
Monotone starts out very small, but incurs high costs on branches.
==== Memory usage ====
Determining reliable figures for memory usage, especially peak usage, seems to require using valgrind which is - according to its
documentation - slowing down program execution by a factor of 5 (which would mean 65h for the already reduced sample set).
Instead I've done a few simple measurements using [http://www.gnu.org/software/acct/ BSD process accounting] for a run of the
benchmark with a single file as sample set. This should at least give a very rough idea of memory consumption, especially
relative to the other VCSs in the run.
{|
|-
|arch (incl. tar)
|align="right" | 1340k
|-
|bazaar
|align="right" | 5060k
|-
|cvs
|align="right" | 957k
|-
|darcs
|align="right" | 7385k
|-
|git
|align="right" | 1614k
|-
|mercurial
|align="right" | 3501k
|-
|monotone
|align="right" | 9865k
|-
|subversion
|align="right" | 4121k
|-
|}