Version support for datastore/Progress: Difference between revisions
Sascha silbe (talk | contribs) →2009-06-01: report on VCS evaluation progress |
Sascha silbe (talk | contribs) final benchmark run |
||
| Line 115: | Line 115: | ||
it's comparing very well with the other systems: It's on second place both for total (unweighted) runtime and | it's comparing very well with the other systems: It's on second place both for total (unweighted) runtime and | ||
final repository size, giving perfect balance between those two goals. | final repository size, giving perfect balance between those two goals. | ||
=== 2009-06-08 === | |||
Finished the planned enhancements to the benchmark. After two runs (with the timings differing by about 20% on average) of about 13h each | |||
(with only the 100 of the Project Gutenberg files) there are a few loosers, but no clear winner. Monotone comes close but doesn't seem | |||
to scale well with branches (both size and runtime) and also has the highest memory consumption. | |||
The results emphasize need to use an abstraction layer so we can switch between different VCSs (depending on what resource is | |||
scarcest) or even to a homegrown backend later. For the prototype git seems to be a good candidate as it combines average | |||
resource consumption with a rich API, including low-level access. | |||
==== Benchmark results ==== | |||
[[Image:Vcs-benchmark-run-2-total.png|thumb|Summary of the final run of the VCS benchmark]] | |||
Darcs is fast, but needs over four times the original size to store the data. | |||
[[Image:Vcs-benchmark-run-2-op-vs-size.png|thumb|Repository size summary]] | |||
[[Image:Vcs-benchmark-run-2-op-vs-time.png|thumb|Runtime summary]] | |||
[[Image:Vcs-benchmark-run-2-op-vs-size-monotone.png|thumb|Repository sizes for monotone]] | |||
[[Image:Vcs-benchmark-run-2-op-vs-time-monotone.png|thumb|Timings for monotone]] | |||
Monotone starts out very small, but incurs high costs on branches. | |||
==== Memory usage ==== | |||
Determining reliable figures for memory usage, especially peak usage, seems to require using valgrind which is - according to its | |||
documentation - slowing down program execution by a factor of 5 (which would mean 65h for the already reduced sample set). | |||
Instead I've done a few simple measurements using [http://www.gnu.org/software/acct/ BSD process accounting] for a run of the | |||
benchmark with a single file as sample set. This should at least give a very rough idea of memory consumption, especially | |||
relative to the other VCSs in the run. | |||
{| | |||
|- | |||
|arch (incl. tar) | |||
|align="right" | 1340k | |||
|- | |||
|bazaar | |||
|align="right" | 5060k | |||
|- | |||
|cvs | |||
|align="right" | 957k | |||
|- | |||
|darcs | |||
|align="right" | 7385k | |||
|- | |||
|git | |||
|align="right" | 1614k | |||
|- | |||
|mercurial | |||
|align="right" | 3501k | |||
|- | |||
|monotone | |||
|align="right" | 9865k | |||
|- | |||
|subversion | |||
|align="right" | 4121k | |||
|- | |||
|} | |||