Ugarit performance (by alaric)
Ugarit was once renowned for its poor import performance, and rightly so. However, it's a lot faster these days - not, sadly, due to amazing optimisation work on my part, but because Thomas Hintz made write-u8vector! faster. It's not released yet, but will be in Chicken 4.10.
There's still work to be done, though. In my experiments with archival mode, I imported 9GiB into an archive in:
real 24m14.822s user 17m51.485s sys 1m59.920s
Writing an uncompressed tarball of the same 9GiB took:
real 8m49.931s user 0m1.076s sys 1m1.315s
That's a factor of 3. Ugarit spends four minutes waiting for I/O while tar spent eight minutes, which is puzzling, but Ugarit spent seventeen minutes of CPU time while tar spent one second; this will be down to the fact that Ugarit still copies each byte of the file several times between reading it in and writing it out, and I know how to fix that!
There will be some unavoidable architectural cost in the fact that Ugarit will always use at least two processes - a frontend and a backend, with the data sent over a pipe between them - but I think there's a lot I can reduce first. Onwards and downwards!
Wed 22nd Apr 2015 11:38 pm 
Subscribe
By Dakota, Fri 24th Apr 2015 @ 8:15 pm
Awesome!
By @ndy, Mon 11th May 2015 @ 10:28 am
Maybe the pipe can be zero-copy on operating systems that support splice?
How much do you thrash the GC whilst doing those copies?