cabal patch to improve "cabal update" behavior.

[Duncan Coutts]

On Tue, 2012-04-17 at 10:32 +0300, Eyal Lotem wrote:
> I built a preliminary prototype of "cabal update" enhancements:
> 
> A) Using a Lazy I/O bytestring with the HTTP download, so that it can do a
> streaming-decompress on-the-fly (This gives a 30% speedup on my machine)

That's interesting. I'm surprised it's that much.

So you're decompressing and writing the decompressed .tar to disk? But
not unpacking the .tar on the fly right? Is that 30% figure just of the
time to write the uncompressed file to disk, or also of the download and
unpack? Or are we just talking about the index file here?

I guess I typically assume that it makes sense to write the compressed
file to disk, and decompresses and unpack on the fly, less disk I/O.
That said, for the index we do keep the uncompressed version on disk, so
doing the decompression on the fly there would make sense.

> B) Using the bytestring-progress library to show a useful progress report
> on the whole process. Unfortunately, due to the PVP, the version
> constraints are (as usual) over-restrictive, so they need to be relaxed.
> Until the fix is put on hackage, you have to install bytestring-progress
> from https://github.com/Peaker/bytestring-progress

Interesting. I'm a bit sceptical of it's subtle side effects. I wonder
if we can do it in a slightly nicer way.

Actually, something that would help is the new interfaces in the zlib
binding I've been working on. That'd let you push blocks in one at a
time and keep track of the original length, and calculate progress.

> C) It's quick&dirty (Proof of concept), because:
> C.1) No version constraints on bytestring-progress
> C.2) Using hSetBuffering on stdout and ANSI terminal codes, which may be
> problematic in different kinds of terminals
> C.3) The only way to disable it is setting verbosity=quiet. It probably
> deserves its own option
> 
> I hope it's a useful demonstration of the utility of this, though. It makes
> "cabal update" something I'd dread a little less to run :)

Yes, thanks.

BTW, the other way to make that better is to do an incremental index
update, and so simply download much less data. If you're interested in
hacking on that, lemme know.

> Also, I was wondering why there's no monad for the passing of Verbosity
> around to so many functions. Then I could easily add more options that go
> around to control things like whether to show progress bars...

There should be, but you actually want rather more than that otherwise
the saving of the verbosity params would be offset by rather a lot of
liftIO $ ...  What it needs is a shell monad of some kind. I've done a
few test designs. It's a major overhaul, but should be done.

Duncan