[Haskell-cafe] binary IO
joelr1 at gmail.com
Tue Dec 27 13:35:13 EST 2005
On Dec 27, 2005, at 4:52 PM, Bulat Ziganshin wrote:
> spending several weeks to random
> optimization is like buying a gold computer case trying to speed up
> the game :)
I did not spend several weeks on optimization. I went through about
25 iterations with the timeleak code and the profiler. I watched the
contribution of each function to the overall time and tried to
improve that contribution.
> sorry, but your negative results say more about you than about GHC or
> Haskell. you are working 3 months but still don't know what is a
> bottleneck in your program!
You are right in that I spent the first few weeks learning. By now I
know that pickling is the bottleneck, though. The timeleak code is
very simple. It forks X threads where each thread opens a file for
reading. Each thread reads the file record by record and sounds an
alarm if the time taken to unpickle a record exceeds, say, 3 seconds.
There's also a clear relationship between the number of threads
launched and the length of the delay so concurrency has an effect as
I'm waiting for Simon Marlow to come back to take a look at it with him.
> i say about timer thread. you created complex design using Map to find
> first event to complete, but instead you can use just 3 timer threads,
> each one serving a fixed timing interval. each thread will serve
> requests just in the order they arrive
I use just one timer thread right now, profiling does not tell me
that timers are a problem. Your solution is also not demonstrably
superior so I'm not pressed to try it.
> it's strange what your pocker server don't required the same pickler
> library as client. this 50k structure is constructed on the server
> end, i'm right?
I'm working with a different poker server now. Mine actually uses
lots of small packets that are easy to process. This particular
server is way different.
> no, type classes allow code economy comparing to combinators you use.
> you need to define only one time how to process each type. just one
> definition for all lists, one for all Word16, one for all GameType and
> so on. all the records defined by the easy code like this one for
Quite possibly. I'm looking for hard proven results at this point and
while you are making bold claims about the throughput of your
pickling library you are not willing to demonstrate this throughput
in a real-life situation. I cannot wager on your approach as it's not
proven. I'm going with a proven solution for now (Erlang) but will
come back to Haskell in a pinch as soon as I know "how to do the
This discussion is a bit of a dead-end unless you are willing to take
the next step and apply your solution to my timeleak repro case. If
you or someone else is willing to do that than I can continue my set
of profiler reports and eventually get some closure. It will happen
once either 1) the last bit of performance is squeezed out of
pickling and it's determined that threading or the scheduler is the
bottleneck or 2) things work out nicely.
I carefully kept track of what changes I made and how they affected
speed, memory consumption, etc. I maxed out on the optimizations I
have been able to make without dropping pickler combinators
altogether. I no longer have the time to continue this but since a
lot of people are concerned with Haskell speed I'm suggesting that
someone takes over with their "best of the breed" solution to the
problem and uses the same process of saving profiling reports and
tracking changes until they optimize to the max or give up.
Then we can lay out the series of profiling reports in a storyboard
of sorts, with changes from report to report described. This would
serve a great "how to write efficient Haskell" manual.
More information about the Haskell-Cafe