[Haskell-cafe] Web application interface

[Michael Snoyman]

Jeremy,

What I meant is, if you use a sendfile system call to send raw files from
the disk, how does this interact with gzip compression, which clearly cannot
be used when using a sendfile call? I ask because you implied there were
significant performance gains from using sendfile.

Michael

On Fri, Jan 22, 2010 at 6:38 PM, Jeremy Shaw <jeremy at n-heptane.com> wrote:

> Hello,
>
> In happstack, there is a Writer monad which holds a list of filters which
> will be applied to the Response before sending it out. One of these filters
> is the gzip filter.
>
> The compression filters are defined here:
>
>
> http://patch-tag.com/r/mae/happstack/snapshot/current/content/pretty/happstack-server/src/Happstack/Server/Parts.hs
>
> The filters are apply when runWebT is called:
>
>
> http://patch-tag.com/r/mae/happstack/snapshot/current/content/pretty/happstack-server/src/Happstack/Server/SimpleHTTP.hs
>
> runWebT is called automatically by the top-level function, simpleHTTP, that
> people actually call in their programs.
>
>
> We do not do anything fancy to cache gzip results to the disk. We don't
> even assume you *have* a disk. I believe that functionality could be added
> as a 3rd party library with out modifying core happstack. That is how we
> would prefer to see it done so that the core is simple, and so that people
> can implement their own caching system if their needs are different.
>
> - jeremy
>
> On Jan 21, 2010, at 10:37 PM, Michael Snoyman wrote:
>
> Hey Jeremy,
>
> I was just wondering: how does Happstack deal with gzip encoding when it
> uses sendfile? I can think of a few ways (cache gziped versions to the
> disk), but was wondering if you'd already come up with a good solution. I'm
> trying to keep all these things in mind when designing WAI.
>
> Thanks,
> Michael
>
> On Thu, Jan 14, 2010 at 5:42 PM, Jeremy Shaw <jeremy at n-heptane.com> wrote:
>
>> Hello,
>>
>> Happstack is currently bundled with it's own lazy I/O based HTTP backend.
>> Ideally, we would like to split that out, and allow happstack to be used
>> with that backend, hyena, or other options.
>>
>> A primary using for using hyena would be for the benefits of
>> predictability and constant space usage that iterators bring. People do
>> actually running into the issues that come with lazy I/O, such as running
>> out of file descriptors, etc.  So, I feel like I would want to stick with
>> using iterators the whole way when using hyena, and not convert back to a
>> lazy ByteString?
>>
>> Happstack now includes support for sendfile(). This is done by adding
>> another constructor to the Response type:
>>
>> (line 94):
>>
>> http://patch-tag.com/r/mae/happstack/snapshot/current/content/pretty/happstack-server/src/Happstack/Server/HTTP/Types.hs
>>
>> Then here on line 197, we match on that case and use sendfile to send the
>> data:
>>
>>
>> http://patch-tag.com/r/mae/happstack/snapshot/current/content/pretty/happstack-server/src/Happstack/Server/HTTP/Handler.hs
>>
>> This makes it difficult for use to be compatible with WAI. We can write a
>> wrapper that converts the sendfile case to use lazy bytestrings instead, but
>> then we lose the advantages of using sendfile.
>>
>> I wonder if the 'Response' portion of WAI should support all three
>> currently used methods:
>>  - lazy I/O
>>  - Enumerator
>>  - sendFile
>>
>> I haven't really thought about how that would work..
>>
>> hyena currently includes a Network.WAI which uses ByteString:
>>
>>
>> http://hackage.haskell.org/packages/archive/hyena/0.1/doc/html/Network-Wai.html
>>
>> gotta run, sorry about any typos!
>> - jeremy
>>
>>
>>
>> On Jan 13, 2010, at 8:46 AM, Michael Snoyman wrote:
>>
>>  Hi,
>>>
>>> I recently read (again) the wiki page on a web application interface[1]
>>> for Haskell. It seems like this basically works out to Hack[2], but using an
>>> enumerator instead of lazy bytestring in the response type. Is anyone
>>> working on implementing this? If not, I would like to create the package,
>>> though I wouldn't mind some community input on some design decisions:
>>>
>>> * Hack has been fairly well-tested in the past year and I think it
>>> provides the features that people want. Therefore, I would want to model the
>>> Environment variable for WAI from Hack. I *could* just import Hack in WAI
>>> and use the exact same Environment data type. Thoughts?
>>>
>>> * If using a different data type for Environment, should I replace the
>>> String parts with ByteStrings? On the one hand, ByteStrings are the
>>> "correct" data type since the HTTP protocol does not specify a character
>>> encoding; on the other hand, Strings are easier to deal with.
>>>
>>> * It's simple to write a function to convert between a lazy bytestring
>>> and an enumerator, meaning it would be very easy to write conversion
>>> functions between Hack and WAI applications. This would make it simpler for
>>> people to use either backend.
>>>
>>> If someone else is already working on WAI, please let me know, I don't
>>> want to have duplicate implementations. The idea here is to consolidate, not
>>> split the community. I have a few Hack handlers (simpleserver, cgi, fastcgi)
>>> that I would happily convert to WAI handlers as well.
>>>
>>> Michael
>>>
>>> [1] http://www.haskell.org/haskellwiki/WebApplicationInterface
>>> [2] http://hackage.haskell.org/cgi-bin/hackage-scripts/package/hack
>>> _______________________________________________
>>> Haskell-Cafe mailing list
>>> Haskell-Cafe at haskell.org
>>> http://www.haskell.org/mailman/listinfo/haskell-cafe
>>>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.haskell.org/pipermail/haskell-cafe/attachments/20100123/755edfb1/attachment.html