<div dir="ltr">ByteString itself works through a ForeignPtr, rather than a ByteArray# and it does a lot of 'inlinePerformIO' tricks to avoid the MVar implicit in a full unsafePerformIO as well as eke out some extra performance benefits over and above unsafeDupablePerformIO that I forget the details of.<div>
<br></div><div>Text, which came later on the other hand does work under ByteArray# and MutableByteArray# s in a much more principled fashion.</div><div><br></div><div>There is, however, a case for keeping some ability to work with ForeignPtr's in a ByteString style, because there are a lot of tricks that live on the ByteString stack for doing things like mmap'ing that can't work with the ByteArray# approach.</div>
<div><br></div><div><span style="color:rgb(0,0,0)">-Edward</span><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Sep 6, 2013 at 3:11 AM, Simon Peyton-Jones <span dir="ltr"><<a href="mailto:simonpj@microsoft.com" target="_blank">simonpj@microsoft.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">| The overhead<br>
| seems to be coming from inlinePerformIO (am I right here? Also, am<br>
<br>
</div>Why do you need all this 'unsafe' stuff? I think because bytestrings work internally using side effects. But why can't you use runST?<br>
<br>
If for some reason you really need IO, it's presumably not because you are really doing I/O. It's a kind of specialised application. The overheads from unsafePerformIO and friends are partly related (I think) to the bad IO-ish things that might happen inside.<br>
<br>
So perhaps you (plural) can look carefully at what you are really trying to do, and propose new primpops or whatever to support it at low cost.<br>
<br>
Don't give *me* the answers! I'm just pointing out that there is no fundamental reason for this to run slower than it would in C, and if it does it might be worth digging a bit.<br>
<span class="HOEnZb"><font color="#888888"><br>
Simon<br>
</font></span><div class="HOEnZb"><div class="h5"><br>
| -----Original Message-----<br>
| From: Libraries [mailto:<a href="mailto:libraries-bounces@haskell.org">libraries-bounces@haskell.org</a>] On Behalf Of<br>
| Artyom Kazak<br>
| Sent: 05 September 2013 21:38<br>
| To: <a href="mailto:libraries@haskell.org">libraries@haskell.org</a><br>
| Subject: Re: mapM_ for bytestring<br>
|<br>
| So, I have written several implementations of mapM_:<br>
| * bsMapM_gen — generic, works for any monad<br>
| * bsMapM_short — essentially (\f s -> mapM_ f $ unpack s)<br>
| * bsMapM_IO — hand-written version specifically for IO<br>
|<br>
| Generic and hand-written versions don’t differ much. The overhead<br>
| seems to be coming from inlinePerformIO (am I right here? Also, am<br>
| I using inlinePerformIO legitimately?), which is only needed when<br>
| we’re not in the IO monad.<br>
|<br>
| {-# SPECIALISE ... IO #-}<br>
| {-# SPECIALISE ... ST #-}<br>
| bsMapM_gen :: Monad m => (Word8 -> m a) -> ByteString -> m ()<br>
| bsMapM_gen f s = unsafePerformIO $ unsafeUseAsCStringLen s mapp<br>
| where<br>
| mapp (ptr, len) = return $ go 0<br>
| where<br>
| go i | i == len = return ()<br>
| | otherwise = let !b = inlinePerformIO $<br>
| peekByteOff ptr i<br>
| in f b >> go (i+1)<br>
|<br>
| The short version relies on fusion of `unpack` and `mapM_`. Its<br>
| advantage is that even when compiled without optimisations, it’s<br>
| still fast. (Question: would the same happen to other versions,<br>
| when put into Data.ByteString module? I suppose packages like<br>
| bytestring are compiled with optimisations, so it probably would.)<br>
|<br>
| {-# SPECIALISE ... IO #-}<br>
| {-# SPECIALISE ... ST #-}<br>
| bsMapM_shortIO :: (Word8 -> IO a) -> ByteString -> IO ()<br>
| bsMapM_shortIO f s = mapM_ f (unpack s)<br>
|<br>
| Finally, the IO-specialised version. It’s faster than generic<br>
| version (and, similarly, an ST-specialised version using<br>
| unsafeIOToST would be just as fast), so I assume a SPECIALISE pragma<br>
| involving bsMapM_IO and bsMapM_ST should be present.<br>
| (Question: are there other monads for which unsafeIOToMonad exists?)<br>
|<br>
| bsMapM_IO :: (Word8 -> IO a) -> ByteString -> IO ()<br>
| bsMapM_IO f s = unsafeUseAsCStringLen s mapp<br>
| where<br>
| mapp (ptr, len) = go 0<br>
| where<br>
| go i | i == len = return ()<br>
| | otherwise = peekByteOff ptr i >>= f >> go (i+1)<br>
|<br>
| A-and here’s a table comparing performance of all three functions.<br>
| All timings are in milliseconds.<br>
|<br>
| ghci ghc ghc -O ghc -O2<br>
| +----------+----------+----------+----------+<br>
| gen | 380 | 85 | 4.1 | 4.0 |<br>
| short | 45 | 46 | 17.2 | 16.5 |<br>
| IO | 434 | 92 | 2.4 | 2.4 |<br>
| +----------+----------+----------+----------+<br>
|<br>
| Here’s the code I used. (Question: have I messed up anything?)<br>
|<br>
| import qualified Data.ByteString as BS<br>
| import Data.Random<br>
| import System.Random<br>
| import System.IO.Unsafe<br>
| import Control.Monad<br>
| import Data.IORef<br>
| import Criterion.Main<br>
| import BSMaps<br>
|<br>
| --a bytestring consisting of 65536 random bytes<br>
| testCase = BS.pack $ fst $<br>
| flip sampleState (mkStdGen 8) $<br>
| replicateM (2^16) stdUniform<br>
|<br>
| --sums elements of a bytestring, using given mapM_<br>
| sumIO :: ((Word8 -> IO ()) -> BS.ByteString -> IO ()) -><br>
| BS.ByteString -> Word8<br>
| sumIO f s = unsafePerformIO $ do<br>
| sm <- newIORef 0<br>
| f (modifyIORef' sm . (+)) s<br>
| readIORef sm<br>
|<br>
| --runs the tests<br>
| main = defaultMain [<br>
| bench "IO" $ whnf (sumIO bsMapM_IO) testCase,<br>
| bench "short" $ whnf (sumIO bsMapM_short) testCase,<br>
| bench "gen" $ whnf (sumIO bsMapM_gen) testCase]<br>
|<br>
| Finally, if there isn’t anything wrong, what are my next steps to see<br>
| this included into next version of bytestring?<br>
|<br>
| _______________________________________________<br>
| Libraries mailing list<br>
| <a href="mailto:Libraries@haskell.org">Libraries@haskell.org</a><br>
| <a href="http://www.haskell.org/mailman/listinfo/libraries" target="_blank">http://www.haskell.org/mailman/listinfo/libraries</a><br>
_______________________________________________<br>
Libraries mailing list<br>
<a href="mailto:Libraries@haskell.org">Libraries@haskell.org</a><br>
<a href="http://www.haskell.org/mailman/listinfo/libraries" target="_blank">http://www.haskell.org/mailman/listinfo/libraries</a><br>
</div></div></blockquote></div><br></div>