<div dir="ltr">OK<div><br></div><div>I have a bunch of BSON documents that I convert to ByteStrings, put in a Map, and write to a socket based on the response. I noticed some high memory usage (in the GBs) so I decided to investigate. I simplified my problem into a small program that demonstrates clearer what is happening. </div>
<div><br></div><div>I wrote two versions, one with a Lazy Map and Lazy ByteStrings and one with a Strict Map and Strict ByteStrings. Both share the same memory behavior (except the lazy BS one is faster)</div><div><br></div>
<div>Here is the strict version:</div><div><br></div><div><a href="http://lpaste.net/92298">http://lpaste.net/92298</a><br></div><div><br></div><div>And here is the lazy version:</div><div><br></div><div><a href="http://lpaste.net/92299">http://lpaste.net/92299</a><br>
</div><div><br></div><div>I wrote this and compared the memory and speed behavior of ByteStrings generated by converting it from a BSON document and ByteStrings generated more purely.</div><div><br></div><div>The length of the ByteString from a BSON document is 68k and the length of the "pure" BS is 70k. </div>
<div><br></div><div>This is my weird memory behavior, both BSON and "pure" methods use the same amount of memory after inserting 10k of them (90mb)</div><div><br></div><div>However when I go to lookup a value, the BSON Map explodes the memory to over 250mb. Even if I lookup just 1 value. Looking up any number of values in the "pure BS" keeps the memory usage stable (90mb).</div>
<div><br></div><div>I am hoping someone can help me understand this. I have read some posts about Temporary ByteStrings causing memory issues but I don't know how to get started debugging.</div><div><br></div><div>--</div>
<div>Kyle Hanson</div></div>