[GHC] #9647: allocation of 10790760 bytes too large
GHC
ghc-devs at haskell.org
Mon Sep 29 19:12:10 UTC 2014
#9647: allocation of 10790760 bytes too large
-------------------------------------+-------------------------------------
Reporter: mirko | Owner:
Type: bug | Status: new
Priority: normal | Milestone:
Component: Compiler | Version: 7.8.3
Resolution: | Keywords: run time
Operating System: Windows | insufficient memory
Type of failure: Incorrect | Architecture: x86_64 (amd64)
warning at compile-time | Difficulty: Unknown
Test Case: | Blocked By:
Blocking: | Related Tickets:
Differential Revisions: |
-------------------------------------+-------------------------------------
Old description:
> I got an error message and invitation to submit a bug
> report. So here is the error message first, followed
> by my description of what happened.
>
> C:\Users\Owner\Documents\hs>ghc -o void void.hs -rtsopts
> [1 of 1] Compiling Main ( void.hs, void.o )
> Linking void.exe ...
>
> C:\Users\Owner\Documents\hs>void +RTS -K400M -RTS > void.txt
> void: internal error: allocation of 10790760 bytes too large
> (GHC should have complained at compile-time)
> (GHC version 7.8.3 for x86_64_unknown_mingw32)
> Please report this as a GHC bug:
> http://www.haskell.org/ghc/reportabug
>
> C:\Users\Owner\Documents\hs>void +RTS -K40000M -RTS > void.txt
> void: internal error: allocation of 10790760 bytes too large
> (GHC should have complained at compile-time)
> (GHC version 7.8.3 for x86_64_unknown_mingw32)
> Please report this as a GHC bug:
> http://www.haskell.org/ghc/reportabug
>
> Description of what happened.
> I gave my program really large numbers.
> It contains the following line:
>
> rr = (of40)*(r250^3041)
>
> In the above of40 is (if I remember correctly) the product
> of the first 40000 odd primes. I computed it with reduce
> (a computer algebra system) which saved the result (as an
> explicitly written out number in decimal notation), which
> I copied and inserted in my hs file, initializing of40 with
> its value. In the file it takes 2457 rows, 1024 cols, so
> it appears to be about 2 515 968 (that is two million+)
> decimal digits. As seen it is multiplied by r250^3041,
> here r250 is a 250-digit number so r250^3041 is about
> 760 250 (seven hundred sixty thousand+) decimal digits,
> so the product rr = (of40)*(r250^3041) seems to be more
> than 3 000 000 (three million+) decimal digits.
>
> I tried to compile it yesterday, I did get an error
> message from the compiler: I do not remember exactly
> what it was, I may try to reproduce it later and include
> it, but it was something along the lines of insufficient
> amount of memory. I run a Hewlett-Packard HPE 500y,
> upgraded to 16 Gb RAM (the max it would take), Windows 7,
> AMD Phenom(tm) II X6 1045T Processor 2.70 GHz (6 core).
> So yesterday after I got the first error message I increased
> the paging file size from automatic to 32768 Mb initial, and
> 65536 Mb max. Then tried to compile again, the compiler
> gradually took more and more memory (I was watching the performance
> using the Task Manager) so it gradually went up to using
> all 16 Gb of RAM, and stayed there for I think at least 40 min,
> could have been more than an hour (or even 2 hours, sorry
> can't quite remember). At some point it seemed the computer
> froze, the task manager stopped showing any activity, but I
> did not attempt to immediately restart it, the mouse would not
> respond anyway, and in a few minutes it fixed itself, it
> seemed to continue compiling, after which it eventually
> gave up and complained about insufficient amount
> of memory again. So this morning I increased
> the paging file to initial size 49152 Mb,
> max size 196608 Mb (I don't know if such big numbers
> make sense but I tried). I tried to compile again
> starting around 9am, left my apt at around 4pm and
> the compiler had not yet finished, and came back after
> midnight, the compiler had finished. It had produced
> a big exe file (I see now, around 9 pm, so it must
> have kept compiling for around 12 hours):
> 09/27/2014 09:38 PM 20,602,960 void.exe
> AVG (antivirus) claimed that this file was a threat and
> refused to let me run it, so I had to disable AVG,
> after which I got the error messages as already
> enclosed at the beginning of this description.
> The Paging file reports as: Recommended 24574 Mb,
> Currently allocated: 163765 Mb.
>
> Note I gave the options -rtsopts to the compiler,
> I do not remember anymore what they mean, and whether
> they are valid (or perhaps not used anymore), I got
> them from a webpage perhaps a couple of years ago
> and at that time they made sense, so I just kept using them.
> I also gave the option +RTS -K400M -RTS when I tried
> to execute the compiled program, and then
> +RTS -K40000M -RTS, both versions generated the error
> messages shown above. I tend to believe that giving
> Haskell such large numbers inside the .hs file explains
> all of the above problems, it might even be a bit
> surprising it went so far before it finally gave up.
> But I do not really know, so let me just report it.
>
> So I changed the paging file back to automatic, and
> I am compiling the same file (after renaming it) to
> try to reproduce the error message that I got yesterday
> (or rather the other day, now being after midnight).
> It has been compiling for about 4 min so far, nothing
> really visible, memory usage stays at 1.28 Gb (where
> it was before I started compiling). I don't quite remember
> what happened the first time when I tried to compile,
> I think I got the error message (about insufficient
> amount of memory) immediately, or almost immediately.
> This does not happen now, I wonder if the computer
> took into account that I changed the paging file
> back to automatic, I may need to restart to make sure.
> Well, ok, 13 min after it started compiling, the memory
> usage started going up gradually. It goes from e.g.
> 13.9 to 14 Gb in about a second or two. So in about
> 2 min it went from 1.28 to 15.9 Gb, and now stays there
> (I do not know how to interpret all that Task Manager shows me).
> I may not be able to reproduce the first error message:
> At that time I was running 5 copies of reduce, each
> busy multiplying numbers for some hours. Right now
> I am only running the GHC by itself, so no other load.
> It is running for about 23 min now, the last 6 of which
> the memory usage stays at 15.9 Gb. My computer may
> have learned to increase the paging file on its own.
> The computer refuses to give up this time, but I can't
> wait, going to bed is long overdue, if it generates an
> error message I'll copy it in the morning. Well, memory
> usage decreased a bit to 15.1, this happened after it stayed
> at 15.9 for about 15 min. It I remember correctly what had
> happened the other day, it will increase again, and then give
> an error message. It's at 15.3 Gb for about 3 min now. On the
> other hand the processor is doing almost nothing, going between
> 0% and 1%. Memory 15.4, 15.8, processor went up to 16% to 17%
> (which is = 1 out of 6 cores),
> memory 15.9, this is about 40 min after it started compiling
> and about 23 min after it went up to 15.9 Gb (15.8 presently).
> I should perhaps be patient and wait for the compiler to give up
> and give me an error message that I could copy, but I do not have
> the patience to do that: Again the original error message was
> something to the effect that the compiler refused to work
> because of insufficient amount of memory. I kept trying to
> compile, so apparently the compiler tried to be nice to me,
> and eventually compiled my program in 9 hours, after which
> when I tried to execute it, I was invited to write this report.
> It keeps compiling right now, at 16% (1 of 6 cores) processor,
> and 15.9 Gb memory usage, for about 47 min (30 min at 15.9).
>
> P.S. Thanks for Richard Eisenberg and Herbert Valerio Riedel
> for replying to my email. I initially had problems, trying
> to register and submit a new bug report, the BotScout
> stopped me, and it was too late at night to try to figure it,
> so I just emailed Richard in my frustration. Now, BotScout
> may not be the culprit: I did not read carefully what
> it asked me to do, and it was something like
> Evaluate `(!!6) ...'
> My knowledge of Haskell is rather superficial (I use it since it
> compiles and is relatively fast, and does not need an extra library
> to work with large numbers), so I have to often consult the
> tutorial for simple constructs. I have had some previous
> experience with Common Lisp (or Scheme, a course taken 12
> years ago) and in Lisp there is a command "evaluate", and
> also single quotation marks are used within the language
> occasionally. So, I just copied the entire line
> Evaluate `(!!6) (whatever it was)'
> and pasted it in the interpreter which gave me some
> nonsense which I sent back to BotScout. BotScout then
> gave me a second chance, and I do not quite remember
> if I tried to be more careful, perhaps not, so BotScout
> wouldn't even let me try anymore after that (even if
> I started from scratch ... perhaps it remembered me).
> After that happened I copied only the (!!6), etc, and
> the my GHC interpreter did evaluate something meaningful
> (a single number, no error message). I just didn't want
> to spend the time to think that late at night, and
> at first it seemed to me that the word Evaluate
> might have been part of a Haskell expression.
> I question if the above (perhaps too lengthy) report
> is of any use, but again, I follow the instruction,
> if in doubt, submit it. Thank you again!
New description:
Here is the error message first, followed
by my description of what happened.
hs> ghc -o void void.hs -rtsopts
[1 of 1] Compiling Main ( void.hs, void.o )
Linking void.exe ...
hs> void +RTS -K400M -RTS > void.txt
void: internal error: allocation of 10790760 bytes too large
(GHC should have complained at compile-time)
(GHC version 7.8.3 for x86_64_unknown_mingw32)
Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug
hs> void +RTS -K40000M -RTS > void.txt
void: internal error: allocation of 10790760 bytes too large
(GHC should have complained at compile-time)
(GHC version 7.8.3 for x86_64_unknown_mingw32)
Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug
Description of what happened:
I gave my program really large numbers.
It contains the following line:
rr = (of40)*(r3041)
where of40 is an integer which has about 2 515 968
(that is two million+) decimal digits (explicitly
written out and initialized in the .hs file).
In addition, of40 is multiplied by (r to the power or 3041),
where r is a 250-digit number so (r to the power or 3041)
has about 760 250 (seven hundred sixty thousand+) decimal digits.
This makes the product
rr = (of40)*(r3041)
to have more than 3 000 000 (three million+) decimal digits.
I tried to compile it three times, twice unsuccessful
(insufficient amount of memory during compilation),
and the third time successful, after 12 hours work by
the compiler, and me increasing the paging file to
initial size 49152 Mb, max size 196608 Mb.
(At the end of compilation the Paging file reported as:
Recommended 24574 Mb, Currently allocated: 163765 Mb.)
I run a Hewlett-Packard HPE 500y, upgraded to 16 Gb RAM,
Windows 7, AMD Phenom(tm) II X6 1045T Processor 2.70 GHz (6 core).
The compiler produced a big exe file:
09/27/2014 09:38 PM 20,602,960 void.exe
When I tried to run the exe file I got the error messages
as already enclosed at the beginning of this description.
Most of the time during compilation the computer used
all of the available memory.
Note that I gave the options -rtsopts to the compiler,
and the option +RTS -K400M -RTS when I tried
to execute the compiled program (I believe this is
some kind of a request for more memory), and one more
attempt with +RTS -K40000M -RTS, both versions generated
the error messages shown above.
P.S. Thanks to Richard Eisenberg and Herbert Valerio Riedel
for replying to my email, after BotScout kept me from
submitting my report. Apparently I passed the
evaluation-captcha (on my second attempt) but
nevertheless BotScout insisted I was spam.
--
Comment (by mirko):
Here is the error message first, followed
by my description of what happened.
hs> ghc -o void void.hs -rtsopts
[1 of 1] Compiling Main ( void.hs, void.o )
Linking void.exe ...
hs> void +RTS -K400M -RTS > void.txt
void: internal error: allocation of 10790760 bytes too large
(GHC should have complained at compile-time)
(GHC version 7.8.3 for x86_64_unknown_mingw32)
Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug
hs> void +RTS -K40000M -RTS > void.txt
void: internal error: allocation of 10790760 bytes too large
(GHC should have complained at compile-time)
(GHC version 7.8.3 for x86_64_unknown_mingw32)
Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug
Description of what happened:
I gave my program really large numbers.
It contains the following line:
rr = (of40)*(r^3041)
where of40 is an integer which has about 2 515 968
(that is two million+) decimal digits (explicitly
written out and initialized in the .hs file).
In addition, of40 is multiplied to (r to the power or 3041),
where r is a 250-digit number so (r to the power or 3041)
has about 760 250 (seven hundred sixty thousand+) decimal digits.
This makes the product
rr = (of40)*(r^3041)
to have more than 3 000 000 (three million+) decimal digits.
I tried to compile it three times, twice unsuccessful
(insufficient amount of memory during compilation),
and the third time successful, after 12 hours work by
the compiler, and me increasing the paging file to
initial size 49152 Mb, max size 196608 Mb.
(At the end of compilation the Paging file reported as:
Recommended 24574 Mb, Currently allocated: 163765 Mb.)
I run a Hewlett-Packard HPE 500y, upgraded to 16 Gb RAM,
Windows 7, AMD Phenom(tm) II X6 1045T Processor 2.70 GHz (6 core).
The compiler produced a big exe file:
09/27/2014 09:38 PM 20,602,960 void.exe
When I tried to run the exe file I got the error messages
as already enclosed at the beginning of this description.
Most of the time during compilation the computer used
all of the available memory.
Note that I gave the options -rtsopts to the compiler,
and the option +RTS -K400M -RTS when I tried
to execute the compiled program (I believe this is
some kind of a request for more memory), and one more
attempt with +RTS -K40000M -RTS, both versions generated
the error messages shown above.
P.S. Thanks to Richard Eisenberg and Herbert Valerio Riedel
for replying to my email, after BotScout kept me from
submitting my report. Apparently I passed the
evaluation-captcha (on my second attempt) but
nevertheless BotScout insisted I was spam.
--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/9647#comment:3>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler
More information about the ghc-tickets
mailing list