Skip to content

md5 on huge file fails with memory exhausted error #1459

Closed
@bdbaddog

Description

@bdbaddog

This issue was originally created at: 2006-09-05 08:15:33.
This issue was reported by: issues@scons.
issues@scons said at 2006-09-05 08:15:33

On a windows XP machine having 2GB of RAM (+ 2GB of swap), the MD5 signature
computation of a huge library (~1GB) fails.
This is due to a memory exhausted condition.

Moreover, the output error generated by SCons is not user-firendly:

'scons: ***'

Below is an excerpt of the logs using the --debug=stacktrace option:

scons: building associated BuildDir targets: out\NNX_rvct22_435\OCDP_UMTS_SYS
scons: ***
scons: internal stack trace:
File "C:\Dvpt_tools\Python24\scons-0.96.92\SCons\Job.py", line 110, in start
task.prepare()
File "C:\Dvpt_tools\Python24\scons-0.96.92\SCons\Taskmaster.py", line 110, in
prepare
self.exception_raise()
File "C:\Dvpt_tools\Python24\scons-0.96.92\SCons\Taskmaster.py", line 260, in
exception_raise
self.tm.exception_raise(self.exc_info())
File "C:\Dvpt_tools\Python24\scons-0.96.92\SCons\Taskmaster.py", line 493, in
next_task
task.make_ready()
File "C:\Dvpt_tools\Python24\scons-0.96.92\SCons\Script\Main.py", line 201, in
make_ready
SCons.Taskmaster.Task.make_ready(self)
File "C:\Dvpt_tools\Python24\scons-0.96.92\SCons\Taskmaster.py", line 226, in
make_ready_current
if t.disambiguate().current():
File "C:\Dvpt_tools\Python24\scons-0.96.92\SCons\Node\FS.py", line 2015, in
current
self.binfo = self.gen_binfo(calc)
File "...\SCons\Memoizer-cache_get-lambda<gen_binfo>", line 4, in
File "C:\Dvpt_tools\Python24\scons-0.96.92\SCons\Memoize.py", line 287, in
Memoizer_cache_get
rval = cdict[ckey] = apply(func, args, kw)
File "C:\Dvpt_tools\Python24\scons-0.96.92\SCons\Node_init
.py", line 677,
in gen_binfo
sourcesigs = executor.process_sources(calc_signature, self.ignore)
File "...\SCons\Memoizer-cache_get-lambda<process_sources>", line 4, in
File "C:\Dvpt_tools\Python24\scons-0.96.92\SCons\Memoize.py", line 287, in
Memoizer_cache_get
rval = cdict[ckey] = apply(func, args, kw)
File "C:\Dvpt_tools\Python24\scons-0.96.92\SCons\Executor.py", line 233, in
process_sources
return map(func, self.get_unignored_sources(ignore))
File "C:\Dvpt_tools\Python24\scons-0.96.92\SCons\Node_init_.py", line 674,
in calc_signature
return node.calc_signature(calc)
File "...\SCons\Memoizer-cache_get_one-lambda<calc_signature>", line 3, in

File "C:\Dvpt_tools\Python24\scons-0.96.92\SCons\Memoize.py", line 347, in
Memoizer_cache_get_one
rval = cdict[ckey] = func(self, arg)
File "C:\Dvpt_tools\Python24\scons-0.96.92\SCons\Node_init_.py", line 626,
in calc_signature
return self.get_csig(calc)
File "C:\Dvpt_tools\Python24\scons-0.96.92\SCons\Node\FS.py", line 1986, in
get_csig
csig = calc.module.signature(self)
File "C:\Dvpt_tools\Python24\scons-0.96.92\SCons\Sig\MD5.py", line 96, in
signature
return new_md5(gc()).hexdigest()
File "C:\Dvpt_tools\Python24\scons-0.96.92\SCons\Node\FS.py", line 1729, in
get_contents
return open(self.rfile().abspath, "rb").read()
scons: building terminated because of errors.

stevenknight said at 2008-07-01 11:30:43

Target milestone and priority per bug party triage.
Reassigning to Ludwig.

pankrat said at 2008-07-29 08:48:55

That's a different symptom of the same problem. Applying the patch of issue 1646
should also fix this problem.

*** This issue has been marked as a duplicate of 1646 ***

pankrat said this issue is a duplicate of #1646 at 2008-07-29 08:48:55.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions