Task 15895952

Name	hadcm3n_o0zt_1940_40_008382269_1
Workunit	8533128
Created	18 Jul 2013, 14:33:19 UTC
Sent	18 Jul 2013, 16:16:01 UTC
Report deadline	17 Oct 2013, 23:43:12 UTC
Received	25 Jul 2013, 17:10:12 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1101384
Run time	6 days 20 hours 11 min 35 sec
CPU time	5 days 9 hours 0 min 2 sec
Validate state	Invalid
Credit	2,488.32
Device peak FLOPS	2.33 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> MonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7C812FD3 Engaging BOINC Windows Runtime Debugger... ****************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 07/25/13 12:06:38 Install Directory : C:\Program Files\BOINC\ Data Directory : C:\Documents and Settings\All Users\Application Data\BOINC Project Symstore : Loaded Library : C:\Program Files\BOINC\\dbghelp.dll Loaded Library : C:\Program Files\BOINC\\symsrv.dll Loaded Library : C:\Program Files\BOINC\\srcsrv.dll LoadLibraryA( C:\Program Files\BOINC\\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\hadcm3n_o0zt_1940_40_008382269;C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net ModLoad: 00400000 00b28000 C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\hadcm3n_um_6.07_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 7c900000 000b2000 C:\WINDOWS\system32\ntdll.dll (5.1.2600.6055) (-exported- Symbols Loaded) Linked PDB Filename : ntdll.pdb File Version : 5.1.2600.6055 (xpsp_sp3_gdr.101209-1647) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.6055 ModLoad: 7c800000 000f6000 C:\WINDOWS\system32\kernel32.dll (5.1.2600.6293) (-exported- Symbols Loaded) Linked PDB Filename : kernel32.pdb File Version : 5.1.2600.6293 (xpsp_sp3_gdr.121001-1622) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.6293 ModLoad: 65240000 0006d000 C:\WINDOWS\System32\SYSFER.DLL (12.1.2100.2093) (-nosymbols- Symbols Loaded) Linked PDB Filename : c:\bld_area\SEP_12.1\Output\SEPClientProtection\Bin.iru\sysfer.pdb File Version : 12.1.2100.2093 Company Name : Symantec Corporation Product Name : Symantec CMC Firewall Product Version : 12.1.2100.2093 ModLoad: 7e410000 00091000 C:\WINDOWS\system32\USER32.dll (5.1.2600.5512) (-nosymbols- Symbols Loaded) Linked PDB Filename : user32.pdb File Version : 5.1.2600.5512 (xpsp.080413-2105) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.5512 ModLoad: 77f10000 00049000 C:\WINDOWS\system32\GDI32.dll (5.1.2600.5698) (-nosymbols- Symbols Loaded) Linked PDB Filename : gdi32.pdb File Version : 5.1.2600.5698 (xpsp_sp3_gdr.081022-1932) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.5698 ModLoad: 77dd0000 0009b000 C:\WINDOWS\system32\ADVAPI32.dll (5.1.2600.5755) (-nosymbols- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 5.1.2600.5755 (xpsp_sp3_gdr.090206-1234) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.5755 ModLoad: 77e70000 00093000 C:\WINDOWS\system32\RPCRT4.dll (5.1.2600.6022) (-nosymbols- Symbols Loaded) Linked PDB Filename : rpcrt4.pdb File Version : 5.1.2600.6022 (xpsp_sp3_gdr.100813-1643) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.6022 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 28, Write: 0, Other 187 - I/O Transfers Counters - Read: 0, Write: 929, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 324836, QuotaPeakPagedPoolUsage: 324836 QuotaNonPagedPoolUsage: 2584, QuotaPeakNonPagedPoolUsage: 3712 - Virtual Memory Usage - VirtualSize: 1007865856, PeakVirtualSize: 1007865856 - Pagefile Usage - PagefileUsage: 34607104, PeakPagefileUsage: 107966464 - Working Set Size - WorkingSetSize: 4947968, PeakWorkingSetSize: 4947968, PageFaultCount: 1272 * Dump of thread ID 8368 (state: Waiting): * - Information - Status: Wait Reason: UserRequest, , Kernel Time: 4375000.000000, User Time: 0.000000, Wait Time: 42309208.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7C812FD3 - Registers - eax=1132fa04 ebx=00000001 ecx=00000000 edx=009fce98 esi=1132fa8c edi=fffffffe eip=7c812fd3 esp=1132fa00 ebp=1132fa54 cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000206 - Callstack - ChildEBP RetAddr Args to Child 1132fa54 007bfa90 e06d7363 00000001 00000003 1132fa80 kernel32!RaiseException+0x0 1132fa8c 007bbbcb 1132fa9c 00860978 0082f1b4 0085d354 hadcm3n_um_6.07_windows_intelx8!+0x0 1132faa8 00406e8c 04600000 01000000 00500000 004083ee hadcm3n_um_6.07_windows_intelx8!+0x0 1132fb10 00408130 1132fb24 000623af 1132fb24 00062311 hadcm3n_um_6.07_windows_intelx8!+0x0 1132ffc0 7c81776f 00000fa0 00480c5f 7ffdf000 e06d7363 hadcm3n_um_6.07_windows_intelx8!+0x0 1132fff0 00000000 007bde7f 00000000 10f300c4 ffffffff kernel32!RegisterWaitForInputIdle+0x0 * Dump of thread ID 9232 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 42309204.000000 - Registers - eax=00749910 ebx=00000000 ecx=00000005 edx=0000305f esi=00000000 edi=3a6eff70 eip=7c90e514 esp=3a6eff40 ebp=3a6eff98 cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246 - Callstack - ChildEBP RetAddr Args to Child 3a6eff98 7c802455 00000064 00000000 3a6effb4 00749924 ntdll!KiFastSystemCallRet+0x0 3a6effa8 00749924 00000064 3a6effec 7c80b729 00000000 kernel32!Sleep+0x0 3a6effb4 7c80b729 00000000 735c6c61 625f6d68 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 3a6effec 00000000 00749910 00000000 00000000 3ac00000 kernel32!GetModuleFileNameA+0x0 * Debug Message Dump ** * Foreground Window Data * Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7044, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7C812FD3 Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 07/25/13 12:06:42 Install Directory : C:\Program Files\BOINC\ Data Directory : C:\Documents and Settings\All Users\Application Data\BOINC Project Symstore : Loaded Library : C:\Program Files\BOINC\\dbghelp.dll Loaded Library : C:\Program Files\BOINC\\symsrv.dll Loaded Library : C:\Program Files\BOINC\\srcsrv.dll LoadLibraryA( C:\Program Files\BOINC\\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\hadcm3n_o0zt_1940_40_008382269;C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net ModLoad: 00400000 00b28000 C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\hadcm3n_um_6.07_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 7c900000 000b2000 C:\WINDOWS\system32\ntdll.dll (5.1.2600.6055) (-exported- Symbols Loaded) Linked PDB Filename : ntdll.pdb File Version : 5.1.2600.6055 (xpsp_sp3_gdr.101209-1647) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.6055 ModLoad: 7c800000 000f6000 C:\WINDOWS\system32\kernel32.dll (5.1.2600.6293) (-exported- Symbols Loaded) Linked PDB Filename : kernel32.pdb File Version : 5.1.2600.6293 (xpsp_sp3_gdr.121001-1622) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.6293 ModLoad: 65240000 0006d000 C:\WINDOWS\System32\SYSFER.DLL (12.1.2100.2093) (-exported- Symbols Loaded) Linked PDB Filename : c:\bld_area\SEP_12.1\Output\SEPClientProtection\Bin.iru\sysfer.pdb File Version : 12.1.2100.2093 Company Name : Symantec Corporation Product Name : Symantec CMC Firewall Product Version : 12.1.2100.2093 ModLoad: 7e410000 00091000 C:\WINDOWS\system32\USER32.dll (5.1.2600.5512) (-exported- Symbols Loaded) Linked PDB Filename : user32.pdb File Version : 5.1.2600.5512 (xpsp.080413-2105) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.5512 ModLoad: 77f10000 00049000 C:\WINDOWS\system32\GDI32.dll (5.1.2600.5698) (-exported- Symbols Loaded) Linked PDB Filename : gdi32.pdb File Version : 5.1.2600.5698 (xpsp_sp3_gdr.081022-1932) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.5698 ModLoad: 77dd0000 0009b000 C:\WINDOWS\system32\ADVAPI32.dll (5.1.2600.5755) (-nosymbols- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 5.1.2600.5755 (xpsp_sp3_gdr.090206-1234) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.5755 ModLoad: 77e70000 00093000 C:\WINDOWS\system32\RPCRT4.dll (5.1.2600.6022) (-nosymbols- Symbols Loaded) Linked PDB Filename : rpcrt4.pdb File Version : 5.1.2600.6022 (xpsp_sp3_gdr.100813-1643) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.6022 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 28, Write: 0, Other 187 - I/O Transfers Counters - Read: 0, Write: 251, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 324836, QuotaPeakPagedPoolUsage: 324836 QuotaNonPagedPoolUsage: 2584, QuotaPeakNonPagedPoolUsage: 3712 - Virtual Memory Usage - VirtualSize: 1007865856, PeakVirtualSize: 1007865856 - Pagefile Usage - PagefileUsage: 33763328, PeakPagefileUsage: 107122688 - Working Set Size - WorkingSetSize: 4939776, PeakWorkingSetSize: 4939776, PageFaultCount: 1270 * Dump of thread ID 620 (state: Ready): * - Information - Status: Base Priority: Above Normal, Priority: Above Normal, , Kernel Time: 4062500.000000, User Time: 156250.000000, Wait Time: 42309496.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7C812FD3 - Registers - eax=1132fa04 ebx=00000001 ecx=00000000 edx=009fce98 esi=1132fa8c edi=fffffffe eip=7c812fd3 esp=1132fa00 ebp=1132fa54 cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000206 - Callstack - ChildEBP RetAddr Args to Child 1132fa54 007bfa90 e06d7363 00000001 00000003 1132fa80 kernel32!RaiseException+0x0 1132fa8c 007bbbcb 1132fa9c 00860978 0082f1b4 0085d354 hadcm3n_um_6.07_windows_intelx8!+0x0 1132faa8 00406e8c 04600000 01000000 00500000 004083ee hadcm3n_um_6.07_windows_intelx8!+0x0 1132fb10 00408130 1132fb24 000623af 1132fb24 00062311 hadcm3n_um_6.07_windows_intelx8!+0x0 1132ffc0 7c81776f 00000fa0 00480c5f 7ffd9000 e06d7363 hadcm3n_um_6.07_windows_intelx8!+0x0 1132fff0 00000000 007bde7f 00000000 10f300c4 ffffffff kernel32!RegisterWaitForInputIdle+0x0 * Dump of thread ID 2376 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 42309496.000000 - Registers - eax=00749910 ebx=00000000 ecx=00000005 edx=0000305f esi=00000000 edi=3a6eff70 eip=7c90e514 esp=3a6eff40 ebp=3a6eff98 cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246 - Callstack - ChildEBP RetAddr Args to Child 3a6eff98 7c802455 00000064 00000000 3a6effb4 00749924 ntdll!KiFastSystemCallRet+0x0 3a6effa8 00749924 00000064 3a6effec 7c80b729 00000000 kernel32!Sleep+0x0 3a6effb4 7c80b729 00000000 735c6c61 625f6d68 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 3a6effec 00000000 00749910 00000000 00000000 3ac00000 kernel32!GetModuleFileNameA+0x0 * Debug Message Dump ** * Foreground Window Data * Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7044, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7C812FD3 Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 07/25/13 12:06:47 Install Directory : C:\Program Files\BOINC\ Data Directory : C:\Documents and Settings\All Users\Application Data\BOINC Project Symstore : Loaded Library : C:\Program Files\BOINC\\dbghelp.dll Loaded Library : C:\Program Files\BOINC\\symsrv.dll Loaded Library : C:\Program Files\BOINC\\srcsrv.dll LoadLibraryA( C:\Program Files\BOINC\\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: $\| ModLoad: 00400000 00b28000 C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\hadcm3n_um_6.07_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 7c900000 000b2000 C:\WINDOWS\system32\ntdll.dll (5.1.2600.6055) (-exported- Symbols Loaded) Linked PDB Filename : ntdll.pdb File Version : 5.1.2600.6055 (xpsp_sp3_gdr.101209-1647) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.6055 ModLoad: 7c800000 000f6000 C:\WINDOWS\system32\kernel32.dll (5.1.2600.6293) (-exported- Symbols Loaded) Linked PDB Filename : kernel32.pdb File Version : 5.1.2600.6293 (xpsp_sp3_gdr.121001-1622) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.6293 ModLoad: 65240000 0006d000 C:\WINDOWS\System32\SYSFER.DLL (12.1.2100.2093) (-exported- Symbols Loaded) Linked PDB Filename : c:\bld_area\SEP_12.1\Output\SEPClientProtection\Bin.iru\sysfer.pdb File Version : 12.1.2100.2093 Company Name : Symantec Corporation Product Name : Symantec CMC Firewall Product Version : 12.1.2100.2093 ModLoad: 7e410000 00091000 C:\WINDOWS\system32\USER32.dll (5.1.2600.5512) (-exported- Symbols Loaded) Linked PDB Filename : user32.pdb File Version : 5.1.2600.5512 (xpsp.080413-2105) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.5512 ModLoad: 77f10000 00049000 C:\WINDOWS\system32\GDI32.dll (5.1.2600.5698) (-exported- Symbols Loaded) Linked PDB Filename : gdi32.pdb File Version : 5.1.2600.5698 (xpsp_sp3_gdr.081022-1932) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.5698 ModLoad: 77dd0000 0009b000 C:\WINDOWS\system32\ADVAPI32.dll (5.1.2600.5755) (-exported- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 5.1.2600.5755 (xpsp_sp3_gdr.090206-1234) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.5755 ModLoad: 77e70000 00093000 C:\WINDOWS\system32\RPCRT4.dll (5.1.2600.6022) (-nosymbols- Symbols Loaded) Linked PDB Filename : rpcrt4.pdb File Version : 5.1.2600.6022 (xpsp_sp3_gdr.100813-1643) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.6022 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 28, Write: 0, Other 187 - I/O Transfers Counters - Read: 0, Write: 251, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 324836, QuotaPeakPagedPoolUsage: 324836 QuotaNonPagedPoolUsage: 2584, QuotaPeakNonPagedPoolUsage: 3712 - Virtual Memory Usage - VirtualSize: 1007865856, PeakVirtualSize: 1007865856 - Pagefile Usage - PagefileUsage: 33763328, PeakPagefileUsage: 107122688 - Working Set Size - WorkingSetSize: 4186112, PeakWorkingSetSize: 4186112, PageFaultCount: 1086 * Dump of thread ID 10204 (state: Waiting): * - Information - Status: Wait Reason: UserRequest, , Kernel Time: 4218750.000000, User Time: 156250.000000, Wait Time: 42309824.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7C812FD3 - Registers - eax=1132fa04 ebx=00000001 ecx=00000000 edx=009fce98 esi=1132fa8c edi=fffffffe eip=7c812fd3 esp=1132fa00 ebp=1132fa54 cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000206 - Callstack - ChildEBP RetAddr Args to Child 1132fa54 007bfa90 e06d7363 00000001 00000003 1132fa80 kernel32!RaiseException+0x0 1132fa8c 007bbbcb 1132fa9c 00860978 0082f1b4 0085d354 hadcm3n_um_6.07_windows_intelx8!+0x0 1132faa8 00406e8c 04600000 01000000 00500000 004083ee hadcm3n_um_6.07_windows_intelx8!+0x0 1132fb10 00408130 1132fb24 000623af 1132fb24 00062311 hadcm3n_um_6.07_windows_intelx8!+0x0 1132ffc0 7c81776f 00000fa0 00480c5f 7ffd9000 e06d7363 hadcm3n_um_6.07_windows_intelx8!+0x0 1132fff0 00000000 007bde7f 00000000 10f300c4 ffffffff kernel32!RegisterWaitForInputIdle+0x0 * Dump of thread ID 6200 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 42309824.000000 - Registers - eax=00749910 ebx=00000000 ecx=00000005 edx=0000305f esi=00000000 edi=3a6eff70 eip=7c90e514 esp=3a6eff40 ebp=3a6eff98 cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246 - Callstack - ChildEBP RetAddr Args to Child 3a6eff98 7c802455 00000064 00000000 3a6effb4 00749924 ntdll!KiFastSystemCallRet+0x0 3a6effa8 00749924 00000064 3a6effec 7c80b729 00000000 kernel32!Sleep+0x0 3a6effb4 7c80b729 00000000 735c6c61 625f6d68 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 3a6effec 00000000 00749910 00000000 00000000 3ac00000 kernel32!GetModuleFileNameA+0x0 * Debug Message Dump ** * Foreground Window Data * Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7044, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7C812FD3 Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 07/25/13 12:06:52 Install Directory : C:\Program Files\BOINC\ Data Directory : C:\Documents and Settings\All Users\Application Data\BOINC Project Symstore : Loaded Library : C:\Program Files\BOINC\\dbghelp.dll Loaded Library : C:\Program Files\BOINC\\symsrv.dll Loaded Library : C:\Program Files\BOINC\\srcsrv.dll LoadLibraryA( C:\Program Files\BOINC\\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\hadcm3n_o0zt_1940_40_008382269;C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net ModLoad: 00400000 00b28000 C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\hadcm3n_um_6.07_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 7c900000 000b2000 C:\WINDOWS\system32\ntdll.dll (5.1.2600.6055) (-exported- Symbols Loaded) Linked PDB Filename : ntdll.pdb File Version : 5.1.2600.6055 (xpsp_sp3_gdr.101209-1647) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.6055 ModLoad: 7c800000 000f6000 C:\WINDOWS\system32\kernel32.dll (5.1.2600.6293) (-exported- Symbols Loaded) Linked PDB Filename : kernel32.pdb File Version : 5.1.2600.6293 (xpsp_sp3_gdr.121001-1622) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.6293 ModLoad: 65240000 0006d000 C:\WINDOWS\System32\SYSFER.DLL (12.1.2100.2093) (-exported- Symbols Loaded) Linked PDB Filename : c:\bld_area\SEP_12.1\Output\SEPClientProtection\Bin.iru\sysfer.pdb File Version : 12.1.2100.2093 Company Name : Symantec Corporation Product Name : Symantec CMC Firewall Product Version : 12.1.2100.2093 ModLoad: 7e410000 00091000 C:\WINDOWS\system32\USER32.dll (5.1.2600.5512) (-exported- Symbols Loaded) Linked PDB Filename : user32.pdb File Version : 5.1.2600.5512 (xpsp.080413-2105) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.5512 ModLoad: 77f10000 00049000 C:\WINDOWS\system32\GDI32.dll (5.1.2600.5698) (-exported- Symbols Loaded) Linked PDB Filename : gdi32.pdb File Version : 5.1.2600.5698 (xpsp_sp3_gdr.081022-1932) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.5698 ModLoad: 77dd0000 0009b000 C:\WINDOWS\system32\ADVAPI32.dll (5.1.2600.5755) (-exported- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 5.1.2600.5755 (xpsp_sp3_gdr.090206-1234) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.5755 ModLoad: 77e70000 00093000 C:\WINDOWS\system32\RPCRT4.dll (5.1.2600.6022) (-exported- Symbols Loaded) Linked PDB Filename : rpcrt4.pdb File Version : 5.1.2600.6022 (xpsp_sp3_gdr.100813-1643) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.6022 ModLoad: 77fe0000 00011000 C:\WINDOWS\system32\Secur32.dll (5.1.2600.5834) (-exported- Symbols Loaded) Linked PDB Filename : secur32.pdb File Version : 5.1.2600.5834 (xpsp_sp3_gdr.090624-1305) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.5834 ModLoad: 5ed00000 000cc000 C:\WINDOWS\system32\OPENGL32.dll (5.1.2600.5512) (-exported- Symbols Loaded) Linked PDB Filename : opengl32.pdb File Version : 5.1.2600.5512 (xpsp.080413-0845) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.5512 ModLoad: 77c10000 00058000 C:\WINDOWS\system32\msvcrt.dll (7.0.2600.5512) (-exported- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.2600.5512 (xpsp.080413-2111) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 7.0.2600.5512 ModLoad: 68b20000 00020000 C:\WINDOWS\system32\GLU32.dll (5.1.2600.5512) (-exported- Symbols Loaded) Linked PDB Filename : glu32.pdb File Version : 5.1.2600.5512 (xpsp.080413-0845) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.5512 ModLoad: 73760000 0004b000 C:\WINDOWS\system32\DDRAW.dll (5.3.2600.5512) (-exported- Symbols Loaded) Linked PDB Filename : ddraw.pdb File Version : 5.03.2600.5512 (xpsp.080413-0845) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.03.2600.5512 ModLoad: 73bc0000 00006000 C:\WINDOWS\system32\DCIMAN32.dll (5.1.2600.5512) (-exported- Symbols Loaded) Linked PDB Filename : dciman32.pdb File Version : 5.1.2600.5512 (xpsp.080413-2105) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.5512 ModLoad: 76c90000 00028000 C:\WINDOWS\system32\imagehlp.dll (5.1.2600.6198) (-exported- Symbols Loaded) Linked PDB Filename : imagehlp.pdb File Version : 5.1.2600.6198 (xpsp_sp3_gdr.120229-1643) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.6198 ModLoad: 76390000 0001d000 C:\WINDOWS\system32\IMM32.DLL (5.1.2600.5512) (-exported- Symbols Loaded) Linked PDB Filename : imm32.pdb File Version : 5.1.2600.5512 (xpsp.080413-2105) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.5512 ModLoad: 003a0000 00011000 C:\WINDOWS\system32\AMINIT.dll (6.1.31.0) (-nosymbols- Symbols Loaded) Linked PDB Filename : File Version : 6, 1, 31, 0 Company Name : Altiris, Inc. Product Name : Application Metering Agent Product Version : 6, 1, 31, 0 ModLoad: 3ac00000 00115000 C:\Program Files\BOINC\dbghelp.dll (6.8.4.0) (-exported- Symbols Loaded) Linked PDB Filename : dbghelp.pdb File Version : 6.8.0004.0 (debuggers(dbg).070515-1751) Company Name : Microsoft Corporation Product Name : Debugging Tools for Windows(R) Product Version : 6.8.0004.0 ModLoad: 3ae20000 00048000 C:\Program Files\BOINC\symsrv.dll (6.8.4.0) (-exported- Symbols Loaded) Linked PDB Filename : symsrv.pdb File Version : 6.8.0004.0 (debuggers(dbg).070515-1751) Company Name : Microsoft Corporation Product Name : Debugging Tools for Windows(R) Product Version : 6.8.0004.0 ModLoad: 3ae70000 0003b000 C:\Program Files\BOINC\srcsrv.dll (6.8.4.0) (-exported- Symbols Loaded) Linked PDB Filename : srcsrv.pdb File Version : 6.8.0004.0 (debuggers(dbg).070515-1751) Company Name : Microsoft Corporation Product Name : Debugging Tools for Windows(R) Product Version : 6.8.0004.0 ModLoad: 77c00000 00008000 C:\WINDOWS\system32\version.dll (5.1.2600.5512) (-exported- Symbols Loaded) Linked PDB Filename : version.pdb File Version : 5.1.2600.5512 (xpsp.080413-2105) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.5512 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 28, Write: 0, Other 187 - I/O Transfers Counters - Read: 0, Write: 252, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 324836, QuotaPeakPagedPoolUsage: 324836 QuotaNonPagedPoolUsage: 2544, QuotaPeakNonPagedPoolUsage: 3712 - Virtual Memory Usage - VirtualSize: 991084544, PeakVirtualSize: 991084544 - Pagefile Usage - PagefileUsage: 16945152, PeakPagefileUsage: 33574912 - Working Set Size - WorkingSetSize: 4177920, PeakWorkingSetSize: 4177920, PageFaultCount: 1084 * Dump of thread ID 9676 (state: Ready): * - Information - Status: Base Priority: Above Normal, Priority: Above Normal, , Kernel Time: 4062500.000000, User Time: 0.000000, Wait Time: 42310160.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7C812FD3 - Registers - eax=1132fa08 ebx=00000001 ecx=00000000 edx=009fce98 esi=1132fa90 edi=fffffffe eip=7c812fd3 esp=1132fa04 ebp=1132fa58 cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000206 - Callstack - ChildEBP RetAddr Args to Child 1132fa58 007bfa90 e06d7363 00000001 00000003 1132fa84 kernel32!RaiseException+0x0 1132fa90 007bbbcb 1132faa0 00860978 0082f1b4 0085d354 hadcm3n_um_6.07_windows_intelx8!+0x0 1132faac 00406e7d 01000000 00500000 004083ee 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 1132fb10 00408130 1132fb24 000623af 1132fb24 00062311 hadcm3n_um_6.07_windows_intelx8!+0x0 1132ffc0 7c81776f 00000fa0 00480c5f 7ffdf000 e06d7363 hadcm3n_um_6.07_windows_intelx8!+0x0 1132fff0 00000000 007bde7f 00000000 10f300c4 ffffffff kernel32!RegisterWaitForInputIdle+0x0 * Dump of thread ID 2348 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 42310160.000000 - Registers - eax=00749910 ebx=00000000 ecx=00000005 edx=0000305f esi=00000000 edi=3a6eff70 eip=7c90e514 esp=3a6eff40 ebp=3a6eff98 cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246 - Callstack - ChildEBP RetAddr Args to Child 3a6eff98 7c802455 00000064 00000000 3a6effb4 00749924 ntdll!KiFastSystemCallRet+0x0 3a6effa8 00749924 00000064 3a6effec 7c80b729 00000000 kernel32!Sleep+0x0 3a6effb4 7c80b729 00000000 735c6c61 625f6d68 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 3a6effec 00000000 00749910 00000000 00000000 003e0050 kernel32!GetModuleFileNameA+0x0 * Debug Message Dump ** * Foreground Window Data * Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7044, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7C812FD3 Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 07/25/13 12:06:57 Install Directory : C:\Program Files\BOINC\ Data Directory : C:\Documents and Settings\All Users\Application Data\BOINC Project Symstore : Loaded Library : C:\Program Files\BOINC\\dbghelp.dll Loaded Library : C:\Program Files\BOINC\\symsrv.dll Loaded Library : C:\Program Files\BOINC\\srcsrv.dll LoadLibraryA( C:\Program Files\BOINC\\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\hadcm3n_o0zt_1940_40_008382269;C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net ModLoad: 00400000 00b28000 C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\hadcm3n_um_6.07_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 7c900000 000b2000 C:\WINDOWS\system32\ntdll.dll (5.1.2600.6055) (-exported- Symbols Loaded) Linked PDB Filename : ntdll.pdb File Version : 5.1.2600.6055 (xpsp_sp3_gdr.101209-1647) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.6055 ModLoad: 7c800000 000f6000 C:\WINDOWS\system32\kernel32.dll (5.1.2600.6293) (-exported- Symbols Loaded) Linked PDB Filename : kernel32.pdb File Version : 5.1.2600.6293 (xpsp_sp3_gdr.121001-1622) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.6293 ModLoad: 65240000 0006d000 C:\WINDOWS\System32\SYSFER.DLL (12.1.2100.2093) (-exported- Symbols Loaded) Linked PDB Filename : c:\bld_area\SEP_12.1\Output\SEPClientProtection\Bin.iru\sysfer.pdb File Version : 12.1.2100.2093 Company Name : Symantec Corporation Product Name : Symantec CMC Firewall Product Version : 12.1.2100.2093 ModLoad: 7e410000 00091000 C:\WINDOWS\system32\USER32.dll (5.1.2600.5512) (-exported- Symbols Loaded) Linked PDB Filename : user32.pdb File Version : 5.1.2600.5512 (xpsp.080413-2105) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.5512 ModLoad: 77f10000 00049000 C:\WINDOWS\system32\GDI32.dll (5.1.2600.5698) (-exported- Symbols Loaded) Linked PDB Filename : gdi32.pdb File Version : 5.1.2600.5698 (xpsp_sp3_gdr.081022-1932) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.5698 ModLoad: 77dd0000 0009b000 C:\WINDOWS\system32\ADVAPI32.dll (5.1.2600.5755) (-exported- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 5.1.2600.5755 (xpsp_sp3_gdr.090206-1234) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.5755 ModLoad: 77e70000 00093000 C:\WINDOWS\system32\RPCRT4.dll (5.1.2600.6022) (-exported- Symbols Loaded) Linked PDB Filename : rpcrt4.pdb File Version : 5.1.2600.6022 (xpsp_sp3_gdr.100813-1643) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.6022 ModLoad: 77fe0000 00011000 C:\WINDOWS\system32\Secur32.dll (5.1.2600.5834) (-exported- Symbols Loaded) Linked PDB Filename : secur32.pdb File Version : 5.1.2600.5834 (xpsp_sp3_gdr.090624-1305) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.5834 ModLoad: 5ed00000 000cc000 C:\WINDOWS\system32\OPENGL32.dll (5.1.2600.5512) (-exported- Symbols Loaded) Linked PDB Filename : opengl32.pdb File Version : 5.1.2600.5512 (xpsp.080413-0845) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.5512 ModLoad: 77c10000 00058000 C:\WINDOWS\system32\msvcrt.dll (7.0.2600.5512) (-exported- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.2600.5512 (xpsp.080413-2111) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 7.0.2600.5512 ModLoad: 68b20000 00020000 C:\WINDOWS\system32\GLU32.dll (5.1.2600.5512) (-exported- Symbols Loaded) Linked PDB Filename : glu32.pdb File Version : 5.1.2600.5512 (xpsp.080413-0845) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.5512 ModLoad: 73760000 0004b000 C:\WINDOWS\system32\DDRAW.dll (5.3.2600.5512) (-exported- Symbols Loaded) Linked PDB Filename : ddraw.pdb File Version : 5.03.2600.5512 (xpsp.080413-0845) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.03.2600.5512 ModLoad: 73bc0000 00006000 C:\WINDOWS\system32\DCIMAN32.dll (5.1.2600.5512) (-exported- Symbols Loaded) Linked PDB Filename : dciman32.pdb File Version : 5.1.2600.5512 (xpsp.080413-2105) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.5512 ModLoad: 76c90000 00028000 C:\WINDOWS\system32\imagehlp.dll (5.1.2600.6198) (-exported- Symbols Loaded) Linked PDB Filename : imagehlp.pdb File Version : 5.1.2600.6198 (xpsp_sp3_gdr.120229-1643) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.6198 ModLoad: 76390000 0001d000 C:\WINDOWS\system32\IMM32.DLL (5.1.2600.5512) (-exported- Symbols Loaded) Linked PDB Filename : imm32.pdb File Version : 5.1.2600.5512 (xpsp.080413-2105) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.5512 ModLoad: 003a0000 00011000 C:\WINDOWS\system32\AMINIT.dll (6.1.31.0) (-nosymbols- Symbols Loaded) Linked PDB Filename : File Version : 6, 1, 31, 0 Company Name : Altiris, Inc. Product Name : Application Metering Agent Product Version : 6, 1, 31, 0 ModLoad: 3ac00000 00115000 C:\Program Files\BOINC\dbghelp.dll (6.8.4.0) (-exported- Symbols Loaded) Linked PDB Filename : dbghelp.pdb File Version : 6.8.0004.0 (debuggers(dbg).070515-1751) Company Name : Microsoft Corporation Product Name : Debugging Tools for Windows(R) Product Version : 6.8.0004.0 ModLoad: 3ae20000 00048000 C:\Program Files\BOINC\symsrv.dll (6.8.4.0) (-exported- Symbols Loaded) Linked PDB Filename : symsrv.pdb File Version : 6.8.0004.0 (debuggers(dbg).070515-1751) Company Name : Microsoft Corporation Product Name : Debugging Tools for Windows(R) Product Version : 6.8.0004.0 ModLoad: 3ae70000 0003b000 C:\Program Files\BOINC\srcsrv.dll (6.8.4.0) (-exported- Symbols Loaded) Linked PDB Filename : srcsrv.pdb File Version : 6.8.0004.0 (debuggers(dbg).070515-1751) Company Name : Microsoft Corporation Product Name : Debugging Tools for Windows(R) Product Version : 6.8.0004.0 ModLoad: 77c00000 00008000 C:\WINDOWS\system32\version.dll (5.1.2600.5512) (-exported- Symbols Loaded) Linked PDB Filename : version.pdb File Version : 5.1.2600.5512 (xpsp.080413-2105) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 5.1.2600.5512 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 28, Write: 0, Other 187 - I/O Transfers Counters - Read: 0, Write: 252, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 324836, QuotaPeakPagedPoolUsage: 324836 QuotaNonPagedPoolUsage: 2544, QuotaPeakNonPagedPoolUsage: 3712 - Virtual Memory Usage - VirtualSize: 991084544, PeakVirtualSize: 991084544 - Pagefile Usage - PagefileUsage: 16945152, PeakPagefileUsage: 33574912 - Working Set Size - WorkingSetSize: 4177920, PeakWorkingSetSize: 4177920, PageFaultCount: 1084 * Dump of thread ID 8892 (state: Ready): * - Information - Status: Base Priority: Above Normal, Priority: Above Normal, , Kernel Time: 3906250.000000, User Time: 0.000000, Wait Time: 42310484.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7C812FD3 - Registers - eax=1132fa08 ebx=00000001 ecx=00000000 edx=009fce98 esi=1132fa90 edi=fffffffe eip=7c812fd3 esp=1132fa04 ebp=1132fa58 cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000206 - Callstack - ChildEBP RetAddr Args to Child 1132fa58 007bfa90 e06d7363 00000001 00000003 1132fa84 kernel32!RaiseException+0x0 1132fa90 007bbbcb 1132faa0 00860978 0082f1b4 0085d354 hadcm3n_um_6.07_windows_intelx8!+0x0 1132faac 00406e7d 01000000 00500000 004083ee 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 1132fb10 00408130 1132fb24 000623af 1132fb24 00062311 hadcm3n_um_6.07_windows_intelx8!+0x0 1132ffc0 7c81776f 00000fa0 00480c5f 7ffdb000 e06d7363 hadcm3n_um_6.07_windows_intelx8!+0x0 1132fff0 00000000 007bde7f 00000000 10f300c4 ffffffff kernel32!RegisterWaitForInputIdle+0x0 * Dump of thread ID 5288 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 42310484.000000 - Registers - eax=00749910 ebx=00000000 ecx=00000005 edx=0000305f esi=00000000 edi=3a6eff70 eip=7c90e514 esp=3a6eff40 ebp=3a6eff98 cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246 - Callstack - ChildEBP RetAddr Args to Child 3a6eff98 7c802455 00000064 00000000 3a6effb4 00749924 ntdll!KiFastSystemCallRet+0x0 3a6effa8 00749924 00000064 3a6effec 7c80b729 00000000 kernel32!Sleep+0x0 3a6effb4 7c80b729 00000000 735c6c61 625f6d68 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 3a6effec 00000000 00749910 00000000 00000000 003e0050 kernel32!GetModuleFileNameA+0x0 * Debug Message Dump ** * Foreground Window Data * Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7044, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7C812FD3 Engaging BOINC Windows Runtime Debugger... **************** BOINC Windows Runtime Debugger Version 6.13.0 Dump Timestamp : 07/25/13 12:07:03 Install Directory : C:\Program Files\BOINC\ Data Directory : C:\Documents and Settings\All Users\Application Data\BOINC Project Symstore : LoadLibraryA( C:\Program Files\BOINC\\dbghelp.dll ): GetLastError = 1455 LoadLibraryA( dbghelp.dll ): GetLastError = 1455 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 28, Write: 0, Other 187 - I/O Transfers Counters - Read: 0, Write: 252, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 324836, QuotaPeakPagedPoolUsage: 324836 QuotaNonPagedPoolUsage: 2584, QuotaPeakNonPagedPoolUsage: 3712 - Virtual Memory Usage - VirtualSize: 1007865856, PeakVirtualSize: 1007865856 - Pagefile Usage - PagefileUsage: 33763328, PeakPagefileUsage: 107122688 - Working Set Size - WorkingSetSize: 4190208, PeakWorkingSetSize: 4190208, PageFaultCount: 1087 * Dump of thread ID 9320 (state: Ready): * - Information - Status: Base Priority: Above Normal, Priority: Above Normal, , Kernel Time: 3750000.000000, User Time: 156250.000000, Wait Time: 42310812.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7C812FD3 * Dump of thread ID 1972 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 42310812.000000 * Debug Message Dump ** * Foreground Window Data *** Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7044, iMonCtr=1 Model crash detected, will try to restart... So

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
25 Jul 2013 15:23:02	1101384	15895952	hadcm3n_o0zt_1940_40_008382269_1	207,360	461,326	2.2248
23 Jul 2013 22:15:25	1101384	15895952	hadcm3n_o0zt_1940_40_008382269_1	181,440	344,856	1.9007
23 Jul 2013 21:58:18	1101384	15895952	hadcm3n_o0zt_1940_40_008382269_1	155,520	295,389	1.8994
23 Jul 2013 21:39:51	1101384	15895952	hadcm3n_o0zt_1940_40_008382269_1	129,600	249,118	1.9222
23 Jul 2013 21:18:00	1101384	15895952	hadcm3n_o0zt_1940_40_008382269_1	103,680	199,102	1.9204
23 Jul 2013 20:40:55	1101384	15895952	hadcm3n_o0zt_1940_40_008382269_1	77,760	145,975	1.8773
23 Jul 2013 20:12:31	1101384	15895952	hadcm3n_o0zt_1940_40_008382269_1	51,840	96,355	1.8587
23 Jul 2013 19:43:41	1101384	15895952	hadcm3n_o0zt_1940_40_008382269_1	25,920	48,979	1.8896