climateprediction.net home page
Task 15475414

Task 15475414

Name hadcm3n_zi2i_1920_40_008245016_2
Workunit 8400140
Created 13 Dec 2012, 20:51:44 UTC
Sent 13 Dec 2012, 20:52:17 UTC
Report deadline 15 Mar 2013, 4:19:28 UTC
Received 22 Dec 2012, 20:09:06 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1278542
Run time 8 days 14 hours 33 min 30 sec
CPU time 7 days 22 hours 17 min 48 sec
Validate state Invalid
Credit 4,354.56
Device peak FLOPS 2.27 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
windows_intelx86
Stderr 
<core_client_version>7.0.28</core_client_version>
<![CDATA[
<message>
The device does not recognize the command. (0x16) - exit code 22 (0x16)
</message>
<stderr_txt>
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
BUFFOUT: C I/O Error - Return code = 32

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    
Signal 11 received, exiting...
Called boinc_finish
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4888, iMonCtr=1
Model crash detected, will try to restart...
Signal 11 received, exiting...
Called boinc_finish
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4888, iMonCtr=1
Model crash detected, will try to restart...
Signal 11 received, exiting...
Called boinc_finish
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4888, iMonCtr=1
Model crash detected, will try to restart...
Signal 11 received, exiting...
Called boinc_finish
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4888, iMonCtr=1
Model crash detected, will try to restart...


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7535C41F

Engaging BOINC Windows Runtime Debugger...



********************


BOINC Windows Runtime Debugger Version 6.13.0


Dump Timestamp    : 12/22/12 19:44:30
Install Directory : 
Data Directory    : C:\ProgramData\BOINC
Project Symstore  : 
LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126
Loaded Library    : dbghelp.dll
LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126
Loaded Library    : version.dll
Debugger Engine   : 4.0.5.0
Symbol Search Path: C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_zi2i_1920_40_008245016;C:\ProgramData\BOINC\projects\climateprediction.net


ModLoad: 00400000 00b28000 C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_um_6.07_windows_intelx86.exe (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 76ed0000 00180000 C:\Windows\SysWOW64\ntdll.dll (6.1.7601.17725) (-exported- Symbols Loaded)
    Linked PDB Filename   : wntdll.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 76500000 00110000 C:\Windows\syswow64\kernel32.dll (6.1.7601.17965) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernel32.pdb
    File Version          : 6.1.7601.17965 (win7sp1_gdr.121004-0333)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17965

ModLoad: 75350000 00047000 C:\Windows\syswow64\KERNELBASE.dll (6.1.7601.17965) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernelbase.pdb
    File Version          : 6.1.7601.17965 (win7sp1_gdr.121004-0333)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17965

ModLoad: 747a0000 00100000 C:\Windows\syswow64\USER32.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : wuser32.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17514

ModLoad: 746e0000 00090000 C:\Windows\syswow64\GDI32.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : wgdi32.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17514

ModLoad: 75330000 0000a000 C:\Windows\syswow64\LPK.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : wlpk.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 75290000 0009d000 C:\Windows\syswow64\USP10.dll (1.626.7601.17561) (-exported- Symbols Loaded)
    Linked PDB Filename   : usp10.pdb
    File Version          : 1.0626.7601.17561 (win7sp1_gdr.110216-1502)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft(R) Uniscribe Unicode script processor
    Product Version       : 1.0626.7601.17561

ModLoad: 75020000 000ac000 C:\Windows\syswow64\msvcrt.dll (7.0.7601.17744) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcrt.pdb
    File Version          : 7.0.7601.17744 (win7sp1_gdr.111215-1535)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 7.0.7601.17744

ModLoad: 748a0000 000a0000 C:\Windows\syswow64\ADVAPI32.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : advapi32.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 74a60000 00019000 C:\Windows\SysWOW64\sechost.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : sechost.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 74c20000 000f0000 C:\Windows\syswow64\RPCRT4.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : wrpcrt4.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 745b0000 00060000 C:\Windows\syswow64\SspiCli.dll (6.1.7601.17940) (-exported- Symbols Loaded)
    Linked PDB Filename   : wsspicli.pdb
    File Version          : 6.1.7601.17940 (win7sp1_gdr.120824-0334)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17940

ModLoad: 745a0000 0000c000 C:\Windows\syswow64\CRYPTBASE.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : cryptbase.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 6ea70000 000c8000 C:\Windows\system32\OPENGL32.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : opengl32.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 6ed50000 00022000 C:\Windows\system32\GLU32.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : glu32.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 6e980000 000e7000 C:\Windows\system32\DDRAW.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : ddraw.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 6ff40000 00006000 C:\Windows\system32\DCIMAN32.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : dciman32.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 753a0000 0019d000 C:\Windows\syswow64\SETUPAPI.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : setupapi.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 74a80000 00027000 C:\Windows\syswow64\CFGMGR32.dll (6.1.7601.17621) (-exported- Symbols Loaded)
    Linked PDB Filename   : cfgmgr32.pdb
    File Version          : 6.1.7601.17621 (win7sp1_gdr.110523-2108)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17621

Get Product Name Failed.
ModLoad: 751e0000 0008f000 C:\Windows\syswow64\OLEAUT32.dll (6.1.7601.17676) (-exported- Symbols Loaded)
    Linked PDB Filename   : oleaut32.pdb
    File Version          : 6.1.7601.17676
    Company Name          : Microsoft Corporation
    Product Name          : 
    Product Version       : 6.1.7601.17676

ModLoad: 75590000 0015c000 C:\Windows\syswow64\ole32.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : ole32.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 75270000 00012000 C:\Windows\syswow64\DEVOBJ.dll (6.1.7601.17621) (-exported- Symbols Loaded)
    Linked PDB Filename   : devobj.pdb
    File Version          : 6.1.7601.17621 (win7sp1_gdr.110523-2108)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17621

ModLoad: 73160000 00013000 C:\Windows\system32\dwmapi.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : dwmapi.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 74770000 0002a000 C:\Windows\syswow64\imagehlp.dll (6.1.7601.17787) (-exported- Symbols Loaded)
    Linked PDB Filename   : imagehlp.pdb
    File Version          : 6.1.7601.17787 (win7sp1_gdr.120229-1502)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17787

ModLoad: 74fc0000 00060000 C:\Windows\system32\IMM32.DLL (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : wimm32.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17514

ModLoad: 74610000 000cc000 C:\Windows\syswow64\MSCTF.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : msctf.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 71640000 000eb000 C:\Windows\system32\dbghelp.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : dbghelp.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17514

ModLoad: 741c0000 00009000 C:\Windows\system32\version.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : version.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385



*** Dump of the Process Statistics: ***

- I/O Operations Counters -
Read: 0, Write: 0, Other 0

- I/O Transfers Counters -
Read: 0, Write: 0, Other 0

- Paged Pool Usage -
QuotaPagedPoolUsage: 0, QuotaPeakPagedPoolUsage: 0
QuotaNonPagedPoolUsage: 0, QuotaPeakNonPagedPoolUsage: 0

- Virtual Memory Usage -
VirtualSize: 0, PeakVirtualSize: 0

- Pagefile Usage -
PagefileUsage: 0, PeakPagefileUsage: 0

- Working Set Size -
WorkingSetSize: 0, PeakWorkingSetSize: 0, PageFaultCount: 0

*** Dump of thread ID 4456 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7535C41F

- Registers -
eax=1132f9c4 ebx=00000001 ecx=00000003 edx=00000000 esi=009fca88 edi=fffffffe
eip=7535c41f esp=1132f9c4 ebp=1132fa14
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000212

- Callstack -
ChildEBP RetAddr  Args to Child
1132fa14 007bfa90 e06d7363 00000001 00000003 1132fa40 KERNELBASE!RaiseException+0x0 
1132fa4c 007bbbcb 1132fa5c 00860978 0082f1b4 0085d354 hadcm3n_um_6.07_windows_intelx8!+0x0 
1132fa68 00406e8c 04600000 01000000 00500000 004083ee hadcm3n_um_6.07_windows_intelx8!+0x0 
1132fad8 00408130 1132faec 114b31e9 1132faec 114b3171 hadcm3n_um_6.07_windows_intelx8!+0x0 
1132ff88 765133aa 7efde000 1132ffd4 76f09ef2 7efde000 hadcm3n_um_6.07_windows_intelx8!+0x0 
1132ff94 76f09ef2 7efde000 63bcc16d 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 
1132ffd4 76f09ec5 007bde7f 7efde000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 
1132ffec 00000000 007bde7f 7efde000 00000000 38fc9330 ntdll!RtlInitializeExceptionChain+0x0 

*** Dump of thread ID 3352 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Registers -
eax=00749910 ebx=00000000 ecx=00000000 edx=00000000 esi=33adff48 edi=00000000
eip=76eefd71 esp=33adff04 ebp=33adff6c
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
33adff6c 75364498 00000064 00000000 33adff88 00749924 ntdll!ZwDelayExecution+0x0 
33adff7c 00749924 00000064 33adff94 765133aa 00000000 KERNELBASE!Sleep+0x0 
33adff88 765133aa 00000000 33adffd4 76f09ef2 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 
33adff94 76f09ef2 00000000 4123c16d 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 
33adffd4 76f09ec5 00749910 00000000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 
33adffec 00000000 00749910 00000000 00000000 33ff0000 ntdll!RtlInitializeExceptionChain+0x0 


*** Debug Message Dump ****


*** Foreground Window Data ***
    Window Name      : 
    Window Class     : 
    Window Process ID: 0
    Window Thread ID : 0

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4888, iMonCtr=1
Model crash detected, will try to restart...
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
22 Dec 2012 16:25:55 1253746 15475414 hadcm3n_zi2i_1920_40_008245016_2 362,880 715,944 1.9729
21 Dec 2012 19:57:34 1253746 15475414 hadcm3n_zi2i_1920_40_008245016_2 336,960 643,062 1.9084
21 Dec 2012 00:43:16 1253746 15475414 hadcm3n_zi2i_1920_40_008245016_2 311,040 574,587 1.8473
20 Dec 2012 10:32:14 1253746 15475414 hadcm3n_zi2i_1920_40_008245016_2 285,120 524,582 1.8399
19 Dec 2012 22:59:09 1253746 15475414 hadcm3n_zi2i_1920_40_008245016_2 259,200 483,188 1.8642
19 Dec 2012 05:46:25 1253746 15475414 hadcm3n_zi2i_1920_40_008245016_2 233,280 427,922 1.8344
18 Dec 2012 14:35:33 1253746 15475414 hadcm3n_zi2i_1920_40_008245016_2 207,360 381,985 1.8421
17 Dec 2012 23:30:03 1253746 15475414 hadcm3n_zi2i_1920_40_008245016_2 181,440 339,535 1.8713
17 Dec 2012 11:10:09 1253746 15475414 hadcm3n_zi2i_1920_40_008245016_2 155,520 298,268 1.9179
16 Dec 2012 23:12:49 1253746 15475414 hadcm3n_zi2i_1920_40_008245016_2 129,600 255,308 1.9700
16 Dec 2012 11:25:03 1253746 15475414 hadcm3n_zi2i_1920_40_008245016_2 103,680 212,885 2.0533
15 Dec 2012 23:02:33 1253746 15475414 hadcm3n_zi2i_1920_40_008245016_2 77,760 167,013 2.1478
15 Dec 2012 07:23:20 1253746 15475414 hadcm3n_zi2i_1920_40_008245016_2 51,840 112,729 2.1746
14 Dec 2012 16:36:51 1253746 15475414 hadcm3n_zi2i_1920_40_008245016_2 25,920 56,311 2.1725


©2024 climateprediction.net