Task 19830082

Name	wah2_sas50_ixkh_201412_13_423_010644819_0
Workunit	10644819
Created	20 Jul 2016, 11:06:59 UTC
Sent	23 Jul 2016, 10:43:29 UTC
Report deadline	5 Jul 2017, 16:03:29 UTC
Received	1 Aug 2016, 11:06:36 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	9 (0x00000009) Unknown error code
Computer ID	907613
Run time	2 days 4 hours 59 min 58 sec
CPU time	2 days 3 hours 33 min 34 sec
Validate state	Invalid
Credit	6,859.15
Device peak FLOPS	4.61 GFLOPS
Application version	Weather At Home 2 (wah2) (region independent) v8.12 windows_intelx86
Peak working set size	242.20 MB
Peak swap size	205.68 MB
Peak disk usage	41.82 MB
Stderr	<core_client_version>7.6.22</core_client_version> <![CDATA[ <message> - exit code 9 (0x9) </message> <stderr_txt> 18:55:21 (9952): start_timer_thread(): CreateThread() failed, errno 0 18:55:22 (12384): start_timer_thread(): CreateThread() failed, errno 0 Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x00463327 read attempt to address 0x0173C3AC Engaging BOINC Windows Runtime Debugger... ****************** BOINC Windows Runtime Debugger Version 7.7.0 Dump Timestamp : 07/27/16 23:16:35 Install Directory : Data Directory : F:\BOINC Project Symstore : LoadLibraryA( F:\BOINC\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( F:\BOINC\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( F:\BOINC\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( F:\BOINC\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: F:\BOINC\slots\5;F:\BOINC\projects\climateprediction.net ModLoad: 0000000000400000 0000000000152000 F:\BOINC\projects\climateprediction.net\wah2_8.12_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 00000000773e0000 0000000000182000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.14393.0) (-exported- Symbols Loaded) Linked PDB Filename : wntdll.pdb File Version : 10.0.14393.0 (rs1_release.160715-1616) Company Name : Microsoft Corporation Product Name : Microsoft? Windows? Operating System Product Version : 10.0.14393.0 ModLoad: 0000000074d20000 00000000000e0000 C:\WINDOWS\System32\KERNEL32.DLL (6.2.14393.0) (-exported- Symbols Loaded) Linked PDB Filename : wkernel32.pdb File Version : 10.0.14393.0 (rs1_release.160715-1616) Company Name : Microsoft Corporation Product Name : Microsoft? Windows? Operating System Product Version : 10.0.14393.0 ModLoad: 0000000076ce0000 00000000001a1000 C:\WINDOWS\System32\KERNELBASE.dll (6.2.14393.0) (-exported- Symbols Loaded) Linked PDB Filename : wkernelbase.pdb File Version : 10.0.14393.0 (rs1_release.160715-1616) Company Name : Microsoft Corporation Product Name : Microsoft? Windows? Operating System Product Version : 10.0.14393.0 ModLoad: 000000006c4f0000 0000000000092000 C:\WINDOWS\system32\apphelp.dll (6.2.14393.0) (-exported- Symbols Loaded) Linked PDB Filename : apphelp.pdb File Version : 10.0.14393.0 (rs1_release.160715-1616) Company Name : Microsoft Corporation Product Name : Microsoft? Windows? Operating System Product Version : 10.0.14393.0 ModLoad: 0000000074e00000 000000000015f000 C:\WINDOWS\System32\USER32.dll (6.2.14393.0) (-exported- Symbols Loaded) Linked PDB Filename : wuser32.pdb File Version : 10.0.14393.0 (rs1_release.160715-1616) Company Name : Microsoft Corporation Product Name : Microsoft? Windows? Operating System Product Version : 10.0.14393.0 ModLoad: 0000000075210000 0000000000015000 C:\WINDOWS\System32\win32u.dll (6.2.14393.0) (-exported- Symbols Loaded) Linked PDB Filename : wwin32u.pdb File Version : 10.0.14393.0 (rs1_release.160715-1616) Company Name : Microsoft Corporation Product Name : Microsoft? Windows? Operating System Product Version : 10.0.14393.0 ModLoad: 00000000751e0000 000000000002b000 C:\WINDOWS\System32\GDI32.dll (6.2.14393.0) (-exported- Symbols Loaded) Linked PDB Filename : wgdi32.pdb File Version : 10.0.14393.0 (rs1_release.160715-1616) Company Name : Microsoft Corporation Product Name : Microsoft? Windows? Operating System Product Version : 10.0.14393.0 ModLoad: 0000000075370000 000000000015b000 C:\WINDOWS\System32\gdi32full.dll (6.2.14393.0) (-exported- Symbols Loaded) Linked PDB Filename : wgdi32full.pdb File Version : 10.0.14393.0 (rs1_release.160715-1616) Company Name : Microsoft Corporation Product Name : Microsoft? Windows? Operating System Product Version : 10.0.14393.0 ModLoad: 0000000075240000 0000000000077000 C:\WINDOWS\System32\ADVAPI32.dll (6.2.14393.0) (-exported- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 10.0.14393.0 (rs1_release.160715-1616) Company Name : Microsoft Corporation Product Name : Microsoft? Windows? Operating System Product Version : 10.0.14393.0 ModLoad: 0000000073f10000 00000000000be000 C:\WINDOWS\System32\msvcrt.dll (7.0.14393.0) (-exported- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.14393.0 (rs1_release.160715-1616) Company Name : Microsoft Corporation Product Name : Microsoft? Windows? Operating System Product Version : 7.0.14393.0 ModLoad: 0000000073ec0000 0000000000041000 C:\WINDOWS\System32\sechost.dll (6.2.14393.0) (-exported- Symbols Loaded) Linked PDB Filename : sechost.pdb File Version : 10.0.14393.0 (rs1_release.160715-1616) Company Name : Microsoft Corporation Product Name : Microsoft? Windows? Operating System Product Version : 10.0.14393.0 ModLoad: 0000000075110000 00000000000c1000 C:\WINDOWS\System32\RPCRT4.dll (6.2.14393.0) (-exported- Symbols Loaded) Linked PDB Filename : wrpcrt4.pdb File Version : 10.0.14393.0 (rs1_release.160715-1616) Company Name : Microsoft Corporation Product Name : Microsoft? Windows? Operating System Product Version : 10.0.14393.0 ModLoad: 0000000073d10000 000000000001e000 C:\WINDOWS\System32\SspiCli.dll (6.2.14393.0) (-exported- Symbols Loaded) Linked PDB Filename : wsspicli.pdb File Version : 10.0.14393.0 (rs1_release.160715-1616) Company Name : Microsoft Corporation Product Name : Microsoft? Windows? Operating System Product Version : 10.0.14393.0 ModLoad: 0000000073d00000 000000000000a000 C:\WINDOWS\System32\CRYPTBASE.dll (6.2.14393.0) (-exported- Symbols Loaded) Linked PDB Filename : cryptbase.pdb File Version : 10.0.14393.0 (rs1_release.160715-1616) Company Name : Microsoft Corporation Product Name : Microsoft? Windows? Operating System Product Version : 10.0.14393.0 ModLoad: 0000000074020000 000000000005a000 C:\WINDOWS\System32\bcryptPrimitives.dll (6.2.14393.0) (-exported- Symbols Loaded) Linked PDB Filename : bcryptprimitives.pdb File Version : 10.0.14393.0 (rs1_release.160715-1616) Company Name : Microsoft Corporation Product Name : Microsoft? Windows? Operating System Product Version : 10.0.14393.0 ModLoad: 0000000073c30000 0000000000028000 C:\WINDOWS\SYSTEM32\ntmarta.dll (6.2.14393.0) (-exported- Symbols Loaded) Linked PDB Filename : ntmarta.pdb File Version : 10.0.14393.0 (rs1_release.160715-1616) Company Name : Microsoft Corporation Product Name : Microsoft? Windows? Operating System Product Version : 10.0.14393.0 ModLoad: 0000000075020000 00000000000e0000 C:\WINDOWS\System32\ucrtbase.dll (6.2.14393.0) (-exported- Symbols Loaded) Linked PDB Filename : ucrtbase.pdb File Version : 10.0.14393.0 (rs1_release.160715-1616) Company Name : Microsoft Corporation Product Name : Microsoft? Windows? Operating System Product Version : 10.0.14393.0 ModLoad: 00000000709b0000 0000000000145000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.2.14321.1024) (-exported- Symbols Loaded) Linked PDB Filename : dbghelp.pdb File Version : 10.0.14321.1024 (rs1_release.160715-1616) Company Name : Microsoft Product Name : Microsoft? Windows? Operating System Product Version : 10.0.14321.1024 ModLoad: 0000000073cf0000 0000000000008000 C:\WINDOWS\SYSTEM32\version.dll (6.2.14393.0) (-exported- Symbols Loaded) Linked PDB Filename : version.pdb File Version : 10.0.14393.0 (rs1_release.160715-1616) Company Name : Microsoft Corporation Product Name : Microsoft? Windows? Operating System Product Version : 10.0.14393.0 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 21581, Write: 0, Other 78921 - I/O Transfers Counters - Read: 0, Write: 3216, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 199616, QuotaPeakPagedPoolUsage: 215608 QuotaNonPagedPoolUsage: 8200, QuotaPeakNonPagedPoolUsage: 9264 - Virtual Memory Usage - VirtualSize: 109473792, PeakVirtualSize: 117923840 - Pagefile Usage - PagefileUsage: 2809856, PeakPagefileUsage: 3186688 - Working Set Size - WorkingSetSize: 2375680, PeakWorkingSetSize: 80183296, PageFaultCount: 21400 * Dump of thread ID 11900 (state: Waiting): * - Information - Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 1058034.000000 - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x00463327 read attempt to address 0x0173C3AC - Registers - eax=00000015 ebx=fffffffe ecx=00000000 edx=00000000 esi=0166c020 edi=00000000 eip=00463327 esp=00a1fe7c ebp=00a1fe80 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010216 - Callstack - ChildEBP RetAddr Args to Child 00a1fe80 004484ff 0166c020 00000000 0166c020 fffffffe wah2_8.12_windows_intelx86!+0x0 00a1fe98 0044fd08 0166c020 00000000 0051d6a8 00a1fee8 wah2_8.12_windows_intelx86!+0x0 00a1feac 00485b9c 00000015 dd82557a 00000006 00485b0c wah2_8.12_windows_intelx86!+0x0 00a1fee8 76e1399f 00000006 7133ae34 76e137e0 76e137e0 wah2_8.12_windows_intelx86!+0x0 00a1ff80 74d362c4 80000006 74d362a0 16a74fc0 00a1ffdc KERNELBASE!CtrlRoutine+0x0 00a1ff94 77440609 80000006 a6023f0e 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 00a1ffdc 774405d4 ffffffff 77462525 00000000 00000000 ntdll!RtlSubscribeWnfStateChangeNotification+0x0 00a1ffec 00000000 76e137e0 80000006 00000000 00000000 ntdll!RtlSubscribeWnfStateChangeNotification+0x0 * Dump of thread ID 248 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 8437500.000000, User Time: 40468752.000000, Wait Time: 1058024.000000 - Registers - eax=00000000 ebx=0019efb4 ecx=00000000 edx=00000000 esi=00000000 edi=0019efb4 eip=7744df4c esp=0019ef74 ebp=0019efd8 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000206 - Callstack - ChildEBP RetAddr Args to Child 0019efd8 76db2a5f 000000c8 00000000 0019effc 004691c1 ntdll!NtDelayExecution+0x0 0019efe8 004691c1 000000c8 000000c8 00000000 0019f034 KERNELBASE!Sleep+0x0 0019effc 00407f7d a0000000 027f9999 004d0c48 00000020 wah2_8.12_windows_intelx86!+0x0 0019f034 00409542 00000000 dd3a56d2 0048616e 00000000 wah2_8.12_windows_intelx86!+0x0 0019fd40 00409731 0019fef0 004097af 004d1368 0019fd60 wah2_8.12_windows_intelx86!+0x0 0019fd48 004097af 004d1368 0019fd60 006e3730 0019fd60 wah2_8.12_windows_intelx86!+0x0 0019fef0 0048611b 00400000 00000000 006e376e 0000000a wah2_8.12_windows_intelx86!+0x0 0019ff80 74d362c4 0023b000 74d362a0 161f4fc0 0019ffdc wah2_8.12_windows_intelx86!+0x0 0019ff94 77440609 0023b000 a6ba3f0e 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 0019ffdc 774405d4 ffffffff 77462525 00000000 00000000 ntdll!RtlSubscribeWnfStateChangeNotification+0x0 0019ffec 00000000 0048616e 0023b000 00000000 78746341 ntdll!RtlSubscribeWnfStateChangeNotification+0x0 * Dump of thread ID 13092 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 156250.000000, Wait Time: 1058033.000000 - Registers - eax=c0000022 ebx=0165fd1c ecx=00000000 edx=00000000 esi=00000000 edi=0165fd1c eip=7744df4c esp=0165fcdc ebp=0165fd40 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 0165fd40 76db2a5f 00000064 00000000 0165ff80 004790c4 ntdll!NtDelayExecution+0x0 0165fd50 004790c4 00000064 004790a0 004790a0 00000000 KERNELBASE!Sleep+0x0 0165ff80 74d362c4 00000000 74d362a0 17634fc0 0165ffdc wah2_8.12_windows_intelx86!+0x0 0165ff94 77440609 00000000 a7c63f0e 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 0165ffdc 774405d4 ffffffff 77462525 00000000 00000000 ntdll!RtlSubscribeWnfStateChangeNotification+0x0 0165ffec 00000000 004790a0 00000000 00000000 00000000 ntdll!RtlSubscribeWnfStateChangeNotification+0x0 * Dump of thread ID 11376 (state: Waiting): * - Information - Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 1058026.000000 - Registers - eax=00000000 ebx=006e07c8 ecx=00000000 edx=00000000 esi=006e3070 edi=006e3230 eip=7744f7fc esp=008dfddc ebp=008dff80 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 008dff80 74d362c4 006e07c8 74d362a0 168b4fc0 008dffdc ntdll!NtWaitForWorkViaWorkerFactory+0x0 008dff94 77440609 006e07c8 a62e3f0e 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 008dffdc 774405d4 ffffffff 77462525 00000000 00000000 ntdll!RtlSubscribeWnfStateChangeNotification+0x0 008dffec 00000000 77416020 006e07c8 00000000 00000000 ntdll!RtlSubscribeWnfStateChangeNotification+0x0 * Debug Message Dump ** * Foreground Window Data *** Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9952, selfPID=9952, iMonCtr=1 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9952, selfPID=12384, iMonCtr=1 23:19:00 (15332): start_timer_thread(): CreateThread() failed, errno 0 23:19:02 (14784): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 18:59:44 (3840): start_timer_thread(): CreateThread() failed, errno 0 18:59:46 (4136): start_timer_thread(): CreateThread() failed, errno 0 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3840, selfPID=3840, iMonCtr=1 No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3840, selfPID=4136, iMonCtr=1 19:42:22 (11392): start_timer_thread(): CreateThread() failed, errno 0 19:42:23 (11488): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 01:08:31 (1264): start_timer_thread(): CreateThread() failed, errno 0 01:08:33 (12956): start_timer_thread(): CreateThread() failed, errno 0 No Process Handle Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1264, selfPID=1264, iMonCtr=1 No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1264, selfPID=12956, iMonCtr=1 19:57:12 (8912): start_timer_thread(): CreateThread() failed, errno 0 19:57:13 (7544): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 02:30:04 (9184): start_timer_thread(): CreateThread() failed, errno 0 02:30:06 (7760): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:37:57 (11520): start_timer_thread(): CreateThread() failed, errno 0 13:37:59 (6940): start_timer_thread(): CreateThread() failed, errno 0 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11520, selfPID=6940, iMonCtr=1 No Process Handle Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11520, selfPID=11520, iMonCtr=1 06:56:56 (11428): start_timer_thread(): CreateThread() failed, errno 0 06:56:58 (236): start_timer_thread(): CreateThread() failed, errno 0 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11428, selfPID=11428, iMonCtr=1 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11428, selfPID=236, iMonCtr=1 20:26:31 (11800): start_timer_thread(): CreateThread() failed, errno 0 20:26:33 (9336): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 21:01:06 (2660): start_timer_thread(): CreateThread() failed, errno 0 21:01:07 (3612): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... No Process Handle Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2660, selfPID=2660, iMonCtr=1 No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2660, selfPID=3612, iMonCtr=1 </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
31 Jul 2016 15:01:31	907613	19830082	wah2_sas50_ixkh_201412_13_423_010644819_0	103,979	175,585	1.6887
31 Jul 2016 08:55:58	907613	19830082	wah2_sas50_ixkh_201412_13_423_010644819_0	92,459	155,573	1.6826
31 Jul 2016 03:50:33	907613	19830082	wah2_sas50_ixkh_201412_13_423_010644819_0	80,939	136,362	1.6848
30 Jul 2016 15:54:25	907613	19830082	wah2_sas50_ixkh_201412_13_423_010644819_0	69,419	117,854	1.6977
30 Jul 2016 10:08:44	907613	19830082	wah2_sas50_ixkh_201412_13_423_010644819_0	57,899	98,008	1.6927
29 Jul 2016 22:47:12	907613	19830082	wah2_sas50_ixkh_201412_13_423_010644819_0	46,379	79,328	1.7104
29 Jul 2016 17:31:19	907613	19830082	wah2_sas50_ixkh_201412_13_423_010644819_0	34,859	61,000	1.7499
28 Jul 2016 17:51:21	907613	19830082	wah2_sas50_ixkh_201412_13_423_010644819_0	23,339	40,467	1.7339
28 Jul 2016 11:20:35	907613	19830082	wah2_sas50_ixkh_201412_13_423_010644819_0	11,819	19,923	1.6857