Task 20613146

Name	wah2_global_a01o_208812_145_615_011130682_1
Workunit	11130682
Created	8 Aug 2017, 10:59:24 UTC
Sent	8 Aug 2017, 19:19:04 UTC
Report deadline	22 Jul 2018, 0:39:04 UTC
Received	26 Sep 2017, 9:41:39 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	194 (0x000000C2) EXIT_ABORTED_BY_CLIENT
Computer ID	1339864
Run time	33 days 15 hours 38 min 4 sec
CPU time	29 days 1 hours 49 min 37 sec
Validate state	Invalid
Credit	25,462.29
Device peak FLOPS	3.65 GFLOPS
Application version	Weather At Home 2 (wah2) v8.24 windows_intelx86
Peak working set size	248.22 MB
Peak swap size	173.68 MB
Peak disk usage	47.74 MB
Stderr	<core_client_version>7.6.33</core_client_version> <![CDATA[ <message> finish file present too long </message> <stderr_txt> diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 15:34:35 (14280): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 18:30:46 (15196): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 15:27:14 (8116): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 22:25:20 (11964): start_timer_thread(): CreateThread() failed, errno 0 22:25:21 (13164): start_timer_thread(): CreateThread() failed, errno 0 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11964, selfPID=13164, iMonCtr=1 23:52:25 (8096): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 19:41:39 (12236): start_timer_thread(): CreateThread() failed, errno 0 19:41:41 (14836): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 15:21:07 (19572): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 20:23:07 (13600): start_timer_thread(): CreateThread() failed, errno 0 20:23:09 (8088): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 15:48:01 (16632): start_timer_thread(): CreateThread() failed, errno 0 15:48:01 (9596): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 15:17:01 (18336): start_timer_thread(): CreateThread() failed, errno 0 15:17:02 (6844): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 06:20:27 (16196): start_timer_thread(): CreateThread() failed, errno 0 06:20:29 (10560): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 21:12:07 (13308): start_timer_thread(): CreateThread() failed, errno 0 21:12:08 (17696): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:25:33 (18080): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 23:00:05 (3648): start_timer_thread(): CreateThread() failed, errno 0 23:00:06 (17916): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 03:50:07 (2656): start_timer_thread(): CreateThread() failed, errno 0 03:50:09 (19412): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 17:30:18 (11784): start_timer_thread(): CreateThread() failed, errno 0 17:30:19 (6568): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 15:56:51 (19220): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 00:35:30 (15112): start_timer_thread(): CreateThread() failed, errno 0 00:35:32 (18180): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 18:25:31 (17576): start_timer_thread(): CreateThread() failed, errno 0 18:25:33 (14356): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 20:51:03 (8100): start_timer_thread(): CreateThread() failed, errno 0 20:51:05 (16988): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 15:31:09 (18920): start_timer_thread(): CreateThread() failed, errno 0 15:31:11 (11684): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 17:49:59 (1532): start_timer_thread(): CreateThread() failed, errno 0 17:50:01 (7512): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 21:11:15 (14828): start_timer_thread(): CreateThread() failed, errno 0 21:11:16 (9236): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 15:41:48 (11428): start_timer_thread(): CreateThread() failed, errno 0 15:41:50 (15640): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 15:44:57 (10660): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 17:13:33 (13468): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 17:57:40 (14364): start_timer_thread(): CreateThread() failed, errno 0 17:57:42 (11728): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 17:57:04 (2924): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 15:30:29 (14172): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 01:35:41 (11244): start_timer_thread(): CreateThread() failed, errno 0 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 01:35:43 (15012): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 15:29:34 (9640): start_timer_thread(): CreateThread() failed, errno 0 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9640, selfPID=8772, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_ain::Monitor... 04:25:47 (8772): called boinc_finish(0) CPDN Monitor - Abort request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x0048EE46 write attempt to address 0x0268112C Engaging BOINC Windows Runtime Debugger... ****************** BOINC Windows Runtime Debugger Version 7.7.0 Dump Timestamp : 09/26/17 04:40:39 Install Directory : Data Directory : C:\ProgramData\BOINC Project Symstore : LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: C:\ProgramData\BOINC\projects\climateprediction.net;C:\ProgramData\BOINC\projects\climateprediction.net ModLoad: 0000000000400000 0000000000152000 C:\ProgramData\BOINC\projects\climateprediction.net\wah2_8.24_windows_intelx86.exe (-nosymbols- Symbols Loaded) Linked PDB Filename : ModLoad: 0000000077170000 000000000018e000 C:\WINDOWS\SYSTEM32\ntdll.dll (6.2.15063.608) (-exported- Symbols Loaded) Linked PDB Filename : wntdll.pdb File Version : 10.0.15063.608 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Système dexploitation Microsoft® Windows® Product Version : 10.0.15063.608 ModLoad: 0000000075030000 00000000000d0000 C:\WINDOWS\System32\KERNEL32.DLL (6.2.15063.296) (-exported- Symbols Loaded) Linked PDB Filename : wkernel32.pdb File Version : 10.0.15063.608 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Système dexploitation Microsoft® Windows® Product Version : 10.0.15063.608 ModLoad: 0000000074e60000 00000000001c2000 C:\WINDOWS\System32\KERNELBASE.dll (6.2.15063.608) (-exported- Symbols Loaded) Linked PDB Filename : wkernelbase.pdb File Version : 10.0.15063.608 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Système dexploitation Microsoft® Windows® Product Version : 10.0.15063.608 ModLoad: 00000000749f0000 000000000013c000 C:\WINDOWS\System32\USER32.dll (6.2.15063.608) (-exported- Symbols Loaded) Linked PDB Filename : wuser32.pdb File Version : 10.0.15063.0 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Système dexploitation MicrosoftMD WindowsMD Product Version : 10.0.15063.0 ModLoad: 0000000074cd0000 0000000000016000 C:\WINDOWS\System32\win32u.dll (6.2.15063.608) (-exported- Symbols Loaded) Linked PDB Filename : wwin32u.pdb File Version : 10.0.15063.608 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.608 ModLoad: 0000000074080000 0000000000021000 C:\WINDOWS\System32\GDI32.dll (6.2.15063.483) (-exported- Symbols Loaded) Linked PDB Filename : wgdi32.pdb File Version : 10.0.15063.483 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.483 ModLoad: 0000000077000000 0000000000158000 C:\WINDOWS\System32\gdi32full.dll (6.2.15063.608) (-exported- Symbols Loaded) Linked PDB Filename : wgdi32full.pdb File Version : 10.0.15063.608 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.608 ModLoad: 0000000076f80000 0000000000079000 C:\WINDOWS\System32\msvcp_win.dll (6.2.15063.0) (-exported- Symbols Loaded) Linked PDB Filename : msvcp_win.pdb File Version : 10.0.15063.0 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.0 ModLoad: 00000000748d0000 0000000000118000 C:\WINDOWS\System32\ucrtbase.dll (6.2.15063.447) (-exported- Symbols Loaded) Linked PDB Filename : ucrtbase.pdb File Version : 10.0.15063.447 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.447 ModLoad: 0000000076e10000 0000000000077000 C:\WINDOWS\System32\ADVAPI32.dll (6.2.15063.0) (-exported- Symbols Loaded) Linked PDB Filename : advapi32.pdb File Version : 10.0.15063.0 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Système dexploitation Microsoft® Windows® Product Version : 10.0.15063.0 ModLoad: 0000000075850000 00000000000bd000 C:\WINDOWS\System32\msvcrt.dll (7.0.15063.0) (-exported- Symbols Loaded) Linked PDB Filename : msvcrt.pdb File Version : 7.0.15063.0 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 7.0.15063.0 ModLoad: 0000000074200000 0000000000041000 C:\WINDOWS\System32\sechost.dll (6.2.15063.0) (-exported- Symbols Loaded) Linked PDB Filename : sechost.pdb File Version : 10.0.15063.0 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.0 ModLoad: 0000000076d20000 00000000000c0000 C:\WINDOWS\System32\RPCRT4.dll (6.2.15063.447) (-exported- Symbols Loaded) Linked PDB Filename : wrpcrt4.pdb File Version : 10.0.15063.0 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Système dexploitation Microsoft® Windows® Product Version : 10.0.15063.0 ModLoad: 0000000073c40000 0000000000020000 C:\WINDOWS\System32\SspiCli.dll (6.2.15063.0) (-exported- Symbols Loaded) Linked PDB Filename : wsspicli.pdb File Version : 10.0.15063.0 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.0 ModLoad: 0000000073c30000 000000000000a000 C:\WINDOWS\System32\CRYPTBASE.dll (6.2.15063.0) (-exported- Symbols Loaded) Linked PDB Filename : cryptbase.pdb File Version : 10.0.15063.0 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.0 ModLoad: 00000000740b0000 0000000000057000 C:\WINDOWS\System32\bcryptPrimitives.dll (6.2.15063.138) (-exported- Symbols Loaded) Linked PDB Filename : bcryptprimitives.pdb File Version : 10.0.15063.138 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.138 ModLoad: 0000000074310000 0000000000025000 C:\WINDOWS\System32\IMM32.DLL (6.2.15063.0) (-exported- Symbols Loaded) Linked PDB Filename : wimm32.pdb File Version : 10.0.15063.0 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.0 ModLoad: 000000006fb70000 000000000015d000 C:\WINDOWS\SYSTEM32\dbghelp.dll (6.2.15063.447) (-exported- Symbols Loaded) Linked PDB Filename : dbghelp.pdb File Version : 10.0.15063.447 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.447 ModLoad: 0000000073b60000 0000000000008000 C:\WINDOWS\SYSTEM32\version.dll (6.2.15063.0) (-exported- Symbols Loaded) Linked PDB Filename : version.pdb File Version : 10.0.15063.0 (WinBuild.160101.0800) Company Name : Microsoft Corporation Product Name : Microsoft® Windows® Operating System Product Version : 10.0.15063.0 * Dump of the Process Statistics: * - I/O Operations Counters - Read: 327, Write: 0, Other 2453 - I/O Transfers Counters - Read: 0, Write: 1944, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 90504, QuotaPeakPagedPoolUsage: 90640 QuotaNonPagedPoolUsage: 8624, QuotaPeakNonPagedPoolUsage: 9048 - Virtual Memory Usage - VirtualSize: 55345152, PeakVirtualSize: 55345152 - Pagefile Usage - PagefileUsage: 3440640, PeakPagefileUsage: 3440640 - Working Set Size - WorkingSetSize: 6320128, PeakWorkingSetSize: 6320128, PageFaultCount: 1842 * Dump of thread ID 11104 (state: Waiting): * - Information - Status: Wait Reason: UserRequest, , Kernel Time: 1718750.000000, User Time: 468750.000000, Wait Time: 54441.000000 - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x0048EE46 write attempt to address 0x0268112C - Registers - eax=00000065 ebx=004b63b9 ecx=0019d8e8 edx=0268112c esi=0019d6a0 edi=0019d918 eip=0048ee46 esp=0019d634 ebp=0019d8c4 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010202 - Callstack - ChildEBP RetAddr Args to Child 0019d8c4 0048161c 0019d8e8 004b63b8 00000000 0019d918 wah2_8.24_windows_intelx86!+0x0 0019d908 0044fa52 0268112c 004b63b8 004b6484 00000421 wah2_8.24_windows_intelx86!+0x0 0019d928 00496e90 0000000b 00000000 00000000 00000000 wah2_8.24_windows_intelx86!+0x0 0019d940 00485ca1 c0000005 0019d974 00490c54 00000000 wah2_8.24_windows_intelx86!+0x0 0019ff80 75048744 00260000 75048720 6bfdcd2e 0019ffdc wah2_8.24_windows_intelx86!+0x0 0019ff94 771d582d 00260000 3ec5a747 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 0019ffdc 771d57fd ffffffff 771f636d 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 0019ffec 00000000 00485cce 00260000 00000000 78746341 ntdll!RtlGetAppContainerNamedObjectPath+0x0 * Dump of thread ID 13356 (state: Waiting): * - Information - Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 53136.000000 - Registers - eax=00000000 ebx=007b4bb8 ecx=00000000 edx=00000000 esi=007b51e0 edi=007b53a0 eip=771e396c esp=0079fd7c ebp=0079ff80 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 0079ff80 75048744 007b4bb8 75048720 6b9dcd2e 0079ffdc ntdll!ZwWaitForWorkViaWorkerFactory+0x0 0079ff94 771d582d 007b4bb8 3ea5a747 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 0079ffdc 771d57fd ffffffff 771f636d 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 0079ffec 00000000 771aa470 007b4bb8 00000000 00905a4d ntdll!RtlGetAppContainerNamedObjectPath+0x0 * Dump of thread ID 14020 (state: Waiting): * - Information - Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 53136.000000 - Registers - eax=00000000 ebx=007b4bb8 ecx=00000000 edx=00000000 esi=007b5bf8 edi=007b5db8 eip=771e396c esp=009efd7c ebp=009eff80 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 009eff80 75048744 007b4bb8 75048720 6b7acd2e 009effdc ntdll!ZwWaitForWorkViaWorkerFactory+0x0 009eff94 771d582d 007b4bb8 3e42a747 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 009effdc 771d57fd ffffffff 771f636d 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 009effec 00000000 771aa470 007b4bb8 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 * Dump of thread ID 11012 (state: Waiting): * - Information - Status: Wait Reason: EventPairLow, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 53136.000000 - Registers - eax=00000000 ebx=007b4bb8 ecx=00000000 edx=00000000 esi=007b60b0 edi=007b6270 eip=771e396c esp=00b2fd7c ebp=00b2ff80 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 00b2ff80 75048744 007b4bb8 75048720 6b56cd2e 00b2ffdc ntdll!ZwWaitForWorkViaWorkerFactory+0x0 00b2ff94 771d582d 007b4bb8 3e6ea747 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 00b2ffdc 771d57fd ffffffff 771f636d 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 00b2ffec 00000000 771aa470 007b4bb8 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 * Dump of thread ID 12520 (state: Waiting): * - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 54437.000000 - Registers - eax=00000000 ebx=025bfd1c ecx=00000000 edx=00000000 esi=00000000 edi=00000001 eip=771e205c esp=025bfcdc ebp=025bfd40 cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 025bfd40 74f5354f 00000064 00000000 025bff80 00478b44 ntdll!NtDelayExecution+0x0 025bfd50 00478b44 00000064 00478b20 00478b20 00000000 KERNELBASE!Sleep+0x0 025bff80 75048744 00000000 75048720 69bfcd2e 025bffdc wah2_8.24_windows_intelx86!+0x0 025bff94 771d582d 00000000 3c87a747 00000000 00000000 KERNEL32!BaseThreadInitThunk+0x0 025bffdc 771d57fd ffffffff 771f636d 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 025bffec 00000000 00478b20 00000000 00000000 00000000 ntdll!RtlGetAppContainerNamedObjectPath+0x0 * Debug Message Dump ** * Foreground Window Data *** Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
26 Sep 2017 10:13:55	1339864	20613146	wah2_global_a01o_208812_145_615_011130682_1	385,987	2,500,601	6.4785
26 Sep 2017 10:10:06	1339864	20613146	wah2_global_a01o_208812_145_615_011130682_1	383,107	2,479,453	6.4720
26 Sep 2017 09:58:43	1339864	20613146	wah2_global_a01o_208812_145_615_011130682_1	380,227	2,458,215	6.4651
24 Sep 2017 17:17:59	1339864	20613146	wah2_global_a01o_208812_145_615_011130682_1	377,347	2,437,565	6.4597
24 Sep 2017 11:23:26	1339864	20613146	wah2_global_a01o_208812_145_615_011130682_1	374,467	2,416,612	6.4535
24 Sep 2017 05:29:03	1339864	20613146	wah2_global_a01o_208812_145_615_011130682_1	371,587	2,395,891	6.4477
23 Sep 2017 23:14:33	1339864	20613146	wah2_global_a01o_208812_145_615_011130682_1	368,707	2,374,145	6.4391
23 Sep 2017 18:11:47	1339864	20613146	wah2_global_a01o_208812_145_615_011130682_1	365,827	2,354,731	6.4367
23 Sep 2017 12:28:12	1339864	20613146	wah2_global_a01o_208812_145_615_011130682_1	362,947	2,335,723	6.4354
23 Sep 2017 07:06:07	1339864	20613146	wah2_global_a01o_208812_145_615_011130682_1	360,067	2,317,045	6.4350
22 Sep 2017 06:41:23	1339864	20613146	wah2_global_a01o_208812_145_615_011130682_1	357,187	2,296,710	6.4300
21 Sep 2017 23:31:22	1339864	20613146	wah2_global_a01o_208812_145_615_011130682_1	354,307	2,273,593	6.4170
21 Sep 2017 07:52:35	1339864	20613146	wah2_global_a01o_208812_145_615_011130682_1	351,427	2,250,961	6.4052
21 Sep 2017 00:09:46	1339864	20613146	wah2_global_a01o_208812_145_615_011130682_1	348,547	2,228,573	6.3939
20 Sep 2017 06:34:17	1339864	20613146	wah2_global_a01o_208812_145_615_011130682_1	345,667	2,205,420	6.3802
20 Sep 2017 00:11:41	1339864	20613146	wah2_global_a01o_208812_145_615_011130682_1	342,787	2,182,939	6.3682
19 Sep 2017 17:41:37	1339864	20613146	wah2_global_a01o_208812_145_615_011130682_1	339,907	2,162,166	6.3611
19 Sep 2017 11:12:59	1339864	20613146	wah2_global_a01o_208812_145_615_011130682_1	337,027	2,140,884	6.3523
19 Sep 2017 05:41:05	1339864	20613146	wah2_global_a01o_208812_145_615_011130682_1	334,147	2,121,934	6.3503
19 Sep 2017 00:19:58	1339864	20613146	wah2_global_a01o_208812_145_615_011130682_1	331,267	2,103,377	6.3495