climateprediction.net home page
Task 21315648

Task 21315648

Name wah2_pnw25_ptf5_205509_28_757_011646696_0
Workunit 11646696
Created 14 Sep 2018, 13:10:14 UTC
Sent 16 Sep 2018, 15:48:22 UTC
Report deadline 29 Aug 2019, 21:08:22 UTC
Received 17 Oct 2018, 22:04:10 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1291402
Run time 5 days 21 hours 52 min 14 sec
CPU time 5 days 18 hours 36 min 10 sec
Validate state Valid
Credit 21,297.94
Device peak FLOPS 3.51 GFLOPS
Application version Weather At Home 2 (wah2) v8.24
windows_intelx86
Peak working set size 186.46 MB
Peak swap size 167.66 MB
Peak disk usage 12.70 MB
Stderr
<core_client_version>7.12.1</core_client_version>
<![CDATA[
<stderr_txt>


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Access Violation (0xc0000005) at address 0x76F1E084 read attempt to address 0x746FFFFF

Engaging BOINC Windows Runtime Debugger...



********************


BOINC Windows Runtime Debugger Version 7.7.0


Dump Timestamp    : 09/17/18 13:29:53
Install Directory : 
Data Directory    : D:\ProgramData\BOINC
Project Symstore  : 
LoadLibraryA( D:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126
Loaded Library    : dbghelp.dll
LoadLibraryA( D:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( D:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( D:\ProgramData\BOINC\version.dll ): GetLastError = 126
Loaded Library    : version.dll
Debugger Engine   : 4.0.5.0
Symbol Search Path: D:\ProgramData\BOINC\slots\0;D:\ProgramData\BOINC\projects\climateprediction.net


ModLoad: 0000000000400000 0000000000152000 D:\ProgramData\BOINC\projects\climateprediction.net\wah2_8.24_windows_intelx86.exe (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 0000000076ef0000 0000000000180000 C:\Windows\SysWOW64\ntdll.dll (6.1.7601.24231) (-exported- Symbols Loaded)
    Linked PDB Filename   : wntdll.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 0000000076040000 0000000000110000 C:\Windows\syswow64\kernel32.dll (6.1.7601.24231) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernel32.pdb
    File Version          : 6.1.7601.18015 (win7sp1_gdr.121129-1432)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.18015

ModLoad: 0000000075fe0000 0000000000047000 C:\Windows\syswow64\KERNELBASE.dll (6.1.7601.24231) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernelbase.pdb
    File Version          : 6.1.7601.18015 (win7sp1_gdr.121129-1432)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.18015

ModLoad: 0000000076930000 0000000000100000 C:\Windows\syswow64\USER32.dll (6.1.7601.23594) (-exported- Symbols Loaded)
    Linked PDB Filename   : wuser32.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17514

ModLoad: 0000000075f50000 0000000000090000 C:\Windows\syswow64\GDI32.dll (6.1.7601.24234) (-exported- Symbols Loaded)
    Linked PDB Filename   : wgdi32.pdb
    File Version          : 6.1.7601.24234 (win7sp1_ldr.180813-0600)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.24234

ModLoad: 0000000076030000 000000000000a000 C:\Windows\syswow64\LPK.dll (6.1.7601.24231) (-exported- Symbols Loaded)
    Linked PDB Filename   : wlpk.pdb
    File Version          : 6.1.7601.24231 (win7sp1_ldr.180810-0600)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.24231

ModLoad: 0000000074ba0000 000000000009d000 C:\Windows\syswow64\USP10.dll (1.626.7601.23894) (-exported- Symbols Loaded)
    Linked PDB Filename   : usp10.pdb
    File Version          : 1.0626.7601.23894 (win7sp1_ldr.170816-0600)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft(R) Uniscribe Unicode script processor
    Product Version       : 1.0626.7601.23894

ModLoad: 00000000748a0000 00000000000ac000 C:\Windows\syswow64\msvcrt.dll (7.0.7601.17744) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcrt.pdb
    File Version          : 7.0.7601.17744 (win7sp1_gdr.111215-1535)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 7.0.7601.17744

ModLoad: 00000000747f0000 00000000000a1000 C:\Windows\syswow64\ADVAPI32.dll (6.1.7601.24231) (-exported- Symbols Loaded)
    Linked PDB Filename   : advapi32.pdb
    File Version          : 6.1.7601.24231 (win7sp1_ldr.180810-0600)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.24231

ModLoad: 00000000764b0000 0000000000019000 C:\Windows\SysWOW64\sechost.dll (6.1.7601.18869) (-exported- Symbols Loaded)
    Linked PDB Filename   : sechost.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 0000000075e60000 00000000000f0000 C:\Windows\syswow64\RPCRT4.dll (6.1.7601.24231) (-exported- Symbols Loaded)
    Linked PDB Filename   : wrpcrt4.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 0000000074660000 0000000000060000 C:\Windows\syswow64\SspiCli.dll (6.1.7601.24231) (-exported- Symbols Loaded)
    Linked PDB Filename   : wsspicli.pdb
    File Version          : 6.1.7601.24231 (win7sp1_ldr.180810-0600)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.24231

ModLoad: 0000000074650000 000000000000c000 C:\Windows\syswow64\CRYPTBASE.dll (6.1.7601.24231) (-exported- Symbols Loaded)
    Linked PDB Filename   : cryptbase.pdb
    File Version          : 6.1.7601.24231 (win7sp1_ldr.180810-0600)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.24231

ModLoad: 0000000075a00000 0000000000060000 C:\Windows\system32\IMM32.DLL (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : wimm32.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17514

ModLoad: 0000000076320000 00000000000cd000 C:\Windows\syswow64\MSCTF.dll (6.1.7601.23915) (-exported- Symbols Loaded)
    Linked PDB Filename   : msctf.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 0000000072bc0000 0000000000021000 C:\Windows\system32\ntmarta.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : ntmarta.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 0000000074790000 0000000000045000 C:\Windows\syswow64\WLDAP32.dll (6.1.7601.23889) (-exported- Symbols Loaded)
    Linked PDB Filename   : wldap32.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 0000000066460000 000000000004c000 C:\Windows\system32\apphelp.dll (6.1.7601.19050) (-exported- Symbols Loaded)
    Linked PDB Filename   : apphelp.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 000000006ffa0000 00000000000eb000 C:\Windows\system32\dbghelp.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : dbghelp.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17514

ModLoad: 0000000072bf0000 0000000000009000 C:\Windows\system32\version.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : version.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385



*** Dump of the Process Statistics: ***

- I/O Operations Counters -
Read: 27359, Write: 0, Other 118367

- I/O Transfers Counters -
Read: 0, Write: 3834, Other 0

- Paged Pool Usage -
QuotaPagedPoolUsage: 194504, QuotaPeakPagedPoolUsage: 194504
QuotaNonPagedPoolUsage: 7688, QuotaPeakNonPagedPoolUsage: 7768

- Virtual Memory Usage -
VirtualSize: 105816064, PeakVirtualSize: 105816064

- Pagefile Usage -
PagefileUsage: 2363392, PeakPagefileUsage: 3092480

- Working Set Size -
WorkingSetSize: 78336000, PeakWorkingSetSize: 78872576, PageFaultCount: 20342

*** Dump of thread ID 928 (state: Waiting): ***

- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 1227186.000000

- Unhandled Exception Record -
Reason: Access Violation (0xc0000005) at address 0x76F1E084 read attempt to address 0x746FFFFF

- Registers -
eax=746ffff8 ebx=74700000 ecx=db447d0b edx=005de8b8 esi=005f0000 edi=00000000
eip=76f1e084 esp=0580fe5c ebp=0580fe6c
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010246

- Callstack -
ChildEBP RetAddr  Args to Child
0580fe6c 760514ad 005f0000 00000000 74700000 0580fe94 ntdll!RtlFreeHeap+0x0 
0580fe80 00482c3b 005f0000 00000000 74700000 0580feac kernel32!HeapFree+0x0 
0580fe94 00448311 74700000 00000000 01120020 80000000 wah2_8.24_windows_intelx86!+0x0 
0580feac 0044fa68 01120020 00000000 0051d6b0 0580fefc wah2_8.24_windows_intelx86!+0x0 
0580fec0 004856fc 00000015 dec483f7 00000001 00000006 wah2_8.24_windows_intelx86!+0x0 
0580fefc 760f7651 00000006 dec4f547 00000000 00000000 wah2_8.24_windows_intelx86!+0x0 
0580ff88 7605343d 00000006 0580ffd4 76f29802 80000006 kernel32!CtrlRoutine+0x0 
0580ff94 76f29802 80000006 7332fa08 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 
0580ffd4 76f297d5 760f749c 80000006 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 
0580ffec 00000000 760f749c 80000006 00000000 6272a894 ntdll!RtlInitializeExceptionChain+0x0 

*** Dump of thread ID 6688 (state: Waiting): ***

- Information -
Status: Wait Reason: ExecutionDelay, , Kernel Time: 13437500.000000, User Time: 85781248.000000, Wait Time: 1227176.000000

- Registers -
eax=00000000 ebx=00000041 ecx=00000000 edx=00000000 esi=0018ed80 edi=00000000
eip=76f0fdc1 esp=0018ed3c ebp=0018eda4
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
0018eda4 75ff460c 000000c8 00000000 0018edc8 00468ec1 ntdll!NtDelayExecution+0x0 
0018edb4 00468ec1 000000c8 000000c8 00000000 0018ee00 KERNELBASE!Sleep+0x0 
0018edc8 00407a89 a0000000 027f9999 00000000 004d0c50 wah2_8.24_windows_intelx86!+0x0 
0018ee00 004092c5 00000000 db5c8043 00000000 00000000 wah2_8.24_windows_intelx86!+0x0 
0018fd48 004094b7 0018fef8 00409535 004d1370 0018fd68 wah2_8.24_windows_intelx86!+0x0 
0018fd50 00409535 004d1370 0018fd68 00662718 0018fd68 wah2_8.24_windows_intelx86!+0x0 
0018fef8 00485c7b 00400000 00000000 00662756 0000000a wah2_8.24_windows_intelx86!+0x0 
0018ff88 7605343d 7efde000 0018ffd4 76f29802 7efde000 wah2_8.24_windows_intelx86!+0x0 
0018ff94 76f29802 7efde000 76aafa08 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 
0018ffd4 76f297d5 00485cce 7efde000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 
0018ffec 00000000 00485cce 7efde000 00000000 78746341 ntdll!RtlInitializeExceptionChain+0x0 

*** Dump of thread ID 6676 (state: Waiting): ***

- Information -
Status: Wait Reason: ExecutionDelay, , Kernel Time: 781250.000000, User Time: 625000.000000, Wait Time: 1227182.000000

- Registers -
eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=0111fd24 edi=00000000
eip=76f0fdc1 esp=0111fce0 ebp=0111fd48
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
0111fd48 75ff460c 00000064 00000000 0111ff88 00478b44 ntdll!NtDelayExecution+0x0 
0111fd58 00478b44 00000064 00000000 00000000 00000000 KERNELBASE!Sleep+0x0 
0111ff88 7605343d 00000000 0111ffd4 76f29802 00000000 wah2_8.24_windows_intelx86!+0x0 
0111ff94 76f29802 00000000 77a3fa08 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 
0111ffd4 76f297d5 00478b20 00000000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 
0111ffec 00000000 00478b20 00000000 00000000 32686177 ntdll!RtlInitializeExceptionChain+0x0 


*** Debug Message Dump ****


*** Foreground Window Data ***
    Window Name      : 
    Window Class     : 
    Window Process ID: 0
    Window Thread ID : 0

Exiting...
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6896, selfPID=6160, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6896, selfPID=6896, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5424, selfPID=5944, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5424, selfPID=5424, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5408, selfPID=5408, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5408, selfPID=5516, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5676, selfPID=5776, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5676, selfPID=5676, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6032, selfPID=6032, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6032, selfPID=5528, iMonCtr=1
19:45:49 (3588): BOINC client no longer exists - exiting
19:45:49 (3588): timer handler: client dead, exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5860, selfPID=5860, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5860, selfPID=6872, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3524, selfPID=3524, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3524, selfPID=5128, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
10:46:49 (3572): BOINC client no longer exists - exiting
10:46:49 (3572): timer handler: client dead, exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1864, selfPID=1864, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1864, selfPID=5908, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5884, selfPID=5884, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5884, selfPID=5176, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5884, selfPID=4000, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5884, selfPID=5884, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5888, selfPID=5888, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5888, selfPID=4164, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6820, selfPID=6820, iMonCtr=1
No Process Handle
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6820, selfPID=644, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4160, selfPID=5848, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4160, selfPID=4160, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5720, selfPID=5360, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5720, selfPID=5720, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3116, selfPID=3116, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3116, selfPID=5140, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6136, selfPID=5304, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6136, selfPID=6136, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4584, selfPID=4584, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4584, selfPID=6328, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5712, selfPID=4824, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5712, selfPID=5712, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5792, selfPID=5956, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5792, selfPID=5792, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6132, selfPID=6132, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6132, selfPID=6120, iMonCtr=1
Leaving CPDN_ain::Monitor...
23:03:00 (1092): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
17 Oct 2018 21:02:15 1291402 21315648 wah2_pnw25_ptf5_205509_28_757_011646696_0 322,859 498,886 1.5452
17 Oct 2018 06:36:03 1291402 21315648 wah2_pnw25_ptf5_205509_28_757_011646696_0 311,339 481,246 1.5457
14 Oct 2018 18:25:46 1291402 21315648 wah2_pnw25_ptf5_205509_28_757_011646696_0 299,819 463,625 1.5463
14 Oct 2018 07:51:06 1291402 21315648 wah2_pnw25_ptf5_205509_28_757_011646696_0 288,299 445,864 1.5465
13 Oct 2018 17:20:19 1291402 21315648 wah2_pnw25_ptf5_205509_28_757_011646696_0 276,779 427,811 1.5457
13 Oct 2018 12:28:21 1291402 21315648 wah2_pnw25_ptf5_205509_28_757_011646696_0 265,259 409,930 1.5454
13 Oct 2018 07:18:49 1291402 21315648 wah2_pnw25_ptf5_205509_28_757_011646696_0 253,739 392,493 1.5468
10 Oct 2018 18:35:48 1291402 21315648 wah2_pnw25_ptf5_205509_28_757_011646696_0 242,219 375,095 1.5486
09 Oct 2018 17:39:45 1291402 21315648 wah2_pnw25_ptf5_205509_28_757_011646696_0 230,699 356,382 1.5448
08 Oct 2018 17:34:08 1291402 21315648 wah2_pnw25_ptf5_205509_28_757_011646696_0 219,179 338,979 1.5466
07 Oct 2018 16:37:11 1291402 21315648 wah2_pnw25_ptf5_205509_28_757_011646696_0 207,659 321,236 1.5469
05 Oct 2018 11:01:10 1291402 21315648 wah2_pnw25_ptf5_205509_28_757_011646696_0 196,139 302,534 1.5424
04 Oct 2018 21:04:55 1291402 21315648 wah2_pnw25_ptf5_205509_28_757_011646696_0 184,619 284,193 1.5393
04 Oct 2018 07:13:22 1291402 21315648 wah2_pnw25_ptf5_205509_28_757_011646696_0 173,099 266,585 1.5401
03 Oct 2018 08:51:55 1291402 21315648 wah2_pnw25_ptf5_205509_28_757_011646696_0 161,579 249,362 1.5433
02 Oct 2018 08:12:24 1291402 21315648 wah2_pnw25_ptf5_205509_28_757_011646696_0 150,059 231,637 1.5436
01 Oct 2018 11:08:04 1291402 21315648 wah2_pnw25_ptf5_205509_28_757_011646696_0 138,539 214,314 1.5470
30 Sep 2018 19:50:51 1291402 21315648 wah2_pnw25_ptf5_205509_28_757_011646696_0 127,019 196,729 1.5488
30 Sep 2018 12:47:48 1291402 21315648 wah2_pnw25_ptf5_205509_28_757_011646696_0 115,499 178,991 1.5497
30 Sep 2018 07:50:46 1291402 21315648 wah2_pnw25_ptf5_205509_28_757_011646696_0 103,979 161,454 1.5528


©2024 climateprediction.net