climateprediction.net home page
Task 18635076

Task 18635076

Name hadam3p_pnw_phcc_2013_1_009970235_0
Workunit 9976593
Created 29 Jun 2015, 17:04:58 UTC
Sent 4 Jul 2015, 7:38:14 UTC
Report deadline 15 Jun 2016, 12:58:14 UTC
Received 2 Aug 2015, 20:18:38 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1331296
Run time 2 days 14 hours 18 min 25 sec
CPU time 2 days 13 hours 40 min 13 sec
Validate state Workunit error - check skipped
Credit 4,512.18
Device peak FLOPS 3.84 GFLOPS
Application version UK Met Office HadAM3P-HadRM3P Pacific North West v7.27
windows_intelx86
Stderr
<core_client_version>7.2.42</core_client_version>
<![CDATA[
<stderr_txt>


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Access Violation (0xc0000005) at address 0x7784DFE4 read attempt to address 0x686FFFFF

Engaging BOINC Windows Runtime Debugger...



********************


BOINC Windows Runtime Debugger Version 7.5.1


Dump Timestamp    : 07/04/15 13:07:10
Install Directory : 
Data Directory    : G:\ProgramData\BOINC
Project Symstore  : 
LoadLibraryA( G:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126
Loaded Library    : dbghelp.dll
LoadLibraryA( G:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( G:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( G:\ProgramData\BOINC\version.dll ): GetLastError = 126
Loaded Library    : version.dll
Debugger Engine   : 4.0.5.0
Symbol Search Path: G:\ProgramData\BOINC\slots\0;G:\ProgramData\BOINC\projects\climateprediction.net


ModLoad: 0000000000400000 000000000018e000 G:\ProgramData\BOINC\projects\climateprediction.net\hadam3p_pnw_7.27_windows_intelx86.exe (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 0000000077820000 0000000000180000 C:\Windows\SysWOW64\ntdll.dll (6.1.7601.17725) (-exported- Symbols Loaded)
    Linked PDB Filename   : wntdll.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : &#206;&#239;&#229;&#240;&#224;&#246;&#232;&#238;&#237;&#237;&#224;&#255; &#241;&#232;&#241;&#242;&#229;&#236;&#224; Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 0000000075fb0000 0000000000110000 C:\Windows\syswow64\kernel32.dll (6.1.7601.18409) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernel32.pdb
    File Version          : 6.1.7601.17965 (win7sp1_gdr.121004-0333)
    Company Name          : Microsoft Corporation
    Product Name          : &#206;&#239;&#229;&#240;&#224;&#246;&#232;&#238;&#237;&#237;&#224;&#255; &#241;&#232;&#241;&#242;&#229;&#236;&#224; Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7601.17965

ModLoad: 00000000765d0000 0000000000047000 C:\Windows\syswow64\KERNELBASE.dll (6.1.7601.18409) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernelbase.pdb
    File Version          : 6.1.7601.17965 (win7sp1_gdr.121004-0333)
    Company Name          : Microsoft Corporation
    Product Name          : &#206;&#239;&#229;&#240;&#224;&#246;&#232;&#238;&#237;&#237;&#224;&#255; &#241;&#232;&#241;&#242;&#229;&#236;&#224; Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7601.17965

ModLoad: 0000000075440000 0000000000100000 C:\Windows\syswow64\USER32.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : wuser32.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : &#206;&#239;&#229;&#240;&#224;&#246;&#232;&#238;&#237;&#237;&#224;&#255; &#241;&#232;&#241;&#242;&#229;&#236;&#224; Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7601.17514

ModLoad: 0000000076260000 0000000000090000 C:\Windows\syswow64\GDI32.dll (6.1.7601.18577) (-exported- Symbols Loaded)
    Linked PDB Filename   : wgdi32.pdb
    File Version          : 6.1.7601.18577 (win7sp1_gdr.140822-1508)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.18577

ModLoad: 0000000075bc0000 000000000000a000 C:\Windows\syswow64\LPK.dll (6.1.7601.18177) (-exported- Symbols Loaded)
    Linked PDB Filename   : wlpk.pdb
    File Version          : 6.1.7601.18177 (win7sp1_gdr.130605-1534)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.18177

ModLoad: 0000000075f10000 000000000009d000 C:\Windows\syswow64\USP10.dll (1.626.7601.18454) (-exported- Symbols Loaded)
    Linked PDB Filename   : usp10.pdb
    File Version          : 1.0626.7601.18454 (win7sp1_gdr.140424-1533)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft(R) Uniscribe Unicode script processor
    Product Version       : 1.0626.7601.18454

ModLoad: 00000000761b0000 00000000000ac000 C:\Windows\syswow64\msvcrt.dll (7.0.7601.17744) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcrt.pdb
    File Version          : 7.0.7601.17744 (win7sp1_gdr.111215-1535)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 7.0.7601.17744

ModLoad: 00000000762f0000 00000000000a0000 C:\Windows\syswow64\ADVAPI32.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : advapi32.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : &#206;&#239;&#229;&#240;&#224;&#246;&#232;&#238;&#237;&#237;&#224;&#255; &#241;&#232;&#241;&#242;&#229;&#236;&#224; Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 0000000075750000 0000000000019000 C:\Windows\SysWOW64\sechost.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : sechost.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 00000000760c0000 00000000000f0000 C:\Windows\syswow64\RPCRT4.dll (6.1.7601.18532) (-exported- Symbols Loaded)
    Linked PDB Filename   : wrpcrt4.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : &#206;&#239;&#229;&#240;&#224;&#246;&#232;&#238;&#237;&#237;&#224;&#255; &#241;&#232;&#241;&#242;&#229;&#236;&#224; Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 0000000075340000 0000000000060000 C:\Windows\syswow64\SspiCli.dll (6.1.7601.18526) (-exported- Symbols Loaded)
    Linked PDB Filename   : wsspicli.pdb
    File Version          : 6.1.7601.18526 (win7sp1_gdr.140706-1506)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.18526

ModLoad: 0000000075330000 000000000000c000 C:\Windows\syswow64\CRYPTBASE.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : cryptbase.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 0000000075970000 0000000000060000 C:\Windows\system32\IMM32.DLL (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : wimm32.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17514

ModLoad: 0000000076390000 00000000000cc000 C:\Windows\syswow64\MSCTF.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : msctf.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : &#206;&#239;&#229;&#240;&#224;&#246;&#232;&#238;&#237;&#237;&#224;&#255; &#241;&#232;&#241;&#242;&#229;&#236;&#224; Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 0000000070ba0000 0000000000021000 C:\Windows\system32\ntmarta.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : ntmarta.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : &#206;&#239;&#229;&#240;&#224;&#246;&#232;&#238;&#237;&#237;&#224;&#255; &#241;&#232;&#241;&#242;&#229;&#236;&#224; Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 0000000075800000 0000000000045000 C:\Windows\syswow64\WLDAP32.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : wldap32.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : &#206;&#239;&#229;&#240;&#224;&#246;&#232;&#238;&#237;&#237;&#224;&#255; &#241;&#232;&#241;&#242;&#229;&#236;&#224; Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 0000000070b10000 000000000004c000 C:\Windows\system32\apphelp.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : apphelp.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : &#206;&#239;&#229;&#240;&#224;&#246;&#232;&#238;&#237;&#237;&#224;&#255; &#241;&#232;&#241;&#242;&#229;&#236;&#224; Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 00000000707e0000 00000000000eb000 C:\Windows\system32\dbghelp.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : dbghelp.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17514

ModLoad: 0000000073910000 0000000000009000 C:\Windows\system32\version.dll (6.1.7601.17647) (-exported- Symbols Loaded)
    Linked PDB Filename   : version.pdb
    File Version          : 6.1.7601.17647 (win7sp1_gdr.110708-1503)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17647



*** Dump of the Process Statistics: ***

- I/O Operations Counters -
Read: 17949, Write: 0, Other 69424

- I/O Transfers Counters -
Read: 0, Write: 17553, Other 0

- Paged Pool Usage -
QuotaPagedPoolUsage: 100672, QuotaPeakPagedPoolUsage: 100672
QuotaNonPagedPoolUsage: 7808, QuotaPeakNonPagedPoolUsage: 8064

- Virtual Memory Usage -
VirtualSize: 57765888, PeakVirtualSize: 57765888

- Pagefile Usage -
PagefileUsage: 2686976, PeakPagefileUsage: 3444736

- Working Set Size -
WorkingSetSize: 29605888, PeakWorkingSetSize: 30117888, PageFaultCount: 8490

*** Dump of thread ID 5232 (state: Waiting): ***

- Information -
Status: Wait Reason: UserRequest, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 577050.000000

- Unhandled Exception Record -
Reason: Access Violation (0xc0000005) at address 0x7784DFE4 read attempt to address 0x686FFFFF

- Registers -
eax=686ffff8 ebx=68700000 ecx=5057dfb1 edx=003ee8b8 esi=00be0000 edi=00000000
eip=7784dfe4 esp=02b5fe5c ebp=02b5fe6c
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010246

- Callstack -
ChildEBP RetAddr  Args to Child
02b5fe6c 75fc14ad 00be0000 00000000 68700000 02b5fe94 ntdll!RtlFreeHeap+0x0 
02b5fe80 00482de7 00be0000 00000000 68700000 02b5feac kernel32!HeapFree+0x0 
02b5fe94 00447e11 68700000 00000000 012d0020 80000000 hadam3p_pnw_7.27_windows_intelx!+0x0 
02b5feac 0044f568 012d0020 00000000 005597a4 02b5fefc hadam3p_pnw_7.27_windows_intelx!+0x0 
02b5fec0 0048374c 00000015 52e2214d 00000001 00000006 hadam3p_pnw_7.27_windows_intelx!+0x0 
02b5fefc 760673b1 00000006 52e44075 00000000 00000000 hadam3p_pnw_7.27_windows_intelx!+0x0 
02b5ff88 75fc338a 00000006 02b5ffd4 77859ef2 80000006 kernel32!CtrlRoutine+0x0 
02b5ff94 77859ef2 80000006 75358d79 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 
02b5ffd4 77859ec5 760671fc 80000006 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 
02b5ffec 00000000 760671fc 80000006 00000000 2bbbb92c ntdll!RtlInitializeExceptionChain+0x0 

*** Dump of thread ID 3896 (state: Waiting): ***

- Information -
Status: Wait Reason: ExecutionDelay, , Kernel Time: 4368028.000000, User Time: 35256224.000000, Wait Time: 577043.000000

- Registers -
eax=00000000 ebx=004cf841 ecx=00000000 edx=00000000 esi=0018f9d8 edi=00000000
eip=7783fd71 esp=0018f994 ebp=0018f9fc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
0018f9fc 765e44a5 000000c8 00000000 0018fa20 004689c1 ntdll!ZwDelayExecution+0x0 
0018fa0c 004689c1 000000c8 000000c8 00000000 0018fa58 KERNELBASE!Sleep+0x0 
0018fa20 00403ddc a0000000 027f9999 004cf858 004cfb88 hadam3p_pnw_7.27_windows_intelx!+0x0 
0018fa58 00404d76 00000000 504f22f9 00000000 00000000 hadam3p_pnw_7.27_windows_intelx!+0x0 
0018fd48 00404f67 0018fef8 00404fe5 004d02a8 0018fd68 hadam3p_pnw_7.27_windows_intelx!+0x0 
0018fd50 00404fe5 004d02a8 0018fd68 007035c0 0018fd68 hadam3p_pnw_7.27_windows_intelx!+0x0 
0018fef8 00484e1b 00400000 00000000 00703605 0000000a hadam3p_pnw_7.27_windows_intelx!+0x0 
0018ff88 75fc338a 7efde000 0018ffd4 77859ef2 7efde000 hadam3p_pnw_7.27_windows_intelx!+0x0 
0018ff94 77859ef2 7efde000 77988d79 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 
0018ffd4 77859ec5 00484e6e 7efde000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 
0018ffec 00000000 00484e6e 7efde000 00000000 78746341 ntdll!RtlInitializeExceptionChain+0x0 

*** Dump of thread ID 6096 (state: Waiting): ***

- Information -
Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 156001.000000, Wait Time: 577049.000000

- Registers -
eax=c0000022 ebx=00000000 ecx=00000000 edx=00000000 esi=012cfd24 edi=00000000
eip=7783fd71 esp=012cfce0 ebp=012cfd48
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
012cfd48 765e44a5 00000064 00000000 012cff88 00477e94 ntdll!ZwDelayExecution+0x0 
012cfd58 00477e94 00000064 00000000 00000000 00000000 KERNELBASE!Sleep+0x0 
012cff88 75fc338a 00000000 012cffd4 77859ef2 00000000 hadam3p_pnw_7.27_windows_intelx!+0x0 
012cff94 77859ef2 00000000 76ac8d79 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 
012cffd4 77859ec5 00477e70 00000000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 
012cffec 00000000 00477e70 00000000 00000000 61646168 ntdll!RtlInitializeExceptionChain+0x0 


*** Debug Message Dump ****


*** Foreground Window Data ***
    Window Name      : 
    Window Class     : 
    Window Process ID: 0
    Window Thread ID : 0

Exiting...
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5044, selfPID=4224, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5044, selfPID=5044, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5944, selfPID=4772, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5944, selfPID=5944, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5220, selfPID=5408, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5220, selfPID=5220, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5044, selfPID=5044, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5044, selfPID=4320, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6364, selfPID=6364, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6364, selfPID=6508, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4648, selfPID=4648, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4648, selfPID=4300, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6508, selfPID=6508, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6508, selfPID=6656, iMonCtr=1
Glegiobal Workerr:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3696, selfPID=3696, iMonCtr=1

CPDN Monitor - Quit request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3624, selfPID=3624, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3624, selfPID=3640, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4204, selfPID=4204, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4204, selfPID=5160, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5368, selfPID=5484, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5368, selfPID=5368, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2796, selfPID=2796, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2796, selfPID=1484, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6128, selfPID=5480, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6128, selfPID=6128, iMonCtr=1
Suspended CPDN Monitor - Suspend request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6596, selfPID=6596, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6596, selfPID=6768, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2608, selfPID=2608, iMonCtr=2
CPDN Monitor - Quit request from BOINC...
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3580, selfPID=3580, iMonCtr=1
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3580, selfPID=3688, iMonCtr=1
CPDN Monitor - Quit request from BOINC...
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6624, selfPID=6728, iMonCtr=1
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6624, selfPID=6624, iMonCtr=1
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Leaving CPDN_Main::Monitor...
22:16:47 (6424): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
02 Aug 2015 19:17:43 1331296 18635076 hadam3p_pnw_phcc_2013_1_009970235_0 207,659 221,935 1.0687
18 Jul 2015 20:11:08 1331296 18635076 hadam3p_pnw_phcc_2013_1_009970235_0 196,139 209,448 1.0679
17 Jul 2015 20:55:16 1331296 18635076 hadam3p_pnw_phcc_2013_1_009970235_0 184,619 197,432 1.0694
17 Jul 2015 17:24:25 1331296 18635076 hadam3p_pnw_phcc_2013_1_009970235_0 173,099 184,732 1.0672
14 Jul 2015 20:58:18 1331296 18635076 hadam3p_pnw_phcc_2013_1_009970235_0 161,579 172,324 1.0665
13 Jul 2015 18:20:33 1331296 18635076 hadam3p_pnw_phcc_2013_1_009970235_0 150,059 159,926 1.0658
12 Jul 2015 19:40:46 1331296 18635076 hadam3p_pnw_phcc_2013_1_009970235_0 138,539 147,497 1.0647
12 Jul 2015 13:19:14 1331296 18635076 hadam3p_pnw_phcc_2013_1_009970235_0 127,019 135,311 1.0653
11 Jul 2015 22:02:07 1331296 18635076 hadam3p_pnw_phcc_2013_1_009970235_0 115,499 122,798 1.0632
11 Jul 2015 12:34:40 1331296 18635076 hadam3p_pnw_phcc_2013_1_009970235_0 103,979 110,466 1.0624
10 Jul 2015 18:36:51 1331296 18635076 hadam3p_pnw_phcc_2013_1_009970235_0 92,459 98,359 1.0638
09 Jul 2015 20:12:30 1331296 18635076 hadam3p_pnw_phcc_2013_1_009970235_0 80,939 86,175 1.0647
08 Jul 2015 19:40:58 1331296 18635076 hadam3p_pnw_phcc_2013_1_009970235_0 69,419 73,496 1.0587
08 Jul 2015 16:05:02 1331296 18635076 hadam3p_pnw_phcc_2013_1_009970235_0 57,899 60,580 1.0463
07 Jul 2015 18:13:45 1331296 18635076 hadam3p_pnw_phcc_2013_1_009970235_0 46,379 48,540 1.0466
06 Jul 2015 17:55:52 1331296 18635076 hadam3p_pnw_phcc_2013_1_009970235_0 34,859 36,524 1.0478
05 Jul 2015 11:37:27 1331296 18635076 hadam3p_pnw_phcc_2013_1_009970235_0 23,339 24,673 1.0572
04 Jul 2015 21:36:19 1331296 18635076 hadam3p_pnw_phcc_2013_1_009970235_0 11,819 12,495 1.0572


©2024 climateprediction.net