Name | hadsm3dhet2_u2fc_006725879_9 |
Workunit | 6929222 |
Created | 17 Sep 2010, 8:08:41 UTC |
Sent | 20 Sep 2010, 23:35:51 UTC |
Report deadline | 3 Sep 2011, 4:55:51 UTC |
Received | 9 Oct 2010, 23:29:52 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 22 (0x00000016) Unknown error code |
Computer ID | 963734 |
Run time | 11 days 3 hours 32 min 12 sec |
CPU time | 10 days 17 hours 20 min 51 sec |
Validate state | Invalid |
Credit | 6,351.58 |
Device peak FLOPS | 2.63 GFLOPS |
Application version | UK Met Office HadSM3 Slab Model v6.07 windows_intelx86 |
Stderr | <core_client_version>6.6.28</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. MainError: 11:39:35 AM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN process is not running, exiting, bRetVal = 1, checkPID=4000, selfPID=4000, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. MainError: 10:35:15 AM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=6836, selfPID=6836, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4328, iMonCtr=1 Model crash detected, will try to restart... CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4328, iMonCtr=1 Model crash detected, will try to restart... CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4328, iMonCtr=1 Model crash detected, will try to restart... CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4328, iMonCtr=1 Model crash detected, will try to restart... Model crashed: (null) Unhandled Exception Detected... - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7D4E237E Engaging BOINC Windows Runtime Debugger... ******************** BOINC Windows Runtime Debugger Version 6.3.8 Dump Timestamp : 10/08/10 23:24:55 Install Directory : Data Directory : C:\Documents and Settings\All Users\Application Data\BOINC Project Symstore : LoadLibraryA( C:\Documents and Settings\All Users\Application Data\BOINC\dbghelp.dll ): GetLastError = 126 Loaded Library : dbghelp.dll LoadLibraryA( C:\Documents and Settings\All Users\Application Data\BOINC\symsrv.dll ): GetLastError = 126 LoadLibraryA( symsrv.dll ): GetLastError = 126 LoadLibraryA( C:\Documents and Settings\All Users\Application Data\BOINC\srcsrv.dll ): GetLastError = 126 LoadLibraryA( srcsrv.dll ): GetLastError = 126 LoadLibraryA( C:\Documents and Settings\All Users\Application Data\BOINC\version.dll ): GetLastError = 126 Loaded Library : version.dll Debugger Engine : 4.0.5.0 Symbol Search Path: C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net\hadsm3dhet2_u2fc_006725879;C:\Documents and Settings\All Users\Application Data\BOINC\projects\climateprediction.net;srv*C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\symbols*http://msdl.microsoft.com/download/symbols;srv*C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\symbols*http://boinc.berkeley.edu/symstore SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : SymGetModuleInfo(): GetLastError = 87 ModLoad: 00000000 00000000 ( Symbols Loaded) Linked PDB Filename : *** Dump of the Process Statistics: *** - I/O Operations Counters - Read: 2, Write: 0, Other 186 - I/O Transfers Counters - Read: 0, Write: 55, Other 0 - Paged Pool Usage - QuotaPagedPoolUsage: 90320, QuotaPeakPagedPoolUsage: 90320 QuotaNonPagedPoolUsage: 5216, QuotaPeakNonPagedPoolUsage: 5216 - Virtual Memory Usage - VirtualSize: 241008640, PeakVirtualSize: 241008640 - Pagefile Usage - PagefileUsage: 7839744, PeakPagefileUsage: 9437184 - Working Set Size - WorkingSetSize: 3366912, PeakWorkingSetSize: 3366912, PageFaultCount: 837 *** Dump of thread ID 2860 (state: Waiting): *** - Information - Status: Wait Reason: UserRequest, , Kernel Time: 156250.000000, User Time: 0.000000, Wait Time: 123369216.000000 - Unhandled Exception Record - Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x7D4E237E - Registers - eax=0487fccc ebx=00000001 ecx=00000000 edx=00000000 esi=0487fd54 edi=fffffffe eip=7d4e237e esp=0487fcc8 ebp=0487fd1c cs=120023 ss=12002b ds=002b es=002b fs=0053 gs=002b efl=00000206 - Callstack - ChildEBP RetAddr Args to Child 0487fd1c 005c1c17 e06d7363 00000001 00000003 0487fd48 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4e237e' 0487fd54 005c1bce 0487fd64 0069ecb8 0065bb7c 0065b39c !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '005c1c17' 0487fd70 004096c6 00200000 0040bb2b 060a0000 060a3a06 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '005c1bce' 0487ffc0 7d4e7d42 00000000 00000000 7efde000 e06d7363 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '004096c6' 0487fff0 00000000 005c570d 00000000 00000000 00000000 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4e7d42' *** Dump of thread ID 4664 (state: Waiting): *** - Information - Status: Wait Reason: ExecutionDelay, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 123369216.000000 - Registers - eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=00000000 edi=0db5ff74 eip=7d61ccc6 esp=0db5ff38 ebp=0db5ff9c cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202 - Callstack - ChildEBP RetAddr Args to Child 0db5ff9c 7d4d14ef 00000064 00000000 0db5ffb8 004213b4 !NtDelayExecution+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d61ccc6' 0db5ffac 004213b4 00000064 0db5ffec 7d4dfe37 00000000 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4d14ef' 0db5ffb8 7d4dfe37 00000000 00000000 00000000 00000000 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '004213b4' 0db5ffec 00000000 004213a0 00000000 00000000 00000000 !+0x0 SymGetModuleInfo(): GetLastError = '87' Address = '7d4dfe37' *** Debug Message Dump **** *** Foreground Window Data *** Window Name : Window Class : Window Process ID: 0 Window Thread ID : 0 Exiting... CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4328, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( called boinc_finish </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
09 Oct 2010 23:33:33 | 963734 | 11899215 | hadsm3dhet2_u2fc_006725879_9 | 172,832 | 917,453 | 1.3271 |
09 Oct 2010 23:33:33 | 963734 | 11899215 | hadsm3dhet2_u2fc_006725879_9 | 162,030 | 903,215 | 1.3272 |
09 Oct 2010 23:33:33 | 963734 | 11899215 | hadsm3dhet2_u2fc_006725879_9 | 151,228 | 888,980 | 1.3274 |
06 Oct 2010 04:22:19 | 963734 | 11899215 | hadsm3dhet2_u2fc_006725879_9 | 140,426 | 875,425 | 1.3286 |
05 Oct 2010 18:24:25 | 963734 | 11899215 | hadsm3dhet2_u2fc_006725879_9 | 129,624 | 861,801 | 1.3297 |
05 Oct 2010 13:13:07 | 963734 | 11899215 | hadsm3dhet2_u2fc_006725879_9 | 118,822 | 848,147 | 1.3308 |
05 Oct 2010 08:34:27 | 963734 | 11899215 | hadsm3dhet2_u2fc_006725879_9 | 108,020 | 833,102 | 1.3297 |
05 Oct 2010 03:48:10 | 963734 | 11899215 | hadsm3dhet2_u2fc_006725879_9 | 97,218 | 817,833 | 1.3283 |
04 Oct 2010 23:04:34 | 963734 | 11899215 | hadsm3dhet2_u2fc_006725879_9 | 86,416 | 802,585 | 1.3268 |
04 Oct 2010 18:19:14 | 963734 | 11899215 | hadsm3dhet2_u2fc_006725879_9 | 75,614 | 787,374 | 1.3253 |
04 Oct 2010 13:53:04 | 963734 | 11899215 | hadsm3dhet2_u2fc_006725879_9 | 64,812 | 773,181 | 1.3255 |
04 Oct 2010 03:11:03 | 963734 | 11899215 | hadsm3dhet2_u2fc_006725879_9 | 54,010 | 759,391 | 1.3264 |
03 Oct 2010 21:50:21 | 963734 | 11899215 | hadsm3dhet2_u2fc_006725879_9 | 43,208 | 745,722 | 1.3276 |
03 Oct 2010 12:27:02 | 963734 | 11899215 | hadsm3dhet2_u2fc_006725879_9 | 32,406 | 732,113 | 1.3289 |
02 Oct 2010 19:45:41 | 963734 | 11899215 | hadsm3dhet2_u2fc_006725879_9 | 21,604 | 717,278 | 1.3280 |
02 Oct 2010 15:11:26 | 963734 | 11899215 | hadsm3dhet2_u2fc_006725879_9 | 10,802 | 702,716 | 1.3276 |
02 Oct 2010 10:36:04 | 963734 | 11899215 | hadsm3dhet2_u2fc_006725879_9 | 259,248 | 688,136 | 1.3272 |
02 Oct 2010 03:07:36 | 963734 | 11899215 | hadsm3dhet2_u2fc_006725879_9 | 248,446 | 673,705 | 1.3270 |
01 Oct 2010 11:17:46 | 963734 | 11899215 | hadsm3dhet2_u2fc_006725879_9 | 237,644 | 658,892 | 1.3260 |
01 Oct 2010 06:29:16 | 963734 | 11899215 | hadsm3dhet2_u2fc_006725879_9 | 226,842 | 643,630 | 1.3241 |
©2024 climateprediction.net