climateprediction.net home page
Task 11014999

Task 11014999

Name hadsm3dhet2_joy5_006595327_1
Workunit 6798700
Created 15 Mar 2010, 12:01:06 UTC
Sent 4 Oct 2010, 10:41:59 UTC
Report deadline 16 Sep 2011, 16:01:59 UTC
Received 6 Nov 2010, 10:13:14 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 514694
Run time
CPU time 16 days 9 hours 36 min 50 sec
Validate state Invalid
Credit 6,252.33
Device peak FLOPS 1.26 GFLOPS
Application version UK Met Office HadSM3 Slab Model v6.07
windows_intelx86
Stderr
<core_client_version>6.2.18</core_client_version>
<![CDATA[
<message>
The device does not recognize the command. (0x16) - exit code 22 (0x16)
</message>
<stderr_txt>
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Access Violation (0xc0000005) at address 0x00405CC9 write attempt to address 0x000000C0

Engaging BOINC Windows Runtime Debugger...



********************


BOINC Windows Runtime Debugger Version 6.3.8


Dump Timestamp    : 10/07/10 07:27:04
Install Directory : C:\Program Files\BOINC\
Data Directory    : C:\Documents and Settings\All Users\Application Data\BOINC
Project Symstore  : 
Loaded Library    : C:\Program Files\BOINC\\dbghelp.dll
Loaded Library    : C:\Program Files\BOINC\\symsrv.dll
Loaded Library    : C:\Program Files\BOINC\\srcsrv.dll
LoadLibraryA( C:\Program Files\BOINC\\version.dll ): GetLastError = 126
Loaded Library    : version.dll


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Access Violation (0xc0000005) at address 0x00405CC9 write attempt to address 0x000000C0

Engaging BOINC Windows Runtime Debugger...



********************


BOINC Windows Runtime Debugger Version 6.3.8


Dump Timestamp    : 10/07/10 07:37:08
Install Directory : C:\Program Files\BOINC\
Data Directory    : C:\Documents and Settings\All Users\Application Data\BOINC
Project Symstore  : 
Loaded Library    : C:\Program Files\BOINC\\dbghelp.dll
Loaded Library    : C:\Program Files\BOINC\\symsrv.dll
Loaded Library    : C:\Program Files\BOINC\\srcsrv.dll
LoadLibraryA( C:\Program Files\BOINC\\version.dll ): GetLastError = 126
Loaded Library    : version.dll
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
MainError:	12:32:27 AM	No files match the supplied pattern.
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
error:  cannot delete old hadsm3dhet2_joy5_006595327/jobs/climate.spin
error:  cannot delete old hadsm3dhet2_joy5_006595327/jobs/climate.cont
error:  cannot delete old hadsm3dhet2_joy5_006595327/jobs/climate.doub
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
MainError:	02:59:29 PM	No files match the supplied pattern.
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
error:  cannot delete old hadsm3dhet2_joy5_006595327/jobs/climate.doub
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
error:  cannot delete old hadsm3dhet2_joy5_006595327/jobs/climate.spin
error:  cannot delete old hadsm3dhet2_joy5_006595327/jobs/climate.cont
error:  cannot delete old hadsm3dhet2_joy5_006595327/jobs/climate.doub
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

CreateFile error 32 when trying set file time
error:  cannot delete old hadsm3dhet2_joy5_006595327/jobs/climate.cont
error:  cannot delete old hadsm3dhet2_joy5_006595327/jobs/climate.doub
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

Model crashed: (null)

Model crashed: (null)

Model crashed: (null)

Model crashed: (null)

Model crashed: (null)

Model crashed: (null)
Sorry, too many model crashes! :-(
called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
06 Nov 2010 04:46:13 514694 11014999 hadsm3dhet2_joy5_006595327_1 162,030 1,396,764 2.0525
05 Nov 2010 09:56:25 514694 11014999 hadsm3dhet2_joy5_006595327_1 151,228 1,374,492 2.0523
04 Nov 2010 16:11:58 514694 11014999 hadsm3dhet2_joy5_006595327_1 140,426 1,351,845 2.0516
04 Nov 2010 05:01:15 514694 11014999 hadsm3dhet2_joy5_006595327_1 129,624 1,329,403 2.0512
03 Nov 2010 08:34:19 514694 11014999 hadsm3dhet2_joy5_006595327_1 118,822 1,306,899 2.0506
02 Nov 2010 16:48:24 514694 11014999 hadsm3dhet2_joy5_006595327_1 108,020 1,284,333 2.0500
02 Nov 2010 04:54:39 514694 11014999 hadsm3dhet2_joy5_006595327_1 97,218 1,261,796 2.0493
01 Nov 2010 18:52:36 514694 11014999 hadsm3dhet2_joy5_006595327_1 86,416 1,239,311 2.0487
01 Nov 2010 04:53:03 514694 11014999 hadsm3dhet2_joy5_006595327_1 75,614 1,216,358 2.0474
31 Oct 2010 15:09:14 514694 11014999 hadsm3dhet2_joy5_006595327_1 64,812 1,193,289 2.0457
31 Oct 2010 05:16:13 514694 11014999 hadsm3dhet2_joy5_006595327_1 54,010 1,170,831 2.0451
30 Oct 2010 16:52:41 514694 11014999 hadsm3dhet2_joy5_006595327_1 43,208 1,148,673 2.0450
30 Oct 2010 06:32:10 514694 11014999 hadsm3dhet2_joy5_006595327_1 32,406 1,126,331 2.0445
29 Oct 2010 16:00:17 514694 11014999 hadsm3dhet2_joy5_006595327_1 21,604 1,103,685 2.0435
29 Oct 2010 03:35:49 514694 11014999 hadsm3dhet2_joy5_006595327_1 10,802 1,081,324 2.0429
28 Oct 2010 15:04:25 514694 11014999 hadsm3dhet2_joy5_006595327_1 259,248 1,059,084 2.0426
28 Oct 2010 04:28:43 514694 11014999 hadsm3dhet2_joy5_006595327_1 248,446 1,037,113 2.0428
27 Oct 2010 13:23:04 514694 11014999 hadsm3dhet2_joy5_006595327_1 237,644 1,014,741 2.0422
27 Oct 2010 03:25:54 514694 11014999 hadsm3dhet2_joy5_006595327_1 226,842 992,110 2.0410
26 Oct 2010 12:40:31 514694 11014999 hadsm3dhet2_joy5_006595327_1 216,040 970,044 2.0410


©2024 climateprediction.net