Task 15592469

Name	hadcm3n_4drk_1940_40_008306201_0
Workunit	8457336
Created	7 Feb 2013, 9:00:21 UTC
Sent	7 Feb 2013, 9:04:53 UTC
Report deadline	9 May 2013, 16:32:04 UTC
Received	30 Mar 2013, 18:36:55 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1356796
Run time	19 days 19 hours 47 min 9 sec
CPU time	18 days 17 hours 27 min 52 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	2.64 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2808, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2808, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2808, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2808, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2808, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2808, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 22:10:02 (4464): No heartbeat from core client for 30 sec - exiting 22:10:03 (4464): No heartbeat from core client for 30 sec - exiting 22:10:04 (4464): No heartbeat from core client for 30 sec - exiting 22:10:05 (4464): No heartbeat from core client for 30 sec - exiting 22:10:06 (4464): No heartbeat from core client for 30 sec - exiting 22:10:07 (4464): No heartbeat from core client for 30 sec - exiting 22:10:08 (4464): No heartbeat from core client for 30 sec - exiting 22:10:09 (4464): No heartbeat from core client for 30 sec - exiting 22:10:10 (4464): No heartbeat from core client for 30 sec - exiting 22:10:11 (4464): No heartbeat from core client for 30 sec - exiting 22:10:12 (4464): No heartbeat from core client for 30 sec - exiting 22:10:13 (4464): No heartbeat from core client for 30 sec - exiting 22:10:14 (4464): No heartbeat from core client for 30 sec - exiting 22:10:15 (4464): No heartbeat from core client for 30 sec - exiting 22:10:16 (4464): No heartbeat from core client for 30 sec - exiting 22:10:17 (4464): No heartbeat from core client for 30 sec - exiting 22:10:18 (4464): No heartbeat from core client for 30 sec - exiting 22:10:19 (4464): No heartbeat from core client for 30 sec - exiting 22:10:20 (4464): No heartbeat from core client for 30 sec - exiting 22:10:21 (4464): No heartbeat from core client for 30 sec - exiting 22:10:22 (4464): No heartbeat from core client for 30 sec - exiting 22:10:23 (4464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 forrtl: There is not enough space on the disk. Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 Suspended CPDN Monitor - Suspend request from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
30 Mar 2013 16:45:52	1209938	15592469	hadcm3n_4drk_1940_40_008306201_0	1,036,800	1,618,070	1.5606
30 Mar 2013 06:18:37	1209938	15592469	hadcm3n_4drk_1940_40_008306201_0	1,010,880	1,581,388	1.5644
28 Mar 2013 14:20:35	1209938	15592469	hadcm3n_4drk_1940_40_008306201_0	984,960	1,545,231	1.5688
28 Mar 2013 03:57:03	1209938	15592469	hadcm3n_4drk_1940_40_008306201_0	959,040	1,508,963	1.5734
27 Mar 2013 18:17:06	1209938	15592469	hadcm3n_4drk_1940_40_008306201_0	933,120	1,472,402	1.5779
27 Mar 2013 06:36:33	1209938	15592469	hadcm3n_4drk_1940_40_008306201_0	907,200	1,435,845	1.5827
26 Mar 2013 19:31:41	1209938	15592469	hadcm3n_4drk_1940_40_008306201_0	881,280	1,399,472	1.5880
26 Mar 2013 08:51:11	1209938	15592469	hadcm3n_4drk_1940_40_008306201_0	855,360	1,363,173	1.5937
22 Mar 2013 11:14:46	1209938	15592469	hadcm3n_4drk_1940_40_008306201_0	829,440	1,326,873	1.5997
21 Mar 2013 23:46:08	1209938	15592469	hadcm3n_4drk_1940_40_008306201_0	803,520	1,291,098	1.6068
21 Mar 2013 13:30:01	1209938	15592469	hadcm3n_4drk_1940_40_008306201_0	777,600	1,255,221	1.6142
21 Mar 2013 02:17:33	1209938	15592469	hadcm3n_4drk_1940_40_008306201_0	751,680	1,218,972	1.6217
20 Mar 2013 15:40:32	1209938	15592469	hadcm3n_4drk_1940_40_008306201_0	725,760	1,182,695	1.6296
20 Mar 2013 05:21:12	1209938	15592469	hadcm3n_4drk_1940_40_008306201_0	699,840	1,147,138	1.6391
19 Mar 2013 19:40:32	1209938	15592469	hadcm3n_4drk_1940_40_008306201_0	673,920	1,111,645	1.6495
19 Mar 2013 08:44:01	1209938	15592469	hadcm3n_4drk_1940_40_008306201_0	648,000	1,076,223	1.6608
18 Mar 2013 22:11:18	1209938	15592469	hadcm3n_4drk_1940_40_008306201_0	622,080	1,041,137	1.6736
18 Mar 2013 12:24:49	1209938	15592469	hadcm3n_4drk_1940_40_008306201_0	596,160	1,006,159	1.6877
17 Mar 2013 04:07:58	1209938	15592469	hadcm3n_4drk_1940_40_008306201_0	570,240	971,002	1.7028
16 Mar 2013 18:01:33	1209938	15592469	hadcm3n_4drk_1940_40_008306201_0	544,320	936,067	1.7197