Task 13021759

Name	hadcm3n_t29z_1940_40_007314566_1
Workunit	7511996
Created	28 Jun 2011, 16:33:42 UTC
Sent	28 Jun 2011, 16:49:25 UTC
Report deadline	28 Sep 2011, 0:16:36 UTC
Received	11 May 2012, 22:16:29 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	940299
Run time	50 days 18 hours 5 min 17 sec
CPU time	50 days 18 hours 5 min 17 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	0.71 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>5.8.16</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:20:26 (2920): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... No Process Handle Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1164, iMonCtr=1 Model crash detected, will try to restart... No Process Handle Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1164, iMonCtr=1 Model crash detected, will try to restart... No Process Handle Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1164, iMonCtr=1 Model crash detected, will try to restart... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1924, selfPID=1924, iMonCtr=1 forrtl: There is not enough space on the disk. Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... No Process Handle Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1980, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITHEAD: I/O error tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITHEAD: I/O error tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 08:51:16 (2648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:52:17 (2644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: There is not enough space on the disk. 08:53:17 (2648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: There is not enough space on the disk. 08:54:17 (2644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: There is not enough space on the disk. 08:55:17 (2648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: There is not enough space on the disk. 08:56:17 (2644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: There is not enough space on the disk. 08:57:17 (2648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: There is not enough space on the disk. 08:58:17 (2644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: There is not enough space on the disk. 08:59:17 (2648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: There is not enough space on the disk. 09:00:17 (2644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: There is not enough space on the disk. 09:01:17 (2648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: There is not enough space on the disk. 09:02:17 (2644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: There is not enough space on the disk. 09:03:17 (2648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: There is not enough space on the disk. 09:04:17 (2644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: There is not enough space on the disk. 09:05:17 (2648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: There is not enough space on the disk. 09:06:17 (2644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: There is not enough space on the disk. 09:07:17 (2648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: There is not enough space on the disk. 09:08:17 (2644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: There is not enough space on the disk. C16:22:18 (2648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: There is not enough space on the disk. 16:23:18 (1840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: There is not enough space on the disk. 16:24:18 (2648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: There is not enough space on the disk. 16:25:18 (1840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: There is not enough space on the disk. 16:26:18 (2648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: There is not enough space on the disk. 16:27:18 (1840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: There is not enough space on the disk. 16:28:18 (2648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: There is not enough space on the disk. 16:29:18 (1840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: There is not enough space on the disk. 16:30:18 (2648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: There is not enough space on the disk. 16:31:18 (1840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: There is not enough space on the disk. 16:32:18 (2648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: There is not enough space on the disk. 16:33:18 (2740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: There is not enough space on the disk. 16:34:18 (2648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: There is not enough space on the disk. 16:35:18 (2740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: There is not enough space on the disk. Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:23:21 (2648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:23:27 (2648): No heartbeat from core client for 30 sec - exiting 01:23:29 (2648): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
11 May 2012 20:55:58	940299	13021759	hadcm3n_t29z_1940_40_007314566_1	1,036,800	4,381,899	4.2264
07 May 2012 10:50:05	940299	13021759	hadcm3n_t29z_1940_40_007314566_1	1,010,880	4,164,266	4.1194
03 May 2012 02:18:20	940299	13021759	hadcm3n_t29z_1940_40_007314566_1	984,960	3,945,365	4.0056
28 Apr 2012 18:18:02	940299	13021759	hadcm3n_t29z_1940_40_007314566_1	959,040	3,727,737	3.8869
24 Apr 2012 08:16:06	940299	13021759	hadcm3n_t29z_1940_40_007314566_1	933,120	3,509,655	3.7612
20 Apr 2012 00:47:25	940299	13021759	hadcm3n_t29z_1940_40_007314566_1	907,200	3,289,729	3.6262
19 Feb 2012 04:27:40	940299	13021759	hadcm3n_t29z_1940_40_007314566_1	881,280	3,661,748	4.1550
13 Feb 2012 23:26:25	940299	13021759	hadcm3n_t29z_1940_40_007314566_1	855,360	3,441,105	4.0230
08 Feb 2012 18:29:40	940299	13021759	hadcm3n_t29z_1940_40_007314566_1	829,440	3,218,886	3.8808
31 Jan 2012 14:59:07	940299	13021759	hadcm3n_t29z_1940_40_007314566_1	803,520	2,999,186	3.7326
26 Jan 2012 06:14:26	940299	13021759	hadcm3n_t29z_1940_40_007314566_1	777,600	2,774,740	3.5683
13 Dec 2011 05:14:38	940299	13021759	hadcm3n_t29z_1940_40_007314566_1	751,680	2,552,758	3.3961
20 Nov 2011 00:30:59	940299	13021759	hadcm3n_t29z_1940_40_007314566_1	725,760	6,299,295	8.6796
20 Nov 2011 00:30:59	940299	13021759	hadcm3n_t29z_1940_40_007314566_1	699,840	6,075,324	8.6810
20 Nov 2011 00:30:59	940299	13021759	hadcm3n_t29z_1940_40_007314566_1	673,920	5,851,324	8.6825
20 Nov 2011 00:30:59	940299	13021759	hadcm3n_t29z_1940_40_007314566_1	648,000	5,630,698	8.6893
20 Nov 2011 00:30:59	940299	13021759	hadcm3n_t29z_1940_40_007314566_1	622,080	5,409,814	8.6963
20 Nov 2011 00:30:59	940299	13021759	hadcm3n_t29z_1940_40_007314566_1	596,160	5,187,511	8.7015
20 Nov 2011 00:30:59	940299	13021759	hadcm3n_t29z_1940_40_007314566_1	570,240	4,965,805	8.7083
20 Nov 2011 00:30:59	940299	13021759	hadcm3n_t29z_1940_40_007314566_1	544,320	4,743,728	8.7150