Task 15400999

Name	hadcm3n_o3yc_2100_40_008239789_0
Workunit	8394913
Created	26 Oct 2012, 20:16:42 UTC
Sent	26 Oct 2012, 20:16:50 UTC
Report deadline	26 Jan 2013, 3:44:01 UTC
Received	1 Jan 2013, 4:12:48 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	684701
Run time	53 days 6 hours 7 min 32 sec
CPU time	44 days 6 hours 46 min 10 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	1.35 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <stderr_txt> 01:40:29 (3188): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:40:30 (3188): No heartbeat from core client for 30 sec - exiting 01:40:31 (3188): No heartbeat from core client for 30 sec - exiting 01:40:32 (3188): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:56:49 (2244): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:56:50 (2244): No heartbeat from core client for 30 sec - exiting 14:56:51 (2244): No heartbeat from core client for 30 sec - exiting 14:56:52 (2244): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 04:00:57 (7200): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:00:58 (7200): No heartbeat from core client for 30 sec - exiting 04:00:59 (7200): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:12:05 (3104): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 10:12:08 (3104): No heartbeat from core client for 30 sec - exiting 10:12:09 (3104): No heartbeat from core client for 30 sec - exiting 10:12:10 (3104): No heartbeat from core client for 30 sec - exiting 10:12:12 (3104): No heartbeat from core client for 30 sec - exiting 10:12:13 (3104): No heartbeat from core client for 30 sec - exiting 10:12:14 (3104): No heartbeat from core client for 30 sec - exiting 10:12:15 (3104): No heartbeat from core client for 30 sec - exiting 10:12:16 (3104): No heartbeat from core client for 30 sec - exiting 10:12:17 (3104): No heartbeat from core client for 30 sec - exiting 10:12:18 (3104): No heartbeat from core client for 30 sec - exiting 10:12:19 (3104): No heartbeat from core client for 30 sec - exiting 10:12:20 (3104): No heartbeat from core client for 30 sec - exiting 10:12:21 (3104): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7644, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:11:46 (4284): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:11:49 (4284): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 02:24:07 (4796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:24:08 (4796): No heartbeat from core client for 30 sec - exiting 02:24:09 (4796): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 04:21:53 (5972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:21:54 (5972): No heartbeat from core client for 30 sec - exiting 04:21:55 (5972): No heartbeat from core client for 30 sec - exiting 04:21:56 (5972): No heartbeat from core client for 30 sec - exiting 04:21:57 (5972): No heartbeat from core client for 30 sec - exiting 04:21:58 (5972): No heartbeat from core client for 30 sec - exiting 04:22:00 (5972): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 02:24:44 (4528): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 02:24:46 (4528): No heartbeat from core client for 30 sec - exiting 02:24:47 (4528): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:33:05 (7932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:33:06 (7932): No heartbeat from core client for 30 sec - exiting 09:33:07 (7932): No heartbeat from core client for 30 sec - exiting 09:33:08 (7932): No heartbeat from core client for 30 sec - exiting 09:33:09 (7932): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6828, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:35:25 (4632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:35:26 (4632): No heartbeat from core client for 30 sec - exiting 03:35:27 (4632): No heartbeat from core client for 30 sec - exiting 03:35:28 (4632): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:27:45 (7708): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 06:27:46 (7708): No heartbeat from core client for 30 sec - exiting 06:27:47 (7708): No heartbeat from core client for 30 sec - exiting 06:27:48 (7708): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:33:32 (7572): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 23:33:33 (7572): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:44:55 (6260): No heartbeat from core client for 30 sec - exiting 18:44:56 (6260): No heartbeat from core client for 30 sec - exiting 18:44:57 (6260): No heartbeat from core client for 30 sec - exiting 18:44:58 (6260): No heartbeat from core client for 30 sec - exiting 18:44:59 (6260): No heartbeat from core client for 30 sec - exiting 18:45:00 (6260): No heartbeat from core client for 30 sec - exiting 18:45:01 (6260): No heartbeat from core client for 30 sec - exiting 18:45:02 (6260): No heartbeat from core client for 30 sec - exiting 18:45:03 (6260): No heartbeat from core client for 30 sec - exiting 18:45:04 (6260): No heartbeat from core client for 30 sec - exiting 18:45:05 (6260): No heartbeat from core client for 30 sec - exiting 18:45:06 (6260): No heartbeat from core client for 30 sec - exiting 18:45:07 (6260): No heartbeat from core client for 30 sec - exiting 18:45:08 (6260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/o3ycko.pjw4c10 Error converting file to netcdf: dataout/o3ycko.piw4c10 Error converting file to netcdf: dataout/o3ycko.pfw4c10 Error converting file to netcdf: dataout/o3ycka.phw4c10 Error converting file to netcdf: dataout/o3ycka.pgw4c10 Error converting file to netcdf: dataout/o3ycka.pew4c10 Error converting file to netcdf: dataout/o3ycka.pdw4c10 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 04:26:50 (1712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:26:52 (1712): No heartbeat from core client for 30 sec - exiting 04:26:53 (1712): No heartbeat from core client for 30 sec - exiting 04:26:54 (1712): No heartbeat from core client for 30 sec - exiting 04:26:55 (1712): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 05:24:01 (3872): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 05:24:02 (3872): No heartbeat from core client for 30 sec - exiting 05:24:03 (3872): No heartbeat from core client for 30 sec - exiting 05:24:04 (3872): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:17:35 (8164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:17:36 (8164): No heartbeat from core client for 30 sec - exiting 03:17:37 (8164): No heartbeat from core client for 30 sec - exiting 03:17:38 (8164): No heartbeat from core client for 30 sec - exiting 03:17:39 (8164): No heartbeat from core client for 30 sec - exiting 03:17:40 (8164): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 02:50:30 (7000): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 02:50:31 (7000): No heartbeat from core client for 30 sec - exiting 02:50:32 (7000): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:19:18 (5888): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:19:19 (5888): No heartbeat from core client for 30 sec - exiting 00:19:20 (5888): No heartbeat from core client for 30 sec - exiting 00:19:21 (5888): No heartbeat from core client for 30 sec - exiting 00:19:22 (5888): No heartbeat from core client for 30 sec - exiting 04:23:15 (6756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:23:16 (6756): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7692, iMonCtr=1 Model crash detected, will try to restart... 19:59:26 (6104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 02:21:43 (4624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:21:44 (4624): No heartbeat from core client for 30 sec - exiting 02:21:45 (4624): No heartbeat from core client for 30 sec - exiting 02:21:46 (4624): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8040, iMonCtr=1 Model crash detected, will try to restart... 20:33:43 (2164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:10:20 (4184): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7420, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
31 Dec 2012 04:17:33	684701	15400999	hadcm3n_o3yc_2100_40_008239789_0	1,036,800	3,824,631	3.6889
29 Dec 2012 13:36:41	684701	15400999	hadcm3n_o3yc_2100_40_008239789_0	1,010,880	3,730,508	3.6904
27 Dec 2012 23:47:18	684701	15400999	hadcm3n_o3yc_2100_40_008239789_0	984,960	3,640,492	3.6961
26 Dec 2012 19:38:50	684701	15400999	hadcm3n_o3yc_2100_40_008239789_0	959,040	3,552,711	3.7044
25 Dec 2012 13:30:25	684701	15400999	hadcm3n_o3yc_2100_40_008239789_0	933,120	3,466,017	3.7144
24 Dec 2012 04:26:36	684701	15400999	hadcm3n_o3yc_2100_40_008239789_0	907,200	3,377,582	3.7231
20 Dec 2012 07:36:34	684701	15400999	hadcm3n_o3yc_2100_40_008239789_0	881,280	3,266,624	3.7067
18 Dec 2012 21:03:43	684701	15400999	hadcm3n_o3yc_2100_40_008239789_0	855,360	3,164,085	3.6991
17 Dec 2012 09:29:49	684701	15400999	hadcm3n_o3yc_2100_40_008239789_0	829,440	3,067,668	3.6985
15 Dec 2012 23:22:38	684701	15400999	hadcm3n_o3yc_2100_40_008239789_0	803,520	2,973,240	3.7003
15 Dec 2012 08:33:33	684701	15400999	hadcm3n_o3yc_2100_40_008239789_0	777,600	2,881,365	3.7055
15 Dec 2012 08:33:33	684701	15400999	hadcm3n_o3yc_2100_40_008239789_0	751,680	2,787,506	3.7084
15 Dec 2012 08:33:33	684701	15400999	hadcm3n_o3yc_2100_40_008239789_0	725,760	2,689,345	3.7056
07 Dec 2012 21:33:29	684701	15400999	hadcm3n_o3yc_2100_40_008239789_0	699,840	2,591,991	3.7037
06 Dec 2012 07:28:06	684701	15400999	hadcm3n_o3yc_2100_40_008239789_0	673,920	2,495,860	3.7035
04 Dec 2012 17:04:06	684701	15400999	hadcm3n_o3yc_2100_40_008239789_0	648,000	2,401,060	3.7053
02 Dec 2012 17:55:39	684701	15400999	hadcm3n_o3yc_2100_40_008239789_0	622,080	2,306,004	3.7069
01 Dec 2012 11:11:44	684701	15400999	hadcm3n_o3yc_2100_40_008239789_0	596,160	2,212,240	3.7108
29 Nov 2012 04:16:36	684701	15400999	hadcm3n_o3yc_2100_40_008239789_0	570,240	2,117,220	3.7129
27 Nov 2012 17:00:26	684701	15400999	hadcm3n_o3yc_2100_40_008239789_0	544,320	2,020,589	3.7121