Task 13318463

Name	hadcm3n_o5ab_1940_40_007432993_1
Workunit	7630496
Created	31 Aug 2011, 20:59:07 UTC
Sent	31 Aug 2011, 21:04:39 UTC
Report deadline	1 Dec 2011, 4:31:50 UTC
Received	28 Sep 2011, 11:26:14 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1120434
Run time	22 days 13 hours 25 min 53 sec
CPU time	22 days 8 hours 25 min 16 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	3.08 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7144, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/o5abko.pjh1c10 Error converting file to netcdf: dataout/o5abko.pih1c10 Error converting file to netcdf: dataout/o5abko.pfh1c10 Error converting file to netcdf: dataout/o5abka.phh1c10 Error converting file to netcdf: dataout/o5abka.pgh1c10 Error converting file to netcdf: dataout/o5abka.peh1c10 Error converting file to netcdf: dataout/o5abka.pdh1c10 07:41:54 (3076): No heartbeat from core client for 30 sec - exiting 07:41:55 (3076): No heartbeat from core client for 30 sec - exiting 07:41:56 (3076): No heartbeat from core client for 30 sec - exiting 07:41:57 (3076): No heartbeat from core client for 30 sec - exiting 07:41:58 (3076): No heartbeat from core client for 30 sec - exiting 07:41:59 (3076): No heartbeat from core client for 30 sec - exiting 07:42:00 (3076): No heartbeat from core client for 30 sec - exiting 07:42:01 (3076): No heartbeat from core client for 30 sec - exiting 07:42:02 (3076): No heartbeat from core client for 30 sec - exiting 07:42:03 (3076): No heartbeat from core client for 30 sec - exiting 07:42:04 (3076): No heartbeat from core client for 30 sec - exiting 07:42:05 (3076): No heartbeat from core client for 30 sec - exiting 07:42:06 (3076): No heartbeat from core client for 30 sec - exiting 07:42:07 (3076): No heartbeat from core client for 30 sec - exiting 07:42:08 (3076): No heartbeat from core client for 30 sec - exiting 07:42:09 (3076): No heartbeat from core client for 30 sec - exiting 07:42:10 (3076): No heartbeat from core client for 30 sec - exiting 07:42:11 (3076): No heartbeat from core client for 30 sec - exiting 07:42:12 (3076): No heartbeat from core client for 30 sec - exiting 07:42:13 (3076): No heartbeat from core client for 30 sec - exiting 07:42:14 (3076): No heartbeat from core client for 30 sec - exiting 07:42:15 (3076): No heartbeat from core client for 30 sec - exiting 07:42:16 (3076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:42:17 (3076): No heartbeat from core client for 30 sec - exiting BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/o5abko.pjh1c10 Error converting file to netcdf: dataout/o5abko.pih1c10 Error converting file to netcdf: dataout/o5abko.pfh1c10 Error converting file to netcdf: dataout/o5abka.phh1c10 Error converting file to netcdf: dataout/o5abka.pgh1c10 Error converting file to netcdf: dataout/o5abka.peh1c10 Error converting file to netcdf: dataout/o5abka.pdh1c10 07:44:13 (6988): No heartbeat from core client for 30 sec - exiting 07:44:14 (6988): No heartbeat from core client for 30 sec - exiting 07:44:15 (6988): No heartbeat from core client for 30 sec - exiting 07:44:16 (6988): No heartbeat from core client for 30 sec - exiting 07:44:17 (6988): No heartbeat from core client for 30 sec - exiting 07:44:18 (6988): No heartbeat from core client for 30 sec - exiting 07:44:19 (6988): No heartbeat from core client for 30 sec - exiting 07:44:20 (6988): No heartbeat from core client for 30 sec - exiting 07:44:21 (6988): No heartbeat from core client for 30 sec - exiting 07:44:22 (6988): No heartbeat from core client for 30 sec - exiting 07:44:23 (6988): No heartbeat from core client for 30 sec - exiting 07:44:24 (6988): No heartbeat from core client for 30 sec - exiting 07:44:25 (6988): No heartbeat from core client for 30 sec - exiting 07:44:26 (6988): No heartbeat from core client for 30 sec - exiting 07:44:27 (6988): No heartbeat from core client for 30 sec - exiting 07:44:28 (6988): No heartbeat from core client for 30 sec - exiting 07:44:29 (6988): No heartbeat from core client for 30 sec - exiting 07:44:30 (6988): No heartbeat from core client for 30 sec - exiting 07:44:31 (6988): No heartbeat from core client for 30 sec - exiting 07:44:32 (6988): No heartbeat from core client for 30 sec - exiting 07:44:33 (6988): No heartbeat from core client for 30 sec - exiting 07:44:34 (6988): No heartbeat from core client for 30 sec - exiting 07:44:35 (6988): No heartbeat from core client for 30 sec - exiting 07:44:36 (6988): No heartbeat from core client for 30 sec - exiting 07:44:37 (6988): No heartbeat from core client for 30 sec - exiting 07:44:38 (6988): No heartbeat from core client for 30 sec - exiting 07:44:39 (6988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/o5abko.pjh1c10 Error converting file to netcdf: dataout/o5abko.pih1c10 Error converting file to netcdf: dataout/o5abko.pfh1c10 Error converting file to netcdf: dataout/o5abka.phh1c10 Error converting file to netcdf: dataout/o5abka.pgh1c10 Error converting file to netcdf: dataout/o5abka.peh1c10 Error converting file to netcdf: dataout/o5abka.pdh1c10 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5796, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5796, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5796, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5796, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5796, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5796, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5796, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5796, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5796, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5796, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5796, iMonCtr=1 Model crash detected, will try to restart... 07:51:54 (4060): No heartbeat from core client for 30 sec - exiting 07:51:55 (4060): No heartbeat from core client for 30 sec - exiting 07:51:56 (4060): No heartbeat from core client for 30 sec - exiting 07:51:57 (4060): No heartbeat from core client for 30 sec - exiting 07:51:58 (4060): No heartbeat from core client for 30 sec - exiting 07:51:59 (4060): No heartbeat from core client for 30 sec - exiting 07:52:00 (4060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/o5abko.pjh1c10 Error converting file to netcdf: dataout/o5abko.pih1c10 Error converting file to netcdf: dataout/o5abko.pfh1c10 Error converting file to netcdf: dataout/o5abka.phh1c10 Error converting file to netcdf: dataout/o5abka.pgh1c10 Error converting file to netcdf: dataout/o5abka.peh1c10 Error converting file to netcdf: dataout/o5abka.pdh1c10 09:58:24 (1356): No heartbeat from core client for 30 sec - exiting 09:58:25 (1356): No heartbeat from core client for 30 sec - exiting 09:58:26 (1356): No heartbeat from core client for 30 sec - exiting 09:58:27 (1356): No heartbeat from core client for 30 sec - exiting 09:58:28 (1356): No heartbeat from core client for 30 sec - exiting 09:58:29 (1356): No heartbeat from core client for 30 sec - exiting 09:58:30 (1356): No heartbeat from core client for 30 sec - exiting 09:58:31 (1356): No heartbeat from core client for 30 sec - exiting 09:58:32 (1356): No heartbeat from core client for 30 sec - exiting 09:58:33 (1356): No heartbeat from core client for 30 sec - exiting 09:58:34 (1356): No heartbeat from core client for 30 sec - exiting 09:58:35 (1356): No heartbeat from core client for 30 sec - exiting 09:58:36 (1356): No heartbeat from core client for 30 sec - exiting 09:58:37 (1356): No heartbeat from core client for 30 sec - exiting 09:58:38 (1356): No heartbeat from core client for 30 sec - exiting 09:58:39 (1356): No heartbeat from core client for 30 sec - exiting 09:58:40 (1356): No heartbeat from core client for 30 sec - exiting 09:58:41 (1356): No heartbeat from core client for 30 sec - exiting 09:58:42 (1356): No heartbeat from core client for 30 sec - exiting 09:58:43 (1356): No heartbeat from core client for 30 sec - exiting 09:58:44 (1356): No heartbeat from core client for 30 sec - exiting 09:58:45 (1356): No heartbeat from core client for 30 sec - exiting 09:58:46 (1356): No heartbeat from core client for 30 sec - exiting 09:58:47 (1356): No heartbeat from core client for 30 sec - exiting 09:58:48 (1356): No heartbeat from core client for 30 sec - exiting 09:58:49 (1356): No heartbeat from core client for 30 sec - exiting 09:58:50 (1356): No heartbeat from core client for 30 sec - exiting 09:58:51 (1356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
28 Sep 2011 11:26:26	1120434	13318463	hadcm3n_o5ab_1940_40_007432993_1	1,036,800	1,930,435	1.8619
27 Sep 2011 22:06:20	1120434	13318463	hadcm3n_o5ab_1940_40_007432993_1	1,010,880	1,880,440	1.8602
27 Sep 2011 07:07:01	1120434	13318463	hadcm3n_o5ab_1940_40_007432993_1	984,960	1,830,729	1.8587
26 Sep 2011 17:49:50	1120434	13318463	hadcm3n_o5ab_1940_40_007432993_1	959,040	1,781,025	1.8571
26 Sep 2011 03:11:04	1120434	13318463	hadcm3n_o5ab_1940_40_007432993_1	933,120	1,731,795	1.8559
25 Sep 2011 13:18:27	1120434	13318463	hadcm3n_o5ab_1940_40_007432993_1	907,200	1,682,609	1.8547
24 Sep 2011 23:33:26	1120434	13318463	hadcm3n_o5ab_1940_40_007432993_1	881,280	1,633,402	1.8534
24 Sep 2011 09:42:05	1120434	13318463	hadcm3n_o5ab_1940_40_007432993_1	855,360	1,584,200	1.8521
23 Sep 2011 09:39:28	1120434	13318463	hadcm3n_o5ab_1940_40_007432993_1	829,440	1,535,072	1.8507
22 Sep 2011 19:58:07	1120434	13318463	hadcm3n_o5ab_1940_40_007432993_1	803,520	1,486,068	1.8494
21 Sep 2011 09:21:52	1120434	13318463	hadcm3n_o5ab_1940_40_007432993_1	777,600	1,435,893	1.8466
20 Sep 2011 02:38:20	1120434	13318463	hadcm3n_o5ab_1940_40_007432993_1	751,680	1,385,507	1.8432
19 Sep 2011 08:47:47	1120434	13318463	hadcm3n_o5ab_1940_40_007432993_1	725,760	1,336,322	1.8413
18 Sep 2011 18:10:39	1120434	13318463	hadcm3n_o5ab_1940_40_007432993_1	699,840	1,287,629	1.8399
18 Sep 2011 04:42:25	1120434	13318463	hadcm3n_o5ab_1940_40_007432993_1	673,920	1,240,184	1.8403
17 Sep 2011 15:38:44	1120434	13318463	hadcm3n_o5ab_1940_40_007432993_1	648,000	1,193,350	1.8416
17 Sep 2011 00:41:09	1120434	13318463	hadcm3n_o5ab_1940_40_007432993_1	622,080	1,145,991	1.8422
16 Sep 2011 04:20:40	1120434	13318463	hadcm3n_o5ab_1940_40_007432993_1	596,160	1,097,679	1.8412
15 Sep 2011 13:48:32	1120434	13318463	hadcm3n_o5ab_1940_40_007432993_1	570,240	1,049,362	1.8402
15 Sep 2011 00:04:40	1120434	13318463	hadcm3n_o5ab_1940_40_007432993_1	544,320	1,000,505	1.8381