Task 13627757

Name	hadcm3n_y8ww_1940_40_007543747_1
Workunit	7740979
Created	10 Nov 2011, 2:46:39 UTC
Sent	10 Nov 2011, 2:54:02 UTC
Report deadline	9 Feb 2012, 10:21:13 UTC
Received	28 Jan 2012, 1:30:45 UTC
Server state	Over
Outcome	Computation error
Client state	Aborted by user
Exit status	-197 (0xFFFFFF3B) ERR_ABORTED_VIA_GUI
Computer ID	1102379
Run time	8 days 16 hours 56 min 14 sec
CPU time	8 days 7 hours 45 min 13 sec
Validate state	Invalid
Credit	4,354.56
Device peak FLOPS	2.92 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.60</core_client_version> <![CDATA[ <message> aborted by user </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3600, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 00:49:31 (2876): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3336, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3444, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3444, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3444, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3444, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3444, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3444, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3444, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3444, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3444, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3444, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3444, iMonCtr=1 Model crash detected, will try to restart... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/y8wwko.pje4c10 Error converting file to netcdf: dataout/y8wwko.pie4c10 Error converting file to netcdf: dataout/y8wwko.pfe4c10 Error converting file to netcdf: dataout/y8wwka.phe4c10 Error converting file to netcdf: dataout/y8wwka.pge4c10 Error converting file to netcdf: dataout/y8wwka.pee4c10 Error converting file to netcdf: dataout/y8wwka.pde4c10 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3528, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3352, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4872, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4872, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3408, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3688, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3580, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3592, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4308, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3988, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4048, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4200, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3988, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4304, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3288, iMonCtr=1 Model crash detected, will try to restart... CCPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2044, iMonCtr=1 Model crash detected, will try to restart... 16:03:50 (608): No heartbeat from core client for 30 sec - exiting 16:03:51 (608): No heartbeat from core client for 30 sec - exiting 16:03:52 (608): No heartbeat from core client for 30 sec - exiting 16:03:53 (608): No heartbeat from core client for 30 sec - exiting 16:03:54 (608): No heartbeat from core client for 30 sec - exiting 16:03:55 (608): No heartbeat from core client for 30 sec - exiting 16:03:56 (608): No heartbeat from core client for 30 sec - exiting 16:03:57 (608): No heartbeat from core client for 30 sec - exiting 16:03:58 (608): No heartbeat from core client for 30 sec - exiting 16:03:59 (608): No heartbeat from core client for 30 sec - exiting 16:04:01 (608): No heartbeat from core client for 30 sec - exiting 16:04:02 (608): No heartbeat from core client for 30 sec - exiting 16:04:03 (608): No heartbeat from core client for 30 sec - exiting 16:04:04 (608): No heartbeat from core client for 30 sec - exiting 16:04:05 (608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4240, iMonCtr=1 Model crash detected, will try to restart... 19:11:10 (3608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:11:11 (3608): No heartbeat from core client for 30 sec - exiting 19:11:12 (3608): No heartbeat from core client for 30 sec - exiting 19:11:13 (3608): No heartbeat from core client for 30 sec - exiting 19:33:42 (4356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:21:39 (4756): No heartbeat from core client for 30 sec - exiting 23:21:40 (4756): No heartbeat from core client for 30 sec - exiting 23:21:41 (4756): No heartbeat from core client for 30 sec - exiting 23:21:42 (4756): No heartbeat from core client for 30 sec - exiting 23:21:43 (4756): No heartbeat from core client for 30 sec - exiting 23:21:44 (4756): No heartbeat from core client for 30 sec - exiting 23:21:45 (4756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:21:46 (4756): No heartbeat from core client for 30 sec - exiting 23:21:48 (4756): No heartbeat from core client for 30 sec - exiting 20:47:05 (3944): No heartbeat from core client for 30 sec - exiting 20:47:06 (3944): No heartbeat from core client for 30 sec - exiting 20:47:07 (3944): No heartbeat from core client for 30 sec - exiting 20:47:08 (3944): No heartbeat from core client for 30 sec - exiting 20:47:09 (3944): No heartbeat from core client for 30 sec - exiting 20:47:10 (3944): No heartbeat from core client for 30 sec - exiting 20:47:11 (3944): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2676, iMonCtr=1 Model crash detected, will try to restart... 21:01:38 (4428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:14:02 (3464): No heartbeat from core client for 30 sec - exiting 23:14:04 (3464): No heartbeat from core client for 30 sec - exiting 23:14:05 (3464): No heartbeat from core client for 30 sec - exiting 23:14:06 (3464): No heartbeat from core client for 30 sec - exiting 23:14:07 (3464): No heartbeat from core client for 30 sec - exiting 23:14:08 (3464): No heartbeat from core client for 30 sec - exiting 23:14:09 (3464): No heartbeat from core client for 30 sec - exiting 23:14:10 (3464): No heartbeat from core client for 30 sec - exiting 23:14:11 (3464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:35:28 (4784): No heartbeat from core client for 30 sec - exiting 08:35:29 (4784): No heartbeat from core client for 30 sec - exiting 08:35:30 (4784): No heartbeat from core client for 30 sec - exiting 08:35:31 (4784): No heartbeat from core client for 30 sec - exiting 08:35:32 (4784): No heartbeat from core client for 30 sec - exiting 08:35:33 (4784): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:35:34 (4784): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 12:21:42 (3412): No heartbeat from core client for 30 sec - exiting 12:21:43 (3412): No heartbeat from core client for 30 sec - exiting 12:21:44 (3412): No heartbeat from core client for 30 sec - exiting 12:21:45 (3412): No heartbeat from core client for 30 sec - exiting 12:21:46 (3412): No heartbeat from core client for 30 sec - exiting 12:21:47 (3412): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:33:06 (3632): No heartbeat from core client for 30 sec - exiting 18:33:07 (3632): No heartbeat from core client for 30 sec - exiting 18:33:08 (3632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:33:09 (3632): No heartbeat from core client for 30 sec - exiting 11:12:58 (4868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:03:16 (3664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:03:17 (3664): No heartbeat from core client for 30 sec - exiting 23:03:18 (3664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 23:07:50 (4612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:29:50 (4076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:17:30 (3412): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:24:29 (5056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4712, iMonCtr=1 Model crash detected, will try to restart... 08:21:00 (4876): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:11:32 (3804): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:11:33 (3804): No heartbeat from core client for 30 sec - exiting 23:03:44 (4244): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... C08:33:45 (5024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:33:46 (5024): No heartbeat from core client for 30 sec - exiting C14:44:42 (3156): No heartbeat from core client for 30 sec - exiting 14:44:43 (3156): No heartbeat from core client for 30 sec - exiting 14:44:44 (3156): No heartbeat from core client for 30 sec - exiting 14:44:45 (3156): No heartbeat from core client for 30 sec - exiting 14:44:46 (3156): No heartbeat from core client for 30 sec - exiting 14:44:47 (3156): No heartbeat from core client for 30 sec - exiting 14:44:48 (3156): No heartbeat from core client for 30 sec - exiting 14:44:49 (3156): No heartbeat from core client for 30 sec - exiting 14:44:51 (3156): No heartbeat from core client for 30 sec - exiting 14:44:52 (3156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:20:05 (4828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:33:02 (3476): No heartbeat from core client for 30 sec - exiting 09:33:04 (3476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:33:05 (3476): No heartbeat from core client for 30 sec - exiting 17:29:25 (4060): No heartbeat from core client for 30 sec - exiting 17:29:26 (4060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:00:27 (3984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:05:36 (4040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Abort request from BOINC... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
19 Jan 2012 08:26:27	1102379	13627757	hadcm3n_y8ww_1940_40_007543747_1	362,880	681,942	1.8792
11 Jan 2012 13:50:21	1102379	13627757	hadcm3n_y8ww_1940_40_007543747_1	336,960	635,348	1.8855
01 Jan 2012 13:28:25	1102379	13627757	hadcm3n_y8ww_1940_40_007543747_1	311,040	587,165	1.8877
25 Dec 2011 12:19:39	1102379	13627757	hadcm3n_y8ww_1940_40_007543747_1	285,120	538,737	1.8895
08 Dec 2011 05:22:08	1102379	13627757	hadcm3n_y8ww_1940_40_007543747_1	259,200	489,918	1.8901
06 Dec 2011 06:04:01	1102379	13627757	hadcm3n_y8ww_1940_40_007543747_1	233,280	441,228	1.8914
02 Dec 2011 09:07:04	1102379	13627757	hadcm3n_y8ww_1940_40_007543747_1	207,360	390,196	1.8817
28 Nov 2011 07:20:08	1102379	13627757	hadcm3n_y8ww_1940_40_007543747_1	181,440	340,993	1.8794
25 Nov 2011 15:32:40	1102379	13627757	hadcm3n_y8ww_1940_40_007543747_1	155,520	292,870	1.8832
25 Nov 2011 01:23:57	1102379	13627757	hadcm3n_y8ww_1940_40_007543747_1	129,600	244,539	1.8869
23 Nov 2011 00:25:31	1102379	13627757	hadcm3n_y8ww_1940_40_007543747_1	103,680	193,322	1.8646
22 Nov 2011 01:16:59	1102379	13627757	hadcm3n_y8ww_1940_40_007543747_1	77,760	144,725	1.8612
20 Nov 2011 21:15:23	1102379	13627757	hadcm3n_y8ww_1940_40_007543747_1	51,840	96,866	1.8686
18 Nov 2011 19:23:57	1102379	13627757	hadcm3n_y8ww_1940_40_007543747_1	25,920	49,137	1.8957