Task 14379064

Name	hadcm3n_o4gv_2020_40_007857331_3
Workunit	8012443
Created	8 Apr 2012, 2:00:18 UTC
Sent	8 Apr 2012, 2:00:22 UTC
Report deadline	8 Jul 2012, 9:27:33 UTC
Received	21 Jul 2012, 3:28:05 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1059924
Run time	33 days 14 hours 10 min
CPU time	28 days 9 hours 38 min 54 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	1.92 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.36</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3372, iMonCtr=1 Model crash detected, will try to restart... 05:48:50 (3456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:26:57 (3416): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:28:59 (3944): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:30:53 (4012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:34:03 (5684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3544, iMonCtr=1 Model crash detected, will try to restart... 23:39:38 (3256): No heartbeat from core client for 30 sec - exiting 23:39:39 (3256): No heartbeat from core client for 30 sec - exiting 23:39:40 (3256): No heartbeat from core client for 30 sec - exiting 23:39:42 (3256): No heartbeat from core client for 30 sec - exiting 23:39:43 (3256): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 23:10:43 (940): No heartbeat from core client for 30 sec - exiting 23:10:45 (940): No heartbeat from core client for 30 sec - exiting 23:10:46 (940): No heartbeat from core client for 30 sec - exiting 23:10:47 (940): No heartbeat from core client for 30 sec - exiting 23:10:48 (940): No heartbeat from core client for 30 sec - exiting 23:10:49 (940): No heartbeat from core client for 30 sec - exiting 23:10:50 (940): No heartbeat from core client for 30 sec - exiting 23:10:51 (940): No heartbeat from core client for 30 sec - exiting 23:10:52 (940): No heartbeat from core client for 30 sec - exiting 23:10:54 (940): No heartbeat from core client for 30 sec - exiting 23:10:55 (940): No heartbeat from core client for 30 sec - exiting 23:10:56 (940): No heartbeat from core client for 30 sec - exiting 23:10:57 (940): No heartbeat from core client for 30 sec - exiting 23:10:58 (940): No heartbeat from core client for 30 sec - exiting 23:10:59 (940): No heartbeat from core client for 30 sec - exiting 23:11:00 (940): No heartbeat from core client for 30 sec - exiting 23:11:01 (940): No heartbeat from core client for 30 sec - exiting 23:11:02 (940): No heartbeat from core client for 30 sec - exiting 23:11:03 (940): No heartbeat from core client for 30 sec - exiting 23:11:04 (940): No heartbeat from core client for 30 sec - exiting 23:11:05 (940): No heartbeat from core client for 30 sec - exiting 23:11:07 (940): No heartbeat from core client for 30 sec - exiting 23:11:08 (940): No heartbeat from core client for 30 sec - exiting 23:11:09 (940): No heartbeat from core client for 30 sec - exiting 23:11:10 (940): No heartbeat from core client for 30 sec - exiting 23:11:12 (940): No heartbeat from core client for 30 sec - exiting 23:11:13 (940): No heartbeat from core client for 30 sec - exiting 23:11:14 (940): No heartbeat from core client for 30 sec - exiting 23:11:17 (940): No heartbeat from core client for 30 sec - exiting 23:11:19 (940): No heartbeat from core client for 30 sec - exiting 23:11:20 (940): No heartbeat from core client for 30 sec - exiting 23:11:21 (940): No heartbeat from core client for 30 sec - exiting 23:11:22 (940): No heartbeat from core client for 30 sec - exiting 23:11:23 (940): No heartbeat from core client for 30 sec - exiting 23:11:24 (940): No heartbeat from core client for 30 sec - exiting 23:11:25 (940): No heartbeat from core client for 30 sec - exiting 23:11:26 (940): No heartbeat from core client for 30 sec - exiting 23:11:27 (940): No heartbeat from core client for 30 sec - exiting 23:11:28 (940): No heartbeat from core client for 30 sec - exiting 23:11:29 (940): No heartbeat from core client for 30 sec - exiting 23:11:30 (940): No heartbeat from core client for 30 sec - exiting 23:11:31 (940): No heartbeat from core client for 30 sec - exiting 23:11:33 (940): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:11:34 (940): No heartbeat from core client for 30 sec - exiting CCPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3652, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 06:26:21 (3520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:22:52 (3572): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:22:53 (3572): No heartbeat from core client for 30 sec - exiting 22:22:54 (3572): No heartbeat from core client for 30 sec - exiting 22:22:55 (3572): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4952, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 10:31:26 (1992): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:36:24 (1540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:36:26 (1540): No heartbeat from core client for 30 sec - exiting 18:36:27 (1540): No heartbeat from core client for 30 sec - exiting 18:36:28 (1540): No heartbeat from core client for 30 sec - exiting 18:38:07 (11588): No heartbeat from core client for 30 sec - exiting 18:38:08 (11588): No heartbeat from core client for 30 sec - exiting 18:38:09 (11588): No heartbeat from core client for 30 sec - exiting 18:38:10 (11588): No heartbeat from core client for 30 sec - exiting 18:38:11 (11588): No heartbeat from core client for 30 sec - exiting 18:38:12 (11588): No heartbeat from core client for 30 sec - exiting 18:38:13 (11588): No heartbeat from core client for 30 sec - exiting 18:38:14 (11588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=868, iMonCtr=1 Model crash detected, will try to restart... 21:01:42 (3620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:37:11 (3484): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:55:29 (5448): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:55:31 (5448): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3516, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2868, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2868, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2868, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2868, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2868, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2868, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3340, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3300, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3016, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 06:08:04 (5052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4804, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3080, iMonCtr=1 Model crash detected, will try to restart... 06:49:32 (3196): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:49:33 (3196): No heartbeat from core client for 30 sec - exiting 06:49:35 (3196): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5812, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 05:50:57 (6120): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:02:09 (4768): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:02:11 (4768): No heartbeat from core client for 30 sec - exiting 00:02:12 (4768): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:45:00 (5524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:45:02 (5524): No heartbeat from core client for 30 sec - exiting 00:45:03 (5524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3932, iMonCtr=1 Model crash detected, will try16:43:21 (3280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:57:20 (3520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2572, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 21:26:05 (1660): No heartbeat from core client for 30 sec - exiting 21:26:06 (1660): No heartbeat from core client for 30 sec - exiting 21:26:07 (1660): No heartbeat from core client for 30 sec - exiting 21:26:08 (1660): No heartbeat from core client for 30 sec - exiting 21:26:09 (1660): No heartbeat from core client for 30 sec - exiting 21:26:10 (1660): No heartbeat from core client for 30 sec - exiting 21:26:11 (1660): No heartbeat from core client for 30 sec - exiting 21:26:12 (1660): No heartbeat from core client for 30 sec - exiting 21:26:13 (1660): No heartbeat from core client for 30 sec - exiting 21:26:14 (1660): No heartbeat from core client for 30 sec - exiting 21:26:16 (1660): No heartbeat from core client for 30 sec - exiting 21:26:17 (1660): No heartbeat from core client for 30 sec - exiting 21:26:18 (1660): No heartbeat from core client for 30 sec - exiting 21:26:19 (1660): No heartbeat from core client for 30 sec - exiting 21:26:20 (1660): No heartbeat from core client for 30 sec - exiting 21:26:21 (1660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:14:45 (5348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:14:49 (5348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 06:45:03 (4292): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:04:10 (3872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4060, iMonCtr=1 Model crash detected, will try to restart... 21:12:55 (3540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:19:26 (568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:22:07 (656): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:25:01 (3640): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:28:47 (3332): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:32:35 (3012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:35:54 (1792): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:38:39 (2532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:41:13 (648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:51:48 (3352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:51:49 (3352): No heartbeat from core client for 30 sec - exiting 21:51:50 (3352): No heartbeat from core client for 30 sec - exiting 21:51:51 (3352): No heartbeat from core client for 30 sec - exiting 21:52:51 (4988): No heartbeat from core client for 30 sec - exiting 21:52:52 (4988): No heartbeat from core client for 30 sec - exiting 21:52:53 (4988): No heartbeat from core client for 30 sec - exiting 21:52:54 (4988): No heartbeat from core client for 30 sec - exiting 21:52:56 (4988): No heartbeat from core client for 30 sec - exiting 21:52:57 (4988): No heartbeat from core client for 30 sec - exiting 21:52:59 (4988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:34:38 (3592): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3360, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3360, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3260, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3216, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 00:12:41 (5096): No heartbeat from core client for 30 sec - exiting 00:12:42 (5096): No heartbeat from core client for 30 sec - exiting 00:12:43 (5096): No heartbeat from core client for 30 sec - exiting 00:12:44 (5096): No heartbeat from core client for 30 sec - exiting 00:12:45 (5096): No heartbeat from core client for 30 sec - exiting 00:12:46 (5096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:36:54 (3596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:37:44 (4280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:08:16 (904): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:50:36 (2024): No heartbeat from core client for 30 sec - exiting 23:50:39 (2024): No heartbeat from core client for 30 sec - exiting 23:50:41 (2024): No heartbeat from core client for 30 sec - exiting 23:50:42 (2024): No heartbeat from core client for 30 sec - exiting 23:50:43 (2024): No heartbeat from core client for 30 sec - exiting 23:50:44 (2024): No heartbeat from core client for 30 sec - exiting 23:50:46 (2024): No heartbeat from core client for 30 sec - exiting 23:50:47 (2024): No heartbeat from core client for 30 sec - exiting 23:50:48 (2024): No heartbeat from core client for 30 sec - exiting 23:50:49 (2024): No heartbeat from core client for 30 sec - exiting 23:50:50 (2024): No heartbeat from core client for 30 sec - exiting 23:50:51 (2024): No heartbeat from core client for 30 sec - exiting 23:50:53 (2024): No heartbeat from core client for 30 sec - exiting 23:50:54 (2024): No heartbeat from core client for 30 sec - exiting 23:50:55 (2024): No heartbeat from core client for 30 sec - exiting 23:50:56 (2024): No heartbeat from core client for 30 sec - exiting 23:50:57 (2024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:50:58 (2024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
21 Jul 2012 03:30:18	1059924	14379064	hadcm3n_o4gv_2020_40_007857331_3	1,036,800	2,453,931	2.3668
19 Jul 2012 17:08:48	1059924	14379064	hadcm3n_o4gv_2020_40_007857331_3	1,010,880	2,397,805	2.3720
17 Jul 2012 13:06:13	1059924	14379064	hadcm3n_o4gv_2020_40_007857331_3	984,960	2,340,275	2.3760
15 Jul 2012 16:18:54	1059924	14379064	hadcm3n_o4gv_2020_40_007857331_3	959,040	2,283,651	2.3812
14 Jul 2012 21:49:48	1059924	14379064	hadcm3n_o4gv_2020_40_007857331_3	933,120	2,225,710	2.3852
14 Jul 2012 00:53:30	1059924	14379064	hadcm3n_o4gv_2020_40_007857331_3	907,200	2,166,320	2.3879
12 Jul 2012 19:55:38	1059924	14379064	hadcm3n_o4gv_2020_40_007857331_3	881,280	2,108,423	2.3925
10 Jul 2012 19:34:41	1059924	14379064	hadcm3n_o4gv_2020_40_007857331_3	855,360	2,049,918	2.3966
08 Jul 2012 12:05:34	1059924	14379064	hadcm3n_o4gv_2020_40_007857331_3	829,440	1,992,208	2.4019
07 Jul 2012 16:27:17	1059924	14379064	hadcm3n_o4gv_2020_40_007857331_3	803,520	1,934,424	2.4074
06 Jul 2012 22:38:21	1059924	14379064	hadcm3n_o4gv_2020_40_007857331_3	777,600	1,877,439	2.4144
04 Jul 2012 23:06:40	1059924	14379064	hadcm3n_o4gv_2020_40_007857331_3	751,680	1,822,010	2.4239
02 Jul 2012 22:11:04	1059924	14379064	hadcm3n_o4gv_2020_40_007857331_3	725,760	1,764,798	2.4317
02 Jul 2012 17:16:31	1059924	14379064	hadcm3n_o4gv_2020_40_007857331_3	699,840	1,707,376	2.4397
30 Jun 2012 05:29:12	1059924	14379064	hadcm3n_o4gv_2020_40_007857331_3	673,920	1,649,215	2.4472
28 Jun 2012 16:37:01	1059924	14379064	hadcm3n_o4gv_2020_40_007857331_3	648,000	1,593,138	2.4585
26 Jun 2012 15:47:36	1059924	14379064	hadcm3n_o4gv_2020_40_007857331_3	622,080	1,535,701	2.4687
12 Jun 2012 22:41:07	1059924	14379064	hadcm3n_o4gv_2020_40_007857331_3	596,160	1,480,892	2.4841
11 Jun 2012 16:39:10	1059924	14379064	hadcm3n_o4gv_2020_40_007857331_3	570,240	1,424,744	2.4985
08 Jun 2012 13:14:56	1059924	14379064	hadcm3n_o4gv_2020_40_007857331_3	544,320	1,367,568	2.5124