Task 16041156

Name	hadcm3n_obue_1900_40_008470473_0
Workunit	8621312
Created	27 Sep 2013, 9:57:41 UTC
Sent	1 Oct 2013, 18:28:57 UTC
Report deadline	1 Jan 2014, 1:56:08 UTC
Received	22 Oct 2013, 10:15:29 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1160537
Run time	3 days 19 hours 40 min 54 sec
CPU time	3 days 10 hours 58 min 2 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	3.56 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> (unknown error) - exit code 193 (0xc1) </message> <stderr_txt> 23:02:09 (4744): No heartbeat from core client for 30 sec - exiting 23:02:10 (4744): No heartbeat from core client for 30 sec - exiting 23:02:11 (4744): No heartbeat from core client for 30 sec - exiting 23:02:12 (4744): No heartbeat from core client for 30 sec - exiting 23:02:13 (4744): No heartbeat from core client for 30 sec - exiting 23:02:14 (4744): No heartbeat from core client for 30 sec - exiting 23:02:15 (4744): No heartbeat from core client for 30 sec - exiting 23:02:16 (4744): No heartbeat from core client for 30 sec - exiting 23:02:18 (4744): No heartbeat from core client for 30 sec - exiting 23:02:19 (4744): No heartbeat from core client for 30 sec - exiting 23:02:20 (4744): No heartbeat from core client for 30 sec - exiting 23:02:21 (4744): No heartbeat from core client for 30 sec - exiting 23:02:22 (4744): No heartbeat from core client for 30 sec - exiting 23:02:23 (4744): No heartbeat from core client for 30 sec - exiting 23:02:24 (4744): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 07:35:23 (7912): No heartbeat from core client for 30 sec - exiting 07:35:24 (7912): No heartbeat from core client for 30 sec - exiting 07:35:25 (7912): No heartbeat from core client for 30 sec - exiting 07:35:26 (7912): No heartbeat from core client for 30 sec - exiting 07:35:28 (7912): No heartbeat from core client for 30 sec - exiting 07:35:29 (7912): No heartbeat from core client for 30 sec - exiting 07:35:30 (7912): No heartbeat from core client for 30 sec - exiting 07:35:31 (7912): No heartbeat from core client for 30 sec - exiting 07:35:32 (7912): No heartbeat from core client for 30 sec - exiting 07:35:33 (7912): No heartbeat from core client for 30 sec - exiting 07:35:34 (7912): No heartbeat from core client for 30 sec - exiting 07:35:35 (7912): No heartbeat from core client for 30 sec - exiting 07:35:36 (7912): No heartbeat from core client for 30 sec - exiting 07:35:37 (7912): No heartbeat from core client for 30 sec - exiting 07:35:39 (7912): No heartbeat from core client for 30 sec - exiting 07:35:40 (7912): No heartbeat from core client for 30 sec - exiting 07:35:41 (7912): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:08:42 (7956): No heartbeat from core client for 30 sec - exiting 19:08:44 (7956): No heartbeat from core client for 30 sec - exiting 19:08:45 (7956): No heartbeat from core client for 30 sec - exiting 19:08:46 (7956): No heartbeat from core client for 30 sec - exiting 19:08:47 (7956): No heartbeat from core client for 30 sec - exiting 19:08:48 (7956): No heartbeat from core client for 30 sec - exiting 19:08:49 (7956): No heartbeat from core client for 30 sec - exiting 19:08:50 (7956): No heartbeat from core client for 30 sec - exiting 19:08:51 (7956): No heartbeat from core client for 30 sec - exiting 19:08:52 (7956): No heartbeat from core client for 30 sec - exiting 19:08:54 (7956): No heartbeat from core client for 30 sec - exiting 19:08:55 (7956): No heartbeat from core client for 30 sec - exiting 19:08:56 (7956): No heartbeat from core client for 30 sec - exiting 19:08:57 (7956): No heartbeat from core client for 30 sec - exiting 19:08:58 (7956): No heartbeat from core client for 30 sec - exiting 19:08:59 (7956): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:12:20 (6004): No heartbeat from core client for 30 sec - exiting 07:12:21 (6004): No heartbeat from core client for 30 sec - exiting 07:12:22 (6004): No heartbeat from core client for 30 sec - exiting 07:12:23 (6004): No heartbeat from core client for 30 sec - exiting 07:12:24 (6004): No heartbeat from core client for 30 sec - exiting 07:12:25 (6004): No heartbeat from core client for 30 sec - exiting 07:12:27 (6004): No heartbeat from core client for 30 sec - exiting 07:12:28 (6004): No heartbeat from core client for 30 sec - exiting 07:12:29 (6004): No heartbeat from core client for 30 sec - exiting 07:12:30 (6004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 17:59:11 (1108): No heartbeat from core client for 30 sec - exiting 17:59:12 (1108): No heartbeat from core client for 30 sec - exiting 17:59:13 (1108): No heartbeat from core client for 30 sec - exiting 17:59:14 (1108): No heartbeat from core client for 30 sec - exiting 17:59:15 (1108): No heartbeat from core client for 30 sec - exiting 17:59:16 (1108): No heartbeat from core client for 30 sec - exiting 17:59:17 (1108): No heartbeat from core client for 30 sec - exiting 17:59:18 (1108): No heartbeat from core client for 30 sec - exiting 17:59:19 (1108): No heartbeat from core client for 30 sec - exiting 17:59:20 (1108): No heartbeat from core client for 30 sec - exiting 17:59:21 (1108): No heartbeat from core client for 30 sec - exiting 17:59:22 (1108): No heartbeat from core client for 30 sec - exiting 17:59:23 (1108): No heartbeat from core client for 30 sec - exiting 17:59:24 (1108): No heartbeat from core client for 30 sec - exiting 17:59:25 (1108): No heartbeat from core client for 30 sec - exiting 17:59:26 (1108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:01:05 (1660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 08:15:08 (6428): No heartbeat from core client for 30 sec - exiting 08:15:10 (6428): No heartbeat from core client for 30 sec - exiting 08:15:11 (6428): No heartbeat from core client for 30 sec - exiting 08:15:12 (6428): No heartbeat from core client for 30 sec - exiting 08:15:13 (6428): No heartbeat from core client for 30 sec - exiting 08:15:14 (6428): No heartbeat from core client for 30 sec - exiting 08:15:15 (6428): No heartbeat from core client for 30 sec - exiting 08:15:16 (6428): No heartbeat from core client for 30 sec - exiting 08:15:17 (6428): No heartbeat from core client for 30 sec - exiting 08:15:18 (6428): No heartbeat from core client for 30 sec - exiting 08:15:19 (6428): No heartbeat from core client for 30 sec - exiting 08:15:20 (6428): No heartbeat from core client for 30 sec - exiting 08:15:22 (6428): No heartbeat from core client for 30 sec - exiting 08:15:23 (6428): No heartbeat from core client for 30 sec - exiting 08:15:24 (6428): No heartbeat from core client for 30 sec - exiting 08:15:25 (6428): No heartbeat from core client for 30 sec - exiting 08:15:26 (6428): No heartbeat from core client for 30 sec - exiting 08:15:27 (6428): No heartbeat from core client for 30 sec - exiting 08:15:28 (6428): No heartbeat from core client for 30 sec - exiting 08:15:29 (6428): No heartbeat from core client for 30 sec - exiting 08:15:30 (6428): No heartbeat from core client for 30 sec - exiting 08:15:31 (6428): No heartbeat from core client for 30 sec - exiting 08:15:32 (6428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:25:29 (6900): No heartbeat from core client for 30 sec - exiting 08:25:30 (6900): No heartbeat from core client for 30 sec - exiting 08:25:31 (6900): No heartbeat from core client for 30 sec - exiting 08:25:32 (6900): No heartbeat from core client for 30 sec - exiting 08:25:33 (6900): No heartbeat from core client for 30 sec - exiting 08:25:35 (6900): No heartbeat from core client for 30 sec - exiting 08:25:36 (6900): No heartbeat from core client for 30 sec - exiting 08:25:37 (6900): No heartbeat from core client for 30 sec - exiting 08:25:38 (6900): No heartbeat from core client for 30 sec - exiting 08:25:39 (6900): No heartbeat from core client for 30 sec - exiting 08:25:40 (6900): No heartbeat from core client for 30 sec - exiting 08:25:41 (6900): No heartbeat from core client for 30 sec - exiting 08:25:42 (6900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6616, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 13:59:13 (7100): No heartbeat from core client for 30 sec - exiting 13:59:14 (7100): No heartbeat from core client for 30 sec - exiting 13:59:15 (7100): No heartbeat from core client for 30 sec - exiting 13:59:16 (7100): No heartbeat from core client for 30 sec - exiting 13:59:17 (7100): No heartbeat from core client for 30 sec - exiting 13:59:18 (7100): No heartbeat from core client for 30 sec - exiting 13:59:19 (7100): No heartbeat from core client for 30 sec - exiting 13:59:20 (7100): No heartbeat from core client for 30 sec - exiting 13:59:21 (7100): No heartbeat from core client for 30 sec - exiting 13:59:22 (7100): No heartbeat from core client for 30 sec - exiting 13:59:24 (7100): No heartbeat from core client for 30 sec - exiting 13:59:25 (7100): No heartbeat from core client for 30 sec - exiting 13:59:26 (7100): No heartbeat from core client for 30 sec - exiting 13:59:27 (7100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1652, iMonCtr=1 Model crash detected, will try to restart... 18:42:54 (7984): No heartbeat from core client for 30 sec - exiting 18:42:55 (7984): No heartbeat from core client for 30 sec - exiting 18:42:56 (7984): No heartbeat from core client for 30 sec - exiting 18:42:57 (7984): No heartbeat from core client for 30 sec - exiting 18:42:58 (7984): No heartbeat from core client for 30 sec - exiting 18:42:59 (7984): No heartbeat from core client for 30 sec - exiting 18:43:00 (7984): No heartbeat from core client for 30 sec - exiting 18:43:01 (7984): No heartbeat from core client for 30 sec - exiting 18:43:03 (7984): No heartbeat from core client for 30 sec - exiting 18:43:04 (7984): No heartbeat from core client for 30 sec - exiting 18:43:05 (7984): No heartbeat from core client for 30 sec - exiting 18:43:06 (7984): No heartbeat from core client for 30 sec - exiting 18:43:07 (7984): No heartbeat from core client for 30 sec - exiting 18:43:08 (7984): No heartbeat from core client for 30 sec - exiting 18:43:09 (7984): No heartbeat from core client for 30 sec - exiting 18:43:10 (7984): No heartbeat from core client for 30 sec - exiting 18:43:11 (7984): No heartbeat from core client for 30 sec - exiting 18:43:12 (7984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 05:35:37 (6972): No heartbeat from core client for 30 sec - exiting 05:35:38 (6972): No heartbeat from core client for 30 sec - exiting 05:35:39 (6972): No heartbeat from core client for 30 sec - exiting 05:35:40 (6972): No heartbeat from core client for 30 sec - exiting 05:35:41 (6972): No heartbeat from core client for 30 sec - exiting 05:35:43 (6972): No heartbeat from core client for 30 sec - exiting 05:35:44 (6972): No heartbeat from core client for 30 sec - exiting 05:35:45 (6972): No heartbeat from core client for 30 sec - exiting 05:35:46 (6972): No heartbeat from core client for 30 sec - exiting 05:35:47 (6972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7540, iMonCtr=1 Model crash detected, will try to restart... 17:21:16 (7116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 17:31:51 (5520): No heartbeat from core client for 30 sec - exiting 17:31:53 (5520): No heartbeat from core client for 30 sec - exiting 17:31:54 (5520): No heartbeat from core client for 30 sec - exiting 17:31:55 (5520): No heartbeat from core client for 30 sec - exiting 17:31:56 (5520): No heartbeat from core client for 30 sec - exiting 17:31:57 (5520): No heartbeat from core client for 30 sec - exiting 17:31:58 (5520): No heartbeat from core client for 30 sec - exiting 17:31:59 (5520): No heartbeat from core client for 30 sec - exiting 17:32:00 (5520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:39:39 (6736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
22 Oct 2013 10:18:30	1160537	16041156	hadcm3n_obue_1900_40_008470473_0	259,200	298,676	1.1523
21 Oct 2013 15:33:42	1160537	16041156	hadcm3n_obue_1900_40_008470473_0	233,280	268,302	1.1501
17 Oct 2013 17:43:35	1160537	16041156	hadcm3n_obue_1900_40_008470473_0	207,360	237,791	1.1468
16 Oct 2013 08:59:31	1160537	16041156	hadcm3n_obue_1900_40_008470473_0	181,440	194,310	1.0709
14 Oct 2013 11:33:22	1160537	16041156	hadcm3n_obue_1900_40_008470473_0	155,520	155,396	0.9992
13 Oct 2013 16:16:57	1160537	16041156	hadcm3n_obue_1900_40_008470473_0	129,600	136,882	1.0562
13 Oct 2013 11:30:56	1160537	16041156	hadcm3n_obue_1900_40_008470473_0	103,680	118,326	1.1413
12 Oct 2013 16:05:51	1160537	16041156	hadcm3n_obue_1900_40_008470473_0	77,760	94,941	1.2209
11 Oct 2013 19:13:09	1160537	16041156	hadcm3n_obue_1900_40_008470473_0	51,840	63,254	1.2202
07 Oct 2013 15:57:03	1160537	16041156	hadcm3n_obue_1900_40_008470473_0	25,920	29,851	1.1517