Task 16824108

Name	hadcm3n_8bko_1980_40_008724419_4
Workunit	8870397
Created	25 Jul 2014, 20:32:50 UTC
Sent	25 Jul 2014, 20:39:25 UTC
Report deadline	25 Oct 2014, 4:06:36 UTC
Received	6 Nov 2014, 1:41:30 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1184169
Run time	13 days 4 hours 25 min 25 sec
CPU time	12 days 2 hours 9 min 56 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	3.70 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.2.42</core_client_version> <![CDATA[ <message> (unknown error) - exit code 193 (0xc1) </message> <stderr_txt> 14:06:20 (11920): No heartbeat from core client for 30 sec - exiting 14:06:21 (11920): No heartbeat from core client for 30 sec - exiting 14:06:22 (11920): No heartbeat from core client for 30 sec - exiting 14:06:23 (11920): No heartbeat from core client for 30 sec - exiting 14:06:24 (11920): No heartbeat from core client for 30 sec - exiting 14:06:25 (11920): No heartbeat from core client for 30 sec - exiting 14:06:26 (11920): No heartbeat from core client for 30 sec - exiting 14:06:27 (11920): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:06:28 (11920): No heartbeat from core client for 30 sec - exiting 14:06:29 (11920): No heartbeat from core client for 30 sec - exiting 22:01:25 (6132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:23:34 (4692): No heartbeat from core client for 30 sec - exiting 22:23:35 (4692): No heartbeat from core client for 30 sec - exiting 22:23:36 (4692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:23:37 (4692): No heartbeat from core client for 30 sec - exiting 20:26:48 (8044): No heartbeat from core client for 30 sec - exiting 20:26:49 (8044): No heartbeat from core client for 30 sec - exiting 20:26:50 (8044): No heartbeat from core client for 30 sec - exiting 20:26:51 (8044): No heartbeat from core client for 30 sec - exiting 20:26:52 (8044): No heartbeat from core client for 30 sec - exiting 20:26:53 (8044): No heartbeat from core client for 30 sec - exiting 20:26:54 (8044): No heartbeat from core client for 30 sec - exiting 20:26:55 (8044): No heartbeat from core client for 30 sec - exiting 20:26:56 (8044): No heartbeat from core client for 30 sec - exiting 20:26:57 (8044): No heartbeat from core client for 30 sec - exiting 20:26:58 (8044): No heartbeat from core client for 30 sec - exiting 20:26:59 (8044): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:21:18 (4020): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:12:44 (9364): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 22:19:30 (4840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:20:23 (4728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:03:34 (57584): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 02:13:45 (125448): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:15:03 (241788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:11:12 (8824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:12:05 (5824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 00:00:15 (6696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:00:01 (7836): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:20:20 (7456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:59:17 (7900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:51:55 (8536): No heartbeat from core client for 30 sec - exiting 20:51:56 (8536): No heartbeat from core client for 30 sec - exiting 20:51:57 (8536): No heartbeat from core client for 30 sec - exiting 20:51:58 (8536): No heartbeat from core client for 30 sec - exiting 20:51:59 (8536): No heartbeat from core client for 30 sec - exiting 20:52:00 (8536): No heartbeat from core client for 30 sec - exiting 20:52:01 (8536): No heartbeat from core client for 30 sec - exiting 20:52:02 (8536): No heartbeat from core client for 30 sec - exiting 20:52:03 (8536): No heartbeat from core client for 30 sec - exiting 20:52:04 (8536): No heartbeat from core client for 30 sec - exiting 20:52:05 (8536): No heartbeat from core client for 30 sec - exiting 20:52:06 (8536): No heartbeat from core client for 30 sec - exiting 20:52:07 (8536): No heartbeat from core client for 30 sec - exiting 20:52:08 (8536): No heartbeat from core client for 30 sec - exiting 20:52:09 (8536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:01:29 (4920): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:47:15 (7600): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:30:05 (2900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:47:47 (8576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:04:09 (6452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:19:58 (9268): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:23:35 (17256): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:45:25 (17308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:22:25 (20752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:53:17 (16528): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:59:15 (23860): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:22:24 (23364): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 02:03:43 (23760): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:20:52 (26084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:21:56 (25812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:23:44 (31016): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:35:38 (33744): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:19:29 (6796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:34:36 (10296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:43:45 (16868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:05:49 (16712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:14:48 (16856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:48:54 (1576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:49:36 (8380): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:39:01 (2720): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7688, iMonCtr=1 Model crash detected, will try to restart... 22:15:52 (7316): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:59:39 (8288): No heartbeat from core client for 30 sec - exiting 20:59:40 (8288): No heartbeat from core client for 30 sec - exiting 20:59:41 (8288): No heartbeat from core client for 30 sec - exiting 20:59:42 (8288): No heartbeat from core client for 30 sec - exiting 20:59:43 (8288): No heartbeat from core client for 30 sec - exiting 20:59:44 (8288): No heartbeat from core client for 30 sec - exiting 20:59:45 (8288): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:51:48 (1312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=46384, iMonCtr=1 Model crash detected, will try to restart... 19:13:35 (12488): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 20:06:41 (2112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:24:58 (1584): No heartbeat from core client for 30 sec - exiting 23:24:59 (1584): No heartbeat from core client for 30 sec - exiting 23:25:00 (1584): No heartbeat from core client for 30 sec - exiting 23:25:01 (1584): No heartbeat from core client for 30 sec - exiting 23:25:02 (1584): No heartbeat from core client for 30 sec - exiting 23:25:03 (1584): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7544, iMonCtr=1 Model crash detected, will try to restart... 19:59:43 (9168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:09:18 (8308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:59:26 (6272): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 23:35:27 (7556): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:29:47 (8748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:33:47 (7288): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8064, iMonCtr=1 Model crash detected, will try to restart... 13:07:45 (5016): No heartbeat from core client for 30 sec - exiting 13:07:46 (5016): No heartbeat from core client for 30 sec - exiting 13:07:47 (5016): No heartbeat from core client for 30 sec - exiting 13:07:48 (5016): No heartbeat from core client for 30 sec - exiting 13:07:49 (5016): No heartbeat from core client for 30 sec - exiting 13:07:50 (5016): No heartbeat from core client for 30 sec - exiting 13:07:51 (5016): No heartbeat from core client for 30 sec - exiting 13:07:52 (5016): No heartbeat from core client for 30 sec - exiting 13:07:53 (5016): No heartbeat from core client for 30 sec - exiting 13:07:54 (5016): No heartbeat from core client for 30 sec - exiting 13:07:55 (5016): No heartbeat from core client for 30 sec - exiting 13:07:56 (5016): No heartbeat from core client for 30 sec - exiting 13:07:57 (5016): No heartbeat from core client for 30 sec - exiting 13:07:58 (5016): No heartbeat from core client for 30 sec - exiting 13:07:59 (5016): No heartbeat from core client for 30 sec - exiting 13:08:00 (5016): No heartbeat from core client for 30 sec - exiting 13:08:01 (5016): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:09:38 (8128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:52:07 (28740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:03:06 (7384): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:07:58 (8864): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:03:49 (10112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:53:03 (9528): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:22:25 (4412): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 10:57:16 (11036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 00:59:39 (216308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:51:35 (216324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:51:50 (8268): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:51:51 (8268): No heartbeat from core client for 30 sec - exiting 20:32:27 (3756): No heartbeat from core client for 30 sec - exiting 20:32:28 (3756): No heartbeat from core client for 30 sec - exiting 20:32:29 (3756): No heartbeat from core client for 30 sec - exiting 20:32:30 (3756): No heartbeat from core client for 30 sec - exiting 20:32:31 (3756): No heartbeat from core client for 30 sec - exiting 20:32:32 (3756): No heartbeat from core client for 30 sec - exiting 20:32:33 (3756): No heartbeat from core client for 30 sec - exiting 20:32:34 (3756): No heartbeat from core client for 30 sec - exiting 20:32:35 (3756): No heartbeat from core client for 30 sec - exiting 20:32:36 (3756): No heartbeat from core client for 30 sec - exiting 20:32:37 (3756): No heartbeat from core client for 30 sec - exiting 20:32:38 (3756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:32:39 (3756): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=47248, iMonCtr=1 Model crash detected, will try to restart... 23:01:02 (1396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
06 Nov 2014 00:40:57	1184169	16824108	hadcm3n_8bko_1980_40_008724419_4	1,036,800	1,044,593	1.0075
04 Nov 2014 06:28:13	1184169	16824108	hadcm3n_8bko_1980_40_008724419_4	1,010,880	1,017,836	1.0069
03 Nov 2014 20:20:00	1184169	16824108	hadcm3n_8bko_1980_40_008724419_4	984,960	990,513	1.0056
02 Nov 2014 15:33:45	1184169	16824108	hadcm3n_8bko_1980_40_008724419_4	959,040	963,428	1.0046
02 Nov 2014 00:37:49	1184169	16824108	hadcm3n_8bko_1980_40_008724419_4	933,120	937,248	1.0044
01 Nov 2014 16:05:15	1184169	16824108	hadcm3n_8bko_1980_40_008724419_4	907,200	911,310	1.0045
01 Nov 2014 00:01:28	1184169	16824108	hadcm3n_8bko_1980_40_008724419_4	881,280	885,331	1.0046
29 Oct 2014 22:22:43	1184169	16824108	hadcm3n_8bko_1980_40_008724419_4	855,360	858,768	1.0040
26 Oct 2014 22:16:47	1184169	16824108	hadcm3n_8bko_1980_40_008724419_4	829,440	832,247	1.0034
26 Oct 2014 14:34:44	1184169	16824108	hadcm3n_8bko_1980_40_008724419_4	803,520	806,521	1.0037
25 Oct 2014 18:01:14	1184169	16824108	hadcm3n_8bko_1980_40_008724419_4	777,600	780,522	1.0038
25 Oct 2014 09:50:02	1184169	16824108	hadcm3n_8bko_1980_40_008724419_4	751,680	754,813	1.0042
21 Oct 2014 18:15:22	1184169	16824108	hadcm3n_8bko_1980_40_008724419_4	725,760	728,764	1.0041
19 Oct 2014 21:13:28	1184169	16824108	hadcm3n_8bko_1980_40_008724419_4	699,840	701,660	1.0026
18 Oct 2014 01:54:52	1184169	16824108	hadcm3n_8bko_1980_40_008724419_4	673,920	674,953	1.0015
17 Oct 2014 17:33:21	1184169	16824108	hadcm3n_8bko_1980_40_008724419_4	648,000	648,356	1.0005
15 Oct 2014 20:32:46	1184169	16824108	hadcm3n_8bko_1980_40_008724419_4	622,080	622,771	1.0011
14 Oct 2014 01:52:06	1184169	16824108	hadcm3n_8bko_1980_40_008724419_4	596,160	596,298	1.0002
13 Oct 2014 00:00:45	1184169	16824108	hadcm3n_8bko_1980_40_008724419_4	570,240	568,704	0.9973
25 Aug 2014 23:15:16	1184169	16824108	hadcm3n_8bko_1980_40_008724419_4	544,320	541,198	0.9943