Task 16048311

Name	hadcm3n_o0kv_2060_40_008410491_2
Workunit	8561347
Created	28 Sep 2013, 15:01:11 UTC
Sent	28 Sep 2013, 15:24:41 UTC
Report deadline	28 Dec 2013, 22:51:52 UTC
Received	27 Nov 2013, 4:01:25 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1288264
Run time	21 days 19 hours 53 min 37 sec
CPU time	20 days 21 hours 2 min 12 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	2.82 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.2.31</core_client_version> <![CDATA[ <message> (unknown error) - exit code 193 (0xc1) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:49:25 (5028): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:32:30 (4180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:32:31 (4180): No heartbeat from core client for 30 sec - exiting 17:32:32 (4180): No heartbeat from core client for 30 sec - exiting 17:32:33 (4180): No heartbeat from core client for 30 sec - exiting 17:32:34 (4180): No heartbeat from core client for 30 sec - exiting 17:32:35 (4180): No heartbeat from core client for 30 sec - exiting 17:32:36 (4180): No heartbeat from core client for 30 sec - exiting 17:32:37 (4180): No heartbeat from core client for 30 sec - exiting 17:32:38 (4180): No heartbeat from core client for 30 sec - exiting 17:32:39 (4180): No heartbeat from core client for 30 sec - exiting 17:32:40 (4180): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6120, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3244, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3540, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6228, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5028, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5752, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:08:03 (4928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:08:04 (4928): No heartbeat from core client for 30 sec - exiting 18:08:05 (4928): No heartbeat from core client for 30 sec - exiting 18:08:06 (4928): No heartbeat from core client for 30 sec - exiting 18:08:07 (4928): No heartbeat from core client for 30 sec - exiting 18:08:08 (4928): No heartbeat from core client for 30 sec - exiting 18:08:09 (4928): No heartbeat from core client for 30 sec - exiting 18:08:11 (4928): No heartbeat from core client for 30 sec - exiting 18:08:12 (4928): No heartbeat from core client for 30 sec - exiting 18:08:13 (4928): No heartbeat from core client for 30 sec - exiting 18:08:14 (4928): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
27 Nov 2013 04:04:35	1288264	16048311	hadcm3n_o0kv_2060_40_008410491_2	1,036,800	1,803,723	1.7397
26 Nov 2013 13:30:15	1288264	16048311	hadcm3n_o0kv_2060_40_008410491_2	1,010,880	1,758,418	1.7395
26 Nov 2013 00:23:07	1288264	16048311	hadcm3n_o0kv_2060_40_008410491_2	984,960	1,713,742	1.7399
25 Nov 2013 11:05:56	1288264	16048311	hadcm3n_o0kv_2060_40_008410491_2	959,040	1,669,519	1.7408
24 Nov 2013 20:40:12	1288264	16048311	hadcm3n_o0kv_2060_40_008410491_2	933,120	1,624,469	1.7409
24 Nov 2013 06:06:05	1288264	16048311	hadcm3n_o0kv_2060_40_008410491_2	907,200	1,575,009	1.7361
23 Nov 2013 16:36:46	1288264	16048311	hadcm3n_o0kv_2060_40_008410491_2	881,280	1,533,176	1.7397
23 Nov 2013 04:36:12	1288264	16048311	hadcm3n_o0kv_2060_40_008410491_2	855,360	1,489,811	1.7417
22 Nov 2013 12:21:47	1288264	16048311	hadcm3n_o0kv_2060_40_008410491_2	829,440	1,444,226	1.7412
21 Nov 2013 22:07:04	1288264	16048311	hadcm3n_o0kv_2060_40_008410491_2	803,520	1,397,137	1.7388
21 Nov 2013 05:22:16	1288264	16048311	hadcm3n_o0kv_2060_40_008410491_2	777,600	1,353,055	1.7400
20 Nov 2013 14:56:21	1288264	16048311	hadcm3n_o0kv_2060_40_008410491_2	751,680	1,307,298	1.7392
19 Nov 2013 23:38:55	1288264	16048311	hadcm3n_o0kv_2060_40_008410491_2	725,760	1,258,010	1.7334
19 Nov 2013 08:08:39	1288264	16048311	hadcm3n_o0kv_2060_40_008410491_2	699,840	1,211,957	1.7318
18 Nov 2013 17:19:57	1288264	16048311	hadcm3n_o0kv_2060_40_008410491_2	673,920	1,168,225	1.7335
18 Nov 2013 04:18:40	1288264	16048311	hadcm3n_o0kv_2060_40_008410491_2	648,000	1,124,571	1.7354
17 Nov 2013 13:19:05	1288264	16048311	hadcm3n_o0kv_2060_40_008410491_2	622,080	1,080,537	1.7370
17 Nov 2013 00:22:02	1288264	16048311	hadcm3n_o0kv_2060_40_008410491_2	596,160	1,035,368	1.7367
16 Nov 2013 10:25:47	1288264	16048311	hadcm3n_o0kv_2060_40_008410491_2	570,240	991,396	1.7386
15 Nov 2013 19:31:56	1288264	16048311	hadcm3n_o0kv_2060_40_008410491_2	544,320	947,191	1.7401