Task 16084321

Name	hadcm3n_84ev_1980_40_008463675_2
Workunit	8614514
Created	25 Nov 2013, 22:21:37 UTC
Sent	26 Nov 2013, 1:07:54 UTC
Report deadline	25 Feb 2014, 8:35:05 UTC
Received	29 Dec 2013, 23:55:02 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1286177
Run time	25 days 13 hours 37 min 6 sec
CPU time	24 days 7 hours 41 min 4 sec
Validate state	Invalid
Credit	11,508.48
Device peak FLOPS	2.66 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.2.33</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9860, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:05:32 (5704): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2132, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6024, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6024, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5256, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5256, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4876, iMonCtr=1 Model crash detected, will try to restart... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
29 Dec 2013 19:13:42	1286177	16084321	hadcm3n_84ev_1980_40_008463675_2	959,040	2,120,578	2.2111
27 Dec 2013 16:30:48	1286177	16084321	hadcm3n_84ev_1980_40_008463675_2	933,120	2,061,991	2.2098
26 Dec 2013 15:35:41	1286177	16084321	hadcm3n_84ev_1980_40_008463675_2	907,200	2,003,641	2.2086
25 Dec 2013 22:36:18	1286177	16084321	hadcm3n_84ev_1980_40_008463675_2	881,280	1,945,453	2.2075
24 Dec 2013 23:03:12	1286177	16084321	hadcm3n_84ev_1980_40_008463675_2	855,360	1,887,188	2.2063
24 Dec 2013 06:13:03	1286177	16084321	hadcm3n_84ev_1980_40_008463675_2	829,440	1,828,751	2.2048
23 Dec 2013 12:37:05	1286177	16084321	hadcm3n_84ev_1980_40_008463675_2	803,520	1,769,852	2.2026
22 Dec 2013 06:43:57	1286177	16084321	hadcm3n_84ev_1980_40_008463675_2	777,600	1,711,267	2.2007
21 Dec 2013 09:41:34	1286177	16084321	hadcm3n_84ev_1980_40_008463675_2	751,680	1,652,647	2.1986
20 Dec 2013 16:56:27	1286177	16084321	hadcm3n_84ev_1980_40_008463675_2	725,760	1,593,902	2.1962
20 Dec 2013 00:13:11	1286177	16084321	hadcm3n_84ev_1980_40_008463675_2	699,840	1,535,341	2.1938
19 Dec 2013 06:04:37	1286177	16084321	hadcm3n_84ev_1980_40_008463675_2	673,920	1,476,257	2.1906
18 Dec 2013 13:13:22	1286177	16084321	hadcm3n_84ev_1980_40_008463675_2	648,000	1,417,757	2.1879
17 Dec 2013 10:27:01	1286177	16084321	hadcm3n_84ev_1980_40_008463675_2	622,080	1,359,201	2.1849
16 Dec 2013 17:20:20	1286177	16084321	hadcm3n_84ev_1980_40_008463675_2	596,160	1,300,007	2.1806
16 Dec 2013 00:26:05	1286177	16084321	hadcm3n_84ev_1980_40_008463675_2	570,240	1,241,629	2.1774
15 Dec 2013 07:21:17	1286177	16084321	hadcm3n_84ev_1980_40_008463675_2	544,320	1,183,079	2.1735
14 Dec 2013 14:36:17	1286177	16084321	hadcm3n_84ev_1980_40_008463675_2	518,400	1,124,863	2.1699
13 Dec 2013 13:54:41	1286177	16084321	hadcm3n_84ev_1980_40_008463675_2	492,480	1,066,357	2.1653
12 Dec 2013 17:02:23	1286177	16084321	hadcm3n_84ev_1980_40_008463675_2	466,560	1,007,874	2.1602