Task 16000472

Name	hadcm3n_o209_1980_40_008386472_1
Workunit	8537331
Created	2 Sep 2013, 14:12:11 UTC
Sent	2 Sep 2013, 14:28:25 UTC
Report deadline	2 Dec 2013, 21:55:36 UTC
Received	20 Sep 2013, 1:58:43 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1291326
Run time	6 days 12 hours 19 min 6 sec
CPU time	6 days 9 hours 45 min 49 sec
Validate state	Invalid
Credit	10,264.32
Device peak FLOPS	3.70 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:53:32 (3940): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:12:38 (3552): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:12:39 (3552): No heartbeat from core client for 30 sec - exiting 09:12:40 (3552): No heartbeat from core client for 30 sec - exiting 09:12:41 (3552): No heartbeat from core client for 30 sec - exiting 09:12:42 (3552): No heartbeat from core client for 30 sec - exiting 09:12:43 (3552): No heartbeat from core client for 30 sec - exiting 09:12:44 (3552): No heartbeat from core client for 30 sec - exiting 09:12:45 (3552): No heartbeat from core client for 30 sec - exiting 09:12:46 (3552): No heartbeat from core client for 30 sec - exiting 09:12:47 (3552): No heartbeat from core client for 30 sec - exiting 09:12:49 (3552): No heartbeat from core client for 30 sec - exiting 09:27:01 (632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:27:02 (632): No heartbeat from core client for 30 sec - exiting 09:27:03 (632): No heartbeat from core client for 30 sec - exiting 09:27:05 (632): No heartbeat from core client for 30 sec - exiting 09:27:06 (632): No heartbeat from core client for 30 sec - exiting 09:27:07 (632): No heartbeat from core client for 30 sec - exiting 09:27:08 (632): No heartbeat from core client for 30 sec - exiting 09:27:09 (632): No heartbeat from core client for 30 sec - exiting 09:27:10 (632): No heartbeat from core client for 30 sec - exiting 09:27:11 (632): No heartbeat from core client for 30 sec - exiting 09:27:12 (632): No heartbeat from core client for 30 sec - exiting Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3248, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3248, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3248, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3248, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3248, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3248, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
20 Sep 2013 01:01:30	1291326	16000472	hadcm3n_o209_1980_40_008386472_1	855,360	555,996	0.6500
19 Sep 2013 00:50:21	1291326	16000472	hadcm3n_o209_1980_40_008386472_1	829,440	539,369	0.6503
18 Sep 2013 15:13:49	1291326	16000472	hadcm3n_o209_1980_40_008386472_1	803,520	523,052	0.6510
18 Sep 2013 10:49:02	1291326	16000472	hadcm3n_o209_1980_40_008386472_1	777,600	506,426	0.6513
18 Sep 2013 06:16:42	1291326	16000472	hadcm3n_o209_1980_40_008386472_1	751,680	489,743	0.6515
18 Sep 2013 01:28:09	1291326	16000472	hadcm3n_o209_1980_40_008386472_1	725,760	472,831	0.6515
17 Sep 2013 20:23:58	1291326	16000472	hadcm3n_o209_1980_40_008386472_1	699,840	455,960	0.6515
17 Sep 2013 20:23:58	1291326	16000472	hadcm3n_o209_1980_40_008386472_1	699,840	455,960	0.6515
17 Sep 2013 16:17:40	1291326	16000472	hadcm3n_o209_1980_40_008386472_1	673,920	439,179	0.6517
17 Sep 2013 11:36:53	1291326	16000472	hadcm3n_o209_1980_40_008386472_1	648,000	422,538	0.6521
17 Sep 2013 07:02:31	1291326	16000472	hadcm3n_o209_1980_40_008386472_1	622,080	405,760	0.6523
17 Sep 2013 02:06:07	1291326	16000472	hadcm3n_o209_1980_40_008386472_1	596,160	388,872	0.6523
16 Sep 2013 21:35:15	1291326	16000472	hadcm3n_o209_1980_40_008386472_1	570,240	371,894	0.6522
16 Sep 2013 16:29:03	1291326	16000472	hadcm3n_o209_1980_40_008386472_1	544,320	355,130	0.6524
16 Sep 2013 16:29:03	1291326	16000472	hadcm3n_o209_1980_40_008386472_1	518,400	338,369	0.6527
16 Sep 2013 06:40:59	1291326	16000472	hadcm3n_o209_1980_40_008386472_1	492,480	321,626	0.6531
16 Sep 2013 01:59:10	1291326	16000472	hadcm3n_o209_1980_40_008386472_1	466,560	304,777	0.6532
15 Sep 2013 21:17:38	1291326	16000472	hadcm3n_o209_1980_40_008386472_1	440,640	287,821	0.6532
15 Sep 2013 07:08:26	1291326	16000472	hadcm3n_o209_1980_40_008386472_1	414,720	270,978	0.6534
15 Sep 2013 00:36:49	1291326	16000472	hadcm3n_o209_1980_40_008386472_1	388,800	253,859	0.6529