Task 15280334

Name	hadcm3n_z8cf_1880_40_008200748_1
Workunit	8355872
Created	13 Sep 2012, 9:01:40 UTC
Sent	13 Sep 2012, 23:06:35 UTC
Report deadline	14 Dec 2012, 6:33:46 UTC
Received	8 Oct 2012, 16:31:22 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1281115
Run time	13 days 11 hours 32 min 18 sec
CPU time	13 days 9 hours 1 min 19 sec
Validate state	Invalid
Credit	9,020.16
Device peak FLOPS	2.88 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... 22:24:29 (3180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:24:35 (3180): No heartbeat from core client for 30 sec - exiting 22:24:36 (3180): No heartbeat from core client for 30 sec - exiting 22:24:37 (3180): No heartbeat from core client for 30 sec - exiting 22:24:38 (3180): No heartbeat from core client for 30 sec - exiting 22:24:39 (3180): No heartbeat from core client for 30 sec - exiting 22:24:40 (3180): No heartbeat from core client for 30 sec - exiting 22:24:41 (3180): No heartbeat from core client for 30 sec - exiting 22:24:42 (3180): No heartbeat from core client for 30 sec - exiting 22:24:43 (3180): No heartbeat from core client for 30 sec - exiting 22:24:44 (3180): No heartbeat from core client for 30 sec - exiting 22:34:08 (6392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:34:11 (6392): No heartbeat from core client for 30 sec - exiting 22:34:12 (6392): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:42:18 (5644): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 16:45:22 (5932): No heartbeat from core client for 30 sec - exiting 16:45:24 (5932): No heartbeat from core client for 30 sec - exiting 16:45:25 (5932): No heartbeat from core client for 30 sec - exiting 16:45:26 (5932): No heartbeat from core client for 30 sec - exiting 16:45:27 (5932): No heartbeat from core client for 30 sec - exiting 16:45:28 (5932): No heartbeat from core client for 30 sec - exiting 16:45:29 (5932): No heartbeat from core client for 30 sec - exiting 16:45:30 (5932): No heartbeat from core client for 30 sec - exiting 16:45:31 (5932): No heartbeat from core client for 30 sec - exiting 16:45:32 (5932): No heartbeat from core client for 30 sec - exiting 16:45:33 (5932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5600, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5600, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5600, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5600, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5600, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5600, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
08 Oct 2012 09:36:12	1129713	15280334	hadcm3n_z8cf_1880_40_008200748_1	751,680	1,138,490	1.5146
07 Oct 2012 21:48:11	1129713	15280334	hadcm3n_z8cf_1880_40_008200748_1	725,760	1,097,872	1.5127
06 Oct 2012 09:02:40	1129713	15280334	hadcm3n_z8cf_1880_40_008200748_1	699,840	1,057,202	1.5106
05 Oct 2012 20:59:07	1129713	15280334	hadcm3n_z8cf_1880_40_008200748_1	673,920	1,014,962	1.5061
05 Oct 2012 09:12:36	1129713	15280334	hadcm3n_z8cf_1880_40_008200748_1	648,000	972,834	1.5013
04 Oct 2012 23:15:21	1129713	15280334	hadcm3n_z8cf_1880_40_008200748_1	622,080	934,656	1.5025
04 Oct 2012 12:08:54	1129713	15280334	hadcm3n_z8cf_1880_40_008200748_1	596,160	897,525	1.5055
04 Oct 2012 01:21:29	1129713	15280334	hadcm3n_z8cf_1880_40_008200748_1	570,240	858,130	1.5049
03 Oct 2012 14:37:35	1129713	15280334	hadcm3n_z8cf_1880_40_008200748_1	544,320	820,868	1.5081
03 Oct 2012 04:20:42	1129713	15280334	hadcm3n_z8cf_1880_40_008200748_1	518,400	784,054	1.5124
02 Oct 2012 17:23:20	1129713	15280334	hadcm3n_z8cf_1880_40_008200748_1	492,480	744,879	1.5125
02 Oct 2012 05:51:22	1129713	15280334	hadcm3n_z8cf_1880_40_008200748_1	466,560	704,176	1.5093
01 Oct 2012 18:59:54	1129713	15280334	hadcm3n_z8cf_1880_40_008200748_1	440,640	665,266	1.5098
01 Oct 2012 08:05:16	1129713	15280334	hadcm3n_z8cf_1880_40_008200748_1	414,720	626,372	1.5103
30 Sep 2012 21:32:53	1129713	15280334	hadcm3n_z8cf_1880_40_008200748_1	388,800	586,935	1.5096
30 Sep 2012 10:36:53	1129713	15280334	hadcm3n_z8cf_1880_40_008200748_1	362,880	549,657	1.5147
30 Sep 2012 00:09:49	1129713	15280334	hadcm3n_z8cf_1880_40_008200748_1	336,960	512,344	1.5205
29 Sep 2012 12:36:36	1129713	15280334	hadcm3n_z8cf_1880_40_008200748_1	311,040	481,075	1.5467
29 Sep 2012 03:04:38	1129713	15280334	hadcm3n_z8cf_1880_40_008200748_1	285,120	448,562	1.5732
28 Sep 2012 15:37:43	1129713	15280334	hadcm3n_z8cf_1880_40_008200748_1	259,200	409,790	1.5810