Task 15486538

Name	hadcm3n_3gvj_1940_40_008258935_0
Workunit	8414059
Created	20 Dec 2012, 12:43:31 UTC
Sent	20 Dec 2012, 12:44:06 UTC
Report deadline	21 Mar 2013, 20:11:17 UTC
Received	20 Jan 2013, 22:27:05 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1235335
Run time	9 days 6 hours 45 min 43 sec
CPU time	8 days 16 hours 25 min 48 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	3.26 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 23:19:22 (2544): No heartbeat from core client for 30 sec - exiting 23:19:23 (2544): No heartbeat from core client for 30 sec - exiting 23:19:24 (2544): No heartbeat from core client for 30 sec - exiting 23:19:25 (2544): No heartbeat from core client for 30 sec - exiting 23:19:26 (2544): No heartbeat from core client for 30 sec - exiting 23:19:27 (2544): No heartbeat from core client for 30 sec - exiting 23:19:28 (2544): No heartbeat from core client for 30 sec - exiting 23:19:29 (2544): No heartbeat from core client for 30 sec - exiting 23:19:30 (2544): No heartbeat from core client for 30 sec - exiting 23:19:31 (2544): No heartbeat from core client for 30 sec - exiting 23:19:32 (2544): No heartbeat from core client for 30 sec - exiting 23:19:33 (2544): No heartbeat from core client for 30 sec - exiting 23:19:34 (2544): No heartbeat from core client for 30 sec - exiting 23:19:35 (2544): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4156, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3068, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
20 Jan 2013 21:28:12	1235335	15486538	hadcm3n_3gvj_1940_40_008258935_0	777,600	767,364	0.9868
20 Jan 2013 00:18:00	1235335	15486538	hadcm3n_3gvj_1940_40_008258935_0	751,680	741,547	0.9865
19 Jan 2013 01:23:35	1235335	15486538	hadcm3n_3gvj_1940_40_008258935_0	725,760	716,577	0.9873
18 Jan 2013 18:35:13	1235335	15486538	hadcm3n_3gvj_1940_40_008258935_0	699,840	691,764	0.9885
18 Jan 2013 11:36:02	1235335	15486538	hadcm3n_3gvj_1940_40_008258935_0	673,920	666,889	0.9896
16 Jan 2013 01:20:41	1235335	15486538	hadcm3n_3gvj_1940_40_008258935_0	648,000	641,231	0.9896
13 Jan 2013 21:29:20	1235335	15486538	hadcm3n_3gvj_1940_40_008258935_0	622,080	615,240	0.9890
12 Jan 2013 22:22:24	1235335	15486538	hadcm3n_3gvj_1940_40_008258935_0	596,160	589,478	0.9888
11 Jan 2013 01:33:04	1235335	15486538	hadcm3n_3gvj_1940_40_008258935_0	570,240	563,613	0.9884
09 Jan 2013 20:45:38	1235335	15486538	hadcm3n_3gvj_1940_40_008258935_0	544,320	537,526	0.9875
06 Jan 2013 23:49:06	1235335	15486538	hadcm3n_3gvj_1940_40_008258935_0	518,400	511,035	0.9858
05 Jan 2013 20:46:54	1235335	15486538	hadcm3n_3gvj_1940_40_008258935_0	492,480	485,422	0.9857
04 Jan 2013 21:11:47	1235335	15486538	hadcm3n_3gvj_1940_40_008258935_0	466,560	459,444	0.9847
02 Jan 2013 22:48:39	1235335	15486538	hadcm3n_3gvj_1940_40_008258935_0	440,640	433,802	0.9845
01 Jan 2013 20:09:36	1235335	15486538	hadcm3n_3gvj_1940_40_008258935_0	414,720	408,137	0.9841
01 Jan 2013 02:50:05	1235335	15486538	hadcm3n_3gvj_1940_40_008258935_0	388,800	382,564	0.9840
31 Dec 2012 19:35:54	1235335	15486538	hadcm3n_3gvj_1940_40_008258935_0	362,880	356,854	0.9834
31 Dec 2012 00:16:36	1235335	15486538	hadcm3n_3gvj_1940_40_008258935_0	336,960	331,280	0.9831
30 Dec 2012 00:02:12	1235335	15486538	hadcm3n_3gvj_1940_40_008258935_0	311,040	305,663	0.9827
29 Dec 2012 14:06:49	1235335	15486538	hadcm3n_3gvj_1940_40_008258935_0	285,120	279,617	0.9807