Task 13125460

Name	hadcm3n_ylk0_1900_40_007360794_1
Workunit	7558224
Created	6 Jul 2011, 15:15:02 UTC
Sent	7 Jul 2011, 16:53:17 UTC
Report deadline	7 Oct 2011, 0:20:28 UTC
Received	13 Aug 2011, 6:19:47 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	993897
Run time	21 days 3 hours 20 min 35 sec
CPU time	21 days 3 hours 20 min 35 sec
Validate state	Invalid
Credit	6,531.84
Device peak FLOPS	1.77 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.2.28</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> 08:08:10 (21204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:08:21 (21204): No heartbeat from core client for 30 sec - exiting 08:08:22 (21204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:45:54 (5644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:45:55 (5644): No heartbeat from core client for 30 sec - exiting 09:45:56 (5644): No heartbeat from core client for 30 sec - exiting 09:45:57 (5644): No heartbeat from core client for 30 sec - exiting 09:45:58 (5644): No heartbeat from core client for 30 sec - exiting 09:45:59 (5644): No heartbeat from core client for 30 sec - exiting 09:46:00 (5644): No heartbeat from core client for 30 sec - exiting 09:46:01 (5644): No heartbeat from core client for 30 sec - exiting 09:46:02 (5644): No heartbeat from core client for 30 sec - exiting 09:46:03 (5644): No heartbeat from core client for 30 sec - exiting 09:46:04 (5644): No heartbeat from core client for 30 sec - exiting 09:46:05 (5644): No heartbeat from core client for 30 sec - exiting 09:46:06 (5644): No heartbeat from core client for 30 sec - exiting 09:46:07 (5644): No heartbeat from core client for 30 sec - exiting 09:46:08 (5644): No heartbeat from core client for 30 sec - exiting 09:46:09 (5644): No heartbeat from core client for 30 sec - exiting 09:46:10 (5644): No heartbeat from core client for 30 sec - exiting 09:46:11 (5644): No heartbeat from core client for 30 sec - exiting 09:46:12 (5644): No heartbeat from core client for 30 sec - exiting 09:46:13 (5644): No heartbeat from core client for 30 sec - exiting 09:46:14 (5644): No heartbeat from core client for 30 sec - exiting 09:46:15 (5644): No heartbeat from core client for 30 sec - exiting 09:46:16 (5644): No heartbeat from core client for 30 sec - exiting 09:46:17 (5644): No heartbeat from core client for 30 sec - exiting 09:46:18 (5644): No heartbeat from core client for 30 sec - exiting 09:46:19 (5644): No heartbeat from core client for 30 sec - exiting 09:46:20 (5644): No heartbeat from core client for 30 sec - exiting 09:46:21 (5644): No heartbeat from core client for 30 sec - exiting 09:46:22 (5644): No heartbeat from core client for 30 sec - exiting 09:46:23 (5644): No heartbeat from core client for 30 sec - exiting 09:46:25 (5644): No heartbeat from core client for 30 sec - exiting 09:46:26 (5644): No heartbeat from core client for 30 sec - exiting 09:46:27 (5644): No heartbeat from core client for 30 sec - exiting 09:46:28 (5644): No heartbeat from core client for 30 sec - exiting 09:46:29 (5644): No heartbeat from core client for 30 sec - exiting 09:46:30 (5644): No heartbeat from core client for 30 sec - exiting 09:46:31 (5644): No heartbeat from core client for 30 sec - exiting 09:46:32 (5644): No heartbeat from core client for 30 sec - exiting 09:46:33 (5644): No heartbeat from core client for 30 sec - exiting 09:46:34 (5644): No heartbeat from core client for 30 sec - exiting 09:46:35 (5644): No heartbeat from core client for 30 sec - exiting 09:46:36 (5644): No heartbeat from core client for 30 sec - exiting 09:46:37 (5644): No heartbeat from core client for 30 sec - exiting 09:46:38 (5644): No heartbeat from core client for 30 sec - exiting 09:46:39 (5644): No heartbeat from core client for 30 sec - exiting 09:46:40 (5644): No heartbeat from core client for 30 sec - exiting 09:46:41 (5644): No heartbeat from core client for 30 sec - exiting 09:46:42 (5644): No heartbeat from core client for 30 sec - exiting 09:46:43 (5644): No heartbeat from core client for 30 sec - exiting 09:46:44 (5644): No heartbeat from core client for 30 sec - exiting 09:46:45 (5644): No heartbeat from core client for 30 sec - exiting 09:46:46 (5644): No heartbeat from core client for 30 sec - exiting 09:46:47 (5644): No heartbeat from core client for 30 sec - exiting 09:46:48 (5644): No heartbeat from core client for 30 sec - exiting 09:46:49 (5644): No heartbeat from core client for 30 sec - exiting 09:46:50 (5644): No heartbeat from core client for 30 sec - exiting 09:46:51 (5644): No heartbeat from core client for 30 sec - exiting 09:46:52 (5644): No heartbeat from core client for 30 sec - exiting 09:46:53 (5644): No heartbeat from core client for 30 sec - exiting 09:46:54 (5644): No heartbeat from core client for 30 sec - exiting 09:46:55 (5644): No heartbeat from core client for 30 sec - exiting 09:46:56 (5644): No heartbeat from core client for 30 sec - exiting 09:46:57 (5644): No heartbeat from core client for 30 sec - exiting 09:46:58 (5644): No heartbeat from core client for 30 sec - exiting 09:46:59 (5644): No heartbeat from core client for 30 sec - exiting 09:47:00 (5644): No heartbeat from core client for 30 sec - exiting 09:47:01 (5644): No heartbeat from core client for 30 sec - exiting 09:47:02 (5644): No heartbeat from core client for 30 sec - exiting 09:47:03 (5644): No heartbeat from core client for 30 sec - exiting 09:47:04 (5644): No heartbeat from core client for 30 sec - exiting 09:47:05 (5644): No heartbeat from core client for 30 sec - exiting 09:47:06 (5644): No heartbeat from core client for 30 sec - exiting 09:47:07 (5644): No heartbeat from core client for 30 sec - exiting 09:47:08 (5644): No heartbeat from core client for 30 sec - exiting 09:47:09 (5644): No heartbeat from core client for 30 sec - exiting 09:47:10 (5644): No heartbeat from core client for 30 sec - exiting 09:47:11 (5644): No heartbeat from core client for 30 sec - exiting 09:47:12 (5644): No heartbeat from core client for 30 sec - exiting 09:47:13 (5644): No heartbeat from core client for 30 sec - exiting 09:47:14 (5644): No heartbeat from core client for 30 sec - exiting 09:47:15 (5644): No heartbeat from core client for 30 sec - exiting 09:47:16 (5644): No heartbeat from core client for 30 sec - exiting 09:47:17 (5644): No heartbeat from core client for 30 sec - exiting 09:47:18 (5644): No heartbeat from core client for 30 sec - exiting 09:47:19 (5644): No heartbeat from core client for 30 sec - exiting 09:47:20 (5644): No heartbeat from core client for 30 sec - exiting 09:47:21 (5644): No heartbeat from core client for 30 sec - exiting 09:47:22 (5644): No heartbeat from core client for 30 sec - exiting 09:47:23 (5644): No heartbeat from core client for 30 sec - exiting 09:47:24 (5644): No heartbeat from core client for 30 sec - exiting 09:47:25 (5644): No heartbeat from core client for 30 sec - exiting 09:47:26 (5644): No heartbeat from core client for 30 sec - exiting 09:47:27 (5644): No heartbeat from core client for 30 sec - exiting 09:47:28 (5644): No heartbeat from core client for 30 sec - exiting 09:47:29 (5644): No heartbeat from core client for 30 sec - exiting 09:47:30 (5644): No heartbeat from core client for 30 sec - exiting 09:47:31 (5644): No heartbeat from core client for 30 sec - exiting 09:47:32 (5644): No heartbeat from core client for 30 sec - exiting 09:47:33 (5644): No heartbeat from core client for 30 sec - exiting 09:47:34 (5644): No heartbeat from core client for 30 sec - exiting 09:47:35 (5644): No heartbeat from core client for 30 sec - exiting 09:47:36 (5644): No heartbeat from core client for 30 sec - exiting 09:47:37 (5644): No heartbeat from core client for 30 sec - exiting 09:47:38 (5644): No heartbeat from core client for 30 sec - exiting 09:47:39 (5644): No heartbeat from core client for 30 sec - exiting 09:47:40 (5644): No heartbeat from core client for 30 sec - exiting 09:47:41 (5644): No heartbeat from core client for 30 sec - exiting 09:47:42 (5644): No heartbeat from core client for 30 sec - exiting 09:47:43 (5644): No heartbeat from core client for 30 sec - exiting 09:47:44 (5644): No heartbeat from core client for 30 sec - exiting 09:47:45 (5644): No heartbeat from core client for 30 sec - exiting 09:47:46 (5644): No heartbeat from core client for 30 sec - exiting 09:47:47 (5644): No heartbeat from core client for 30 sec - exiting 09:47:48 (5644): No heartbeat from core client for 30 sec - exiting 09:47:49 (5644): No heartbeat from core client for 30 sec - exiting 09:47:50 (5644): No heartbeat from core client for 30 sec - exiting 09:47:51 (5644): No heartbeat from core client for 30 sec - exiting 09:47:52 (5644): No heartbeat from core client for 30 sec - exiting 09:47:53 (5644): No heartbeat from core client for 30 sec - exiting 09:47:54 (5644): No heartbeat from core client for 30 sec - exiting 09:47:55 (5644): No heartbeat from core client for 30 sec - exiting 09:47:56 (5644): No heartbeat from core client for 30 sec - exiting 09:47:57 (5644): No heartbeat from core client for 30 sec - exiting 09:47:58 (5644): No heartbeat from core client for 30 sec - exiting 09:47:59 (5644): No heartbeat from core client for 30 sec - exiting 09:48:00 (5644): No heartbeat from core client for 30 sec - exiting 09:56:08 (12612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:56:09 (12612): No heartbeat from core client for 30 sec - exiting 09:56:10 (12612): No heartbeat from core client for 30 sec - exiting 09:56:11 (12612): No heartbeat from core client for 30 sec - exiting 09:56:12 (12612): No heartbeat from core client for 30 sec - exiting 09:56:13 (12612): No heartbeat from core client for 30 sec - exiting 09:56:14 (12612): No heartbeat from core client for 30 sec - exiting 09:56:15 (12612): No heartbeat from core client for 30 sec - exiting 09:56:16 (12612): No heartbeat from core client for 30 sec - exiting 09:56:17 (12612): No heartbeat from core client for 30 sec - exiting 09:56:18 (12612): No heartbeat from core client for 30 sec - exiting 10:13:04 (15300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:17:53 (14528): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:17:54 (14528): No heartbeat from core client for 30 sec - exiting 10:20:04 (15884): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:15:20 (16280): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 12:49:32 (47604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:49:33 (47604): No heartbeat from core client for 30 sec - exiting 12:49:34 (47604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5472, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5472, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5472, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5472, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5472, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5472, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
12 Aug 2011 15:53:30	993897	13125460	hadcm3n_ylk0_1900_40_007360794_1	544,320	1,810,415	3.3260
10 Aug 2011 14:40:59	993897	13125460	hadcm3n_ylk0_1900_40_007360794_1	518,400	1,723,035	3.3238
09 Aug 2011 05:18:06	993897	13125460	hadcm3n_ylk0_1900_40_007360794_1	492,480	1,635,655	3.3213
07 Aug 2011 19:50:07	993897	13125460	hadcm3n_ylk0_1900_40_007360794_1	466,560	1,544,576	3.3106
06 Aug 2011 14:05:36	993897	13125460	hadcm3n_ylk0_1900_40_007360794_1	440,640	1,454,486	3.3008
05 Aug 2011 07:27:11	993897	13125460	hadcm3n_ylk0_1900_40_007360794_1	414,720	1,366,007	3.2938
03 Aug 2011 01:16:24	993897	13125460	hadcm3n_ylk0_1900_40_007360794_1	388,800	1,274,443	3.2779
01 Aug 2011 02:36:43	993897	13125460	hadcm3n_ylk0_1900_40_007360794_1	362,880	1,186,162	3.2687
30 Jul 2011 17:43:00	993897	13125460	hadcm3n_ylk0_1900_40_007360794_1	336,960	1,096,170	3.2531
28 Jul 2011 12:49:46	993897	13125460	hadcm3n_ylk0_1900_40_007360794_1	311,040	1,012,294	3.2545
28 Jul 2011 12:49:46	993897	13125460	hadcm3n_ylk0_1900_40_007360794_1	285,120	944,501	3.3126
28 Jul 2011 12:49:46	993897	13125460	hadcm3n_ylk0_1900_40_007360794_1	259,200	862,265	3.3266
28 Jul 2011 12:49:46	993897	13125460	hadcm3n_ylk0_1900_40_007360794_1	233,280	771,903	3.3089
28 Jul 2011 12:49:46	993897	13125460	hadcm3n_ylk0_1900_40_007360794_1	207,360	683,114	3.2943
28 Jul 2011 12:49:46	993897	13125460	hadcm3n_ylk0_1900_40_007360794_1	181,440	595,084	3.2798
25 Jul 2011 16:38:42	993897	13125460	hadcm3n_ylk0_1900_40_007360794_1	155,520	508,775	3.2714
25 Jul 2011 15:37:36	993897	13125460	hadcm3n_ylk0_1900_40_007360794_1	129,600	424,465	3.2752
25 Jul 2011 13:17:21	993897	13125460	hadcm3n_ylk0_1900_40_007360794_1	103,680	338,631	3.2661
25 Jul 2011 13:17:21	993897	13125460	hadcm3n_ylk0_1900_40_007360794_1	77,760	251,269	3.2313
10 Jul 2011 17:48:12	993897	13125460	hadcm3n_ylk0_1900_40_007360794_1	51,840	167,347	3.2281