Task 15442119

Name	hadcm3n_zn2r_1880_40_008246325_1
Workunit	8401449
Created	21 Nov 2012, 0:40:16 UTC
Sent	21 Nov 2012, 0:40:16 UTC
Report deadline	20 Feb 2013, 8:07:27 UTC
Received	4 Dec 2012, 21:06:09 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1120190
Run time	3 days 20 hours 50 min 22 sec
CPU time	2 days 15 hours 27 min 44 sec
Validate state	Invalid
Credit	1,866.24
Device peak FLOPS	1.98 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.17</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:07:19 (2025): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:43:37 (2255): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:44:22 (6111): No heartbeat from core client for 30 sec - exiting 14:44:29 (6111): No heartbeat from core client for 30 sec - exiting 14:44:30 (6111): No heartbeat from core client for 30 sec - exiting 14:44:31 (6111): No heartbeat from core client for 30 sec - exiting 14:44:32 (6111): No heartbeat from core client for 30 sec - exiting 14:44:33 (6111): No heartbeat from core client for 30 sec - exiting 14:44:34 (6111): No heartbeat from core client for 30 sec - exiting 14:44:35 (6111): No heartbeat from core client for 30 sec - exiting 14:44:36 (6111): No heartbeat from core client for 30 sec - exiting 14:44:37 (6111): No heartbeat from core client for 30 sec - exiting 14:44:38 (6111): No heartbeat from core client for 30 sec - exiting 14:44:39 (6111): No heartbeat from core client for 30 sec - exiting 14:44:40 (6111): No heartbeat from core client for 30 sec - exiting 14:44:41 (6111): No heartbeat from core client for 30 sec - exiting 14:44:42 (6111): No heartbeat from core client for 30 sec - exiting 14:44:43 (6111): No heartbeat from core client for 30 sec - exiting 14:44:44 (6111): No heartbeat from core client for 30 sec - exiting 14:44:45 (6111): No heartbeat from core client for 30 sec - exiting 14:44:46 (6111): No heartbeat from core client for 30 sec - exiting 14:44:47 (6111): No heartbeat from core client for 30 sec - exiting 14:44:48 (6111): No heartbeat from core client for 30 sec - exiting 14:44:49 (6111): No heartbeat from core client for 30 sec - exiting 14:44:50 (6111): No heartbeat from core client for 30 sec - exiting 14:44:51 (6111): No heartbeat from core client for 30 sec - exiting 14:44:52 (6111): No heartbeat from core client for 30 sec - exiting 14:44:53 (6111): No heartbeat from core client for 30 sec - exiting 14:44:54 (6111): No heartbeat from core client for 30 sec - exiting 14:44:55 (6111): No heartbeat from core client for 30 sec - exiting 14:44:56 (6111): No heartbeat from core client for 30 sec - exiting 14:44:57 (6111): No heartbeat from core client for 30 sec - exiting 14:44:58 (6111): No heartbeat from core client for 30 sec - exiting 14:44:59 (6111): No heartbeat from core client for 30 sec - exiting 14:45:00 (6111): No heartbeat from core client for 30 sec - exiting 14:45:01 (6111): No heartbeat from core client for 30 sec - exiting 14:45:02 (6111): No heartbeat from core client for 30 sec - exiting 14:45:03 (6111): No heartbeat from core client for 30 sec - exiting 14:45:04 (6111): No heartbeat from core client for 30 sec - exiting 14:45:05 (6111): No heartbeat from core client for 30 sec - exiting 14:45:06 (6111): No heartbeat from core client for 30 sec - exiting 14:45:07 (6111): No heartbeat from core client for 30 sec - exiting 14:45:08 (6111): No heartbeat from core client for 30 sec - exiting 14:45:09 (6111): No heartbeat from core client for 30 sec - exiting 14:45:10 (6111): No heartbeat from core client for 30 sec - exiting 14:45:11 (6111): No heartbeat from core client for 30 sec - exiting 14:45:12 (6111): No heartbeat from core client for 30 sec - exiting 14:45:13 (6111): No heartbeat from core client for 30 sec - exiting 14:45:14 (6111): No heartbeat from core client for 30 sec - exiting 14:45:15 (6111): No heartbeat from core client for 30 sec - exiting 14:45:16 (6111): No heartbeat from core client for 30 sec - exiting 14:45:17 (6111): No heartbeat from core client for 30 sec - exiting 14:45:18 (6111): No heartbeat from core client for 30 sec - exiting 14:45:19 (6111): No heartbeat from core client for 30 sec - exiting 14:45:20 (6111): No heartbeat from core client for 30 sec - exiting 14:45:21 (6111): No heartbeat from core client for 30 sec - exiting 14:45:22 (6111): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 09:06:40 (2063): No heartbeat from core client for 30 sec - exiting 09:06:41 (2063): No heartbeat from core client for 30 sec - exiting 09:06:43 (2063): No heartbeat from core client for 30 sec - exiting 09:06:44 (2063): No heartbeat from core client for 30 sec - exiting 09:06:48 (2063): No heartbeat from core client for 30 sec - exiting 09:06:49 (2063): No heartbeat from core client for 30 sec - exiting 09:06:50 (2063): No heartbeat from core client for 30 sec - exiting 09:06:51 (2063): No heartbeat from core client for 30 sec - exiting 09:06:52 (2063): No heartbeat from core client for 30 sec - exiting 09:06:53 (2063): No heartbeat from core client for 30 sec - exiting 09:06:54 (2063): No heartbeat from core client for 30 sec - exiting 09:06:55 (2063): No heartbeat from core client for 30 sec - exiting 09:06:56 (2063): No heartbeat from core client for 30 sec - exiting 09:06:57 (2063): No heartbeat from core client for 30 sec - exiting 09:06:58 (2063): No heartbeat from core client for 30 sec - exiting 09:07:00 (2063): No heartbeat from core client for 30 sec - exiting 09:07:01 (2063): No heartbeat from core client for 30 sec - exiting 09:07:02 (2063): No heartbeat from core client for 30 sec - exiting 09:07:03 (2063): No heartbeat from core client for 30 sec - exiting 09:07:04 (2063): No heartbeat from core client for 30 sec - exiting 09:07:05 (2063): No heartbeat from core client for 30 sec - exiting 09:07:06 (2063): No heartbeat from core client for 30 sec - exiting 09:07:07 (2063): No heartbeat from core client for 30 sec - exiting 09:07:08 (2063): No heartbeat from core client for 30 sec - exiting 09:07:10 (2063): No heartbeat from core client for 30 sec - exiting 09:07:11 (2063): No heartbeat from core client for 30 sec - exiting 09:07:12 (2063): No heartbeat from core client for 30 sec - exiting 09:07:13 (2063): No heartbeat from core client for 30 sec - exiting 09:07:14 (2063): No heartbeat from core client for 30 sec - exiting 09:07:15 (2063): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
04 Dec 2012 10:42:39	1120190	15442119	hadcm3n_zn2r_1880_40_008246325_1	155,520	237,680	1.5283
30 Nov 2012 13:23:47	1120190	15442119	hadcm3n_zn2r_1880_40_008246325_1	129,600	187,968	1.4504
29 Nov 2012 23:29:40	1120190	15442119	hadcm3n_zn2r_1880_40_008246325_1	103,680	140,274	1.3530
29 Nov 2012 09:22:42	1120190	15442119	hadcm3n_zn2r_1880_40_008246325_1	77,760	90,302	1.1613
28 Nov 2012 19:37:47	1120190	15442119	hadcm3n_zn2r_1880_40_008246325_1	51,840	40,538	0.7820
21 Nov 2012 14:49:19	1120190	15442119	hadcm3n_zn2r_1880_40_008246325_1	25,920	5,880	0.2269