Task 12857594

Name	hadam3p_saf_2896_1990_1_007240296_0
Workunit	7438536
Created	4 May 2011, 7:36:17 UTC
Sent	10 May 2011, 3:10:53 UTC
Report deadline	21 Apr 2012, 8:30:53 UTC
Received	27 May 2011, 23:38:01 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1123650
Run time	3 days 5 hours 57 min 54 sec
CPU time	3 days 3 hours 40 min 6 sec
Validate state	Invalid
Credit	1,683.45
Device peak FLOPS	2.62 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Southern Africa v6.09 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4348, selfPID=4348, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:48:29 (2996): No heartbeat from core client for 30 sec - exiting 09:48:30 (2996): No heartbeat from core client for 30 sec - exiting 09:48:31 (2996): No heartbeat from core client for 30 sec - exiting 09:48:32 (2996): No heartbeat from core client for 30 sec - exiting 09:48:33 (2996): No heartbeat from core client for 30 sec - exiting 09:48:34 (2996): No heartbeat from core client for 30 sec - exiting 09:48:35 (2996): No heartbeat from core client for 30 sec - exiting 09:48:36 (2996): No heartbeat from core client for 30 sec - exiting 09:48:37 (2996): No heartbeat from core client for 30 sec - exiting 09:48:38 (2996): No heartbeat from core client for 30 sec - exiting 09:48:39 (2996): No heartbeat from core client for 30 sec - exiting 09:48:40 (2996): No heartbeat from core client for 30 sec - exiting 09:48:41 (2996): No heartbeat from core client for 30 sec - exiting 09:48:43 (2996): No heartbeat from core client for 30 sec - exiting 09:48:44 (2996): No heartbeat from core client for 30 sec - exiting 09:48:45 (2996): No heartbeat from core client for 30 sec - exiting 09:48:46 (2996): No heartbeat from core client for 30 sec - exiting 09:48:47 (2996): No heartbeat from core client for 30 sec - exiting 09:48:48 (2996): No heartbeat from core client for 30 sec - exiting 09:48:49 (2996): No heartbeat from core client for 30 sec - exiting 09:48:50 (2996): No heartbeat from core client for 30 sec - exiting 09:48:51 (2996): No heartbeat from core client for 30 sec - exiting 09:48:52 (2996): No heartbeat from core client for 30 sec - exiting 09:48:53 (2996): No heartbeat from core client for 30 sec - exiting 09:48:55 (2996): No heartbeat from core client for 30 sec - exiting 09:48:56 (2996): No heartbeat from core client for 30 sec - exiting 09:48:57 (2996): No heartbeat from core client for 30 sec - exiting 09:48:58 (2996): No heartbeat from core client for 30 sec - exiting 09:48:59 (2996): No heartbeat from core client for 30 sec - exiting 09:49:00 (2996): No heartbeat from core client for 30 sec - exiting 09:49:01 (2996): No heartbeat from core client for 30 sec - exiting 09:49:02 (2996): No heartbeat from core client for 30 sec - exiting 09:49:03 (2996): No heartbeat from core client for 30 sec - exiting 09:49:04 (2996): No heartbeat from core client for 30 sec - exiting 09:49:05 (2996): No heartbeat from core client for 30 sec - exiting 09:49:07 (2996): No heartbeat from core client for 30 sec - exiting 09:49:08 (2996): No heartbeat from core client for 30 sec - exiting 09:49:09 (2996): No heartbeat from core client for 30 sec - exiting 09:49:10 (2996): No heartbeat from core client for 30 sec - exiting 09:49:11 (2996): No heartbeat from core client for 30 sec - exiting 09:49:12 (2996): No heartbeat from core client for 30 sec - exiting 09:49:13 (2996): No heartbeat from core client for 30 sec - exiting 09:49:14 (2996): No heartbeat from core client for 30 sec - exiting 09:49:15 (2996): No heartbeat from core client for 30 sec - exiting 09:49:16 (2996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:49:18 (2996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3112, selfPID=3112, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5568, selfPID=5568, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7660, selfPID=7660, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5360, selfPID=5360, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8108, selfPID=8108, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=744, selfPID=744, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3056, selfPID=3056, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3208, selfPID=3208, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7612, selfPID=7612, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4776, selfPID=4776, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5424, selfPID=5424, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7804, selfPID=7804, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2044, selfPID=2044, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8188, selfPID=8188, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3056, selfPID=3056, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:33:01 (2984): No heartbeat from core client for 30 sec - exiting 17:33:02 (2984): No heartbeat from core client for 30 sec - exiting 17:33:03 (2984): No heartbeat from core client for 30 sec - exiting 17:33:04 (2984): No heartbeat from core client for 30 sec - exiting 17:33:05 (2984): No heartbeat from core client for 30 sec - exiting 17:33:07 (2984): No heartbeat from core client for 30 sec - exiting 17:33:08 (2984): No heartbeat from core client for 30 sec - exiting 17:33:09 (2984): No heartbeat from core client for 30 sec - exiting 17:33:10 (2984): No heartbeat from core client for 30 sec - exiting 17:33:11 (2984): No heartbeat from core client for 30 sec - exiting 17:33:12 (2984): No heartbeat from core client for 30 sec - exiting 17:33:13 (2984): No heartbeat from core client for 30 sec - exiting 17:33:14 (2984): No heartbeat from core client for 30 sec - exiting 17:33:15 (2984): No heartbeat from core client for 30 sec - exiting 17:33:16 (2984): No heartbeat from core client for 30 sec - exiting 17:33:17 (2984): No heartbeat from core client for 30 sec - exiting 17:33:19 (2984): No heartbeat from core client for 30 sec - exiting 17:33:20 (2984): No heartbeat from core client for 30 sec - exiting 17:33:21 (2984): No heartbeat from core client for 30 sec - exiting 17:33:22 (2984): No heartbeat from core client for 30 sec - exiting 17:33:23 (2984): No heartbeat from core client for 30 sec - exiting 17:33:24 (2984): No heartbeat from core client for 30 sec - exiting 17:33:25 (2984): No heartbeat from core client for 30 sec - exiting 17:33:26 (2984): No heartbeat from core client for 30 sec - exiting 17:33:27 (2984): No heartbeat from core client for 30 sec - exiting 17:33:28 (2984): No heartbeat from core client for 30 sec - exiting 17:33:29 (2984): No heartbeat from core client for 30 sec - exiting 17:33:31 (2984): No heartbeat from core client for 30 sec - exiting 17:33:32 (2984): No heartbeat from core client for 30 sec - exiting 17:33:33 (2984): No heartbeat from core client for 30 sec - exiting 17:33:34 (2984): No heartbeat from core client for 30 sec - exiting 17:33:35 (2984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:33:36 (2984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1628, selfPID=1628, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1872, selfPID=1872, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5412, selfPID=5412, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2888, selfPID=2888, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4720, selfPID=4720, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1912, selfPID=1912, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7572, selfPID=7572, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6712, selfPID=6712, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:31:03 (2924): No heartbeat from core client for 30 sec - exiting 17:31:04 (2924): No heartbeat from core client for 30 sec - exiting 17:31:06 (2924): No heartbeat from core client for 30 sec - exiting 17:31:07 (2924): No heartbeat from core client for 30 sec - exiting 17:31:08 (2924): No heartbeat from core client for 30 sec - exiting 17:31:09 (2924): No heartbeat from core client for 30 sec - exiting 17:31:10 (2924): No heartbeat from core client for 30 sec - exiting 17:31:11 (2924): No heartbeat from core client for 30 sec - exiting 17:31:12 (2924): No heartbeat from core client for 30 sec - exiting 17:31:13 (2924): No heartbeat from core client for 30 sec - exiting 17:31:14 (2924): No heartbeat from core client for 30 sec - exiting 17:31:15 (2924): No heartbeat from core client for 30 sec - exiting 17:31:16 (2924): No heartbeat from core client for 30 sec - exiting 17:31:18 (2924): No heartbeat from core client for 30 sec - exiting 17:31:19 (2924): No heartbeat from core client for 30 sec - exiting 17:31:20 (2924): No heartbeat from core client for 30 sec - exiting 17:31:21 (2924): No heartbeat from core client for 30 sec - exiting 17:31:22 (2924): No heartbeat from core client for 30 sec - exiting 17:31:23 (2924): No heartbeat from core client for 30 sec - exiting 17:31:24 (2924): No heartbeat from core client for 30 sec - exiting 17:31:25 (2924): No heartbeat from core client for 30 sec - exiting 17:31:26 (2924): No heartbeat from core client for 30 sec - exiting 17:31:27 (2924): No heartbeat from core client for 30 sec - exiting 17:31:28 (2924): No heartbeat from core client for 30 sec - exiting 17:31:30 (2924): No heartbeat from core client for 30 sec - exiting 17:31:31 (2924): No heartbeat from core client for 30 sec - exiting 17:31:32 (2924): No heartbeat from core client for 30 sec - exiting 17:31:33 (2924): No heartbeat from core client for 30 sec - exiting 17:31:34 (2924): No heartbeat from core client for 30 sec - exiting 17:31:35 (2924): No heartbeat from core client for 30 sec - exiting 17:31:36 (2924): No heartbeat from core client for 30 sec - exiting 17:31:37 (2924): No heartbeat from core client for 30 sec - exiting 17:31:38 (2924): No heartbeat from core client for 30 sec - exiting 17:31:39 (2924): No heartbeat from core client for 30 sec - exiting 17:31:40 (2924): No heartbeat from core client for 30 sec - exiting 17:31:41 (2924): No heartbeat from core client for 30 sec - exiting 17:31:42 (2924): No heartbeat from core client for 30 sec - exiting 17:31:44 (2924): No heartbeat from core client for 30 sec - exiting 17:31:45 (2924): No heartbeat from core client for 30 sec - exiting 17:31:46 (2924): No heartbeat from core client for 30 sec - exiting 17:31:47 (2924): No heartbeat from core client for 30 sec - exiting 17:31:48 (2924): No heartbeat from core client for 30 sec - exiting 17:31:49 (2924): No heartbeat from core client for 30 sec - exiting 17:31:50 (2924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:31:51 (2924): No heartbeat from core client for 30 sec - exiting 17:31:52 (2924): No heartbeat from core client for 30 sec - exiting Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/xaakm.pipe_dummy 2048 Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> <file_xfer_error> <file_name>hadam3p_saf_2896_1990_1_007240296_0_10.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_saf_2896_1990_1_007240296_0_11.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_saf_2896_1990_1_007240296_0_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
26 May 2011 17:56:43	1123650	12857594	hadam3p_saf_2896_1990_1_007240296_0	103,776	267,822	2.5808
26 May 2011 09:38:30	1123650	12857594	hadam3p_saf_2896_1990_1_007240296_0	92,256	238,445	2.5846
26 May 2011 01:16:14	1123650	12857594	hadam3p_saf_2896_1990_1_007240296_0	80,736	209,122	2.5902
25 May 2011 15:06:35	1123650	12857594	hadam3p_saf_2896_1990_1_007240296_0	69,216	179,929	2.5995
25 May 2011 06:42:59	1123650	12857594	hadam3p_saf_2896_1990_1_007240296_0	57,696	150,748	2.6128
24 May 2011 21:52:45	1123650	12857594	hadam3p_saf_2896_1990_1_007240296_0	46,176	121,042	2.6213
24 May 2011 10:14:14	1123650	12857594	hadam3p_saf_2896_1990_1_007240296_0	34,656	91,022	2.6264
24 May 2011 01:40:52	1123650	12857594	hadam3p_saf_2896_1990_1_007240296_0	23,136	61,049	2.6387
22 May 2011 17:05:03	1123650	12857594	hadam3p_saf_2896_1990_1_007240296_0	11,616	30,721	2.6447