Task 16969429

Name	hadam3p_anz_rmg2_2012_1_008955669_1
Workunit	9099844
Created	1 Sep 2014, 20:22:15 UTC
Sent	1 Sep 2014, 20:53:12 UTC
Report deadline	15 Aug 2015, 2:13:12 UTC
Received	19 Sep 2014, 8:24:34 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1332080
Run time	2 days 16 hours 30 min 14 sec
CPU time	2 days 3 hours 54 min 30 sec
Validate state	Invalid
Credit	2,000.18
Device peak FLOPS	3.65 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10 windows_intelx86
Stderr	<core_client_version>7.2.42</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:33:43 (4272): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4564, selfPID=4564, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8092, selfPID=8092, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6796, selfPID=6796, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:07:59 (3996): No heartbeat from core client for 30 sec - exiting 08:08:00 (3996): No heartbeat from core client for 30 sec - exiting 08:08:01 (3996): No heartbeat from core client for 30 sec - exiting 08:08:02 (3996): No heartbeat from core client for 30 sec - exiting 08:08:04 (3996): No heartbeat from core client for 30 sec - exiting 08:08:05 (3996): No heartbeat from core client for 30 sec - exiting 08:08:06 (3996): No heartbeat from core client for 30 sec - exiting 08:08:07 (3996): No heartbeat from core client for 30 sec - exiting 08:08:08 (3996): No heartbeat from core client for 30 sec - exiting 08:08:09 (3996): No heartbeat from core client for 30 sec - exiting 08:08:10 (3996): No heartbeat from core client for 30 sec - exiting 08:08:11 (3996): No heartbeat from core client for 30 sec - exiting 08:08:12 (3996): No heartbeat from core client for 30 sec - exiting 08:08:13 (3996): No heartbeat from core client for 30 sec - exiting 08:08:14 (3996): No heartbeat from core client for 30 sec - exiting 08:08:16 (3996): No heartbeat from core client for 30 sec - exiting 08:08:17 (3996): No heartbeat from core client for 30 sec - exiting 08:08:18 (3996): No heartbeat from core client for 30 sec - exiting 08:08:19 (3996): No heartbeat from core client for 30 sec - exiting 08:08:20 (3996): No heartbeat from core client for 30 sec - exiting 08:08:21 (3996): No heartbeat from core client for 30 sec - exiting 08:08:22 (3996): No heartbeat from core client for 30 sec - exiting 08:08:23 (3996): No heartbeat from core client for 30 sec - exiting 08:08:24 (3996): No heartbeat from core client for 30 sec - exiting 08:08:25 (3996): No heartbeat from core client for 30 sec - exiting 08:08:26 (3996): No heartbeat from core client for 30 sec - exiting 08:08:28 (3996): No heartbeat from core client for 30 sec - exiting 08:08:29 (3996): No heartbeat from core client for 30 sec - exiting 08:08:30 (3996): No heartbeat from core client for 30 sec - exiting 08:08:31 (3996): No heartbeat from core client for 30 sec - exiting 08:08:32 (3996): No heartbeat from core client for 30 sec - exiting 08:08:33 (3996): No heartbeat from core client for 30 sec - exiting 08:08:34 (3996): No heartbeat from core client for 30 sec - exiting 08:08:35 (3996): No heartbeat from core client for 30 sec - exiting 08:08:36 (3996): No heartbeat from core client for 30 sec - exiting 08:08:37 (3996): No heartbeat from core client for 30 sec - exiting 08:08:38 (3996): No heartbeat from core client for 30 sec - exiting 08:08:40 (3996): No heartbeat from core client for 30 sec - exiting 08:08:41 (3996): No heartbeat from core client for 30 sec - exiting 08:08:42 (3996): No heartbeat from core client for 30 sec - exiting 08:08:43 (3996): No heartbeat from core client for 30 sec - exiting 08:08:44 (3996): No heartbeat from core client for 30 sec - exiting 08:08:45 (3996): No heartbeat from core client for 30 sec - exiting 08:08:46 (3996): No heartbeat from core client for 30 sec - exiting 08:08:47 (3996): No heartbeat from core client for 30 sec - exiting 08:08:48 (3996): No heartbeat from core client for 30 sec - exiting 08:08:49 (3996): No heartbeat from core client for 30 sec - exiting 08:08:50 (3996): No heartbeat from core client for 30 sec - exiting 08:08:51 (3996): No heartbeat from core client for 30 sec - exiting 08:08:53 (3996): No heartbeat from core client for 30 sec - exiting 08:08:54 (3996): No heartbeat from core client for 30 sec - exiting 08:08:55 (3996): No heartbeat from core client for 30 sec - exiting 08:08:56 (3996): No heartbeat from core client for 30 sec - exiting 08:08:57 (3996): No heartbeat from core client for 30 sec - exiting 08:08:58 (3996): No heartbeat from core client for 30 sec - exiting 08:08:59 (3996): No heartbeat from core client for 30 sec - exiting 08:09:00 (3996): No heartbeat from core client for 30 sec - exiting 08:09:01 (3996): No heartbeat from core client for 30 sec - exiting 08:09:02 (3996): No heartbeat from core client for 30 sec - exiting 08:09:03 (3996): No heartbeat from core client for 30 sec - exiting 08:09:05 (3996): No heartbeat from core client for 30 sec - exiting 08:09:06 (3996): No heartbeat from core client for 30 sec - exiting 08:09:07 (3996): No heartbeat from core client for 30 sec - exiting 08:09:08 (3996): No heartbeat from core client for 30 sec - exiting 08:09:09 (3996): No heartbeat from core client for 30 sec - exiting 08:09:10 (3996): No heartbeat from core client for 30 sec - exiting 08:09:11 (3996): No heartbeat from core client for 30 sec - exiting 08:09:12 (3996): No heartbeat from core client for 30 sec - exiting 08:09:13 (3996): No heartbeat from core client for 30 sec - exiting 08:09:14 (3996): No heartbeat from core client for 30 sec - exiting 08:09:15 (3996): No heartbeat from core client for 30 sec - exiting 08:09:17 (3996): No heartbeat from core client for 30 sec - exiting 08:09:18 (3996): No heartbeat from core client for 30 sec - exiting 08:09:19 (3996): No heartbeat from core client for 30 sec - exiting 08:09:20 (3996): No heartbeat from core client for 30 sec - exiting 08:09:21 (3996): No heartbeat from core client for 30 sec - exiting 08:09:22 (3996): No heartbeat from core client for 30 sec - exiting 08:09:23 (3996): No heartbeat from core client for 30 sec - exiting 08:09:24 (3996): No heartbeat from core client for 30 sec - exiting 08:09:25 (3996): No heartbeat from core client for 30 sec - exiting 08:09:26 (3996): No heartbeat from core client for 30 sec - exiting 08:09:27 (3996): No heartbeat from core client for 30 sec - exiting 08:09:29 (3996): No heartbeat from core client for 30 sec - exiting 08:09:30 (3996): No heartbeat from core client for 30 sec - exiting 08:09:31 (3996): No heartbeat from core client for 30 sec - exiting 08:09:32 (3996): No heartbeat from core client for 30 sec - exiting 08:09:33 (3996): No heartbeat from core client for 30 sec - exiting 08:09:34 (3996): No heartbeat from core client for 30 sec - exiting 08:09:35 (3996): No heartbeat from core client for 30 sec - exiting 08:09:36 (3996): No heartbeat from core client for 30 sec - exiting 08:09:37 (3996): No heartbeat from core client for 30 sec - exiting 08:09:38 (3996): No heartbeat from core client for 30 sec - exiting 08:09:39 (3996): No heartbeat from core client for 30 sec - exiting 08:09:40 (3996): No heartbeat from core client for 30 sec - exiting 08:09:42 (3996): No heartbeat from core client for 30 sec - exiting 08:09:43 (3996): No heartbeat from core client for 30 sec - exiting 08:09:44 (3996): No heartbeat from core client for 30 sec - exiting 08:09:45 (3996): No heartbeat from core client for 30 sec - exiting 08:09:46 (3996): No heartbeat from core client for 30 sec - exiting 08:09:47 (3996): No heartbeat from core client for 30 sec - exiting 08:09:48 (3996): No heartbeat from core client for 30 sec - exiting 08:09:49 (3996): No heartbeat from core client for 30 sec - exiting 08:09:50 (3996): No heartbeat from core client for 30 sec - exiting 08:09:51 (3996): No heartbeat from core client for 30 sec - exiting 08:09:52 (3996): No heartbeat from core client for 30 sec - exiting 08:09:54 (3996): No heartbeat from core client for 30 sec - exiting 08:09:55 (3996): No heartbeat from core client for 30 sec - exiting 08:09:56 (3996): No heartbeat from core client for 30 sec - exiting 08:09:57 (3996): No heartbeat from core client for 30 sec - exiting 08:09:58 (3996): No heartbeat from core client for 30 sec - exiting 08:09:59 (3996): No heartbeat from core client for 30 sec - exiting 08:10:00 (3996): No heartbeat from core client for 30 sec - exiting 08:10:01 (3996): No heartbeat from core client for 30 sec - exiting 08:10:02 (3996): No heartbeat from core client for 30 sec - exiting 08:10:03 (3996): No heartbeat from core client for 30 sec - exiting 08:10:04 (3996): No heartbeat from core client for 30 sec - exiting 08:10:06 (3996): No heartbeat from core client for 30 sec - exiting 08:10:07 (3996): No heartbeat from core client for 30 sec - exiting 08:10:08 (3996): No heartbeat from core client for 30 sec - exiting 08:10:09 (3996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:10:10 (3996): No heartbeat from core client for 30 sec - exiting Global Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=6716, iMonCtr=1 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1872, selfPID=6756, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 08:11:32 (6756): No heartbeat from core client for 30 sec - exiting Called boinc_finish 08:11:40 (6756): No heartbeat from core client for 30 sec - exiting 08:11:41 (6756): No heartbeat from core client for 30 sec - exiting </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_anz_rmg2_2012_1_008955669_1_5.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_rmg2_2012_1_008955669_1_6.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_rmg2_2012_1_008955669_1_7.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_rmg2_2012_1_008955669_1_8.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_rmg2_2012_1_008955669_1_9.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_rmg2_2012_1_008955669_1_10.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_rmg2_2012_1_008955669_1_11.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_rmg2_2012_1_008955669_1_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
16 Sep 2014 09:58:21	1332080	16969429	hadam3p_anz_rmg2_2012_1_008955669_1	46,379	177,788	3.8334
12 Sep 2014 12:27:00	1332080	16969429	hadam3p_anz_rmg2_2012_1_008955669_1	34,859	133,531	3.8306
08 Sep 2014 17:32:32	1332080	16969429	hadam3p_anz_rmg2_2012_1_008955669_1	23,339	89,059	3.8159
03 Sep 2014 16:48:04	1332080	16969429	hadam3p_anz_rmg2_2012_1_008955669_1	11,819	44,522	3.7670