Task 18281637

Name	hadam3p_anz_f15h_2013_1_009728135_0
Workunit	9799980
Created	8 Apr 2015, 19:29:48 UTC
Sent	9 Apr 2015, 17:39:57 UTC
Report deadline	21 Mar 2016, 22:59:57 UTC
Received	17 Apr 2015, 14:01:00 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1113142
Run time	6 days 14 hours 49 min 26 sec
CPU time	5 days 11 hours 34 min 31 sec
Validate state	Invalid
Credit	2,993.82
Device peak FLOPS	2.58 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... 15:23:04 (5660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2916, selfPID=2916, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=15928, selfPID=15928, iMonCtr=2 15:22:09 (7092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:22:10 (7092): No heartbeat from core client for 30 sec - exiting 15:22:11 (7092): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 16:44:35 (15332): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:23:30 (12828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=16144, selfPID=15416, iMonCtr=1 09:20:00 (15208): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:23:16 (18060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 17:55:20 (16744): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:24:07 (4456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=14296, selfPID=1740, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 11:46:11 (13116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:35:40 (10820): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:01:34 (6556): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:20:27 (8116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:34:57 (5180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:38:25 (5512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:39:26 (5512): No heartbeat from core client for 30 sec - exiting 22:48:02 (12692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:54:00 (9264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:55:26 (9264): No heartbeat from core client for 30 sec - exiting 23:01:57 (9572): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:05:26 (8884): No heartbeat from core client for 30 sec - exiting 23:05:27 (8884): No heartbeat from core client for 30 sec - exiting 23:05:28 (8884): No heartbeat from core client for 30 sec - exiting 23:05:29 (8884): No heartbeat from core client for 30 sec - exiting 23:05:30 (8884): No heartbeat from core client for 30 sec - exiting 23:05:31 (8884): No heartbeat from core client for 30 sec - exiting 23:05:32 (8884): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:10:03 (5336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:16:59 (12108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8332, selfPID=8332, iMonCtr=2 23:35:57 (2928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:48:56 (1624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:20:19 (9472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:55:56 (9472): No heartbeat from core client for 30 sec - exiting 01:17:27 (9688): No heartbeat from core client for 30 sec - exiting 01:17:28 (9688): No heartbeat from core client for 30 sec - exiting 01:17:29 (9688): No heartbeat from core client for 30 sec - exiting 01:17:30 (9688): No heartbeat from core client for 30 sec - exiting 01:17:31 (9688): No heartbeat from core client for 30 sec - exiting 01:17:32 (9688): No heartbeat from core client for 30 sec - exiting 01:18:03 (9688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:31:58 (11760): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:42:27 (11760): No heartbeat from core client for 30 sec - exiting 01:51:05 (5336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:59:28 (5336): No heartbeat from core client for 30 sec - exiting 02:28:42 (12972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:47:20 (7136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:12:59 (7956): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:32:47 (8612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:43:29 (8612): No heartbeat from core client for 30 sec - exiting 03:50:31 (8612): No heartbeat from core client for 30 sec - exiting 04:46:59 (5764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:05:29 (5764): No heartbeat from core client for 30 sec - exiting 06:17:26 (10132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:17:57 (10132): No heartbeat from core client for 30 sec - exiting 06:18:34 (10132): No heartbeat from core client for 30 sec - exiting 06:55:13 (12996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:02:49 (12996): No heartbeat from core client for 30 sec - exiting 07:02:50 (12996): No heartbeat from core client for 30 sec - exiting 07:26:33 (10412): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:46:59 (11420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:57:04 (1008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:59:05 (1008): No heartbeat from core client for 30 sec - exiting 08:10:58 (12500): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:20:13 (2332): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:24:11 (2332): No heartbeat from core client for 30 sec - exiting 08:35:52 (12392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:59:02 (10132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:13:06 (14020): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:53:06 (7680): No heartbeat from core client for 30 sec - exiting 09:53:08 (7680): No heartbeat from core client for 30 sec - exiting 09:53:09 (7680): No heartbeat from core client for 30 sec - exiting 09:53:10 (7680): No heartbeat from core client for 30 sec - exiting 09:53:11 (7680): No heartbeat from core client for 30 sec - exiting 09:53:12 (7680): No heartbeat from core client for 30 sec - exiting 09:53:13 (7680): No heartbeat from core client for 30 sec - exiting 09:53:14 (7680): No heartbeat from core client for 30 sec - exiting 09:53:15 (7680): No heartbeat from core client for 30 sec - exiting 09:53:16 (7680): No heartbeat from core client for 30 sec - exiting 09:53:18 (7680): No heartbeat from core client for 30 sec - exiting 09:53:19 (7680): No heartbeat from core client for 30 sec - exiting 09:53:20 (7680): No heartbeat from core client for 30 sec - exiting 09:53:21 (7680): No heartbeat from core client for 30 sec - exiting 09:53:22 (7680): No heartbeat from core client for 30 sec - exiting 09:53:23 (7680): No heartbeat from core client for 30 sec - exiting 09:53:24 (7680): No heartbeat from core client for 30 sec - exiting 09:53:25 (7680): No heartbeat from core client for 30 sec - exiting 09:53:26 (7680): No heartbeat from core client for 30 sec - exiting 09:53:27 (7680): No heartbeat from core client for 30 sec - exiting 09:53:28 (7680): No heartbeat from core client for 30 sec - exiting 09:53:30 (7680): No heartbeat from core client for 30 sec - exiting 09:53:31 (7680): No heartbeat from core client for 30 sec - exiting 09:53:32 (7680): No heartbeat from core client for 30 sec - exiting 09:53:33 (7680): No heartbeat from core client for 30 sec - exiting 09:53:34 (7680): No heartbeat from core client for 30 sec - exiting 09:53:35 (7680): No heartbeat from core client for 30 sec - exiting 09:53:36 (7680): No heartbeat from core client for 30 sec - exiting 09:53:37 (7680): No heartbeat from core client for 30 sec - exiting 09:53:38 (7680): No heartbeat from core client for 30 sec - exiting 09:53:39 (7680): No heartbeat from core client for 30 sec - exiting 09:53:41 (7680): No heartbeat from core client for 30 sec - exiting 09:53:42 (7680): No heartbeat from core client for 30 sec - exiting 09:53:43 (7680): No heartbeat from core client for 30 sec - exiting 09:53:44 (7680): No heartbeat from core client for 30 sec - exiting 09:53:45 (7680): No heartbeat from core client for 30 sec - exiting 09:53:46 (7680): No heartbeat from core client for 30 sec - exiting 09:53:47 (7680): No heartbeat from core client for 30 sec - exiting 09:53:48 (7680): No heartbeat from core client for 30 sec - exiting 09:53:49 (7680): No heartbeat from core client for 30 sec - exiting 09:53:50 (7680): No heartbeat from core client for 30 sec - exiting 09:53:51 (7680): No heartbeat from core client for 30 sec - exiting 09:53:53 (7680): No heartbeat from core client for 30 sec - exiting 09:53:54 (7680): No heartbeat from core client for 30 sec - exiting 09:53:55 (7680): No heartbeat from core client for 30 sec - exiting 09:53:56 (7680): No heartbeat from core client for 30 sec - exiting 09:53:57 (7680): No heartbeat from core client for 30 sec - exiting 09:53:58 (7680): No heartbeat from core client for 30 sec - exiting 09:53:59 (7680): No heartbeat from core client for 30 sec - exiting 09:54:00 (7680): No heartbeat from core client for 30 sec - exiting 09:54:01 (7680): No heartbeat from core client for 30 sec - exiting 09:54:02 (7680): No heartbeat from core client for 30 sec - exiting 09:54:03 (7680): No heartbeat from core client for 30 sec - exiting 09:54:05 (7680): No heartbeat from core client for 30 sec - exiting 09:54:06 (7680): No heartbeat from core client for 30 sec - exiting 09:54:07 (7680): No heartbeat from core client for 30 sec - exiting 09:54:08 (7680): No heartbeat from core client for 30 sec - exiting 09:54:09 (7680): No heartbeat from core client for 30 sec - exiting 09:54:10 (7680): No heartbeat from core client for 30 sec - exiting 09:54:11 (7680): No heartbeat from core client for 30 sec - exiting 09:54:12 (7680): No heartbeat from core client for 30 sec - exiting 09:54:13 (7680): No heartbeat from core client for 30 sec - exiting 09:54:14 (7680): No heartbeat from core client for 30 sec - exiting 09:54:15 (7680): No heartbeat from core client for 30 sec - exiting 09:54:17 (7680): No heartbeat from core client for 30 sec - exiting 09:54:18 (7680): No heartbeat from core client for 30 sec - exiting 09:54:19 (7680): No heartbeat from core client for 30 sec - exiting 09:54:20 (7680): No heartbeat from core client for 30 sec - exiting 09:54:21 (7680): No heartbeat from core client for 30 sec - exiting 09:54:22 (7680): No heartbeat from core client for 30 sec - exiting 09:54:23 (7680): No heartbeat from core client for 30 sec - exiting 09:54:24 (7680): No heartbeat from core client for 30 sec - exiting 09:54:25 (7680): No heartbeat from core client for 30 sec - exiting 09:54:26 (7680): No heartbeat from core client for 30 sec - exiting 09:54:27 (7680): No heartbeat from core client for 30 sec - exiting 09:54:29 (7680): No heartbeat from core client for 30 sec - exiting 09:54:30 (7680): No heartbeat from core client for 30 sec - exiting 09:54:31 (7680): No heartbeat from core client for 30 sec - exiting 09:54:32 (7680): No heartbeat from core client for 30 sec - exiting 09:54:33 (7680): No heartbeat from core client for 30 sec - exiting 09:54:34 (7680): No heartbeat from core client for 30 sec - exiting 09:54:35 (7680): No heartbeat from core client for 30 sec - exiting 09:54:36 (7680): No heartbeat from core client for 30 sec - exiting 09:54:37 (7680): No heartbeat from core client for 30 sec - exiting 09:54:38 (7680): No heartbeat from core client for 30 sec - exiting 09:54:39 (7680): No heartbeat from core client for 30 sec - exiting 09:54:41 (7680): No heartbeat from core client for 30 sec - exiting 09:54:42 (7680): No heartbeat from core client for 30 sec - exiting 09:54:43 (7680): No heartbeat from core client for 30 sec - exiting 09:54:44 (7680): No heartbeat from core client for 30 sec - exiting 09:55:18 (7680): No heartbeat from core client for 30 sec - exiting 09:55:19 (7680): No heartbeat from core client for 30 sec - exiting 09:55:20 (7680): No heartbeat from core client for 30 sec - exiting 09:55:21 (7680): No heartbeat from core client for 30 sec - exiting 09:55:22 (7680): No heartbeat from core client for 30 sec - exiting 09:55:23 (7680): No heartbeat from core client for 30 sec - exiting 09:55:24 (7680): No heartbeat from core client for 30 sec - exiting 09:55:25 (7680): No heartbeat from core client for 30 sec - exiting 09:55:26 (7680): No heartbeat from core client for 30 sec - exiting 09:55:28 (7680): No heartbeat from core client for 30 sec - exiting 09:55:29 (7680): No heartbeat from core client for 30 sec - exiting 09:55:30 (7680): No heartbeat from core client for 30 sec - exiting 09:55:31 (7680): No heartbeat from core client for 30 sec - exiting 09:55:32 (7680): No heartbeat from core client for 30 sec - exiting 09:55:33 (7680): No heartbeat from core client for 30 sec - exiting 09:55:34 (7680): No heartbeat from core client for 30 sec - exiting 09:55:35 (7680): No heartbeat from core client for 30 sec - exiting 09:55:36 (7680): No heartbeat from core client for 30 sec - exiting 09:55:37 (7680): No heartbeat from core client for 30 sec - exiting 09:55:38 (7680): No heartbeat from core client for 30 sec - exiting 09:55:40 (7680): No heartbeat from core client for 30 sec - exiting 09:55:41 (7680): No heartbeat from core client for 30 sec - exiting 09:55:42 (7680): No heartbeat from core client for 30 sec - exiting 09:55:43 (7680): No heartbeat from core client for 30 sec - exiting 09:55:44 (7680): No heartbeat from core client for 30 sec - exiting 09:55:45 (7680): No heartbeat from core client for 30 sec - exiting 09:55:46 (7680): No heartbeat from core client for 30 sec - exiting 09:55:47 (7680): No heartbeat from core client for 30 sec - exiting 09:55:48 (7680): No heartbeat from core client for 30 sec - exiting Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/xaakm.pipe_dummy 2048 CPDN Monitor - No 'heartbeat' from BOINC... Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/xaakm.pipe_dummy 2048 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8852, selfPID=8852, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8852, selfPID=9556, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> <file_xfer_error> <file_name>hadam3p_anz_f15h_2013_1_009728135_0_7.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_f15h_2013_1_009728135_0_8.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_f15h_2013_1_009728135_0_9.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_f15h_2013_1_009728135_0_10.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_f15h_2013_1_009728135_0_11.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_f15h_2013_1_009728135_0_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
16 Apr 2015 21:49:31	1113142	18281637	hadam3p_anz_f15h_2013_1_009728135_0	69,419	458,233	6.6010
15 Apr 2015 06:50:53	1113142	18281637	hadam3p_anz_f15h_2013_1_009728135_0	57,899	383,994	6.6321
14 Apr 2015 02:14:21	1113142	18281637	hadam3p_anz_f15h_2013_1_009728135_0	46,379	309,131	6.6653
12 Apr 2015 19:46:35	1113142	18281637	hadam3p_anz_f15h_2013_1_009728135_0	34,859	233,353	6.6942
11 Apr 2015 19:47:47	1113142	18281637	hadam3p_anz_f15h_2013_1_009728135_0	23,339	156,363	6.6996
10 Apr 2015 19:11:26	1113142	18281637	hadam3p_anz_f15h_2013_1_009728135_0	11,819	78,967	6.6814