Task 16610583

Name	hadam3p_anz_a8tb_2012_1_008617759_1
Workunit	8764271
Created	5 May 2014, 8:07:52 UTC
Sent	5 May 2014, 8:09:45 UTC
Report deadline	17 Apr 2015, 13:29:45 UTC
Received	28 May 2014, 7:33:35 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1241169
Run time	5 days 12 hours 43 min 20 sec
CPU time	5 days 8 hours 16 min 8 sec
Validate state	Invalid
Credit	2,497.00
Device peak FLOPS	2.55 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10 windows_intelx86
Stderr	<core_client_version>7.2.33</core_client_version> <![CDATA[ <stderr_txt> Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7988, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7896, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4880, iMonCtr=2 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=896, selfPID=3092, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9100, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7000, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:01:40 (1716): No heartbeat from core client for 30 sec - exiting 09:01:41 (1716): No heartbeat from core client for 30 sec - exiting 09:01:42 (1716): No heartbeat from core client for 30 sec - exiting 09:01:44 (1716): No heartbeat from core client for 30 sec - exiting 09:01:45 (1716): No heartbeat from core client for 30 sec - exiting 09:01:46 (1716): No heartbeat from core client for 30 sec - exiting 09:01:47 (1716): No heartbeat from core client for 30 sec - exiting 09:01:48 (1716): No heartbeat from core client for 30 sec - exiting 09:01:49 (1716): No heartbeat from core client for 30 sec - exiting 09:01:50 (1716): No heartbeat from core client for 30 sec - exiting 09:01:51 (1716): No heartbeat from core client for 30 sec - exiting 09:01:52 (1716): No heartbeat from core client for 30 sec - exiting 09:01:53 (1716): No heartbeat from core client for 30 sec - exiting 09:01:54 (1716): No heartbeat from core client for 30 sec - exiting 09:01:56 (1716): No heartbeat from core client for 30 sec - exiting 09:01:57 (1716): No heartbeat from core client for 30 sec - exiting 09:01:58 (1716): No heartbeat from core client for 30 sec - exiting 09:01:59 (1716): No heartbeat from core client for 30 sec - exiting 09:02:00 (1716): No heartbeat from core client for 30 sec - exiting 09:02:01 (1716): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2828, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5840, iMonCtr=2 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5396, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2560, selfPID=1796, iMonCtr=1 Model crash detected, will try to restart... GGlobal Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7128, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=240, selfPID=3388, iMonCtr=1 Model crash detected, will try to restart... 14:24:57 (4076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4388, selfPID=1304, iMonCtr=1 Model crash detected, will try to restart... 09:13:39 (6652): No heartbeat from core client for 30 sec - exiting 09:13:40 (6652): No heartbeat from core client for 30 sec - exiting 09:13:41 (6652): No heartbeat from core client for 30 sec - exiting 09:13:42 (6652): No heartbeat from core client for 30 sec - exiting 09:13:43 (6652): No heartbeat from core client for 30 sec - exiting 09:13:44 (6652): No heartbeat from core client for 30 sec - exiting 09:13:45 (6652): No heartbeat from core client for 30 sec - exiting 09:13:46 (6652): No heartbeat from core client for 30 sec - exiting 09:13:47 (6652): No heartbeat from core client for 30 sec - exiting 09:13:49 (6652): No heartbeat from core client for 30 sec - exiting 09:13:50 (6652): No heartbeat from core client for 30 sec - exiting 09:13:51 (6652): No heartbeat from core client for 30 sec - exiting 09:13:52 (6652): No heartbeat from core client for 30 sec - exiting 09:13:53 (6652): No heartbeat from core client for 30 sec - exiting 09:13:54 (6652): No heartbeat from core client for 30 sec - exiting 09:13:55 (6652): No heartbeat from core client for 30 sec - exiting 09:13:56 (6652): No heartbeat from core client for 30 sec - exiting 09:13:57 (6652): No heartbeat from core client for 30 sec - exiting 09:13:58 (6652): No heartbeat from core client for 30 sec - exiting 09:13:59 (6652): No heartbeat from core client for 30 sec - exiting 09:14:01 (6652): No heartbeat from core client for 30 sec - exiting 09:14:02 (6652): No heartbeat from core client for 30 sec - exiting 09:14:03 (6652): No heartbeat from core client for 30 sec - exiting 09:14:04 (6652): No heartbeat from core client for 30 sec - exiting 09:14:05 (6652): No heartbeat from core client for 30 sec - exiting 09:14:06 (6652): No heartbeat from core client for 30 sec - exiting 09:14:07 (6652): No heartbeat from core client for 30 sec - exiting 09:14:08 (6652): No heartbeat from core client for 30 sec - exiting 09:14:09 (6652): No heartbeat from core client for 30 sec - exiting 09:14:10 (6652): No heartbeat from core client for 30 sec - exiting 09:14:11 (6652): No heartbeat from core client for 30 sec - exiting 09:14:13 (6652): No heartbeat from core client for 30 sec - exiting 09:14:14 (6652): No heartbeat from core client for 30 sec - exiting 09:14:15 (6652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6832, iMonCtr=2 Model crash detected, will try to restart... 18:24:25 (3376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7108, iMonCtr=2 Model crash detected, will try to restart... 08:27:27 (4372): No heartbeat from core client for 30 sec - exiting 08:27:28 (4372): No heartbeat from core client for 30 sec - exiting 08:27:29 (4372): No heartbeat from core client for 30 sec - exiting 08:27:30 (4372): No heartbeat from core client for 30 sec - exiting 08:27:31 (4372): No heartbeat from core client for 30 sec - exiting 08:27:32 (4372): No heartbeat from core client for 30 sec - exiting 08:27:34 (4372): No heartbeat from core client for 30 sec - exiting 08:27:35 (4372): No heartbeat from core client for 30 sec - exiting 08:27:36 (4372): No heartbeat from core client for 30 sec - exiting 08:27:37 (4372): No heartbeat from core client for 30 sec - exiting 08:27:38 (4372): No heartbeat from core client for 30 sec - exiting 08:27:39 (4372): No heartbeat from core client for 30 sec - exiting 08:27:40 (4372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:40:13 (4404): No heartbeat from core client for 30 sec - exiting 08:40:14 (4404): No heartbeat from core client for 30 sec - exiting 08:40:15 (4404): No heartbeat from core client for 30 sec - exiting 08:40:16 (4404): No heartbeat from core client for 30 sec - exiting 08:40:17 (4404): No heartbeat from core client for 30 sec - exiting 08:40:18 (4404): No heartbeat from core client for 30 sec - exiting 08:40:20 (4404): No heartbeat from core client for 30 sec - exiting 08:40:21 (4404): No heartbeat from core client for 30 sec - exiting 08:40:22 (4404): No heartbeat from core client for 30 sec - exiting 08:40:23 (4404): No heartbeat from core client for 30 sec - exiting 08:40:24 (4404): No heartbeat from core client for 30 sec - exiting 08:40:25 (4404): No heartbeat from core client for 30 sec - exiting 08:40:26 (4404): No heartbeat from core client for 30 sec - exiting 08:40:27 (4404): No heartbeat from core client for 30 sec - exiting 08:40:28 (4404): No heartbeat from core client for 30 sec - exiting 08:40:29 (4404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:33:59 (3444): No heartbeat from core client for 30 sec - exiting 20:34:00 (3444): No heartbeat from core client for 30 sec - exiting 20:34:01 (3444): No heartbeat from core client for 30 sec - exiting 20:34:02 (3444): No heartbeat from core client for 30 sec - exiting 20:34:04 (3444): No heartbeat from core client for 30 sec - exiting 20:34:05 (3444): No heartbeat from core client for 30 sec - exiting 20:34:06 (3444): No heartbeat from core client for 30 sec - exiting 20:34:07 (3444): No heartbeat from core client for 30 sec - exiting 20:34:08 (3444): No heartbeat from core client for 30 sec - exiting 20:34:09 (3444): No heartbeat from core client for 30 sec - exiting 20:34:10 (3444): No heartbeat from core client for 30 sec - exiting 20:34:11 (3444): No heartbeat from core client for 30 sec - exiting 20:34:12 (3444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:12:31 (6504): No heartbeat from core client for 30 sec - exiting 12:12:32 (6504): No heartbeat from core client for 30 sec - exiting 12:12:33 (6504): No heartbeat from core client for 30 sec - exiting 12:12:34 (6504): No heartbeat from core client for 30 sec - exiting 12:12:35 (6504): No heartbeat from core client for 30 sec - exiting 12:12:36 (6504): No heartbeat from core client for 30 sec - exiting 12:12:38 (6504): No heartbeat from core client for 30 sec - exiting 12:12:39 (6504): No heartbeat from core client for 30 sec - exiting 12:12:40 (6504): No heartbeat from core client for 30 sec - exiting 12:12:41 (6504): No heartbeat from core client for 30 sec - exiting 12:12:42 (6504): No heartbeat from core client for 30 sec - exiting 12:12:43 (6504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4960, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7344, iMonCtr=2 Model crash detected, will try to restart... 09:12:51 (6028): No heartbeat from core client for 30 sec - exiting 09:12:52 (6028): No heartbeat from core client for 30 sec - exiting 09:12:53 (6028): No heartbeat from core client for 30 sec - exiting 09:12:54 (6028): No heartbeat from core client for 30 sec - exiting 09:12:56 (6028): No heartbeat from core client for 30 sec - exiting 09:12:57 (6028): No heartbeat from core client for 30 sec - exiting 09:12:58 (6028): No heartbeat from core client for 30 sec - exiting 09:12:59 (6028): No heartbeat from core client for 30 sec - exiting 09:13:00 (6028): No heartbeat from core client for 30 sec - exiting 09:13:01 (6028): No heartbeat from core client for 30 sec - exiting 09:13:02 (6028): No heartbeat from core client for 30 sec - exiting 09:13:03 (6028): No heartbeat from core client for 30 sec - exiting 09:13:04 (6028): No heartbeat from core client for 30 sec - exiting 09:13:05 (6028): No heartbeat from core client for 30 sec - exiting 09:13:07 (6028): No heartbeat from core client for 30 sec - exiting 09:13:08 (6028): No heartbeat from core client for 30 sec - exiting 09:13:09 (6028): No heartbeat from core client for 30 sec - exiting 09:13:10 (6028): No heartbeat from core client for 30 sec - exiting 09:13:11 (6028): No heartbeat from core client for 30 sec - exiting 09:13:12 (6028): No heartbeat from core client for 30 sec - exiting 09:13:13 (6028): No heartbeat from core client for 30 sec - exiting 09:13:14 (6028): No heartbeat from core client for 30 sec - exiting 09:13:15 (6028): No heartbeat from core client for 30 sec - exiting 09:13:16 (6028): No heartbeat from core client for 30 sec - exiting 09:13:18 (6028): No heartbeat from core client for 30 sec - exiting 09:13:19 (6028): No heartbeat from core client for 30 sec - exiting 09:13:20 (6028): No heartbeat from core client for 30 sec - exiting 09:13:21 (6028): No heartbeat from core client for 30 sec - exiting 09:13:22 (6028): No heartbeat from core client for 30 sec - exiting 09:13:23 (6028): No heartbeat from core client for 30 sec - exiting 09:13:24 (6028): No heartbeat from core client for 30 sec - exiting 09:13:25 (6028): No heartbeat from core client for 30 sec - exiting 09:13:26 (6028): No heartbeat from core client for 30 sec - exiting 09:13:27 (6028): No heartbeat from core client for 30 sec - exiting 09:13:28 (6028): No heartbeat from core client for 30 sec - exiting 09:13:30 (6028): No heartbeat from core client for 30 sec - exiting 09:13:31 (6028): No heartbeat from core client for 30 sec - exiting 09:13:32 (6028): No heartbeat from core client for 30 sec - exiting 09:13:33 (6028): No heartbeat from core client for 30 sec - exiting 09:13:34 (6028): No heartbeat from core client for 30 sec - exiting 09:13:35 (6028): No heartbeat from core client for 30 sec - exiting 09:13:36 (6028): No heartbeat from core client for 30 sec - exiting 09:13:37 (6028): No heartbeat from core client for 30 sec - exiting 09:13:38 (6028): No heartbeat from core client for 30 sec - exiting 09:13:39 (6028): No heartbeat from core client for 30 sec - exiting 09:13:40 (6028): No heartbeat from core client for 30 sec - exiting 09:13:42 (6028): No heartbeat from core client for 30 sec - exiting 09:13:43 (6028): No heartbeat from core client for 30 sec - exiting 09:13:44 (6028): No heartbeat from core client for 30 sec - exiting 09:13:45 (6028): No heartbeat from core client for 30 sec - exiting 09:13:46 (6028): No heartbeat from core client for 30 sec - exiting 09:13:47 (6028): No heartbeat from core client for 30 sec - exiting 09:13:48 (6028): No heartbeat from core client for 30 sec - exiting 09:13:49 (6028): No heartbeat from core client for 30 sec - exiting 09:13:50 (6028): No heartbeat from core client for 30 sec - exiting 09:13:51 (6028): No heartbeat from core client for 30 sec - exiting 09:13:52 (6028): No heartbeat from core client for 30 sec - exiting 09:13:54 (6028): No heartbeat from core client for 30 sec - exiting 09:13:55 (6028): No heartbeat from core client for 30 sec - exiting 09:13:56 (6028): No heartbeat from core client for 30 sec - exiting 09:13:57 (6028): No heartbeat from core client for 30 sec - exiting 09:13:58 (6028): No heartbeat from core client for 30 sec - exiting 09:13:59 (6028): No heartbeat from core client for 30 sec - exiting 09:14:00 (6028): No heartbeat from core client for 30 sec - exiting 09:14:01 (6028): No heartbeat from core client for 30 sec - exiting 09:14:02 (6028): No heartbeat from core client for 30 sec - exiting 09:14:03 (6028): No heartbeat from core client for 30 sec - exiting 09:14:04 (6028): No heartbeat from core client for 30 sec - exiting 09:14:06 (6028): No heartbeat from core client for 30 sec - exiting 09:14:07 (6028): No heartbeat from core client for 30 sec - exiting 09:14:08 (6028): No heartbeat from core client for 30 sec - exiting 09:14:09 (6028): No heartbeat from core client for 30 sec - exiting 09:14:10 (6028): No heartbeat from core client for 30 sec - exiting 09:14:11 (6028): No heartbeat from core client for 30 sec - exiting 09:14:12 (6028): No heartbeat from core client for 30 sec - exiting 09:14:13 (6028): No heartbeat from core client for 30 sec - exiting 09:14:14 (6028): No heartbeat from core client for 30 sec - exiting 09:14:15 (6028): No heartbeat from core client for 30 sec - exiting 09:14:16 (6028): No heartbeat from core client for 30 sec - exiting 09:14:18 (6028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:22:34 (4388): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7748, selfPID=6348, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2912, iMonCtr=2 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1112, iMonCtr=2 Leaving CPDN_Main::Monitor... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2512, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 09:07:58 (2572): No heartbeat from core client for 30 sec - exiting 09:07:59 (2572): No heartbeat from core client for 30 sec - exiting 09:08:00 (2572): No heartbeat from core client for 30 sec - exiting 09:08:01 (2572): No heartbeat from core client for 30 sec - exiting 09:08:03 (2572): No heartbeat from core client for 30 sec - exiting 09:08:04 (2572): No heartbeat from core client for 30 sec - exiting </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_anz_a8tb_2012_1_008617759_1_6.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_a8tb_2012_1_008617759_1_7.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_a8tb_2012_1_008617759_1_8.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_a8tb_2012_1_008617759_1_9.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_a8tb_2012_1_008617759_1_10.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_a8tb_2012_1_008617759_1_11.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_a8tb_2012_1_008617759_1_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
23 May 2014 13:16:46	1241169	16610583	hadam3p_anz_a8tb_2012_1_008617759_1	57,899	429,456	7.4173
19 May 2014 16:26:09	1241169	16610583	hadam3p_anz_a8tb_2012_1_008617759_1	46,379	334,238	7.2067
15 May 2014 09:30:16	1241169	16610583	hadam3p_anz_a8tb_2012_1_008617759_1	34,859	254,442	7.2992
12 May 2014 22:09:02	1241169	16610583	hadam3p_anz_a8tb_2012_1_008617759_1	23,339	169,924	7.2807
08 May 2014 08:58:50	1241169	16610583	hadam3p_anz_a8tb_2012_1_008617759_1	11,819	81,402	6.8874