Task 18506071

Name	hadam3p_anz_n3jn_2007_1_009861183_0
Workunit	9899680
Created	29 May 2015, 15:48:16 UTC
Sent	1 Jun 2015, 18:52:00 UTC
Report deadline	14 May 2016, 0:12:00 UTC
Received	29 Sep 2015, 20:10:44 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1169010
Run time	7 days 23 hours 19 min 37 sec
CPU time	5 days 16 hours 13 min 45 sec
Validate state	Invalid
Credit	2,993.82
Device peak FLOPS	2.43 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10 windows_intelx86
Stderr	<core_client_version>7.4.27</core_client_version> <![CDATA[ <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=812, selfPID=4968, iMonCtr=1 Model crash detected, will try to restart... 21:01:23 (4480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:01:24 (4480): No heartbeat from core client for 30 sec - exiting 21:01:25 (4480): No heartbeat from core client for 30 sec - exiting 21:01:26 (4480): No heartbeat from core client for 30 sec - exiting 21:01:28 (4480): No heartbeat from core client for 30 sec - exiting 21:01:29 (4480): No heartbeat from core client for 30 sec - exiting 21:01:30 (4480): No heartbeat from core client for 30 sec - exiting 21:01:31 (4480): No heartbeat from core client for 30 sec - exiting 21:01:32 (4480): No heartbeat from core client for 30 sec - exiting 21:01:33 (4480): No heartbeat from core client for 30 sec - exiting 21:01:34 (4480): No heartbeat from core client for 30 sec - exiting 21:01:35 (4480): No heartbeat from core client for 30 sec - exiting 21:01:36 (4480): No heartbeat from core client for 30 sec - exiting 21:01:37 (4480): No heartbeat from core client for 30 sec - exiting 21:01:38 (4480): No heartbeat from core client for 30 sec - exiting 21:01:39 (4480): No heartbeat from core client for 30 sec - exiting 21:01:40 (4480): No heartbeat from core client for 30 sec - exiting 21:01:41 (4480): No heartbeat from core client for 30 sec - exiting 21:01:42 (4480): No heartbeat from core client for 30 sec - exiting 21:01:43 (4480): No heartbeat from core client for 30 sec - exiting 21:01:44 (4480): No heartbeat from core client for 30 sec - exiting 21:01:45 (4480): No heartbeat from core client for 30 sec - exiting 21:01:46 (4480): No heartbeat from core client for 30 sec - exiting 21:01:47 (4480): No heartbeat from core client for 30 sec - exiting 21:01:48 (4480): No heartbeat from core client for 30 sec - exiting 21:01:49 (4480): No heartbeat from core client for 30 sec - exiting 20:45:08 (2356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:45:09 (2356): No heartbeat from core client for 30 sec - exiting 20:45:10 (2356): No heartbeat from core client for 30 sec - exiting 20:45:11 (2356): No heartbeat from core client for 30 sec - exiting 20:45:12 (2356): No heartbeat from core client for 30 sec - exiting 20:45:13 (2356): No heartbeat from core client for 30 sec - exiting 20:45:14 (2356): No heartbeat from core client for 30 sec - exiting 20:45:15 (2356): No heartbeat from core client for 30 sec - exiting 20:45:16 (2356): No heartbeat from core client for 30 sec - exiting 20:45:17 (2356): No heartbeat from core client for 30 sec - exiting 20:45:18 (2356): No heartbeat from core client for 30 sec - exiting 21:02:53 (4832): No heartbeat from core client for 30 sec - exiting 21:02:54 (4832): No heartbeat from core client for 30 sec - exiting 21:02:55 (4832): No heartbeat from core client for 30 sec - exiting 21:02:56 (4832): No heartbeat from core client for 30 sec - exiting 21:02:57 (4832): No heartbeat from core client for 30 sec - exiting 21:02:58 (4832): No heartbeat from core client for 30 sec - exiting 21:02:59 (4832): No heartbeat from core client for 30 sec - exiting 21:03:00 (4832): No heartbeat from core client for 30 sec - exiting 21:03:01 (4832): No heartbeat from core client for 30 sec - exiting 21:03:02 (4832): No heartbeat from core client for 30 sec - exiting 21:03:03 (4832): No heartbeat from core client for 30 sec - exiting 21:03:04 (4832): No heartbeat from core client for 30 sec - exiting 21:03:05 (4832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3336, selfPID=5848, iMonCtr=1 Model crash detected, will try to restart... 19:47:49 (4788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:47:43 (5348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5920, iMonCtr=2 Model crash detected, will try to restart... 21:52:54 (4960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:52:55 (4960): No heartbeat from core client for 30 sec - exiting 21:52:56 (4960): No heartbeat from core client for 30 sec - exiting 21:52:57 (4960): No heartbeat from core client for 30 sec - exiting 21:52:58 (4960): No heartbeat from core client for 30 sec - exiting 21:52:59 (4960): No heartbeat from core client for 30 sec - exiting 21:53:00 (4960): No heartbeat from core client for 30 sec - exiting 21:53:01 (4960): No heartbeat from core client for 30 sec - exiting 21:53:02 (4960): No heartbeat from core client for 30 sec - exiting 21:53:03 (4960): No heartbeat from core client for 30 sec - exiting 21:53:04 (4960): No heartbeat from core client for 30 sec - exiting 23:23:24 (4372): No heartbeat from core client for 30 sec - exiting 23:23:25 (4372): No heartbeat from core client for 30 sec - exiting 23:23:26 (4372): No heartbeat from core client for 30 sec - exiting 23:23:27 (4372): No heartbeat from core client for 30 sec - exiting 23:23:28 (4372): No heartbeat from core client for 30 sec - exiting 23:23:29 (4372): No heartbeat from core client for 30 sec - exiting 23:23:30 (4372): No heartbeat from core client for 30 sec - exiting 23:23:31 (4372): No heartbeat from core client for 30 sec - exiting 23:23:32 (4372): No heartbeat from core client for 30 sec - exiting 23:23:33 (4372): No heartbeat from core client for 30 sec - exiting 23:23:34 (4372): No heartbeat from core client for 30 sec - exiting 23:23:35 (4372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5344, selfPID=5344, iMonCtr=2 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2476, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4716, iMonCtr=2 Model crash detected, will try to restart... 22:15:40 (5740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:57:18 (2072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:57:28 (2712): Can't acquire lockfile (32) - waiting 35s CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1244, selfPID=3460, iMonCtr=1 Model crash detected, will try to restart... 10:03:46 (4352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4156, selfPID=5136, iMonCtr=1 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5120, iMonCtr=2 19:48:48 (5020): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:24:04 (5960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:24:06 (5960): No heartbeat from core client for 30 sec - exiting 19:24:07 (5960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 19:17:56 (5716): No heartbeat from core client for 30 sec - exiting 19:17:57 (5716): No heartbeat from core client for 30 sec - exiting 19:17:58 (5716): No heartbeat from core client for 30 sec - exiting 19:17:59 (5716): No heartbeat from core client for 30 sec - exiting 19:18:00 (5716): No heartbeat from core client for 30 sec - exiting 19:18:01 (5716): No heartbeat from core client for 30 sec - exiting 19:18:02 (5716): No heartbeat from core client for 30 sec - exiting 19:18:03 (5716): No heartbeat from core client for 30 sec - exiting 19:18:04 (5716): No heartbeat from core client for 30 sec - exiting 19:18:05 (5716): No heartbeat from core client for 30 sec - exiting 19:18:06 (5716): No heartbeat from core client for 30 sec - exiting 19:18:07 (5716): No heartbeat from core client for 30 sec - exiting 19:18:08 (5716): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2976, iMonCtr=2 Model crash detected, will try to restart... 05:04:57 (1076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:06:32 (4872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5544, selfPID=5544, iMonCtr=2 05:21:13 (1596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:05:32 (3556): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:31:44 (3408): No heartbeat from core client for 30 sec - exiting 22:31:45 (3408): No heartbeat from core client for 30 sec - exiting 22:31:46 (3408): No heartbeat from core client for 30 sec - exiting 22:31:48 (3408): No heartbeat from core client for 30 sec - exiting 22:31:49 (3408): No heartbeat from core client for 30 sec - exiting 22:31:50 (3408): No heartbeat from core client for 30 sec - exiting 22:31:51 (3408): No heartbeat from core client for 30 sec - exiting 22:31:52 (3408): No heartbeat from core client for 30 sec - exiting 22:31:53 (3408): No heartbeat from core client for 30 sec - exiting 22:31:54 (3408): No heartbeat from core client for 30 sec - exiting 22:31:55 (3408): No heartbeat from core client for 30 sec - exiting 22:31:56 (3408): No heartbeat from core client for 30 sec - exiting 22:31:57 (3408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:40:53 (4676): No heartbeat from core client for 30 sec - exiting 12:40:55 (4676): No heartbeat from core client for 30 sec - exiting 12:40:56 (4676): No heartbeat from core client for 30 sec - exiting 12:40:57 (4676): No heartbeat from core client for 30 sec - exiting 12:40:58 (4676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5608, selfPID=5428, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5740, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5984, selfPID=6048, iMonCtr=1 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4288, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5296, selfPID=4728, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:36:37 (4236): No heartbeat from core client for 30 sec - exiting 21:36:38 (4236): No heartbeat from core client for 30 sec - exiting 21:36:39 (4236): No heartbeat from core client for 30 sec - exiting 21:36:40 (4236): No heartbeat from core client for 30 sec - exiting 21:36:41 (4236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:20:21 (3980): No heartbeat from core client for 30 sec - exiting 23:20:22 (3980): No heartbeat from core client for 30 sec - exiting 23:20:23 (3980): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3092, selfPID=4740, iMonCtr=1 Model crash detected, will try to restart... 22:20:12 (5892): No heartbeat from core client for 30 sec - exiting 22:20:13 (5892): No heartbeat from core client for 30 sec - exiting 22:20:14 (5892): No heartbeat from core client for 30 sec - exiting 22:20:15 (5892): No heartbeat from core client for 30 sec - exiting 22:20:16 (5892): No heartbeat from core client for 30 sec - exiting 22:20:17 (5892): No heartbeat from core client for 30 sec - exiting 22:20:18 (5892): No heartbeat from core client for 30 sec - exiting 22:20:19 (5892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:27:56 (2516): No heartbeat from core client for 30 sec - exiting 00:27:57 (2516): No heartbeat from core client for 30 sec - exiting 00:27:58 (2516): No heartbeat from core client for 30 sec - exiting 00:27:59 (2516): No heartbeat from core client for 30 sec - exiting 00:28:00 (2516): No heartbeat from core client for 30 sec - exiting 00:28:01 (2516): No heartbeat from core client for 30 sec - exiting 00:28:02 (2516): No heartbeat from core client for 30 sec - exiting 00:28:03 (2516): No heartbeat from core client for 30 sec - exiting 00:28:04 (2516): No heartbeat from core client for 30 sec - exiting 00:28:05 (2516): No heartbeat from core client for 30 sec - exiting 00:28:06 (2516): No heartbeat from core client for 30 sec - exiting 00:28:07 (2516): No heartbeat from core client for 30 sec - exiting 00:28:08 (2516): No heartbeat from core client for 30 sec - exiting 00:28:09 (2516): No heartbeat from core client for 30 sec - exiting 00:28:10 (2516): No heartbeat from core client for 30 sec - exiting 00:28:11 (2516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:18:31 (3432): No heartbeat from core client for 30 sec - exiting 01:18:32 (3432): No heartbeat from core client for 30 sec - exiting 01:18:33 (3432): No heartbeat from core client for 30 sec - exiting 01:18:34 (3432): No heartbeat from core client for 30 sec - exiting 01:18:35 (3432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:07:13 (2304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5136, iMonCtr=2 Model crash detected, will try to restart... 19:35:09 (5684): No heartbeat from core client for 30 sec - exiting 19:35:10 (5684): No heartbeat from core client for 30 sec - exiting 19:35:11 (5684): No heartbeat from core client for 30 sec - exiting 19:35:12 (5684): No heartbeat from core client for 30 sec - exiting 19:35:13 (5684): No heartbeat from core client for 30 sec - exiting 19:35:14 (5684): No heartbeat from core client for 30 sec - exiting 19:35:15 (5684): No heartbeat from core client for 30 sec - exiting 19:35:16 (5684): No heartbeat from core client for 30 sec - exiting 19:35:17 (5684): No heartbeat from core client for 30 sec - exiting 19:35:18 (5684): No heartbeat from core client for 30 sec - exiting 19:35:19 (5684): No heartbeat from core client for 30 sec - exiting 19:35:20 (5684): No heartbeat from core client for 30 sec - exiting 19:35:21 (5684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4876, iMonCtr=2 19:42:53 (5768): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3776, selfPID=3776, iMonCtr=2 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3240, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4856, iMonCtr=2 Model crash detected, will try to restart... 23:49:17 (5204): No heartbeat from core client for 30 sec - exiting 23:49:19 (5204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Model crashed: Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_anz_n3jn_2007_1_009861183_0_7.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_n3jn_2007_1_009861183_0_8.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_n3jn_2007_1_009861183_0_9.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_n3jn_2007_1_009861183_0_10.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_n3jn_2007_1_009861183_0_11.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_n3jn_2007_1_009861183_0_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
19 Sep 2015 08:05:34	1169010	18506071	hadam3p_anz_n3jn_2007_1_009861183_0	69,419	451,579	6.5051
09 Sep 2015 21:26:47	1169010	18506071	hadam3p_anz_n3jn_2007_1_009861183_0	57,899	383,051	6.6158
29 Aug 2015 21:38:54	1169010	18506071	hadam3p_anz_n3jn_2007_1_009861183_0	46,379	319,319	6.8850
22 Aug 2015 17:07:27	1169010	18506071	hadam3p_anz_n3jn_2007_1_009861183_0	34,859	240,978	6.9129
21 Jul 2015 22:37:55	1169010	18506071	hadam3p_anz_n3jn_2007_1_009861183_0	23,339	159,240	6.8229
16 Jun 2015 22:11:20	1169010	18506071	hadam3p_anz_n3jn_2007_1_009861183_0	11,819	88,541	7.4914