Task 18344191

Name	hadam3p_anz_f783_2012_1_009778728_0
Workunit	9834692
Created	24 Apr 2015, 15:18:01 UTC
Sent	29 Apr 2015, 22:53:19 UTC
Report deadline	11 Apr 2016, 4:13:19 UTC
Received	13 May 2015, 6:57:59 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1362412
Run time	2 days 20 hours 53 min 14 sec
CPU time	2 days 17 hours 45 min 8 sec
Validate state	Invalid
Credit	3,987.46
Device peak FLOPS	4.38 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10 windows_intelx86
Stderr	<core_client_version>7.4.42</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1696, selfPID=1696, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:22:56 (9992): Can't acquire lockfile (32) - waiting 35s 18:23:01 (3180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9948, selfPID=9948, iMonCtr=2 CPDN Monitor - Quit request from BOINC... 09:13:45 (8160): No heartbeat from core client for 30 sec - exiting 09:13:46 (8160): No heartbeat from core client for 30 sec - exiting 09:13:47 (8160): No heartbeat from core client for 30 sec - exiting 09:13:48 (8160): No heartbeat from core client for 30 sec - exiting 09:13:49 (8160): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:54:46 (8364): No heartbeat from core client for 30 sec - exiting 08:54:48 (8364): No heartbeat from core client for 30 sec - exiting 08:54:49 (8364): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4652, selfPID=4652, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5076, selfPID=5076, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10020, selfPID=10020, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6392, selfPID=6392, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:50:49 (2964): No heartbeat from core client for 30 sec - exiting 09:50:50 (2964): No heartbeat from core client for 30 sec - exiting 09:50:51 (2964): No heartbeat from core client for 30 sec - exiting 09:50:52 (2964): No heartbeat from core client for 30 sec - exiting 09:50:53 (2964): No heartbeat from core client for 30 sec - exiting 09:50:54 (2964): No heartbeat from core client for 30 sec - exiting 09:50:55 (2964): No heartbeat from core client for 30 sec - exiting 09:50:56 (2964): No heartbeat from core client for 30 sec - exiting 09:50:57 (2964): No heartbeat from core client for 30 sec - exiting 09:50:58 (2964): No heartbeat from core client for 30 sec - exiting 09:50:59 (2964): No heartbeat from core client for 30 sec - exiting 09:51:00 (2964): No heartbeat from core client for 30 sec - exiting 09:51:01 (2964): No heartbeat from core client for 30 sec - exiting 09:51:02 (2964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7220, selfPID=7220, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4244, selfPID=4244, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10184, selfPID=10184, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7428, selfPID=7428, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9412, selfPID=9412, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3172, selfPID=3172, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9636, selfPID=9636, iMonCtr=2 10:33:35 (8240): No heartbeat from core client for 30 sec - exiting 10:33:36 (8240): No heartbeat from core client for 30 sec - exiting 10:33:37 (8240): No heartbeat from core client for 30 sec - exiting 10:33:39 (8240): No heartbeat from core client for 30 sec - exiting 10:34:10 (8240): No heartbeat from core client for 30 sec - exiting 10:34:11 (8240): No heartbeat from core client for 30 sec - exiting 10:34:12 (8240): No heartbeat from core client for 30 sec - exiting 10:34:13 (8240): No heartbeat from core client for 30 sec - exiting 10:34:14 (8240): No heartbeat from core client for 30 sec - exiting 10:34:15 (8240): No heartbeat from core client for 30 sec - exiting 10:34:16 (8240): No heartbeat from core client for 30 sec - exiting 10:34:17 (8240): No heartbeat from core client for 30 sec - exiting 10:34:18 (8240): No heartbeat from core client for 30 sec - exiting 10:34:19 (8240): No heartbeat from core client for 30 sec - exiting 10:34:20 (8240): No heartbeat from core client for 30 sec - exiting 10:34:21 (8240): No heartbeat from core client for 30 sec - exiting 10:34:22 (8240): No heartbeat from core client for 30 sec - exiting 10:34:23 (8240): No heartbeat from core client for 30 sec - exiting 10:34:24 (8240): No heartbeat from core client for 30 sec - exiting 10:34:25 (8240): No heartbeat from core client for 30 sec - exiting 10:34:26 (8240): No heartbeat from core client for 30 sec - exiting 10:34:27 (8240): No heartbeat from core client for 30 sec - exiting 10:34:28 (8240): No heartbeat from core client for 30 sec - exiting 10:34:29 (8240): No heartbeat from core client for 30 sec - exiting 10:34:30 (8240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7504, selfPID=7504, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8872, selfPID=8872, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3104, selfPID=3104, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2092, selfPID=2092, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6352, selfPID=6352, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9068, selfPID=9068, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:20:55 (7996): No heartbeat from core client for 30 sec - exiting 09:20:56 (7996): No heartbeat from core client for 30 sec - exiting 09:20:57 (7996): No heartbeat from core client for 30 sec - exiting 09:20:58 (7996): No heartbeat from core client for 30 sec - exiting 09:20:59 (7996): No heartbeat from core client for 30 sec - exiting 09:21:01 (7996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:04:12 (8840): No heartbeat from core client for 30 sec - exiting 08:04:13 (8840): No heartbeat from core client for 30 sec - exiting 08:04:14 (8840): No heartbeat from core client for 30 sec - exiting 08:04:15 (8840): No heartbeat from core client for 30 sec - exiting 08:04:16 (8840): No heartbeat from core client for 30 sec - exiting 08:04:17 (8840): No heartbeat from core client for 30 sec - exiting 08:04:18 (8840): No heartbeat from core client for 30 sec - exiting 08:04:19 (8840): No heartbeat from core client for 30 sec - exiting 08:04:20 (8840): No heartbeat from core client for 30 sec - exiting 08:04:21 (8840): No heartbeat from core client for 30 sec - exiting 08:04:22 (8840): No heartbeat from core client for 30 sec - exiting 08:04:23 (8840): No heartbeat from core client for 30 sec - exiting Global Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=6352, iMonCtr=1 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=0, iMonCtr=2 CPDN Monitor - No 'heartbeat' from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=1396, iMonCtr=1 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=0, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5100, selfPID=5880, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_anz_f783_2012_1_009778728_0_9.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_f783_2012_1_009778728_0_10.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_f783_2012_1_009778728_0_11.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_f783_2012_1_009778728_0_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
12 May 2015 09:34:36	1362412	18344191	hadam3p_anz_f783_2012_1_009778728_0	92,459	227,700	2.4627
10 May 2015 17:35:38	1362412	18344191	hadam3p_anz_f783_2012_1_009778728_0	80,939	199,095	2.4598
08 May 2015 20:10:04	1362412	18344191	hadam3p_anz_f783_2012_1_009778728_0	69,419	170,560	2.4570
08 May 2015 19:26:18	1362412	18344191	hadam3p_anz_f783_2012_1_009778728_0	57,899	141,580	2.4453
08 May 2015 19:14:29	1362412	18344191	hadam3p_anz_f783_2012_1_009778728_0	46,379	113,315	2.4432
08 May 2015 19:12:15	1362412	18344191	hadam3p_anz_f783_2012_1_009778728_0	34,859	85,153	2.4428
08 May 2015 19:10:05	1362412	18344191	hadam3p_anz_f783_2012_1_009778728_0	23,339	57,033	2.4437
08 May 2015 19:02:32	1362412	18344191	hadam3p_anz_f783_2012_1_009778728_0	11,819	29,017	2.4551