Task 15509823

Name	hadam3p_pnw_7715_2006_1_007674262_2
Workunit	7829349
Created	24 Dec 2012, 16:06:53 UTC
Sent	24 Dec 2012, 16:15:58 UTC
Report deadline	6 Dec 2013, 21:35:58 UTC
Received	4 Apr 2013, 23:03:38 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1237595
Run time	3 days 20 hours 35 min 47 sec
CPU time	3 days 12 hours 0 min 12 sec
Validate state	Invalid
Credit	1,754.30
Device peak FLOPS	2.76 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Pacific North West v6.09 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4584, selfPID=4584, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1216, selfPID=1216, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2116, selfPID=2116, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3820, selfPID=3604, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 17:07:32 (3808): No heartbeat from core client for 30 sec - exiting 17:07:33 (3808): No heartbeat from core client for 30 sec - exiting 17:07:34 (3808): No heartbeat from core client for 30 sec - exiting 17:07:35 (3808): No heartbeat from core client for 30 sec - exiting 17:07:36 (3808): No heartbeat from core client for 30 sec - exiting 17:07:37 (3808): No heartbeat from core client for 30 sec - exiting 17:07:38 (3808): No heartbeat from core client for 30 sec - exiting 17:07:39 (3808): No heartbeat from core client for 30 sec - exiting 17:07:40 (3808): No heartbeat from core client for 30 sec - exiting 17:07:41 (3808): No heartbeat from core client for 30 sec - exiting 17:07:42 (3808): No heartbeat from core client for 30 sec - exiting 17:07:43 (3808): No heartbeat from core client for 30 sec - exiting 17:07:44 (3808): No heartbeat from core client for 30 sec - exiting 17:07:45 (3808): No heartbeat from core client for 30 sec - exiting 17:07:46 (3808): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1932, selfPID=1932, iMonCtr=2 CPDN Monitor - Quit request from BOINC... 14:44:53 (4988): Can't acquire lockfile (32) - waiting 35s 14:45:04 (3496): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN procesController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5512, selfPID=5348, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5824, selfPID=5824, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:49:19 (3056): No heartbeat from core client for 30 sec - exiting 19:49:21 (3056): No heartbeat from core client for 30 sec - exiting 19:49:22 (3056): No heartbeat from core client for 30 sec - exiting 19:49:23 (3056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:28:08 (3772): No heartbeat from core client for 30 sec - exiting 15:28:09 (3772): No heartbeat from core client for 30 sec - exiting 15:28:10 (3772): No heartbeat from core client for 30 sec - exiting 15:28:11 (3772): No heartbeat from core client for 30 sec - exiting 15:28:12 (3772): No heartbeat from core client for 30 sec - exiting 15:28:13 (3772): No heartbeat from core client for 30 sec - exiting 15:28:14 (3772): No heartbeat from core client for 30 sec - exiting 15:28:15 (3772): No heartbeat from core client for 30 sec - exiting 15:28:16 (3772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5192, selfPID=5192, iMonCtr=2 02:32:42 (3656): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:14:19 (3436): No heartbeat from core client for 30 sec - exiting 00:14:20 (3436): No heartbeat from core client for 30 sec - exiting 00:14:21 (3436): No heartbeat from core client for 30 sec - exiting 00:14:22 (3436): No heartbeat from core client for 30 sec - exiting 00:14:23 (3436): No heartbeat from core client for 30 sec - exiting 00:14:24 (3436): No heartbeat from core client for 30 sec - exiting 00:14:25 (3436): No heartbeat from core client for 30 sec - exiting 00:14:26 (3436): No heartbeat from core client for 30 sec - exiting 00:14:28 (3436): No heartbeat from core client for 30 sec - exiting 00:14:29 (3436): No heartbeat from core client for 30 sec - exiting 00:14:30 (3436): No heartbeat from core client for 30 sec - exiting 00:14:30 (2704): Can't acquire lockfile (32) - waiting 35s 00:14:31 (3436): No heartbeat from core client for 30 sec - exiting 00:14:32 (3436): No heartbeat from core client for 30 sec - exiting 00:14:33 (3436): No heartbeat from core client for 30 sec - exiting 00:14:34 (3436): No heartbeat from core client for 30 sec - exiting 00:14:35 (3436): No heartbeat from core client for 30 sec - exiting 00:14:36 (3436): No heartbeat from core client for 30 sec - exiting 00:14:37 (3436): No heartbeat from core client for 30 sec - exiting 00:14:38 (3436): No heartbeat from core client for 30 sec - exiting 00:14:40 (3436): No heartbeat from core client for 30 sec - exiting 00:14:41 (3436): No heartbeat from core client for 30 sec - exiting 00:14:42 (3436): No heartbeat from core client for 30 sec - exiting 00:14:43 (3436): No heartbeat from core client for 30 sec - exiting 00:14:44 (3436): No heartbeat from core client for 30 sec - exiting 00:14:45 (3436): No heartbeat from core client for 30 sec - exiting 00:14:46 (3436): No heartbeat from core client for 30 sec - exiting 00:14:47 (3436): No heartbeat from core client for 30 sec - exiting 00:14:48 (3436): No heartbeat from core client for 30 sec - exiting 00:14:49 (3436): No heartbeat from core client for 30 sec - exiting 00:14:50 (3436): No heartbeat from core client for 30 sec - exiting 00:14:52 (3436): No heartbeat from core client for 30 sec - exiting 00:14:53 (3436): No heartbeat from core client for 30 sec - exiting 00:14:54 (3436): No heartbeat from core client for 30 sec - exiting 00:14:55 (3436): No heartbeat from core client for 30 sec - exiting 00:14:56 (3436): No heartbeat from core client for 30 sec - exiting 00:14:57 (3436): No heartbeat from core client for 30 sec - exiting 00:14:58 (3436): No heartbeat from core client for 30 sec - exiting 00:14:59 (3436): No heartbeat from core client for 30 sec - exiting 00:15:00 (3436): No heartbeat from core client for 30 sec - exiting 00:15:01 (3436): No heartbeat from core client for 30 sec - exiting 00:15:02 (3436): No heartbeat from core client for 30 sec - exiting 00:15:04 (3436): No heartbeat from core client for 30 sec - exiting 00:15:05 (3436): No heartbeat from core client for 30 sec - exiting 00:15:05 (2704): Can't acquire lockfile (32) - exiting 00:15:05 (2704): Error: The process cannot access the file because it is being used by another process. (0x20) 00:15:06 (3436): No heartbeat from core client for 30 sec - exiting 00:15:07 (3436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:51:32 (4128): Can't acquire lockfile (32) - waiting 35s 17:51:58 (2548): No heartbeat from core client for 30 sec - exiting 17:51:59 (2548): No heartbeat from core client for 30 sec - exiting 17:52:00 (2548): No heartbeat from core client for 30 sec - exiting 17:52:01 (2548): No heartbeat from core client for 30 sec - exiting 17:52:03 (2548): No heartbeat from core client for 30 sec - exiting 17:52:04 (2548): No heartbeat from core client for 30 sec - exiting 17:52:05 (2548): No heartbeat from core client for 30 sec - exiting 17:52:06 (2548): No heartbeat from core client for 30 sec - exiting 17:52:07 (2548): No heartbeat from core client for 30 sec - exiting 17:52:07 (4128): Can't acquire lockfile (32) - exiting 17:52:07 (4128): Error: The process cannot access the file because it is being used by another process. (0x20) 17:52:08 (2548): No heartbeat from core client for 30 sec - exiting 17:52:09 (2548): No heartbeat from core client for 30 sec - exiting 17:52:10 (2548): No heartbeat from core client for 30 sec - exiting 17:52:11 (2548): No heartbeat from core client for 30 sec - exiting 17:52:12 (2548): No heartbeat from core client for 30 sec - exiting 17:52:13 (2548): No heartbeat from core client for 30 sec - exiting 17:52:15 (2548): No heartbeat from core client for 30 sec - exiting 17:52:16 (2548): No heartbeat from core client for 30 sec - exiting 17:52:16 (7952): Can't acquire lockfile (32) - waiting 35s 17:52:17 (2548): No heartbeat from core client for 30 sec - exiting 17:52:18 (2548): No heartbeat from core client for 30 sec - exiting 17:52:19 (2548): No heartbeat from core client for 30 sec - exiting 17:52:20 (2548): No heartbeat from core client for 30 sec - exiting 17:52:21 (2548): No heartbeat from core client for 30 sec - exiting 17:52:22 (2548): No heartbeat from core client for 30 sec - exiting 17:52:23 (2548): No heartbeat from core client for 30 sec - exiting 17:52:24 (2548): No heartbeat from core client for 30 sec - exiting 17:52:26 (2548): No heartbeat from core client for 30 sec - exiting 17:52:27 (2548): No heartbeat from core client for 30 sec - exiting 17:52:28 (2548): No heartbeat from core client for 30 sec - exiting 17:52:29 (2548): No heartbeat from core client for 30 sec - exiting 17:52:30 (2548): No heartbeat from core client for 30 sec - exiting 17:52:31 (2548): No heartbeat from core client for 30 sec - exiting 17:52:32 (2548): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: READHIST: End of file in READ from history file for namelist NLIHISTO tmp/xaakg.pipe_dummy 2048 Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 7 Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_pnw_7715_2006_1_007674262_2_8.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_7715_2006_1_007674262_2_9.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_7715_2006_1_007674262_2_10.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_7715_2006_1_007674262_2_11.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_7715_2006_1_007674262_2_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
20 Mar 2013 01:10:11	1237595	15509823	hadam3p_pnw_7715_2006_1_007674262_2	80,736	268,984	3.3316
25 Feb 2013 00:11:18	1237595	15509823	hadam3p_pnw_7715_2006_1_007674262_2	69,216	230,972	3.3370
18 Feb 2013 02:46:57	1237595	15509823	hadam3p_pnw_7715_2006_1_007674262_2	57,717	192,287	3.3315
18 Feb 2013 01:46:43	1237595	15509823	hadam3p_pnw_7715_2006_1_007674262_2	57,709	191,802	3.3236
18 Feb 2013 00:01:17	1237595	15509823	hadam3p_pnw_7715_2006_1_007674262_2	57,700	191,375	3.3167
14 Feb 2013 01:03:58	1237595	15509823	hadam3p_pnw_7715_2006_1_007674262_2	57,696	190,938	3.3094
03 Feb 2013 07:52:37	1237595	15509823	hadam3p_pnw_7715_2006_1_007674262_2	46,176	152,170	3.2954
24 Jan 2013 08:16:02	1237595	15509823	hadam3p_pnw_7715_2006_1_007674262_2	34,656	112,755	3.2535
17 Jan 2013 03:53:34	1237595	15509823	hadam3p_pnw_7715_2006_1_007674262_2	23,136	73,916	3.1948
04 Jan 2013 06:59:54	1237595	15509823	hadam3p_pnw_7715_2006_1_007674262_2	11,616	36,530	3.1448