Task 13983539

Name	hadam3p_pnw_8vqz_2000_1_007714103_0
Workunit	7869211
Created	25 Jan 2012, 19:59:14 UTC
Sent	30 Jan 2012, 10:44:13 UTC
Report deadline	11 Jan 2013, 16:04:13 UTC
Received	12 Mar 2012, 12:37:37 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1025527
Run time	6 days 13 hours 30 min 33 sec
CPU time	5 days 14 hours 18 min 5 sec
Validate state	Invalid
Credit	2,254.93
Device peak FLOPS	1.50 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Pacific North West v6.09 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:35:35 (3836): No heartbeat from core client for 30 sec - exiting 07:35:37 (3836): No heartbeat from core client for 30 sec - exiting 07:35:38 (3836): No heartbeat from core client for 30 sec - exiting 07:35:39 (3836): No heartbeat from core client for 30 sec - exiting 07:35:40 (3836): No heartbeat from core client for 30 sec - exiting 07:35:41 (3836): No heartbeat from core client for 30 sec - exiting 07:35:43 (3836): No heartbeat from core client for 30 sec - exiting 07:35:44 (3836): No heartbeat from core client for 30 sec - exiting 07:35:45 (3836): No heartbeat from core client for 30 sec - exiting 07:35:46 (3836): No heartbeat from core client for 30 sec - exiting 07:35:47 (3836): No heartbeat from core client for 30 sec - exiting 07:35:48 (3836): No heartbeat from core client for 30 sec - exiting 07:35:50 (3836): No heartbeat from core client for 30 sec - exiting 07:35:51 (3836): No heartbeat from core client for 30 sec - exiting 07:35:52 (3836): No heartbeat from core client for 30 sec - exiting 07:35:53 (3836): No heartbeat from core client for 30 sec - exiting 07:35:54 (3836): No heartbeat from core client for 30 sec - exiting 07:35:55 (3836): No heartbeat from core client for 30 sec - exiting 07:35:57 (3836): No heartbeat from core client for 30 sec - exiting 07:35:58 (3836): No heartbeat from core client for 30 sec - exiting 07:35:59 (3836): No heartbeat from core client for 30 sec - exiting 07:36:00 (3836): No heartbeat from core client for 30 sec - exiting 07:36:01 (3836): No heartbeat from core client for 30 sec - exiting 07:36:02 (3836): No heartbeat from core client for 30 sec - exiting 07:36:04 (3836): No heartbeat from core client for 30 sec - exiting 07:36:05 (3836): No heartbeat from core client for 30 sec - exiting 07:36:06 (3836): No heartbeat from core client for 30 sec - exiting 07:36:07 (3836): No heartbeat from core client for 30 sec - exiting 07:36:08 (3836): No heartbeat from core client for 30 sec - exiting 07:36:10 (3836): No heartbeat from core client for 30 sec - exiting 07:36:11 (3836): No heartbeat from core client for 30 sec - exiting 07:36:12 (3836): No heartbeat from core client for 30 sec - exiting 07:36:13 (3836): No heartbeat from core client for 30 sec - exiting 07:36:14 (3836): No heartbeat from core client for 30 sec - exiting 07:36:15 (3836): No heartbeat from core client for 30 sec - exiting 07:36:17 (3836): No heartbeat from core client for 30 sec - exiting 07:36:18 (3836): No heartbeat from core client for 30 sec - exiting 07:36:19 (3836): No heartbeat from core client for 30 sec - exiting 07:36:20 (3836): No heartbeat from core client for 30 sec - exiting 07:36:21 (3836): No heartbeat from core client for 30 sec - exiting 07:36:22 (3836): No heartbeat from core client for 30 sec - exiting 07:36:24 (3836): No heartbeat from core client for 30 sec - exiting 07:36:25 (3836): No heartbeat from core client for 30 sec - exiting 07:37:01 (3836): No heartbeat from core client for 30 sec - exiting 07:37:02 (3836): No heartbeat from core client for 30 sec - exiting 07:37:04 (3836): No heartbeat from core client for 30 sec - exiting 07:37:05 (3836): No heartbeat from core client for 30 sec - exiting 07:37:06 (3836): No heartbeat from core client for 30 sec - exiting 07:37:07 (3836): No heartbeat from core client for 30 sec - exiting 07:37:08 (3836): No heartbeat from core client for 30 sec - exiting 07:37:10 (3836): No heartbeat from core client for 30 sec - exiting 07:37:11 (3836): No heartbeat from core client for 30 sec - exiting 07:37:12 (3836): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:37:13 (3836): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... RegCPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... ReCPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3792, selfPID=3792, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3448, selfPID=3448, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3720, selfPID=3720, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5500, selfPID=5500, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4180, selfPID=4180, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 06:42:13 (428): No heartbeat from core client for 30 sec - exiting 06:42:14 (428): No heartbeat from core client for 30 sec - exiting 06:42:15 (428): No heartbeat from core client for 30 sec - exiting 06:42:17 (428): No heartbeat from core client for 30 sec - exiting 06:42:18 (428): No heartbeat from core client for 30 sec - exiting 06:42:19 (428): No heartbeat from core client for 30 sec - exiting 06:42:20 (428): No heartbeat from core client for 30 sec - exiting 06:42:21 (428): No heartbeat from core client for 30 sec - exiting 06:42:23 (428): No heartbeat from core client for 30 sec - exiting 06:42:24 (428): No heartbeat from core client for 30 sec - exiting 06:42:25 (428): No heartbeat from core client for 30 sec - exiting 06:42:26 (428): No heartbeat from core client for 30 sec - exiting 06:42:27 (428): No heartbeat from core client for 30 sec - exiting 06:42:29 (428): No heartbeat from core client for 30 sec - exiting 06:42:30 (428): No heartbeat from core client for 30 sec - exiting 06:42:31 (428): No heartbeat from core client for 30 sec - exiting 06:42:32 (428): No heartbeat from core client for 30 sec - exiting 06:42:34 (428): No heartbeat from core client for 30 sec - exiting 06:42:35 (428): No heartbeat from core client for 30 sec - exiting 06:42:36 (428): No heartbeat from core client for 30 sec - exiting 06:42:37 (428): No heartbeat from core client for 30 sec - exiting 06:42:38 (428): No heartbeat from core client for 30 sec - exiting 06:42:40 (428): No heartbeat from core client for 30 sec - exiting 06:42:41 (428): No heartbeat from core client for 30 sec - exiting 06:42:42 (428): No heartbeat from core client for 30 sec - exiting 06:42:43 (428): No heartbeat from core client for 30 sec - exiting 06:42:44 (428): No heartbeat from core client for 30 sec - exiting 06:42:46 (428): No heartbeat from core client for 30 sec - exiting 06:42:47 (428): No heartbeat from core client for 30 sec - exiting 06:42:48 (428): No heartbeat from core client for 30 sec - exiting 06:42:49 (428): No heartbeat from core client for 30 sec - exiting 06:42:51 (428): No heartbeat from core client for 30 sec - exiting 06:42:52 (428): No heartbeat from core client for 30 sec - exiting 06:42:53 (428): No heartbeat from core client for 30 sec - exiting 06:42:54 (428): No heartbeat from core client for 30 sec - exiting 06:42:55 (428): No heartbeat from core client for 30 sec - exiting 06:42:57 (428): No heartbeat from core client for 30 sec - exiting 06:42:58 (428): No heartbeat from core client for 30 sec - exiting 06:42:59 (428): No heartbeat from core client for 30 sec - exiting 06:43:00 (428): No heartbeat from core client for 30 sec - exiting 06:43:01 (428): No heartbeat from core client for 30 sec - exiting 06:43:03 (428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 05:33:15 (3892): No heartbeat from core client for 30 sec - exiting 05:33:16 (3892): No heartbeat from core client for 30 sec - exiting 05:33:17 (3892): No heartbeat from core client for 30 sec - exiting 05:33:18 (3892): No heartbeat from core client for 30 sec - exiting 05:33:19 (3892): No heartbeat from core client for 30 sec - exiting 05:33:21 (3892): No heartbeat from core client for 30 sec - exiting 05:33:22 (3892): No heartbeat from core client for 30 sec - exiting 05:33:23 (3892): No heartbeat from core client for 30 sec - exiting 05:33:24 (3892): No heartbeat from core client for 30 sec - exiting 05:33:25 (3892): No heartbeat from core client for 30 sec - exiting 05:33:27 (3892): No heartbeat from core client for 30 sec - exiting 05:33:28 (3892): No heartbeat from core client for 30 sec - exiting 05:33:29 (3892): No heartbeat from core client for 30 sec - exiting 05:33:30 (3892): No heartbeat from core client for 30 sec - exiting 05:33:32 (3892): No heartbeat from core client for 30 sec - exiting 05:34:09 (3892): No heartbeat from core client for 30 sec - exiting 05:34:10 (3892): No heartbeat from core client for 30 sec - exiting 05:34:12 (3892): No heartbeat from core client for 30 sec - exiting 05:34:13 (3892): No heartbeat from core client for 30 sec - exiting 05:34:14 (3892): No heartbeat from core client for 30 sec - exiting 05:34:15 (3892): No heartbeat from core client for 30 sec - exiting 05:34:16 (3892): No heartbeat from core client for 30 sec - exiting 05:34:18 (3892): No heartbeat from core client for 30 sec - exiting 05:34:19 (3892): No heartbeat from core client for 30 sec - exiting 05:34:20 (3892): No heartbeat from core client for 30 sec - exiting 05:34:21 (3892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:34:23 (3892): No heartbeat from core client for 30 sec - exiting 05:34:24 (3892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 14:35:28 (3476): No heartbeat from core client for 30 sec - exiting 14:35:30 (3476): No heartbeat from core client for 30 sec - exiting 14:35:31 (3476): No heartbeat from core client for 30 sec - exiting 14:35:32 (3476): No heartbeat from core client for 30 sec - exiting 14:35:33 (3476): No heartbeat from core client for 30 sec - exiting 14:35:34 (3476): No heartbeat from core client for 30 sec - exiting 14:35:36 (3476): No heartbeat from core client for 30 sec - exiting 14:35:37 (3476): No heartbeat from core client for 30 sec - exiting 14:35:38 (3476): No heartbeat from core client for 30 sec - exiting 14:35:39 (3476): No heartbeat from core client for 30 sec - exiting 14:35:41 (3476): No heartbeat from core client for 30 sec - exiting 14:35:42 (3476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: C06:24:55 (740): No heartbeat from core client for 30 sec - exiting 06:24:56 (740): No heartbeat from core client for 30 sec - exiting 06:24:57 (740): No heartbeat from core client for 30 sec - exiting 06:24:58 (740): No heartbeat from core client for 30 sec - exiting 06:25:00 (740): No heartbeat from core client for 30 sec - exiting 06:25:01 (740): No heartbeat from core client for 30 sec - exiting 06:25:02 (740): No heartbeat from core client for 30 sec - exiting 06:25:03 (740): No heartbeat from core client for 30 sec - exiting 06:25:04 (740): No heartbeat from core client for 30 sec - exiting 06:25:06 (740): No heartbeat from core client for 30 sec - exiting 06:25:07 (740): No heartbeat from core client for 30 sec - exiting 06:25:08 (740): No heartbeat from core client for 30 sec - exiting 06:25:09 (740): No heartbeat from core client for 30 sec - exiting 06:25:10 (740): No heartbeat from core client for 30 sec - exiting 06:25:12 (740): No heartbeat from core client for 30 sec - exiting 06:25:13 (740): No heartbeat from core client for 30 sec - exiting 06:25:14 (740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 12:48:04 (3888): No heartbeat from core client for 30 sec - exiting 12:48:06 (3888): No heartbeat from core client for 30 sec - exiting 12:48:07 (3888): No heartbeat from core client for 30 sec - exiting 12:48:08 (3888): No heartbeat from core client for 30 sec - exiting 12:48:09 (3888): No heartbeat from core client for 30 sec - exiting 12:48:11 (3888): No heartbeat from core client for 30 sec - exiting 12:48:12 (3888): No heartbeat from core client for 30 sec - exiting 12:48:13 (3888): No heartbeat from core client for 30 sec - exiting 12:48:14 (3888): No heartbeat from core client for 30 sec - exiting 12:48:15 (3888): No heartbeat from core client for 30 sec - exiting 12:48:17 (3888): No heartbeat from core client for 30 sec - exiting 12:48:18 (3888): No heartbeat from core client for 30 sec - exiting 12:48:19 (3888): No heartbeat from core client for 30 sec - exiting 12:48:20 (3888): No heartbeat from core client for 30 sec - exiting 12:48:22 (3888): No heartbeat from core client for 30 sec - exiting 12:48:23 (3888): No heartbeat from core client for 30 sec - exiting 12:48:24 (3888): No heartbeat from core client for 30 sec - exiting 12:48:25 (3888): No heartbeat from core client for 30 sec - exiting 12:48:26 (3888): No heartbeat from core client for 30 sec - exiting 12:48:28 (3888): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:12:40 (3696): No heartbeat from core client for 30 sec - exiting 12:12:41 (3696): No heartbeat from core client for 30 sec - exiting 12:12:42 (3696): No heartbeat from core client for 30 sec - exiting 12:12:43 (3696): No heartbeat from core client for 30 sec - exiting 12:12:45 (3696): No heartbeat from core client for 30 sec - exiting 12:12:46 (3696): No heartbeat from core client for 30 sec - exiting 12:12:47 (3696): No heartbeat from core client for 30 sec - exiting 12:12:48 (3696): No heartbeat from core client for 30 sec - exiting 12:12:49 (3696): No heartbeat from core client for 30 sec - exiting 12:12:51 (3696): No heartbeat from core client for 30 sec - exiting 12:12:52 (3696): No heartbeat from core client for 30 sec - exiting 12:12:53 (3696): No heartbeat from core client for 30 sec - exiting 12:12:54 (3696): No heartbeat from core client for 30 sec - exiting 12:12:56 (3696): No heartbeat from core client for 30 sec - exiting 12:12:57 (3696): No heartbeat from core client for 30 sec - exiting 12:12:58 (3696): No heartbeat from core client for 30 sec - exiting 12:12:59 (3696): No heartbeat from core client for 30 sec - exiting 12:13:00 (3696): No heartbeat from core client for 30 sec - exiting 12:13:02 (3696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 06:39:01 (3432): No heartbeat from core client for 30 sec - exiting 06:39:02 (3432): No heartbeat from core client for 30 sec - exiting 06:39:04 (3432): No heartbeat from core client for 30 sec - exiting 06:39:05 (3432): No heartbeat from core client for 30 sec - exiting 06:39:06 (3432): No heartbeat from core client for 30 sec - exiting 06:39:07 (3432): No heartbeat from core client for 30 sec - exiting 06:39:09 (3432): No heartbeat from core client for 30 sec - exiting 06:39:10 (3432): No heartbeat from core client for 30 sec - exiting 06:39:11 (3432): No heartbeat from core client for 30 sec - exiting 06:39:12 (3432): No heartbeat from core client for 30 sec - exiting 06:39:13 (3432): No heartbeat from core client for 30 sec - exiting 06:39:15 (3432): No heartbeat from core client for 30 sec - exiting 06:39:16 (3432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1064, selfPID=1064, iMonCtr=2 14:28:29 (3520): No heartbeat from core client for 30 sec - exiting 14:28:30 (3520): No heartbeat from core client for 30 sec - exiting 14:28:31 (3520): No heartbeat from core client for 30 sec - exiting 14:28:32 (3520): No heartbeat from core client for 30 sec - exiting 14:28:33 (3520): No heartbeat from core client for 30 sec - exiting 14:28:35 (3520): No heartbeat from core client for 30 sec - exiting 14:28:36 (3520): No heartbeat from core client for 30 sec - exiting 14:28:37 (3520): No heartbeat from core client for 30 sec - exiting 14:28:38 (3520): No heartbeat from core client for 30 sec - exiting 14:28:40 (3520): No heartbeat from core client for 30 sec - exiting 14:28:41 (3520): No heartbeat from core client for 30 sec - exiting 14:28:42 (3520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: READHIST: End of file in READ from history file for namelist NLCFILES tmp/xaakm.pipe_dummy 2048 Leaving CPDN_Main::Monitor... Regional yearly means requires 12 input files got 9 Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_pnw_8vqz_2000_1_007714103_0_10.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_8vqz_2000_1_007714103_0_11.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_pnw_8vqz_2000_1_007714103_0_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
11 Mar 2012 03:34:14	1025527	13983539	hadam3p_pnw_8vqz_2000_1_007714103_0	103,776	444,766	4.2858
09 Mar 2012 09:52:04	1025527	13983539	hadam3p_pnw_8vqz_2000_1_007714103_0	92,256	395,787	4.2901
07 Mar 2012 21:46:35	1025527	13983539	hadam3p_pnw_8vqz_2000_1_007714103_0	80,736	350,500	4.3413
05 Mar 2012 08:06:50	1025527	13983539	hadam3p_pnw_8vqz_2000_1_007714103_0	69,216	302,147	4.3653
03 Mar 2012 10:40:56	1025527	13983539	hadam3p_pnw_8vqz_2000_1_007714103_0	57,696	251,979	4.3674
25 Feb 2012 10:47:39	1025527	13983539	hadam3p_pnw_8vqz_2000_1_007714103_0	46,176	201,318	4.3598
22 Feb 2012 07:26:27	1025527	13983539	hadam3p_pnw_8vqz_2000_1_007714103_0	34,656	150,062	4.3300
21 Feb 2012 13:54:14	1025527	13983539	hadam3p_pnw_8vqz_2000_1_007714103_0	23,136	100,375	4.3385
20 Feb 2012 03:12:51	1025527	13983539	hadam3p_pnw_8vqz_2000_1_007714103_0	11,616	51,225	4.4099