Task 15156238

Name	hadam3p_saf_1s6h_2000_1_007002753_1
Workunit	7206069
Created	20 Aug 2012, 10:52:38 UTC
Sent	20 Aug 2012, 10:53:22 UTC
Report deadline	2 Aug 2013, 16:13:22 UTC
Received	25 Aug 2013, 7:48:40 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1110046
Run time	9 days 13 hours 27 min 11 sec
CPU time	5 days 11 hours 30 min 23 sec
Validate state	Invalid
Credit	2,057.21
Device peak FLOPS	1.85 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Southern Africa v6.09 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <stderr_txt> CreateFile error 32 when trying set file time CreateFile error 32 when trying set file time Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6108, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6124, selfPID=4108, iMonCtr=1 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4840, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4872, selfPID=644, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5380, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4644, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3376, selfPID=3996, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5988, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=520, selfPID=5748, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=288, selfPID=1132, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3224, iMonCtr=2 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5132, iMonCtr=2 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5572, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2868, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4188, selfPID=4188, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5484, selfPID=4872, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2988, selfPID=2988, iMonCtr=1 No Process Handle Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2988, selfPID=2688, iMonCtr=1 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2920, selfPID=5820, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3352, selfPID=3768, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5952, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5584, selfPID=3148, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4888, selfPID=4888, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is noCPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5408, selfPID=5408, iMonCtr=2 12:27:19 (1908): No heartbeat from core client for 30 sec - exiting 12:27:22 (1908): No heartbeat from core client for 30 sec - exiting 12:27:23 (1908): No heartbeat from core client for 30 sec - exiting 12:27:24 (1908): No heartbeat from core client for 30 sec - exiting 12:27:25 (1908): No heartbeat from core client for 30 sec - exiting 12:27:26 (1908): No heartbeat from core client for 30 sec - exiting 12:27:27 (1908): No heartbeat from core client for 30 sec - exiting 12:27:28 (1908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4476, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2660, selfPID=1796, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2780, iMonCtr=2 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5380, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5428, selfPID=3616, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3552, selfPID=2952, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2408, selfPID=3172, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3396, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3004, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3752, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3452, selfPID=3512, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3628, selfPID=2916, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3400, selfPID=2736, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4952, selfPID=2948, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5332, selfPID=4676, iMonCtr=1 Model crash detected, will try to restart... GController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4360, iMonCtr=2 Model crash detected, will try to restart... lobal Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4196, iMonCtr=2 Leaving CPDN_Main::Monitor... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3372, selfPID=2760, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5868, selfPID=5868, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5944, selfPID=2268, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 08:13:38 (868): No heartbeat from core client for 30 sec - exiting 08:13:39 (868): No heartbeat from core client for 30 sec - exiting 08:13:40 (868): No heartbeat from core client for 30 sec - exiting 08:13:41 (868): No heartbeat from core client for 30 sec - exiting 08:13:42 (868): No heartbeat from core client for 30 sec - exiting 08:13:43 (868): No heartbeat from core client for 30 sec - exiting 08:13:44 (868): No heartbeat from core client for 30 sec - exiting 08:13:45 (868): No heartbeat from core client for 30 sec - exiting 08:13:46 (868): No heartbeat from core client for 30 sec - exiting 08:13:47 (868): No heartbeat from core client for 30 sec - exiting 08:13:48 (868): No heartbeat from core client for 30 sec - exiting 08:13:49 (868): No heartbeat from core client for 30 sec - exiting 08:13:51 (868): No heartbeat from core client for 30 sec - exiting 08:13:52 (868): No heartbeat from core client for 30 sec - exiting 08:13:53 (868): No heartbeat from core client for 30 sec - exiting 08:13:54 (868): No heartbeat from core client for 30 sec - exiting 08:13:55 (868): No heartbeat from core client for 30 sec - exiting 08:13:57 (868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4876, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=54624, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=40952, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4692, selfPID=4692, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=48500, selfPID=48500, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4500, selfPID=4500, iMonCtr=2 Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=35624, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=36940, selfPID=36932, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3892, iMonCtr=2 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3192, selfPID=3180, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=18872, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... RCPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7620, selfPID=6164, iMonCtr=1 Model crash detected, will try to restart... GController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5428, selfPID=4492, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=55108, selfPID=54528, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4312, selfPID=4312, iMonCtr=2 CGlobal Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3936, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3516, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3240, selfPID=3496, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4532, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... GController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=932, selfPID=2744, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... GController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1132, selfPID=3676, iMonCtr=1 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2828, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3216, selfPID=3736, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=61512, selfPID=59716, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=532, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1368, selfPID=3800, iMonCtr=1 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5584, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5588, selfPID=4752, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4860, selfPID=2776, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5600, selfPID=3680, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2832, selfPID=2832, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2832, selfPID=4436, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> <file_xfer_error> <file_name>hadam3p_saf_1s6h_2000_1_007002753_1_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
19 Aug 2013 11:15:12	1110046	15156238	hadam3p_saf_1s6h_2000_1_007002753_1	126,816	453,271	3.5742
14 Aug 2013 15:58:43	1110046	15156238	hadam3p_saf_1s6h_2000_1_007002753_1	115,296	415,442	3.6033
08 May 2013 15:55:49	1110046	15156238	hadam3p_saf_1s6h_2000_1_007002753_1	103,776	375,842	3.6217
14 Apr 2013 11:12:23	1110046	15156238	hadam3p_saf_1s6h_2000_1_007002753_1	92,276	337,793	3.6607
08 Apr 2013 10:02:35	1110046	15156238	hadam3p_saf_1s6h_2000_1_007002753_1	92,256	337,200	3.6550
06 Mar 2013 16:35:48	1110046	15156238	hadam3p_saf_1s6h_2000_1_007002753_1	80,741	300,092	3.7167
06 Mar 2013 13:50:13	1110046	15156238	hadam3p_saf_1s6h_2000_1_007002753_1	80,736	299,526	3.7099
24 Feb 2013 07:19:46	1110046	15156238	hadam3p_saf_1s6h_2000_1_007002753_1	69,216	263,279	3.8037
10 Feb 2013 06:31:26	1110046	15156238	hadam3p_saf_1s6h_2000_1_007002753_1	57,696	227,298	3.9396
27 Jan 2013 07:35:48	1110046	15156238	hadam3p_saf_1s6h_2000_1_007002753_1	46,190	191,818	4.1528
23 Jan 2013 16:50:48	1110046	15156238	hadam3p_saf_1s6h_2000_1_007002753_1	46,176	191,252	4.1418
23 Dec 2012 06:47:37	1110046	15156238	hadam3p_saf_1s6h_2000_1_007002753_1	34,656	136,698	3.9444
28 Oct 2012 09:59:01	1110046	15156238	hadam3p_saf_1s6h_2000_1_007002753_1	23,136	81,189	3.5092
02 Sep 2012 07:55:32	1110046	15156238	hadam3p_saf_1s6h_2000_1_007002753_1	11,616	40,889	3.5201