Task 15923257

Name	hadam3p_saf_1hrc_1983_1_006960448_2
Workunit	7163764
Created	15 Aug 2013, 23:41:38 UTC
Sent	15 Aug 2013, 23:41:52 UTC
Report deadline	29 Jul 2014, 5:01:52 UTC
Received	13 Sep 2013, 7:58:56 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-2 (0xFFFFFFFE) Unknown error code
Computer ID	1131315
Run time	3 days 3 hours 24 min 49 sec
CPU time	3 days 0 hours 22 min 32 sec
Validate state	Invalid
Credit	1,870.33
Device peak FLOPS	3.02 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Southern Africa v6.09 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> - exit code -2 (0xfffffffe) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11512, selfPID=11512, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11444, selfPID=11444, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=15780, selfPID=15780, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10644, selfPID=10644, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=15356, selfPID=15356, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=18100, selfPID=18100, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:56:50 (12136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:01:51 (13276): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=16024, selfPID=16024, iMonCtr=2 10:08:06 (10248): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=25644, selfPID=25644, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=165832, selfPID=165832, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=165784, selfPID=165784, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=166056, selfPID=166056, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=156528, selfPID=156528, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=141508, selfPID=141508, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=168788, selfPID=168788, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=164760, selfPID=164760, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=169216, selfPID=169216, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=166256, selfPID=166256, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=170100, selfPID=170100, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=170484, selfPID=170484, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=170496, selfPID=170496, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=14996, selfPID=14996, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=130532, selfPID=130532, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=171744, selfPID=171744, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=168520, selfPID=168520, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... No PCPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=166252, selfPID=166252, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=178600, selfPID=178600, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=180660, selfPID=180660, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=187708, selfPID=187708, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=185876, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=417916, selfPID=419140, iMonCtr=1 Model crash detected, will try to restart... 03:38:26 (7820): No heartbeat from core client for 30 sec - exiting 03:38:27 (7820): No heartbeat from core client for 30 sec - exiting 03:38:28 (7820): No heartbeat from core client for 30 sec - exiting 03:38:29 (7820): No heartbeat from core client for 30 sec - exiting 03:38:30 (7820): No heartbeat from core client for 30 sec - exiting 03:38:31 (7820): No heartbeat from core client for 30 sec - exiting 03:38:32 (7820): No heartbeat from core client for 30 sec - exiting 03:38:33 (7820): No heartbeat from core client for 30 sec - exiting 03:38:34 (7820): No heartbeat from core client for 30 sec - exiting 03:38:35 (7820): No heartbeat from core client for 30 sec - exiting 03:38:36 (7820): No heartbeat from core client for 30 sec - exiting 03:38:37 (7820): No heartbeat from core client for 30 sec - exiting 03:38:38 (7820): No heartbeat from core client for 30 sec - exiting 03:38:39 (7820): No heartbeat from core client for 30 sec - exiting 03:38:40 (7820): No heartbeat from core client for 30 sec - exiting 03:38:41 (7820): No heartbeat from core client for 30 sec - exiting 03:38:42 (7820): No heartbeat from core client for 30 sec - exiting 03:38:43 (7820): No heartbeat from core client for 30 sec - exiting 03:38:44 (7820): No heartbeat from core client for 30 sec - exiting 03:38:45 (7820): No heartbeat from core client for 30 sec - exiting 03:38:46 (7820): No heartbeat from core client for 30 sec - exiting 03:38:47 (7820): No heartbeat from core client for 30 sec - exiting 03:38:48 (7820): No heartbeat from core client for 30 sec - exiting 03:38:49 (7820): No heartbeat from core client for 30 sec - exiting 03:38:50 (7820): No heartbeat from core client for 30 sec - exiting 03:38:51 (7820): No heartbeat from core client for 30 sec - exiting 03:38:52 (7820): No heartbeat from core client for 30 sec - exiting 03:38:53 (7820): No heartbeat from core client for 30 sec - exiting 03:38:54 (7820): No heartbeat from core client for 30 sec - exiting 03:38:55 (7820): No heartbeat from core client for 30 sec - exiting 03:38:56 (7820): No heartbeat from core client for 30 sec - exiting 03:38:57 (7820): No heartbeat from core client for 30 sec - exiting 03:38:58 (7820): No heartbeat from core client for 30 sec - exiting 03:38:59 (7820): No heartbeat from core client for 30 sec - exiting 03:39:00 (7820): No heartbeat from core client for 30 sec - exiting 03:39:01 (7820): No heartbeat from core client for 30 sec - exiting 03:39:02 (7820): No heartbeat from core client for 30 sec - exiting 03:39:03 (7820): No heartbeat from core client for 30 sec - exiting Could not launch model process. Last Error=5 03:39:04 (7820): No heartbeat from core client for 30 sec - exiting 03:39:05 (7820): No heartbeat from core client for 30 sec - exiting 03:39:06 (7820): No heartbeat from core client for 30 sec - exiting 03:39:07 (7820): No heartbeat from core client for 30 sec - exiting Called boinc_finish 03:39:10 (7820): No heartbeat from core client for 30 sec - exiting 03:39:11 (7820): No heartbeat from core client for 30 sec - exiting </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
13 Sep 2013 03:59:55	1131315	15923257	hadam3p_saf_1hrc_1983_1_006960448_2	115,296	249,281	2.1621
12 Sep 2013 21:03:19	1131315	15923257	hadam3p_saf_1hrc_1983_1_006960448_2	103,776	224,806	2.1663
12 Sep 2013 11:45:18	1131315	15923257	hadam3p_saf_1hrc_1983_1_006960448_2	92,256	200,248	2.1706
12 Sep 2013 03:52:28	1131315	15923257	hadam3p_saf_1hrc_1983_1_006960448_2	80,736	175,287	2.1711
11 Sep 2013 20:24:30	1131315	15923257	hadam3p_saf_1hrc_1983_1_006960448_2	69,216	150,567	2.1753
11 Sep 2013 06:00:27	1131315	15923257	hadam3p_saf_1hrc_1983_1_006960448_2	57,696	125,768	2.1798
10 Sep 2013 06:58:44	1131315	15923257	hadam3p_saf_1hrc_1983_1_006960448_2	46,176	100,560	2.1778
09 Sep 2013 15:22:03	1131315	15923257	hadam3p_saf_1hrc_1983_1_006960448_2	34,656	75,315	2.1732
09 Sep 2013 03:01:47	1131315	15923257	hadam3p_saf_1hrc_1983_1_006960448_2	23,136	50,404	2.1786
08 Sep 2013 16:19:13	1131315	15923257	hadam3p_saf_1hrc_1983_1_006960448_2	11,616	25,316	2.1794