Task 14775275

Name	hadam3p_eu_cwnb_2000_1_007961865_1
Workunit	8116979
Created	5 Jun 2012, 18:51:39 UTC
Sent	5 Jun 2012, 18:55:06 UTC
Report deadline	19 May 2013, 0:15:06 UTC
Received	23 Jun 2012, 18:02:07 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-226 (0xFFFFFF1E) ERR_TOO_MANY_EXITS
Computer ID	1219704
Run time	1 days 18 hours 44 min 58 sec
CPU time	1 days 15 hours 58 min 52 sec
Validate state	Invalid
Credit	399.63
Device peak FLOPS	1.69 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Europe v6.09 windows_intelx86
Stderr	<core_client_version>7.0.25</core_client_version> <![CDATA[ <message> too many exit(0)s </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5216, selfPID=6088, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4892, selfPID=4892, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4268, selfPID=5260, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5892, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6944, selfPID=4880, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8568, selfPID=10180, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8896, selfPID=6452, iMonCtr=1 Model crash detected, will try to restart... 19:34:07 (716): No heartbeat from core client for 30 sec - exiting 19:34:08 (716): No heartbeat from core client for 30 sec - exiting 19:34:09 (716): No heartbeat from core client for 30 sec - exiting 19:34:10 (716): No heartbeat from core client for 30 sec - exiting 19:34:11 (716): No heartbeat from core client for 30 sec - exiting 19:34:12 (716): No heartbeat from core client for 30 sec - exiting 19:34:13 (716): No heartbeat from core client for 30 sec - exiting 19:34:14 (716): No heartbeat from core client for 30 sec - exiting 19:34:16 (716): No heartbeat from core client for 30 sec - exiting 19:34:17 (716): No heartbeat from core client for 30 sec - exiting 19:34:18 (716): No heartbeat from core client for 30 sec - exiting 19:34:19 (716): No heartbeat from core client for 30 sec - exiting 19:34:20 (716): No heartbeat from core client for 30 sec - exiting 19:34:21 (716): No heartbeat from core client for 30 sec - exiting 19:34:22 (716): No heartbeat from core client for 30 sec - exiting 19:34:23 (716): No heartbeat from core client for 30 sec - exiting 19:34:24 (716): No heartbeat from core client for 30 sec - exiting 19:34:25 (716): No heartbeat from core client for 30 sec - exiting 19:34:26 (716): No heartbeat from core client for 30 sec - exiting 19:34:28 (716): No heartbeat from core client for 30 sec - exiting 19:34:29 (716): No heartbeat from core client for 30 sec - exiting 19:34:30 (716): No heartbeat from core client for 30 sec - exiting 19:34:31 (716): No heartbeat from core client for 30 sec - exiting 19:34:32 (716): No heartbeat from core client for 30 sec - exiting 19:34:33 (716): No heartbeat from core client for 30 sec - exiting 19:34:34 (716): No heartbeat from core client for 30 sec - exiting 19:34:35 (716): No heartbeat from core client for 30 sec - exiting 19:34:36 (716): No heartbeat from core client for 30 sec - exiting 19:34:37 (716): No heartbeat from core client for 30 sec - exiting 19:34:39 (716): No heartbeat from core client for 30 sec - exiting 19:34:40 (716): No heartbeat from core client for 30 sec - exiting 19:34:41 (716): No heartbeat from core client for 30 sec - exiting 19:34:42 (716): No heartbeat from core client for 30 sec - exiting 19:34:43 (716): No heartbeat from core client for 30 sec - exiting 19:34:44 (716): No heartbeat from core client for 30 sec - exiting 19:34:45 (716): No heartbeat from core client for 30 sec - exiting 19:34:46 (716): No heartbeat from core client for 30 sec - exiting 19:34:47 (716): No heartbeat from core client for 30 sec - exiting 19:34:48 (716): No heartbeat from core client for 30 sec - exiting 19:34:49 (716): No heartbeat from core client for 30 sec - exiting 19:34:51 (716): No heartbeat from core client for 30 sec - exiting 19:34:52 (716): No heartbeat from core client for 30 sec - exiting 19:34:53 (716): No heartbeat from core client for 30 sec - exiting 19:34:54 (716): No heartbeat from core client for 30 sec - exiting 19:34:55 (716): No heartbeat from core client for 30 sec - exiting 19:34:56 (716): No heartbeat from core client for 30 sec - exiting 19:34:57 (716): No heartbeat from core client for 30 sec - exiting 19:34:58 (716): No heartbeat from core client for 30 sec - exiting 19:34:59 (716): No heartbeat from core client for 30 sec - exiting 19:35:00 (716): No heartbeat from core client for 30 sec - exiting 19:35:01 (716): No heartbeat from core client for 30 sec - exiting 19:35:03 (716): No heartbeat from core client for 30 sec - exiting 19:35:04 (716): No heartbeat from core client for 30 sec - exiting 19:35:05 (716): No heartbeat from core client for 30 sec - exiting 19:35:06 (716): No heartbeat from core client for 30 sec - exiting 19:35:07 (716): No heartbeat from core client for 30 sec - exiting 19:35:08 (716): No heartbeat from core client for 30 sec - exiting 19:35:09 (716): No heartbeat from core client for 30 sec - exiting 19:35:10 (716): No heartbeat from core client for 30 sec - exiting 19:35:11 (716): No heartbeat from core client for 30 sec - exiting 19:35:12 (716): No heartbeat from core client for 30 sec - exiting 19:35:13 (716): No heartbeat from core client for 30 sec - exiting 19:35:15 (716): No heartbeat from core client for 30 sec - exiting 19:35:16 (716): No heartbeat from core client for 30 sec - exiting 19:35:17 (716): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3400, selfPID=4944, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8388, selfPID=8388, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1552, selfPID=7280, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7496, selfPID=7496, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8856, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1496, selfPID=1496, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3104, selfPID=2556, iMonCtr=1 Model crash detected, will try to restart... 21:17:13 (6344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7572, selfPID=7572, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7464, selfPID=856, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7768, selfPID=5428, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9684, selfPID=9992, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1192, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3288, selfPID=6376, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8228, selfPID=8968, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6132, selfPID=6636, iMonCtr=1 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8996, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9004, selfPID=2576, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1960, selfPID=5704, iMonCtr=1 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6832, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8116, selfPID=3468, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3304, selfPID=3304, iMonCtr=2 </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
19 Jun 2012 15:13:18	1219704	14775275	hadam3p_eu_cwnb_2000_1_007961865_1	23,166	104,501	4.5110
19 Jun 2012 14:12:21	1219704	14775275	hadam3p_eu_cwnb_2000_1_007961865_1	23,159	103,557	4.4716
19 Jun 2012 12:14:44	1219704	14775275	hadam3p_eu_cwnb_2000_1_007961865_1	23,153	102,849	4.4421
19 Jun 2012 12:14:44	1219704	14775275	hadam3p_eu_cwnb_2000_1_007961865_1	23,145	102,171	4.4144
19 Jun 2012 11:13:48	1219704	14775275	hadam3p_eu_cwnb_2000_1_007961865_1	23,138	101,509	4.3871
19 Jun 2012 11:13:48	1219704	14775275	hadam3p_eu_cwnb_2000_1_007961865_1	23,136	100,958	4.3637
11 Jun 2012 12:33:58	1219704	14775275	hadam3p_eu_cwnb_2000_1_007961865_1	11,616	52,359	4.5075