Task 15651162

Name	hadcm3n_4ko7_1940_40_008308344_3
Workunit	8459479
Created	6 Mar 2013, 17:06:59 UTC
Sent	6 Mar 2013, 17:07:06 UTC
Report deadline	6 Jun 2013, 0:34:17 UTC
Received	23 Jun 2013, 15:18:43 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-226 (0xFFFFFF1E) ERR_TOO_MANY_EXITS
Computer ID	1231443
Run time	10 days 3 hours 36 min 48 sec
CPU time	8 days 16 hours 22 min 45 sec
Validate state	Invalid
Credit	3,421.44
Device peak FLOPS	2.50 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> too many exit(0)s </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:37:35 (5740): No heartbeat from core client for 30 sec - exiting 18:37:38 (5740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:37:39 (5740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:01:18 (6272): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:39:41 (9828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:17:39 (8120): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:07:39 (8540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8096, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 20:50:00 (5004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:15:10 (1988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CCPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6872, iMonCtr=1 ModelController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5608, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5344, iMonCtr=1 Model crash detected, will try to restart... 18:16:35 (5308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:17:00 (5308): No heartbeat from core client for 30 sec - exiting 18:17:10 (5308): No heartbeat from core client for 30 sec - exiting 18:18:08 (5308): No heartbeat from core client for 30 sec - exiting 18:18:09 (5308): No heartbeat from core client for 30 sec - exiting 18:18:10 (5308): No heartbeat from core client for 30 sec - exiting 18:18:11 (5308): No heartbeat from core client for 30 sec - exiting 18:18:12 (5308): No heartbeat from core client for 30 sec - exiting 18:18:14 (5308): No heartbeat from core client for 30 sec - exiting 18:18:15 (5308): No heartbeat from core client for 30 sec - exiting 18:18:16 (5308): No heartbeat from core client for 30 sec - exiting 18:18:17 (5308): No heartbeat from core client for 30 sec - exiting 18:18:18 (5308): No heartbeat from core client for 30 sec - exiting 18:18:19 (5308): No heartbeat from core client for 30 sec - exiting 18:18:20 (5308): No heartbeat from core client for 30 sec - exiting 18:18:21 (5308): No heartbeat from core client for 30 sec - exiting 18:18:22 (5308): No heartbeat from core client for 30 sec - exiting 18:18:23 (5308): No heartbeat from core client for 30 sec - exiting 18:18:24 (5308): No heartbeat from core client for 30 sec - exiting 18:18:26 (5308): No heartbeat from core client for 30 sec - exiting 18:18:27 (5308): No heartbeat from core client for 30 sec - exiting 18:18:28 (5308): No heartbeat from core client for 30 sec - exiting 18:18:29 (5308): No heartbeat from core client for 30 sec - exiting 18:18:30 (5308): No heartbeat from core client for 30 sec - exiting 18:18:31 (5308): No heartbeat from core client for 30 sec - exiting 18:18:32 (5308): No heartbeat from core client for 30 sec - exiting 18:18:33 (5308): No heartbeat from core client for 30 sec - exiting 18:18:34 (5308): No heartbeat from core client for 30 sec - exiting 18:18:35 (5308): No heartbeat from core client for 30 sec - exiting 18:18:36 (5308): No heartbeat from core client for 30 sec - exiting 18:49:57 (4440): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:47:57 (4940): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:49:08 (7108): No heartbeat from core client for 30 sec - exiting 18:49:09 (7108): No heartbeat from core client for 30 sec - exiting 18:49:10 (7108): No heartbeat from core client for 30 sec - exiting 18:49:11 (7108): No heartbeat from core client for 30 sec - exiting 18:49:12 (7108): No heartbeat from core client for 30 sec - exiting 18:49:13 (7108): No heartbeat from core client for 30 sec - exiting 18:49:14 (7108): No heartbeat from core client for 30 sec - exiting 18:49:58 (7108): No heartbeat from core client for 30 sec - exiting 18:49:59 (7108): No heartbeat from core client for 30 sec - exiting 18:50:00 (7108): No heartbeat from core client for 30 sec - exiting 18:50:01 (7108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:01:32 (7560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:01:44 (7560): No heartbeat from core client for 30 sec - exiting 18:01:45 (7560): No heartbeat from core client for 30 sec - exiting 18:01:46 (7560): No heartbeat from core client for 30 sec - exiting 18:01:47 (7560): No heartbeat from core client for 30 sec - exiting 18:01:48 (7560): No heartbeat from core client for 30 sec - exiting 18:01:49 (7560): No heartbeat from core client for 30 sec - exiting 18:01:50 (7560): No heartbeat from core client for 30 sec - exiting 18:01:51 (7560): No heartbeat from core client for 30 sec - exiting 18:01:52 (7560): No heartbeat from core client for 30 sec - exiting 18:01:53 (7560): No heartbeat from core client for 30 sec - exiting 18:01:54 (7560): No heartbeat from core client for 30 sec - exiting 18:01:55 (7560): No heartbeat from core client for 30 sec - exiting 18:01:56 (7560): No heartbeat from core client for 30 sec - exiting 18:01:57 (7560): No heartbeat from core client for 30 sec - exiting 18:01:58 (7560): No heartbeat from core client for 30 sec - exiting 18:01:59 (7560): No heartbeat from core client for 30 sec - exiting 18:02:00 (7560): No heartbeat from core client for 30 sec - exiting 18:02:01 (7560): No heartbeat from core client for 30 sec - exiting 18:02:02 (7560): No heartbeat from core client for 30 sec - exiting 18:02:03 (7560): No heartbeat from core client for 30 sec - exiting 18:02:04 (7560): No heartbeat from core client for 30 sec - exiting 18:02:05 (7560): No heartbeat from core client for 30 sec - exiting 18:02:06 (7560): No heartbeat from core client for 30 sec - exiting 18:02:07 (7560): No heartbeat from core client for 30 sec - exiting 18:02:08 (7560): No heartbeat from core client for 30 sec - exiting 18:02:09 (7560): No heartbeat from core client for 30 sec - exiting 18:02:10 (7560): No heartbeat from core client for 30 sec - exiting 18:02:11 (7560): No heartbeat from core client for 30 sec - exiting 18:02:12 (7560): No heartbeat from core client for 30 sec - exiting 18:02:13 (7560): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9260, iMonCtr=1 Model crash detected, will try to restart... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
17 Jun 2013 20:25:47	1231443	15651162	hadcm3n_4ko7_1940_40_008308344_3	285,120	709,563	2.4886
15 Jun 2013 03:47:39	1231443	15651162	hadcm3n_4ko7_1940_40_008308344_3	259,200	657,299	2.5359
17 Apr 2013 00:49:12	1231443	15651162	hadcm3n_4ko7_1940_40_008308344_3	233,280	595,916	2.5545
12 Apr 2013 20:48:35	1231443	15651162	hadcm3n_4ko7_1940_40_008308344_3	207,360	536,496	2.5873
08 Apr 2013 17:39:34	1231443	15651162	hadcm3n_4ko7_1940_40_008308344_3	181,440	465,339	2.5647
02 Apr 2013 19:58:53	1231443	15651162	hadcm3n_4ko7_1940_40_008308344_3	155,520	397,346	2.5550
28 Mar 2013 19:19:51	1231443	15651162	hadcm3n_4ko7_1940_40_008308344_3	129,600	326,066	2.5159
26 Mar 2013 15:40:30	1231443	15651162	hadcm3n_4ko7_1940_40_008308344_3	103,680	259,419	2.5021
22 Mar 2013 13:35:07	1231443	15651162	hadcm3n_4ko7_1940_40_008308344_3	77,760	192,584	2.4766
17 Mar 2013 18:44:58	1231443	15651162	hadcm3n_4ko7_1940_40_008308344_3	51,840	120,619	2.3268
11 Mar 2013 17:46:24	1231443	15651162	hadcm3n_4ko7_1940_40_008308344_3	25,920	60,017	2.3155