Task 15480446

Name	hadcm3n_z8tx_1880_40_008253503_2
Workunit	8408627
Created	16 Dec 2012, 11:58:07 UTC
Sent	16 Dec 2012, 11:58:16 UTC
Report deadline	17 Mar 2013, 19:25:27 UTC
Received	14 Feb 2013, 7:36:08 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1131474
Run time	23 days 20 hours 2 min 58 sec
CPU time	21 days 15 hours 54 min 55 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	1.64 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... 12:14:24 (4192): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4344, iMonCtr=1 Model crash detected, will try to restart... 21:41:26 (5432): No heartbeat from core client for 30 sec - exiting 21:41:27 (5432): No heartbeat from core client for 30 sec - exiting 21:41:28 (5432): No heartbeat from core client for 30 sec - exiting 21:41:29 (5432): No heartbeat from core client for 30 sec - exiting 21:41:30 (5432): No heartbeat from core client for 30 sec - exiting 21:41:31 (5432): No heartbeat from core client for 30 sec - exiting 21:41:32 (5432): No heartbeat from core client for 30 sec - exiting 21:41:33 (5432): No heartbeat from core client for 30 sec - exiting 21:41:34 (5432): No heartbeat from core client for 30 sec - exiting 21:41:35 (5432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2132, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5056, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:46:04 (4296): No heartbeat from core client for 30 sec - exiting 01:46:05 (4296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:28:18 (4556): No heartbeat from core client for 30 sec - exiting 19:28:19 (4556): No heartbeat from core client for 30 sec - exiting 19:28:20 (4556): No heartbeat from core client for 30 sec - exiting 19:28:21 (4556): No heartbeat from core client for 30 sec - exiting 19:28:22 (4556): No heartbeat from core client for 30 sec - exiting 19:28:23 (4556): No heartbeat from core client for 30 sec - exiting 19:28:24 (4556): No heartbeat from core client for 30 sec - exiting 19:28:25 (4556): No heartbeat from core client for 30 sec - exiting 19:28:26 (4556): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4660, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 20:52:40 (1828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 02:01:56 (4452): No heartbeat from core client for 30 sec - exiting 02:01:57 (4452): No heartbeat from core client for 30 sec - exiting 02:01:58 (4452): No heartbeat from core client for 30 sec - exiting 02:01:59 (4452): No heartbeat from core client for 30 sec - exiting 02:02:00 (4452): No heartbeat from core client for 30 sec - exiting 02:02:01 (4452): No heartbeat from core client for 30 sec - exiting 02:02:02 (4452): No heartbeat from core client for 30 sec - exiting 02:02:03 (4452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... parse_init_data_file: no end tag 21:22:28 (4568): Can't parse init data file - running in standalone mode Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:32:48 (4376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CSuspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:08:09 (7712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7668, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 06:59:58 (4548): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4648, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4976, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3136, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5056, iMonCtr=1 Model crash detected, will try to restart... 06:17:52 (5096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5028, iMonCtr=1 Model crash detected, will try to restart... 06:10:46 (5108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5320, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 09:22:39 (5852): No heartbeat from core client for 30 sec - exiting 09:22:41 (5852): No heartbeat from core client for 30 sec - exiting 09:22:42 (5852): No heartbeat from core client for 30 sec - exiting 09:22:43 (5852): No heartbeat from core client for 30 sec - exiting 09:22:44 (5852): No heartbeat from core client for 30 sec - exiting 09:22:45 (5852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:22:46 (5852): No heartbeat from core client for 30 sec - exiting 09:22:47 (5852): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2840, iMonCtr=1 Model crash detected, will try to restart... 19:00:33 (2188): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
14 Feb 2013 06:40:16	1131474	15480446	hadcm3n_z8tx_1880_40_008253503_2	518,400	1,871,674	3.6105
12 Feb 2013 06:24:06	1131474	15480446	hadcm3n_z8tx_1880_40_008253503_2	492,480	1,773,804	3.6018
09 Feb 2013 17:56:35	1131474	15480446	hadcm3n_z8tx_1880_40_008253503_2	466,560	1,674,020	3.5880
07 Feb 2013 06:32:08	1131474	15480446	hadcm3n_z8tx_1880_40_008253503_2	440,640	1,577,056	3.5790
04 Feb 2013 23:44:54	1131474	15480446	hadcm3n_z8tx_1880_40_008253503_2	414,720	1,480,930	3.5709
02 Feb 2013 22:40:05	1131474	15480446	hadcm3n_z8tx_1880_40_008253503_2	388,800	1,382,416	3.5556
31 Jan 2013 23:39:48	1131474	15480446	hadcm3n_z8tx_1880_40_008253503_2	362,880	1,284,450	3.5396
28 Jan 2013 23:17:52	1131474	15480446	hadcm3n_z8tx_1880_40_008253503_2	336,960	1,193,410	3.5417
26 Jan 2013 22:38:46	1131474	15480446	hadcm3n_z8tx_1880_40_008253503_2	311,040	1,091,576	3.5094
24 Jan 2013 06:05:40	1131474	15480446	hadcm3n_z8tx_1880_40_008253503_2	285,120	991,859	3.4787
21 Jan 2013 17:31:17	1131474	15480446	hadcm3n_z8tx_1880_40_008253503_2	259,200	897,817	3.4638
13 Jan 2013 16:05:34	1131474	15480446	hadcm3n_z8tx_1880_40_008253503_2	233,280	804,352	3.4480
09 Jan 2013 20:40:23	1131474	15480446	hadcm3n_z8tx_1880_40_008253503_2	207,360	712,518	3.4361
06 Jan 2013 14:27:22	1131474	15480446	hadcm3n_z8tx_1880_40_008253503_2	181,440	617,807	3.4050
03 Jan 2013 13:35:24	1131474	15480446	hadcm3n_z8tx_1880_40_008253503_2	155,520	528,705	3.3996
31 Dec 2012 09:13:32	1131474	15480446	hadcm3n_z8tx_1880_40_008253503_2	129,600	441,838	3.4092
28 Dec 2012 22:02:30	1131474	15480446	hadcm3n_z8tx_1880_40_008253503_2	103,680	351,849	3.3936
25 Dec 2012 19:10:54	1131474	15480446	hadcm3n_z8tx_1880_40_008253503_2	77,760	261,586	3.3640
23 Dec 2012 18:42:51	1131474	15480446	hadcm3n_z8tx_1880_40_008253503_2	51,840	172,910	3.3355
19 Dec 2012 23:39:18	1131474	15480446	hadcm3n_z8tx_1880_40_008253503_2	25,920	85,896	3.3139