Task 15279713

Name	hadcm3n_zc74_1880_40_008200516_0
Workunit	8355640
Created	13 Sep 2012, 7:37:13 UTC
Sent	13 Sep 2012, 7:50:32 UTC
Report deadline	13 Dec 2012, 15:17:43 UTC
Received	30 Nov 2012, 9:36:32 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	940614
Run time	13 days 22 hours 18 min 53 sec
CPU time	13 days 22 hours 18 min 53 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	2.61 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.4.5</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... 12:48:49 (880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:06:51 (2824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:06:53 (2824): No heartbeat from core client for 30 sec - exiting 19:06:54 (2824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:15:53 (3200): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:15:54 (3200): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:24:54 (4060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:24:56 (4060): No heartbeat from core client for 30 sec - exiting 14:33:27 (724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:33:29 (724): No heartbeat from core client for 30 sec - exiting 14:33:30 (724): No heartbeat from core client for 30 sec - exiting 19:56:00 (4944): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:56:02 (4944): No heartbeat from core client for 30 sec - exiting 19:56:04 (4944): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 17:01:55 (2264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:07:48 (2580): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 16:34:40 (4836): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:00:59 (820): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:39:53 (5944): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5896, selfPID=5896, iMonCtr=1 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/zc74ko.pja0c10 Error converting file to netcdf: dataout/zc74ko.pia0c10 Error converting file to netcdf: dataout/zc74ko.pfa0c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:17:30 (3624): No heartbeat from core client for 30 sec - exiting 11:17:31 (3624): No heartbeat from core client for 30 sec - exiting 11:17:32 (3624): No heartbeat from core client for 30 sec - exiting 11:17:33 (3624): No heartbeat from core client for 30 sec - exiting 11:17:34 (3624): No heartbeat from core client for 30 sec - exiting 11:17:35 (3624): No heartbeat from core client for 30 sec - exiting 11:17:37 (3624): No heartbeat from core client for 30 sec - exiting 11:17:38 (3624): No heartbeat from core client for 30 sec - exiting 11:17:39 (3624): No heartbeat from core client for 30 sec - exiting 11:17:40 (3624): No heartbeat from core client for 30 sec - exiting 11:17:41 (3624): No heartbeat from core client for 30 sec - exiting 11:17:42 (3624): No heartbeat from core client for 30 sec - exiting 11:17:43 (3624): No heartbeat from core client for 30 sec - exiting 11:17:44 (3624): No heartbeat from core client for 30 sec - exiting 11:17:45 (3624): No heartbeat from core client for 30 sec - exiting 11:17:46 (3624): No heartbeat from core client for 30 sec - exiting 11:17:48 (3624): No heartbeat from core client for 30 sec - exiting 11:17:49 (3624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 16:50:08 (2252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:28:58 (5516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:26:42 (2620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:25:01 (4368): No heartbeat from core client for 30 sec - exiting 08:25:02 (4368): No heartbeat from core client for 30 sec - exiting 08:25:03 (4368): No heartbeat from core client for 30 sec - exiting 08:25:04 (4368): No heartbeat from core client for 30 sec - exiting 08:25:05 (4368): No heartbeat from core client for 30 sec - exiting 08:25:06 (4368): No heartbeat from core client for 30 sec - exiting 08:25:07 (4368): No heartbeat from core client for 30 sec - exiting 08:25:09 (4368): No heartbeat from core client for 30 sec - exiting 08:25:10 (4368): No heartbeat from core client for 30 sec - exiting 08:25:11 (4368): No heartbeat from core client for 30 sec - exiting 08:25:12 (4368): No heartbeat from core client for 30 sec - exiting 08:25:13 (4368): No heartbeat from core client for 30 sec - exiting 08:25:14 (4368): No heartbeat from core client for 30 sec - exiting 08:25:15 (4368): No heartbeat from core client for 30 sec - exiting 08:25:16 (4368): No heartbeat from core client for 30 sec - exiting 08:25:17 (4368): No heartbeat from core client for 30 sec - exiting 08:25:18 (4368): No heartbeat from core client for 30 sec - exiting 08:25:20 (4368): No heartbeat from core client for 30 sec - exiting 08:25:21 (4368): No heartbeat from core client for 30 sec - exiting 08:25:22 (4368): No heartbeat from core client for 30 sec - exiting 08:25:23 (4368): No heartbeat from core client for 30 sec - exiting 08:25:24 (4368): No heartbeat from core client for 30 sec - exiting 08:25:25 (4368): No heartbeat from core client for 30 sec - exiting 08:25:26 (4368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 09:47:10 (4260): No heartbeat from core client for 30 sec - exiting 09:47:11 (4260): No heartbeat from core client for 30 sec - exiting 09:47:12 (4260): No heartbeat from core client for 30 sec - exiting 09:47:13 (4260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=492, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=492, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
29 Nov 2012 09:37:45	940614	15279713	hadcm3n_zc74_1880_40_008200516_0	777,600	1,203,495	1.5477
28 Nov 2012 09:36:58	940614	15279713	hadcm3n_zc74_1880_40_008200516_0	751,680	1,163,862	1.5483
26 Nov 2012 16:02:29	940614	15279713	hadcm3n_zc74_1880_40_008200516_0	725,760	1,123,887	1.5486
23 Nov 2012 10:05:49	940614	15279713	hadcm3n_zc74_1880_40_008200516_0	699,840	1,093,217	1.5621
19 Nov 2012 16:19:43	940614	15279713	hadcm3n_zc74_1880_40_008200516_0	673,920	1,052,105	1.5612
16 Nov 2012 15:36:40	940614	15279713	hadcm3n_zc74_1880_40_008200516_0	648,000	1,012,237	1.5621
15 Nov 2012 05:54:01	940614	15279713	hadcm3n_zc74_1880_40_008200516_0	622,080	971,450	1.5616
14 Nov 2012 11:29:19	940614	15279713	hadcm3n_zc74_1880_40_008200516_0	596,160	929,714	1.5595
12 Nov 2012 13:48:07	940614	15279713	hadcm3n_zc74_1880_40_008200516_0	570,240	889,624	1.5601
09 Nov 2012 08:45:27	940614	15279713	hadcm3n_zc74_1880_40_008200516_0	544,320	849,914	1.5614
08 Nov 2012 07:39:25	940614	15279713	hadcm3n_zc74_1880_40_008200516_0	518,400	810,462	1.5634
06 Nov 2012 15:41:36	940614	15279713	hadcm3n_zc74_1880_40_008200516_0	492,480	767,803	1.5591
06 Nov 2012 01:34:43	940614	15279713	hadcm3n_zc74_1880_40_008200516_0	466,560	726,839	1.5579
05 Nov 2012 11:58:37	940614	15279713	hadcm3n_zc74_1880_40_008200516_0	440,640	685,757	1.5563
31 Oct 2012 12:44:28	940614	15279713	hadcm3n_zc74_1880_40_008200516_0	414,720	645,391	1.5562
30 Oct 2012 22:52:37	940614	15279713	hadcm3n_zc74_1880_40_008200516_0	388,800	605,536	1.5574
29 Oct 2012 15:52:32	940614	15279713	hadcm3n_zc74_1880_40_008200516_0	362,880	564,879	1.5567
25 Oct 2012 07:29:41	940614	15279713	hadcm3n_zc74_1880_40_008200516_0	336,960	523,424	1.5534
24 Oct 2012 14:17:27	940614	15279713	hadcm3n_zc74_1880_40_008200516_0	311,040	482,952	1.5527
22 Oct 2012 17:31:22	940614	15279713	hadcm3n_zc74_1880_40_008200516_0	285,120	442,592	1.5523