Task 16099627

Name	hadcm3n_729m_1980_40_008414221_1
Workunit	8565077
Created	29 Nov 2013, 1:16:14 UTC
Sent	29 Nov 2013, 1:16:40 UTC
Report deadline	28 Feb 2014, 8:43:51 UTC
Received	1 Feb 2014, 14:16:26 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1263764
Run time	46 days 3 hours 34 min 40 sec
CPU time	42 days 18 hours 14 min 32 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	1.33 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> (unknown error) - exit code 193 (0xc1) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:39:36 (4920): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:39:37 (4920): No heartbeat from core client for 30 sec - exiting 01:49:34 (3600): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:35:35 (4352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:35:36 (4352): No heartbeat from core client for 30 sec - exiting 10:08:00 (3700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:08:01 (3700): No heartbeat from core client for 30 sec - exiting 10:08:46 (4392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:43:43 (4328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:05:58 (816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:47:55 (2948): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:55:30 (4876): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:42:32 (4060): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 23:42:33 (4060): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 18:01:52 (3880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:41:27 (3940): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:28:52 (876): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:13:38 (4444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:05:43 (4996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:05:44 (4996): No heartbeat from core client for 30 sec - exiting 18:12:16 (4428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:12:18 (4428): No heartbeat from core client for 30 sec - exiting 18:12:19 (4428): No heartbeat from core client for 30 sec - exiting 18:12:20 (4428): No heartbeat from core client for 30 sec - exiting 19:02:33 (3732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5452, iMonCtr=1 Model crash detected, will try to restart... 09:50:19 (4820): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1864, iMonCtr=1 Model crash detected, will try to restart... 10:14:04 (2588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:12:59 (5116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:01:54 (4236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:01:55 (4236): No heartbeat from core client for 30 sec - exiting 03:54:37 (4360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2320, iMonCtr=1 Model crash detected, will try to restart... 12:34:39 (4668): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4244, iMonCtr=1 Model crash detected, will try to restart... 11:49:49 (4588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:46:27 (2752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:46:29 (2752): No heartbeat from core client for 30 sec - exiting 19:51:25 (4884): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:51:26 (4884): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 02:04:52 (3252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:46:18 (6584): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 10:46:20 (6584): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 04:54:21 (6760): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 04:54:23 (6760): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77593AC3 read attempt to address 0x40B327CD Engaging BOINC Windows Runtime Debugger... Cannot serialize file D:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_729m_1980_40_008414221/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
01 Feb 2014 06:16:20	1263764	16099627	hadcm3n_729m_1980_40_008414221_1	777,600	3,694,470	4.7511
31 Jan 2014 12:20:03	1263764	16099627	hadcm3n_729m_1980_40_008414221_1	751,680	3,572,656	4.7529
27 Jan 2014 14:38:54	1263764	16099627	hadcm3n_729m_1980_40_008414221_1	725,760	3,448,312	4.7513
24 Jan 2014 19:59:31	1263764	16099627	hadcm3n_729m_1980_40_008414221_1	699,840	3,321,573	4.7462
22 Jan 2014 05:27:48	1263764	16099627	hadcm3n_729m_1980_40_008414221_1	673,920	3,195,083	4.7410
20 Jan 2014 15:43:59	1263764	16099627	hadcm3n_729m_1980_40_008414221_1	648,000	3,068,246	4.7349
19 Jan 2014 00:53:28	1263764	16099627	hadcm3n_729m_1980_40_008414221_1	622,080	2,940,511	4.7269
17 Jan 2014 10:17:31	1263764	16099627	hadcm3n_729m_1980_40_008414221_1	596,160	2,812,139	4.7171
15 Jan 2014 18:34:44	1263764	16099627	hadcm3n_729m_1980_40_008414221_1	570,240	2,684,319	4.7073
14 Jan 2014 06:12:32	1263764	16099627	hadcm3n_729m_1980_40_008414221_1	544,320	2,562,176	4.7071
12 Jan 2014 14:45:00	1263764	16099627	hadcm3n_729m_1980_40_008414221_1	518,400	2,439,148	4.7051
10 Jan 2014 22:17:44	1263764	16099627	hadcm3n_729m_1980_40_008414221_1	492,480	2,315,276	4.7013
09 Jan 2014 05:41:19	1263764	16099627	hadcm3n_729m_1980_40_008414221_1	466,560	2,193,914	4.7023
07 Jan 2014 19:16:30	1263764	16099627	hadcm3n_729m_1980_40_008414221_1	440,640	2,078,521	4.7171
06 Jan 2014 07:43:24	1263764	16099627	hadcm3n_729m_1980_40_008414221_1	414,720	1,959,438	4.7247
04 Jan 2014 20:26:57	1263764	16099627	hadcm3n_729m_1980_40_008414221_1	388,800	1,840,329	4.7334
03 Jan 2014 05:39:57	1263764	16099627	hadcm3n_729m_1980_40_008414221_1	362,880	1,711,388	4.7161
18 Dec 2013 20:06:28	1263764	16099627	hadcm3n_729m_1980_40_008414221_1	336,960	1,583,387	4.6990
17 Dec 2013 07:00:48	1263764	16099627	hadcm3n_729m_1980_40_008414221_1	311,040	1,456,957	4.6841
15 Dec 2013 17:28:53	1263764	16099627	hadcm3n_729m_1980_40_008414221_1	285,120	1,329,535	4.6631