Task 15453244

Name	hadcm3n_z7q8_1880_40_008251080_0
Workunit	8406204
Created	22 Nov 2012, 9:56:19 UTC
Sent	22 Nov 2012, 9:57:24 UTC
Report deadline	21 Feb 2013, 17:24:35 UTC
Received	4 Jan 2013, 4:17:38 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1210970
Run time	11 days 11 hours 36 min 57 sec
CPU time	9 days 19 hours 31 min 50 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.01 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5052, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4228, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 14:14:20 (4412): No heartbeat from core client for 30 sec - exiting 14:14:21 (4412): No heartbeat from core client for 30 sec - exiting 14:14:22 (4412): No heartbeat from core client for 30 sec - exiting 14:14:23 (4412): No heartbeat from core client for 30 sec - exiting 14:14:24 (4412): No heartbeat from core client for 30 sec - exiting 14:14:25 (4412): No heartbeat from core client for 30 sec - exiting 14:14:26 (4412): No heartbeat from core client for 30 sec - exiting 14:14:27 (4412): No heartbeat from core client for 30 sec - exiting 14:14:28 (4412): No heartbeat from core client for 30 sec - exiting 14:14:30 (4412): No heartbeat from core client for 30 sec - exiting 14:14:31 (4412): No heartbeat from core client for 30 sec - exiting 14:14:32 (4412): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:06:43 (4544): No heartbeat from core client for 30 sec - exiting 09:06:44 (4544): No heartbeat from core client for 30 sec - exiting 09:06:45 (4544): No heartbeat from core client for 30 sec - exiting 09:06:46 (4544): No heartbeat from core client for 30 sec - exiting 09:06:47 (4544): No heartbeat from core client for 30 sec - exiting 09:06:48 (4544): No heartbeat from core client for 30 sec - exiting 09:06:49 (4544): No heartbeat from core client for 30 sec - exiting 09:06:51 (4544): No heartbeat from core client for 30 sec - exiting 09:06:52 (4544): No heartbeat from core client for 30 sec - exiting 09:06:53 (4544): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:18:17 (4368): No heartbeat from core client for 30 sec - exiting 18:18:20 (4368): No heartbeat from core client for 30 sec - exiting 18:18:21 (4368): No heartbeat from core client for 30 sec - exiting 18:18:22 (4368): No heartbeat from core client for 30 sec - exiting 18:18:24 (4368): No heartbeat from core client for 30 sec - exiting 18:18:25 (4368): No heartbeat from core client for 30 sec - exiting 18:18:26 (4368): No heartbeat from core client for 30 sec - exiting 18:18:27 (4368): No heartbeat from core client for 30 sec - exiting 18:18:28 (4368): No heartbeat from core client for 30 sec - exiting 18:18:29 (4368): No heartbeat from core client for 30 sec - exiting 18:18:30 (4368): No heartbeat from core client for 30 sec - exiting 18:18:31 (4368): No heartbeat from core client for 30 sec - exiting 18:18:32 (4368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:12:57 (1620): No heartbeat from core client for 30 sec - exiting 05:12:58 (1620): No heartbeat from core client for 30 sec - exiting 05:12:59 (1620): No heartbeat from core client for 30 sec - exiting 05:13:00 (1620): No heartbeat from core client for 30 sec - exiting 05:13:01 (1620): No heartbeat from core client for 30 sec - exiting 05:13:02 (1620): No heartbeat from core client for 30 sec - exiting 05:13:03 (1620): No heartbeat from core client for 30 sec - exiting 05:13:04 (1620): No heartbeat from core client for 30 sec - exiting 05:13:06 (1620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:39:54 (4728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:51:50 (4936): No heartbeat from core client for 30 sec - exiting 05:51:51 (4936): No heartbeat from core client for 30 sec - exiting 05:51:53 (4936): No heartbeat from core client for 30 sec - exiting 05:51:54 (4936): No heartbeat from core client for 30 sec - exiting 05:51:55 (4936): No heartbeat from core client for 30 sec - exiting 05:51:56 (4936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:51:57 (4936): No heartbeat from core client for 30 sec - exiting 05:51:58 (4936): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4660, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2856, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2856, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2712, iMonCtr=1 Model crash detected, will try to restart... CController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3424, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77C33AB3 read attempt to address 0x40BEEEB2 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77A23AB3 read attempt to address 0x40BEEEB2 Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_z7q8_1880_40_008251080/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
29 Dec 2012 18:28:19	1210970	15453244	hadcm3n_z7q8_1880_40_008251080_0	259,200	822,889	3.1747
28 Dec 2012 12:00:06	1210970	15453244	hadcm3n_z7q8_1880_40_008251080_0	233,280	730,017	3.1294
18 Dec 2012 10:11:54	1210970	15453244	hadcm3n_z7q8_1880_40_008251080_0	207,360	649,903	3.1342
14 Dec 2012 14:32:18	1210970	15453244	hadcm3n_z7q8_1880_40_008251080_0	181,440	571,903	3.1520
06 Dec 2012 17:15:48	1210970	15453244	hadcm3n_z7q8_1880_40_008251080_0	155,520	487,437	3.1342
04 Dec 2012 17:19:12	1210970	15453244	hadcm3n_z7q8_1880_40_008251080_0	129,600	405,127	3.1260
03 Dec 2012 12:20:06	1210970	15453244	hadcm3n_z7q8_1880_40_008251080_0	103,680	317,399	3.0613
01 Dec 2012 12:27:07	1210970	15453244	hadcm3n_z7q8_1880_40_008251080_0	77,760	232,237	2.9866
30 Nov 2012 13:58:54	1210970	15453244	hadcm3n_z7q8_1880_40_008251080_0	51,840	157,299	3.0343
29 Nov 2012 12:43:20	1210970	15453244	hadcm3n_z7q8_1880_40_008251080_0	25,920	82,374	3.1780