Task 12834605

Name	hadcm3n_p7ix_1900_40_007227337_1
Workunit	7425577
Created	26 Apr 2011, 15:39:53 UTC
Sent	26 Apr 2011, 18:28:56 UTC
Report deadline	27 Jul 2011, 1:56:07 UTC
Received	15 Jun 2011, 17:59:02 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1106657
Run time	16 days 23 hours 9 min 2 sec
CPU time	16 days 18 hours 3 min 42 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	2.65 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1076, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:26:10 (4896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3452, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3852, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3628, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITHEAD: I/O error tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 07:15:30 (2564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x76FF6436 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x76FFA645 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77D3A645 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_p7ix_1900_40_007227337/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
14 Jun 2011 22:44:48	1106657	12834605	hadcm3n_p7ix_1900_40_007227337_1	777,600	1,447,504	1.8615
13 Jun 2011 18:14:31	1106657	12834605	hadcm3n_p7ix_1900_40_007227337_1	751,680	1,399,597	1.8620
11 Jun 2011 20:27:40	1106657	12834605	hadcm3n_p7ix_1900_40_007227337_1	725,760	1,350,835	1.8613
10 Jun 2011 02:34:12	1106657	12834605	hadcm3n_p7ix_1900_40_007227337_1	699,840	1,301,582	1.8598
08 Jun 2011 21:50:07	1106657	12834605	hadcm3n_p7ix_1900_40_007227337_1	673,920	1,252,077	1.8579
07 Jun 2011 18:38:25	1106657	12834605	hadcm3n_p7ix_1900_40_007227337_1	648,000	1,203,102	1.8566
06 Jun 2011 01:11:36	1106657	12834605	hadcm3n_p7ix_1900_40_007227337_1	622,080	1,153,254	1.8539
04 Jun 2011 03:14:23	1106657	12834605	hadcm3n_p7ix_1900_40_007227337_1	596,160	1,106,535	1.8561
02 Jun 2011 23:18:54	1106657	12834605	hadcm3n_p7ix_1900_40_007227337_1	570,240	1,059,621	1.8582
01 Jun 2011 18:34:16	1106657	12834605	hadcm3n_p7ix_1900_40_007227337_1	544,320	1,011,341	1.8580
29 May 2011 01:10:31	1106657	12834605	hadcm3n_p7ix_1900_40_007227337_1	518,400	962,777	1.8572
27 May 2011 22:11:08	1106657	12834605	hadcm3n_p7ix_1900_40_007227337_1	492,480	914,567	1.8571
26 May 2011 18:53:01	1106657	12834605	hadcm3n_p7ix_1900_40_007227337_1	466,560	866,526	1.8573
25 May 2011 01:39:21	1106657	12834605	hadcm3n_p7ix_1900_40_007227337_1	440,640	818,366	1.8572
23 May 2011 22:19:33	1106657	12834605	hadcm3n_p7ix_1900_40_007227337_1	414,720	770,295	1.8574
21 May 2011 23:15:52	1106657	12834605	hadcm3n_p7ix_1900_40_007227337_1	388,800	722,854	1.8592
20 May 2011 20:15:53	1106657	12834605	hadcm3n_p7ix_1900_40_007227337_1	362,880	677,965	1.8683
16 May 2011 05:15:13	1106657	12834605	hadcm3n_p7ix_1900_40_007227337_1	336,960	628,748	1.8659
14 May 2011 19:18:36	1106657	12834605	hadcm3n_p7ix_1900_40_007227337_1	311,040	579,806	1.8641
12 May 2011 19:47:32	1106657	12834605	hadcm3n_p7ix_1900_40_007227337_1	285,120	531,460	1.8640