Task 15677552

Name	hadcm3n_3jpi_1980_40_008334427_0
Workunit	8485288
Created	22 Mar 2013, 23:09:44 UTC
Sent	22 Mar 2013, 23:11:21 UTC
Report deadline	22 Jun 2013, 6:38:32 UTC
Received	25 Apr 2013, 10:49:49 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1097360
Run time	19 days 9 hours 36 min 21 sec
CPU time	17 days 17 hours 43 min 16 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	2.99 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1952, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/3jpiko.pjj8c10 Error converting file to netcdf: dataout/3jpiko.pij8c10 Error converting file to netcdf: dataout/3jpiko.pfj8c10 Error converting file to netcdf: dataout/3jpika.phj8c10 Error converting file to netcdf: dataout/3jpika.pgj8c10 Error converting file to netcdf: dataout/3jpika.pej8c10 Error converting file to netcdf: dataout/3jpika.pdj8c10 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=11884, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 10:01:42 (1248): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6444, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=12876, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4804, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4804, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4804, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4804, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4804, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4804, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4804, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4804, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4804, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4804, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4804, iMonCtr=1 Model crash detected, will try to restart... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/3jpiko.pjk5c10 Error converting file to netcdf: dataout/3jpiko.pik5c10 Error converting file to netcdf: dataout/3jpiko.pfk5c10 Error converting file to netcdf: dataout/3jpika.phk5c10 Error converting file to netcdf: dataout/3jpika.pgk5c10 Error converting file to netcdf: dataout/3jpika.pek5c10 Error converting file to netcdf: dataout/3jpika.pdk5c10 10:08:44 (5308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77723AB3 read attempt to address 0x40E1F62A Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_3jpi_1980_40_008334427/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
25 Apr 2013 10:52:50	1097360	15677552	hadcm3n_3jpi_1980_40_008334427_0	1,036,800	1,532,593	1.4782
24 Apr 2013 13:21:35	1097360	15677552	hadcm3n_3jpi_1980_40_008334427_0	1,010,880	1,493,374	1.4773
23 Apr 2013 18:11:56	1097360	15677552	hadcm3n_3jpi_1980_40_008334427_0	984,960	1,454,767	1.4770
21 Apr 2013 23:10:16	1097360	15677552	hadcm3n_3jpi_1980_40_008334427_0	959,040	1,416,573	1.4771
20 Apr 2013 20:11:37	1097360	15677552	hadcm3n_3jpi_1980_40_008334427_0	933,120	1,378,447	1.4772
20 Apr 2013 07:58:42	1097360	15677552	hadcm3n_3jpi_1980_40_008334427_0	907,200	1,340,163	1.4773
19 Apr 2013 09:08:49	1097360	15677552	hadcm3n_3jpi_1980_40_008334427_0	881,280	1,302,182	1.4776
17 Apr 2013 18:48:34	1097360	15677552	hadcm3n_3jpi_1980_40_008334427_0	855,360	1,264,207	1.4780
16 Apr 2013 09:09:49	1097360	15677552	hadcm3n_3jpi_1980_40_008334427_0	829,440	1,226,487	1.4787
15 Apr 2013 21:19:47	1097360	15677552	hadcm3n_3jpi_1980_40_008334427_0	803,520	1,188,368	1.4790
15 Apr 2013 05:09:14	1097360	15677552	hadcm3n_3jpi_1980_40_008334427_0	777,600	1,150,685	1.4798
14 Apr 2013 11:32:28	1097360	15677552	hadcm3n_3jpi_1980_40_008334427_0	751,680	1,112,899	1.4805
13 Apr 2013 15:10:47	1097360	15677552	hadcm3n_3jpi_1980_40_008334427_0	725,760	1,074,829	1.4810
12 Apr 2013 17:27:26	1097360	15677552	hadcm3n_3jpi_1980_40_008334427_0	699,840	1,036,886	1.4816
11 Apr 2013 20:28:49	1097360	15677552	hadcm3n_3jpi_1980_40_008334427_0	673,920	999,163	1.4826
11 Apr 2013 09:09:08	1097360	15677552	hadcm3n_3jpi_1980_40_008334427_0	648,000	960,398	1.4821
08 Apr 2013 15:35:37	1097360	15677552	hadcm3n_3jpi_1980_40_008334427_0	622,080	920,690	1.4800
07 Apr 2013 19:14:45	1097360	15677552	hadcm3n_3jpi_1980_40_008334427_0	596,160	881,248	1.4782
06 Apr 2013 23:29:43	1097360	15677552	hadcm3n_3jpi_1980_40_008334427_0	570,240	841,852	1.4763
06 Apr 2013 11:39:41	1097360	15677552	hadcm3n_3jpi_1980_40_008334427_0	544,320	802,765	1.4748