Task 15762866

Name	hadcm3n_3hyk_1980_40_008320658_2
Workunit	8471793
Created	5 May 2013, 18:29:54 UTC
Sent	5 May 2013, 18:30:00 UTC
Report deadline	5 Aug 2013, 1:57:11 UTC
Received	23 Aug 2013, 14:19:26 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1280163
Run time	9 days 12 hours 8 min 20 sec
CPU time	7 days 5 hours 29 min 12 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.00 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5028, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5452, iMonCtr=1 Model crash detected, will try to restart... 21:42:36 (5604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5344, iMonCtr=1 Model crash detected, will try to restart... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/3hykko.pji3c10 Error converting file to netcdf: dataout/3hykko.pii3c10 Error converting file to netcdf: dataout/3hykko.pfi3c10 Error converting file to netcdf: dataout/3hykka.phi3c10 Error converting file to netcdf: dataout/3hykka.pgi3c10 Error converting file to netcdf: dataout/3hykka.pei3c10 Error converting file to netcdf: dataout/3hykka.pdi3c10 19:02:31 (4612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5528, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5816, iMonCtr=1 Model crash detected, will try to restart... CCPDN Monitor - Quit request from BOINC... 14:03:36 (5420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:06:13 (5336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CCController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5368, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/3hykko.pji6c10 Error converting file to netcdf: dataout/3hykko.pii6c10 Error converting file to netcdf: dataout/3hykko.pfi6c10 Error converting file to netcdf: dataout/3hykka.phi6c10 Error converting file to netcdf: dataout/3hykka.pgi6c10 Error converting file to netcdf: dataout/3hykka.pei6c10 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5728, iMonCtr=1 Model crash detected, will try to restart... CCPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5960, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5496, iMonCtr=1 Model crash detected, will try to restart... 16:13:55 (4588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:13:50 (5992): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5200, iMonCtr=1 Model crash detected, will try to restart... CCController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4560, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4892, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5924, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CBUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
23 Aug 2013 14:22:39	1280163	15762866	hadcm3n_3hyk_1980_40_008320658_2	259,200	624,548	2.4095
23 Aug 2013 14:22:39	1280163	15762866	hadcm3n_3hyk_1980_40_008320658_2	233,280	561,870	2.4086
25 Jul 2013 18:38:56	1280163	15762866	hadcm3n_3hyk_1980_40_008320658_2	207,360	497,634	2.3999
23 Jul 2013 19:52:32	1280163	15762866	hadcm3n_3hyk_1980_40_008320658_2	181,440	432,363	2.3830
23 Jul 2013 19:52:32	1280163	15762866	hadcm3n_3hyk_1980_40_008320658_2	155,520	367,221	2.3612
22 Jun 2013 11:17:24	1280163	15762866	hadcm3n_3hyk_1980_40_008320658_2	129,600	303,164	2.3392
13 Jun 2013 13:45:47	1280163	15762866	hadcm3n_3hyk_1980_40_008320658_2	103,680	238,657	2.3019
01 Jun 2013 14:36:42	1280163	15762866	hadcm3n_3hyk_1980_40_008320658_2	77,760	175,676	2.2592
24 May 2013 11:08:38	1280163	15762866	hadcm3n_3hyk_1980_40_008320658_2	51,840	117,006	2.2571
14 May 2013 19:18:04	1280163	15762866	hadcm3n_3hyk_1980_40_008320658_2	25,920	58,039	2.2392