Task 15542648

Name	hadcm3n_3mg6_1980_40_008283377_0
Workunit	8434512
Created	14 Jan 2013, 5:54:29 UTC
Sent	14 Jan 2013, 5:54:40 UTC
Report deadline	15 Apr 2013, 13:21:51 UTC
Received	3 Apr 2013, 13:17:48 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1145695
Run time	31 days 21 hours 40 min 41 sec
CPU time	30 days 6 hours 3 min 55 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	2.31 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4304, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5368, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6068, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4720, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4092, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4740, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5908, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1488, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6084, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1512, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4040, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4284, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1872, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5076, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5076, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4280, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4728, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4100, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4100, iMonCtr=1 Model crash detected, will try to restart... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/3mg6ko.pjj9c10 Error converting file to netcdf: dataout/3mg6ko.pij9c10 Error converting file to netcdf: dataout/3mg6ko.pfj9c10 Error converting file to netcdf: dataout/3mg6ka.phj9c10 Error converting file to netcdf: dataout/3mg6ka.pgj9c10 Error converting file to netcdf: dataout/3mg6ka.pej9c10 Error converting file to netcdf: dataout/3mg6ka.pdj9c10 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5760, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5892, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4624, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4552, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1788, iMonCtr=1 Model crash detected, will try to restart... CController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2640, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6112, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6140, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4604, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5252, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5016, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=128, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=128, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7548, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4928, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4084, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4168, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3296, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4324, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3644, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5036, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4184, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4976, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5204, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5204, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6104, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
03 Apr 2013 13:22:22	1145695	15542648	hadcm3n_3mg6_1980_40_008283377_0	1,036,800	2,613,827	2.5211
01 Apr 2013 17:52:51	1145695	15542648	hadcm3n_3mg6_1980_40_008283377_0	1,010,880	2,545,487	2.5181
31 Mar 2013 11:24:56	1145695	15542648	hadcm3n_3mg6_1980_40_008283377_0	984,960	2,477,513	2.5153
30 Mar 2013 06:53:49	1145695	15542648	hadcm3n_3mg6_1980_40_008283377_0	959,040	2,409,644	2.5126
28 Mar 2013 07:28:15	1145695	15542648	hadcm3n_3mg6_1980_40_008283377_0	933,120	2,341,564	2.5094
26 Mar 2013 16:15:37	1145695	15542648	hadcm3n_3mg6_1980_40_008283377_0	907,200	2,272,812	2.5053
25 Mar 2013 11:17:52	1145695	15542648	hadcm3n_3mg6_1980_40_008283377_0	881,280	2,205,460	2.5026
23 Mar 2013 18:11:22	1145695	15542648	hadcm3n_3mg6_1980_40_008283377_0	855,360	2,139,292	2.5010
21 Mar 2013 14:35:25	1145695	15542648	hadcm3n_3mg6_1980_40_008283377_0	829,440	2,072,240	2.4984
20 Mar 2013 09:37:14	1145695	15542648	hadcm3n_3mg6_1980_40_008283377_0	803,520	2,004,843	2.4951
18 Mar 2013 13:50:09	1145695	15542648	hadcm3n_3mg6_1980_40_008283377_0	777,600	1,942,551	2.4981
15 Mar 2013 20:01:52	1145695	15542648	hadcm3n_3mg6_1980_40_008283377_0	751,680	1,879,061	2.4998
13 Mar 2013 21:35:58	1145695	15542648	hadcm3n_3mg6_1980_40_008283377_0	725,760	1,812,266	2.4971
12 Mar 2013 14:05:17	1145695	15542648	hadcm3n_3mg6_1980_40_008283377_0	699,840	1,745,332	2.4939
07 Mar 2013 18:21:55	1145695	15542648	hadcm3n_3mg6_1980_40_008283377_0	673,920	1,678,924	2.4913
06 Mar 2013 10:11:06	1145695	15542648	hadcm3n_3mg6_1980_40_008283377_0	648,000	1,611,438	2.4868
03 Mar 2013 14:10:20	1145695	15542648	hadcm3n_3mg6_1980_40_008283377_0	622,080	1,546,975	2.4868
28 Feb 2013 16:10:36	1145695	15542648	hadcm3n_3mg6_1980_40_008283377_0	596,160	1,482,460	2.4867
27 Feb 2013 09:16:55	1145695	15542648	hadcm3n_3mg6_1980_40_008283377_0	570,240	1,417,190	2.4853
25 Feb 2013 11:55:50	1145695	15542648	hadcm3n_3mg6_1980_40_008283377_0	544,320	1,351,155	2.4823