Task 15856762

Name	hadcm3n_zdyb_1960_40_008394587_0
Workunit	8545446
Created	22 Jun 2013, 4:26:10 UTC
Sent	22 Jun 2013, 6:29:00 UTC
Report deadline	21 Sep 2013, 13:56:11 UTC
Received	3 Sep 2013, 2:36:35 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	1 (0x00000001) Unknown error code
Computer ID	1138291
Run time	8 days 9 hours 53 min 16 sec
CPU time	5 days 13 hours 29 min 48 sec
Validate state	Invalid
Credit	2,799.36
Device peak FLOPS	2.52 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> Incorrect function. (0x1) - exit code 1 (0x1) </message> <stderr_txt> 15:50:08 (8076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:50:09 (8076): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3224, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6052, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/zdybko.pjg1c10 Error converting file to netcdf: dataout/zdybko.pig1c10 Error converting file to netcdf: dataout/zdybko.pfg1c10 Error converting file to netcdf: dataout/zdybka.phg1c10 Error converting file to netcdf: dataout/zdybka.pgg1c10 Error converting file to netcdf: dataout/zdybka.peg1c10 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3088, iMonCtr=1 Model crash detected, will try to restart... CController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5840, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5764, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2636, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3656, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3656, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3656, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4408, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5252, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4916, iMonCtr=1 Model crash detected, will try to restart... 06:06:45 (5716): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1268, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2768, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 18:51:19 (4960): No heartbeat from core client for 30 sec - exiting 18:51:20 (4960): No heartbeat from core client for 30 sec - exiting 18:51:21 (4960): No heartbeat from core client for 30 sec - exiting 18:51:22 (4960): No heartbeat from core client for 30 sec - exiting 18:51:24 (4960): No heartbeat from core client for 30 sec - exiting 18:51:25 (4960): No heartbeat from core client for 30 sec - exiting 18:51:26 (4960): No heartbeat from core client for 30 sec - exiting 18:51:27 (4960): No heartbeat from core client for 30 sec - exiting 18:51:28 (4960): No heartbeat from core client for 30 sec - exiting 18:51:30 (4960): No heartbeat from core client for 30 sec - exiting 18:51:31 (4960): No heartbeat from core client for 30 sec - exiting 18:51:32 (4960): No heartbeat from core client for 30 sec - exiting 18:51:33 (4960): No heartbeat from core client for 30 sec - exiting 18:51:35 (4960): No heartbeat from core client for 30 sec - exiting 18:51:36 (4960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:06:21 (1836): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=456, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 15:46:46 (5036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:46:47 (5036): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6904, iMonCtr=1 Model crash detected, will try to restart... 16:44:52 (5392): No heartbeat from core client for 30 sec - exiting 16:44:53 (5392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:46:12 (4064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 15:21:49 (4544): No heartbeat from core client for 30 sec - exiting 15:21:50 (4544): No heartbeat from core client for 30 sec - exiting 15:21:51 (4544): No heartbeat from core client for 30 sec - exiting 15:21:52 (4544): No heartbeat from core client for 30 sec - exiting 15:21:53 (4544): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:30:14 (1848): No heartbeat from core client for 30 sec - exiting 16:30:16 (1848): No heartbeat from core client for 30 sec - exiting 16:30:17 (1848): No heartbeat from core client for 30 sec - exiting 16:30:18 (1848): No heartbeat from core client for 30 sec - exiting 16:30:19 (1848): No heartbeat from core client for 30 sec - exiting 16:30:20 (1848): No heartbeat from core client for 30 sec - exiting 16:30:21 (1848): No heartbeat from core client for 30 sec - exiting 16:30:22 (1848): No heartbeat from core client for 30 sec - exiting 16:30:23 (1848): No heartbeat from core client for 30 sec - exiting 16:30:24 (1848): No heartbeat from core client for 30 sec - exiting 16:30:25 (1848): No heartbeat from core client for 30 sec - exiting 16:30:26 (1848): No heartbeat from core client for 30 sec - exiting 16:30:27 (1848): No heartbeat from core client for 30 sec - exiting 16:30:28 (1848): No heartbeat from core client for 30 sec - exiting 16:30:29 (1848): No heartbeat from core client for 30 sec - exiting 16:30:30 (1848): No heartbeat from core client for 30 sec - exiting 16:30:31 (1848): No heartbeat from core client for 30 sec - exiting 16:30:32 (1848): No heartbeat from core client for 30 sec - exiting 16:30:33 (1848): No heartbeat from core client for 30 sec - exiting 16:30:34 (1848): No heartbeat from core client for 30 sec - exiting 16:30:35 (1848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 07:43:02 (5464): No heartbeat from core client for 30 sec - exiting 07:43:03 (5464): No heartbeat from core client for 30 sec - exiting 07:43:04 (5464): No heartbeat from core client for 30 sec - exiting 07:43:05 (5464): No heartbeat from core client for 30 sec - exiting 07:43:06 (5464): No heartbeat from core client for 30 sec - exiting 07:43:07 (5464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:36:23 (3248): No heartbeat from core client for 30 sec - exiting 14:36:24 (3248): No heartbeat from core client for 30 sec - exiting 14:36:25 (3248): No heartbeat from core client for 30 sec - exiting 14:36:26 (3248): No heartbeat from core client for 30 sec - exiting 14:36:27 (3248): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 17:19:45 (6368): No heartbeat from core client for 30 sec - exiting 07:46:59 (2892): No heartbeat from core client for 30 sec - exiting 07:47:00 (2892): No heartbeat from core client for 30 sec - exiting 07:47:01 (2892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:23:50 (4788): No heartbeat from core client for 30 sec - exiting 07:23:52 (4788): No heartbeat from core client for 30 sec - exiting 07:23:53 (4788): No heartbeat from core client for 30 sec - exiting 07:23:54 (4788): No heartbeat from core client for 30 sec - exiting 07:23:55 (4788): No heartbeat from core client for 30 sec - exiting 07:23:56 (4788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3968, iMonCtr=1 Model crash detected, will try to restart... 18:36:24 (992): No heartbeat from core client for 30 sec - exiting 18:36:26 (992): No heartbeat from core client for 30 sec - exiting 18:36:27 (992): No heartbeat from core client for 30 sec - exiting 18:36:28 (992): No heartbeat from core client for 30 sec - exiting 18:36:29 (992): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:36:30 (992): No heartbeat from core client for 30 sec - exiting 18:36:31 (992): No heartbeat from core client for 30 sec - exiting 18:36:32 (992): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3928, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5820, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5424, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4624, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4728, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5080, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5644, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3972, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5188, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6304, iMonCtr=1 Model crash detected, will try to restart... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
02 Sep 2013 05:27:44	1138291	15856762	hadcm3n_zdyb_1960_40_008394587_0	233,280	463,818	1.9882
28 Aug 2013 18:46:05	1138291	15856762	hadcm3n_zdyb_1960_40_008394587_0	207,360	416,111	2.0067
25 Aug 2013 12:43:49	1138291	15856762	hadcm3n_zdyb_1960_40_008394587_0	181,440	365,666	2.0154
23 Aug 2013 10:17:13	1138291	15856762	hadcm3n_zdyb_1960_40_008394587_0	155,520	313,198	2.0139
15 Aug 2013 01:12:57	1138291	15856762	hadcm3n_zdyb_1960_40_008394587_0	129,600	262,769	2.0275
15 Aug 2013 01:12:57	1138291	15856762	hadcm3n_zdyb_1960_40_008394587_0	103,680	211,909	2.0439
23 Jul 2013 21:41:05	1138291	15856762	hadcm3n_zdyb_1960_40_008394587_0	77,760	160,451	2.0634
23 Jul 2013 20:13:29	1138291	15856762	hadcm3n_zdyb_1960_40_008394587_0	51,840	106,945	2.0630
02 Jul 2013 10:27:10	1138291	15856762	hadcm3n_zdyb_1960_40_008394587_0	25,920	54,430	2.0999