Task 15923538

Name	hadcm3n_o4fs_2020_40_008399097_3
Workunit	8549953
Created	16 Aug 2013, 5:38:10 UTC
Sent	16 Aug 2013, 5:38:25 UTC
Report deadline	15 Nov 2013, 13:05:36 UTC
Received	9 Sep 2013, 18:19:57 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	25 (0x00000019) Unknown error code
Computer ID	1204618
Run time	7 days 17 hours 12 min 34 sec
CPU time	7 days 0 hours 57 min 49 sec
Validate state	Invalid
Credit	2,799.36
Device peak FLOPS	2.07 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> The drive cannot locate a specific area or track on the disk. (0x19) - exit code 25 (0x19) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1484, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy 2048 Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7012, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6808, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6808, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6808, iMonCtr=1 Model crash detected, will try to restart... Signal 4 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6808, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6696, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6696, iMonCtr=1 Model crash detected, will try to restart... Signal 4 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6696, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6696, iMonCtr=1 Model crash detected, will try to restart... Signal 4 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6696, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6696, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6696, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6696, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6696, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6696, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6696, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2420, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2420, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2420, iMonCtr=1 Model crash detected, will try to restart... Signal 4 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2420, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2420, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2420, iMonCtr=1 Model crash detected, will try to restart... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2420, iMonCtr=1 Model crash detected, will try to restart... Signal 4 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2420, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 17:05:38 (3472): No heartbeat from core client for 30 sec - exiting 17:05:39 (3472): No heartbeat from core client for 30 sec - exiting 17:05:40 (3472): No heartbeat from core client for 30 sec - exiting 17:05:41 (3472): No heartbeat from core client for 30 sec - exiting 17:05:42 (3472): No heartbeat from core client for 30 sec - exiting 17:05:43 (3472): No heartbeat from core client for 30 sec - exiting 17:05:44 (3472): No heartbeat from core client for 30 sec - exiting 17:05:45 (3472): No heartbeat from core client for 30 sec - exiting 17:05:46 (3472): No heartbeat from core client for 30 sec - exiting 17:05:47 (3472): No heartbeat from core client for 30 sec - exiting 17:05:48 (3472): No heartbeat from core client for 30 sec - exiting 17:05:49 (3472): No heartbeat from core client for 30 sec - exiting 17:05:50 (3472): No heartbeat from core client for 30 sec - exiting 17:05:51 (3472): No heartbeat from core client for 30 sec - exiting 17:05:52 (3472): No heartbeat from core client for 30 sec - exiting 17:05:53 (3472): No heartbeat from core client for 30 sec - exiting 17:05:54 (3472): No heartbeat from core client for 30 sec - exiting 17:05:55 (3472): No heartbeat from core client for 30 sec - exiting 17:05:56 (3472): No heartbeat from core client for 30 sec - exiting 17:05:57 (3472): No heartbeat from core client for 30 sec - exiting 17:05:58 (3472): No heartbeat from core client for 30 sec - exiting 17:05:59 (3472): No heartbeat from core client for 30 sec - exiting 17:06:00 (3472): No heartbeat from core client for 30 sec - exiting 17:06:01 (3472): No heartbeat from core client for 30 sec - exiting 17:06:02 (3472): No heartbeat from core client for 30 sec - exiting 17:06:03 (3472): No heartbeat from core client for 30 sec - exiting 17:06:04 (3472): No heartbeat from core client for 30 sec - exiting 17:06:05 (3472): No heartbeat from core client for 30 sec - exiting 17:06:06 (3472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5264, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5264, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5016, iMonCtr=1 Model crash detected, will try to restart... 13:03:21 (7288): Can't acquire lockfile (32) - waiting 35s Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5016, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5016, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2784, iMonCtr=1 Model crash detected, will try to restart... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
05 Sep 2013 11:48:53	1204618	15923538	hadcm3n_o4fs_2020_40_008399097_3	233,280	574,003	2.4606
04 Sep 2013 20:15:10	1204618	15923538	hadcm3n_o4fs_2020_40_008399097_3	207,360	524,264	2.5283
30 Aug 2013 04:40:09	1204618	15923538	hadcm3n_o4fs_2020_40_008399097_3	181,440	473,183	2.6079
30 Aug 2013 04:40:09	1204618	15923538	hadcm3n_o4fs_2020_40_008399097_3	155,520	421,618	2.7110
28 Aug 2013 02:16:08	1204618	15923538	hadcm3n_o4fs_2020_40_008399097_3	129,600	370,236	2.8568
27 Aug 2013 12:07:16	1204618	15923538	hadcm3n_o4fs_2020_40_008399097_3	103,680	319,223	3.0789
19 Aug 2013 07:04:22	1204618	15923538	hadcm3n_o4fs_2020_40_008399097_3	77,760	119,502	1.5368
18 Aug 2013 06:24:15	1204618	15923538	hadcm3n_o4fs_2020_40_008399097_3	51,840	102,231	1.9720
17 Aug 2013 14:40:41	1204618	15923538	hadcm3n_o4fs_2020_40_008399097_3	25,920	50,575	1.9512