Task 13361570

Name	hadcm3n_t6l4_1940_40_007448503_0
Workunit	7646006
Created	9 Sep 2011, 21:18:55 UTC
Sent	15 Sep 2011, 6:32:07 UTC
Report deadline	15 Dec 2011, 13:59:18 UTC
Received	28 Oct 2011, 20:09:29 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1102379
Run time	17 days 22 hours 39 min 32 sec
CPU time	16 days 21 hours 1 min 40 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	2.91 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.60</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3796, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:45:54 (4144): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:46:28 (5000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Restart file copy failed on t6l4ka.dae59m0 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file D:\BOINC\BOINC_DATA/projects/climateprediction.net/hadcm3n_se_6.07_windows_intelx86.dll after 11 attempts cpdnmonitor: cannot open input file D:\BOINC\BOINC_DATA/projects/climateprediction.net/hadcm3n_um_6.07_windows_intelx86.exe after 11 attempts cpdnmonitor: cannot open input file D:\BOINC\BOINC_DATA/projects/climateprediction.net/hadcm3n_t6l4_1940_40_007448503/jobs/xabnk.ihist after 11 attempts cpdnmonitor: cannot open input file D:\BOINC\BOINC_DATA/projects/climateprediction.net/hadcm3n_t6l4_1940_40_007448503/jobs/xabnk.namelists after 11 attempts cpdnmonitor: cannot open input file D:\BOINC\BOINC_DATA/projects/climateprediction.net/hadcm3n_t6l4_1940_40_007448503/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file D:\BOINC\BOINC_DATA/projects/climateprediction.net/hadcm3n_t6l4_1940_40_007448503/dataout/ocean_restart.day after 11 attempts 15:29:26 (3108): Can't open init data file - running in standalone mode Could not launch model process. Last Error=2 Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3428, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3436, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3512, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3540, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3424, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3376, iMonCtr=1 Model crash detected, will try to restart... 00:01:47 (3736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:02:47 (136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:15:01 (344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... C07:53:52 (3492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:54:52 (3576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:01:57 (3848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:02:36 (1188): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:27:03 (4756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:46:59 (3700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:48:43 (3624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:49:44 (1220): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3576, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3948, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3596, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3440, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3568, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3696, iMonCtr=1 Model crash detected, will try to restart... CController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3668, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3548, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
31 Oct 2011 18:13:40	1102379	13361570	hadcm3n_t6l4_1940_40_007448503_0	777,600	1,458,092	1.8751
31 Oct 2011 16:50:34	1102379	13361570	hadcm3n_t6l4_1940_40_007448503_0	751,680	1,411,700	1.8781
31 Oct 2011 16:33:59	1102379	13361570	hadcm3n_t6l4_1940_40_007448503_0	725,760	1,364,260	1.8798
31 Oct 2011 16:33:59	1102379	13361570	hadcm3n_t6l4_1940_40_007448503_0	699,840	1,315,143	1.8792
18 Oct 2011 19:22:09	1102379	13361570	hadcm3n_t6l4_1940_40_007448503_0	673,920	1,265,775	1.8782
17 Oct 2011 15:28:50	1102379	13361570	hadcm3n_t6l4_1940_40_007448503_0	648,000	1,216,639	1.8775
17 Oct 2011 01:03:57	1102379	13361570	hadcm3n_t6l4_1940_40_007448503_0	622,080	1,167,445	1.8767
14 Oct 2011 20:51:11	1102379	13361570	hadcm3n_t6l4_1940_40_007448503_0	596,160	1,118,213	1.8757
13 Oct 2011 23:20:46	1102379	13361570	hadcm3n_t6l4_1940_40_007448503_0	570,240	1,069,183	1.8750
12 Oct 2011 23:09:34	1102379	13361570	hadcm3n_t6l4_1940_40_007448503_0	544,320	1,020,733	1.8752
12 Oct 2011 01:54:00	1102379	13361570	hadcm3n_t6l4_1940_40_007448503_0	518,400	971,422	1.8739
11 Oct 2011 02:47:08	1102379	13361570	hadcm3n_t6l4_1940_40_007448503_0	492,480	922,268	1.8727
08 Oct 2011 21:51:32	1102379	13361570	hadcm3n_t6l4_1940_40_007448503_0	466,560	873,266	1.8717
06 Oct 2011 07:01:19	1102379	13361570	hadcm3n_t6l4_1940_40_007448503_0	440,640	824,295	1.8707
05 Oct 2011 08:39:59	1102379	13361570	hadcm3n_t6l4_1940_40_007448503_0	414,720	775,066	1.8689
05 Oct 2011 00:06:14	1102379	13361570	hadcm3n_t6l4_1940_40_007448503_0	388,800	726,011	1.8673
03 Oct 2011 16:54:33	1102379	13361570	hadcm3n_t6l4_1940_40_007448503_0	362,880	676,425	1.8640
02 Oct 2011 19:08:55	1102379	13361570	hadcm3n_t6l4_1940_40_007448503_0	336,960	627,385	1.8619
30 Sep 2011 17:09:04	1102379	13361570	hadcm3n_t6l4_1940_40_007448503_0	311,040	578,404	1.8596
30 Sep 2011 02:30:09	1102379	13361570	hadcm3n_t6l4_1940_40_007448503_0	285,120	529,225	1.8561