Task 15613046

Name	hadcm3n_3ls4_1940_40_008260881_1
Workunit	8416005
Created	19 Feb 2013, 4:11:49 UTC
Sent	19 Feb 2013, 4:12:08 UTC
Report deadline	21 May 2013, 11:39:19 UTC
Received	25 Feb 2013, 5:37:29 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1336997
Run time	5 days 4 hours 49 min 54 sec
CPU time	4 days 13 hours 58 min 13 sec
Validate state	Invalid
Credit	5,287.68
Device peak FLOPS	3.76 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4936, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4936, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4936, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:29:56 (5088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7052, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7052, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7052, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7052, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7052, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7052, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
25 Feb 2013 00:36:33	1269906	15613046	hadcm3n_3ls4_1940_40_008260881_1	440,640	383,346	0.8700
24 Feb 2013 14:00:37	1269906	15613046	hadcm3n_3ls4_1940_40_008260881_1	414,720	358,329	0.8640
24 Feb 2013 04:13:32	1269906	15613046	hadcm3n_3ls4_1940_40_008260881_1	388,800	333,096	0.8567
23 Feb 2013 20:28:52	1269906	15613046	hadcm3n_3ls4_1940_40_008260881_1	362,880	307,837	0.8483
23 Feb 2013 12:50:08	1269906	15613046	hadcm3n_3ls4_1940_40_008260881_1	336,960	282,719	0.8390
23 Feb 2013 04:56:20	1269906	15613046	hadcm3n_3ls4_1940_40_008260881_1	311,040	257,691	0.8285
22 Feb 2013 20:58:36	1269906	15613046	hadcm3n_3ls4_1940_40_008260881_1	285,120	232,861	0.8167
22 Feb 2013 12:17:40	1269906	15613046	hadcm3n_3ls4_1940_40_008260881_1	259,200	208,167	0.8031
22 Feb 2013 02:50:43	1269906	15613046	hadcm3n_3ls4_1940_40_008260881_1	233,280	183,715	0.7875
21 Feb 2013 22:04:22	1269906	15613046	hadcm3n_3ls4_1940_40_008260881_1	207,360	166,445	0.8027
21 Feb 2013 13:00:10	1269906	15613046	hadcm3n_3ls4_1940_40_008260881_1	181,440	146,138	0.8054
21 Feb 2013 02:08:12	1269906	15613046	hadcm3n_3ls4_1940_40_008260881_1	155,520	123,177	0.7920
20 Feb 2013 19:39:04	1269906	15613046	hadcm3n_3ls4_1940_40_008260881_1	129,600	99,577	0.7683
20 Feb 2013 11:48:41	1269906	15613046	hadcm3n_3ls4_1940_40_008260881_1	103,680	75,644	0.7296
20 Feb 2013 02:31:51	1269906	15613046	hadcm3n_3ls4_1940_40_008260881_1	77,760	70,715	0.9094
19 Feb 2013 19:42:43	1269906	15613046	hadcm3n_3ls4_1940_40_008260881_1	51,840	47,301	0.9124
19 Feb 2013 13:15:15	1269906	15613046	hadcm3n_3ls4_1940_40_008260881_1	25,920	23,926	0.9231