Task 13791416

Name	hadam3p_saf_7c1o_2002_1_007572549_1
Workunit	7750679
Created	17 Dec 2011, 18:49:29 UTC
Sent	17 Dec 2011, 19:31:08 UTC
Report deadline	29 Nov 2012, 0:51:08 UTC
Received	10 Jun 2012, 16:50:48 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1162375
Run time	4 days 5 hours 59 min 51 sec
CPU time	11 hours 43 min 5 sec
Validate state	Invalid
Credit	1,870.33
Device peak FLOPS	2.50 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Southern Africa v6.09 windows_intelx86
Stderr	<core_client_version>7.0.25</core_client_version> <![CDATA[ <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 17:38:48 (7108): Can't acquire lockfile (32) - waiting 35s 17:39:17 (6956): No heartbeat from core client for 30 sec - exiting 17:39:18 (6956): No heartbeat from core client for 30 sec - exiting 17:39:19 (6956): No heartbeat from core client for 30 sec - exiting 17:39:20 (6956): No heartbeat from core client for 30 sec - exiting 17:39:21 (6956): No heartbeat from core client for 30 sec - exiting 17:39:22 (6956): No heartbeat from core client for 30 sec - exiting 17:39:23 (7108): Can't acquire lockfile (32) - exiting 17:39:23 (7108): Error: Der Prozess kann nicht auf die Datei zugreifen, da sie von einem anderen Prozess verwendet wird. (0x20) 17:39:23 (6956): No heartbeat from core client for 30 sec - exiting 17:39:24 (6956): No heartbeat from core client for 30 sec - exiting 17:39:25 (6956): No heartbeat from core client for 30 sec - exiting 17:39:26 (6956): No heartbeat from core client for 30 sec - exiting 17:39:27 (6956): No heartbeat from core client for 30 sec - exiting 17:39:28 (6956): No heartbeat from core client for 30 sec - exiting 17:39:29 (6956): No heartbeat from core client for 30 sec - exiting 17:39:30 (6956): No heartbeat from core client for 30 sec - exiting 17:39:31 (6956): No heartbeat from core client for 30 sec - exiting 17:39:32 (6956): No heartbeat from core client for 30 sec - exiting 17:39:33 (6956): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:39:34 (6956): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 18:58:57 (5020): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:26:16 (4220): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:29:18 (7004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 07:25:30 (2080): No heartbeat from core client for 30 sec - exiting 07:25:31 (2080): No heartbeat from core client for 30 sec - exiting 07:25:32 (2080): No heartbeat from core client for 30 sec - exiting 07:25:33 (2080): No heartbeat from core client for 30 sec - exiting 07:25:34 (2080): No heartbeat from core client for 30 sec - exiting 07:25:35 (2080): No heartbeat from core client for 30 sec - exiting 07:25:36 (2080): No heartbeat from core client for 30 sec - exiting 07:25:37 (2080): No heartbeat from core client for 30 sec - exiting 07:25:38 (2080): No heartbeat from core client for 30 sec - exiting 07:25:39 (2080): No heartbeat from core client for 30 sec - exiting 07:25:40 (2080): No heartbeat from core client for 30 sec - exiting 07:25:41 (2080): No heartbeat from core client for 30 sec - exiting 07:25:42 (2080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:25:43 (2080): No heartbeat from core client for 30 sec - exiting 18:16:43 (4980): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:52:57 (4744): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:08:02 (4920): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:10:56 (6324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 23:02:02 (5928): No heartbeat from core client for 30 sec - exiting 23:02:03 (5928): No heartbeat from core client for 30 sec - exiting 23:02:04 (5928): No heartbeat from core client for 30 sec - exiting 23:02:05 (5928): No heartbeat from core client for 30 sec - exiting 23:02:06 (5928): No heartbeat from core client for 30 sec - exiting 23:02:07 (5928): No heartbeat from core client for 30 sec - exiting 23:02:08 (5928): No heartbeat from core client for 30 sec - exiting 23:02:09 (5928): No heartbeat from core client for 30 sec - exiting 23:02:10 (5928): No heartbeat from core client for 30 sec - exiting 23:02:11 (5928): No heartbeat from core client for 30 sec - exiting 23:02:12 (5928): No heartbeat from core client for 30 sec - exiting 23:02:13 (5928): No heartbeat from core client for 30 sec - exiting 23:02:14 (5928): No heartbeat from core client for 30 sec - exiting 23:02:15 (5928): No heartbeat from core client for 30 sec - exiting 23:02:16 (5928): No heartbeat from core client for 30 sec - exiting 23:02:17 (5928): No heartbeat from core client for 30 sec - exiting 23:02:18 (5928): No heartbeat from core client for 30 sec - exiting 23:02:19 (5928): No heartbeat from core client for 30 sec - exiting 23:02:20 (5928): No heartbeat from core client for 30 sec - exiting 23:02:21 (5928): No heartbeat from core client for 30 sec - exiting 23:02:22 (5928): No heartbeat from core client for 30 sec - exiting 23:02:23 (5928): No heartbeat from core client for 30 sec - exiting 23:02:24 (5928): No heartbeat from core client for 30 sec - exiting 23:02:25 (5928): No heartbeat from core client for 30 sec - exiting 23:02:26 (5928): No heartbeat from core client for 30 sec - exiting 23:02:27 (5928): No heartbeat from core client for 30 sec - exiting 23:02:28 (5928): No heartbeat from core client for 30 sec - exiting 23:02:29 (5928): No heartbeat from core client for 30 sec - exiting 23:02:30 (5928): No heartbeat from core client for 30 sec - exiting 23:02:31 (5928): No heartbeat from core client for 30 sec - exiting 23:02:32 (5928): No heartbeat from core client for 30 sec - exiting 23:02:33 (5928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 21:54:55 (4488): No heartbeat from core client for 30 sec - exiting 21:54:56 (4488): No heartbeat from core client for 30 sec - exiting 21:54:57 (4488): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:22:09 (5032): No heartbeat from core client for 30 sec - exiting 19:22:10 (5032): No heartbeat from core client for 30 sec - exiting 19:22:11 (5032): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 19:13:46 (4776): No heartbeat from core client for 30 sec - exiting 19:13:47 (4776): No heartbeat from core client for 30 sec - exiting 19:13:48 (4776): No heartbeat from core client for 30 sec - exiting 19:13:49 (4776): No heartbeat from core client for 30 sec - exiting 19:13:50 (4776): No heartbeat from core client for 30 sec - exiting 19:13:52 (4776): No heartbeat from core client for 30 sec - exiting 19:13:53 (4776): No heartbeat from core client for 30 sec - exiting 19:13:54 (4776): No heartbeat from core client for 30 sec - exiting 19:13:55 (4776): No heartbeat from core client for 30 sec - exiting 19:13:56 (4776): No heartbeat from core client for 30 sec - exiting 19:13:57 (4776): No heartbeat from core client for 30 sec - exiting 19:13:58 (4776): No heartbeat from core client for 30 sec - exiting 19:13:59 (4776): No heartbeat from core client for 30 sec - exiting 19:14:00 (4776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 06:47:18 (2768): No heartbeat from core client for 30 sec - exiting 06:47:19 (2768): No heartbeat from core client for 30 sec - exiting 06:47:20 (2768): No heartbeat from core client for 30 sec - exiting 06:47:21 (2768): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6732, selfPID=4232, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:05:05 (4472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2828, selfPID=2828, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:12:29 (4588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4908, selfPID=4908, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6528, selfPID=4616, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6532, selfPID=4568, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7016, selfPID=7016, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6580, selfPID=6580, iMonCtr=2 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6784, selfPID=6784, iMonCtr=2 18:38:19 (4552): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5284, selfPID=5284, iMonCtr=2 </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_saf_7c1o_2002_1_007572549_1_11.zip</file_name> <error_code>-161</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_saf_7c1o_2002_1_007572549_1_12.zip</file_name> <error_code>-161</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
29 Apr 2012 17:25:32	1162375	13791416	hadam3p_saf_7c1o_2002_1_007572549_1	115,296	250,075	2.1690
18 Apr 2012 18:49:59	1162375	13791416	hadam3p_saf_7c1o_2002_1_007572549_1	103,776	224,499	2.1633
15 Apr 2012 15:21:11	1162375	13791416	hadam3p_saf_7c1o_2002_1_007572549_1	92,256	201,229	2.1812
24 Feb 2012 22:54:26	1162375	13791416	hadam3p_saf_7c1o_2002_1_007572549_1	80,736	176,090	2.1811
14 Feb 2012 19:25:30	1162375	13791416	hadam3p_saf_7c1o_2002_1_007572549_1	69,216	150,966	2.1811
12 Feb 2012 20:33:27	1162375	13791416	hadam3p_saf_7c1o_2002_1_007572549_1	57,696	125,697	2.1786
07 Feb 2012 18:53:48	1162375	13791416	hadam3p_saf_7c1o_2002_1_007572549_1	46,176	99,921	2.1639
04 Feb 2012 15:53:23	1162375	13791416	hadam3p_saf_7c1o_2002_1_007572549_1	34,656	76,778	2.2154
22 Jan 2012 12:16:53	1162375	13791416	hadam3p_saf_7c1o_2002_1_007572549_1	23,136	51,502	2.2261
11 Jan 2012 19:08:13	1162375	13791416	hadam3p_saf_7c1o_2002_1_007572549_1	11,616	25,958	2.2347