Task 16118377

Name	hadam3p_eu_izl5_1999_1_008491310_0
Workunit	8642123
Created	3 Dec 2013, 21:08:25 UTC
Sent	4 Dec 2013, 19:22:31 UTC
Report deadline	17 Nov 2014, 0:42:31 UTC
Received	27 Jan 2014, 21:22:13 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1263465
Run time	6 days 18 hours 34 min 42 sec
CPU time	5 days 22 hours 16 min 24 sec
Validate state	Workunit error - check skipped
Credit	2,386.39
Device peak FLOPS	1.71 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Europe v6.09 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5296, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 17:57:29 (4700): No heartbeat from core client for 30 sec - exiting 17:57:30 (4700): No heartbeat from core client for 30 sec - exiting 17:57:31 (4700): No heartbeat from core client for 30 sec - exiting 17:57:32 (4700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6024, selfPID=436, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4180, iMonCtr=2 Model crash detected, will try to restart... 11:26:48 (4312): No heartbeat from core client for 30 sec - exiting 11:26:49 (4312): No heartbeat from core client for 30 sec - exiting 11:26:50 (4312): No heartbeat from core client for 30 sec - exiting 11:26:51 (4312): No heartbeat from core client for 30 sec - exiting 11:26:53 (4312): No heartbeat from core client for 30 sec - exiting 11:26:54 (4312): No heartbeat from core client for 30 sec - exiting 11:26:55 (4312): No heartbeat from core client for 30 sec - exiting 11:26:56 (4312): No heartbeat from core client for 30 sec - exiting 11:26:57 (4312): No heartbeat from core client for 30 sec - exiting 11:26:58 (4312): No heartbeat from core client for 30 sec - exiting 11:26:59 (4312): No heartbeat from core client for 30 sec - exiting 11:27:00 (4312): No heartbeat from core client for 30 sec - exiting 11:27:01 (4312): No heartbeat from core client for 30 sec - exiting 11:27:02 (4312): No heartbeat from core client for 30 sec - exiting 11:27:03 (4312): No heartbeat from core client for 30 sec - exiting 11:27:05 (4312): No heartbeat from core client for 30 sec - exiting 11:27:06 (4312): No heartbeat from core client for 30 sec - exiting 11:27:07 (4312): No heartbeat from core client for 30 sec - exiting 11:27:08 (4312): No heartbeat from core client for 30 sec - exiting 11:27:09 (4312): No heartbeat from core client for 30 sec - exiting 11:27:10 (4312): No heartbeat from core client for 30 sec - exiting 11:27:11 (4312): No heartbeat from core client for 30 sec - exiting 11:27:12 (4312): No heartbeat from core client for 30 sec - exiting 11:27:13 (4312): No heartbeat from core client for 30 sec - exiting 11:27:14 (4312): No heartbeat from core client for 30 sec - exiting 11:27:15 (4312): No heartbeat from core client for 30 sec - exiting 11:27:17 (4312): No heartbeat from core client for 30 sec - exiting 11:27:18 (4312): No heartbeat from core client for 30 sec - exiting 11:27:19 (4312): No heartbeat from core client for 30 sec - exiting 11:27:20 (4312): No heartbeat from core client for 30 sec - exiting 11:27:21 (4312): No heartbeat from core client for 30 sec - exiting 11:27:22 (4312): No heartbeat from core client for 30 sec - exiting 11:27:23 (4312): No heartbeat from core client for 30 sec - exiting 11:27:24 (4312): No heartbeat from core client for 30 sec - exiting 11:27:25 (4312): No heartbeat from core client for 30 sec - exiting 11:27:26 (4312): No heartbeat from core client for 30 sec - exiting 11:27:28 (4312): No heartbeat from core client for 30 sec - exiting 11:27:29 (4312): No heartbeat from core client for 30 sec - exiting 11:27:30 (4312): No heartbeat from core client for 30 sec - exiting 11:27:31 (4312): No heartbeat from core client for 30 sec - exiting 11:27:32 (4312): No heartbeat from core client for 30 sec - exiting 11:27:33 (4312): No heartbeat from core client for 30 sec - exiting 11:27:34 (4312): No heartbeat from core client for 30 sec - exiting 11:27:35 (4312): No heartbeat from core client for 30 sec - exiting 11:27:36 (4312): No heartbeat from core client for 30 sec - exiting 11:27:37 (4312): No heartbeat from core client for 30 sec - exiting 11:27:38 (4312): No heartbeat from core client for 30 sec - exiting 11:27:40 (4312): No heartbeat from core client for 30 sec - exiting 11:27:41 (4312): No heartbeat from core client for 30 sec - exiting 11:27:42 (4312): No heartbeat from core client for 30 sec - exiting 11:27:43 (4312): No heartbeat from core client for 30 sec - exiting 11:27:44 (4312): No heartbeat from core client for 30 sec - exiting 11:27:45 (4312): No heartbeat from core client for 30 sec - exiting 11:27:46 (4312): No heartbeat from core client for 30 sec - exiting 11:27:47 (4312): No heartbeat from core client for 30 sec - exiting 11:27:48 (4312): No heartbeat from core client for 30 sec - exiting 11:28:42 (4312): No heartbeat from core client for 30 sec - exiting 11:28:43 (4312): No heartbeat from core client for 30 sec - exiting 11:28:44 (4312): No heartbeat from core client for 30 sec - exiting 11:28:45 (4312): No heartbeat from core client for 30 sec - exiting 11:28:46 (4312): No heartbeat from core client for 30 sec - exiting 11:28:47 (4312): No heartbeat from core client for 30 sec - exiting 11:28:48 (4312): No heartbeat from core client for 30 sec - exiting 11:28:49 (4312): No heartbeat from core client for 30 sec - exiting 11:28:50 (4312): No heartbeat from core client for 30 sec - exiting 11:28:51 (4312): No heartbeat from core client for 30 sec - exiting 11:28:52 (4312): No heartbeat from core client for 30 sec - exiting 11:28:53 (4312): No heartbeat from core client for 30 sec - exiting 11:28:54 (4312): No heartbeat from core client for 30 sec - exiting 11:28:55 (4312): No heartbeat from core client for 30 sec - exiting 11:28:56 (4312): No heartbeat from core client for 30 sec - exiting 11:28:57 (4312): No heartbeat from core client for 30 sec - exiting 11:28:58 (4312): No heartbeat from core client for 30 sec - exiting 11:28:59 (4312): No heartbeat from core client for 30 sec - exiting 11:29:00 (4312): No heartbeat from core client for 30 sec - exiting 11:29:01 (4312): No heartbeat from core client for 30 sec - exiting 11:29:02 (4312): No heartbeat from core client for 30 sec - exiting 11:29:03 (4312): No heartbeat from core client for 30 sec - exiting 11:29:04 (4312): No heartbeat from core client for 30 sec - exiting 11:29:05 (4312): No heartbeat from core client for 30 sec - exiting 11:29:06 (4312): No heartbeat from core client for 30 sec - exiting 11:29:07 (4312): No heartbeat from core client for 30 sec - exiting 11:29:09 (4312): No heartbeat from core client for 30 sec - exiting 11:29:10 (4312): No heartbeat from core client for 30 sec - exiting 11:29:11 (4312): No heartbeat from core client for 30 sec - exiting 11:29:12 (4312): No heartbeat from core client for 30 sec - exiting 11:29:13 (4312): No heartbeat from core client for 30 sec - exiting 11:29:14 (4312): No heartbeat from core client for 30 sec - exiting 11:29:15 (4312): No heartbeat from core client for 30 sec - exiting 11:29:16 (4312): No heartbeat from core client for 30 sec - exiting 11:29:17 (4312): No heartbeat from core client for 30 sec - exiting 11:29:18 (4312): No heartbeat from core client for 30 sec - exiting 11:29:19 (4312): No heartbeat from core client for 30 sec - exiting 11:29:21 (4312): No heartbeat from core client for 30 sec - exiting 11:29:22 (4312): No heartbeat from core client for 30 sec - exiting 11:29:23 (4312): No heartbeat from core client for 30 sec - exiting 11:29:24 (4312): No heartbeat from core client for 30 sec - exiting 11:29:25 (4312): No heartbeat from core client for 30 sec - exiting 11:29:26 (4312): No heartbeat from core client for 30 sec - exiting 11:29:27 (4312): No heartbeat from core client for 30 sec - exiting 11:29:28 (4312): No heartbeat from core client for 30 sec - exiting 11:29:29 (4312): No heartbeat from core client for 30 sec - exiting 11:29:30 (4312): No heartbeat from core client for 30 sec - exiting 11:29:31 (4312): No heartbeat from core client for 30 sec - exiting 11:29:33 (4312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2808, selfPID=4668, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5928, selfPID=4272, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1812, selfPID=2992, iMonCtr=1 Model crash detected, will try to restart... 17:18:52 (5080): No heartbeat from core client for 30 sec - exiting 17:18:53 (5080): No heartbeat from core client for 30 sec - exiting 17:18:54 (5080): No heartbeat from core client for 30 sec - exiting 17:18:55 (5080): No heartbeat from core client for 30 sec - exiting 17:18:57 (5080): No heartbeat from core client for 30 sec - exiting 17:18:58 (5080): No heartbeat from core client for 30 sec - exiting 17:18:59 (5080): No heartbeat from core client for 30 sec - exiting 17:19:00 (5080): No heartbeat from core client for 30 sec - exiting 17:19:01 (5080): No heartbeat from core client for 30 sec - exiting 17:19:02 (5080): No heartbeat from core client for 30 sec - exiting 17:19:03 (5080): No heartbeat from core client for 30 sec - exiting 17:19:04 (5080): No heartbeat from core client for 30 sec - exiting 17:19:05 (5080): No heartbeat from core client for 30 sec - exiting 17:19:06 (5080): No heartbeat from core client for 30 sec - exiting 17:19:08 (5080): No heartbeat from core client for 30 sec - exiting 17:19:09 (5080): No heartbeat from core client for 30 sec - exiting 17:19:10 (5080): No heartbeat from core client for 30 sec - exiting 17:19:11 (5080): No heartbeat from core client for 30 sec - exiting 17:19:12 (5080): No heartbeat from core client for 30 sec - exiting 17:19:13 (5080): No heartbeat from core client for 30 sec - exiting 17:19:14 (5080): No heartbeat from core client for 30 sec - exiting 17:19:15 (5080): No heartbeat from core client for 30 sec - exiting 17:19:16 (5080): No heartbeat from core client for 30 sec - exiting 17:19:17 (5080): No heartbeat from core client for 30 sec - exiting 17:19:18 (5080): No heartbeat from core client for 30 sec - exiting 17:19:20 (5080): No heartbeat from core client for 30 sec - exiting 17:19:21 (5080): No heartbeat from core client for 30 sec - exiting 17:19:22 (5080): No heartbeat from core client for 30 sec - exiting 17:19:23 (5080): No heartbeat from core client for 30 sec - exiting 17:19:24 (5080): No heartbeat from core client for 30 sec - exiting 17:19:25 (5080): No heartbeat from core client for 30 sec - exiting 17:19:26 (5080): No heartbeat from core client for 30 sec - exiting 17:19:27 (5080): No heartbeat from core client for 30 sec - exiting 17:19:28 (5080): No heartbeat from core client for 30 sec - exiting 17:19:29 (5080): No heartbeat from core client for 30 sec - exiting 17:19:30 (5080): No heartbeat from core client for 30 sec - exiting 17:19:32 (5080): No heartbeat from core client for 30 sec - exiting 17:19:33 (5080): No heartbeat from core client for 30 sec - exiting 17:19:34 (5080): No heartbeat from core client for 30 sec - exiting 17:19:35 (5080): No heartbeat from core client for 30 sec - exiting 17:19:36 (5080): No heartbeat from core client for 30 sec - exiting 17:19:37 (5080): No heartbeat from core client for 30 sec - exiting 17:19:38 (5080): No heartbeat from core client for 30 sec - exiting 17:19:39 (5080): No heartbeat from core client for 30 sec - exiting 17:19:40 (5080): No heartbeat from core client for 30 sec - exiting 17:19:42 (5080): No heartbeat from core client for 30 sec - exiting 17:19:43 (5080): No heartbeat from core client for 30 sec - exiting 17:19:44 (5080): No heartbeat from core client for 30 sec - exiting 17:19:45 (5080): No heartbeat from core client for 30 sec - exiting 17:19:46 (5080): No heartbeat from core client for 30 sec - exiting 17:19:47 (5080): No heartbeat from core client for 30 sec - exiting 17:19:48 (5080): No heartbeat from core client for 30 sec - exiting 17:19:49 (5080): No heartbeat from core client for 30 sec - exiting 17:19:50 (5080): No heartbeat from core client for 30 sec - exiting 17:19:51 (5080): No heartbeat from core client for 30 sec - exiting 17:19:53 (5080): No heartbeat from core client for 30 sec - exiting 17:19:54 (5080): No heartbeat from core client for 30 sec - exiting 17:19:55 (5080): No heartbeat from core client for 30 sec - exiting 17:19:56 (5080): No heartbeat from core client for 30 sec - exiting 17:19:57 (5080): No heartbeat from core client for 30 sec - exiting 17:19:58 (5080): No heartbeat from core client for 30 sec - exiting 17:19:59 (5080): No heartbeat from core client for 30 sec - exiting 17:20:00 (5080): No heartbeat from core client for 30 sec - exiting 17:20:01 (5080): No heartbeat from core client for 30 sec - exiting 17:20:02 (5080): No heartbeat from core client for 30 sec - exiting 17:20:03 (5080): No heartbeat from core client for 30 sec - exiting 17:20:05 (5080): No heartbeat from core client for 30 sec - exiting 17:20:06 (5080): No heartbeat from core client for 30 sec - exiting 17:20:07 (5080): No heartbeat from core client for 30 sec - exiting 17:20:08 (5080): No heartbeat from core client for 30 sec - exiting 17:20:09 (5080): No heartbeat from core client for 30 sec - exiting 17:20:10 (5080): No heartbeat from core client for 30 sec - exiting 17:20:11 (5080): No heartbeat from core client for 30 sec - exiting 17:20:12 (5080): No heartbeat from core client for 30 sec - exiting 17:20:13 (5080): No heartbeat from core client for 30 sec - exiting 17:20:14 (5080): No heartbeat from core client for 30 sec - exiting 17:20:15 (5080): No heartbeat from core client for 30 sec - exiting 17:20:17 (5080): No heartbeat from core client for 30 sec - exiting 17:20:18 (5080): No heartbeat from core client for 30 sec - exiting 17:20:19 (5080): No heartbeat from core client for 30 sec - exiting 17:20:20 (5080): No heartbeat from core client for 30 sec - exiting 17:20:21 (5080): No heartbeat from core client for 30 sec - exiting 17:20:22 (5080): No heartbeat from core client for 30 sec - exiting 17:20:24 (5080): No heartbeat from core client for 30 sec - exiting 17:20:25 (5080): No heartbeat from core client for 30 sec - exiting 17:20:26 (5080): No heartbeat from core client for 30 sec - exiting 17:20:27 (5080): No heartbeat from core client for 30 sec - exiting 17:20:28 (5080): No heartbeat from core client for 30 sec - exiting 17:20:29 (5080): No heartbeat from core client for 30 sec - exiting 17:20:30 (5080): No heartbeat from core client for 30 sec - exiting 17:20:32 (5080): No heartbeat from core client for 30 sec - exiting 17:20:33 (5080): No heartbeat from core client for 30 sec - exiting 17:20:34 (5080): No heartbeat from core client for 30 sec - exiting 17:20:35 (5080): No heartbeat from core client for 30 sec - exiting 17:20:36 (5080): No heartbeat from core client for 30 sec - exiting 17:20:37 (5080): No heartbeat from core client for 30 sec - exiting 17:20:38 (5080): No heartbeat from core client for 30 sec - exiting 17:20:39 (5080): No heartbeat from core client for 30 sec - exiting 17:20:40 (5080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6196, selfPID=5100, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... GController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5552, selfPID=4752, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 15:53:52 (4000): No heartbeat from core client for 30 sec - exiting 15:53:53 (4000): No heartbeat from core client for 30 sec - exiting 15:53:54 (4000): No heartbeat from core client for 30 sec - exiting 15:53:55 (4000): No heartbeat from core client for 30 sec - exiting 15:53:57 (4000): No heartbeat from core client for 30 sec - exiting 15:53:58 (4000): No heartbeat from core client for 30 sec - exiting 15:53:59 (4000): No heartbeat from core client for 30 sec - exiting 15:54:00 (4000): No heartbeat from core client for 30 sec - exiting 15:54:01 (4000): No heartbeat from core client for 30 sec - exiting 15:54:02 (4000): No heartbeat from core client for 30 sec - exiting 15:54:03 (4000): No heartbeat from core client for 30 sec - exiting 15:54:04 (4000): No heartbeat from core client for 30 sec - exiting 15:54:05 (4000): No heartbeat from core client for 30 sec - exiting 15:54:06 (4000): No heartbeat from core client for 30 sec - exiting 15:54:07 (4000): No heartbeat from core client for 30 sec - exiting 15:54:09 (4000): No heartbeat from core client for 30 sec - exiting 15:54:10 (4000): No heartbeat from core client for 30 sec - exiting 15:54:11 (4000): No heartbeat from core client for 30 sec - exiting 15:54:12 (4000): No heartbeat from core client for 30 sec - exiting 15:54:13 (4000): No heartbeat from core client for 30 sec - exiting 15:54:14 (4000): No heartbeat from core client for 30 sec - exiting 15:54:15 (4000): No heartbeat from core client for 30 sec - exiting 15:54:16 (4000): No heartbeat from core client for 30 sec - exiting 15:54:17 (4000): No heartbeat from core client for 30 sec - exiting 15:54:18 (4000): No heartbeat from core client for 30 sec - exiting 15:54:20 (4000): No heartbeat from core client for 30 sec - exiting 15:54:21 (4000): No heartbeat from core client for 30 sec - exiting 15:54:22 (4000): No heartbeat from core client for 30 sec - exiting 15:54:23 (4000): No heartbeat from core client for 30 sec - exiting 15:54:24 (4000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5144, selfPID=4344, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5300, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3888, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4856, selfPID=5152, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 13:12:47 (5108): No heartbeat from core client for 30 sec - exiting 13:12:49 (5108): No heartbeat from core client for 30 sec - exiting 13:12:50 (5108): No heartbeat from core client for 30 sec - exiting 13:12:51 (5108): No heartbeat from core client for 30 sec - exiting 13:12:52 (5108): No heartbeat from core client for 30 sec - exiting 13:12:53 (5108): No heartbeat from core client for 30 sec - exiting 13:12:54 (5108): No heartbeat from core client for 30 sec - exiting 13:12:55 (5108): No heartbeat from core client for 30 sec - exiting 13:12:56 (5108): No heartbeat from core client for 30 sec - exiting 13:12:58 (5108): No heartbeat from core client for 30 sec - exiting 13:12:59 (5108): No heartbeat from core client for 30 sec - exiting 13:13:00 (5108): No heartbeat from core client for 30 sec - exiting 13:13:01 (5108): No heartbeat from core client for 30 sec - exiting 13:13:02 (5108): No heartbeat from core client for 30 sec - exiting 13:13:03 (5108): No heartbeat from core client for 30 sec - exiting 13:13:04 (5108): No heartbeat from core client for 30 sec - exiting 13:13:05 (5108): No heartbeat from core client for 30 sec - exiting 13:13:06 (5108): No heartbeat from core client for 30 sec - exiting 13:13:07 (5108): No heartbeat from core client for 30 sec - exiting 13:13:09 (5108): No heartbeat from core client for 30 sec - exiting 13:13:10 (5108): No heartbeat from core client for 30 sec - exiting 13:13:11 (5108): No heartbeat from core client for 30 sec - exiting 13:13:12 (5108): No heartbeat from core client for 30 sec - exiting 13:13:13 (5108): No heartbeat from core client for 30 sec - exiting 13:13:14 (5108): No heartbeat from core client for 30 sec - exiting 13:13:15 (5108): No heartbeat from core client for 30 sec - exiting 13:13:16 (5108): No heartbeat from core client for 30 sec - exiting 13:13:17 (5108): No heartbeat from core client for 30 sec - exiting 13:13:18 (5108): No heartbeat from core client for 30 sec - exiting 13:13:19 (5108): No heartbeat from core client for 30 sec - exiting 13:13:21 (5108): No heartbeat from core client for 30 sec - exiting 13:13:22 (5108): No heartbeat from core client for 30 sec - exiting 13:13:23 (5108): No heartbeat from core client for 30 sec - exiting 13:13:24 (5108): No heartbeat from core client for 30 sec - exiting 13:13:25 (5108): No heartbeat from core client for 30 sec - exiting 13:13:26 (5108): No heartbeat from core client for 30 sec - exiting 13:13:27 (5108): No heartbeat from core client for 30 sec - exiting 13:13:28 (5108): No heartbeat from core client for 30 sec - exiting 13:13:29 (5108): No heartbeat from core client for 30 sec - exiting 13:13:31 (5108): No heartbeat from core client for 30 sec - exiting 13:13:32 (5108): No heartbeat from core client for 30 sec - exiting 13:13:33 (5108): No heartbeat from core client for 30 sec - exiting 13:13:34 (5108): No heartbeat from core client for 30 sec - exiting 13:13:35 (5108): No heartbeat from core client for 30 sec - exiting 13:13:36 (5108): No heartbeat from core client for 30 sec - exiting 13:13:37 (5108): No heartbeat from core client for 30 sec - exiting 13:13:38 (5108): No heartbeat from core client for 30 sec - exiting 13:13:39 (5108): No heartbeat from core client for 30 sec - exiting 13:13:40 (5108): No heartbeat from core client for 30 sec - exiting 13:13:41 (5108): No heartbeat from core client for 30 sec - exiting 13:13:43 (5108): No heartbeat from core client for 30 sec - exiting 13:13:44 (5108): No heartbeat from core client for 30 sec - exiting 13:13:45 (5108): No heartbeat from core client for 30 sec - exiting 13:13:46 (5108): No heartbeat from core client for 30 sec - exiting 13:13:47 (5108): No heartbeat from core client for 30 sec - exiting 13:13:48 (5108): No heartbeat from core client for 30 sec - exiting 13:13:49 (5108): No heartbeat from core client for 30 sec - exiting 13:13:50 (5108): No heartbeat from core client for 30 sec - exiting 13:13:51 (5108): No heartbeat from core client for 30 sec - exiting 13:13:52 (5108): No heartbeat from core client for 30 sec - exiting 13:13:53 (5108): No heartbeat from core client for 30 sec - exiting 13:13:54 (5108): No heartbeat from core client for 30 sec - exiting 13:13:55 (5108): No heartbeat from core client for 30 sec - exiting 13:13:56 (5108): No heartbeat from core client for 30 sec - exiting 13:13:57 (5108): No heartbeat from core client for 30 sec - exiting 13:13:58 (5108): No heartbeat from core client for 30 sec - exiting 13:13:59 (5108): No heartbeat from core client for 30 sec - exiting 13:14:00 (5108): No heartbeat from core client for 30 sec - exiting 13:14:01 (5108): No heartbeat from core client for 30 sec - exiting 13:14:02 (5108): No heartbeat from core client for 30 sec - exiting 13:14:03 (5108): No heartbeat from core client for 30 sec - exiting 13:14:04 (5108): No heartbeat from core client for 30 sec - exiting 13:14:05 (5108): No heartbeat from core client for 30 sec - exiting 13:14:07 (5108): No heartbeat from core client for 30 sec - exiting 13:14:08 (5108): No heartbeat from core client for 30 sec - exiting 13:14:09 (5108): No heartbeat from core client for 30 sec - exiting 13:14:10 (5108): No heartbeat from core client for 30 sec - exiting 13:14:11 (5108): No heartbeat from core client for 30 sec - exiting 13:14:12 (5108): No heartbeat from core client for 30 sec - exiting 13:14:13 (5108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5384, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3948, iMonCtr=2 Model crash detected, will try to restart... 10:32:41 (2400): No heartbeat from core client for 30 sec - exiting 10:32:42 (2400): No heartbeat from core client for 30 sec - exiting 10:32:44 (2400): No heartbeat from core client for 30 sec - exiting 10:32:45 (2400): No heartbeat from core client for 30 sec - exiting 10:32:46 (2400): No heartbeat from core client for 30 sec - exiting 10:32:47 (2400): No heartbeat from core client for 30 sec - exiting 10:32:48 (2400): No heartbeat from core client for 30 sec - exiting 10:32:49 (2400): No heartbeat from core client for 30 sec - exiting 10:32:50 (2400): No heartbeat from core client for 30 sec - exiting 10:32:51 (2400): No heartbeat from core client for 30 sec - exiting 10:32:52 (2400): No heartbeat from core client for 30 sec - exiting 10:32:53 (2400): No heartbeat from core client for 30 sec - exiting 10:32:55 (2400): No heartbeat from core client for 30 sec - exiting 10:32:56 (2400): No heartbeat from core client for 30 sec - exiting 10:32:57 (2400): No heartbeat from core client for 30 sec - exiting 10:32:58 (2400): No heartbeat from core client for 30 sec - exiting 10:32:59 (2400): No heartbeat from core client for 30 sec - exiting 10:33:00 (2400): No heartbeat from core client for 30 sec - exiting 10:33:01 (2400): No heartbeat from core client for 30 sec - exiting 10:33:02 (2400): No heartbeat from core client for 30 sec - exiting 10:33:03 (2400): No heartbeat from core client for 30 sec - exiting 10:33:04 (2400): No heartbeat from core client for 30 sec - exiting 10:33:05 (2400): No heartbeat from core client for 30 sec - exiting 10:33:06 (2400): No heartbeat from core client for 30 sec - exiting 10:33:07 (2400): No heartbeat from core client for 30 sec - exiting 10:33:08 (2400): No heartbeat from core client for 30 sec - exiting 10:33:09 (2400): No heartbeat from core client for 30 sec - exiting 10:33:10 (2400): No heartbeat from core client for 30 sec - exiting 10:33:11 (2400): No heartbeat from core client for 30 sec - exiting 10:33:12 (2400): No heartbeat from core client for 30 sec - exiting 10:33:13 (2400): No heartbeat from core client for 30 sec - exiting 10:33:14 (2400): No heartbeat from core client for 30 sec - exiting 10:33:15 (2400): No heartbeat from core client for 30 sec - exiting 10:33:16 (2400): No heartbeat from core client for 30 sec - exiting 10:33:17 (2400): No heartbeat from core client for 30 sec - exiting 10:33:18 (2400): No heartbeat from core client for 30 sec - exiting 10:33:19 (2400): No heartbeat from core client for 30 sec - exiting 10:33:20 (2400): No heartbeat from core client for 30 sec - exiting 10:33:21 (2400): No heartbeat from core client for 30 sec - exiting 10:33:22 (2400): No heartbeat from core client for 30 sec - exiting 10:33:23 (2400): No heartbeat from core client for 30 sec - exiting 10:33:24 (2400): No heartbeat from core client for 30 sec - exiting 10:33:25 (2400): No heartbeat from core client for 30 sec - exiting 10:33:26 (2400): No heartbeat from core client for 30 sec - exiting 10:33:27 (2400): No heartbeat from core client for 30 sec - exiting 10:33:28 (2400): No heartbeat from core client for 30 sec - exiting 10:33:29 (2400): No heartbeat from core client for 30 sec - exiting 10:33:30 (2400): No heartbeat from core client for 30 sec - exiting 10:33:31 (2400): No heartbeat from core client for 30 sec - exiting 10:33:32 (2400): No heartbeat from core client for 30 sec - exiting 10:33:33 (2400): No heartbeat from core client for 30 sec - exiting 10:33:34 (2400): No heartbeat from core client for 30 sec - exiting 10:33:35 (2400): No heartbeat from core client for 30 sec - exiting 10:33:36 (2400): No heartbeat from core client for 30 sec - exiting 10:33:37 (2400): No heartbeat from core client for 30 sec - exiting 10:33:38 (2400): No heartbeat from core client for 30 sec - exiting 10:33:39 (2400): No heartbeat from core client for 30 sec - exiting 10:33:40 (2400): No heartbeat from core client for 30 sec - exiting 10:33:41 (2400): No heartbeat from core client for 30 sec - exiting 10:33:42 (2400): No heartbeat from core client for 30 sec - exiting 10:33:43 (2400): No heartbeat from core client for 30 sec - exiting 10:33:44 (2400): No heartbeat from core client for 30 sec - exiting 10:33:45 (2400): No heartbeat from core client for 30 sec - exiting 10:33:46 (2400): No heartbeat from core client for 30 sec - exiting 10:33:47 (2400): No heartbeat from core client for 30 sec - exiting 10:33:48 (2400): No heartbeat from core client for 30 sec - exiting 10:33:49 (2400): No heartbeat from core client for 30 sec - exiting 10:33:50 (2400): No heartbeat from core client for 30 sec - exiting 10:33:51 (2400): No heartbeat from core client for 30 sec - exiting 10:33:52 (2400): No heartbeat from core client for 30 sec - exiting 10:33:53 (2400): No heartbeat from core client for 30 sec - exiting 10:33:54 (2400): No heartbeat from core client for 30 sec - exiting 10:33:55 (2400): No heartbeat from core client for 30 sec - exiting 10:33:56 (2400): No heartbeat from core client for 30 sec - exiting 10:33:57 (2400): No heartbeat from core client for 30 sec - exiting 10:33:58 (2400): No heartbeat from core client for 30 sec - exiting 10:33:59 (2400): No heartbeat from core client for 30 sec - exiting 10:34:00 (2400): No heartbeat from core client for 30 sec - exiting 10:34:01 (2400): No heartbeat from core client for 30 sec - exiting 10:34:02 (2400): No heartbeat from core client for 30 sec - exiting 10:34:03 (2400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6772, selfPID=7080, iMonCtr=1 Model crash detected, will try to restart... 11:14:06 (2468): No heartbeat from core client for 30 sec - exiting 11:14:07 (2468): No heartbeat from core client for 30 sec - exiting 11:14:08 (2468): No heartbeat from core client for 30 sec - exiting 11:14:09 (2468): No heartbeat from core client for 30 sec - exiting 11:14:10 (2468): No heartbeat from core client for 30 sec - exiting 11:14:11 (2468): No heartbeat from core client for 30 sec - exiting 11:14:12 (2468): No heartbeat from core client for 30 sec - exiting 11:14:13 (2468): No heartbeat from core client for 30 sec - exiting 11:14:14 (2468): No heartbeat from core client for 30 sec - exiting 11:14:15 (2468): No heartbeat from core client for 30 sec - exiting 11:14:17 (2468): No heartbeat from core client for 30 sec - exiting 11:14:18 (2468): No heartbeat from core client for 30 sec - exiting 11:14:19 (2468): No heartbeat from core client for 30 sec - exiting 11:14:20 (2468): No heartbeat from core client for 30 sec - exiting 11:14:21 (2468): No heartbeat from core client for 30 sec - exiting 11:14:22 (2468): No heartbeat from core client for 30 sec - exiting 11:14:23 (2468): No heartbeat from core client for 30 sec - exiting 11:14:24 (2468): No heartbeat from core client for 30 sec - exiting 11:14:25 (2468): No heartbeat from core client for 30 sec - exiting 11:14:26 (2468): No heartbeat from core client for 30 sec - exiting 11:14:27 (2468): No heartbeat from core client for 30 sec - exiting 11:14:29 (2468): No heartbeat from core client for 30 sec - exiting 11:14:30 (2468): No heartbeat from core client for 30 sec - exiting 11:14:31 (2468): No heartbeat from core client for 30 sec - exiting 11:14:32 (2468): No heartbeat from core client for 30 sec - exiting 11:14:33 (2468): No heartbeat from core client for 30 sec - exiting 11:14:34 (2468): No heartbeat from core client for 30 sec - exiting 11:14:35 (2468): No heartbeat from core client for 30 sec - exiting 11:14:36 (2468): No heartbeat from core client for 30 sec - exiting 11:14:38 (2468): No heartbeat from core client for 30 sec - exiting 11:14:39 (2468): No heartbeat from core client for 30 sec - exiting 11:14:40 (2468): No heartbeat from core client for 30 sec - exiting 11:14:41 (2468): No heartbeat from core client for 30 sec - exiting 11:14:42 (2468): No heartbeat from core client for 30 sec - exiting 11:14:43 (2468): No heartbeat from core client for 30 sec - exiting 11:14:44 (2468): No heartbeat from core client for 30 sec - exiting 11:14:45 (2468): No heartbeat from core client for 30 sec - exiting 11:14:46 (2468): No heartbeat from core client for 30 sec - exiting 11:14:47 (2468): No heartbeat from core client for 30 sec - exiting 11:14:48 (2468): No heartbeat from core client for 30 sec - exiting 11:14:50 (2468): No heartbeat from core client for 30 sec - exiting 11:14:51 (2468): No heartbeat from core client for 30 sec - exiting 11:14:52 (2468): No heartbeat from core client for 30 sec - exiting 11:14:53 (2468): No heartbeat from core client for 30 sec - exiting 11:14:54 (2468): No heartbeat from core client for 30 sec - exiting 11:14:55 (2468): No heartbeat from core client for 30 sec - exiting 11:14:56 (2468): No heartbeat from core client for 30 sec - exiting 11:14:57 (2468): No heartbeat from core client for 30 sec - exiting 11:14:58 (2468): No heartbeat from core client for 30 sec - exiting 11:14:59 (2468): No heartbeat from core client for 30 sec - exiting 11:15:00 (2468): No heartbeat from core client for 30 sec - exiting 11:15:02 (2468): No heartbeat from core client for 30 sec - exiting 11:15:03 (2468): No heartbeat from core client for 30 sec - exiting 11:15:04 (2468): No heartbeat from core client for 30 sec - exiting 11:15:05 (2468): No heartbeat from core client for 30 sec - exiting 11:15:06 (2468): No heartbeat from core client for 30 sec - exiting 11:15:07 (2468): No heartbeat from core client for 30 sec - exiting 11:15:08 (2468): No heartbeat from core client for 30 sec - exiting 11:15:09 (2468): No heartbeat from core client for 30 sec - exiting 11:15:10 (2468): No heartbeat from core client for 30 sec - exiting 11:15:11 (2468): No heartbeat from core client for 30 sec - exiting 11:15:13 (2468): No heartbeat from core client for 30 sec - exiting 11:15:14 (2468): No heartbeat from core client for 30 sec - exiting 11:15:15 (2468): No heartbeat from core client for 30 sec - exiting 11:15:16 (2468): No heartbeat from core client for 30 sec - exiting 11:15:17 (2468): No heartbeat from core client for 30 sec - exiting 11:15:18 (2468): No heartbeat from core client for 30 sec - exiting 11:15:19 (2468): No heartbeat from core client for 30 sec - exiting 11:16:13 (2468): No heartbeat from core client for 30 sec - exiting 11:16:14 (2468): No heartbeat from core client for 30 sec - exiting 11:16:15 (2468): No heartbeat from core client for 30 sec - exiting 11:16:17 (2468): No heartbeat from core client for 30 sec - exiting 11:16:18 (2468): No heartbeat from core client for 30 sec - exiting 11:16:19 (2468): No heartbeat from core client for 30 sec - exiting 11:16:20 (2468): No heartbeat from core client for 30 sec - exiting 11:16:21 (2468): No heartbeat from core client for 30 sec - exiting 11:16:22 (2468): No heartbeat from core client for 30 sec - exiting 11:16:23 (2468): No heartbeat from core client for 30 sec - exiting 11:16:24 (2468): No heartbeat from core client for 30 sec - exiting 11:16:25 (2468): No heartbeat from core client for 30 sec - exiting 11:16:26 (2468): No heartbeat from core client for 30 sec - exiting 11:16:27 (2468): No heartbeat from core client for 30 sec - exiting 11:16:29 (2468): No heartbeat from core client for 30 sec - exiting 11:16:30 (2468): No heartbeat from core client for 30 sec - exiting 11:16:31 (2468): No heartbeat from core client for 30 sec - exiting 11:16:32 (2468): No heartbeat from core client for 30 sec - exiting 11:16:33 (2468): No heartbeat from core client for 30 sec - exiting 11:16:34 (2468): No heartbeat from core client for 30 sec - exiting 11:16:35 (2468): No heartbeat from core client for 30 sec - exiting 11:16:36 (2468): No heartbeat from core client for 30 sec - exiting 11:16:37 (2468): No heartbeat from core client for 30 sec - exiting 11:16:38 (2468): No heartbeat from core client for 30 sec - exiting 11:16:40 (2468): No heartbeat from core client for 30 sec - exiting 11:16:41 (2468): No heartbeat from core client for 30 sec - exiting 11:16:42 (2468): No heartbeat from core client for 30 sec - exiting 11:16:43 (2468): No heartbeat from core client for 30 sec - exiting 11:16:44 (2468): No heartbeat from core client for 30 sec - exiting 11:16:45 (2468): No heartbeat from core client for 30 sec - exiting 11:16:46 (2468): No heartbeat from core client for 30 sec - exiting 11:16:47 (2468): No heartbeat from core client for 30 sec - exiting 11:16:48 (2468): No heartbeat from core client for 30 sec - exiting 11:16:49 (2468): No heartbeat from core client for 30 sec - exiting 11:16:50 (2468): No heartbeat from core client for 30 sec - exiting 11:16:52 (2468): No heartbeat from core client for 30 sec - exiting 11:16:53 (2468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2664, selfPID=3536, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 19:03:56 (4836): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:12:17 (4572): No heartbeat from core client for 30 sec - exiting 10:12:18 (4572): No heartbeat from core client for 30 sec - exiting 10:12:19 (4572): No heartbeat from core client for 30 sec - exiting 10:12:20 (4572): No heartbeat from core client for 30 sec - exiting 10:12:21 (4572): No heartbeat from core client for 30 sec - exiting 10:12:22 (4572): No heartbeat from core client for 30 sec - exiting 10:12:23 (4572): No heartbeat from core client for 30 sec - exiting 10:12:24 (4572): No heartbeat from core client for 30 sec - exiting 10:12:25 (4572): No heartbeat from core client for 30 sec - exiting 10:12:26 (4572): No heartbeat from core client for 30 sec - exiting 10:12:27 (4572): No heartbeat from core client for 30 sec - exiting 10:12:28 (4572): No heartbeat from core client for 30 sec - exiting 10:12:29 (4572): No heartbeat from core client for 30 sec - exiting 10:12:30 (4572): No heartbeat from core client for 30 sec - exiting 10:12:31 (4572): No heartbeat from core client for 30 sec - exiting 10:12:32 (4572): No heartbeat from core client for 30 sec - exiting 10:12:33 (4572): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5280, selfPID=5524, iMonCtr=1 Model crash detected, will try to restart... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1632, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4936, selfPID=4568, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 09:39:04 (5856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2696, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1872, selfPID=1872, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:26:09 (928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3056, selfPID=3056, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Atmos Restart file copy failed on atmos_restart.day CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3448, selfPID=4660, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1264, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6320, selfPID=2980, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
27 Jan 2014 21:23:57	1263465	16118377	hadam3p_eu_izl5_1999_1_008491310_0	138,336	511,268	3.6958
24 Jan 2014 22:00:01	1263465	16118377	hadam3p_eu_izl5_1999_1_008491310_0	126,816	468,299	3.6927
10 Jan 2014 17:05:01	1263465	16118377	hadam3p_eu_izl5_1999_1_008491310_0	115,296	425,901	3.6940
05 Jan 2014 18:59:29	1263465	16118377	hadam3p_eu_izl5_1999_1_008491310_0	103,776	383,378	3.6943
03 Jan 2014 21:46:43	1263465	16118377	hadam3p_eu_izl5_1999_1_008491310_0	92,256	341,207	3.6985
29 Dec 2013 20:59:10	1263465	16118377	hadam3p_eu_izl5_1999_1_008491310_0	80,736	299,360	3.7079
27 Dec 2013 16:25:46	1263465	16118377	hadam3p_eu_izl5_1999_1_008491310_0	69,220	257,450	3.7193
27 Dec 2013 15:22:48	1263465	16118377	hadam3p_eu_izl5_1999_1_008491310_0	69,216	256,888	3.7114
24 Dec 2013 18:01:19	1263465	16118377	hadam3p_eu_izl5_1999_1_008491310_0	57,696	214,285	3.7140
21 Dec 2013 18:20:55	1263465	16118377	hadam3p_eu_izl5_1999_1_008491310_0	46,176	171,962	3.7241
16 Dec 2013 21:17:43	1263465	16118377	hadam3p_eu_izl5_1999_1_008491310_0	34,656	129,347	3.7323
09 Dec 2013 17:40:17	1263465	16118377	hadam3p_eu_izl5_1999_1_008491310_0	23,142	86,382	3.7327
08 Dec 2013 21:59:52	1263465	16118377	hadam3p_eu_izl5_1999_1_008491310_0	23,136	85,801	3.7085
07 Dec 2013 18:12:37	1263465	16118377	hadam3p_eu_izl5_1999_1_008491310_0	11,616	43,230	3.7216