climateprediction.net home page
Task 15527027

Task 15527027

Name hadam3p_pnw_dhqp_2042_1_008279357_1
Workunit 8430492
Created 9 Jan 2013, 12:01:45 UTC
Sent 9 Jan 2013, 12:01:58 UTC
Report deadline 22 Dec 2013, 17:21:58 UTC
Received 7 Mar 2013, 9:43:22 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1245167
Run time 4 days 21 hours 41 min 37 sec
CPU time 4 days 17 hours 11 min 9 sec
Validate state Workunit error - check skipped
Credit 3,005.88
Device peak FLOPS 2.70 GFLOPS
Application version UK Met Office HadAM3P-HadRM3P Pacific North West v6.09
windows_intelx86
Stderr
<core_client_version>7.0.28</core_client_version>
<![CDATA[
<stderr_txt>
Atmos Restart file copy failed on atmos_restart.day
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1604, selfPID=4688, iMonCtr=1
Model crash detected, will try to restart...
Leaving CPDN_Main::Monitor...
Regional yearly means requires 12 input files got 1
08:58:04 (4848): No heartbeat from core client for 30 sec - exiting
08:58:06 (4848): No heartbeat from core client for 30 sec - exiting
08:58:07 (4848): No heartbeat from core client for 30 sec - exiting
08:58:08 (4848): No heartbeat from core client for 30 sec - exiting
08:58:09 (4848): No heartbeat from core client for 30 sec - exiting
08:58:10 (4848): No heartbeat from core client for 30 sec - exiting
08:58:11 (4848): No heartbeat from core client for 30 sec - exiting
08:58:12 (4848): No heartbeat from core client for 30 sec - exiting
08:58:13 (4848): No heartbeat from core client for 30 sec - exiting
08:58:14 (4848): No heartbeat from core client for 30 sec - exiting
08:58:15 (4848): No heartbeat from core client for 30 sec - exiting
08:58:16 (4848): No heartbeat from core client for 30 sec - exiting
08:58:18 (4848): No heartbeat from core client for 30 sec - exiting
08:58:19 (4848): No heartbeat from core client for 30 sec - exiting
08:58:20 (4848): No heartbeat from core client for 30 sec - exiting
08:58:21 (4848): No heartbeat from core client for 30 sec - exiting
08:58:22 (4848): No heartbeat from core client for 30 sec - exiting
08:58:23 (4848): No heartbeat from core client for 30 sec - exiting
08:58:24 (4848): No heartbeat from core client for 30 sec - exiting
08:58:25 (4848): No heartbeat from core client for 30 sec - exiting
08:58:26 (4848): No heartbeat from core client for 30 sec - exiting
08:58:27 (4848): No heartbeat from core client for 30 sec - exiting
08:58:28 (4848): No heartbeat from core client for 30 sec - exiting
08:58:30 (4848): No heartbeat from core client for 30 sec - exiting
08:58:31 (4848): No heartbeat from core client for 30 sec - exiting
08:58:32 (4848): No heartbeat from core client for 30 sec - exiting
08:58:33 (4848): No heartbeat from core client for 30 sec - exiting
08:58:34 (4848): No heartbeat from core client for 30 sec - exiting
08:58:35 (4848): No heartbeat from core client for 30 sec - exiting
08:58:36 (4848): No heartbeat from core client for 30 sec - exiting
08:58:37 (4848): No heartbeat from core client for 30 sec - exiting
08:58:38 (4848): No heartbeat from core client for 30 sec - exiting
08:58:39 (4848): No heartbeat from core client for 30 sec - exiting
08:58:41 (4848): No heartbeat from core client for 30 sec - exiting
08:58:42 (4848): No heartbeat from core client for 30 sec - exiting
08:58:43 (4848): No heartbeat from core client for 30 sec - exiting
08:58:44 (4848): No heartbeat from core client for 30 sec - exiting
08:58:45 (4848): No heartbeat from core client for 30 sec - exiting
08:58:46 (4848): No heartbeat from core client for 30 sec - exiting
08:58:47 (4848): No heartbeat from core client for 30 sec - exiting
08:58:48 (4848): No heartbeat from core client for 30 sec - exiting
08:58:49 (4848): No heartbeat from core client for 30 sec - exiting
08:58:50 (4848): No heartbeat from core client for 30 sec - exiting
08:58:51 (4848): No heartbeat from core client for 30 sec - exiting
08:58:53 (4848): No heartbeat from core client for 30 sec - exiting
08:58:54 (4848): No heartbeat from core client for 30 sec - exiting
08:58:55 (4848): No heartbeat from core client for 30 sec - exiting
08:58:56 (4848): No heartbeat from core client for 30 sec - exiting
08:58:57 (4848): No heartbeat from core client for 30 sec - exiting
08:58:58 (4848): No heartbeat from core client for 30 sec - exiting
08:58:59 (4848): No heartbeat from core client for 30 sec - exiting
08:59:00 (4848): No heartbeat from core client for 30 sec - exiting
08:59:01 (4848): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4836, selfPID=4664, iMonCtr=1
Model crash detected, will try to restart...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
08:40:13 (4300): No heartbeat from core client for 30 sec - exiting
08:40:14 (4300): No heartbeat from core client for 30 sec - exiting
08:40:15 (4300): No heartbeat from core client for 30 sec - exiting
08:40:16 (4300): No heartbeat from core client for 30 sec - exiting
08:40:17 (4300): No heartbeat from core client for 30 sec - exiting
08:40:18 (4300): No heartbeat from core client for 30 sec - exiting
08:40:20 (4300): No heartbeat from core client for 30 sec - exiting
08:40:21 (4300): No heartbeat from core client for 30 sec - exiting
08:40:22 (4300): No heartbeat from core client for 30 sec - exiting
08:40:23 (4300): No heartbeat from core client for 30 sec - exiting
08:40:24 (4300): No heartbeat from core client for 30 sec - exiting
08:40:25 (4300): No heartbeat from core client for 30 sec - exiting
08:40:26 (4300): No heartbeat from core client for 30 sec - exiting
08:40:27 (4300): No heartbeat from core client for 30 sec - exiting
08:40:28 (4300): No heartbeat from core client for 30 sec - exiting
08:40:29 (4300): No heartbeat from core client for 30 sec - exiting
08:40:31 (4300): No heartbeat from core client for 30 sec - exiting
08:40:32 (4300): No heartbeat from core client for 30 sec - exiting
08:40:33 (4300): No heartbeat from core client for 30 sec - exiting
08:40:34 (4300): No heartbeat from core client for 30 sec - exiting
08:40:35 (4300): No heartbeat from core client for 30 sec - exiting
08:40:36 (4300): No heartbeat from core client for 30 sec - exiting
08:40:37 (4300): No heartbeat from core client for 30 sec - exiting
08:40:38 (4300): No heartbeat from core client for 30 sec - exiting
08:40:39 (4300): No heartbeat from core client for 30 sec - exiting
08:40:40 (4300): No heartbeat from core client for 30 sec - exiting
08:40:41 (4300): No heartbeat from core client for 30 sec - exiting
08:40:43 (4300): No heartbeat from core client for 30 sec - exiting
08:40:44 (4300): No heartbeat from core client for 30 sec - exiting
08:40:45 (4300): No heartbeat from core client for 30 sec - exiting
08:40:46 (4300): No heartbeat from core client for 30 sec - exiting
08:40:47 (4300): No heartbeat from core client for 30 sec - exiting
08:40:48 (4300): No heartbeat from core client for 30 sec - exiting
08:40:49 (4300): No heartbeat from core client for 30 sec - exiting
08:40:50 (4300): No heartbeat from core client for 30 sec - exiting
08:40:51 (4300): No heartbeat from core client for 30 sec - exiting
08:40:52 (4300): No heartbeat from core client for 30 sec - exiting
08:40:53 (4300): No heartbeat from core client for 30 sec - exiting
08:40:55 (4300): No heartbeat from core client for 30 sec - exiting
08:40:56 (4300): No heartbeat from core client for 30 sec - exiting
08:40:57 (4300): No heartbeat from core client for 30 sec - exiting
08:40:58 (4300): No heartbeat from core client for 30 sec - exiting
08:40:59 (4300): No heartbeat from core client for 30 sec - exiting
08:41:00 (4300): No heartbeat from core client for 30 sec - exiting
08:41:02 (4300): No heartbeat from core client for 30 sec - exiting
08:41:03 (4300): No heartbeat from core client for 30 sec - exiting
08:41:04 (4300): No heartbeat from core client for 30 sec - exiting
08:41:05 (4300): No heartbeat from core client for 30 sec - exiting
08:41:06 (4300): No heartbeat from core client for 30 sec - exiting
08:41:07 (4300): No heartbeat from core client for 30 sec - exiting
08:41:08 (4300): No heartbeat from core client for 30 sec - exiting
08:41:09 (4300): No heartbeat from core client for 30 sec - exiting
08:41:10 (4300): No heartbeat from core client for 30 sec - exiting
08:41:11 (4300): No heartbeat from core client for 30 sec - exiting
08:41:12 (4300): No heartbeat from core client for 30 sec - exiting
08:41:14 (4300): No heartbeat from core client for 30 sec - exiting
08:41:15 (4300): No heartbeat from core client for 30 sec - exiting
08:41:16 (4300): No heartbeat from core client for 30 sec - exiting
08:41:17 (4300): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:41:18 (4300): No heartbeat from core client for 30 sec - exiting
Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=864, iMonCtr=2
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
08:26:54 (4768): No heartbeat from core client for 30 sec - exiting
08:26:55 (4768): No heartbeat from core client for 30 sec - exiting
08:26:56 (4768): No heartbeat from core client for 30 sec - exiting
08:26:57 (4768): No heartbeat from core client for 30 sec - exiting
08:26:58 (4768): No heartbeat from core client for 30 sec - exiting
08:26:59 (4768): No heartbeat from core client for 30 sec - exiting
08:27:00 (4768): No heartbeat from core client for 30 sec - exiting
08:27:01 (4768): No heartbeat from core client for 30 sec - exiting
08:27:03 (4768): No heartbeat from core client for 30 sec - exiting
08:27:04 (4768): No heartbeat from core client for 30 sec - exiting
08:27:05 (4768): No heartbeat from core client for 30 sec - exiting
08:27:06 (4768): No heartbeat from core client for 30 sec - exiting
08:27:07 (4768): No heartbeat from core client for 30 sec - exiting
08:27:08 (4768): No heartbeat from core client for 30 sec - exiting
08:27:09 (4768): No heartbeat from core client for 30 sec - exiting
08:27:10 (4768): No heartbeat from core client for 30 sec - exiting
08:27:11 (4768): No heartbeat from core client for 30 sec - exiting
08:27:12 (4768): No heartbeat from core client for 30 sec - exiting
08:27:13 (4768): No heartbeat from core client for 30 sec - exiting
08:27:15 (4768): No heartbeat from core client for 30 sec - exiting
08:27:16 (4768): No heartbeat from core client for 30 sec - exiting
08:27:17 (4768): No heartbeat from core client for 30 sec - exiting
08:27:18 (4768): No heartbeat from core client for 30 sec - exiting
08:27:19 (4768): No heartbeat from core client for 30 sec - exiting
08:27:20 (4768): No heartbeat from core client for 30 sec - exiting
08:27:21 (4768): No heartbeat from core client for 30 sec - exiting
08:27:22 (4768): No heartbeat from core client for 30 sec - exiting
08:27:23 (4768): No heartbeat from core client for 30 sec - exiting
08:27:24 (4768): No heartbeat from core client for 30 sec - exiting
08:27:25 (4768): No heartbeat from core client for 30 sec - exiting
08:27:27 (4768): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
08:19:43 (5412): No heartbeat from core client for 30 sec - exiting
08:19:44 (5412): No heartbeat from core client for 30 sec - exiting
08:19:45 (5412): No heartbeat from core client for 30 sec - exiting
08:19:46 (5412): No heartbeat from core client for 30 sec - exiting
08:19:47 (5412): No heartbeat from core client for 30 sec - exiting
08:19:48 (5412): No heartbeat from core client for 30 sec - exiting
08:19:49 (5412): No heartbeat from core client for 30 sec - exiting
08:19:50 (5412): No heartbeat from core client for 30 sec - exiting
08:19:52 (5412): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1180, selfPID=4612, iMonCtr=1
Model crash detected, will try to restart...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1684, selfPID=4976, iMonCtr=1
Model crash detected, will try to restart...
Leaving CPDN_Main::Monitor...
Regional yearly means requires 12 input files got 6
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5876, selfPID=4220, iMonCtr=1
Model crash detected, will try to restart...
Leaving CPDN_Main::Monitor...
Regional yearly means requires 12 input files got 8
08:54:10 (4840): No heartbeat from core client for 30 sec - exiting
08:54:11 (4840): No heartbeat from core client for 30 sec - exiting
08:54:12 (4840): No heartbeat from core client for 30 sec - exiting
08:54:13 (4840): No heartbeat from core client for 30 sec - exiting
08:54:14 (4840): No heartbeat from core client for 30 sec - exiting
08:54:15 (4840): No heartbeat from core client for 30 sec - exiting
08:54:16 (4840): No heartbeat from core client for 30 sec - exiting
08:54:17 (4840): No heartbeat from core client for 30 sec - exiting
08:54:19 (4840): No heartbeat from core client for 30 sec - exiting
08:54:20 (4840): No heartbeat from core client for 30 sec - exiting
08:54:21 (4840): No heartbeat from core client for 30 sec - exiting
08:54:22 (4840): No heartbeat from core client for 30 sec - exiting
08:54:23 (4840): No heartbeat from core client for 30 sec - exiting
08:54:24 (4840): No heartbeat from core client for 30 sec - exiting
08:54:25 (4840): No heartbeat from core client for 30 sec - exiting
08:54:26 (4840): No heartbeat from core client for 30 sec - exiting
08:54:27 (4840): No heartbeat from core client for 30 sec - exiting
08:54:28 (4840): No heartbeat from core client for 30 sec - exiting
08:54:29 (4840): No heartbeat from core client for 30 sec - exiting
08:54:31 (4840): No heartbeat from core client for 30 sec - exiting
08:54:32 (4840): No heartbeat from core client for 30 sec - exiting
08:54:33 (4840): No heartbeat from core client for 30 sec - exiting
08:54:34 (4840): No heartbeat from core client for 30 sec - exiting
08:54:35 (4840): No heartbeat from core client for 30 sec - exiting
08:54:36 (4840): No heartbeat from core client for 30 sec - exiting
08:54:37 (4840): No heartbeat from core client for 30 sec - exiting
08:54:38 (4840): No heartbeat from core client for 30 sec - exiting
08:54:39 (4840): No heartbeat from core client for 30 sec - exiting
08:54:40 (4840): No heartbeat from core client for 30 sec - exiting
08:54:41 (4840): No heartbeat from core client for 30 sec - exiting
08:54:43 (4840): No heartbeat from core client for 30 sec - exiting
08:54:44 (4840): No heartbeat from core client for 30 sec - exiting
08:54:45 (4840): No heartbeat from core client for 30 sec - exiting
08:54:46 (4840): No heartbeat from core client for 30 sec - exiting
08:54:47 (4840): No heartbeat from core client for 30 sec - exiting
08:54:48 (4840): No heartbeat from core client for 30 sec - exiting
08:54:49 (4840): No heartbeat from core client for 30 sec - exiting
08:54:50 (4840): No heartbeat from core client for 30 sec - exiting
08:54:51 (4840): No heartbeat from core client for 30 sec - exiting
08:54:52 (4840): No heartbeat from core client for 30 sec - exiting
08:54:53 (4840): No heartbeat from core client for 30 sec - exiting
08:54:55 (4840): No heartbeat from core client for 30 sec - exiting
08:54:56 (4840): No heartbeat from core client for 30 sec - exiting
08:54:57 (4840): No heartbeat from core client for 30 sec - exiting
08:54:58 (4840): No heartbeat from core client for 30 sec - exiting
08:54:59 (4840): No heartbeat from core client for 30 sec - exiting
08:55:00 (4840): No heartbeat from core client for 30 sec - exiting
08:55:01 (4840): No heartbeat from core client for 30 sec - exiting
08:55:02 (4840): No heartbeat from core client for 30 sec - exiting
08:55:03 (4840): No heartbeat from core client for 30 sec - exiting
08:55:04 (4840): No heartbeat from core client for 30 sec - exiting
08:55:05 (4840): No heartbeat from core client for 30 sec - exiting
08:55:07 (4840): No heartbeat from core client for 30 sec - exiting
08:55:08 (4840): No heartbeat from core client for 30 sec - exiting
08:55:09 (4840): No heartbeat from core client for 30 sec - exiting
08:55:10 (4840): No heartbeat from core client for 30 sec - exiting
08:55:11 (4840): No heartbeat from core client for 30 sec - exiting
08:55:12 (4840): No heartbeat from core client for 30 sec - exiting
08:55:13 (4840): No heartbeat from core client for 30 sec - exiting
08:55:14 (4840): No heartbeat from core client for 30 sec - exiting
08:55:15 (4840): No heartbeat from core client for 30 sec - exiting
08:55:16 (4840): No heartbeat from core client for 30 sec - exiting
08:55:18 (4840): No heartbeat from core client for 30 sec - exiting
08:55:19 (4840): No heartbeat from core client for 30 sec - exiting
08:55:20 (4840): No heartbeat from core client for 30 sec - exiting
08:55:21 (4840): No heartbeat from core client for 30 sec - exiting
08:55:22 (4840): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
08:51:24 (4444): No heartbeat from core client for 30 sec - exiting
08:51:25 (4444): No heartbeat from core client for 30 sec - exiting
08:51:26 (4444): No heartbeat from core client for 30 sec - exiting
08:51:27 (4444): No heartbeat from core client for 30 sec - exiting
08:51:28 (4444): No heartbeat from core client for 30 sec - exiting
08:51:29 (4444): No heartbeat from core client for 30 sec - exiting
08:51:30 (4444): No heartbeat from core client for 30 sec - exiting
08:51:31 (4444): No heartbeat from core client for 30 sec - exiting
08:51:32 (4444): No heartbeat from core client for 30 sec - exiting
08:51:34 (4444): No heartbeat from core client for 30 sec - exiting
08:51:35 (4444): No heartbeat from core client for 30 sec - exiting
08:51:36 (4444): No heartbeat from core client for 30 sec - exiting
08:51:37 (4444): No heartbeat from core client for 30 sec - exiting
08:51:38 (4444): No heartbeat from core client for 30 sec - exiting
08:51:39 (4444): No heartbeat from core client for 30 sec - exiting
08:51:40 (4444): No heartbeat from core client for 30 sec - exiting
08:51:41 (4444): No heartbeat from core client for 30 sec - exiting
08:51:42 (4444): No heartbeat from core client for 30 sec - exiting
08:51:43 (4444): No heartbeat from core client for 30 sec - exiting
08:51:44 (4444): No heartbeat from core client for 30 sec - exiting
08:51:46 (4444): No heartbeat from core client for 30 sec - exiting
08:51:47 (4444): No heartbeat from core client for 30 sec - exiting
08:51:48 (4444): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4524, selfPID=4228, iMonCtr=1
Model crash detected, will try to restart...
09:09:50 (5172): No heartbeat from core client for 30 sec - exiting
09:09:51 (5172): No heartbeat from core client for 30 sec - exiting
09:09:52 (5172): No heartbeat from core client for 30 sec - exiting
09:09:53 (5172): No heartbeat from core client for 30 sec - exiting
09:09:54 (5172): No heartbeat from core client for 30 sec - exiting
09:09:55 (5172): No heartbeat from core client for 30 sec - exiting
09:09:56 (5172): No heartbeat from core client for 30 sec - exiting
09:09:57 (5172): No heartbeat from core client for 30 sec - exiting
09:09:58 (5172): No heartbeat from core client for 30 sec - exiting
09:09:59 (5172): No heartbeat from core client for 30 sec - exiting
09:10:00 (5172): No heartbeat from core client for 30 sec - exiting
09:10:01 (5172): No heartbeat from core client for 30 sec - exiting
09:10:02 (5172): No heartbeat from core client for 30 sec - exiting
09:10:03 (5172): No heartbeat from core client for 30 sec - exiting
09:10:04 (5172): No heartbeat from core client for 30 sec - exiting
09:10:05 (5172): No heartbeat from core client for 30 sec - exiting
09:10:06 (5172): No heartbeat from core client for 30 sec - exiting
09:10:07 (5172): No heartbeat from core client for 30 sec - exiting
09:10:08 (5172): No heartbeat from core client for 30 sec - exiting
09:10:09 (5172): No heartbeat from core client for 30 sec - exiting
09:10:10 (5172): No heartbeat from core client for 30 sec - exiting
09:10:11 (5172): No heartbeat from core client for 30 sec - exiting
09:10:12 (5172): No heartbeat from core client for 30 sec - exiting
09:10:13 (5172): No heartbeat from core client for 30 sec - exiting
09:10:14 (5172): No heartbeat from core client for 30 sec - exiting
09:10:15 (5172): No heartbeat from core client for 30 sec - exiting
09:10:16 (5172): No heartbeat from core client for 30 sec - exiting
09:10:17 (5172): No heartbeat from core client for 30 sec - exiting
09:10:18 (5172): No heartbeat from core client for 30 sec - exiting
09:10:19 (5172): No heartbeat from core client for 30 sec - exiting
09:10:20 (5172): No heartbeat from core client for 30 sec - exiting
09:10:21 (5172): No heartbeat from core client for 30 sec - exiting
09:10:22 (5172): No heartbeat from core client for 30 sec - exiting
09:10:23 (5172): No heartbeat from core client for 30 sec - exiting
09:10:24 (5172): No heartbeat from core client for 30 sec - exiting
09:10:25 (5172): No heartbeat from core client for 30 sec - exiting
09:10:26 (5172): No heartbeat from core client for 30 sec - exiting
09:10:27 (5172): No heartbeat from core client for 30 sec - exiting
09:10:28 (5172): No heartbeat from core client for 30 sec - exiting
09:10:29 (5172): No heartbeat from core client for 30 sec - exiting
09:10:30 (5172): No heartbeat from core client for 30 sec - exiting
09:10:31 (5172): No heartbeat from core client for 30 sec - exiting
09:10:32 (5172): No heartbeat from core client for 30 sec - exiting
09:10:33 (5172): No heartbeat from core client for 30 sec - exiting
09:10:34 (5172): No heartbeat from core client for 30 sec - exiting
09:10:35 (5172): No heartbeat from core client for 30 sec - exiting
09:10:36 (5172): No heartbeat from core client for 30 sec - exiting
09:10:37 (5172): No heartbeat from core client for 30 sec - exiting
09:10:38 (5172): No heartbeat from core client for 30 sec - exiting
09:10:39 (5172): No heartbeat from core client for 30 sec - exiting
09:10:40 (5172): No heartbeat from core client for 30 sec - exiting
09:10:41 (5172): No heartbeat from core client for 30 sec - exiting
09:10:42 (5172): No heartbeat from core client for 30 sec - exiting
09:10:43 (5172): No heartbeat from core client for 30 sec - exiting
09:10:44 (5172): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
09:17:01 (2212): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2516, selfPID=2516, iMonCtr=2
08:48:46 (4792): No heartbeat from core client for 30 sec - exiting
08:48:47 (4792): No heartbeat from core client for 30 sec - exiting
08:48:48 (4792): No heartbeat from core client for 30 sec - exiting
08:48:50 (4792): No heartbeat from core client for 30 sec - exiting
08:48:51 (4792): No heartbeat from core client for 30 sec - exiting
08:48:52 (4792): No heartbeat from core client for 30 sec - exiting
08:48:53 (4792): No heartbeat from core client for 30 sec - exiting
08:48:54 (4792): No heartbeat from core client for 30 sec - exiting
08:48:55 (4792): No heartbeat from core client for 30 sec - exiting
08:48:56 (4792): No heartbeat from core client for 30 sec - exiting
08:48:57 (4792): No heartbeat from core client for 30 sec - exiting
08:48:58 (4792): No heartbeat from core client for 30 sec - exiting
08:48:59 (4792): No heartbeat from core client for 30 sec - exiting
08:49:00 (4792): No heartbeat from core client for 30 sec - exiting
08:49:02 (4792): No heartbeat from core client for 30 sec - exiting
08:49:03 (4792): No heartbeat from core client for 30 sec - exiting
08:49:04 (4792): No heartbeat from core client for 30 sec - exiting
08:49:05 (4792): No heartbeat from core client for 30 sec - exiting
08:49:06 (4792): No heartbeat from core client for 30 sec - exiting
08:49:07 (4792): No heartbeat from core client for 30 sec - exiting
08:49:08 (4792): No heartbeat from core client for 30 sec - exiting
08:49:09 (4792): No heartbeat from core client for 30 sec - exiting
08:49:10 (4792): No heartbeat from core client for 30 sec - exiting
08:49:11 (4792): No heartbeat from core client for 30 sec - exiting
08:49:12 (4792): No heartbeat from core client for 30 sec - exiting
08:49:14 (4792): No heartbeat from core client for 30 sec - exiting
08:49:15 (4792): No heartbeat from core client for 30 sec - exiting
08:49:16 (4792): No heartbeat from core client for 30 sec - exiting
08:49:17 (4792): No heartbeat from core client for 30 sec - exiting
08:49:18 (4792): No heartbeat from core client for 30 sec - exiting
08:49:19 (4792): No heartbeat from core client for 30 sec - exiting
08:49:20 (4792): No heartbeat from core client for 30 sec - exiting
08:49:21 (4792): No heartbeat from core client for 30 sec - exiting
08:49:22 (4792): No heartbeat from core client for 30 sec - exiting
08:49:23 (4792): No heartbeat from core client for 30 sec - exiting
08:49:24 (4792): No heartbeat from core client for 30 sec - exiting
08:49:26 (4792): No heartbeat from core client for 30 sec - exiting
08:49:27 (4792): No heartbeat from core client for 30 sec - exiting
08:49:28 (4792): No heartbeat from core client for 30 sec - exiting
08:49:29 (4792): No heartbeat from core client for 30 sec - exiting
08:49:30 (4792): No heartbeat from core client for 30 sec - exiting
08:49:31 (4792): No heartbeat from core client for 30 sec - exiting
08:49:32 (4792): No heartbeat from core client for 30 sec - exiting
08:49:33 (4792): No heartbeat from core client for 30 sec - exiting
08:49:34 (4792): No heartbeat from core client for 30 sec - exiting
08:49:35 (4792): No heartbeat from core client for 30 sec - exiting
08:49:36 (4792): No heartbeat from core client for 30 sec - exiting
08:49:38 (4792): No heartbeat from core client for 30 sec - exiting
08:49:39 (4792): No heartbeat from core client for 30 sec - exiting
08:49:40 (4792): No heartbeat from core client for 30 sec - exiting
08:49:41 (4792): No heartbeat from core client for 30 sec - exiting
08:49:42 (4792): No heartbeat from core client for 30 sec - exiting
08:49:43 (4792): No heartbeat from core client for 30 sec - exiting
08:49:44 (4792): No heartbeat from core client for 30 sec - exiting
08:49:45 (4792): No heartbeat from core client for 30 sec - exiting
08:49:46 (4792): No heartbeat from core client for 30 sec - exiting
08:49:47 (4792): No heartbeat from core client for 30 sec - exiting
08:49:48 (4792): No heartbeat from core client for 30 sec - exiting
08:49:50 (4792): No heartbeat from core client for 30 sec - exiting
08:49:51 (4792): No heartbeat from core client for 30 sec - exiting
08:49:52 (4792): No heartbeat from core client for 30 sec - exiting
08:49:53 (4792): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:49:54 (4792): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
08:42:03 (4532): No heartbeat from core client for 30 sec - exiting
08:42:04 (4532): No heartbeat from core client for 30 sec - exiting
08:42:05 (4532): No heartbeat from core client for 30 sec - exiting
08:42:06 (4532): No heartbeat from core client for 30 sec - exiting
08:42:07 (4532): No heartbeat from core client for 30 sec - exiting
08:42:09 (4532): No heartbeat from core client for 30 sec - exiting
08:42:10 (4532): No heartbeat from core client for 30 sec - exiting
08:42:11 (4532): No heartbeat from core client for 30 sec - exiting
08:42:12 (4532): No heartbeat from core client for 30 sec - exiting
08:42:13 (4532): No heartbeat from core client for 30 sec - exiting
08:42:14 (4532): No heartbeat from core client for 30 sec - exiting
08:42:15 (4532): No heartbeat from core client for 30 sec - exiting
08:42:16 (4532): No heartbeat from core client for 30 sec - exiting
08:42:17 (4532): No heartbeat from core client for 30 sec - exiting
08:42:18 (4532): No heartbeat from core client for 30 sec - exiting
08:42:19 (4532): No heartbeat from core client for 30 sec - exiting
08:42:21 (4532): No heartbeat from core client for 30 sec - exiting
08:42:22 (4532): No heartbeat from core client for 30 sec - exiting
08:42:23 (4532): No heartbeat from core client for 30 sec - exiting
08:42:24 (4532): No heartbeat from core client for 30 sec - exiting
08:42:25 (4532): No heartbeat from core client for 30 sec - exiting
08:42:26 (4532): No heartbeat from core client for 30 sec - exiting
08:42:27 (4532): No heartbeat from core client for 30 sec - exiting
08:42:28 (4532): No heartbeat from core client for 30 sec - exiting
08:42:29 (4532): No heartbeat from core client for 30 sec - exiting
08:42:30 (4532): No heartbeat from core client for 30 sec - exiting
08:42:32 (4532): No heartbeat from core client for 30 sec - exiting
08:42:33 (4532): No heartbeat from core client for 30 sec - exiting
08:42:34 (4532): No heartbeat from core client for 30 sec - exiting
08:42:35 (4532): No heartbeat from core client for 30 sec - exiting
08:42:36 (4532): No heartbeat from core client for 30 sec - exiting
08:42:37 (4532): No heartbeat from core client for 30 sec - exiting
08:42:38 (4532): No heartbeat from core client for 30 sec - exiting
08:42:39 (4532): No heartbeat from core client for 30 sec - exiting
08:42:40 (4532): No heartbeat from core client for 30 sec - exiting
08:42:41 (4532): No heartbeat from core client for 30 sec - exiting
08:42:42 (4532): No heartbeat from core client for 30 sec - exiting
08:42:44 (4532): No heartbeat from core client for 30 sec - exiting
08:42:45 (4532): No heartbeat from core client for 30 sec - exiting
08:42:46 (4532): No heartbeat from core client for 30 sec - exiting
08:42:47 (4532): No heartbeat from core client for 30 sec - exiting
08:42:48 (4532): No heartbeat from core client for 30 sec - exiting
08:42:49 (4532): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:42:50 (4532): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
11:15:18 (3968): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
16:03:10 (3520): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Leaving CPDN_Main::Monitor...
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
07 Mar 2013 09:46:10 1245167 15527027 hadam3p_pnw_dhqp_2042_1_008279357_1 138,336 406,753 2.9403
26 Feb 2013 10:30:51 1245167 15527027 hadam3p_pnw_dhqp_2042_1_008279357_1 126,816 372,987 2.9412
18 Feb 2013 11:33:54 1245167 15527027 hadam3p_pnw_dhqp_2042_1_008279357_1 115,296 339,574 2.9452
14 Feb 2013 11:01:10 1245167 15527027 hadam3p_pnw_dhqp_2042_1_008279357_1 103,776 305,777 2.9465
12 Feb 2013 15:16:10 1245167 15527027 hadam3p_pnw_dhqp_2042_1_008279357_1 92,256 272,326 2.9519
11 Feb 2013 12:40:07 1245167 15527027 hadam3p_pnw_dhqp_2042_1_008279357_1 80,736 238,756 2.9572
04 Feb 2013 14:40:38 1245167 15527027 hadam3p_pnw_dhqp_2042_1_008279357_1 69,216 205,409 2.9677
01 Feb 2013 10:32:10 1245167 15527027 hadam3p_pnw_dhqp_2042_1_008279357_1 57,696 171,904 2.9795
30 Jan 2013 08:43:08 1245167 15527027 hadam3p_pnw_dhqp_2042_1_008279357_1 46,176 138,458 2.9985
22 Jan 2013 11:00:54 1245167 15527027 hadam3p_pnw_dhqp_2042_1_008279357_1 34,656 104,187 3.0063
15 Jan 2013 13:35:27 1245167 15527027 hadam3p_pnw_dhqp_2042_1_008279357_1 23,136 69,668 3.0112
11 Jan 2013 08:59:19 1245167 15527027 hadam3p_pnw_dhqp_2042_1_008279357_1 11,616 35,069 3.0190


©2024 climateprediction.net