climateprediction.net home page
Task 15469700

Task 15469700

Name hadcm3n_zkvu_1880_40_008249149_2
Workunit 8404273
Created 3 Dec 2012, 17:20:16 UTC
Sent 3 Dec 2012, 17:20:27 UTC
Report deadline 5 Mar 2013, 0:47:38 UTC
Received 13 Mar 2013, 20:18:58 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status -226 (0xFFFFFF1E) ERR_TOO_MANY_EXITS
Computer ID 1165600
Run time 7 days 3 hours 34 min 1 sec
CPU time 6 days 18 hours 21 min 52 sec
Validate state Invalid
Credit 4,976.64
Device peak FLOPS 3.32 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
windows_intelx86
Stderr
<core_client_version>7.0.28</core_client_version>
<![CDATA[
<message>
too many exit(0)s
</message>
<stderr_txt>
ng
CPDN Monitor - No 'heartbeat' from BOINC...
13:40:09 (10712): No heartbeat from core client for 30 sec - exiting
13:40:10 (10712): No heartbeat from core client for 30 sec - exiting
13:40:11 (10712): No heartbeat from core client for 30 sec - exiting
13:40:12 (10712): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:40:49 (2148): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:41:33 (6640): No heartbeat from core client for 30 sec - exiting
13:41:34 (6640): No heartbeat from core client for 30 sec - exiting
13:41:35 (6640): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:42:18 (6248): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:43:00 (11752): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:43:36 (10704): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:43:37 (10704): No heartbeat from core client for 30 sec - exiting
13:44:16 (9572): No heartbeat from core client for 30 sec - exiting
13:44:17 (9572): No heartbeat from core client for 30 sec - exiting
13:44:18 (9572): No heartbeat from core client for 30 sec - exiting
13:44:19 (9572): No heartbeat from core client for 30 sec - exiting
13:44:20 (9572): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:45:02 (7796): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:45:46 (8384): No heartbeat from core client for 30 sec - exiting
13:45:47 (8384): No heartbeat from core client for 30 sec - exiting
13:45:48 (8384): No heartbeat from core client for 30 sec - exiting
13:45:49 (8384): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:46:27 (10020): No heartbeat from core client for 30 sec - exiting
13:46:28 (10020): No heartbeat from core client for 30 sec - exiting
13:46:29 (10020): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:47:09 (11752): No heartbeat from core client for 30 sec - exiting
13:47:10 (11752): No heartbeat from core client for 30 sec - exiting
13:47:11 (11752): No heartbeat from core client for 30 sec - exiting
13:47:12 (11752): No heartbeat from core client for 30 sec - exiting
13:47:13 (11752): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:47:48 (12136): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:48:30 (11992): No heartbeat from core client for 30 sec - exiting
13:48:31 (11992): No heartbeat from core client for 30 sec - exiting
13:48:32 (11992): No heartbeat from core client for 30 sec - exiting
13:48:33 (11992): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:49:10 (12256): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:49:56 (10932): No heartbeat from core client for 30 sec - exiting
13:49:57 (10932): No heartbeat from core client for 30 sec - exiting
13:49:58 (10932): No heartbeat from core client for 30 sec - exiting
13:49:59 (10932): No heartbeat from core client for 30 sec - exiting
13:50:00 (10932): No heartbeat from core client for 30 sec - exiting
13:50:01 (10932): No heartbeat from core client for 30 sec - exiting
13:50:02 (10932): No heartbeat from core client for 30 sec - exiting
13:50:03 (10932): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:50:37 (7560): No heartbeat from core client for 30 sec - exiting
13:50:38 (7560): No heartbeat from core client for 30 sec - exiting
13:50:39 (7560): No heartbeat from core client for 30 sec - exiting
13:50:40 (7560): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:51:17 (11452): No heartbeat from core client for 30 sec - exiting
13:51:18 (11452): No heartbeat from core client for 30 sec - exiting
13:51:19 (11452): No heartbeat from core client for 30 sec - exiting
13:51:20 (11452): No heartbeat from core client for 30 sec - exiting
13:51:21 (11452): No heartbeat from core client for 30 sec - exiting
13:51:22 (11452): No heartbeat from core client for 30 sec - exiting
13:51:23 (11452): No heartbeat from core client for 30 sec - exiting
13:51:24 (11452): No heartbeat from core client for 30 sec - exiting
13:51:25 (11452): No heartbeat from core client for 30 sec - exiting
13:51:26 (11452): No heartbeat from core client for 30 sec - exiting
13:51:27 (11452): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:51:59 (7608): No heartbeat from core client for 30 sec - exiting
13:52:00 (7608): No heartbeat from core client for 30 sec - exiting
13:52:01 (7608): No heartbeat from core client for 30 sec - exiting
13:52:02 (7608): No heartbeat from core client for 30 sec - exiting
13:52:03 (7608): No heartbeat from core client for 30 sec - exiting
13:52:04 (7608): No heartbeat from core client for 30 sec - exiting
13:52:05 (7608): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:52:41 (11012): No heartbeat from core client for 30 sec - exiting
13:52:42 (11012): No heartbeat from core client for 30 sec - exiting
13:52:43 (11012): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:53:34 (11744): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:54:17 (10252): No heartbeat from core client for 30 sec - exiting
13:54:18 (10252): No heartbeat from core client for 30 sec - exiting
13:54:19 (10252): No heartbeat from core client for 30 sec - exiting
13:54:20 (10252): No heartbeat from core client for 30 sec - exiting
13:54:21 (10252): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:54:55 (9412): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:55:39 (10864): No heartbeat from core client for 30 sec - exiting
13:55:40 (10864): No heartbeat from core client for 30 sec - exiting
13:55:41 (10864): No heartbeat from core client for 30 sec - exiting
13:55:42 (10864): No heartbeat from core client for 30 sec - exiting
13:55:43 (10864): No heartbeat from core client for 30 sec - exiting
13:55:44 (10864): No heartbeat from core client for 30 sec - exiting
13:55:45 (10864): No heartbeat from core client for 30 sec - exiting
13:55:46 (10864): No heartbeat from core client for 30 sec - exiting
Signal 11 received, exiting...
Called boinc_finish
CPDN Monitor - No 'heartbeat' from BOINC...
13:56:20 (9888): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:57:07 (4900): No heartbeat from core client for 30 sec - exiting
13:57:08 (4900): No heartbeat from core client for 30 sec - exiting
13:57:09 (4900): No heartbeat from core client for 30 sec - exiting
13:57:10 (4900): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:57:51 (4788): No heartbeat from core client for 30 sec - exiting
13:57:52 (4788): No heartbeat from core client for 30 sec - exiting
13:57:53 (4788): No heartbeat from core client for 30 sec - exiting
13:57:54 (4788): No heartbeat from core client for 30 sec - exiting
13:57:55 (4788): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:57:56 (4788): No heartbeat from core client for 30 sec - exiting
13:58:44 (11908): No heartbeat from core client for 30 sec - exiting
13:58:45 (11908): No heartbeat from core client for 30 sec - exiting
13:58:46 (11908): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:59:31 (11604): No heartbeat from core client for 30 sec - exiting
13:59:32 (11604): No heartbeat from core client for 30 sec - exiting
13:59:33 (11604): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:00:15 (7532): No heartbeat from core client for 30 sec - exiting
14:00:16 (7532): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:00:17 (7532): No heartbeat from core client for 30 sec - exiting
14:01:07 (8620): No heartbeat from core client for 30 sec - exiting
14:01:08 (8620): No heartbeat from core client for 30 sec - exiting
14:01:09 (8620): No heartbeat from core client for 30 sec - exiting
14:01:10 (8620): No heartbeat from core client for 30 sec - exiting
14:01:11 (8620): No heartbeat from core client for 30 sec - exiting
14:01:12 (8620): No heartbeat from core client for 30 sec - exiting
14:01:13 (8620): No heartbeat from core client for 30 sec - exiting
14:01:14 (8620): No heartbeat from core client for 30 sec - exiting
14:01:15 (8620): No heartbeat from core client for 30 sec - exiting
14:01:16 (8620): No heartbeat from core client for 30 sec - exiting
14:01:17 (8620): No heartbeat from core client for 30 sec - exiting
14:01:18 (8620): No heartbeat from core client for 30 sec - exiting
14:01:19 (8620): No heartbeat from core client for 30 sec - exiting
14:01:20 (8620): No heartbeat from core client for 30 sec - exiting
14:01:21 (8620): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:02:00 (7592): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:02:44 (11596): No heartbeat from core client for 30 sec - exiting
14:02:45 (11596): No heartbeat from core client for 30 sec - exiting
14:02:46 (11596): No heartbeat from core client for 30 sec - exiting
14:02:47 (11596): No heartbeat from core client for 30 sec - exiting
14:02:48 (11596): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:02:49 (11596): No heartbeat from core client for 30 sec - exiting
14:03:33 (8804): No heartbeat from core client for 30 sec - exiting
14:03:34 (8804): No heartbeat from core client for 30 sec - exiting
14:03:35 (8804): No heartbeat from core client for 30 sec - exiting
14:03:36 (8804): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:04:31 (4988): No heartbeat from core client for 30 sec - exiting
14:04:32 (4988): No heartbeat from core client for 30 sec - exiting
14:04:33 (4988): No heartbeat from core client for 30 sec - exiting
14:04:34 (4988): No heartbeat from core client for 30 sec - exiting
14:04:35 (4988): No heartbeat from core client for 30 sec - exiting
14:04:36 (4988): No heartbeat from core client for 30 sec - exiting
14:04:37 (4988): No heartbeat from core client for 30 sec - exiting
14:04:38 (4988): No heartbeat from core client for 30 sec - exiting
14:04:39 (4988): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:04:40 (4988): No heartbeat from core client for 30 sec - exiting
14:05:21 (11468): No heartbeat from core client for 30 sec - exiting
14:05:22 (11468): No heartbeat from core client for 30 sec - exiting
14:05:23 (11468): No heartbeat from core client for 30 sec - exiting
14:05:24 (11468): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:05:25 (11468): No heartbeat from core client for 30 sec - exiting
14:06:03 (12220): No heartbeat from core client for 30 sec - exiting
14:06:04 (12220): No heartbeat from core client for 30 sec - exiting
14:06:05 (12220): No heartbeat from core client for 30 sec - exiting
14:06:06 (12220): No heartbeat from core client for 30 sec - exiting
14:06:07 (12220): No heartbeat from core client for 30 sec - exiting
14:06:08 (12220): No heartbeat from core client for 30 sec - exiting
14:06:09 (12220): No heartbeat from core client for 30 sec - exiting
14:06:10 (12220): No heartbeat from core client for 30 sec - exiting
14:06:11 (12220): No heartbeat from core client for 30 sec - exiting
14:06:12 (12220): No heartbeat from core client for 30 sec - exiting
14:06:13 (12220): No heartbeat from core client for 30 sec - exiting
14:06:14 (12220): No heartbeat from core client for 30 sec - exiting
14:06:15 (12220): No heartbeat from core client for 30 sec - exiting
14:06:16 (12220): No heartbeat from core client for 30 sec - exiting
14:06:17 (12220): No heartbeat from core client for 30 sec - exiting
14:06:18 (12220): No heartbeat from core client for 30 sec - exiting
14:06:19 (12220): No heartbeat from core client for 30 sec - exiting
14:06:20 (12220): No heartbeat from core client for 30 sec - exiting
14:06:21 (12220): No heartbeat from core client for 30 sec - exiting
14:06:22 (12220): No heartbeat from core client for 30 sec - exiting
14:06:23 (12220): No heartbeat from core client for 30 sec - exiting
14:06:24 (12220): No heartbeat from core client for 30 sec - exiting
14:06:25 (12220): No heartbeat from core client for 30 sec - exiting
14:06:26 (12220): No heartbeat from core client for 30 sec - exiting
14:06:27 (12220): No heartbeat from core client for 30 sec - exiting
14:06:28 (12220): No heartbeat from core client for 30 sec - exiting
14:06:29 (12220): No heartbeat from core client for 30 sec - exiting
14:06:30 (12220): No heartbeat from core client for 30 sec - exiting
14:06:31 (12220): No heartbeat from core client for 30 sec - exiting
14:06:32 (12220): No heartbeat from core client for 30 sec - exiting
14:06:33 (12220): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:07:18 (7968): No heartbeat from core client for 30 sec - exiting
14:07:19 (7968): No heartbeat from core client for 30 sec - exiting
14:07:20 (7968): No heartbeat from core client for 30 sec - exiting
14:07:21 (7968): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:07:22 (7968): No heartbeat from core client for 30 sec - exiting
14:08:12 (12280): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Signal 11 received, exiting...
Called boinc_finish
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=12160, iMonCtr=1
Model crash detected, will try to restart...
14:08:59 (12160): No heartbeat from core client for 30 sec - exiting
14:09:00 (12160): No heartbeat from core client for 30 sec - exiting
14:09:01 (12160): No heartbeat from core client for 30 sec - exiting
14:09:02 (12160): No heartbeat from core client for 30 sec - exiting
14:09:03 (12160): No heartbeat from core client for 30 sec - exiting
14:09:04 (12160): No heartbeat from core client for 30 sec - exiting
14:09:05 (12160): No heartbeat from core client for 30 sec - exiting
14:09:06 (12160): No heartbeat from core client for 30 sec - exiting
14:09:07 (12160): No heartbeat from core client for 30 sec - exiting
14:09:08 (12160): No heartbeat from core client for 30 sec - exiting
14:09:09 (12160): No heartbeat from core client for 30 sec - exiting
14:09:10 (12160): No heartbeat from core client for 30 sec - exiting
14:09:11 (12160): No heartbeat from core client for 30 sec - exiting
14:09:12 (12160): No heartbeat from core client for 30 sec - exiting
14:09:13 (12160): No heartbeat from core client for 30 sec - exiting
14:09:14 (12160): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:10:29 (5088): No heartbeat from core client for 30 sec - exiting
14:10:30 (5088): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:11:13 (10756): No heartbeat from core client for 30 sec - exiting
14:11:14 (10756): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:11:56 (7592): No heartbeat from core client for 30 sec - exiting
14:11:57 (7592): No heartbeat from core client for 30 sec - exiting
14:11:58 (7592): No heartbeat from core client for 30 sec - exiting
14:11:59 (7592): No heartbeat from core client for 30 sec - exiting
14:12:00 (7592): No heartbeat from core client for 30 sec - exiting
14:12:01 (7592): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:12:45 (10652): No heartbeat from core client for 30 sec - exiting
14:12:46 (10652): No heartbeat from core client for 30 sec - exiting
14:12:47 (10652): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:13:29 (8300): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:13:30 (8300): No heartbeat from core client for 30 sec - exiting
14:14:13 (12056): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:15:00 (10620): No heartbeat from core client for 30 sec - exiting
14:15:01 (10620): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:15:44 (11400): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:16:27 (4596): No heartbeat from core client for 30 sec - exiting
14:16:29 (4596): No heartbeat from core client for 30 sec - exiting
14:16:30 (4596): No heartbeat from core client for 30 sec - exiting
14:16:31 (4596): No heartbeat from core client for 30 sec - exiting
14:16:32 (4596): No heartbeat from core client for 30 sec - exiting
14:16:33 (4596): No heartbeat from core client for 30 sec - exiting
14:16:34 (4596): No heartbeat from core client for 30 sec - exiting
14:16:35 (4596): No heartbeat from core client for 30 sec - exiting
14:16:36 (4596): No heartbeat from core client for 30 sec - exiting
14:16:37 (4596): No heartbeat from core client for 30 sec - exiting
14:16:38 (4596): No heartbeat from core client for 30 sec - exiting
14:16:39 (4596): No heartbeat from core client for 30 sec - exiting
14:16:40 (4596): No heartbeat from core client for 30 sec - exiting
14:16:41 (4596): No heartbeat from core client for 30 sec - exiting
14:16:42 (4596): No heartbeat from core client for 30 sec - exiting
14:16:43 (4596): No heartbeat from core client for 30 sec - exiting
14:16:44 (4596): No heartbeat from core client for 30 sec - exiting
14:16:45 (4596): No heartbeat from core client for 30 sec - exiting
14:16:46 (4596): No heartbeat from core client for 30 sec - exiting
14:16:47 (4596): No heartbeat from core client for 30 sec - exiting
14:16:48 (4596): No heartbeat from core client for 30 sec - exiting
14:16:49 (4596): No heartbeat from core client for 30 sec - exiting
14:16:50 (4596): No heartbeat from core client for 30 sec - exiting
14:16:51 (4596): No heartbeat from core client for 30 sec - exiting
14:16:52 (4596): No heartbeat from core client for 30 sec - exiting
14:16:53 (4596): No heartbeat from core client for 30 sec - exiting
14:16:54 (4596): No heartbeat from core client for 30 sec - exiting
14:16:55 (4596): No heartbeat from core client for 30 sec - exiting
14:16:56 (4596): No heartbeat from core client for 30 sec - exiting
14:16:57 (4596): No heartbeat from core client for 30 sec - exiting
14:16:58 (4596): No heartbeat from core client for 30 sec - exiting
14:16:59 (4596): No heartbeat from core client for 30 sec - exiting
14:17:00 (4596): No heartbeat from core client for 30 sec - exiting
14:17:01 (4596): No heartbeat from core client for 30 sec - exiting
14:17:02 (4596): No heartbeat from core client for 30 sec - exiting
14:17:03 (4596): No heartbeat from core client for 30 sec - exiting
14:17:04 (4596): No heartbeat from core client for 30 sec - exiting
14:17:05 (4596): No heartbeat from core client for 30 sec - exiting
14:17:06 (4596): No heartbeat from core client for 30 sec - exiting
14:17:07 (4596): No heartbeat from core client for 30 sec - exiting
14:17:08 (4596): No heartbeat from core client for 30 sec - exiting
14:17:09 (4596): No heartbeat from core client for 30 sec - exiting
14:17:10 (4596): No heartbeat from core client for 30 sec - exiting
14:17:11 (4596): No heartbeat from core client for 30 sec - exiting
14:17:12 (4596): No heartbeat from core client for 30 sec - exiting
14:17:13 (4596): No heartbeat from core client for 30 sec - exiting
14:17:14 (4596): No heartbeat from core client for 30 sec - exiting
14:17:15 (4596): No heartbeat from core client for 30 sec - exiting
14:17:16 (4596): No heartbeat from core client for 30 sec - exiting
14:17:17 (4596): No heartbeat from core client for 30 sec - exiting
14:17:18 (4596): No heartbeat from core client for 30 sec - exiting
14:17:19 (4596): No heartbeat from core client for 30 sec - exiting
14:17:20 (4596): No heartbeat from core client for 30 sec - exiting
14:17:21 (4596): No heartbeat from core client for 30 sec - exiting
14:17:22 (4596): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Signal 11 received, exiting...
Called boinc_finish
14:18:24 (10348): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:19:06 (10100): No heartbeat from core client for 30 sec - exiting
14:19:07 (10100): No heartbeat from core client for 30 sec - exiting
14:19:08 (10100): No heartbeat from core client for 30 sec - exiting
14:19:09 (10100): No heartbeat from core client for 30 sec - exiting
14:19:10 (10100): No heartbeat from core client for 30 sec - exiting
14:19:11 (10100): No heartbeat from core client for 30 sec - exiting
14:19:12 (10100): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:19:13 (10100): No heartbeat from core client for 30 sec - exiting
14:19:46 (9168): No heartbeat from core client for 30 sec - exiting
14:19:47 (9168): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:20:36 (10164): No heartbeat from core client for 30 sec - exiting
14:20:38 (10164): No heartbeat from core client for 30 sec - exiting
14:20:39 (10164): No heartbeat from core client for 30 sec - exiting
14:20:40 (10164): No heartbeat from core client for 30 sec - exiting
14:20:41 (10164): No heartbeat from core client for 30 sec - exiting
14:20:42 (10164): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:20:43 (10164): No heartbeat from core client for 30 sec - exiting
14:21:19 (876): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:22:01 (10552): No heartbeat from core client for 30 sec - exiting
14:22:02 (10552): No heartbeat from core client for 30 sec - exiting
14:22:03 (10552): No heartbeat from core client for 30 sec - exiting
14:22:04 (10552): No heartbeat from core client for 30 sec - exiting
14:22:05 (10552): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:22:06 (10552): No heartbeat from core client for 30 sec - exiting
14:22:51 (10348): No heartbeat from core client for 30 sec - exiting
14:22:52 (10348): No heartbeat from core client for 30 sec - exiting
14:22:53 (10348): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:23:35 (6892): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:24:17 (11444): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:25:19 (10360): No heartbeat from core client for 30 sec - exiting
14:25:22 (10360): No heartbeat from core client for 30 sec - exiting
14:25:23 (10360): No heartbeat from core client for 30 sec - exiting
14:25:24 (10360): No heartbeat from core client for 30 sec - exiting
14:25:25 (10360): No heartbeat from core client for 30 sec - exiting
14:25:26 (10360): No heartbeat from core client for 30 sec - exiting
14:25:27 (10360): No heartbeat from core client for 30 sec - exiting
14:25:28 (10360): No heartbeat from core client for 30 sec - exiting
14:25:29 (10360): No heartbeat from core client for 30 sec - exiting
14:25:30 (10360): No heartbeat from core client for 30 sec - exiting
14:25:31 (10360): No heartbeat from core client for 30 sec - exiting
14:25:32 (10360): No heartbeat from core client for 30 sec - exiting
14:25:33 (10360): No heartbeat from core client for 30 sec - exiting
14:25:34 (10360): No heartbeat from core client for 30 sec - exiting
14:25:35 (10360): No heartbeat from core client for 30 sec - exiting
14:25:36 (10360): No heartbeat from core client for 30 sec - exiting
14:25:37 (10360): No heartbeat from core client for 30 sec - exiting
14:25:38 (10360): No heartbeat from core client for 30 sec - exiting
14:25:39 (10360): No heartbeat from core client for 30 sec - exiting
14:25:40 (10360): No heartbeat from core client for 30 sec - exiting
14:25:41 (10360): No heartbeat from core client for 30 sec - exiting
14:25:42 (10360): No heartbeat from core client for 30 sec - exiting
14:25:43 (10360): No heartbeat from core client for 30 sec - exiting
14:25:44 (10360): No heartbeat from core client for 30 sec - exiting
14:25:45 (10360): No heartbeat from core client for 30 sec - exiting
14:25:46 (10360): No heartbeat from core client for 30 sec - exiting
14:25:47 (10360): No heartbeat from core client for 30 sec - exiting
14:25:48 (10360): No heartbeat from core client for 30 sec - exiting
14:25:49 (10360): No heartbeat from core client for 30 sec - exiting
14:25:50 (10360): No heartbeat from core client for 30 sec - exiting
14:25:51 (10360): No heartbeat from core client for 30 sec - exiting
14:25:52 (10360): No heartbeat from core client for 30 sec - exiting
14:25:53 (10360): No heartbeat from core client for 30 sec - exiting
14:25:54 (10360): No heartbeat from core client for 30 sec - exiting
14:25:55 (10360): No heartbeat from core client for 30 sec - exiting
14:25:56 (10360): No heartbeat from core client for 30 sec - exiting
14:25:57 (10360): No heartbeat from core client for 30 sec - exiting
14:25:58 (10360): No heartbeat from core client for 30 sec - exiting
14:25:59 (10360): No heartbeat from core client for 30 sec - exiting
14:26:00 (10360): No heartbeat from core client for 30 sec - exiting
14:26:01 (10360): No heartbeat from core client for 30 sec - exiting
14:26:02 (10360): No heartbeat from core client for 30 sec - exiting
14:26:03 (10360): No heartbeat from core client for 30 sec - exiting
14:26:04 (10360): No heartbeat from core client for 30 sec - exiting
14:26:05 (10360): No heartbeat from core client for 30 sec - exiting
14:26:06 (10360): No heartbeat from core client for 30 sec - exiting
14:26:07 (10360): No heartbeat from core client for 30 sec - exiting
14:26:08 (10360): No heartbeat from core client for 30 sec - exiting
14:26:09 (10360): No heartbeat from core client for 30 sec - exiting
14:26:10 (10360): No heartbeat from core client for 30 sec - exiting
14:26:11 (10360): No heartbeat from core client for 30 sec - exiting
14:26:12 (10360): No heartbeat from core client for 30 sec - exiting
14:26:13 (10360): No heartbeat from core client for 30 sec - exiting
14:26:14 (10360): No heartbeat from core client for 30 sec - exiting
14:26:15 (10360): No heartbeat from core client for 30 sec - exiting
14:26:16 (10360): No heartbeat from core client for 30 sec - exiting
14:26:17 (10360): No heartbeat from core client for 30 sec - exiting
14:26:18 (10360): No heartbeat from core client for 30 sec - exiting
14:26:19 (10360): No heartbeat from core client for 30 sec - exiting
14:26:20 (10360): No heartbeat from core client for 30 sec - exiting
14:26:21 (10360): No heartbeat from core client for 30 sec - exiting
14:26:22 (10360): No heartbeat from core client for 30 sec - exiting
14:26:23 (10360): No heartbeat from core client for 30 sec - exiting
14:26:24 (10360): No heartbeat from core client for 30 sec - exiting
14:26:25 (10360): No heartbeat from core client for 30 sec - exiting
14:26:26 (10360): No heartbeat from core client for 30 sec - exiting
14:26:27 (10360): No heartbeat from core client for 30 sec - exiting
14:26:28 (10360): No heartbeat from core client for 30 sec - exiting
14:26:29 (10360): No heartbeat from core client for 30 sec - exiting
14:26:30 (10360): No heartbeat from core client for 30 sec - exiting
14:26:31 (10360): No heartbeat from core client for 30 sec - exiting
14:26:32 (10360): No heartbeat from core client for 30 sec - exiting
14:26:33 (10360): No heartbeat from core client for 30 sec - exiting
14:26:34 (10360): No heartbeat from core client for 30 sec - exiting
14:26:35 (10360): No heartbeat from core client for 30 sec - exiting
14:26:36 (10360): No heartbeat from core client for 30 sec - exiting
14:26:37 (10360): No heartbeat from core client for 30 sec - exiting
14:26:38 (10360): No heartbeat from core client for 30 sec - exiting
14:26:39 (10360): No heartbeat from core client for 30 sec - exiting
14:26:40 (10360): No heartbeat from core client for 30 sec - exiting
14:26:41 (10360): No heartbeat from core client for 30 sec - exiting
14:26:42 (10360): No heartbeat from core client for 30 sec - exiting
14:26:43 (10360): No heartbeat from core client for 30 sec - exiting
14:26:44 (10360): No heartbeat from core client for 30 sec - exiting
14:26:45 (10360): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:26:46 (10360): No heartbeat from core client for 30 sec - exiting


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x757BC41F

Engaging BOINC Windows Runtime Debugger...



********************


BOINC Windows Runtime Debugger Version 6.13.0


Dump Timestamp    : 03/13/13 14:28:15
Install Directory : 
Data Directory    : C:\ProgramData\BOINC
Project Symstore  : 
LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126
Loaded Library    : dbghelp.dll
LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126
Loaded Library    : version.dll
Debugger Engine   : 4.0.5.0
Symbol Search Path: C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_zkvu_1880_40_008249149;C:\ProgramData\BOINC\projects\climateprediction.net


ModLoad: 00400000 00b28000 C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_um_6.07_windows_intelx86.exe (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 777c0000 00180000 C:\Windows\SysWOW64\ntdll.dll (6.1.7601.17725) (-exported- Symbols Loaded)
    Linked PDB Filename   : wntdll.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 76910000 00110000 C:\Windows\syswow64\kernel32.dll (6.1.7601.17965) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernel32.pdb
    File Version          : 6.1.7601.17965 (win7sp1_gdr.121004-0333)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7601.17965

ModLoad: 757b0000 00047000 C:\Windows\syswow64\KERNELBASE.dll (6.1.7601.17965) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernelbase.pdb
    File Version          : 6.1.7601.17965 (win7sp1_gdr.121004-0333)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7601.17965

ModLoad: 76a80000 00100000 C:\Windows\syswow64\USER32.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : wuser32.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7601.17514

ModLoad: 75390000 00090000 C:\Windows\syswow64\GDI32.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : wgdi32.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17514

ModLoad: 75930000 0000a000 C:\Windows\syswow64\LPK.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : wlpk.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 75260000 0009d000 C:\Windows\syswow64\USP10.dll (1.626.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : usp10.pdb
    File Version          : 1.0626.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft(R) Uniscribe Unicode script processor
    Product Version       : 1.0626.7601.17514

ModLoad: 74f00000 000ac000 C:\Windows\syswow64\msvcrt.dll (7.0.7601.17744) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcrt.pdb
    File Version          : 7.0.7601.17744 (win7sp1_gdr.111215-1535)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 7.0.7601.17744

ModLoad: 76840000 000a0000 C:\Windows\syswow64\ADVAPI32.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : advapi32.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 755f0000 00019000 C:\Windows\SysWOW64\sechost.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : sechost.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 74fb0000 000f0000 C:\Windows\syswow64\RPCRT4.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : wrpcrt4.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 74ea0000 00060000 C:\Windows\syswow64\SspiCli.dll (6.1.7601.17856) (-exported- Symbols Loaded)
    Linked PDB Filename   : wsspicli.pdb
    File Version          : 6.1.7601.17856 (win7sp1_gdr.120601-1505)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17856

ModLoad: 74e90000 0000c000 C:\Windows\syswow64\CRYPTBASE.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : cryptbase.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 64f00000 000c8000 C:\Windows\system32\OPENGL32.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : opengl32.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 69da0000 00022000 C:\Windows\system32\GLU32.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : glu32.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 64e10000 000e7000 C:\Windows\system32\DDRAW.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : ddraw.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 69cb0000 00006000 C:\Windows\system32\DCIMAN32.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : dciman32.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 75610000 0019d000 C:\Windows\syswow64\SETUPAPI.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : setupapi.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 759d0000 00027000 C:\Windows\syswow64\CFGMGR32.dll (6.1.7601.17621) (-exported- Symbols Loaded)
    Linked PDB Filename   : cfgmgr32.pdb
    File Version          : 6.1.7601.17621 (win7sp1_gdr.110523-2108)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17621

Get Product Name Failed.
ModLoad: 75940000 0008f000 C:\Windows\syswow64\OLEAUT32.dll (6.1.7601.17676) (-exported- Symbols Loaded)
    Linked PDB Filename   : oleaut32.pdb
    File Version          : 6.1.7601.17676
    Company Name          : Microsoft Corporation
    Product Name          : 
    Product Version       : 6.1.7601.17676

ModLoad: 75100000 0015c000 C:\Windows\syswow64\ole32.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : ole32.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 76b80000 00012000 C:\Windows\syswow64\DEVOBJ.dll (6.1.7601.17621) (-exported- Symbols Loaded)
    Linked PDB Filename   : devobj.pdb
    File Version          : 6.1.7601.17621 (win7sp1_gdr.110523-2108)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17621

ModLoad: 71690000 00013000 C:\Windows\system32\dwmapi.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : dwmapi.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 768e0000 0002a000 C:\Windows\syswow64\imagehlp.dll (6.1.7601.17787) (-exported- Symbols Loaded)
    Linked PDB Filename   : imagehlp.pdb
    File Version          : 6.1.7601.17787 (win7sp1_gdr.120229-1502)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17787

ModLoad: 750a0000 00060000 C:\Windows\system32\IMM32.DLL (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : wimm32.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17514

ModLoad: 75a00000 000cc000 C:\Windows\syswow64\MSCTF.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : msctf.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 713b0000 000eb000 C:\Windows\system32\dbghelp.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : dbghelp.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17514

ModLoad: 74dc0000 00009000 C:\Windows\system32\version.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : version.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385



*** Dump of the Process Statistics: ***

- I/O Operations Counters -
Read: 0, Write: 0, Other 0

- I/O Transfers Counters -
Read: 0, Write: 0, Other 0

- Paged Pool Usage -
QuotaPagedPoolUsage: 0, QuotaPeakPagedPoolUsage: 0
QuotaNonPagedPoolUsage: 0, QuotaPeakNonPagedPoolUsage: 0

- Virtual Memory Usage -
VirtualSize: 0, PeakVirtualSize: 0

- Pagefile Usage -
PagefileUsage: 0, PeakPagefileUsage: 0

- Working Set Size -
WorkingSetSize: 0, PeakWorkingSetSize: 0, PageFaultCount: 0

*** Dump of thread ID 11616 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x757BC41F

- Registers -
eax=1132f9c4 ebx=00000001 ecx=00000003 edx=00000000 esi=009fca88 edi=fffffffe
eip=757bc41f esp=1132f9c4 ebp=1132fa14
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000212

- Callstack -
ChildEBP RetAddr  Args to Child
1132fa14 007bfa90 e06d7363 00000001 00000003 1132fa40 KERNELBASE!RaiseException+0x0 
1132fa4c 007bbbcb 1132fa5c 00860978 0082f1b4 0085d354 hadcm3n_um_6.07_windows_intelx8!+0x0 
1132fa68 00406e8c 04600000 01000000 00500000 004083ee hadcm3n_um_6.07_windows_intelx8!+0x0 
1132fad8 00408130 1132faec 114b3a59 1132faec 114b39e1 hadcm3n_um_6.07_windows_intelx8!+0x0 
1132ff88 769233aa 7efde000 1132ffd4 777f9ef2 7efde000 hadcm3n_um_6.07_windows_intelx8!+0x0 
1132ff94 777f9ef2 7efde000 65645307 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 
1132ffd4 777f9ec5 007bde7f 7efde000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 
1132ffec 00000000 007bde7f 7efde000 00000000 00360000 ntdll!RtlInitializeExceptionChain+0x0 

*** Dump of thread ID 11792 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Registers -
eax=00749910 ebx=00000000 ecx=00000000 edx=00000000 esi=4435ff48 edi=00000000
eip=777dfd71 esp=4435ff04 ebp=4435ff6c
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
4435ff6c 757c4498 00000064 00000000 4435ff88 00749924 ntdll!ZwDelayExecution+0x0 
4435ff7c 00749924 00000064 4435ff94 769233aa 00000000 KERNELBASE!Sleep+0x0 
4435ff88 769233aa 00000000 4435ffd4 777f9ef2 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 
4435ff94 777f9ef2 00000000 30635307 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 
4435ffd4 777f9ec5 00749910 00000000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 
4435ffec 00000000 00749910 00000000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 


*** Debug Message Dump ****


*** Foreground Window Data ***
    Window Name      : 
    Window Class     : 
    Window Process ID: 0
    Window Thread ID : 0

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10676, iMonCtr=1
Model crash detected, will try to restart...
14:28:47 (10676): No heartbeat from core client for 30 sec - exiting
14:28:48 (10676): No heartbeat from core client for 30 sec - exiting
14:28:49 (10676): No heartbeat from core client for 30 sec - exiting
14:28:50 (10676): No heartbeat from core client for 30 sec - exiting
14:28:51 (10676): No heartbeat from core client for 30 sec - exiting
14:28:52 (10676): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:30:07 (11580): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:30:47 (7704): No heartbeat from core client for 30 sec - exiting
14:30:48 (7704): No heartbeat from core client for 30 sec - exiting
14:30:49 (7704): No heartbeat from core client for 30 sec - exiting
14:30:50 (7704): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:31:34 (12272): No heartbeat from core client for 30 sec - exiting


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x757BC41F

Engaging BOINC Windows Runtime Debugger...

14:31:35 (12272): No heartbeat from core client for 30 sec - exiting


********************


BOINC Windows Runtime Debugger Version 6.13.0


Dump Timestamp    : 03/13/13 14:31:35
Install Directory : 
Data Directory    : C:\ProgramData\BOINC
Project Symstore  : 
LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126
Loaded Library    : dbghelp.dll
LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126
Loaded Library    : version.dll
Debugger Engine   : 4.0.5.0
Symbol Search Path: C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_zkvu_1880_40_008249149;C:\ProgramData\BOINC\projects\climateprediction.net


ModLoad: 00400000 00b28000 C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_um_6.07_windows_intelx86.exe (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 777c0000 00180000 C:\Windows\SysWOW64\ntdll.dll (6.1.7601.17725) (-exported- Symbols Loaded)
    Linked PDB Filename   : wntdll.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

CPDN Monitor - No 'heartbeat' from BOINC...
14:31:36 (12272): No heartbeat from core client for 30 sec - exiting
14:32:16 (9432): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:33:12 (11644): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:33:50 (8440): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:34:33 (7660): No heartbeat from core client for 30 sec - exiting
14:34:34 (7660): No heartbeat from core client for 30 sec - exiting
14:34:35 (7660): No heartbeat from core client for 30 sec - exiting
14:34:36 (7660): No heartbeat from core client for 30 sec - exiting
14:34:37 (7660): No heartbeat from core client for 30 sec - exiting
14:34:38 (7660): No heartbeat from core client for 30 sec - exiting
14:34:39 (7660): No heartbeat from core client for 30 sec - exiting
14:34:40 (7660): No heartbeat from core client for 30 sec - exiting
14:34:41 (7660): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:35:18 (9492): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:36:02 (10040): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Signal 11 received, exiting...
Called boinc_finish
14:36:41 (11848): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Signal 11 received, exiting...
Called boinc_finish
14:37:26 (8744): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:38:09 (8440): No heartbeat from core client for 30 sec - exiting
14:38:10 (8440): No heartbeat from core client for 30 sec - exiting
14:38:11 (8440): No heartbeat from core client for 30 sec - exiting
14:38:12 (8440): No heartbeat from core client for 30 sec - exiting
14:38:13 (8440): No heartbeat from core client for 30 sec - exiting
14:38:14 (8440): No heartbeat from core client for 30 sec - exiting
14:38:15 (8440): No heartbeat from core client for 30 sec - exiting
14:38:16 (8440): No heartbeat from core client for 30 sec - exiting
14:38:17 (8440): No heartbeat from core client for 30 sec - exiting
14:38:18 (8440): No heartbeat from core client for 30 sec - exiting
14:38:19 (8440): No heartbeat from core client for 30 sec - exiting
14:38:20 (8440): No heartbeat from core client for 30 sec - exiting
14:38:21 (8440): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:38:55 (11132): No heartbeat from core client for 30 sec - exiting
14:38:56 (11132): No heartbeat from core client for 30 sec - exiting
14:38:57 (11132): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:39:44 (9168): No heartbeat from core client for 30 sec - exiting
14:39:45 (9168): No heartbeat from core client for 30 sec - exiting
14:39:46 (9168): No heartbeat from core client for 30 sec - exiting
14:39:47 (9168): No heartbeat from core client for 30 sec - exiting
14:39:48 (9168): No heartbeat from core client for 30 sec - exiting
14:39:49 (9168): No heartbeat from core client for 30 sec - exiting
14:39:50 (9168): No heartbeat from core client for 30 sec - exiting
14:39:51 (9168): No heartbeat from core client for 30 sec - exiting
14:39:52 (9168): No heartbeat from core client for 30 sec - exiting
14:39:53 (9168): No heartbeat from core client for 30 sec - exiting
14:39:54 (9168): No heartbeat from core client for 30 sec - exiting
14:39:55 (9168): No heartbeat from core client for 30 sec - exiting
14:39:56 (9168): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:40:42 (6932): No heartbeat from core client for 30 sec - exiting
14:40:43 (6932): No heartbeat from core client for 30 sec - exiting
14:40:44 (6932): No heartbeat from core client for 30 sec - exiting
14:40:45 (6932): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:41:27 (11348): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:42:14 (9120): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:42:54 (11032): No heartbeat from core client for 30 sec - exiting


Unhandled Exception Detected...

- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x757BC41F

Engaging BOINC Windows Runtime Debugger...



********************


BOINC Windows Runtime Debugger Version 6.13.0


Dump Timestamp    : 03/13/13 14:42:54
Install Directory : 
Data Directory    : C:\ProgramData\BOINC
Project Symstore  : 
LoadLibraryA( C:\ProgramData\BOINC\dbghelp.dll ): GetLastError = 126
Loaded Library    : dbghelp.dll
LoadLibraryA( C:\ProgramData\BOINC\symsrv.dll ): GetLastError = 126
LoadLibraryA( symsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\srcsrv.dll ): GetLastError = 126
LoadLibraryA( srcsrv.dll ): GetLastError = 126
LoadLibraryA( C:\ProgramData\BOINC\version.dll ): GetLastError = 126
Loaded Library    : version.dll
Debugger Engine   : 4.0.5.0
Symbol Search Path: C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_zkvu_1880_40_008249149;C:\ProgramData\BOINC\projects\climateprediction.net


ModLoad: 00400000 00b28000 C:\ProgramData\BOINC\projects\climateprediction.net\hadcm3n_um_6.07_windows_intelx86.exe (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : 

ModLoad: 777c0000 00180000 C:\Windows\SysWOW64\ntdll.dll (6.1.7601.17725) (-exported- Symbols Loaded)
    Linked PDB Filename   : wntdll.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 76910000 00110000 C:\Windows\syswow64\kernel32.dll (6.1.7601.17965) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernel32.pdb
    File Version          : 6.1.7601.17965 (win7sp1_gdr.121004-0333)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17965

ModLoad: 757b0000 00047000 C:\Windows\syswow64\KERNELBASE.dll (6.1.7601.17965) (-exported- Symbols Loaded)
    Linked PDB Filename   : wkernelbase.pdb
    File Version          : 6.1.7601.17965 (win7sp1_gdr.121004-0333)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17965

ModLoad: 76a80000 00100000 C:\Windows\syswow64\USER32.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : wuser32.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7601.17514

ModLoad: 75390000 00090000 C:\Windows\syswow64\GDI32.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : wgdi32.pdb
    File Version          : 6.1.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17514

ModLoad: 75930000 0000a000 C:\Windows\syswow64\LPK.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : wlpk.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 75260000 0009d000 C:\Windows\syswow64\USP10.dll (1.626.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : usp10.pdb
    File Version          : 1.0626.7601.17514 (win7sp1_rtm.101119-1850)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft(R) Uniscribe Unicode script processor
    Product Version       : 1.0626.7601.17514

ModLoad: 74f00000 000ac000 C:\Windows\syswow64\msvcrt.dll (7.0.7601.17744) (-exported- Symbols Loaded)
    Linked PDB Filename   : msvcrt.pdb
    File Version          : 7.0.7601.17744 (win7sp1_gdr.111215-1535)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 7.0.7601.17744

ModLoad: 76840000 000a0000 C:\Windows\syswow64\ADVAPI32.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : advapi32.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 755f0000 00019000 C:\Windows\SysWOW64\sechost.dll (6.1.7600.16385) (-exported- Symbols Loaded)
    Linked PDB Filename   : sechost.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 74fb0000 000f0000 C:\Windows\syswow64\RPCRT4.dll (6.1.7601.17514) (-exported- Symbols Loaded)
    Linked PDB Filename   : wrpcrt4.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 74ea0000 00060000 C:\Windows\syswow64\SspiCli.dll (6.1.7601.17856) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : wsspicli.pdb
    File Version          : 6.1.7601.17856 (win7sp1_gdr.120601-1505)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17856

ModLoad: 74e90000 0000c000 C:\Windows\syswow64\CRYPTBASE.dll (6.1.7600.16385) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : cryptbase.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 64f00000 000c8000 C:\Windows\system32\OPENGL32.dll (6.1.7600.16385) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : opengl32.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 69da0000 00022000 C:\Windows\system32\GLU32.dll (6.1.7600.16385) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : glu32.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 64e10000 000e7000 C:\Windows\system32\DDRAW.dll (6.1.7600.16385) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : ddraw.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 69cb0000 00006000 C:\Windows\system32\DCIMAN32.dll (6.1.7600.16385) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : dciman32.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7600.16385

ModLoad: 75610000 0019d000 C:\Windows\syswow64\SETUPAPI.dll (6.1.7601.17514) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : setupapi.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385

ModLoad: 759d0000 00027000 C:\Windows\syswow64\CFGMGR32.dll (6.1.7601.17621) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : cfgmgr32.pdb
    File Version          : 6.1.7601.17621 (win7sp1_gdr.110523-2108)
    Company Name          : Microsoft Corporation
    Product Name          : Microsoft&#174; Windows&#174; Operating System
    Product Version       : 6.1.7601.17621

Get Product Name Failed.
ModLoad: 75940000 0008f000 C:\Windows\syswow64\OLEAUT32.dll (6.1.7601.17676) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : oleaut32.pdb
    File Version          : 6.1.7601.17676
    Company Name          : Microsoft Corporation
    Product Name          : 
    Product Version       : 6.1.7601.17676

ModLoad: 75100000 0015c000 C:\Windows\syswow64\ole32.dll (6.1.7601.17514) (-nosymbols- Symbols Loaded)
    Linked PDB Filename   : ole32.pdb
    File Version          : 6.1.7600.16385 (win7_rtm.090713-1255)
    Company Name          : Microsoft Corporation
    Product Name          : Betriebssystem Microsoft&#174; Windows&#174;
    Product Version       : 6.1.7600.16385



*** Dump of the Process Statistics: ***

- I/O Operations Counters -
Read: 0, Write: 0, Other 0

- I/O Transfers Counters -
Read: 0, Write: 0, Other 0

- Paged Pool Usage -
QuotaPagedPoolUsage: 0, QuotaPeakPagedPoolUsage: 0
QuotaNonPagedPoolUsage: 0, QuotaPeakNonPagedPoolUsage: 0

- Virtual Memory Usage -
VirtualSize: 0, PeakVirtualSize: 0

- Pagefile Usage -
PagefileUsage: 0, PeakPagefileUsage: 0

- Working Set Size -
WorkingSetSize: 0, PeakWorkingSetSize: 0, PageFaultCount: 0

*** Dump of thread ID 4784 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Unhandled Exception Record -
Reason: Out Of Memory (C++ Exception) (0xe06d7363) at address 0x757BC41F

- Registers -
eax=1132f9c4 ebx=00000001 ecx=00000003 edx=00000000 esi=009fca88 edi=fffffffe
eip=757bc41f esp=1132f9c4 ebp=1132fa14
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000212

- Callstack -
ChildEBP RetAddr  Args to Child
1132fa14 007bfa90 e06d7363 00000001 00000003 1132fa40 KERNELBASE!RaiseException+0x0 
1132fa4c 007bbbcb 1132fa5c 00860978 0082f1b4 0085d354 hadcm3n_um_6.07_windows_intelx8!+0x0 
1132fa68 00406e8c 04600000 01000000 00500000 004083ee hadcm3n_um_6.07_windows_intelx8!+0x0 
1132fad8 00408130 1132faec 00203a59 1132faec 002039e1 hadcm3n_um_6.07_windows_intelx8!+0x0 
1132ff88 769233aa 7efde000 1132ffd4 777f9ef2 7efde000 hadcm3n_um_6.07_windows_intelx8!+0x0 
1132ff94 777f9ef2 7efde000 6559273a 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 
1132ffd4 777f9ec5 007bde7f 7efde000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 
1132ffec 00000000 007bde7f 7efde000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 

*** Dump of thread ID 10600 (state: Initialized): ***

- Information -
Status: Base Priority: Normal, Priority: Normal, , Kernel Time: 0.000000, User Time: 0.000000, Wait Time: 0.000000

- Registers -
eax=00749910 ebx=00000000 ecx=00000000 edx=00000000 esi=3351ff48 edi=00000000
eip=777dfd71 esp=3351ff04 ebp=3351ff6c
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246

- Callstack -
ChildEBP RetAddr  Args to Child
3351ff6c 757c4498 00000064 00000000 3351ff88 00749924 ntdll!ZwDelayExecution+0x0 
3351ff7c 00749924 00000064 3351ff94 769233aa 00000000 KERNELBASE!Sleep+0x0 
3351ff88 769233aa 00000000 3351ffd4 777f9ef2 00000000 hadcm3n_um_6.07_windows_intelx8!+0x0 
3351ff94 777f9ef2 00000000 473a273a 00000000 00000000 kernel32!BaseThreadInitThunk+0x0 
3351ffd4 777f9ec5 00749910 00000000 00000000 00000000 ntdll!RtlInitializeExceptionChain+0x0 
3351ffec 00000000 00749910 00000000 00000000 43534552 ntdll!RtlInitializeExceptionChain+0x0 


*** Debug Message Dump ****


*** Foreground Window Data ***
    Window Name      : 
    Window Class     : 
    Window Process ID: 0
    Window Thread ID : 0

Exiting...
CPDN Monitor - No 'heartbeat' from BOINC...
14:43:47 (8820): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Signal 11 received, exiting...
Called boinc_finish
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=11016, iMonCtr=1
Model crash detected, will try to restart...
14:45:13 (11016): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:45:14 (11016): No heartbeat from core client for 30 sec - exiting

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
13 Mar 2013 04:23:49 1165600 15469700 hadcm3n_zkvu_1880_40_008249149_2 414,720 560,442 1.3514
12 Mar 2013 17:46:28 1165600 15469700 hadcm3n_zkvu_1880_40_008249149_2 388,800 526,316 1.3537
12 Mar 2013 08:31:27 1165600 15469700 hadcm3n_zkvu_1880_40_008249149_2 362,880 494,472 1.3626
12 Mar 2013 00:03:57 1165600 15469700 hadcm3n_zkvu_1880_40_008249149_2 336,960 462,875 1.3737
11 Mar 2013 14:10:25 1165600 15469700 hadcm3n_zkvu_1880_40_008249149_2 311,040 431,654 1.3878
11 Mar 2013 05:26:47 1165600 15469700 hadcm3n_zkvu_1880_40_008249149_2 285,120 400,687 1.4053
14 Feb 2013 21:39:16 1165600 15469700 hadcm3n_zkvu_1880_40_008249149_2 259,200 364,150 1.4049
14 Feb 2013 10:41:07 1165600 15469700 hadcm3n_zkvu_1880_40_008249149_2 233,280 326,624 1.4001
13 Feb 2013 23:23:30 1165600 15469700 hadcm3n_zkvu_1880_40_008249149_2 207,360 290,733 1.4021
13 Feb 2013 13:45:17 1165600 15469700 hadcm3n_zkvu_1880_40_008249149_2 181,440 256,057 1.4112
12 Feb 2013 22:14:00 1165600 15469700 hadcm3n_zkvu_1880_40_008249149_2 155,520 222,960 1.4336
12 Feb 2013 11:30:11 1165600 15469700 hadcm3n_zkvu_1880_40_008249149_2 129,600 186,728 1.4408
11 Feb 2013 18:01:40 1165600 15469700 hadcm3n_zkvu_1880_40_008249149_2 103,680 151,714 1.4633
11 Feb 2013 07:00:48 1165600 15469700 hadcm3n_zkvu_1880_40_008249149_2 77,760 113,404 1.4584
10 Feb 2013 19:44:58 1165600 15469700 hadcm3n_zkvu_1880_40_008249149_2 51,840 75,507 1.4565
10 Feb 2013 05:46:15 1165600 15469700 hadcm3n_zkvu_1880_40_008249149_2 25,920 37,893 1.4619


©2024 climateprediction.net