climateprediction.net home page
Task 15471855

Task 15471855

Name hadcm3n_zb7d_1880_40_008253943_2
Workunit 8409067
Created 6 Dec 2012, 2:23:00 UTC
Sent 6 Dec 2012, 2:23:08 UTC
Report deadline 7 Mar 2013, 9:50:19 UTC
Received 6 Jun 2013, 7:08:09 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 193 (0x000000C1) EXIT_SIGNAL
Computer ID 1120190
Run time 10 days 23 hours 25 min 25 sec
CPU time 10 days 21 hours 41 min 32 sec
Validate state Invalid
Credit 6,220.80
Device peak FLOPS 1.99 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>7.0.27</core_client_version>
<![CDATA[
<message>
process exited with code 193 (0xc1, -63)
</message>
<stderr_txt>
ing
09:00:39 (2382): No heartbeat from core client for 30 sec - exiting
09:00:40 (2382): No heartbeat from core client for 30 sec - exiting
09:00:41 (2382): No heartbeat from core client for 30 sec - exiting
09:00:42 (2382): No heartbeat from core client for 30 sec - exiting
09:00:43 (2382): No heartbeat from core client for 30 sec - exiting
09:00:44 (2382): No heartbeat from core client for 30 sec - exiting
09:00:45 (2382): No heartbeat from core client for 30 sec - exiting
09:00:46 (2382): No heartbeat from core client for 30 sec - exiting
09:00:47 (2382): No heartbeat from core client for 30 sec - exiting
09:00:48 (2382): No heartbeat from core client for 30 sec - exiting
09:00:51 (2382): No heartbeat from core client for 30 sec - exiting
09:00:52 (2382): No heartbeat from core client for 30 sec - exiting
09:00:53 (2382): No heartbeat from core client for 30 sec - exiting
09:00:54 (2382): No heartbeat from core client for 30 sec - exiting
09:00:55 (2382): No heartbeat from core client for 30 sec - exiting
09:00:56 (2382): No heartbeat from core client for 30 sec - exiting
09:00:57 (2382): No heartbeat from core client for 30 sec - exiting
09:00:58 (2382): No heartbeat from core client for 30 sec - exiting
09:01:01 (2382): No heartbeat from core client for 30 sec - exiting
09:01:02 (2382): No heartbeat from core client for 30 sec - exiting
09:01:03 (2382): No heartbeat from core client for 30 sec - exiting
09:01:04 (2382): No heartbeat from core client for 30 sec - exiting
09:01:05 (2382): No heartbeat from core client for 30 sec - exiting
09:01:06 (2382): No heartbeat from core client for 30 sec - exiting
09:01:07 (2382): No heartbeat from core client for 30 sec - exiting
09:01:08 (2382): No heartbeat from core client for 30 sec - exiting
09:01:09 (2382): No heartbeat from core client for 30 sec - exiting
09:01:10 (2382): No heartbeat from core client for 30 sec - exiting
09:01:11 (2382): No heartbeat from core client for 30 sec - exiting
09:01:12 (2382): No heartbeat from core client for 30 sec - exiting
09:01:13 (2382): No heartbeat from core client for 30 sec - exiting
09:01:14 (2382): No heartbeat from core client for 30 sec - exiting
09:01:15 (2382): No heartbeat from core client for 30 sec - exiting
09:01:16 (2382): No heartbeat from core client for 30 sec - exiting
09:01:17 (2382): No heartbeat from core client for 30 sec - exiting
09:01:18 (2382): No heartbeat from core client for 30 sec - exiting
09:01:19 (2382): No heartbeat from core client for 30 sec - exiting
09:01:20 (2382): No heartbeat from core client for 30 sec - exiting
09:01:21 (2382): No heartbeat from core client for 30 sec - exiting
09:01:22 (2382): No heartbeat from core client for 30 sec - exiting
09:01:23 (2382): No heartbeat from core client for 30 sec - exiting
09:01:24 (2382): No heartbeat from core client for 30 sec - exiting
09:01:25 (2382): No heartbeat from core client for 30 sec - exiting
09:01:26 (2382): No heartbeat from core client for 30 sec - exiting
09:01:27 (2382): No heartbeat from core client for 30 sec - exiting
09:01:28 (2382): No heartbeat from core client for 30 sec - exiting
09:01:29 (2382): No heartbeat from core client for 30 sec - exiting
09:01:30 (2382): No heartbeat from core client for 30 sec - exiting
09:01:31 (2382): No heartbeat from core client for 30 sec - exiting
09:01:32 (2382): No heartbeat from core client for 30 sec - exiting
09:01:33 (2382): No heartbeat from core client for 30 sec - exiting
09:01:34 (2382): No heartbeat from core client for 30 sec - exiting
09:01:35 (2382): No heartbeat from core client for 30 sec - exiting
09:01:36 (2382): No heartbeat from core client for 30 sec - exiting
09:01:37 (2382): No heartbeat from core client for 30 sec - exiting
09:01:38 (2382): No heartbeat from core client for 30 sec - exiting
09:01:39 (2382): No heartbeat from core client for 30 sec - exiting
09:01:40 (2382): No heartbeat from core client for 30 sec - exiting
09:01:41 (2382): No heartbeat from core client for 30 sec - exiting
09:01:42 (2382): No heartbeat from core client for 30 sec - exiting
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09619768 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
09:01:43 (2382): No heartbeat from core client for 30 sec - exiting
09:01:44 (2382): No heartbeat from core client for 30 sec - exiting
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08bae768 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
09:08:22 (2377): No heartbeat from core client for 30 sec - exiting
09:08:28 (2377): No heartbeat from core client for 30 sec - exiting
09:08:29 (2377): No heartbeat from core client for 30 sec - exiting
09:08:30 (2377): No heartbeat from core client for 30 sec - exiting
09:08:31 (2377): No heartbeat from core client for 30 sec - exiting
09:08:32 (2377): No heartbeat from core client for 30 sec - exiting
09:08:35 (2377): No heartbeat from core client for 30 sec - exiting
09:08:38 (2377): No heartbeat from core client for 30 sec - exiting
09:08:39 (2377): No heartbeat from core client for 30 sec - exiting
09:08:40 (2377): No heartbeat from core client for 30 sec - exiting
09:08:41 (2377): No heartbeat from core client for 30 sec - exiting
09:08:42 (2377): No heartbeat from core client for 30 sec - exiting
09:08:43 (2377): No heartbeat from core client for 30 sec - exiting
09:08:44 (2377): No heartbeat from core client for 30 sec - exiting
09:08:45 (2377): No heartbeat from core client for 30 sec - exiting
09:08:46 (2377): No heartbeat from core client for 30 sec - exiting
09:08:47 (2377): No heartbeat from core client for 30 sec - exiting
09:08:48 (2377): No heartbeat from core client for 30 sec - exiting
09:08:49 (2377): No heartbeat from core client for 30 sec - exiting
09:08:50 (2377): No heartbeat from core client for 30 sec - exiting
09:08:51 (2377): No heartbeat from core client for 30 sec - exiting
09:08:52 (2377): No heartbeat from core client for 30 sec - exiting
09:08:53 (2377): No heartbeat from core client for 30 sec - exiting
09:08:54 (2377): No heartbeat from core client for 30 sec - exiting
09:08:55 (2377): No heartbeat from core client for 30 sec - exiting
09:08:56 (2377): No heartbeat from core client for 30 sec - exiting
09:08:57 (2377): No heartbeat from core client for 30 sec - exiting
09:08:58 (2377): No heartbeat from core client for 30 sec - exiting
09:08:59 (2377): No heartbeat from core client for 30 sec - exiting
09:09:00 (2377): No heartbeat from core client for 30 sec - exiting
09:09:01 (2377): No heartbeat from core client for 30 sec - exiting
09:09:02 (2377): No heartbeat from core client for 30 sec - exiting
09:09:03 (2377): No heartbeat from core client for 30 sec - exiting
09:09:04 (2377): No heartbeat from core client for 30 sec - exiting
09:09:05 (2377): No heartbeat from core client for 30 sec - exiting
09:09:06 (2377): No heartbeat from core client for 30 sec - exiting
09:09:07 (2377): No heartbeat from core client for 30 sec - exiting
09:09:08 (2377): No heartbeat from core client for 30 sec - exiting
09:09:09 (2377): No heartbeat from core client for 30 sec - exiting
09:09:10 (2377): No heartbeat from core client for 30 sec - exiting
09:09:11 (2377): No heartbeat from core client for 30 sec - exiting
09:09:12 (2377): No heartbeat from core client for 30 sec - exiting
09:09:13 (2377): No heartbeat from core client for 30 sec - exiting
09:09:14 (2377): No heartbeat from core client for 30 sec - exiting
09:09:15 (2377): No heartbeat from core client for 30 sec - exiting
09:09:16 (2377): No heartbeat from core client for 30 sec - exiting
09:09:17 (2377): No heartbeat from core client for 30 sec - exiting
09:09:18 (2377): No heartbeat from core client for 30 sec - exiting
09:09:19 (2377): No heartbeat from core client for 30 sec - exiting
09:09:20 (2377): No heartbeat from core client for 30 sec - exiting
09:09:21 (2377): No heartbeat from core client for 30 sec - exiting
09:09:22 (2377): No heartbeat from core client for 30 sec - exiting
09:09:23 (2377): No heartbeat from core client for 30 sec - exiting
09:09:24 (2377): No heartbeat from core client for 30 sec - exiting
09:09:25 (2377): No heartbeat from core client for 30 sec - exiting
09:09:26 (2377): No heartbeat from core client for 30 sec - exiting
09:09:27 (2377): No heartbeat from core client for 30 sec - exiting
09:09:29 (2377): No heartbeat from core client for 30 sec - exiting
09:09:30 (2377): No heartbeat from core client for 30 sec - exiting
09:09:31 (2377): No heartbeat from core client for 30 sec - exiting
09:09:32 (2377): No heartbeat from core client for 30 sec - exiting
09:09:33 (2377): No heartbeat from core client for 30 sec - exiting
09:09:34 (2377): No heartbeat from core client for 30 sec - exiting
09:09:36 (2377): No heartbeat from core client for 30 sec - exiting
09:09:37 (2377): No heartbeat from core client for 30 sec - exiting
09:09:38 (2377): No heartbeat from core client for 30 sec - exiting
09:09:39 (2377): No heartbeat from core client for 30 sec - exiting
09:09:40 (2377): No heartbeat from core client for 30 sec - exiting
09:09:41 (2377): No heartbeat from core client for 30 sec - exiting
09:09:42 (2377): No heartbeat from core client for 30 sec - exiting
09:09:45 (2377): No heartbeat from core client for 30 sec - exiting
09:09:46 (2377): No heartbeat from core client for 30 sec - exiting
09:09:47 (2377): No heartbeat from core client for 30 sec - exiting
09:09:48 (2377): No heartbeat from core client for 30 sec - exiting
09:09:49 (2377): No heartbeat from core client for 30 sec - exiting
09:09:50 (2377): No heartbeat from core client for 30 sec - exiting
09:09:51 (2377): No heartbeat from core client for 30 sec - exiting
09:09:52 (2377): No heartbeat from core client for 30 sec - exiting
09:09:53 (2377): No heartbeat from core client for 30 sec - exiting
09:09:54 (2377): No heartbeat from core client for 30 sec - exiting
09:09:55 (2377): No heartbeat from core client for 30 sec - exiting
09:09:56 (2377): No heartbeat from core client for 30 sec - exiting
09:09:57 (2377): No heartbeat from core client for 30 sec - exiting
09:09:58 (2377): No heartbeat from core client for 30 sec - exiting
09:09:59 (2377): No heartbeat from core client for 30 sec - exiting
09:10:00 (2377): No heartbeat from core client for 30 sec - exiting
09:10:01 (2377): No heartbeat from core client for 30 sec - exiting
09:10:02 (2377): No heartbeat from core client for 30 sec - exiting
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x094bc768 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
09:10:03 (2377): No heartbeat from core client for 30 sec - exiting
09:10:04 (2377): No heartbeat from core client for 30 sec - exiting
09:18:22 (2377): No heartbeat from core client for 30 sec - exiting
09:18:26 (2377): No heartbeat from core client for 30 sec - exiting
09:18:27 (2377): No heartbeat from core client for 30 sec - exiting
09:18:28 (2377): No heartbeat from core client for 30 sec - exiting
09:18:29 (2377): No heartbeat from core client for 30 sec - exiting
09:18:30 (2377): No heartbeat from core client for 30 sec - exiting
09:18:31 (2377): No heartbeat from core client for 30 sec - exiting
09:18:32 (2377): No heartbeat from core client for 30 sec - exiting
09:18:33 (2377): No heartbeat from core client for 30 sec - exiting
09:18:34 (2377): No heartbeat from core client for 30 sec - exiting
09:18:35 (2377): No heartbeat from core client for 30 sec - exiting
09:18:38 (2377): No heartbeat from core client for 30 sec - exiting
09:18:39 (2377): No heartbeat from core client for 30 sec - exiting
09:18:40 (2377): No heartbeat from core client for 30 sec - exiting
09:18:52 (2377): No heartbeat from core client for 30 sec - exiting
09:19:02 (2377): No heartbeat from core client for 30 sec - exiting
09:19:11 (2377): No heartbeat from core client for 30 sec - exiting
09:19:21 (2377): No heartbeat from core client for 30 sec - exiting
09:19:22 (2377): No heartbeat from core client for 30 sec - exiting
09:19:23 (2377): No heartbeat from core client for 30 sec - exiting
09:19:24 (2377): No heartbeat from core client for 30 sec - exiting
09:19:25 (2377): No heartbeat from core client for 30 sec - exiting
09:19:26 (2377): No heartbeat from core client for 30 sec - exiting
09:19:27 (2377): No heartbeat from core client for 30 sec - exiting
09:19:28 (2377): No heartbeat from core client for 30 sec - exiting
09:19:29 (2377): No heartbeat from core client for 30 sec - exiting
09:19:30 (2377): No heartbeat from core client for 30 sec - exiting
09:19:31 (2377): No heartbeat from core client for 30 sec - exiting
09:19:32 (2377): No heartbeat from core client for 30 sec - exiting
09:19:33 (2377): No heartbeat from core client for 30 sec - exiting
09:19:34 (2377): No heartbeat from core client for 30 sec - exiting
09:19:35 (2377): No heartbeat from core client for 30 sec - exiting
09:19:36 (2377): No heartbeat from core client for 30 sec - exiting
09:19:37 (2377): No heartbeat from core client for 30 sec - exiting
09:19:38 (2377): No heartbeat from core client for 30 sec - exiting
09:19:39 (2377): No heartbeat from core client for 30 sec - exiting
09:19:40 (2377): No heartbeat from core client for 30 sec - exiting
09:19:41 (2377): No heartbeat from core client for 30 sec - exiting
09:19:42 (2377): No heartbeat from core client for 30 sec - exiting
09:19:43 (2377): No heartbeat from core client for 30 sec - exiting
09:19:44 (2377): No heartbeat from core client for 30 sec - exiting
09:19:45 (2377): No heartbeat from core client for 30 sec - exiting
09:19:46 (2377): No heartbeat from core client for 30 sec - exiting
09:19:47 (2377): No heartbeat from core client for 30 sec - exiting
09:19:48 (2377): No heartbeat from core client for 30 sec - exiting
09:19:49 (2377): No heartbeat from core client for 30 sec - exiting
09:19:50 (2377): No heartbeat from core client for 30 sec - exiting
09:19:51 (2377): No heartbeat from core client for 30 sec - exiting
09:19:52 (2377): No heartbeat from core client for 30 sec - exiting
09:19:53 (2377): No heartbeat from core client for 30 sec - exiting
09:19:54 (2377): No heartbeat from core client for 30 sec - exiting
09:19:55 (2377): No heartbeat from core client for 30 sec - exiting
09:19:56 (2377): No heartbeat from core client for 30 sec - exiting
09:19:57 (2377): No heartbeat from core client for 30 sec - exiting
09:19:58 (2377): No heartbeat from core client for 30 sec - exiting
09:19:59 (2377): No heartbeat from core client for 30 sec - exiting
09:20:00 (2377): No heartbeat from core client for 30 sec - exiting
09:20:04 (2377): No heartbeat from core client for 30 sec - exiting
09:20:05 (2377): No heartbeat from core client for 30 sec - exiting
09:20:06 (2377): No heartbeat from core client for 30 sec - exiting
09:20:07 (2377): No heartbeat from core client for 30 sec - exiting
09:20:08 (2377): No heartbeat from core client for 30 sec - exiting
09:20:09 (2377): No heartbeat from core client for 30 sec - exiting
09:20:10 (2377): No heartbeat from core client for 30 sec - exiting
09:20:11 (2377): No heartbeat from core client for 30 sec - exiting
09:20:12 (2377): No heartbeat from core client for 30 sec - exiting
09:20:13 (2377): No heartbeat from core client for 30 sec - exiting
09:20:14 (2377): No heartbeat from core client for 30 sec - exiting
09:20:15 (2377): No heartbeat from core client for 30 sec - exiting
09:20:16 (2377): No heartbeat from core client for 30 sec - exiting
09:20:17 (2377): No heartbeat from core client for 30 sec - exiting
09:20:18 (2377): No heartbeat from core client for 30 sec - exiting
09:20:19 (2377): No heartbeat from core client for 30 sec - exiting
09:20:20 (2377): No heartbeat from core client for 30 sec - exiting
09:20:21 (2377): No heartbeat from core client for 30 sec - exiting
09:20:22 (2377): No heartbeat from core client for 30 sec - exiting
09:20:23 (2377): No heartbeat from core client for 30 sec - exiting
09:20:27 (2377): No heartbeat from core client for 30 sec - exiting
09:20:28 (2377): No heartbeat from core client for 30 sec - exiting
09:20:29 (2377): No heartbeat from core client for 30 sec - exiting
09:20:30 (2377): No heartbeat from core client for 30 sec - exiting
09:20:31 (2377): No heartbeat from core client for 30 sec - exiting
09:20:32 (2377): No heartbeat from core client for 30 sec - exiting
09:20:33 (2377): No heartbeat from core client for 30 sec - exiting
09:20:34 (2377): No heartbeat from core client for 30 sec - exiting
09:20:35 (2377): No heartbeat from core client for 30 sec - exiting
09:20:36 (2377): No heartbeat from core client for 30 sec - exiting
09:20:37 (2377): No heartbeat from core client for 30 sec - exiting
09:20:48 (2377): No heartbeat from core client for 30 sec - exiting
09:21:01 (2377): No heartbeat from core client for 30 sec - exiting
09:21:03 (2377): No heartbeat from core client for 30 sec - exiting
09:21:04 (2377): No heartbeat from core client for 30 sec - exiting
09:21:05 (2377): No heartbeat from core client for 30 sec - exiting
09:21:06 (2377): No heartbeat from core client for 30 sec - exiting
09:21:07 (2377): No heartbeat from core client for 30 sec - exiting
09:21:08 (2377): No heartbeat from core client for 30 sec - exiting
09:21:09 (2377): No heartbeat from core client for 30 sec - exiting
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09fd1768 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08e8c768 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x094c4768 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08b83768 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
09:09:12 (2421): No heartbeat from core client for 30 sec - exiting
09:09:43 (2421): No heartbeat from core client for 30 sec - exiting
09:09:44 (2421): No heartbeat from core client for 30 sec - exiting
09:09:45 (2421): No heartbeat from core client for 30 sec - exiting
09:09:46 (2421): No heartbeat from core client for 30 sec - exiting
09:09:50 (2421): No heartbeat from core client for 30 sec - exiting
09:09:51 (2421): No heartbeat from core client for 30 sec - exiting
09:10:02 (2421): No heartbeat from core client for 30 sec - exiting
09:10:06 (2421): No heartbeat from core client for 30 sec - exiting
09:14:31 (2382): No heartbeat from core client for 30 sec - exiting
09:14:33 (2382): No heartbeat from core client for 30 sec - exiting
09:14:34 (2382): No heartbeat from core client for 30 sec - exiting
09:14:35 (2382): No heartbeat from core client for 30 sec - exiting
09:14:36 (2382): No heartbeat from core client for 30 sec - exiting
09:14:37 (2382): No heartbeat from core client for 30 sec - exiting
09:14:38 (2382): No heartbeat from core client for 30 sec - exiting
09:14:39 (2382): No heartbeat from core client for 30 sec - exiting
09:14:40 (2382): No heartbeat from core client for 30 sec - exiting
09:14:41 (2382): No heartbeat from core client for 30 sec - exiting
09:14:42 (2382): No heartbeat from core client for 30 sec - exiting
09:14:43 (2382): No heartbeat from core client for 30 sec - exiting
09:14:44 (2382): No heartbeat from core client for 30 sec - exiting
09:14:45 (2382): No heartbeat from core client for 30 sec - exiting
09:14:46 (2382): No heartbeat from core client for 30 sec - exiting
09:14:47 (2382): No heartbeat from core client for 30 sec - exiting
09:14:48 (2382): No heartbeat from core client for 30 sec - exiting
09:14:49 (2382): No heartbeat from core client for 30 sec - exiting
09:14:52 (2382): No heartbeat from core client for 30 sec - exiting
09:14:53 (2382): No heartbeat from core client for 30 sec - exiting
09:14:54 (2382): No heartbeat from core client for 30 sec - exiting
09:14:55 (2382): No heartbeat from core client for 30 sec - exiting
09:14:56 (2382): No heartbeat from core client for 30 sec - exiting
09:14:57 (2382): No heartbeat from core client for 30 sec - exiting
09:14:58 (2382): No heartbeat from core client for 30 sec - exiting
09:14:59 (2382): No heartbeat from core client for 30 sec - exiting
09:15:00 (2382): No heartbeat from core client for 30 sec - exiting
09:15:01 (2382): No heartbeat from core client for 30 sec - exiting
09:15:02 (2382): No heartbeat from core client for 30 sec - exiting
09:15:03 (2382): No heartbeat from core client for 30 sec - exiting
09:15:04 (2382): No heartbeat from core client for 30 sec - exiting
09:15:05 (2382): No heartbeat from core client for 30 sec - exiting
09:15:06 (2382): No heartbeat from core client for 30 sec - exiting
09:15:07 (2382): No heartbeat from core client for 30 sec - exiting
09:15:08 (2382): No heartbeat from core client for 30 sec - exiting
09:15:09 (2382): No heartbeat from core client for 30 sec - exiting
09:15:10 (2382): No heartbeat from core client for 30 sec - exiting
09:15:11 (2382): No heartbeat from core client for 30 sec - exiting
09:15:12 (2382): No heartbeat from core client for 30 sec - exiting
09:15:13 (2382): No heartbeat from core client for 30 sec - exiting
09:15:14 (2382): No heartbeat from core client for 30 sec - exiting
09:15:15 (2382): No heartbeat from core client for 30 sec - exiting
09:15:16 (2382): No heartbeat from core client for 30 sec - exiting
09:15:17 (2382): No heartbeat from core client for 30 sec - exiting
09:15:18 (2382): No heartbeat from core client for 30 sec - exiting
09:15:20 (2382): No heartbeat from core client for 30 sec - exiting
09:15:21 (2382): No heartbeat from core client for 30 sec - exiting
09:15:22 (2382): No heartbeat from core client for 30 sec - exiting
09:15:24 (2382): No heartbeat from core client for 30 sec - exiting
09:15:25 (2382): No heartbeat from core client for 30 sec - exiting
09:15:26 (2382): No heartbeat from core client for 30 sec - exiting
09:15:27 (2382): No heartbeat from core client for 30 sec - exiting
09:15:28 (2382): No heartbeat from core client for 30 sec - exiting
09:15:29 (2382): No heartbeat from core client for 30 sec - exiting
09:15:30 (2382): No heartbeat from core client for 30 sec - exiting
09:15:32 (2382): No heartbeat from core client for 30 sec - exiting
09:15:33 (2382): No heartbeat from core client for 30 sec - exiting
09:15:34 (2382): No heartbeat from core client for 30 sec - exiting
09:15:35 (2382): No heartbeat from core client for 30 sec - exiting
09:15:38 (2382): No heartbeat from core client for 30 sec - exiting
09:15:39 (2382): No heartbeat from core client for 30 sec - exiting
09:15:40 (2382): No heartbeat from core client for 30 sec - exiting
09:15:41 (2382): No heartbeat from core client for 30 sec - exiting
09:15:42 (2382): No heartbeat from core client for 30 sec - exiting
09:15:43 (2382): No heartbeat from core client for 30 sec - exiting
09:15:44 (2382): No heartbeat from core client for 30 sec - exiting
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08bae768 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
09:15:45 (2382): No heartbeat from core client for 30 sec - exiting
09:15:46 (2382): No heartbeat from core client for 30 sec - exiting
09:15:47 (2382): No heartbeat from core client for 30 sec - exiting
11:57:08 (2351): No heartbeat from core client for 30 sec - exiting
11:57:21 (2351): No heartbeat from core client for 30 sec - exiting
11:57:22 (2351): No heartbeat from core client for 30 sec - exiting
11:57:23 (2351): No heartbeat from core client for 30 sec - exiting
11:57:24 (2351): No heartbeat from core client for 30 sec - exiting
11:57:25 (2351): No heartbeat from core client for 30 sec - exiting
11:57:26 (2351): No heartbeat from core client for 30 sec - exiting
11:57:27 (2351): No heartbeat from core client for 30 sec - exiting
11:57:28 (2351): No heartbeat from core client for 30 sec - exiting
11:57:29 (2351): No heartbeat from core client for 30 sec - exiting
11:57:30 (2351): No heartbeat from core client for 30 sec - exiting
11:57:31 (2351): No heartbeat from core client for 30 sec - exiting
11:57:32 (2351): No heartbeat from core client for 30 sec - exiting
11:57:34 (2351): No heartbeat from core client for 30 sec - exiting
11:57:37 (2351): No heartbeat from core client for 30 sec - exiting
11:57:38 (2351): No heartbeat from core client for 30 sec - exiting
11:57:39 (2351): No heartbeat from core client for 30 sec - exiting
11:57:40 (2351): No heartbeat from core client for 30 sec - exiting
11:57:41 (2351): No heartbeat from core client for 30 sec - exiting
11:57:42 (2351): No heartbeat from core client for 30 sec - exiting
11:57:43 (2351): No heartbeat from core client for 30 sec - exiting
11:57:44 (2351): No heartbeat from core client for 30 sec - exiting
11:57:45 (2351): No heartbeat from core client for 30 sec - exiting
11:57:47 (2351): No heartbeat from core client for 30 sec - exiting
11:57:48 (2351): No heartbeat from core client for 30 sec - exiting
11:57:49 (2351): No heartbeat from core client for 30 sec - exiting
11:57:50 (2351): No heartbeat from core client for 30 sec - exiting
11:57:51 (2351): No heartbeat from core client for 30 sec - exiting
11:57:52 (2351): No heartbeat from core client for 30 sec - exiting
11:57:53 (2351): No heartbeat from core client for 30 sec - exiting
11:57:54 (2351): No heartbeat from core client for 30 sec - exiting
11:57:55 (2351): No heartbeat from core client for 30 sec - exiting
11:57:56 (2351): No heartbeat from core client for 30 sec - exiting
11:57:57 (2351): No heartbeat from core client for 30 sec - exiting
11:57:58 (2351): No heartbeat from core client for 30 sec - exiting
11:57:59 (2351): No heartbeat from core client for 30 sec - exiting
11:58:00 (2351): No heartbeat from core client for 30 sec - exiting
11:58:01 (2351): No heartbeat from core client for 30 sec - exiting
11:58:02 (2351): No heartbeat from core client for 30 sec - exiting
11:58:03 (2351): No heartbeat from core client for 30 sec - exiting
11:58:04 (2351): No heartbeat from core client for 30 sec - exiting
11:58:05 (2351): No heartbeat from core client for 30 sec - exiting
11:58:06 (2351): No heartbeat from core client for 30 sec - exiting
11:58:07 (2351): No heartbeat from core client for 30 sec - exiting
11:58:13 (2351): No heartbeat from core client for 30 sec - exiting
11:58:14 (2351): No heartbeat from core client for 30 sec - exiting
11:58:15 (2351): No heartbeat from core client for 30 sec - exiting
11:58:16 (2351): No heartbeat from core client for 30 sec - exiting
11:58:17 (2351): No heartbeat from core client for 30 sec - exiting
11:58:19 (2351): No heartbeat from core client for 30 sec - exiting
11:58:20 (2351): No heartbeat from core client for 30 sec - exiting
11:58:21 (2351): No heartbeat from core client for 30 sec - exiting
11:58:22 (2351): No heartbeat from core client for 30 sec - exiting
11:58:23 (2351): No heartbeat from core client for 30 sec - exiting
11:58:24 (2351): No heartbeat from core client for 30 sec - exiting
11:58:25 (2351): No heartbeat from core client for 30 sec - exiting
11:58:26 (2351): No heartbeat from core client for 30 sec - exiting
11:58:27 (2351): No heartbeat from core client for 30 sec - exiting
11:58:28 (2351): No heartbeat from core client for 30 sec - exiting
11:58:29 (2351): No heartbeat from core client for 30 sec - exiting
11:58:31 (2351): No heartbeat from core client for 30 sec - exiting
11:58:32 (2351): No heartbeat from core client for 30 sec - exiting
11:58:33 (2351): No heartbeat from core client for 30 sec - exiting
11:58:34 (2351): No heartbeat from core client for 30 sec - exiting
11:58:35 (2351): No heartbeat from core client for 30 sec - exiting
11:58:36 (2351): No heartbeat from core client for 30 sec - exiting
11:58:37 (2351): No heartbeat from core client for 30 sec - exiting
11:58:38 (2351): No heartbeat from core client for 30 sec - exiting
11:58:39 (2351): No heartbeat from core client for 30 sec - exiting
11:58:40 (2351): No heartbeat from core client for 30 sec - exiting
11:58:41 (2351): No heartbeat from core client for 30 sec - exiting
11:58:42 (2351): No heartbeat from core client for 30 sec - exiting
11:58:43 (2351): No heartbeat from core client for 30 sec - exiting
11:58:44 (2351): No heartbeat from core client for 30 sec - exiting
11:58:45 (2351): No heartbeat from core client for 30 sec - exiting
11:58:46 (2351): No heartbeat from core client for 30 sec - exiting
11:58:47 (2351): No heartbeat from core client for 30 sec - exiting
11:58:48 (2351): No heartbeat from core client for 30 sec - exiting
11:58:49 (2351): No heartbeat from core client for 30 sec - exiting
11:58:50 (2351): No heartbeat from core client for 30 sec - exiting
11:58:51 (2351): No heartbeat from core client for 30 sec - exiting
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09b81768 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
11:58:52 (2351): No heartbeat from core client for 30 sec - exiting
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09d50768 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
08:56:02 (2438): No heartbeat from core client for 30 sec - exiting
08:56:03 (2438): No heartbeat from core client for 30 sec - exiting
08:56:04 (2438): No heartbeat from core client for 30 sec - exiting
08:56:05 (2438): No heartbeat from core client for 30 sec - exiting
16:42:18 (2438): No heartbeat from core client for 30 sec - exiting
16:42:19 (2438): No heartbeat from core client for 30 sec - exiting
16:42:20 (2438): No heartbeat from core client for 30 sec - exiting
16:42:21 (2438): No heartbeat from core client for 30 sec - exiting
16:42:22 (2438): No heartbeat from core client for 30 sec - exiting
16:42:23 (2438): No heartbeat from core client for 30 sec - exiting
16:42:24 (2438): No heartbeat from core client for 30 sec - exiting
16:42:25 (2438): No heartbeat from core client for 30 sec - exiting
16:42:26 (2438): No heartbeat from core client for 30 sec - exiting
16:42:27 (2438): No heartbeat from core client for 30 sec - exiting
16:42:28 (2438): No heartbeat from core client for 30 sec - exiting
16:42:29 (2438): No heartbeat from core client for 30 sec - exiting
16:42:30 (2438): No heartbeat from core client for 30 sec - exiting
16:42:31 (2438): No heartbeat from core client for 30 sec - exiting
16:42:32 (2438): No heartbeat from core client for 30 sec - exiting
16:42:33 (2438): No heartbeat from core client for 30 sec - exiting
16:42:34 (2438): No heartbeat from core client for 30 sec - exiting
16:42:35 (2438): No heartbeat from core client for 30 sec - exiting
16:42:36 (2438): No heartbeat from core client for 30 sec - exiting
16:42:37 (2438): No heartbeat from core client for 30 sec - exiting
16:42:38 (2438): No heartbeat from core client for 30 sec - exiting
16:42:39 (2438): No heartbeat from core client for 30 sec - exiting
16:42:40 (2438): No heartbeat from core client for 30 sec - exiting
16:42:41 (2438): No heartbeat from core client for 30 sec - exiting
16:42:42 (2438): No heartbeat from core client for 30 sec - exiting
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x082fe768 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
09:09:01 (2363): No heartbeat from core client for 30 sec - exiting
09:09:02 (2363): No heartbeat from core client for 30 sec - exiting
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08467768 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x0a0d9768 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
09:12:07 (2398): No heartbeat from core client for 30 sec - exiting
09:12:08 (2398): No heartbeat from core client for 30 sec - exiting
09:12:09 (2398): No heartbeat from core client for 30 sec - exiting
09:12:10 (2398): No heartbeat from core client for 30 sec - exiting
09:12:11 (2398): No heartbeat from core client for 30 sec - exiting
09:12:12 (2398): No heartbeat from core client for 30 sec - exiting
09:12:13 (2398): No heartbeat from core client for 30 sec - exiting
09:12:14 (2398): No heartbeat from core client for 30 sec - exiting
09:12:15 (2398): No heartbeat from core client for 30 sec - exiting
09:12:16 (2398): No heartbeat from core client for 30 sec - exiting
09:12:17 (2398): No heartbeat from core client for 30 sec - exiting
09:12:18 (2398): No heartbeat from core client for 30 sec - exiting
09:12:19 (2398): No heartbeat from core client for 30 sec - exiting
09:12:20 (2398): No heartbeat from core client for 30 sec - exiting
09:12:21 (2398): No heartbeat from core client for 30 sec - exiting
09:12:22 (2398): No heartbeat from core client for 30 sec - exiting
09:12:23 (2398): No heartbeat from core client for 30 sec - exiting
09:12:24 (2398): No heartbeat from core client for 30 sec - exiting
09:12:25 (2398): No heartbeat from core client for 30 sec - exiting
09:12:26 (2398): No heartbeat from core client for 30 sec - exiting
09:12:27 (2398): No heartbeat from core client for 30 sec - exiting
09:12:28 (2398): No heartbeat from core client for 30 sec - exiting
09:12:29 (2398): No heartbeat from core client for 30 sec - exiting
09:12:30 (2398): No heartbeat from core client for 30 sec - exiting
09:12:31 (2398): No heartbeat from core client for 30 sec - exiting
09:12:32 (2398): No heartbeat from core client for 30 sec - exiting
09:12:33 (2398): No heartbeat from core client for 30 sec - exiting
09:12:34 (2398): No heartbeat from core client for 30 sec - exiting
09:12:35 (2398): No heartbeat from core client for 30 sec - exiting
09:11:24 (2433): No heartbeat from core client for 30 sec - exiting
09:11:28 (2433): No heartbeat from core client for 30 sec - exiting
09:11:29 (2433): No heartbeat from core client for 30 sec - exiting
09:11:30 (2433): No heartbeat from core client for 30 sec - exiting
09:11:31 (2433): No heartbeat from core client for 30 sec - exiting
09:11:32 (2433): No heartbeat from core client for 30 sec - exiting
09:11:33 (2433): No heartbeat from core client for 30 sec - exiting
09:11:34 (2433): No heartbeat from core client for 30 sec - exiting
09:11:36 (2433): No heartbeat from core client for 30 sec - exiting
09:11:37 (2433): No heartbeat from core client for 30 sec - exiting
09:11:38 (2433): No heartbeat from core client for 30 sec - exiting
09:11:39 (2433): No heartbeat from core client for 30 sec - exiting
09:11:40 (2433): No heartbeat from core client for 30 sec - exiting
09:11:41 (2433): No heartbeat from core client for 30 sec - exiting
09:11:42 (2433): No heartbeat from core client for 30 sec - exiting
09:11:43 (2433): No heartbeat from core client for 30 sec - exiting
09:11:44 (2433): No heartbeat from core client for 30 sec - exiting
09:11:45 (2433): No heartbeat from core client for 30 sec - exiting
09:11:46 (2433): No heartbeat from core client for 30 sec - exiting
09:11:48 (2433): No heartbeat from core client for 30 sec - exiting
09:11:49 (2433): No heartbeat from core client for 30 sec - exiting
09:11:50 (2433): No heartbeat from core client for 30 sec - exiting
09:11:51 (2433): No heartbeat from core client for 30 sec - exiting
09:11:52 (2433): No heartbeat from core client for 30 sec - exiting
09:11:53 (2433): No heartbeat from core client for 30 sec - exiting
09:11:54 (2433): No heartbeat from core client for 30 sec - exiting
09:11:56 (2433): No heartbeat from core client for 30 sec - exiting
09:11:57 (2433): No heartbeat from core client for 30 sec - exiting
09:11:58 (2433): No heartbeat from core client for 30 sec - exiting
09:11:59 (2433): No heartbeat from core client for 30 sec - exiting
09:12:00 (2433): No heartbeat from core client for 30 sec - exiting
09:12:01 (2433): No heartbeat from core client for 30 sec - exiting
09:12:02 (2433): No heartbeat from core client for 30 sec - exiting
09:12:03 (2433): No heartbeat from core client for 30 sec - exiting
09:12:04 (2433): No heartbeat from core client for 30 sec - exiting
09:12:05 (2433): No heartbeat from core client for 30 sec - exiting
09:12:06 (2433): No heartbeat from core client for 30 sec - exiting
09:12:07 (2433): No heartbeat from core client for 30 sec - exiting
09:12:08 (2433): No heartbeat from core client for 30 sec - exiting
09:12:09 (2433): No heartbeat from core client for 30 sec - exiting
09:12:10 (2433): No heartbeat from core client for 30 sec - exiting
09:12:11 (2433): No heartbeat from core client for 30 sec - exiting
09:12:12 (2433): No heartbeat from core client for 30 sec - exiting
09:12:13 (2433): No heartbeat from core client for 30 sec - exiting
09:12:14 (2433): No heartbeat from core client for 30 sec - exiting
09:12:16 (2433): No heartbeat from core client for 30 sec - exiting
09:12:17 (2433): No heartbeat from core client for 30 sec - exiting
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x098df768 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
09:12:18 (2433): No heartbeat from core client for 30 sec - exiting
09:12:19 (2433): No heartbeat from core client for 30 sec - exiting
09:12:20 (2433): No heartbeat from core client for 30 sec - exiting
09:12:21 (2433): No heartbeat from core client for 30 sec - exiting
09:12:22 (2433): No heartbeat from core client for 30 sec - exiting
09:12:23 (2433): No heartbeat from core client for 30 sec - exiting
09:12:24 (2433): No heartbeat from core client for 30 sec - exiting
09:12:25 (2433): No heartbeat from core client for 30 sec - exiting
09:12:26 (2433): No heartbeat from core client for 30 sec - exiting
09:12:27 (2433): No heartbeat from core client for 30 sec - exiting
09:12:29 (2433): No heartbeat from core client for 30 sec - exiting
09:12:30 (2433): No heartbeat from core client for 30 sec - exiting
09:12:31 (2433): No heartbeat from core client for 30 sec - exiting
09:12:32 (2433): No heartbeat from core client for 30 sec - exiting
09:12:33 (2433): No heartbeat from core client for 30 sec - exiting
09:12:34 (2433): No heartbeat from core client for 30 sec - exiting
09:12:35 (2433): No heartbeat from core client for 30 sec - exiting
09:12:36 (2433): No heartbeat from core client for 30 sec - exiting
09:12:37 (2433): No heartbeat from core client for 30 sec - exiting
09:12:38 (2433): No heartbeat from core client for 30 sec - exiting
09:12:39 (2433): No heartbeat from core client for 30 sec - exiting
09:12:40 (2433): No heartbeat from core client for 30 sec - exiting
09:12:41 (2433): No heartbeat from core client for 30 sec - exiting
09:12:42 (2433): No heartbeat from core client for 30 sec - exiting
09:12:43 (2433): No heartbeat from core client for 30 sec - exiting
09:12:44 (2433): No heartbeat from core client for 30 sec - exiting
09:12:45 (2433): No heartbeat from core client for 30 sec - exiting
09:12:46 (2433): No heartbeat from core client for 30 sec - exiting
09:12:47 (2433): No heartbeat from core client for 30 sec - exiting
09:12:48 (2433): No heartbeat from core client for 30 sec - exiting
09:12:49 (2433): No heartbeat from core client for 30 sec - exiting
09:12:50 (2433): No heartbeat from core client for 30 sec - exiting
09:12:51 (2433): No heartbeat from core client for 30 sec - exiting
09:12:52 (2433): No heartbeat from core client for 30 sec - exiting
09:12:53 (2433): No heartbeat from core client for 30 sec - exiting
09:12:54 (2433): No heartbeat from core client for 30 sec - exiting
09:12:55 (2433): No heartbeat from core client for 30 sec - exiting
09:12:56 (2433): No heartbeat from core client for 30 sec - exiting
09:12:57 (2433): No heartbeat from core client for 30 sec - exiting
09:12:58 (2433): No heartbeat from core client for 30 sec - exiting
09:12:59 (2433): No heartbeat from core client for 30 sec - exiting
09:13:00 (2433): No heartbeat from core client for 30 sec - exiting
09:13:01 (2433): No heartbeat from core client for 30 sec - exiting
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08c29768 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
09:11:06 (2644): No heartbeat from core client for 30 sec - exiting
09:11:08 (2644): No heartbeat from core client for 30 sec - exiting
09:11:09 (2644): No heartbeat from core client for 30 sec - exiting
09:11:10 (2644): No heartbeat from core client for 30 sec - exiting
09:11:11 (2644): No heartbeat from core client for 30 sec - exiting
09:11:12 (2644): No heartbeat from core client for 30 sec - exiting
09:11:13 (2644): No heartbeat from core client for 30 sec - exiting
09:11:14 (2644): No heartbeat from core client for 30 sec - exiting
09:11:15 (2644): No heartbeat from core client for 30 sec - exiting
09:11:16 (2644): No heartbeat from core client for 30 sec - exiting
09:11:17 (2644): No heartbeat from core client for 30 sec - exiting
09:11:18 (2644): No heartbeat from core client for 30 sec - exiting
09:11:19 (2644): No heartbeat from core client for 30 sec - exiting
09:11:20 (2644): No heartbeat from core client for 30 sec - exiting
09:11:21 (2644): No heartbeat from core client for 30 sec - exiting
09:11:22 (2644): No heartbeat from core client for 30 sec - exiting
09:11:23 (2644): No heartbeat from core client for 30 sec - exiting
09:11:24 (2644): No heartbeat from core client for 30 sec - exiting
09:11:25 (2644): No heartbeat from core client for 30 sec - exiting
09:11:26 (2644): No heartbeat from core client for 30 sec - exiting
09:11:27 (2644): No heartbeat from core client for 30 sec - exiting
09:11:28 (2644): No heartbeat from core client for 30 sec - exiting
09:11:29 (2644): No heartbeat from core client for 30 sec - exiting
09:11:30 (2644): No heartbeat from core client for 30 sec - exiting
09:11:31 (2644): No heartbeat from core client for 30 sec - exiting
09:11:32 (2644): No heartbeat from core client for 30 sec - exiting
09:11:33 (2644): No heartbeat from core client for 30 sec - exiting
09:11:34 (2644): No heartbeat from core client for 30 sec - exiting
09:11:35 (2644): No heartbeat from core client for 30 sec - exiting
09:11:36 (2644): No heartbeat from core client for 30 sec - exiting
09:11:37 (2644): No heartbeat from core client for 30 sec - exiting
09:11:38 (2644): No heartbeat from core client for 30 sec - exiting
09:11:39 (2644): No heartbeat from core client for 30 sec - exiting
09:11:40 (2644): No heartbeat from core client for 30 sec - exiting
09:11:41 (2644): No heartbeat from core client for 30 sec - exiting
09:11:42 (2644): No heartbeat from core client for 30 sec - exiting
09:11:43 (2644): No heartbeat from core client for 30 sec - exiting
09:11:44 (2644): No heartbeat from core client for 30 sec - exiting
09:11:45 (2644): No heartbeat from core client for 30 sec - exiting
09:11:46 (2644): No heartbeat from core client for 30 sec - exiting
09:11:47 (2644): No heartbeat from core client for 30 sec - exiting
09:11:48 (2644): No heartbeat from core client for 30 sec - exiting
09:11:49 (2644): No heartbeat from core client for 30 sec - exiting
09:11:50 (2644): No heartbeat from core client for 30 sec - exiting
09:11:51 (2644): No heartbeat from core client for 30 sec - exiting
09:11:52 (2644): No heartbeat from core client for 30 sec - exiting
09:11:53 (2644): No heartbeat from core client for 30 sec - exiting
09:11:54 (2644): No heartbeat from core client for 30 sec - exiting
09:11:55 (2644): No heartbeat from core client for 30 sec - exiting
09:11:56 (2644): No heartbeat from core client for 30 sec - exiting
09:11:57 (2644): No heartbeat from core client for 30 sec - exiting
09:11:58 (2644): No heartbeat from core client for 30 sec - exiting
09:11:59 (2644): No heartbeat from core client for 30 sec - exiting
09:12:00 (2644): No heartbeat from core client for 30 sec - exiting
09:12:01 (2644): No heartbeat from core client for 30 sec - exiting
09:12:02 (2644): No heartbeat from core client for 30 sec - exiting
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09abf768 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
09:12:03 (2644): No heartbeat from core client for 30 sec - exiting
09:12:04 (2644): No heartbeat from core client for 30 sec - exiting
09:12:05 (2644): No heartbeat from core client for 30 sec - exiting
09:12:06 (2644): No heartbeat from core client for 30 sec - exiting
09:12:07 (2644): No heartbeat from core client for 30 sec - exiting
09:12:09 (2644): No heartbeat from core client for 30 sec - exiting
09:12:10 (2644): No heartbeat from core client for 30 sec - exiting
09:12:11 (2644): No heartbeat from core client for 30 sec - exiting
09:12:12 (2644): No heartbeat from core client for 30 sec - exiting
09:12:13 (2644): No heartbeat from core client for 30 sec - exiting
09:12:14 (2644): No heartbeat from core client for 30 sec - exiting
09:12:15 (2644): No heartbeat from core client for 30 sec - exiting
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08c7f768 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
12:19:27 (2425): No heartbeat from core client for 30 sec - exiting
12:19:29 (2425): No heartbeat from core client for 30 sec - exiting
12:19:30 (2425): No heartbeat from core client for 30 sec - exiting
12:19:31 (2425): No heartbeat from core client for 30 sec - exiting
12:19:32 (2425): No heartbeat from core client for 30 sec - exiting
12:19:33 (2425): No heartbeat from core client for 30 sec - exiting
12:19:34 (2425): No heartbeat from core client for 30 sec - exiting
12:19:35 (2425): No heartbeat from core client for 30 sec - exiting
12:19:36 (2425): No heartbeat from core client for 30 sec - exiting
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x0864b768 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
09:20:08 (2666): No heartbeat from core client for 30 sec - exiting
09:20:10 (2666): No heartbeat from core client for 30 sec - exiting
09:20:11 (2666): No heartbeat from core client for 30 sec - exiting
09:20:12 (2666): No heartbeat from core client for 30 sec - exiting
09:20:13 (2666): No heartbeat from core client for 30 sec - exiting
09:20:14 (2666): No heartbeat from core client for 30 sec - exiting
09:20:15 (2666): No heartbeat from core client for 30 sec - exiting
09:20:16 (2666): No heartbeat from core client for 30 sec - exiting
09:20:17 (2666): No heartbeat from core client for 30 sec - exiting
09:20:18 (2666): No heartbeat from core client for 30 sec - exiting
09:20:19 (2666): No heartbeat from core client for 30 sec - exiting
09:20:20 (2666): No heartbeat from core client for 30 sec - exiting
09:20:21 (2666): No heartbeat from core client for 30 sec - exiting
09:20:22 (2666): No heartbeat from core client for 30 sec - exiting
09:20:23 (2666): No heartbeat from core client for 30 sec - exiting
09:20:24 (2666): No heartbeat from core client for 30 sec - exiting
09:20:25 (2666): No heartbeat from core client for 30 sec - exiting
09:20:26 (2666): No heartbeat from core client for 30 sec - exiting
09:20:27 (2666): No heartbeat from core client for 30 sec - exiting
09:20:28 (2666): No heartbeat from core client for 30 sec - exiting
09:20:29 (2666): No heartbeat from core client for 30 sec - exiting
09:20:30 (2666): No heartbeat from core client for 30 sec - exiting
09:20:31 (2666): No heartbeat from core client for 30 sec - exiting
09:20:32 (2666): No heartbeat from core client for 30 sec - exiting
09:20:33 (2666): No heartbeat from core client for 30 sec - exiting
09:20:34 (2666): No heartbeat from core client for 30 sec - exiting
09:20:35 (2666): No heartbeat from core client for 30 sec - exiting
09:20:36 (2666): No heartbeat from core client for 30 sec - exiting
09:20:37 (2666): No heartbeat from core client for 30 sec - exiting
09:20:38 (2666): No heartbeat from core client for 30 sec - exiting
09:20:39 (2666): No heartbeat from core client for 30 sec - exiting
09:20:40 (2666): No heartbeat from core client for 30 sec - exiting
09:20:41 (2666): No heartbeat from core client for 30 sec - exiting
09:20:42 (2666): No heartbeat from core client for 30 sec - exiting
09:20:43 (2666): No heartbeat from core client for 30 sec - exiting
09:20:44 (2666): No heartbeat from core client for 30 sec - exiting
09:20:45 (2666): No heartbeat from core client for 30 sec - exiting
09:20:46 (2666): No heartbeat from core client for 30 sec - exiting
09:20:47 (2666): No heartbeat from core client for 30 sec - exiting
09:20:48 (2666): No heartbeat from core client for 30 sec - exiting
09:20:49 (2666): No heartbeat from core client for 30 sec - exiting
09:20:50 (2666): No heartbeat from core client for 30 sec - exiting
09:20:51 (2666): No heartbeat from core client for 30 sec - exiting
09:20:52 (2666): No heartbeat from core client for 30 sec - exiting
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08310768 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
09:20:53 (2666): No heartbeat from core client for 30 sec - exiting
09:20:54 (2666): No heartbeat from core client for 30 sec - exiting
09:20:55 (2666): No heartbeat from core client for 30 sec - exiting
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09235768 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08b12768 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09ffa768 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09538768 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
13:27:49 (2293): No heartbeat from core client for 30 sec - exiting
13:27:52 (2293): No heartbeat from core client for 30 sec - exiting
13:27:53 (2293): No heartbeat from core client for 30 sec - exiting
13:27:54 (2293): No heartbeat from core client for 30 sec - exiting
13:27:55 (2293): No heartbeat from core client for 30 sec - exiting
13:27:56 (2293): No heartbeat from core client for 30 sec - exiting
13:27:57 (2293): No heartbeat from core client for 30 sec - exiting
13:27:58 (2293): No heartbeat from core client for 30 sec - exiting
13:27:59 (2293): No heartbeat from core client for 30 sec - exiting
13:28:00 (2293): No heartbeat from core client for 30 sec - exiting
13:28:01 (2293): No heartbeat from core client for 30 sec - exiting
13:28:02 (2293): No heartbeat from core client for 30 sec - exiting
13:28:03 (2293): No heartbeat from core client for 30 sec - exiting
13:28:04 (2293): No heartbeat from core client for 30 sec - exiting
13:28:05 (2293): No heartbeat from core client for 30 sec - exiting
13:28:06 (2293): No heartbeat from core client for 30 sec - exiting
13:28:07 (2293): No heartbeat from core client for 30 sec - exiting
13:28:08 (2293): No heartbeat from core client for 30 sec - exiting
13:28:09 (2293): No heartbeat from core client for 30 sec - exiting
13:28:10 (2293): No heartbeat from core client for 30 sec - exiting
13:28:11 (2293): No heartbeat from core client for 30 sec - exiting
13:28:12 (2293): No heartbeat from core client for 30 sec - exiting
13:28:13 (2293): No heartbeat from core client for 30 sec - exiting
13:28:14 (2293): No heartbeat from core client for 30 sec - exiting
13:28:15 (2293): No heartbeat from core client for 30 sec - exiting
13:28:16 (2293): No heartbeat from core client for 30 sec - exiting
13:28:17 (2293): No heartbeat from core client for 30 sec - exiting
13:28:18 (2293): No heartbeat from core client for 30 sec - exiting
13:28:19 (2293): No heartbeat from core client for 30 sec - exiting
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x082f4768 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08a2e768 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x0865d768 ***
hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed.
SIGABRT: abort called
Signal 3 received, exiting...
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
23 Jan 2013 12:25:04 1120190 15471855 hadcm3n_zb7d_1880_40_008253943_2 518,400 927,916 1.7900
22 Jan 2013 08:25:26 1120190 15471855 hadcm3n_zb7d_1880_40_008253943_2 492,480 885,426 1.7979
18 Jan 2013 11:41:03 1120190 15471855 hadcm3n_zb7d_1880_40_008253943_2 466,560 839,374 1.7991
16 Jan 2013 14:45:46 1120190 15471855 hadcm3n_zb7d_1880_40_008253943_2 440,640 793,330 1.8004
15 Jan 2013 10:18:54 1120190 15471855 hadcm3n_zb7d_1880_40_008253943_2 414,720 747,566 1.8026
14 Jan 2013 21:30:19 1120190 15471855 hadcm3n_zb7d_1880_40_008253943_2 388,800 701,594 1.8045
14 Jan 2013 08:33:44 1120190 15471855 hadcm3n_zb7d_1880_40_008253943_2 362,880 655,302 1.8058
11 Jan 2013 03:43:27 1120190 15471855 hadcm3n_zb7d_1880_40_008253943_2 336,960 609,692 1.8094
10 Jan 2013 14:55:37 1120190 15471855 hadcm3n_zb7d_1880_40_008253943_2 311,040 563,977 1.8132
09 Jan 2013 10:25:29 1120190 15471855 hadcm3n_zb7d_1880_40_008253943_2 285,120 518,376 1.8181
08 Jan 2013 05:31:21 1120190 15471855 hadcm3n_zb7d_1880_40_008253943_2 259,200 472,588 1.8233
07 Jan 2013 16:47:04 1120190 15471855 hadcm3n_zb7d_1880_40_008253943_2 233,280 426,949 1.8302
20 Dec 2012 12:11:18 1120190 15471855 hadcm3n_zb7d_1880_40_008253943_2 207,360 381,764 1.8411
19 Dec 2012 07:46:44 1120190 15471855 hadcm3n_zb7d_1880_40_008253943_2 181,440 337,046 1.8576
18 Dec 2012 19:10:55 1120190 15471855 hadcm3n_zb7d_1880_40_008253943_2 155,520 292,061 1.8780
17 Dec 2012 13:20:18 1120190 15471855 hadcm3n_zb7d_1880_40_008253943_2 129,600 247,073 1.9064
17 Dec 2012 08:14:35 1120190 15471855 hadcm3n_zb7d_1880_40_008253943_2 103,680 199,377 1.9230
17 Dec 2012 08:14:35 1120190 15471855 hadcm3n_zb7d_1880_40_008253943_2 77,760 149,539 1.9231
17 Dec 2012 08:14:35 1120190 15471855 hadcm3n_zb7d_1880_40_008253943_2 51,840 99,546 1.9203
07 Dec 2012 10:57:21 1120190 15471855 hadcm3n_zb7d_1880_40_008253943_2 25,920 49,988 1.9285


©2024 climateprediction.net