Name | hadcm3n_zb7d_1880_40_008253943_2 |
Workunit | 8409067 |
Created | 6 Dec 2012, 2:23:00 UTC |
Sent | 6 Dec 2012, 2:23:08 UTC |
Report deadline | 7 Mar 2013, 9:50:19 UTC |
Received | 6 Jun 2013, 7:08:09 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 193 (0x000000C1) EXIT_SIGNAL |
Computer ID | 1120190 |
Run time | 10 days 23 hours 25 min 25 sec |
CPU time | 10 days 21 hours 41 min 32 sec |
Validate state | Invalid |
Credit | 6,220.80 |
Device peak FLOPS | 1.99 GFLOPS |
Application version | UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu |
Stderr | <core_client_version>7.0.27</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> ing 09:00:39 (2382): No heartbeat from core client for 30 sec - exiting 09:00:40 (2382): No heartbeat from core client for 30 sec - exiting 09:00:41 (2382): No heartbeat from core client for 30 sec - exiting 09:00:42 (2382): No heartbeat from core client for 30 sec - exiting 09:00:43 (2382): No heartbeat from core client for 30 sec - exiting 09:00:44 (2382): No heartbeat from core client for 30 sec - exiting 09:00:45 (2382): No heartbeat from core client for 30 sec - exiting 09:00:46 (2382): No heartbeat from core client for 30 sec - exiting 09:00:47 (2382): No heartbeat from core client for 30 sec - exiting 09:00:48 (2382): No heartbeat from core client for 30 sec - exiting 09:00:51 (2382): No heartbeat from core client for 30 sec - exiting 09:00:52 (2382): No heartbeat from core client for 30 sec - exiting 09:00:53 (2382): No heartbeat from core client for 30 sec - exiting 09:00:54 (2382): No heartbeat from core client for 30 sec - exiting 09:00:55 (2382): No heartbeat from core client for 30 sec - exiting 09:00:56 (2382): No heartbeat from core client for 30 sec - exiting 09:00:57 (2382): No heartbeat from core client for 30 sec - exiting 09:00:58 (2382): No heartbeat from core client for 30 sec - exiting 09:01:01 (2382): No heartbeat from core client for 30 sec - exiting 09:01:02 (2382): No heartbeat from core client for 30 sec - exiting 09:01:03 (2382): No heartbeat from core client for 30 sec - exiting 09:01:04 (2382): No heartbeat from core client for 30 sec - exiting 09:01:05 (2382): No heartbeat from core client for 30 sec - exiting 09:01:06 (2382): No heartbeat from core client for 30 sec - exiting 09:01:07 (2382): No heartbeat from core client for 30 sec - exiting 09:01:08 (2382): No heartbeat from core client for 30 sec - exiting 09:01:09 (2382): No heartbeat from core client for 30 sec - exiting 09:01:10 (2382): No heartbeat from core client for 30 sec - exiting 09:01:11 (2382): No heartbeat from core client for 30 sec - exiting 09:01:12 (2382): No heartbeat from core client for 30 sec - exiting 09:01:13 (2382): No heartbeat from core client for 30 sec - exiting 09:01:14 (2382): No heartbeat from core client for 30 sec - exiting 09:01:15 (2382): No heartbeat from core client for 30 sec - exiting 09:01:16 (2382): No heartbeat from core client for 30 sec - exiting 09:01:17 (2382): No heartbeat from core client for 30 sec - exiting 09:01:18 (2382): No heartbeat from core client for 30 sec - exiting 09:01:19 (2382): No heartbeat from core client for 30 sec - exiting 09:01:20 (2382): No heartbeat from core client for 30 sec - exiting 09:01:21 (2382): No heartbeat from core client for 30 sec - exiting 09:01:22 (2382): No heartbeat from core client for 30 sec - exiting 09:01:23 (2382): No heartbeat from core client for 30 sec - exiting 09:01:24 (2382): No heartbeat from core client for 30 sec - exiting 09:01:25 (2382): No heartbeat from core client for 30 sec - exiting 09:01:26 (2382): No heartbeat from core client for 30 sec - exiting 09:01:27 (2382): No heartbeat from core client for 30 sec - exiting 09:01:28 (2382): No heartbeat from core client for 30 sec - exiting 09:01:29 (2382): No heartbeat from core client for 30 sec - exiting 09:01:30 (2382): No heartbeat from core client for 30 sec - exiting 09:01:31 (2382): No heartbeat from core client for 30 sec - exiting 09:01:32 (2382): No heartbeat from core client for 30 sec - exiting 09:01:33 (2382): No heartbeat from core client for 30 sec - exiting 09:01:34 (2382): No heartbeat from core client for 30 sec - exiting 09:01:35 (2382): No heartbeat from core client for 30 sec - exiting 09:01:36 (2382): No heartbeat from core client for 30 sec - exiting 09:01:37 (2382): No heartbeat from core client for 30 sec - exiting 09:01:38 (2382): No heartbeat from core client for 30 sec - exiting 09:01:39 (2382): No heartbeat from core client for 30 sec - exiting 09:01:40 (2382): No heartbeat from core client for 30 sec - exiting 09:01:41 (2382): No heartbeat from core client for 30 sec - exiting 09:01:42 (2382): No heartbeat from core client for 30 sec - exiting *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09619768 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called 09:01:43 (2382): No heartbeat from core client for 30 sec - exiting 09:01:44 (2382): No heartbeat from core client for 30 sec - exiting *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08bae768 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called 09:08:22 (2377): No heartbeat from core client for 30 sec - exiting 09:08:28 (2377): No heartbeat from core client for 30 sec - exiting 09:08:29 (2377): No heartbeat from core client for 30 sec - exiting 09:08:30 (2377): No heartbeat from core client for 30 sec - exiting 09:08:31 (2377): No heartbeat from core client for 30 sec - exiting 09:08:32 (2377): No heartbeat from core client for 30 sec - exiting 09:08:35 (2377): No heartbeat from core client for 30 sec - exiting 09:08:38 (2377): No heartbeat from core client for 30 sec - exiting 09:08:39 (2377): No heartbeat from core client for 30 sec - exiting 09:08:40 (2377): No heartbeat from core client for 30 sec - exiting 09:08:41 (2377): No heartbeat from core client for 30 sec - exiting 09:08:42 (2377): No heartbeat from core client for 30 sec - exiting 09:08:43 (2377): No heartbeat from core client for 30 sec - exiting 09:08:44 (2377): No heartbeat from core client for 30 sec - exiting 09:08:45 (2377): No heartbeat from core client for 30 sec - exiting 09:08:46 (2377): No heartbeat from core client for 30 sec - exiting 09:08:47 (2377): No heartbeat from core client for 30 sec - exiting 09:08:48 (2377): No heartbeat from core client for 30 sec - exiting 09:08:49 (2377): No heartbeat from core client for 30 sec - exiting 09:08:50 (2377): No heartbeat from core client for 30 sec - exiting 09:08:51 (2377): No heartbeat from core client for 30 sec - exiting 09:08:52 (2377): No heartbeat from core client for 30 sec - exiting 09:08:53 (2377): No heartbeat from core client for 30 sec - exiting 09:08:54 (2377): No heartbeat from core client for 30 sec - exiting 09:08:55 (2377): No heartbeat from core client for 30 sec - exiting 09:08:56 (2377): No heartbeat from core client for 30 sec - exiting 09:08:57 (2377): No heartbeat from core client for 30 sec - exiting 09:08:58 (2377): No heartbeat from core client for 30 sec - exiting 09:08:59 (2377): No heartbeat from core client for 30 sec - exiting 09:09:00 (2377): No heartbeat from core client for 30 sec - exiting 09:09:01 (2377): No heartbeat from core client for 30 sec - exiting 09:09:02 (2377): No heartbeat from core client for 30 sec - exiting 09:09:03 (2377): No heartbeat from core client for 30 sec - exiting 09:09:04 (2377): No heartbeat from core client for 30 sec - exiting 09:09:05 (2377): No heartbeat from core client for 30 sec - exiting 09:09:06 (2377): No heartbeat from core client for 30 sec - exiting 09:09:07 (2377): No heartbeat from core client for 30 sec - exiting 09:09:08 (2377): No heartbeat from core client for 30 sec - exiting 09:09:09 (2377): No heartbeat from core client for 30 sec - exiting 09:09:10 (2377): No heartbeat from core client for 30 sec - exiting 09:09:11 (2377): No heartbeat from core client for 30 sec - exiting 09:09:12 (2377): No heartbeat from core client for 30 sec - exiting 09:09:13 (2377): No heartbeat from core client for 30 sec - exiting 09:09:14 (2377): No heartbeat from core client for 30 sec - exiting 09:09:15 (2377): No heartbeat from core client for 30 sec - exiting 09:09:16 (2377): No heartbeat from core client for 30 sec - exiting 09:09:17 (2377): No heartbeat from core client for 30 sec - exiting 09:09:18 (2377): No heartbeat from core client for 30 sec - exiting 09:09:19 (2377): No heartbeat from core client for 30 sec - exiting 09:09:20 (2377): No heartbeat from core client for 30 sec - exiting 09:09:21 (2377): No heartbeat from core client for 30 sec - exiting 09:09:22 (2377): No heartbeat from core client for 30 sec - exiting 09:09:23 (2377): No heartbeat from core client for 30 sec - exiting 09:09:24 (2377): No heartbeat from core client for 30 sec - exiting 09:09:25 (2377): No heartbeat from core client for 30 sec - exiting 09:09:26 (2377): No heartbeat from core client for 30 sec - exiting 09:09:27 (2377): No heartbeat from core client for 30 sec - exiting 09:09:29 (2377): No heartbeat from core client for 30 sec - exiting 09:09:30 (2377): No heartbeat from core client for 30 sec - exiting 09:09:31 (2377): No heartbeat from core client for 30 sec - exiting 09:09:32 (2377): No heartbeat from core client for 30 sec - exiting 09:09:33 (2377): No heartbeat from core client for 30 sec - exiting 09:09:34 (2377): No heartbeat from core client for 30 sec - exiting 09:09:36 (2377): No heartbeat from core client for 30 sec - exiting 09:09:37 (2377): No heartbeat from core client for 30 sec - exiting 09:09:38 (2377): No heartbeat from core client for 30 sec - exiting 09:09:39 (2377): No heartbeat from core client for 30 sec - exiting 09:09:40 (2377): No heartbeat from core client for 30 sec - exiting 09:09:41 (2377): No heartbeat from core client for 30 sec - exiting 09:09:42 (2377): No heartbeat from core client for 30 sec - exiting 09:09:45 (2377): No heartbeat from core client for 30 sec - exiting 09:09:46 (2377): No heartbeat from core client for 30 sec - exiting 09:09:47 (2377): No heartbeat from core client for 30 sec - exiting 09:09:48 (2377): No heartbeat from core client for 30 sec - exiting 09:09:49 (2377): No heartbeat from core client for 30 sec - exiting 09:09:50 (2377): No heartbeat from core client for 30 sec - exiting 09:09:51 (2377): No heartbeat from core client for 30 sec - exiting 09:09:52 (2377): No heartbeat from core client for 30 sec - exiting 09:09:53 (2377): No heartbeat from core client for 30 sec - exiting 09:09:54 (2377): No heartbeat from core client for 30 sec - exiting 09:09:55 (2377): No heartbeat from core client for 30 sec - exiting 09:09:56 (2377): No heartbeat from core client for 30 sec - exiting 09:09:57 (2377): No heartbeat from core client for 30 sec - exiting 09:09:58 (2377): No heartbeat from core client for 30 sec - exiting 09:09:59 (2377): No heartbeat from core client for 30 sec - exiting 09:10:00 (2377): No heartbeat from core client for 30 sec - exiting 09:10:01 (2377): No heartbeat from core client for 30 sec - exiting 09:10:02 (2377): No heartbeat from core client for 30 sec - exiting *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x094bc768 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called 09:10:03 (2377): No heartbeat from core client for 30 sec - exiting 09:10:04 (2377): No heartbeat from core client for 30 sec - exiting 09:18:22 (2377): No heartbeat from core client for 30 sec - exiting 09:18:26 (2377): No heartbeat from core client for 30 sec - exiting 09:18:27 (2377): No heartbeat from core client for 30 sec - exiting 09:18:28 (2377): No heartbeat from core client for 30 sec - exiting 09:18:29 (2377): No heartbeat from core client for 30 sec - exiting 09:18:30 (2377): No heartbeat from core client for 30 sec - exiting 09:18:31 (2377): No heartbeat from core client for 30 sec - exiting 09:18:32 (2377): No heartbeat from core client for 30 sec - exiting 09:18:33 (2377): No heartbeat from core client for 30 sec - exiting 09:18:34 (2377): No heartbeat from core client for 30 sec - exiting 09:18:35 (2377): No heartbeat from core client for 30 sec - exiting 09:18:38 (2377): No heartbeat from core client for 30 sec - exiting 09:18:39 (2377): No heartbeat from core client for 30 sec - exiting 09:18:40 (2377): No heartbeat from core client for 30 sec - exiting 09:18:52 (2377): No heartbeat from core client for 30 sec - exiting 09:19:02 (2377): No heartbeat from core client for 30 sec - exiting 09:19:11 (2377): No heartbeat from core client for 30 sec - exiting 09:19:21 (2377): No heartbeat from core client for 30 sec - exiting 09:19:22 (2377): No heartbeat from core client for 30 sec - exiting 09:19:23 (2377): No heartbeat from core client for 30 sec - exiting 09:19:24 (2377): No heartbeat from core client for 30 sec - exiting 09:19:25 (2377): No heartbeat from core client for 30 sec - exiting 09:19:26 (2377): No heartbeat from core client for 30 sec - exiting 09:19:27 (2377): No heartbeat from core client for 30 sec - exiting 09:19:28 (2377): No heartbeat from core client for 30 sec - exiting 09:19:29 (2377): No heartbeat from core client for 30 sec - exiting 09:19:30 (2377): No heartbeat from core client for 30 sec - exiting 09:19:31 (2377): No heartbeat from core client for 30 sec - exiting 09:19:32 (2377): No heartbeat from core client for 30 sec - exiting 09:19:33 (2377): No heartbeat from core client for 30 sec - exiting 09:19:34 (2377): No heartbeat from core client for 30 sec - exiting 09:19:35 (2377): No heartbeat from core client for 30 sec - exiting 09:19:36 (2377): No heartbeat from core client for 30 sec - exiting 09:19:37 (2377): No heartbeat from core client for 30 sec - exiting 09:19:38 (2377): No heartbeat from core client for 30 sec - exiting 09:19:39 (2377): No heartbeat from core client for 30 sec - exiting 09:19:40 (2377): No heartbeat from core client for 30 sec - exiting 09:19:41 (2377): No heartbeat from core client for 30 sec - exiting 09:19:42 (2377): No heartbeat from core client for 30 sec - exiting 09:19:43 (2377): No heartbeat from core client for 30 sec - exiting 09:19:44 (2377): No heartbeat from core client for 30 sec - exiting 09:19:45 (2377): No heartbeat from core client for 30 sec - exiting 09:19:46 (2377): No heartbeat from core client for 30 sec - exiting 09:19:47 (2377): No heartbeat from core client for 30 sec - exiting 09:19:48 (2377): No heartbeat from core client for 30 sec - exiting 09:19:49 (2377): No heartbeat from core client for 30 sec - exiting 09:19:50 (2377): No heartbeat from core client for 30 sec - exiting 09:19:51 (2377): No heartbeat from core client for 30 sec - exiting 09:19:52 (2377): No heartbeat from core client for 30 sec - exiting 09:19:53 (2377): No heartbeat from core client for 30 sec - exiting 09:19:54 (2377): No heartbeat from core client for 30 sec - exiting 09:19:55 (2377): No heartbeat from core client for 30 sec - exiting 09:19:56 (2377): No heartbeat from core client for 30 sec - exiting 09:19:57 (2377): No heartbeat from core client for 30 sec - exiting 09:19:58 (2377): No heartbeat from core client for 30 sec - exiting 09:19:59 (2377): No heartbeat from core client for 30 sec - exiting 09:20:00 (2377): No heartbeat from core client for 30 sec - exiting 09:20:04 (2377): No heartbeat from core client for 30 sec - exiting 09:20:05 (2377): No heartbeat from core client for 30 sec - exiting 09:20:06 (2377): No heartbeat from core client for 30 sec - exiting 09:20:07 (2377): No heartbeat from core client for 30 sec - exiting 09:20:08 (2377): No heartbeat from core client for 30 sec - exiting 09:20:09 (2377): No heartbeat from core client for 30 sec - exiting 09:20:10 (2377): No heartbeat from core client for 30 sec - exiting 09:20:11 (2377): No heartbeat from core client for 30 sec - exiting 09:20:12 (2377): No heartbeat from core client for 30 sec - exiting 09:20:13 (2377): No heartbeat from core client for 30 sec - exiting 09:20:14 (2377): No heartbeat from core client for 30 sec - exiting 09:20:15 (2377): No heartbeat from core client for 30 sec - exiting 09:20:16 (2377): No heartbeat from core client for 30 sec - exiting 09:20:17 (2377): No heartbeat from core client for 30 sec - exiting 09:20:18 (2377): No heartbeat from core client for 30 sec - exiting 09:20:19 (2377): No heartbeat from core client for 30 sec - exiting 09:20:20 (2377): No heartbeat from core client for 30 sec - exiting 09:20:21 (2377): No heartbeat from core client for 30 sec - exiting 09:20:22 (2377): No heartbeat from core client for 30 sec - exiting 09:20:23 (2377): No heartbeat from core client for 30 sec - exiting 09:20:27 (2377): No heartbeat from core client for 30 sec - exiting 09:20:28 (2377): No heartbeat from core client for 30 sec - exiting 09:20:29 (2377): No heartbeat from core client for 30 sec - exiting 09:20:30 (2377): No heartbeat from core client for 30 sec - exiting 09:20:31 (2377): No heartbeat from core client for 30 sec - exiting 09:20:32 (2377): No heartbeat from core client for 30 sec - exiting 09:20:33 (2377): No heartbeat from core client for 30 sec - exiting 09:20:34 (2377): No heartbeat from core client for 30 sec - exiting 09:20:35 (2377): No heartbeat from core client for 30 sec - exiting 09:20:36 (2377): No heartbeat from core client for 30 sec - exiting 09:20:37 (2377): No heartbeat from core client for 30 sec - exiting 09:20:48 (2377): No heartbeat from core client for 30 sec - exiting 09:21:01 (2377): No heartbeat from core client for 30 sec - exiting 09:21:03 (2377): No heartbeat from core client for 30 sec - exiting 09:21:04 (2377): No heartbeat from core client for 30 sec - exiting 09:21:05 (2377): No heartbeat from core client for 30 sec - exiting 09:21:06 (2377): No heartbeat from core client for 30 sec - exiting 09:21:07 (2377): No heartbeat from core client for 30 sec - exiting 09:21:08 (2377): No heartbeat from core client for 30 sec - exiting 09:21:09 (2377): No heartbeat from core client for 30 sec - exiting *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09fd1768 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08e8c768 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x094c4768 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08b83768 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called 09:09:12 (2421): No heartbeat from core client for 30 sec - exiting 09:09:43 (2421): No heartbeat from core client for 30 sec - exiting 09:09:44 (2421): No heartbeat from core client for 30 sec - exiting 09:09:45 (2421): No heartbeat from core client for 30 sec - exiting 09:09:46 (2421): No heartbeat from core client for 30 sec - exiting 09:09:50 (2421): No heartbeat from core client for 30 sec - exiting 09:09:51 (2421): No heartbeat from core client for 30 sec - exiting 09:10:02 (2421): No heartbeat from core client for 30 sec - exiting 09:10:06 (2421): No heartbeat from core client for 30 sec - exiting 09:14:31 (2382): No heartbeat from core client for 30 sec - exiting 09:14:33 (2382): No heartbeat from core client for 30 sec - exiting 09:14:34 (2382): No heartbeat from core client for 30 sec - exiting 09:14:35 (2382): No heartbeat from core client for 30 sec - exiting 09:14:36 (2382): No heartbeat from core client for 30 sec - exiting 09:14:37 (2382): No heartbeat from core client for 30 sec - exiting 09:14:38 (2382): No heartbeat from core client for 30 sec - exiting 09:14:39 (2382): No heartbeat from core client for 30 sec - exiting 09:14:40 (2382): No heartbeat from core client for 30 sec - exiting 09:14:41 (2382): No heartbeat from core client for 30 sec - exiting 09:14:42 (2382): No heartbeat from core client for 30 sec - exiting 09:14:43 (2382): No heartbeat from core client for 30 sec - exiting 09:14:44 (2382): No heartbeat from core client for 30 sec - exiting 09:14:45 (2382): No heartbeat from core client for 30 sec - exiting 09:14:46 (2382): No heartbeat from core client for 30 sec - exiting 09:14:47 (2382): No heartbeat from core client for 30 sec - exiting 09:14:48 (2382): No heartbeat from core client for 30 sec - exiting 09:14:49 (2382): No heartbeat from core client for 30 sec - exiting 09:14:52 (2382): No heartbeat from core client for 30 sec - exiting 09:14:53 (2382): No heartbeat from core client for 30 sec - exiting 09:14:54 (2382): No heartbeat from core client for 30 sec - exiting 09:14:55 (2382): No heartbeat from core client for 30 sec - exiting 09:14:56 (2382): No heartbeat from core client for 30 sec - exiting 09:14:57 (2382): No heartbeat from core client for 30 sec - exiting 09:14:58 (2382): No heartbeat from core client for 30 sec - exiting 09:14:59 (2382): No heartbeat from core client for 30 sec - exiting 09:15:00 (2382): No heartbeat from core client for 30 sec - exiting 09:15:01 (2382): No heartbeat from core client for 30 sec - exiting 09:15:02 (2382): No heartbeat from core client for 30 sec - exiting 09:15:03 (2382): No heartbeat from core client for 30 sec - exiting 09:15:04 (2382): No heartbeat from core client for 30 sec - exiting 09:15:05 (2382): No heartbeat from core client for 30 sec - exiting 09:15:06 (2382): No heartbeat from core client for 30 sec - exiting 09:15:07 (2382): No heartbeat from core client for 30 sec - exiting 09:15:08 (2382): No heartbeat from core client for 30 sec - exiting 09:15:09 (2382): No heartbeat from core client for 30 sec - exiting 09:15:10 (2382): No heartbeat from core client for 30 sec - exiting 09:15:11 (2382): No heartbeat from core client for 30 sec - exiting 09:15:12 (2382): No heartbeat from core client for 30 sec - exiting 09:15:13 (2382): No heartbeat from core client for 30 sec - exiting 09:15:14 (2382): No heartbeat from core client for 30 sec - exiting 09:15:15 (2382): No heartbeat from core client for 30 sec - exiting 09:15:16 (2382): No heartbeat from core client for 30 sec - exiting 09:15:17 (2382): No heartbeat from core client for 30 sec - exiting 09:15:18 (2382): No heartbeat from core client for 30 sec - exiting 09:15:20 (2382): No heartbeat from core client for 30 sec - exiting 09:15:21 (2382): No heartbeat from core client for 30 sec - exiting 09:15:22 (2382): No heartbeat from core client for 30 sec - exiting 09:15:24 (2382): No heartbeat from core client for 30 sec - exiting 09:15:25 (2382): No heartbeat from core client for 30 sec - exiting 09:15:26 (2382): No heartbeat from core client for 30 sec - exiting 09:15:27 (2382): No heartbeat from core client for 30 sec - exiting 09:15:28 (2382): No heartbeat from core client for 30 sec - exiting 09:15:29 (2382): No heartbeat from core client for 30 sec - exiting 09:15:30 (2382): No heartbeat from core client for 30 sec - exiting 09:15:32 (2382): No heartbeat from core client for 30 sec - exiting 09:15:33 (2382): No heartbeat from core client for 30 sec - exiting 09:15:34 (2382): No heartbeat from core client for 30 sec - exiting 09:15:35 (2382): No heartbeat from core client for 30 sec - exiting 09:15:38 (2382): No heartbeat from core client for 30 sec - exiting 09:15:39 (2382): No heartbeat from core client for 30 sec - exiting 09:15:40 (2382): No heartbeat from core client for 30 sec - exiting 09:15:41 (2382): No heartbeat from core client for 30 sec - exiting 09:15:42 (2382): No heartbeat from core client for 30 sec - exiting 09:15:43 (2382): No heartbeat from core client for 30 sec - exiting 09:15:44 (2382): No heartbeat from core client for 30 sec - exiting *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08bae768 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called 09:15:45 (2382): No heartbeat from core client for 30 sec - exiting 09:15:46 (2382): No heartbeat from core client for 30 sec - exiting 09:15:47 (2382): No heartbeat from core client for 30 sec - exiting 11:57:08 (2351): No heartbeat from core client for 30 sec - exiting 11:57:21 (2351): No heartbeat from core client for 30 sec - exiting 11:57:22 (2351): No heartbeat from core client for 30 sec - exiting 11:57:23 (2351): No heartbeat from core client for 30 sec - exiting 11:57:24 (2351): No heartbeat from core client for 30 sec - exiting 11:57:25 (2351): No heartbeat from core client for 30 sec - exiting 11:57:26 (2351): No heartbeat from core client for 30 sec - exiting 11:57:27 (2351): No heartbeat from core client for 30 sec - exiting 11:57:28 (2351): No heartbeat from core client for 30 sec - exiting 11:57:29 (2351): No heartbeat from core client for 30 sec - exiting 11:57:30 (2351): No heartbeat from core client for 30 sec - exiting 11:57:31 (2351): No heartbeat from core client for 30 sec - exiting 11:57:32 (2351): No heartbeat from core client for 30 sec - exiting 11:57:34 (2351): No heartbeat from core client for 30 sec - exiting 11:57:37 (2351): No heartbeat from core client for 30 sec - exiting 11:57:38 (2351): No heartbeat from core client for 30 sec - exiting 11:57:39 (2351): No heartbeat from core client for 30 sec - exiting 11:57:40 (2351): No heartbeat from core client for 30 sec - exiting 11:57:41 (2351): No heartbeat from core client for 30 sec - exiting 11:57:42 (2351): No heartbeat from core client for 30 sec - exiting 11:57:43 (2351): No heartbeat from core client for 30 sec - exiting 11:57:44 (2351): No heartbeat from core client for 30 sec - exiting 11:57:45 (2351): No heartbeat from core client for 30 sec - exiting 11:57:47 (2351): No heartbeat from core client for 30 sec - exiting 11:57:48 (2351): No heartbeat from core client for 30 sec - exiting 11:57:49 (2351): No heartbeat from core client for 30 sec - exiting 11:57:50 (2351): No heartbeat from core client for 30 sec - exiting 11:57:51 (2351): No heartbeat from core client for 30 sec - exiting 11:57:52 (2351): No heartbeat from core client for 30 sec - exiting 11:57:53 (2351): No heartbeat from core client for 30 sec - exiting 11:57:54 (2351): No heartbeat from core client for 30 sec - exiting 11:57:55 (2351): No heartbeat from core client for 30 sec - exiting 11:57:56 (2351): No heartbeat from core client for 30 sec - exiting 11:57:57 (2351): No heartbeat from core client for 30 sec - exiting 11:57:58 (2351): No heartbeat from core client for 30 sec - exiting 11:57:59 (2351): No heartbeat from core client for 30 sec - exiting 11:58:00 (2351): No heartbeat from core client for 30 sec - exiting 11:58:01 (2351): No heartbeat from core client for 30 sec - exiting 11:58:02 (2351): No heartbeat from core client for 30 sec - exiting 11:58:03 (2351): No heartbeat from core client for 30 sec - exiting 11:58:04 (2351): No heartbeat from core client for 30 sec - exiting 11:58:05 (2351): No heartbeat from core client for 30 sec - exiting 11:58:06 (2351): No heartbeat from core client for 30 sec - exiting 11:58:07 (2351): No heartbeat from core client for 30 sec - exiting 11:58:13 (2351): No heartbeat from core client for 30 sec - exiting 11:58:14 (2351): No heartbeat from core client for 30 sec - exiting 11:58:15 (2351): No heartbeat from core client for 30 sec - exiting 11:58:16 (2351): No heartbeat from core client for 30 sec - exiting 11:58:17 (2351): No heartbeat from core client for 30 sec - exiting 11:58:19 (2351): No heartbeat from core client for 30 sec - exiting 11:58:20 (2351): No heartbeat from core client for 30 sec - exiting 11:58:21 (2351): No heartbeat from core client for 30 sec - exiting 11:58:22 (2351): No heartbeat from core client for 30 sec - exiting 11:58:23 (2351): No heartbeat from core client for 30 sec - exiting 11:58:24 (2351): No heartbeat from core client for 30 sec - exiting 11:58:25 (2351): No heartbeat from core client for 30 sec - exiting 11:58:26 (2351): No heartbeat from core client for 30 sec - exiting 11:58:27 (2351): No heartbeat from core client for 30 sec - exiting 11:58:28 (2351): No heartbeat from core client for 30 sec - exiting 11:58:29 (2351): No heartbeat from core client for 30 sec - exiting 11:58:31 (2351): No heartbeat from core client for 30 sec - exiting 11:58:32 (2351): No heartbeat from core client for 30 sec - exiting 11:58:33 (2351): No heartbeat from core client for 30 sec - exiting 11:58:34 (2351): No heartbeat from core client for 30 sec - exiting 11:58:35 (2351): No heartbeat from core client for 30 sec - exiting 11:58:36 (2351): No heartbeat from core client for 30 sec - exiting 11:58:37 (2351): No heartbeat from core client for 30 sec - exiting 11:58:38 (2351): No heartbeat from core client for 30 sec - exiting 11:58:39 (2351): No heartbeat from core client for 30 sec - exiting 11:58:40 (2351): No heartbeat from core client for 30 sec - exiting 11:58:41 (2351): No heartbeat from core client for 30 sec - exiting 11:58:42 (2351): No heartbeat from core client for 30 sec - exiting 11:58:43 (2351): No heartbeat from core client for 30 sec - exiting 11:58:44 (2351): No heartbeat from core client for 30 sec - exiting 11:58:45 (2351): No heartbeat from core client for 30 sec - exiting 11:58:46 (2351): No heartbeat from core client for 30 sec - exiting 11:58:47 (2351): No heartbeat from core client for 30 sec - exiting 11:58:48 (2351): No heartbeat from core client for 30 sec - exiting 11:58:49 (2351): No heartbeat from core client for 30 sec - exiting 11:58:50 (2351): No heartbeat from core client for 30 sec - exiting 11:58:51 (2351): No heartbeat from core client for 30 sec - exiting *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09b81768 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called 11:58:52 (2351): No heartbeat from core client for 30 sec - exiting *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09d50768 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called 08:56:02 (2438): No heartbeat from core client for 30 sec - exiting 08:56:03 (2438): No heartbeat from core client for 30 sec - exiting 08:56:04 (2438): No heartbeat from core client for 30 sec - exiting 08:56:05 (2438): No heartbeat from core client for 30 sec - exiting 16:42:18 (2438): No heartbeat from core client for 30 sec - exiting 16:42:19 (2438): No heartbeat from core client for 30 sec - exiting 16:42:20 (2438): No heartbeat from core client for 30 sec - exiting 16:42:21 (2438): No heartbeat from core client for 30 sec - exiting 16:42:22 (2438): No heartbeat from core client for 30 sec - exiting 16:42:23 (2438): No heartbeat from core client for 30 sec - exiting 16:42:24 (2438): No heartbeat from core client for 30 sec - exiting 16:42:25 (2438): No heartbeat from core client for 30 sec - exiting 16:42:26 (2438): No heartbeat from core client for 30 sec - exiting 16:42:27 (2438): No heartbeat from core client for 30 sec - exiting 16:42:28 (2438): No heartbeat from core client for 30 sec - exiting 16:42:29 (2438): No heartbeat from core client for 30 sec - exiting 16:42:30 (2438): No heartbeat from core client for 30 sec - exiting 16:42:31 (2438): No heartbeat from core client for 30 sec - exiting 16:42:32 (2438): No heartbeat from core client for 30 sec - exiting 16:42:33 (2438): No heartbeat from core client for 30 sec - exiting 16:42:34 (2438): No heartbeat from core client for 30 sec - exiting 16:42:35 (2438): No heartbeat from core client for 30 sec - exiting 16:42:36 (2438): No heartbeat from core client for 30 sec - exiting 16:42:37 (2438): No heartbeat from core client for 30 sec - exiting 16:42:38 (2438): No heartbeat from core client for 30 sec - exiting 16:42:39 (2438): No heartbeat from core client for 30 sec - exiting 16:42:40 (2438): No heartbeat from core client for 30 sec - exiting 16:42:41 (2438): No heartbeat from core client for 30 sec - exiting 16:42:42 (2438): No heartbeat from core client for 30 sec - exiting *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x082fe768 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called 09:09:01 (2363): No heartbeat from core client for 30 sec - exiting 09:09:02 (2363): No heartbeat from core client for 30 sec - exiting *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08467768 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x0a0d9768 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called 09:12:07 (2398): No heartbeat from core client for 30 sec - exiting 09:12:08 (2398): No heartbeat from core client for 30 sec - exiting 09:12:09 (2398): No heartbeat from core client for 30 sec - exiting 09:12:10 (2398): No heartbeat from core client for 30 sec - exiting 09:12:11 (2398): No heartbeat from core client for 30 sec - exiting 09:12:12 (2398): No heartbeat from core client for 30 sec - exiting 09:12:13 (2398): No heartbeat from core client for 30 sec - exiting 09:12:14 (2398): No heartbeat from core client for 30 sec - exiting 09:12:15 (2398): No heartbeat from core client for 30 sec - exiting 09:12:16 (2398): No heartbeat from core client for 30 sec - exiting 09:12:17 (2398): No heartbeat from core client for 30 sec - exiting 09:12:18 (2398): No heartbeat from core client for 30 sec - exiting 09:12:19 (2398): No heartbeat from core client for 30 sec - exiting 09:12:20 (2398): No heartbeat from core client for 30 sec - exiting 09:12:21 (2398): No heartbeat from core client for 30 sec - exiting 09:12:22 (2398): No heartbeat from core client for 30 sec - exiting 09:12:23 (2398): No heartbeat from core client for 30 sec - exiting 09:12:24 (2398): No heartbeat from core client for 30 sec - exiting 09:12:25 (2398): No heartbeat from core client for 30 sec - exiting 09:12:26 (2398): No heartbeat from core client for 30 sec - exiting 09:12:27 (2398): No heartbeat from core client for 30 sec - exiting 09:12:28 (2398): No heartbeat from core client for 30 sec - exiting 09:12:29 (2398): No heartbeat from core client for 30 sec - exiting 09:12:30 (2398): No heartbeat from core client for 30 sec - exiting 09:12:31 (2398): No heartbeat from core client for 30 sec - exiting 09:12:32 (2398): No heartbeat from core client for 30 sec - exiting 09:12:33 (2398): No heartbeat from core client for 30 sec - exiting 09:12:34 (2398): No heartbeat from core client for 30 sec - exiting 09:12:35 (2398): No heartbeat from core client for 30 sec - exiting 09:11:24 (2433): No heartbeat from core client for 30 sec - exiting 09:11:28 (2433): No heartbeat from core client for 30 sec - exiting 09:11:29 (2433): No heartbeat from core client for 30 sec - exiting 09:11:30 (2433): No heartbeat from core client for 30 sec - exiting 09:11:31 (2433): No heartbeat from core client for 30 sec - exiting 09:11:32 (2433): No heartbeat from core client for 30 sec - exiting 09:11:33 (2433): No heartbeat from core client for 30 sec - exiting 09:11:34 (2433): No heartbeat from core client for 30 sec - exiting 09:11:36 (2433): No heartbeat from core client for 30 sec - exiting 09:11:37 (2433): No heartbeat from core client for 30 sec - exiting 09:11:38 (2433): No heartbeat from core client for 30 sec - exiting 09:11:39 (2433): No heartbeat from core client for 30 sec - exiting 09:11:40 (2433): No heartbeat from core client for 30 sec - exiting 09:11:41 (2433): No heartbeat from core client for 30 sec - exiting 09:11:42 (2433): No heartbeat from core client for 30 sec - exiting 09:11:43 (2433): No heartbeat from core client for 30 sec - exiting 09:11:44 (2433): No heartbeat from core client for 30 sec - exiting 09:11:45 (2433): No heartbeat from core client for 30 sec - exiting 09:11:46 (2433): No heartbeat from core client for 30 sec - exiting 09:11:48 (2433): No heartbeat from core client for 30 sec - exiting 09:11:49 (2433): No heartbeat from core client for 30 sec - exiting 09:11:50 (2433): No heartbeat from core client for 30 sec - exiting 09:11:51 (2433): No heartbeat from core client for 30 sec - exiting 09:11:52 (2433): No heartbeat from core client for 30 sec - exiting 09:11:53 (2433): No heartbeat from core client for 30 sec - exiting 09:11:54 (2433): No heartbeat from core client for 30 sec - exiting 09:11:56 (2433): No heartbeat from core client for 30 sec - exiting 09:11:57 (2433): No heartbeat from core client for 30 sec - exiting 09:11:58 (2433): No heartbeat from core client for 30 sec - exiting 09:11:59 (2433): No heartbeat from core client for 30 sec - exiting 09:12:00 (2433): No heartbeat from core client for 30 sec - exiting 09:12:01 (2433): No heartbeat from core client for 30 sec - exiting 09:12:02 (2433): No heartbeat from core client for 30 sec - exiting 09:12:03 (2433): No heartbeat from core client for 30 sec - exiting 09:12:04 (2433): No heartbeat from core client for 30 sec - exiting 09:12:05 (2433): No heartbeat from core client for 30 sec - exiting 09:12:06 (2433): No heartbeat from core client for 30 sec - exiting 09:12:07 (2433): No heartbeat from core client for 30 sec - exiting 09:12:08 (2433): No heartbeat from core client for 30 sec - exiting 09:12:09 (2433): No heartbeat from core client for 30 sec - exiting 09:12:10 (2433): No heartbeat from core client for 30 sec - exiting 09:12:11 (2433): No heartbeat from core client for 30 sec - exiting 09:12:12 (2433): No heartbeat from core client for 30 sec - exiting 09:12:13 (2433): No heartbeat from core client for 30 sec - exiting 09:12:14 (2433): No heartbeat from core client for 30 sec - exiting 09:12:16 (2433): No heartbeat from core client for 30 sec - exiting 09:12:17 (2433): No heartbeat from core client for 30 sec - exiting *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x098df768 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called 09:12:18 (2433): No heartbeat from core client for 30 sec - exiting 09:12:19 (2433): No heartbeat from core client for 30 sec - exiting 09:12:20 (2433): No heartbeat from core client for 30 sec - exiting 09:12:21 (2433): No heartbeat from core client for 30 sec - exiting 09:12:22 (2433): No heartbeat from core client for 30 sec - exiting 09:12:23 (2433): No heartbeat from core client for 30 sec - exiting 09:12:24 (2433): No heartbeat from core client for 30 sec - exiting 09:12:25 (2433): No heartbeat from core client for 30 sec - exiting 09:12:26 (2433): No heartbeat from core client for 30 sec - exiting 09:12:27 (2433): No heartbeat from core client for 30 sec - exiting 09:12:29 (2433): No heartbeat from core client for 30 sec - exiting 09:12:30 (2433): No heartbeat from core client for 30 sec - exiting 09:12:31 (2433): No heartbeat from core client for 30 sec - exiting 09:12:32 (2433): No heartbeat from core client for 30 sec - exiting 09:12:33 (2433): No heartbeat from core client for 30 sec - exiting 09:12:34 (2433): No heartbeat from core client for 30 sec - exiting 09:12:35 (2433): No heartbeat from core client for 30 sec - exiting 09:12:36 (2433): No heartbeat from core client for 30 sec - exiting 09:12:37 (2433): No heartbeat from core client for 30 sec - exiting 09:12:38 (2433): No heartbeat from core client for 30 sec - exiting 09:12:39 (2433): No heartbeat from core client for 30 sec - exiting 09:12:40 (2433): No heartbeat from core client for 30 sec - exiting 09:12:41 (2433): No heartbeat from core client for 30 sec - exiting 09:12:42 (2433): No heartbeat from core client for 30 sec - exiting 09:12:43 (2433): No heartbeat from core client for 30 sec - exiting 09:12:44 (2433): No heartbeat from core client for 30 sec - exiting 09:12:45 (2433): No heartbeat from core client for 30 sec - exiting 09:12:46 (2433): No heartbeat from core client for 30 sec - exiting 09:12:47 (2433): No heartbeat from core client for 30 sec - exiting 09:12:48 (2433): No heartbeat from core client for 30 sec - exiting 09:12:49 (2433): No heartbeat from core client for 30 sec - exiting 09:12:50 (2433): No heartbeat from core client for 30 sec - exiting 09:12:51 (2433): No heartbeat from core client for 30 sec - exiting 09:12:52 (2433): No heartbeat from core client for 30 sec - exiting 09:12:53 (2433): No heartbeat from core client for 30 sec - exiting 09:12:54 (2433): No heartbeat from core client for 30 sec - exiting 09:12:55 (2433): No heartbeat from core client for 30 sec - exiting 09:12:56 (2433): No heartbeat from core client for 30 sec - exiting 09:12:57 (2433): No heartbeat from core client for 30 sec - exiting 09:12:58 (2433): No heartbeat from core client for 30 sec - exiting 09:12:59 (2433): No heartbeat from core client for 30 sec - exiting 09:13:00 (2433): No heartbeat from core client for 30 sec - exiting 09:13:01 (2433): No heartbeat from core client for 30 sec - exiting *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08c29768 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called 09:11:06 (2644): No heartbeat from core client for 30 sec - exiting 09:11:08 (2644): No heartbeat from core client for 30 sec - exiting 09:11:09 (2644): No heartbeat from core client for 30 sec - exiting 09:11:10 (2644): No heartbeat from core client for 30 sec - exiting 09:11:11 (2644): No heartbeat from core client for 30 sec - exiting 09:11:12 (2644): No heartbeat from core client for 30 sec - exiting 09:11:13 (2644): No heartbeat from core client for 30 sec - exiting 09:11:14 (2644): No heartbeat from core client for 30 sec - exiting 09:11:15 (2644): No heartbeat from core client for 30 sec - exiting 09:11:16 (2644): No heartbeat from core client for 30 sec - exiting 09:11:17 (2644): No heartbeat from core client for 30 sec - exiting 09:11:18 (2644): No heartbeat from core client for 30 sec - exiting 09:11:19 (2644): No heartbeat from core client for 30 sec - exiting 09:11:20 (2644): No heartbeat from core client for 30 sec - exiting 09:11:21 (2644): No heartbeat from core client for 30 sec - exiting 09:11:22 (2644): No heartbeat from core client for 30 sec - exiting 09:11:23 (2644): No heartbeat from core client for 30 sec - exiting 09:11:24 (2644): No heartbeat from core client for 30 sec - exiting 09:11:25 (2644): No heartbeat from core client for 30 sec - exiting 09:11:26 (2644): No heartbeat from core client for 30 sec - exiting 09:11:27 (2644): No heartbeat from core client for 30 sec - exiting 09:11:28 (2644): No heartbeat from core client for 30 sec - exiting 09:11:29 (2644): No heartbeat from core client for 30 sec - exiting 09:11:30 (2644): No heartbeat from core client for 30 sec - exiting 09:11:31 (2644): No heartbeat from core client for 30 sec - exiting 09:11:32 (2644): No heartbeat from core client for 30 sec - exiting 09:11:33 (2644): No heartbeat from core client for 30 sec - exiting 09:11:34 (2644): No heartbeat from core client for 30 sec - exiting 09:11:35 (2644): No heartbeat from core client for 30 sec - exiting 09:11:36 (2644): No heartbeat from core client for 30 sec - exiting 09:11:37 (2644): No heartbeat from core client for 30 sec - exiting 09:11:38 (2644): No heartbeat from core client for 30 sec - exiting 09:11:39 (2644): No heartbeat from core client for 30 sec - exiting 09:11:40 (2644): No heartbeat from core client for 30 sec - exiting 09:11:41 (2644): No heartbeat from core client for 30 sec - exiting 09:11:42 (2644): No heartbeat from core client for 30 sec - exiting 09:11:43 (2644): No heartbeat from core client for 30 sec - exiting 09:11:44 (2644): No heartbeat from core client for 30 sec - exiting 09:11:45 (2644): No heartbeat from core client for 30 sec - exiting 09:11:46 (2644): No heartbeat from core client for 30 sec - exiting 09:11:47 (2644): No heartbeat from core client for 30 sec - exiting 09:11:48 (2644): No heartbeat from core client for 30 sec - exiting 09:11:49 (2644): No heartbeat from core client for 30 sec - exiting 09:11:50 (2644): No heartbeat from core client for 30 sec - exiting 09:11:51 (2644): No heartbeat from core client for 30 sec - exiting 09:11:52 (2644): No heartbeat from core client for 30 sec - exiting 09:11:53 (2644): No heartbeat from core client for 30 sec - exiting 09:11:54 (2644): No heartbeat from core client for 30 sec - exiting 09:11:55 (2644): No heartbeat from core client for 30 sec - exiting 09:11:56 (2644): No heartbeat from core client for 30 sec - exiting 09:11:57 (2644): No heartbeat from core client for 30 sec - exiting 09:11:58 (2644): No heartbeat from core client for 30 sec - exiting 09:11:59 (2644): No heartbeat from core client for 30 sec - exiting 09:12:00 (2644): No heartbeat from core client for 30 sec - exiting 09:12:01 (2644): No heartbeat from core client for 30 sec - exiting 09:12:02 (2644): No heartbeat from core client for 30 sec - exiting *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09abf768 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called 09:12:03 (2644): No heartbeat from core client for 30 sec - exiting 09:12:04 (2644): No heartbeat from core client for 30 sec - exiting 09:12:05 (2644): No heartbeat from core client for 30 sec - exiting 09:12:06 (2644): No heartbeat from core client for 30 sec - exiting 09:12:07 (2644): No heartbeat from core client for 30 sec - exiting 09:12:09 (2644): No heartbeat from core client for 30 sec - exiting 09:12:10 (2644): No heartbeat from core client for 30 sec - exiting 09:12:11 (2644): No heartbeat from core client for 30 sec - exiting 09:12:12 (2644): No heartbeat from core client for 30 sec - exiting 09:12:13 (2644): No heartbeat from core client for 30 sec - exiting 09:12:14 (2644): No heartbeat from core client for 30 sec - exiting 09:12:15 (2644): No heartbeat from core client for 30 sec - exiting *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08c7f768 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called 12:19:27 (2425): No heartbeat from core client for 30 sec - exiting 12:19:29 (2425): No heartbeat from core client for 30 sec - exiting 12:19:30 (2425): No heartbeat from core client for 30 sec - exiting 12:19:31 (2425): No heartbeat from core client for 30 sec - exiting 12:19:32 (2425): No heartbeat from core client for 30 sec - exiting 12:19:33 (2425): No heartbeat from core client for 30 sec - exiting 12:19:34 (2425): No heartbeat from core client for 30 sec - exiting 12:19:35 (2425): No heartbeat from core client for 30 sec - exiting 12:19:36 (2425): No heartbeat from core client for 30 sec - exiting *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x0864b768 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called 09:20:08 (2666): No heartbeat from core client for 30 sec - exiting 09:20:10 (2666): No heartbeat from core client for 30 sec - exiting 09:20:11 (2666): No heartbeat from core client for 30 sec - exiting 09:20:12 (2666): No heartbeat from core client for 30 sec - exiting 09:20:13 (2666): No heartbeat from core client for 30 sec - exiting 09:20:14 (2666): No heartbeat from core client for 30 sec - exiting 09:20:15 (2666): No heartbeat from core client for 30 sec - exiting 09:20:16 (2666): No heartbeat from core client for 30 sec - exiting 09:20:17 (2666): No heartbeat from core client for 30 sec - exiting 09:20:18 (2666): No heartbeat from core client for 30 sec - exiting 09:20:19 (2666): No heartbeat from core client for 30 sec - exiting 09:20:20 (2666): No heartbeat from core client for 30 sec - exiting 09:20:21 (2666): No heartbeat from core client for 30 sec - exiting 09:20:22 (2666): No heartbeat from core client for 30 sec - exiting 09:20:23 (2666): No heartbeat from core client for 30 sec - exiting 09:20:24 (2666): No heartbeat from core client for 30 sec - exiting 09:20:25 (2666): No heartbeat from core client for 30 sec - exiting 09:20:26 (2666): No heartbeat from core client for 30 sec - exiting 09:20:27 (2666): No heartbeat from core client for 30 sec - exiting 09:20:28 (2666): No heartbeat from core client for 30 sec - exiting 09:20:29 (2666): No heartbeat from core client for 30 sec - exiting 09:20:30 (2666): No heartbeat from core client for 30 sec - exiting 09:20:31 (2666): No heartbeat from core client for 30 sec - exiting 09:20:32 (2666): No heartbeat from core client for 30 sec - exiting 09:20:33 (2666): No heartbeat from core client for 30 sec - exiting 09:20:34 (2666): No heartbeat from core client for 30 sec - exiting 09:20:35 (2666): No heartbeat from core client for 30 sec - exiting 09:20:36 (2666): No heartbeat from core client for 30 sec - exiting 09:20:37 (2666): No heartbeat from core client for 30 sec - exiting 09:20:38 (2666): No heartbeat from core client for 30 sec - exiting 09:20:39 (2666): No heartbeat from core client for 30 sec - exiting 09:20:40 (2666): No heartbeat from core client for 30 sec - exiting 09:20:41 (2666): No heartbeat from core client for 30 sec - exiting 09:20:42 (2666): No heartbeat from core client for 30 sec - exiting 09:20:43 (2666): No heartbeat from core client for 30 sec - exiting 09:20:44 (2666): No heartbeat from core client for 30 sec - exiting 09:20:45 (2666): No heartbeat from core client for 30 sec - exiting 09:20:46 (2666): No heartbeat from core client for 30 sec - exiting 09:20:47 (2666): No heartbeat from core client for 30 sec - exiting 09:20:48 (2666): No heartbeat from core client for 30 sec - exiting 09:20:49 (2666): No heartbeat from core client for 30 sec - exiting 09:20:50 (2666): No heartbeat from core client for 30 sec - exiting 09:20:51 (2666): No heartbeat from core client for 30 sec - exiting 09:20:52 (2666): No heartbeat from core client for 30 sec - exiting *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08310768 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called 09:20:53 (2666): No heartbeat from core client for 30 sec - exiting 09:20:54 (2666): No heartbeat from core client for 30 sec - exiting 09:20:55 (2666): No heartbeat from core client for 30 sec - exiting *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09235768 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08b12768 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09ffa768 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09538768 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called 13:27:49 (2293): No heartbeat from core client for 30 sec - exiting 13:27:52 (2293): No heartbeat from core client for 30 sec - exiting 13:27:53 (2293): No heartbeat from core client for 30 sec - exiting 13:27:54 (2293): No heartbeat from core client for 30 sec - exiting 13:27:55 (2293): No heartbeat from core client for 30 sec - exiting 13:27:56 (2293): No heartbeat from core client for 30 sec - exiting 13:27:57 (2293): No heartbeat from core client for 30 sec - exiting 13:27:58 (2293): No heartbeat from core client for 30 sec - exiting 13:27:59 (2293): No heartbeat from core client for 30 sec - exiting 13:28:00 (2293): No heartbeat from core client for 30 sec - exiting 13:28:01 (2293): No heartbeat from core client for 30 sec - exiting 13:28:02 (2293): No heartbeat from core client for 30 sec - exiting 13:28:03 (2293): No heartbeat from core client for 30 sec - exiting 13:28:04 (2293): No heartbeat from core client for 30 sec - exiting 13:28:05 (2293): No heartbeat from core client for 30 sec - exiting 13:28:06 (2293): No heartbeat from core client for 30 sec - exiting 13:28:07 (2293): No heartbeat from core client for 30 sec - exiting 13:28:08 (2293): No heartbeat from core client for 30 sec - exiting 13:28:09 (2293): No heartbeat from core client for 30 sec - exiting 13:28:10 (2293): No heartbeat from core client for 30 sec - exiting 13:28:11 (2293): No heartbeat from core client for 30 sec - exiting 13:28:12 (2293): No heartbeat from core client for 30 sec - exiting 13:28:13 (2293): No heartbeat from core client for 30 sec - exiting 13:28:14 (2293): No heartbeat from core client for 30 sec - exiting 13:28:15 (2293): No heartbeat from core client for 30 sec - exiting 13:28:16 (2293): No heartbeat from core client for 30 sec - exiting 13:28:17 (2293): No heartbeat from core client for 30 sec - exiting 13:28:18 (2293): No heartbeat from core client for 30 sec - exiting 13:28:19 (2293): No heartbeat from core client for 30 sec - exiting *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x082f4768 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08a2e768 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x0865d768 *** hadcm3n_6.07_i686-pc-linux-gnu: malloc.c:2451: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. SIGABRT: abort called Signal 3 received, exiting... Called boinc_finish </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
23 Jan 2013 12:25:04 | 1120190 | 15471855 | hadcm3n_zb7d_1880_40_008253943_2 | 518,400 | 927,916 | 1.7900 |
22 Jan 2013 08:25:26 | 1120190 | 15471855 | hadcm3n_zb7d_1880_40_008253943_2 | 492,480 | 885,426 | 1.7979 |
18 Jan 2013 11:41:03 | 1120190 | 15471855 | hadcm3n_zb7d_1880_40_008253943_2 | 466,560 | 839,374 | 1.7991 |
16 Jan 2013 14:45:46 | 1120190 | 15471855 | hadcm3n_zb7d_1880_40_008253943_2 | 440,640 | 793,330 | 1.8004 |
15 Jan 2013 10:18:54 | 1120190 | 15471855 | hadcm3n_zb7d_1880_40_008253943_2 | 414,720 | 747,566 | 1.8026 |
14 Jan 2013 21:30:19 | 1120190 | 15471855 | hadcm3n_zb7d_1880_40_008253943_2 | 388,800 | 701,594 | 1.8045 |
14 Jan 2013 08:33:44 | 1120190 | 15471855 | hadcm3n_zb7d_1880_40_008253943_2 | 362,880 | 655,302 | 1.8058 |
11 Jan 2013 03:43:27 | 1120190 | 15471855 | hadcm3n_zb7d_1880_40_008253943_2 | 336,960 | 609,692 | 1.8094 |
10 Jan 2013 14:55:37 | 1120190 | 15471855 | hadcm3n_zb7d_1880_40_008253943_2 | 311,040 | 563,977 | 1.8132 |
09 Jan 2013 10:25:29 | 1120190 | 15471855 | hadcm3n_zb7d_1880_40_008253943_2 | 285,120 | 518,376 | 1.8181 |
08 Jan 2013 05:31:21 | 1120190 | 15471855 | hadcm3n_zb7d_1880_40_008253943_2 | 259,200 | 472,588 | 1.8233 |
07 Jan 2013 16:47:04 | 1120190 | 15471855 | hadcm3n_zb7d_1880_40_008253943_2 | 233,280 | 426,949 | 1.8302 |
20 Dec 2012 12:11:18 | 1120190 | 15471855 | hadcm3n_zb7d_1880_40_008253943_2 | 207,360 | 381,764 | 1.8411 |
19 Dec 2012 07:46:44 | 1120190 | 15471855 | hadcm3n_zb7d_1880_40_008253943_2 | 181,440 | 337,046 | 1.8576 |
18 Dec 2012 19:10:55 | 1120190 | 15471855 | hadcm3n_zb7d_1880_40_008253943_2 | 155,520 | 292,061 | 1.8780 |
17 Dec 2012 13:20:18 | 1120190 | 15471855 | hadcm3n_zb7d_1880_40_008253943_2 | 129,600 | 247,073 | 1.9064 |
17 Dec 2012 08:14:35 | 1120190 | 15471855 | hadcm3n_zb7d_1880_40_008253943_2 | 103,680 | 199,377 | 1.9230 |
17 Dec 2012 08:14:35 | 1120190 | 15471855 | hadcm3n_zb7d_1880_40_008253943_2 | 77,760 | 149,539 | 1.9231 |
17 Dec 2012 08:14:35 | 1120190 | 15471855 | hadcm3n_zb7d_1880_40_008253943_2 | 51,840 | 99,546 | 1.9203 |
07 Dec 2012 10:57:21 | 1120190 | 15471855 | hadcm3n_zb7d_1880_40_008253943_2 | 25,920 | 49,988 | 1.9285 |
©2024 climateprediction.net