 |
|
| Name | hadcm3n_o0m6_1940_40_007267511_0 |
| Workunit | 7465751 |
| Created | 3 Jun 2011, 3:51:30 UTC |
| Sent | 3 Jun 2011, 3:51:44 UTC |
| Report deadline | 2 Sep 2011, 11:18:55 UTC |
| Received | 23 Aug 2011, 10:52:59 UTC |
| Server state | Over |
| Outcome | Computation error |
| Client state | Compute error |
| Exit status | 193 (0x000000C1) EXIT_SIGNAL |
| Computer ID | 919479 |
| Run time | 21 days 15 hours 18 min 34 sec |
| CPU time | 21 days 13 hours 41 min 9 sec |
| Validate state | Invalid |
| Credit | 12,441.60 |
| Device peak FLOPS | 2.02 GFLOPS |
| Application version | UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu |
| Stderr | <core_client_version>6.10.58</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:01:03 (3129): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:01:51 (18793): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:05:59 (3144): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:06:37 (3100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:06:35 (12749): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:57:14 (17308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:39:39 (5382): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:04:18 (20127): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:06:18 (3156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08d8fcc8 *** ======= Backtrace: ========= /lib32/libc.so.6(+0x6bf31)[0xf7570f31] /lib32/libc.so.6(+0x6d7a8)[0xf75727a8] /lib32/libc.so.6(cfree+0x6d)[0xf757588d] /usr/lib32/libstdc++.so.6(_ZdlPv+0x21)[0xf774ba31] /usr/lib32/libstdc++.so.6(_ZdaPv+0x1d)[0xf774ba8d] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf751bc76] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] ======= Memory map: ======== 08048000-080e3000 r-xp 00000000 09:07 36978713 /srv/boinc/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e3000-080e4000 rw-p 0009b000 09:07 36978713 /srv/boinc/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e4000-0813b000 rw-p 00000000 00:00 0 08d3a000-08da0000 rw-p 00000000 00:00 0 [heap] f6f00000-f6f21000 rw-p 00000000 00:00 0 f6f21000-f7000000 ---p 00000000 00:00 0 f7088000-f7503000 rw-s 00000000 09:07 36987023 /srv/boinc/slots/1/133165 f7503000-f7505000 rw-p 00000000 00:00 0 f7505000-f7645000 r-xp 00000000 09:00 852004 /lib32/libc-2.11.2.so f7645000-f7646000 ---p 00140000 09:00 852004 /lib32/libc-2.11.2.so f7646000-f7648000 r--p 00140000 09:00 852004 /lib32/libc-2.11.2.so f7648000-f7649000 rw-p 00142000 09:00 852004 /lib32/libc-2.11.2.so f7649000-f764c000 rw-p 00000000 00:00 0 f764c000-f7669000 r-xp 00000000 09:00 656063 /usr/lib32/libgcc_s.so.1 f7669000-f766a000 rw-p 0001c000 09:00 656063 /usr/lib32/libgcc_s.so.1 f766a000-f768e000 r-xp 00000000 09:00 852013 /lib32/libm-2.11.2.so f768e000-f768f000 r--p 00023000 09:00 852013 /lib32/libm-2.11.2.so f768f000-f7690000 rw-p 00024000 09:00 852013 /lib32/libm-2.11.2.so f7690000-f7779000 r-xp 00000000 09:00 656070 /usr/lib32/libstdc++.so.6.0.13 f7779000-f777d000 r--p 000e9000 09:00 656070 /usr/lib32/libstdc++.so.6.0.13 f777d000-f777e000 rw-p 000ed000 09:00 656070 /usr/lib32/libstdc++.so.6.0.13 f777e000-f7785000 rw-p 00000000 00:00 0 f7785000-f7787000 r-xp 00000000 09:00 852003 /lib32/libdl-2.11.2.so f7787000-f7788000 r--p 00001000 09:00 852003 /lib32/libdl-2.11.2.so f7788000-f7789000 rw-p 00002000 09:00 852003 /lib32/libdl-2.11.2.so f7789000-f778a000 rw-p 00000000 00:00 0 f778a000-f779f000 r-xp 00000000 09:00 852005 /lib32/libpthread-2.11.2.so f779f000-f77a0000 r--p 00014000 09:00 852005 /lib32/libpthread-2.11.2.so f77a0000-f77a1000 rw-p 00015000 09:00 852005 /lib32/libpthread-2.11.2.so f77a1000-f77a3000 rw-p 00000000 00:00 0 f77b5000-f77b6000 rw-p 00000000 00:00 0 f77b6000-f77b7000 ---p 00000000 00:00 0 f77b7000-f77ba000 rw-p 00000000 00:00 0 f77ba000-f77bc000 rw-s 00000000 09:07 36986902 /srv/boinc/slots/1/boinc_mmap_file f77bc000-f77be000 rw-p 00000000 00:00 0 f77be000-f77bf000 r-xp 00000000 00:00 0 [vdso] f77bf000-f77db000 r-xp 00000000 09:00 852016 /lib32/ld-2.11.2.so f77db000-f77dc000 r--p 0001b000 09:00 852016 /lib32/ld-2.11.2.so f77dc000-f77dd000 rw-p 0001c000 09:00 852016 /lib32/ld-2.11.2.so ffcb5000-ffd24000 rw-p 00000000 00:00 0 [stack] SIGABRT: abort called Stack trace (19 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xf77be400] [0xf77be425] /lib32/libc.so.6(gsignal+0x51)[0xf752fa21] /lib32/libc.so.6(abort+0x182)[0xf7532e42] /lib32/libc.so.6(+0x61e35)[0xf7566e35] /lib32/libc.so.6(+0x6bf31)[0xf7570f31] /lib32/libc.so.6(+0x6d7a8)[0xf75727a8] /lib32/libc.so.6(cfree+0x6d)[0xf757588d] /usr/lib32/libstdc++.so.6(_ZdlPv+0x21)[0xf774ba31] /usr/lib32/libstdc++.so.6(_ZdaPv+0x1d)[0xf774ba8d] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf751bc76] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... </stderr_txt> ]]> |
| Latest Trickles Received | ||||||
|---|---|---|---|---|---|---|
| Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
| 23 Aug 2011 10:53:05 | 919479 | 12928618 | hadcm3n_o0m6_1940_40_007267511_0 | 1,036,800 | 1,863,667 | 1.7975 |
| 19 Aug 2011 15:59:33 | 919479 | 12928618 | hadcm3n_o0m6_1940_40_007267511_0 | 1,010,880 | 1,817,095 | 1.7975 |
| 17 Aug 2011 11:07:23 | 919479 | 12928618 | hadcm3n_o0m6_1940_40_007267511_0 | 984,960 | 1,770,616 | 1.7977 |
| 15 Aug 2011 12:40:19 | 919479 | 12928618 | hadcm3n_o0m6_1940_40_007267511_0 | 959,040 | 1,723,649 | 1.7973 |
| 12 Aug 2011 07:03:56 | 919479 | 12928618 | hadcm3n_o0m6_1940_40_007267511_0 | 933,120 | 1,676,981 | 1.7972 |
| 10 Aug 2011 07:33:38 | 919479 | 12928618 | hadcm3n_o0m6_1940_40_007267511_0 | 907,200 | 1,630,356 | 1.7971 |
| 08 Aug 2011 11:23:59 | 919479 | 12928618 | hadcm3n_o0m6_1940_40_007267511_0 | 881,280 | 1,583,761 | 1.7971 |
| 06 Aug 2011 12:58:04 | 919479 | 12928618 | hadcm3n_o0m6_1940_40_007267511_0 | 855,360 | 1,536,882 | 1.7968 |
| 03 Aug 2011 14:10:04 | 919479 | 12928618 | hadcm3n_o0m6_1940_40_007267511_0 | 829,440 | 1,490,128 | 1.7965 |
| 01 Aug 2011 12:01:11 | 919479 | 12928618 | hadcm3n_o0m6_1940_40_007267511_0 | 803,520 | 1,443,462 | 1.7964 |
| 30 Jul 2011 10:38:45 | 919479 | 12928618 | hadcm3n_o0m6_1940_40_007267511_0 | 777,600 | 1,396,941 | 1.7965 |
| 28 Jul 2011 10:00:32 | 919479 | 12928618 | hadcm3n_o0m6_1940_40_007267511_0 | 751,680 | 1,350,286 | 1.7964 |
| 26 Jul 2011 08:52:49 | 919479 | 12928618 | hadcm3n_o0m6_1940_40_007267511_0 | 725,760 | 1,303,657 | 1.7963 |
| 25 Jul 2011 21:50:26 | 919479 | 12928618 | hadcm3n_o0m6_1940_40_007267511_0 | 699,840 | 1,257,101 | 1.7963 |
| 25 Jul 2011 19:02:17 | 919479 | 12928618 | hadcm3n_o0m6_1940_40_007267511_0 | 673,920 | 1,210,358 | 1.7960 |
| 25 Jul 2011 17:53:12 | 919479 | 12928618 | hadcm3n_o0m6_1940_40_007267511_0 | 648,000 | 1,163,878 | 1.7961 |
| 25 Jul 2011 15:32:11 | 919479 | 12928618 | hadcm3n_o0m6_1940_40_007267511_0 | 622,080 | 1,117,199 | 1.7959 |
| 25 Jul 2011 14:30:36 | 919479 | 12928618 | hadcm3n_o0m6_1940_40_007267511_0 | 596,160 | 1,070,669 | 1.7959 |
| 25 Jul 2011 14:30:36 | 919479 | 12928618 | hadcm3n_o0m6_1940_40_007267511_0 | 570,240 | 1,023,976 | 1.7957 |
| 09 Jul 2011 17:45:51 | 919479 | 12928618 | hadcm3n_o0m6_1940_40_007267511_0 | 544,320 | 977,382 | 1.7956 |
©2024 climateprediction.net