 |
|
| Name | hadcm3n_t1ip_1940_40_007543528_0 |
| Workunit | 7740760 |
| Created | 10 Nov 2011, 1:57:19 UTC |
| Sent | 16 Nov 2011, 7:24:18 UTC |
| Report deadline | 15 Feb 2012, 14:51:29 UTC |
| Received | 30 Dec 2011, 4:52:55 UTC |
| Server state | Over |
| Outcome | Computation error |
| Client state | Compute error |
| Exit status | 193 (0x000000C1) EXIT_SIGNAL |
| Computer ID | 1140527 |
| Run time | 18 days 8 hours 34 min 2 sec |
| CPU time | 16 days 15 hours 37 min 9 sec |
| Validate state | Invalid |
| Credit | 9,331.20 |
| Device peak FLOPS | 2.82 GFLOPS |
| Application version | UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu |
| Stderr | <core_client_version>6.10.58</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:55:12 (3942): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:58:25 (7811): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:58:26 (7811): No heartbeat from core client for 30 sec - exiting 16:58:27 (7811): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 20:25:43 (4356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:25:44 (4356): No heartbeat from core client for 30 sec - exiting 20:25:45 (4356): No heartbeat from core client for 30 sec - exiting 20:25:46 (4356): No heartbeat from core client for 30 sec - exiting 20:25:47 (4356): No heartbeat from core client for 30 sec - exiting 20:25:48 (4356): No heartbeat from core client for 30 sec - exiting 20:25:49 (4356): No heartbeat from core client for 30 sec - exiting 20:27:08 (6684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:27:09 (6684): No heartbeat from core client for 30 sec - exiting 20:27:10 (6684): No heartbeat from core client for 30 sec - exiting 20:27:11 (6684): No heartbeat from core client for 30 sec - exiting 20:27:12 (6684): No heartbeat from core client for 30 sec - exiting 20:27:13 (6684): No heartbeat from core client for 30 sec - exiting 20:27:14 (6684): No heartbeat from core client for 30 sec - exiting 20:27:15 (6684): No heartbeat from core client for 30 sec - exiting 20:27:16 (6684): No heartbeat from core client for 30 sec - exiting 20:27:17 (6684): No heartbeat from core client for 30 sec - exiting 20:27:18 (6684): No heartbeat from core client for 30 sec - exiting 20:27:19 (6684): No heartbeat from core client for 30 sec - exiting 20:27:20 (6684): No heartbeat from core client for 30 sec - exiting 20:27:21 (6684): No heartbeat from core client for 30 sec - exiting 20:27:22 (6684): No heartbeat from core client for 30 sec - exiting 20:27:23 (6684): No heartbeat from core client for 30 sec - exiting 20:27:24 (6684): No heartbeat from core client for 30 sec - exiting 20:27:25 (6684): No heartbeat from core client for 30 sec - exiting 20:27:26 (6684): No heartbeat from core client for 30 sec - exiting 20:27:27 (6684): No heartbeat from core client for 30 sec - exiting 20:27:28 (6684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Error: Input file: dataout/t1ipko.pjg6c10 is not a valid UM file. Error converting file to netcdf: dataout/t1ipko.pjg6c10 Error: Input file: dataout/t1ipko.pig6c10 is not a valid UM file. Error converting file to netcdf: dataout/t1ipko.pig6c10 Error: Input file: dataout/t1ipko.pfg6c10 is not a valid UM file. Error converting file to netcdf: dataout/t1ipko.pfg6c10 Error: Input file: dataout/t1ipka.phg6c10 is not a valid UM file. Error converting file to netcdf: dataout/t1ipka.phg6c10 Error: Input file: dataout/t1ipka.pgg6c10 is not a valid UM file. Error converting file to netcdf: dataout/t1ipka.pgg6c10 Error: Input file: dataout/t1ipka.peg6c10 is not a valid UM file. Error converting file to netcdf: dataout/t1ipka.peg6c10 Error: Input file: dataout/t1ipka.pdg6c10 is not a valid UM file. Error converting file to netcdf: dataout/t1ipka.pdg6c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish SIGSEGV: segmentation violation Stack trace (7 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xf76ec400] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805067e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf7432bd6] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: free(): invalid next size (normal): 0x09481168 *** ======= Backtrace: ========= /lib32/libc.so.6(+0x6c231)[0xf750e231] /lib32/libc.so.6(+0x6dab8)[0xf750fab8] /lib32/libc.so.6(cfree+0x6d)[0xf7512b9d] /usr/lib32/libstdc++.so.6(_ZdlPv+0x21)[0xf76fca91] /usr/lib32/libstdc++.so.6(_ZdaPv+0x1d)[0xf76fcaed] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf74b8bd6] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] ======= Memory map: ======== 08048000-080e3000 r-xp 00000000 fc:04 13109854 /igel2/Vol1/BOINC/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e3000-080e4000 rwxp 0009b000 fc:04 13109854 /igel2/Vol1/BOINC/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e4000-0813b000 rwxp 00000000 00:00 0 093bd000-094bd000 rwxp 00000000 00:00 0 [heap] f6f00000-f6f21000 rwxp 00000000 00:00 0 f6f21000-f7000000 ---p 00000000 00:00 0 f7025000-f74a0000 rwxs 00000000 fc:04 13107757 /igel2/Vol1/BOINC/slots/19/137145 f74a0000-f74a2000 rwxp 00000000 00:00 0 f74a2000-f75f5000 r-xp 00000000 08:22 4063237 /lib32/libc-2.11.1.so f75f5000-f75f6000 ---p 00153000 08:22 4063237 /lib32/libc-2.11.1.so f75f6000-f75f8000 r-xp 00153000 08:22 4063237 /lib32/libc-2.11.1.so f75f8000-f75f9000 rwxp 00155000 08:22 4063237 /lib32/libc-2.11.1.so f75f9000-f75fc000 rwxp 00000000 00:00 0 f75fc000-f7619000 r-xp 00000000 08:22 3282217 /usr/lib32/libgcc_s.so.1 f7619000-f761a000 r-xp 0001c000 08:22 3282217 /usr/lib32/libgcc_s.so.1 f761a000-f761b000 rwxp 0001d000 08:22 3282217 /usr/lib32/libgcc_s.so.1 f761b000-f763f000 r-xp 00000000 08:22 4063241 /lib32/libm-2.11.1.so f763f000-f7640000 r-xp 00023000 08:22 4063241 /lib32/libm-2.11.1.so f7640000-f7641000 rwxp 00024000 08:22 4063241 /lib32/libm-2.11.1.so f7641000-f772a000 r-xp 00000000 08:22 3282231 /usr/lib32/libstdc++.so.6.0.13 f772a000-f772b000 ---p 000e9000 08:22 3282231 /usr/lib32/libstdc++.so.6.0.13 f772b000-f772f000 r-xp 000e9000 08:22 3282231 /usr/lib32/libstdc++.so.6.0.13 f772f000-f7730000 rwxp 000ed000 08:22 3282231 /usr/lib32/libstdc++.so.6.0.13 f7730000-f7737000 rwxp 00000000 00:00 0 f7737000-f7739000 r-xp 00000000 08:22 4063240 /lib32/libdl-2.11.1.so f7739000-f773a000 r-xp 00001000 08:22 4063240 /lib32/libdl-2.11.1.so f773a000-f773b000 rwxp 00002000 08:22 4063240 /lib32/libdl-2.11.1.so f773b000-f7750000 r-xp 00000000 08:22 4063251 /lib32/libpthread-2.11.1.so f7750000-f7751000 r-xp 00014000 08:22 4063251 /lib32/libpthread-2.11.1.so f7751000-f7752000 rwxp 00015000 08:22 4063251 /lib32/libpthread-2.11.1.so f7752000-f7755000 rwxp 00000000 00:00 0 f7769000-f776a000 rwxp 00000000 00:00 0 f776a000-f776b000 ---p 00000000 00:00 0 f776b000-f776e000 rwxp 00000000 00:00 0 f776e000-f7770000 rwxs 00000000 fc:04 13107759 /igel2/Vol1/BOINC/slots/19/boinc_mmap_file f7770000-f7772000 rwxp 00000000 00:00 0 f7772000-f7773000 r-xp 00000000 00:00 0 [vdso] f7773000-f778f000 r-xp 00000000 08:22 4063234 /lib32/ld-2.11.1.so f778f000-f7790000 r-xp 0001b000 08:22 4063234 /lib32/ld-2.11.1.so f7790000-f7791000 rwxp 0001c000 08:22 4063234 /lib32/ld-2.11.1.so ffd67000-ffdd8000 rw-p 00000000 00:00 0 [stack] SIGABRT: abort called Stack trace (19 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xf7772400] [0xf7772430] /lib32/libc.so.6(gsignal+0x51)[0xf74cc921] /lib32/libc.so.6(abort+0x182)[0xf74cfd52] /lib32/libc.so.6(+0x6213d)[0xf750413d] /lib32/libc.so.6(+0x6c231)[0xf750e231] /lib32/libc.so.6(+0x6dab8)[0xf750fab8] /lib32/libc.so.6(cfree+0x6d)[0xf7512b9d] /usr/lib32/libstdc++.so.6(_ZdlPv+0x21)[0xf76fca91] /usr/lib32/libstdc++.so.6(_ZdaPv+0x1d)[0xf76fcaed] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf74b8bd6] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... </stderr_txt> ]]> |
| Latest Trickles Received | ||||||
|---|---|---|---|---|---|---|
| Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
| 30 Dec 2011 04:53:54 | 1140527 | 13627056 | hadcm3n_t1ip_1940_40_007543528_0 | 777,600 | 1,438,626 | 1.8501 |
| 29 Dec 2011 11:24:41 | 1140527 | 13627056 | hadcm3n_t1ip_1940_40_007543528_0 | 751,680 | 1,390,168 | 1.8494 |
| 28 Dec 2011 17:16:58 | 1140527 | 13627056 | hadcm3n_t1ip_1940_40_007543528_0 | 725,760 | 1,341,813 | 1.8488 |
| 25 Dec 2011 21:32:46 | 1140527 | 13627056 | hadcm3n_t1ip_1940_40_007543528_0 | 699,840 | 1,292,796 | 1.8473 |
| 25 Dec 2011 07:48:44 | 1140527 | 13627056 | hadcm3n_t1ip_1940_40_007543528_0 | 673,920 | 1,245,409 | 1.8480 |
| 24 Dec 2011 20:00:09 | 1140527 | 13627056 | hadcm3n_t1ip_1940_40_007543528_0 | 648,000 | 1,203,018 | 1.8565 |
| 24 Dec 2011 04:54:14 | 1140527 | 13627056 | hadcm3n_t1ip_1940_40_007543528_0 | 622,080 | 1,149,271 | 1.8475 |
| 21 Dec 2011 04:41:58 | 1140527 | 13627056 | hadcm3n_t1ip_1940_40_007543528_0 | 596,160 | 1,096,270 | 1.8389 |
| 20 Dec 2011 10:49:52 | 1140527 | 13627056 | hadcm3n_t1ip_1940_40_007543528_0 | 570,240 | 1,050,651 | 1.8425 |
| 19 Dec 2011 19:48:42 | 1140527 | 13627056 | hadcm3n_t1ip_1940_40_007543528_0 | 544,320 | 998,009 | 1.8335 |
| 16 Dec 2011 06:56:04 | 1140527 | 13627056 | hadcm3n_t1ip_1940_40_007543528_0 | 518,400 | 955,600 | 1.8434 |
| 15 Dec 2011 18:09:54 | 1140527 | 13627056 | hadcm3n_t1ip_1940_40_007543528_0 | 492,480 | 908,041 | 1.8438 |
| 15 Dec 2011 04:08:06 | 1140527 | 13627056 | hadcm3n_t1ip_1940_40_007543528_0 | 466,560 | 858,741 | 1.8406 |
| 14 Dec 2011 14:44:34 | 1140527 | 13627056 | hadcm3n_t1ip_1940_40_007543528_0 | 440,640 | 810,059 | 1.8384 |
| 14 Dec 2011 00:10:24 | 1140527 | 13627056 | hadcm3n_t1ip_1940_40_007543528_0 | 414,720 | 758,812 | 1.8297 |
| 12 Dec 2011 04:02:10 | 1140527 | 13627056 | hadcm3n_t1ip_1940_40_007543528_0 | 388,800 | 705,130 | 1.8136 |
| 11 Dec 2011 09:26:56 | 1140527 | 13627056 | hadcm3n_t1ip_1940_40_007543528_0 | 362,880 | 653,130 | 1.7999 |
| 10 Dec 2011 17:13:03 | 1140527 | 13627056 | hadcm3n_t1ip_1940_40_007543528_0 | 336,960 | 594,742 | 1.7650 |
| 28 Nov 2011 08:50:34 | 1140527 | 13627056 | hadcm3n_t1ip_1940_40_007543528_0 | 311,040 | 543,930 | 1.7487 |
| 27 Nov 2011 14:07:07 | 1140527 | 13627056 | hadcm3n_t1ip_1940_40_007543528_0 | 285,120 | 494,650 | 1.7349 |
©2024 climateprediction.net