 |
|
| Name | hadcm3n_3gqo_2020_40_008362664_1 |
| Workunit | 8513523 |
| Created | 14 Aug 2013, 11:39:17 UTC |
| Sent | 14 Aug 2013, 18:13:40 UTC |
| Report deadline | 14 Nov 2013, 1:40:51 UTC |
| Received | 15 Sep 2013, 10:27:47 UTC |
| Server state | Over |
| Outcome | Computation error |
| Client state | Compute error |
| Exit status | 193 (0x000000C1) EXIT_SIGNAL |
| Computer ID | 1105896 |
| Run time | 23 days 9 hours 46 min 41 sec |
| CPU time | 22 days 0 hours 1 min 9 sec |
| Validate state | Invalid |
| Credit | 12,441.60 |
| Device peak FLOPS | 3.57 GFLOPS |
| Application version | UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu |
| Stderr | <core_client_version>6.12.34</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... 20:24:03 (2306): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:24:05 (2306): No heartbeat from core client for 30 sec - exiting 21:02:24 (806): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:27:39 (10645): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08190d88 *** ======= Backtrace: ========= /lib/libc.so.6(+0x6fd04)[0xf74d6d04] /lib/libc.so.6(+0x71613)[0xf74d8613] /lib/libc.so.6(cfree+0x6d)[0xf74db74d] /usr/lib/libstdc++.so.6(_ZdlPv+0x1f)[0xf766a4ef] /usr/lib/libstdc++.so.6(_ZdaPv+0x1b)[0xf766a53b] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/libc.so.6(__libc_start_main+0xe5)[0xf747dc25] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] ======= Memory map: ======== 08048000-080e3000 r-xp 00000000 08:11 9052272 /home/boinc/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e3000-080e4000 rw-p 0009b000 08:11 9052272 /home/boinc/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e4000-081a1000 rw-p 00000000 00:00 0 [heap] f6e00000-f6e21000 rw-p 00000000 00:00 0 f6e21000-f6f00000 ---p 00000000 00:00 0 f6fea000-f7465000 rw-s 00000000 08:11 8683596 /home/boinc/slots/4/136715 f7465000-f7467000 rw-p 00000000 00:00 0 f7467000-f75ce000 r-xp 00000000 08:02 16064517 /lib/libc-2.11.3.so f75ce000-f75cf000 ---p 00167000 08:02 16064517 /lib/libc-2.11.3.so f75cf000-f75d1000 r--p 00167000 08:02 16064517 /lib/libc-2.11.3.so f75d1000-f75d2000 rw-p 00169000 08:02 16064517 /lib/libc-2.11.3.so f75d2000-f75d5000 rw-p 00000000 00:00 0 f75d5000-f75f1000 r-xp 00000000 08:02 32014431 /lib/libgcc_s.so.1 f75f1000-f75f2000 r--p 0001b000 08:02 32014431 /lib/libgcc_s.so.1 f75f2000-f75f3000 rw-p 0001c000 08:02 32014431 /lib/libgcc_s.so.1 f75f3000-f75f4000 rw-p 00000000 00:00 0 f75f4000-f761b000 r-xp 00000000 08:02 16064534 /lib/libm-2.11.3.so f761b000-f761c000 r--p 00026000 08:02 16064534 /lib/libm-2.11.3.so f761c000-f761d000 rw-p 00027000 08:02 16064534 /lib/libm-2.11.3.so f761d000-f76fd000 r-xp 00000000 08:02 45678946 /usr/lib/libstdc++.so.6.0.17 f76fd000-f7701000 r--p 000df000 08:02 45678946 /usr/lib/libstdc++.so.6.0.17 f7701000-f7702000 rw-p 000e3000 08:02 45678946 /usr/lib/libstdc++.so.6.0.17 f7702000-f7709000 rw-p 00000000 00:00 0 f7709000-f770c000 r-xp 00000000 08:02 16064532 /lib/libdl-2.11.3.so f770c000-f770d000 r--p 00002000 08:02 16064532 /lib/libdl-2.11.3.so f770d000-f770e000 rw-p 00003000 08:02 16064532 /lib/libdl-2.11.3.so f770e000-f7725000 r-xp 00000000 08:02 16064556 /lib/libpthread-2.11.3.so f7725000-f7726000 r--p 00016000 08:02 16064556 /lib/libpthread-2.11.3.so f7726000-f7727000 rw-p 00017000 08:02 16064556 /lib/libpthread-2.11.3.so f7727000-f7729000 rw-p 00000000 00:00 0 f7779000-f777a000 rw-p 00000000 00:00 0 f777a000-f777b000 ---p 00000000 00:00 0 f777b000-f777e000 rw-p 00000000 00:00 0 f777e000-f7780000 rw-s 00000000 08:11 8683639 /home/boinc/slots/4/boinc_mmap_file f7780000-f7781000 rw-p 00000000 00:00 0 f7781000-f77a0000 r-xp 00000000 08:02 16064520 /lib/ld-2.11.3.so f77a0000-f77a1000 r--p 0001e000 08:02 16064520 /lib/ld-2.11.3.so f77a1000-f77a2000 rw-p 0001f000 08:02 16064520 /lib/ld-2.11.3.so ffbec000-ffc5b000 rw-p 00000000 00:00 0 [stack] ffffe000-fffff000 r-xp 00000000 00:00 0 [vdso] SIGABRT: abort called Stack trace (19 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xffffe400] [0xffffe430] /lib/libc.so.6(gsignal+0x50)[0xf7491e30] /lib/libc.so.6(abort+0x185)[0xf7493765] /lib/libc.so.6(+0x69d75)[0xf74d0d75] /lib/libc.so.6(+0x6fd04)[0xf74d6d04] /lib/libc.so.6(+0x71613)[0xf74d8613] /lib/libc.so.6(cfree+0x6d)[0xf74db74d] /usr/lib/libstdc++.so.6(_ZdlPv+0x1f)[0xf766a4ef] /usr/lib/libstdc++.so.6(_ZdaPv+0x1b)[0xf766a53b] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/libc.so.6(__libc_start_main+0xe5)[0xf747dc25] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... </stderr_txt> ]]> |
| Latest Trickles Received | ||||||
|---|---|---|---|---|---|---|
| Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
| 15 Sep 2013 10:29:34 | 1105896 | 15914762 | hadcm3n_3gqo_2020_40_008362664_1 | 1,036,800 | 1,900,863 | 1.8334 |
| 14 Sep 2013 14:59:34 | 1105896 | 15914762 | hadcm3n_3gqo_2020_40_008362664_1 | 1,010,880 | 1,853,371 | 1.8334 |
| 13 Sep 2013 19:23:26 | 1105896 | 15914762 | hadcm3n_3gqo_2020_40_008362664_1 | 984,960 | 1,805,733 | 1.8333 |
| 13 Sep 2013 03:14:46 | 1105896 | 15914762 | hadcm3n_3gqo_2020_40_008362664_1 | 959,040 | 1,758,220 | 1.8333 |
| 12 Sep 2013 08:54:31 | 1105896 | 15914762 | hadcm3n_3gqo_2020_40_008362664_1 | 933,120 | 1,710,623 | 1.8332 |
| 11 Sep 2013 12:27:00 | 1105896 | 15914762 | hadcm3n_3gqo_2020_40_008362664_1 | 907,200 | 1,663,065 | 1.8332 |
| 10 Sep 2013 19:27:17 | 1105896 | 15914762 | hadcm3n_3gqo_2020_40_008362664_1 | 881,280 | 1,615,600 | 1.8332 |
| 10 Sep 2013 03:02:04 | 1105896 | 15914762 | hadcm3n_3gqo_2020_40_008362664_1 | 855,360 | 1,568,171 | 1.8333 |
| 09 Sep 2013 13:15:51 | 1105896 | 15914762 | hadcm3n_3gqo_2020_40_008362664_1 | 829,440 | 1,520,614 | 1.8333 |
| 08 Sep 2013 23:46:04 | 1105896 | 15914762 | hadcm3n_3gqo_2020_40_008362664_1 | 803,520 | 1,473,026 | 1.8332 |
| 08 Sep 2013 08:57:08 | 1105896 | 15914762 | hadcm3n_3gqo_2020_40_008362664_1 | 777,600 | 1,426,286 | 1.8342 |
| 07 Sep 2013 16:51:15 | 1105896 | 15914762 | hadcm3n_3gqo_2020_40_008362664_1 | 751,680 | 1,379,335 | 1.8350 |
| 07 Sep 2013 00:52:05 | 1105896 | 15914762 | hadcm3n_3gqo_2020_40_008362664_1 | 725,760 | 1,331,138 | 1.8341 |
| 06 Sep 2013 06:31:15 | 1105896 | 15914762 | hadcm3n_3gqo_2020_40_008362664_1 | 699,840 | 1,283,798 | 1.8344 |
| 05 Sep 2013 15:40:54 | 1105896 | 15914762 | hadcm3n_3gqo_2020_40_008362664_1 | 673,920 | 1,236,121 | 1.8342 |
| 04 Sep 2013 22:00:37 | 1105896 | 15914762 | hadcm3n_3gqo_2020_40_008362664_1 | 648,000 | 1,188,556 | 1.8342 |
| 04 Sep 2013 02:18:52 | 1105896 | 15914762 | hadcm3n_3gqo_2020_40_008362664_1 | 622,080 | 1,140,791 | 1.8338 |
| 03 Sep 2013 07:29:57 | 1105896 | 15914762 | hadcm3n_3gqo_2020_40_008362664_1 | 596,160 | 1,093,458 | 1.8342 |
| 02 Sep 2013 13:06:00 | 1105896 | 15914762 | hadcm3n_3gqo_2020_40_008362664_1 | 570,240 | 1,046,098 | 1.8345 |
| 01 Sep 2013 17:37:57 | 1105896 | 15914762 | hadcm3n_3gqo_2020_40_008362664_1 | 544,320 | 997,893 | 1.8333 |
©2024 climateprediction.net