 |
|
| Name | hadcm3n_xd0a_198012_480_351_010332462_1 |
| Workunit | 10332462 |
| Created | 6 Mar 2016, 4:57:34 UTC |
| Sent | 6 Mar 2016, 4:57:43 UTC |
| Report deadline | 16 Feb 2017, 10:17:43 UTC |
| Received | 6 Apr 2016, 16:49:41 UTC |
| Server state | Over |
| Outcome | Computation error |
| Client state | Compute error |
| Exit status | 193 (0x000000C1) EXIT_SIGNAL |
| Computer ID | 1228943 |
| Run time | 15 days 7 hours 3 min 7 sec |
| CPU time | 14 days 16 hours 40 min 24 sec |
| Validate state | Invalid |
| Credit | 6,220.80 |
| Device peak FLOPS | 1.68 GFLOPS |
| Application version | UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu |
| Stderr | <core_client_version>6.10.58</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... 07:26:31 (23111): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 19:33:48 (12018): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:25:01 (26814): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:25:03 (26814): No heartbeat from core client for 30 sec - exiting 09:25:04 (26814): No heartbeat from core client for 30 sec - exiting 09:25:05 (26814): No heartbeat from core client for 30 sec - exiting 09:25:06 (26814): No heartbeat from core client for 30 sec - exiting 09:25:07 (26814): No heartbeat from core client for 30 sec - exiting 09:25:08 (26814): No heartbeat from core client for 30 sec - exiting 09:25:09 (26814): No heartbeat from core client for 30 sec - exiting 09:25:10 (26814): No heartbeat from core client for 30 sec - exiting 09:25:11 (26814): No heartbeat from core client for 30 sec - exiting 09:25:12 (26814): No heartbeat from core client for 30 sec - exiting 09:25:13 (26814): No heartbeat from core client for 30 sec - exiting 09:25:14 (26814): No heartbeat from core client for 30 sec - exiting 09:25:15 (26814): No heartbeat from core client for 30 sec - exiting 09:25:16 (26814): No heartbeat from core client for 30 sec - exiting 09:25:18 (26814): No heartbeat from core client for 30 sec - exiting 09:25:19 (26814): No heartbeat from core client for 30 sec - exiting 09:25:20 (26814): No heartbeat from core client for 30 sec - exiting 09:25:21 (26814): No heartbeat from core client for 30 sec - exiting 09:25:22 (26814): No heartbeat from core client for 30 sec - exiting 09:25:23 (26814): No heartbeat from core client for 30 sec - exiting 09:25:24 (26814): No heartbeat from core client for 30 sec - exiting 09:25:25 (26814): No heartbeat from core client for 30 sec - exiting 09:25:26 (26814): No heartbeat from core client for 30 sec - exiting 09:25:27 (26814): No heartbeat from core client for 30 sec - exiting 09:25:28 (26814): No heartbeat from core client for 30 sec - exiting 09:25:29 (26814): No heartbeat from core client for 30 sec - exiting 09:25:30 (26814): No heartbeat from core client for 30 sec - exiting 09:25:31 (26814): No heartbeat from core client for 30 sec - exiting 09:25:32 (26814): No heartbeat from core client for 30 sec - exiting 09:25:33 (26814): No heartbeat from core client for 30 sec - exiting 09:25:34 (26814): No heartbeat from core client for 30 sec - exiting 09:25:35 (26814): No heartbeat from core client for 30 sec - exiting 09:25:36 (26814): No heartbeat from core client for 30 sec - exiting 09:25:37 (26814): No heartbeat from core client for 30 sec - exiting 09:25:38 (26814): No heartbeat from core client for 30 sec - exiting 09:25:39 (26814): No heartbeat from core client for 30 sec - exiting 09:25:40 (26814): No heartbeat from core client for 30 sec - exiting 09:25:41 (26814): No heartbeat from core client for 30 sec - exiting 09:25:42 (26814): No heartbeat from core client for 30 sec - exiting 09:25:43 (26814): No heartbeat from core client for 30 sec - exiting 09:25:44 (26814): No heartbeat from core client for 30 sec - exiting 09:25:45 (26814): No heartbeat from core client for 30 sec - exiting 09:25:46 (26814): No heartbeat from core client for 30 sec - exiting 09:25:47 (26814): No heartbeat from core client for 30 sec - exiting 09:25:48 (26814): No heartbeat from core client for 30 sec - exiting 09:25:49 (26814): No heartbeat from core client for 30 sec - exiting 09:25:50 (26814): No heartbeat from core client for 30 sec - exiting 09:25:51 (26814): No heartbeat from core client for 30 sec - exiting 09:25:52 (26814): No heartbeat from core client for 30 sec - exiting 09:25:53 (26814): No heartbeat from core client for 30 sec - exiting 09:25:54 (26814): No heartbeat from core client for 30 sec - exiting 09:25:55 (26814): No heartbeat from core client for 30 sec - exiting 09:25:56 (26814): No heartbeat from core client for 30 sec - exiting 09:25:57 (26814): No heartbeat from core client for 30 sec - exiting 09:25:58 (26814): No heartbeat from core client for 30 sec - exiting 09:25:59 (26814): No heartbeat from core client for 30 sec - exiting 09:26:00 (26814): No heartbeat from core client for 30 sec - exiting 09:26:01 (26814): No heartbeat from core client for 30 sec - exiting 09:26:02 (26814): No heartbeat from core client for 30 sec - exiting 09:26:03 (26814): No heartbeat from core client for 30 sec - exiting 09:26:04 (26814): No heartbeat from core client for 30 sec - exiting 09:26:05 (26814): No heartbeat from core client for 30 sec - exiting 09:26:06 (26814): No heartbeat from core client for 30 sec - exiting 09:26:07 (26814): No heartbeat from core client for 30 sec - exiting 09:26:08 (26814): No heartbeat from core client for 30 sec - exiting 09:26:09 (26814): No heartbeat from core client for 30 sec - exiting 09:26:10 (26814): No heartbeat from core client for 30 sec - exiting 09:26:11 (26814): No heartbeat from core client for 30 sec - exiting 09:26:12 (26814): No heartbeat from core client for 30 sec - exiting 09:26:13 (26814): No heartbeat from core client for 30 sec - exiting 09:26:14 (26814): No heartbeat from core client for 30 sec - exiting 09:26:15 (26814): No heartbeat from core client for 30 sec - exiting 09:26:16 (26814): No heartbeat from core client for 30 sec - exiting 09:26:17 (26814): No heartbeat from core client for 30 sec - exiting 09:26:18 (26814): No heartbeat from core client for 30 sec - exiting 09:26:19 (26814): No heartbeat from core client for 30 sec - exiting 09:26:20 (26814): No heartbeat from core client for 30 sec - exiting 09:26:21 (26814): No heartbeat from core client for 30 sec - exiting 09:26:22 (26814): No heartbeat from core client for 30 sec - exiting 09:26:23 (26814): No heartbeat from core client for 30 sec - exiting 09:26:24 (26814): No heartbeat from core client for 30 sec - exiting 09:29:14 (18875): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:29:15 (18875): No heartbeat from core client for 30 sec - exiting 09:29:16 (18875): No heartbeat from core client for 30 sec - exiting 09:29:17 (18875): No heartbeat from core client for 30 sec - exiting 09:29:18 (18875): No heartbeat from core client for 30 sec - exiting 09:29:19 (18875): No heartbeat from core client for 30 sec - exiting 09:29:20 (18875): No heartbeat from core client for 30 sec - exiting 09:29:21 (18875): No heartbeat from core client for 30 sec - exiting 09:29:22 (18875): No heartbeat from core client for 30 sec - exiting 09:29:23 (18875): No heartbeat from core client for 30 sec - exiting 09:29:24 (18875): No heartbeat from core client for 30 sec - exiting 09:29:25 (18875): No heartbeat from core client for 30 sec - exiting 09:30:32 (20964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:32:18 (22747): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:33:50 (24464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:35:15 (26033): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:36:32 (27438): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:57:58 (28697): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:50:38 (5102): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:52:58 (10070): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:59:37 (7049): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:00:39 (32185): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:06:46 (12759): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:06:47 (12759): No heartbeat from core client for 30 sec - exiting 06:06:48 (12759): No heartbeat from core client for 30 sec - exiting 06:06:49 (12759): No heartbeat from core client for 30 sec - exiting 06:06:50 (12759): No heartbeat from core client for 30 sec - exiting 06:06:51 (12759): No heartbeat from core client for 30 sec - exiting 06:06:52 (12759): No heartbeat from core client for 30 sec - exiting 06:06:53 (12759): No heartbeat from core client for 30 sec - exiting 06:06:54 (12759): No heartbeat from core client for 30 sec - exiting 07:09:58 (7212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 19:06:10 (4427): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:06:14 (4427): No heartbeat from core client for 30 sec - exiting 19:06:15 (4427): No heartbeat from core client for 30 sec - exiting 19:06:16 (4427): No heartbeat from core client for 30 sec - exiting 19:06:17 (4427): No heartbeat from core client for 30 sec - exiting 19:06:18 (4427): No heartbeat from core client for 30 sec - exiting 20:20:18 (24421): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:23:16 (22143): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:40:41 (5576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:40:42 (5576): No heartbeat from core client for 30 sec - exiting 01:48:43 (19664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:48:44 (19664): No heartbeat from core client for 30 sec - exiting 09:37:21 (21710): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:38:20 (13699): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:39:33 (14735): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:42:24 (18931): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:44:45 (21621): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:46:36 (23770): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:48:10 (25643): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:49:34 (27232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:50:57 (28821): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:10:47 (30160): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09adbef8 *** ======= Backtrace: ========= /lib/libc.so.6[0x4a3c81] /lib/libc.so.6[0x4a6601] /usr/lib/libstdc++.so.6(_ZdlPv+0x22)[0x70c552] /usr/lib/libstdc++.so.6(_ZdaPv+0x1e)[0x70c5ae] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/libc.so.6(__libc_start_main+0xe6)[0x449d36] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] ======= Memory map: ======== 00411000-0042f000 r-xp 00000000 08:03 1081069 /lib/ld-2.12.so 0042f000-00430000 r-xp 0001d000 08:03 1081069 /lib/ld-2.12.so 00430000-00431000 rwxp 0001e000 08:03 1081069 /lib/ld-2.12.so 00433000-005c3000 r-xp 00000000 08:03 1051683 /lib/libc-2.12.so 005c3000-005c4000 ---p 00190000 08:03 1051683 /lib/libc-2.12.so 005c4000-005c6000 r-xp 00190000 08:03 1051683 /lib/libc-2.12.so 005c6000-005c7000 rwxp 00192000 08:03 1051683 /lib/libc-2.12.so 005c7000-005ca000 rwxp 00000000 00:00 0 005cc000-005f4000 r-xp 00000000 08:03 1080935 /lib/libm-2.12.so 005f4000-005f5000 r-xp 00027000 08:03 1080935 /lib/libm-2.12.so 005f5000-005f6000 rwxp 00028000 08:03 1080935 /lib/libm-2.12.so 005f8000-0060f000 r-xp 00000000 08:03 1080943 /lib/libpthread-2.12.so 0060f000-00610000 r-xp 00016000 08:03 1080943 /lib/libpthread-2.12.so 00610000-00611000 rwxp 00017000 08:03 1080943 /lib/libpthread-2.12.so 00611000-00613000 rwxp 00000000 00:00 0 00615000-00618000 r-xp 00000000 08:03 1080959 /lib/libdl-2.12.so 00618000-00619000 r-xp 00002000 08:03 1080959 /lib/libdl-2.12.so 00619000-0061a000 rwxp 00003000 08:03 1080959 /lib/libdl-2.12.so 0063d000-0065a000 r-xp 00000000 08:03 1081579 /lib/libgcc_s-4.4.7-20120601.so.1 0065a000-0065b000 rwxp 0001d000 08:03 1081579 /lib/libgcc_s-4.4.7-20120601.so.1 0065d000-0073e000 r-xp 00000000 08:03 957284 /usr/lib/libstdc++.so.6.0.13 0073e000-00742000 r-xp 000e0000 08:03 957284 /usr/lib/libstdc++.so.6.0.13 00742000-00744000 rwxp 000e4000 08:03 957284 /usr/lib/libstdc++.so.6.0.13 00744000-0074a000 rwxp 00000000 00:00 0 08048000-080e3000 r-xp 00000000 08:04 1704227 /scratch/boinc/BOINC/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e3000-080e4000 rwxp 0009b000 08:04 1704227 /scratch/boinc/BOINC/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e4000-0813b000 rwxp 00000000 00:00 0 099dd000-09b4f000 rwxp 00000000 00:00 0 [heap] f6f00000-f6f21000 rwxp 00000000 00:00 0 f6f21000-f7000000 ---p 00000000 00:00 0 f7100000-f7121000 rwxp 00000000 00:00 0 f7121000-f7200000 ---p 00000000 00:00 0 f7234000-f76af000 rwxs 00000000 08:04 2490523 /scratch/boinc/BOINC/slots/0/200545 f76af000-f76b2000 rwxp 00000000 00:00 0 f771d000-f771f000 rwxp 00000000 00:00 0 f771f000-f7720000 ---p 00000000 00:00 0 f7720000-f7723000 rwxp 00000000 00:00 0 f7723000-f7725000 rwxs 00000000 08:04 2490520 /scratch/boinc/BOINC/slots/0/boinc_mmap_file f7725000-f7726000 rwxp 00000000 00:00 0 f7726000-f7727000 r-xp 00000000 00:00 0 [vdso] ffa08000-ffa79000 rw-p 00000000 00:00 0 [stack] SIGABRT: abort called Stack trace (18 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xf7726400] [0xf7726430] /lib/libc.so.6(gsignal+0x51)[0x45d871] /lib/libc.so.6(abort+0x17a)[0x45f14a] /lib/libc.so.6[0x49d735] /lib/libc.so.6[0x4a3c81] /lib/libc.so.6[0x4a6601] /usr/lib/libstdc++.so.6(_ZdlPv+0x22)[0x70c552] /usr/lib/libstdc++.so.6(_ZdaPv+0x1e)[0x70c5ae] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/libc.so.6(__libc_start_main+0xe6)[0x449d36] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... </stderr_txt> ]]> |
| Latest Trickles Received | ||||||
|---|---|---|---|---|---|---|
| Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
| 06 Apr 2016 15:56:27 | 1228943 | 19325447 | hadcm3n_xd0a_198012_480_351_010332462_1 | 518,400 | 1,269,619 | 2.4491 |
| 05 Apr 2016 21:00:14 | 1228943 | 19325447 | hadcm3n_xd0a_198012_480_351_010332462_1 | 492,480 | 1,205,485 | 2.4478 |
| 05 Apr 2016 02:43:30 | 1228943 | 19325447 | hadcm3n_xd0a_198012_480_351_010332462_1 | 466,560 | 1,141,355 | 2.4463 |
| 04 Apr 2016 08:08:26 | 1228943 | 19325447 | hadcm3n_xd0a_198012_480_351_010332462_1 | 440,640 | 1,077,164 | 2.4445 |
| 03 Apr 2016 12:37:07 | 1228943 | 19325447 | hadcm3n_xd0a_198012_480_351_010332462_1 | 414,720 | 1,012,918 | 2.4424 |
| 02 Apr 2016 17:43:23 | 1228943 | 19325447 | hadcm3n_xd0a_198012_480_351_010332462_1 | 388,800 | 948,629 | 2.4399 |
| 01 Apr 2016 23:06:03 | 1228943 | 19325447 | hadcm3n_xd0a_198012_480_351_010332462_1 | 362,880 | 884,129 | 2.4364 |
| 01 Apr 2016 04:17:29 | 1228943 | 19325447 | hadcm3n_xd0a_198012_480_351_010332462_1 | 336,960 | 819,787 | 2.4329 |
| 31 Mar 2016 09:04:36 | 1228943 | 19325447 | hadcm3n_xd0a_198012_480_351_010332462_1 | 311,040 | 755,611 | 2.4293 |
| 30 Mar 2016 14:38:53 | 1228943 | 19325447 | hadcm3n_xd0a_198012_480_351_010332462_1 | 285,120 | 691,532 | 2.4254 |
| 29 Mar 2016 20:10:50 | 1228943 | 19325447 | hadcm3n_xd0a_198012_480_351_010332462_1 | 259,200 | 627,486 | 2.4209 |
| 29 Mar 2016 01:10:08 | 1228943 | 19325447 | hadcm3n_xd0a_198012_480_351_010332462_1 | 233,280 | 563,619 | 2.4161 |
| 28 Mar 2016 07:17:53 | 1228943 | 19325447 | hadcm3n_xd0a_198012_480_351_010332462_1 | 207,360 | 501,353 | 2.4178 |
| 27 Mar 2016 13:18:37 | 1228943 | 19325447 | hadcm3n_xd0a_198012_480_351_010332462_1 | 181,440 | 438,959 | 2.4193 |
| 26 Mar 2016 18:38:05 | 1228943 | 19325447 | hadcm3n_xd0a_198012_480_351_010332462_1 | 155,520 | 376,359 | 2.4200 |
| 26 Mar 2016 00:28:34 | 1228943 | 19325447 | hadcm3n_xd0a_198012_480_351_010332462_1 | 129,600 | 313,667 | 2.4203 |
| 25 Mar 2016 06:30:29 | 1228943 | 19325447 | hadcm3n_xd0a_198012_480_351_010332462_1 | 103,680 | 250,948 | 2.4204 |
| 24 Mar 2016 12:31:36 | 1228943 | 19325447 | hadcm3n_xd0a_198012_480_351_010332462_1 | 77,760 | 188,189 | 2.4201 |
| 23 Mar 2016 18:30:52 | 1228943 | 19325447 | hadcm3n_xd0a_198012_480_351_010332462_1 | 51,840 | 125,467 | 2.4203 |
| 23 Mar 2016 01:30:16 | 1228943 | 19325447 | hadcm3n_xd0a_198012_480_351_010332462_1 | 25,920 | 62,762 | 2.4214 |
©2024 climateprediction.net