 |
|
| Name | hadcm3n_8aru_1980_40_008723381_0 |
| Workunit | 8869359 |
| Created | 23 Apr 2014, 12:59:43 UTC |
| Sent | 3 May 2014, 21:06:49 UTC |
| Report deadline | 3 Aug 2014, 4:34:00 UTC |
| Received | 10 May 2014, 8:51:53 UTC |
| Server state | Over |
| Outcome | Computation error |
| Client state | Compute error |
| Exit status | 193 (0x000000C1) EXIT_SIGNAL |
| Computer ID | 1322662 |
| Run time | 6 days 7 hours 30 min 27 sec |
| CPU time | 6 days 3 hours 55 min 38 sec |
| Validate state | Invalid |
| Credit | 5,909.76 |
| Device peak FLOPS | 3.84 GFLOPS |
| Application version | UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu |
| Stderr | <core_client_version>7.2.42</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> 23:20:28 (12170): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:56:50 (13870): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:11:36 (14239): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:12:49 (25692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:16:39 (25721): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:19:36 (25753): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:22:39 (25783): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:25:28 (25811): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:27:51 (25840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:31:00 (25869): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:34:02 (25913): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:36:35 (25943): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:39:43 (25971): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:42:51 (26001): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:45:10 (26029): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 16:47:23 (26059): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:50:25 (26088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:52:53 (26119): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:55:42 (26148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:58:35 (26177): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:01:08 (26207): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:04:12 (26250): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:07:24 (26278): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:10:08 (26308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:13:20 (26338): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:16:34 (26366): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:19:47 (26402): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:22:09 (26432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:24:58 (26460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:27:51 (26489): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:30:44 (26522): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:33:02 (26565): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:35:20 (26593): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:38:03 (26622): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:40:26 (26652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:43:04 (26680): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:45:28 (26709): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:48:37 (26737): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:51:49 (26768): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:54:17 (26797): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:57:21 (26833): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:00:23 (26863): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:03:11 (26905): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:06:10 (26935): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:09:02 (26963): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:11:56 (26993): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:14:54 (27022): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:17:31 (27051): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:20:40 (27084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:23:08 (27112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:25:26 (27141): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:27:54 (27169): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:30:12 (27200): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:32:30 (27242): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:35:24 (27271): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:38:36 (27299): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:41:50 (27329): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:44:38 (27359): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:47:01 (27387): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:49:44 (27416): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:52:07 (27445): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:55:06 (27473): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:57:53 (27504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:00:21 (27533): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:02:44 (27576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:05:38 (27604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:08:31 (27633): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:11:04 (27663): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:13:52 (27693): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:16:46 (27721): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:19:33 (27754): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:22:06 (27784): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:24:44 (27812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:27:02 (27841): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:29:20 (27870): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:31:38 (27898): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:34:32 (27941): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:37:30 (27970): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:40:13 (28000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:42:56 (28029): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:46:04 (28057): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:48:48 (28087): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:51:45 (28116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:54:58 (28145): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:57:46 (28173): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:00:20 (28203): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:03:33 (28246): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:06:16 (28275): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:09:24 (28304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:11:42 (28333): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:14:55 (28363): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:17:13 (28391): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:20:16 (28424): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:23:04 (28452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:25:33 (28481): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:28:46 (28510): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:31:09 (28540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (!prev): 0x095d5ea8 *** ======= Backtrace: ========= /lib/i386-linux-gnu/libc.so.6(+0x75ee2)[0xf74efee2] /usr/lib/i386-linux-gnu/libstdc++.so.6(_ZdlPv+0x1f)[0xf76b7adf] /usr/lib/i386-linux-gnu/libstdc++.so.6(_ZdaPv+0x1b)[0xf76b7b2b] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf74934d3] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] ======= Memory map: ======== 08048000-080e3000 r-xp 00000000 08:04 19791959 /home/allan/Downloads/BOINC/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e3000-080e4000 rw-p 0009b000 08:04 19791959 /home/allan/Downloads/BOINC/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e4000-0813b000 rw-p 00000000 00:00 0 09518000-09612000 rw-p 00000000 00:00 0 [heap] f6dff000-f6e00000 rw-p 00000000 00:00 0 f6e00000-f6e21000 rw-p 00000000 00:00 0 f6e21000-f6f00000 ---p 00000000 00:00 0 f6ffc000-f7477000 rw-s 00000000 08:04 19793598 /home/allan/Downloads/BOINC/slots/0/137590 f7477000-f747a000 rw-p 00000000 00:00 0 f747a000-f761e000 r-xp 00000000 08:02 1051909 /lib/i386-linux-gnu/libc-2.15.so f761e000-f761f000 ---p 001a4000 08:02 1051909 /lib/i386-linux-gnu/libc-2.15.so f761f000-f7621000 r--p 001a4000 08:02 1051909 /lib/i386-linux-gnu/libc-2.15.so f7621000-f7622000 rw-p 001a6000 08:02 1051909 /lib/i386-linux-gnu/libc-2.15.so f7622000-f7625000 rw-p 00000000 00:00 0 f7625000-f7641000 r-xp 00000000 08:02 1051854 /lib/i386-linux-gnu/libgcc_s.so.1 f7641000-f7642000 r--p 0001b000 08:02 1051854 /lib/i386-linux-gnu/libgcc_s.so.1 f7642000-f7643000 rw-p 0001c000 08:02 1051854 /lib/i386-linux-gnu/libgcc_s.so.1 f7643000-f766d000 r-xp 00000000 08:02 1060732 /lib/i386-linux-gnu/libm-2.15.so f766d000-f766e000 r--p 00029000 08:02 1060732 /lib/i386-linux-gnu/libm-2.15.so f766e000-f766f000 rw-p 0002a000 08:02 1060732 /lib/i386-linux-gnu/libm-2.15.so f766f000-f774b000 r-xp 00000000 08:02 1064413 /usr/lib/i386-linux-gnu/libstdc++.so.6.0.17 f774b000-f774c000 ---p 000dc000 08:02 1064413 /usr/lib/i386-linux-gnu/libstdc++.so.6.0.17 f774c000-f7750000 r--p 000dc000 08:02 1064413 /usr/lib/i386-linux-gnu/libstdc++.so.6.0.17 f7750000-f7751000 rw-p 000e0000 08:02 1064413 /usr/lib/i386-linux-gnu/libstdc++.so.6.0.17 f7751000-f7759000 rw-p 00000000 00:00 0 f7759000-f775c000 r-xp 00000000 08:02 1054348 /lib/i386-linux-gnu/libdl-2.15.so f775c000-f775d000 r--p 00002000 08:02 1054348 /lib/i386-linux-gnu/libdl-2.15.so f775d000-f775e000 rw-p 00003000 08:02 1054348 /lib/i386-linux-gnu/libdl-2.15.so f775e000-f7775000 r-xp 00000000 08:02 1054349 /lib/i386-linux-gnu/libpthread-2.15.so f7775000-f7776000 r--p 00016000 08:02 1054349 /lib/i386-linux-gnu/libpthread-2.15.so f7776000-f7777000 rw-p 00017000 08:02 1054349 /lib/i386-linux-gnu/libpthread-2.15.so f7777000-f7779000 rw-p 00000000 00:00 0 f7791000-f7792000 rw-p 00000000 00:00 0 f7792000-f7793000 ---p 00000000 00:00 0 f7793000-f7796000 rw-p 00000000 00:00 0 [stack:28583] f7796000-f7798000 rw-s 00000000 08:04 19791992 /home/allan/Downloads/BOINC/slots/0/boinc_mmap_file f7798000-f779a000 rw-p 00000000 00:00 0 f779a000-f779b000 r-xp 00000000 00:00 0 [vdso] f779b000-f77bb000 r-xp 00000000 08:02 1060658 /lib/i386-linux-gnu/ld-2.15.so f77bb000-f77bc000 r--p 0001f000 08:02 1060658 /lib/i386-linux-gnu/ld-2.15.so f77bc000-f77bd000 rw-p 00020000 08:02 1060658 /lib/i386-linux-gnu/ld-2.15.so ffa78000-ffae8000 rw-p 00000000 00:00 0 [stack] SIGABRT: abort called Stack trace (17 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xf779a400] [0xf779a430] /lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xf74a81df] /lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xf74ab825] /lib/i386-linux-gnu/libc.so.6(+0x6b39a)[0xf74e539a] /lib/i386-linux-gnu/libc.so.6(+0x75ee2)[0xf74efee2] /usr/lib/i386-linux-gnu/libstdc++.so.6(_ZdlPv+0x1f)[0xf76b7adf] /usr/lib/i386-linux-gnu/libstdc++.so.6(_ZdaPv+0x1b)[0xf76b7b2b] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf74934d3] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... </stderr_txt> ]]> |
| Latest Trickles Received | ||||||
|---|---|---|---|---|---|---|
| Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
| 10 May 2014 00:41:36 | 1322662 | 16588209 | hadcm3n_8aru_1980_40_008723381_0 | 492,480 | 505,568 | 1.0266 |
| 09 May 2014 17:09:01 | 1322662 | 16588209 | hadcm3n_8aru_1980_40_008723381_0 | 466,560 | 478,750 | 1.0261 |
| 09 May 2014 10:38:42 | 1322662 | 16588209 | hadcm3n_8aru_1980_40_008723381_0 | 440,640 | 453,775 | 1.0298 |
| 08 May 2014 23:38:49 | 1322662 | 16588209 | hadcm3n_8aru_1980_40_008723381_0 | 414,720 | 427,162 | 1.0300 |
| 08 May 2014 16:00:26 | 1322662 | 16588209 | hadcm3n_8aru_1980_40_008723381_0 | 388,800 | 400,361 | 1.0297 |
| 08 May 2014 08:28:43 | 1322662 | 16588209 | hadcm3n_8aru_1980_40_008723381_0 | 362,880 | 373,594 | 1.0295 |
| 08 May 2014 00:50:04 | 1322662 | 16588209 | hadcm3n_8aru_1980_40_008723381_0 | 336,960 | 346,831 | 1.0293 |
| 07 May 2014 17:17:58 | 1322662 | 16588209 | hadcm3n_8aru_1980_40_008723381_0 | 311,040 | 320,055 | 1.0290 |
| 07 May 2014 09:38:56 | 1322662 | 16588209 | hadcm3n_8aru_1980_40_008723381_0 | 285,120 | 293,308 | 1.0287 |
| 07 May 2014 02:04:55 | 1322662 | 16588209 | hadcm3n_8aru_1980_40_008723381_0 | 259,200 | 266,554 | 1.0284 |
| 06 May 2014 18:25:31 | 1322662 | 16588209 | hadcm3n_8aru_1980_40_008723381_0 | 233,280 | 239,796 | 1.0279 |
| 06 May 2014 10:53:57 | 1322662 | 16588209 | hadcm3n_8aru_1980_40_008723381_0 | 207,360 | 213,172 | 1.0280 |
| 06 May 2014 03:20:15 | 1322662 | 16588209 | hadcm3n_8aru_1980_40_008723381_0 | 181,440 | 186,457 | 1.0277 |
| 05 May 2014 19:48:27 | 1322662 | 16588209 | hadcm3n_8aru_1980_40_008723381_0 | 155,520 | 159,827 | 1.0277 |
| 05 May 2014 12:09:26 | 1322662 | 16588209 | hadcm3n_8aru_1980_40_008723381_0 | 129,600 | 133,136 | 1.0273 |
| 05 May 2014 04:32:58 | 1322662 | 16588209 | hadcm3n_8aru_1980_40_008723381_0 | 103,680 | 106,508 | 1.0273 |
| 04 May 2014 20:56:16 | 1322662 | 16588209 | hadcm3n_8aru_1980_40_008723381_0 | 77,760 | 79,852 | 1.0269 |
| 04 May 2014 14:02:30 | 1322662 | 16588209 | hadcm3n_8aru_1980_40_008723381_0 | 51,840 | 53,288 | 1.0279 |
| 04 May 2014 05:45:03 | 1322662 | 16588209 | hadcm3n_8aru_1980_40_008723381_0 | 25,920 | 26,689 | 1.0297 |
©2024 climateprediction.net