 |
|
| Name | hadcm3n_8arw_1980_40_008723383_0 |
| Workunit | 8869361 |
| Created | 23 Apr 2014, 12:59:43 UTC |
| Sent | 3 May 2014, 21:06:49 UTC |
| Report deadline | 3 Aug 2014, 4:34:00 UTC |
| Received | 10 May 2014, 8:51:53 UTC |
| Server state | Over |
| Outcome | Computation error |
| Client state | Compute error |
| Exit status | 193 (0x000000C1) EXIT_SIGNAL |
| Computer ID | 1322662 |
| Run time | 6 days 7 hours 19 min 28 sec |
| CPU time | 6 days 3 hours 45 min 12 sec |
| Validate state | Invalid |
| Credit | 5,909.76 |
| Device peak FLOPS | 3.84 GFLOPS |
| Application version | UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu |
| Stderr | <core_client_version>7.2.42</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> 23:20:29 (12172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:56:50 (13874): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:11:36 (14243): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:14:01 (25696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:16:39 (25725): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:19:37 (25757): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:22:40 (25787): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:25:28 (25815): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 16:27:51 (25844): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:30:59 (25873): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:34:02 (25917): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:36:35 (25947): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:39:44 (25976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:42:51 (26005): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:45:10 (26033): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:47:23 (26063): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:50:25 (26092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:52:54 (26123): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:55:42 (26152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:58:36 (26181): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:01:08 (26211): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:04:11 (26254): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:07:25 (26282): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:10:07 (26312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:13:20 (26342): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:16:33 (26370): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:19:47 (26406): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:22:10 (26436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:24:58 (26464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:27:51 (26493): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:30:44 (26526): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:33:02 (26569): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:35:20 (26597): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:38:04 (26626): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:40:27 (26656): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:43:04 (26684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:45:27 (26713): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:48:36 (26741): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:51:49 (26772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:54:17 (26801): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:57:21 (26837): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:00:23 (26867): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:03:11 (26909): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:06:09 (26939): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:09:02 (26967): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:11:56 (26997): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:14:53 (27026): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:17:32 (27055): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:20:41 (27088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:23:08 (27116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:25:26 (27145): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:27:54 (27173): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:30:12 (27204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:32:30 (27246): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:35:24 (27275): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:38:37 (27303): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:41:50 (27333): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:44:38 (27363): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:47:01 (27391): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:49:44 (27420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:52:07 (27449): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:55:05 (27477): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:57:53 (27508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:00:22 (27537): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:02:44 (27580): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:05:38 (27608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:08:31 (27637): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:11:05 (27667): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:13:52 (27697): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:16:45 (27726): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:19:33 (27758): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:22:05 (27788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:24:44 (27816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:27:02 (27845): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:29:20 (27874): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:31:39 (27902): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:34:32 (27945): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:37:30 (27974): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:40:13 (28004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:42:56 (28033): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:46:04 (28061): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:48:48 (28091): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:51:45 (28120): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:54:58 (28149): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:57:47 (28177): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:00:20 (28207): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:03:32 (28250): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:06:16 (28279): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:09:24 (28308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:11:42 (28337): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:14:55 (28367): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:17:14 (28395): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:20:16 (28428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:23:04 (28456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:25:32 (28485): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:28:46 (28514): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:31:09 (28544): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (!prev): 0x08f45ec0 *** ======= Backtrace: ========= /lib/i386-linux-gnu/libc.so.6(+0x75ee2)[0xf752bee2] /usr/lib/i386-linux-gnu/libstdc++.so.6(_ZdlPv+0x1f)[0xf76f3adf] /usr/lib/i386-linux-gnu/libstdc++.so.6(_ZdaPv+0x1b)[0xf76f3b2b] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf74cf4d3] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] ======= Memory map: ======== 08048000-080e3000 r-xp 00000000 08:04 19791959 /home/allan/Downloads/BOINC/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e3000-080e4000 rw-p 0009b000 08:04 19791959 /home/allan/Downloads/BOINC/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e4000-0813b000 rw-p 00000000 00:00 0 08e88000-08f82000 rw-p 00000000 00:00 0 [heap] f6f00000-f6f21000 rw-p 00000000 00:00 0 f6f21000-f7000000 ---p 00000000 00:00 0 f7038000-f74b3000 rw-s 00000000 08:04 19794178 /home/allan/Downloads/BOINC/slots/1/137990 f74b3000-f74b6000 rw-p 00000000 00:00 0 f74b6000-f765a000 r-xp 00000000 08:02 1051909 /lib/i386-linux-gnu/libc-2.15.so f765a000-f765b000 ---p 001a4000 08:02 1051909 /lib/i386-linux-gnu/libc-2.15.so f765b000-f765d000 r--p 001a4000 08:02 1051909 /lib/i386-linux-gnu/libc-2.15.so f765d000-f765e000 rw-p 001a6000 08:02 1051909 /lib/i386-linux-gnu/libc-2.15.so f765e000-f7661000 rw-p 00000000 00:00 0 f7661000-f767d000 r-xp 00000000 08:02 1051854 /lib/i386-linux-gnu/libgcc_s.so.1 f767d000-f767e000 r--p 0001b000 08:02 1051854 /lib/i386-linux-gnu/libgcc_s.so.1 f767e000-f767f000 rw-p 0001c000 08:02 1051854 /lib/i386-linux-gnu/libgcc_s.so.1 f767f000-f76a9000 r-xp 00000000 08:02 1060732 /lib/i386-linux-gnu/libm-2.15.so f76a9000-f76aa000 r--p 00029000 08:02 1060732 /lib/i386-linux-gnu/libm-2.15.so f76aa000-f76ab000 rw-p 0002a000 08:02 1060732 /lib/i386-linux-gnu/libm-2.15.so f76ab000-f7787000 r-xp 00000000 08:02 1064413 /usr/lib/i386-linux-gnu/libstdc++.so.6.0.17 f7787000-f7788000 ---p 000dc000 08:02 1064413 /usr/lib/i386-linux-gnu/libstdc++.so.6.0.17 f7788000-f778c000 r--p 000dc000 08:02 1064413 /usr/lib/i386-linux-gnu/libstdc++.so.6.0.17 f778c000-f778d000 rw-p 000e0000 08:02 1064413 /usr/lib/i386-linux-gnu/libstdc++.so.6.0.17 f778d000-f7795000 rw-p 00000000 00:00 0 f7795000-f7798000 r-xp 00000000 08:02 1054348 /lib/i386-linux-gnu/libdl-2.15.so f7798000-f7799000 r--p 00002000 08:02 1054348 /lib/i386-linux-gnu/libdl-2.15.so f7799000-f779a000 rw-p 00003000 08:02 1054348 /lib/i386-linux-gnu/libdl-2.15.so f779a000-f77b1000 r-xp 00000000 08:02 1054349 /lib/i386-linux-gnu/libpthread-2.15.so f77b1000-f77b2000 r--p 00016000 08:02 1054349 /lib/i386-linux-gnu/libpthread-2.15.so f77b2000-f77b3000 rw-p 00017000 08:02 1054349 /lib/i386-linux-gnu/libpthread-2.15.so f77b3000-f77b5000 rw-p 00000000 00:00 0 f77cc000-f77ce000 rw-p 00000000 00:00 0 f77ce000-f77cf000 ---p 00000000 00:00 0 f77cf000-f77d2000 rw-p 00000000 00:00 0 [stack:28587] f77d2000-f77d4000 rw-s 00000000 08:04 19794035 /home/allan/Downloads/BOINC/slots/1/boinc_mmap_file f77d4000-f77d6000 rw-p 00000000 00:00 0 f77d6000-f77d7000 r-xp 00000000 00:00 0 [vdso] f77d7000-f77f7000 r-xp 00000000 08:02 1060658 /lib/i386-linux-gnu/ld-2.15.so f77f7000-f77f8000 r--p 0001f000 08:02 1060658 /lib/i386-linux-gnu/ld-2.15.so f77f8000-f77f9000 rw-p 00020000 08:02 1060658 /lib/i386-linux-gnu/ld-2.15.so ffbb8000-ffc28000 rw-p 00000000 00:00 0 [stack] SIGABRT: abort called Stack trace (17 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xf77d6400] [0xf77d6430] /lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xf74e41df] /lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xf74e7825] /lib/i386-linux-gnu/libc.so.6(+0x6b39a)[0xf752139a] /lib/i386-linux-gnu/libc.so.6(+0x75ee2)[0xf752bee2] /usr/lib/i386-linux-gnu/libstdc++.so.6(_ZdlPv+0x1f)[0xf76f3adf] /usr/lib/i386-linux-gnu/libstdc++.so.6(_ZdaPv+0x1b)[0xf76f3b2b] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xf74cf4d3] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... </stderr_txt> ]]> |
| Latest Trickles Received | ||||||
|---|---|---|---|---|---|---|
| Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
| 10 May 2014 00:41:36 | 1322662 | 16588211 | hadcm3n_8arw_1980_40_008723383_0 | 492,480 | 504,975 | 1.0254 |
| 09 May 2014 17:09:01 | 1322662 | 16588211 | hadcm3n_8arw_1980_40_008723383_0 | 466,560 | 478,201 | 1.0250 |
| 09 May 2014 10:38:42 | 1322662 | 16588211 | hadcm3n_8arw_1980_40_008723383_0 | 440,640 | 453,341 | 1.0288 |
| 08 May 2014 23:38:49 | 1322662 | 16588211 | hadcm3n_8arw_1980_40_008723383_0 | 414,720 | 426,684 | 1.0288 |
| 08 May 2014 16:00:26 | 1322662 | 16588211 | hadcm3n_8arw_1980_40_008723383_0 | 388,800 | 399,933 | 1.0286 |
| 08 May 2014 08:28:43 | 1322662 | 16588211 | hadcm3n_8arw_1980_40_008723383_0 | 362,880 | 373,204 | 1.0285 |
| 08 May 2014 00:50:04 | 1322662 | 16588211 | hadcm3n_8arw_1980_40_008723383_0 | 336,960 | 346,461 | 1.0282 |
| 07 May 2014 17:17:58 | 1322662 | 16588211 | hadcm3n_8arw_1980_40_008723383_0 | 311,040 | 319,718 | 1.0279 |
| 07 May 2014 09:38:56 | 1322662 | 16588211 | hadcm3n_8arw_1980_40_008723383_0 | 285,120 | 292,991 | 1.0276 |
| 07 May 2014 02:04:55 | 1322662 | 16588211 | hadcm3n_8arw_1980_40_008723383_0 | 259,200 | 266,266 | 1.0273 |
| 06 May 2014 18:25:31 | 1322662 | 16588211 | hadcm3n_8arw_1980_40_008723383_0 | 233,280 | 239,538 | 1.0268 |
| 06 May 2014 10:53:57 | 1322662 | 16588211 | hadcm3n_8arw_1980_40_008723383_0 | 207,360 | 212,951 | 1.0270 |
| 06 May 2014 02:20:00 | 1322662 | 16588211 | hadcm3n_8arw_1980_40_008723383_0 | 181,440 | 186,263 | 1.0266 |
| 05 May 2014 18:43:09 | 1322662 | 16588211 | hadcm3n_8arw_1980_40_008723383_0 | 155,520 | 159,673 | 1.0267 |
| 05 May 2014 11:04:14 | 1322662 | 16588211 | hadcm3n_8arw_1980_40_008723383_0 | 129,600 | 133,012 | 1.0263 |
| 05 May 2014 03:32:47 | 1322662 | 16588211 | hadcm3n_8arw_1980_40_008723383_0 | 103,680 | 106,401 | 1.0262 |
| 04 May 2014 19:56:00 | 1322662 | 16588211 | hadcm3n_8arw_1980_40_008723383_0 | 77,760 | 79,779 | 1.0260 |
| 04 May 2014 14:02:30 | 1322662 | 16588211 | hadcm3n_8arw_1980_40_008723383_0 | 51,840 | 53,270 | 1.0276 |
| 04 May 2014 04:44:51 | 1322662 | 16588211 | hadcm3n_8arw_1980_40_008723383_0 | 25,920 | 26,661 | 1.0286 |
©2024 climateprediction.net