 |
|
| Name | hadcm3n_yg0d_1900_40_007353607_0 |
| Workunit | 7551037 |
| Created | 6 Jul 2011, 14:28:05 UTC |
| Sent | 15 Jul 2011, 16:19:11 UTC |
| Report deadline | 14 Oct 2011, 23:46:22 UTC |
| Received | 24 Aug 2011, 23:06:28 UTC |
| Server state | Over |
| Outcome | Computation error |
| Client state | Compute error |
| Exit status | 193 (0x000000C1) EXIT_SIGNAL |
| Computer ID | 1137030 |
| Run time | 27 days 13 hours 58 min 1 sec |
| CPU time | 20 days 18 hours 26 min 7 sec |
| Validate state | Invalid |
| Credit | 9,331.20 |
| Device peak FLOPS | 2.18 GFLOPS |
| Application version | UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu |
| Stderr | <core_client_version>6.10.59</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:40:41 (3095): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:06:36 (4520): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:58:33 (5149): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:28:52 (7389): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:52:08 (9420): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:33:26 (12243): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 04:09:08 (13609): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 22:40:51 (1428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:25:52 (6819): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:52:48 (16018): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:08:06 (1552): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:22:18 (3396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:34:19 (5961): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:50:52 (6320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:29:55 (1575): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:55:52 (2963): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:08:25 (3618): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:03:17 (4238): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:21:17 (7825): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:49:31 (13572): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:04:17 (14307): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:46:58 (2845): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:56:36 (4503): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 04:24:41 (7692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:24:43 (7692): No heartbeat from core client for 30 sec - exiting 04:24:44 (7692): No heartbeat from core client for 30 sec - exiting 04:24:45 (7692): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 05:46:41 (9251): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:33:36 (19984): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:33:55 (1459): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09525640 *** ======= Backtrace: ========= /lib32/libc.so.6(+0x6b511)[0xf74cb511] /lib32/libc.so.6(+0x6ce1b)[0xf74cce1b] /lib32/libc.so.6(cfree+0x6d)[0xf74cff8d] /usr/lib32/libstdc++.so.6(_ZdlPv+0x21)[0xf76a9841] /usr/lib32/libstdc++.so.6(_ZdaPv+0x1d)[0xf76a989d] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib32/libc.so.6(__libc_start_main+0xe7)[0xf7476e37] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] ======= Memory map: ======== 08048000-080e3000 r-xp 00000000 08:01 2228239 /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e3000-080e4000 rw-p 0009b000 08:01 2228239 /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e4000-0813b000 rw-p 00000000 00:00 0 094d0000-09535000 rw-p 00000000 00:00 0 [heap] f6e00000-f6e21000 rw-p 00000000 00:00 0 f6e21000-f6f00000 ---p 00000000 00:00 0 f6fe2000-f745d000 rw-s 00000000 08:01 2752633 /var/lib/boinc-client/slots/9/135560 f745d000-f7460000 rw-p 00000000 00:00 0 f7460000-f75b6000 r-xp 00000000 08:01 7340183 /lib32/libc-2.13.so f75b6000-f75b7000 ---p 00156000 08:01 7340183 /lib32/libc-2.13.so f75b7000-f75b9000 r--p 00156000 08:01 7340183 /lib32/libc-2.13.so f75b9000-f75ba000 rw-p 00158000 08:01 7340183 /lib32/libc-2.13.so f75ba000-f75bd000 rw-p 00000000 00:00 0 f75bd000-f75d7000 r-xp 00000000 08:01 1966816 /usr/lib32/libgcc_s.so.1 f75d7000-f75d8000 r--p 00019000 08:01 1966816 /usr/lib32/libgcc_s.so.1 f75d8000-f75d9000 rw-p 0001a000 08:01 1966816 /usr/lib32/libgcc_s.so.1 f75d9000-f75fd000 r-xp 00000000 08:01 7340187 /lib32/libm-2.13.so f75fd000-f75fe000 r--p 00023000 08:01 7340187 /lib32/libm-2.13.so f75fe000-f75ff000 rw-p 00024000 08:01 7340187 /lib32/libm-2.13.so f75ff000-f76de000 r-xp 00000000 08:01 1966813 /usr/lib32/libstdc++.so.6.0.14 f76de000-f76e2000 r--p 000de000 08:01 1966813 /usr/lib32/libstdc++.so.6.0.14 f76e2000-f76e3000 rw-p 000e2000 08:01 1966813 /usr/lib32/libstdc++.so.6.0.14 f76e3000-f76ea000 rw-p 00000000 00:00 0 f76ea000-f76ec000 r-xp 00000000 08:01 7340186 /lib32/libdl-2.13.so f76ec000-f76ed000 r--p 00001000 08:01 7340186 /lib32/libdl-2.13.so f76ed000-f76ee000 rw-p 00002000 08:01 7340186 /lib32/libdl-2.13.so f76ee000-f7703000 r-xp 00000000 08:01 7340197 /lib32/libpthread-2.13.so f7703000-f7704000 r--p 00014000 08:01 7340197 /lib32/libpthread-2.13.so f7704000-f7705000 rw-p 00015000 08:01 7340197 /lib32/libpthread-2.13.so f7705000-f7708000 rw-p 00000000 00:00 0 f7723000-f7724000 rw-p 00000000 00:00 0 f7724000-f7725000 ---p 00000000 00:00 0 f7725000-f7728000 rw-p 00000000 00:00 0 f7728000-f772a000 rw-s 00000000 08:01 2752542 /var/lib/boinc-client/slots/9/boinc_mmap_file f772a000-f772b000 rw-p 00000000 00:00 0 f772b000-f772c000 r-xp 00000000 00:00 0 [vdso] f772c000-f7748000 r-xp 00000000 08:01 7340180 /lib32/ld-2.13.so f7748000-f7749000 r--p 0001b000 08:01 7340180 /lib32/ld-2.13.so f7749000-f774a000 rw-p 0001c000 08:01 7340180 /lib32/ld-2.13.so ffbc3000-ffc33000 rw-p 00000000 00:00 0 [stack] SIGABRT: abort called Stack trace (19 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xf772b400] [0xf772b430] /lib32/libc.so.6(gsignal+0x51)[0xf748aea1] /lib32/libc.so.6(abort+0x17e)[0xf748e2ce] /lib32/libc.so.6(+0x61277)[0xf74c1277] /lib32/libc.so.6(+0x6b511)[0xf74cb511] /lib32/libc.so.6(+0x6ce1b)[0xf74cce1b] /lib32/libc.so.6(cfree+0x6d)[0xf74cff8d] /usr/lib32/libstdc++.so.6(_ZdlPv+0x21)[0xf76a9841] /usr/lib32/libstdc++.so.6(_ZdaPv+0x1d)[0xf76a989d] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib32/libc.so.6(__libc_start_main+0xe7)[0xf7476e37] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... </stderr_txt> ]]> |
| Latest Trickles Received | ||||||
|---|---|---|---|---|---|---|
| Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
| 24 Aug 2011 12:00:00 | 1137030 | 13111084 | hadcm3n_yg0d_1900_40_007353607_0 | 777,600 | 1,794,360 | 2.3076 |
| 23 Aug 2011 16:25:43 | 1137030 | 13111084 | hadcm3n_yg0d_1900_40_007353607_0 | 751,680 | 1,736,090 | 2.3096 |
| 22 Aug 2011 21:08:07 | 1137030 | 13111084 | hadcm3n_yg0d_1900_40_007353607_0 | 725,760 | 1,678,135 | 2.3122 |
| 21 Aug 2011 12:08:00 | 1137030 | 13111084 | hadcm3n_yg0d_1900_40_007353607_0 | 699,840 | 1,619,738 | 2.3144 |
| 20 Aug 2011 12:49:09 | 1137030 | 13111084 | hadcm3n_yg0d_1900_40_007353607_0 | 673,920 | 1,560,580 | 2.3157 |
| 19 Aug 2011 16:35:42 | 1137030 | 13111084 | hadcm3n_yg0d_1900_40_007353607_0 | 648,000 | 1,502,263 | 2.3183 |
| 18 Aug 2011 20:08:16 | 1137030 | 13111084 | hadcm3n_yg0d_1900_40_007353607_0 | 622,080 | 1,444,419 | 2.3219 |
| 17 Aug 2011 14:54:10 | 1137030 | 13111084 | hadcm3n_yg0d_1900_40_007353607_0 | 596,160 | 1,386,412 | 2.3256 |
| 16 Aug 2011 06:20:20 | 1137030 | 13111084 | hadcm3n_yg0d_1900_40_007353607_0 | 570,240 | 1,326,960 | 2.3270 |
| 13 Aug 2011 20:23:10 | 1137030 | 13111084 | hadcm3n_yg0d_1900_40_007353607_0 | 544,320 | 1,265,032 | 2.3241 |
| 11 Aug 2011 13:03:11 | 1137030 | 13111084 | hadcm3n_yg0d_1900_40_007353607_0 | 518,400 | 1,205,878 | 2.3262 |
| 09 Aug 2011 15:40:56 | 1137030 | 13111084 | hadcm3n_yg0d_1900_40_007353607_0 | 492,480 | 1,147,327 | 2.3297 |
| 06 Aug 2011 15:12:21 | 1137030 | 13111084 | hadcm3n_yg0d_1900_40_007353607_0 | 466,560 | 1,088,750 | 2.3336 |
| 05 Aug 2011 00:45:56 | 1137030 | 13111084 | hadcm3n_yg0d_1900_40_007353607_0 | 440,640 | 1,030,289 | 2.3382 |
| 03 Aug 2011 03:56:30 | 1137030 | 13111084 | hadcm3n_yg0d_1900_40_007353607_0 | 414,720 | 969,289 | 2.3372 |
| 02 Aug 2011 07:14:21 | 1137030 | 13111084 | hadcm3n_yg0d_1900_40_007353607_0 | 388,800 | 909,459 | 2.3391 |
| 01 Aug 2011 08:57:49 | 1137030 | 13111084 | hadcm3n_yg0d_1900_40_007353607_0 | 362,880 | 848,882 | 2.3393 |
| 31 Jul 2011 09:46:00 | 1137030 | 13111084 | hadcm3n_yg0d_1900_40_007353607_0 | 336,960 | 787,765 | 2.3379 |
| 30 Jul 2011 10:12:35 | 1137030 | 13111084 | hadcm3n_yg0d_1900_40_007353607_0 | 311,040 | 725,310 | 2.3319 |
| 29 Jul 2011 12:05:38 | 1137030 | 13111084 | hadcm3n_yg0d_1900_40_007353607_0 | 285,120 | 664,244 | 2.3297 |
©2024 climateprediction.net