 |
|
| Name | hadcm3n_yftf_1900_40_007523474_2 |
| Workunit | 7720949 |
| Created | 28 Oct 2011, 14:25:55 UTC |
| Sent | 28 Oct 2011, 14:29:13 UTC |
| Report deadline | 27 Jan 2012, 21:56:24 UTC |
| Received | 25 Nov 2011, 20:46:46 UTC |
| Server state | Over |
| Outcome | Computation error |
| Client state | Compute error |
| Exit status | 193 (0x000000C1) EXIT_SIGNAL |
| Computer ID | 1137791 |
| Run time | 21 days 2 hours 42 min 7 sec |
| CPU time | 20 days 10 hours 26 min 30 sec |
| Validate state | Invalid |
| Credit | 9,331.20 |
| Device peak FLOPS | 2.10 GFLOPS |
| Application version | UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu |
| Stderr | <core_client_version>6.12.33</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 14:42:31 (14905): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:44:03 (14928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:45:29 (14950): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:46:55 (14973): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:48:16 (14997): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:49:52 (15019): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:51:28 (15042): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:52:54 (15063): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:54:15 (15087): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:55:57 (15110): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:57:32 (15132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:58:58 (15154): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:09:01 (3875): No heartbeat from core client for 30 sec - exiting 21:09:04 (3875): No heartbeat from core client for 30 sec - exiting 21:09:05 (3875): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:09:08 (3875): No heartbeat from core client for 30 sec - exiting 21:09:09 (3875): No heartbeat from core client for 30 sec - exiting 21:09:10 (3875): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:01:38 (32318): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:07:29 (3494): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:08:51 (3520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:10:20 (3551): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:12:02 (3612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:13:33 (3648): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:14:59 (3678): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:16:35 (3698): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:17:51 (3723): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:19:29 (3756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:21:09 (3783): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:22:30 (3805): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:23:56 (3829): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:25:27 (3849): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:26:43 (3869): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:28:05 (3888): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:29:30 (3908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:30:46 (3928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:32:23 (3948): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:33:54 (3970): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:35:25 (3992): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:36:40 (4012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:38:16 (4032): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:39:33 (4054): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:41:14 (4081): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:42:39 (4100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:44:00 (4120): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:45:37 (4147): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:47:03 (4166): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:48:29 (4186): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:50:10 (4205): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:51:51 (4227): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:53:17 (4247): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:54:34 (4267): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:56:05 (4287): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:56:45 (4306): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:59:46 (4325): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:01:03 (4350): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:02:19 (4370): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:03:40 (4389): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:05:17 (4411): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:06:37 (4434): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:08:08 (4454): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:09:29 (4475): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:10:51 (4503): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:12:16 (4524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:13:53 (4544): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:15:33 (4563): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:16:59 (4582): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:18:30 (4607): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:19:42 (4627): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... *** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09ac7538 *** ======= Backtrace: ========= /lib32/libc.so.6(+0x6eb72)[0xf7539b72] /lib32/libc.so.6(+0x6f812)[0xf753a812] /lib32/libc.so.6(cfree+0x6d)[0xf753d8cd] /usr/lib32/libstdc++.so.6(_ZdlPv+0x1f)[0xf773abaf] /usr/lib32/libstdc++.so.6(_ZdaPv+0x1b)[0xf773ac0b] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib32/libc.so.6(__libc_start_main+0xf3)[0xf74e40f3] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] ======= Memory map: ======== 08048000-080e3000 r-xp 00000000 08:11 656690 /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e3000-080e4000 rw-p 0009b000 08:11 656690 /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e4000-0813b000 rw-p 00000000 00:00 0 09a72000-09ad7000 rw-p 00000000 00:00 0 [heap] f6f00000-f6f21000 rw-p 00000000 00:00 0 f6f21000-f7000000 ---p 00000000 00:00 0 f704d000-f74c8000 rw-s 00000000 08:11 922948 /var/lib/boinc-client/slots/2/139165 f74c8000-f74cb000 rw-p 00000000 00:00 0 f74cb000-f763e000 r-xp 00000000 08:11 1707725 /lib32/libc-2.13.so f763e000-f763f000 ---p 00173000 08:11 1707725 /lib32/libc-2.13.so f763f000-f7641000 r--p 00173000 08:11 1707725 /lib32/libc-2.13.so f7641000-f7642000 rw-p 00175000 08:11 1707725 /lib32/libc-2.13.so f7642000-f7645000 rw-p 00000000 00:00 0 f7645000-f7661000 r-xp 00000000 08:11 405814 /usr/lib32/libgcc_s.so.1 f7661000-f7662000 r--p 0001b000 08:11 405814 /usr/lib32/libgcc_s.so.1 f7662000-f7663000 rw-p 0001c000 08:11 405814 /usr/lib32/libgcc_s.so.1 f7663000-f768b000 r-xp 00000000 08:11 1707740 /lib32/libm-2.13.so f768b000-f768c000 r--p 00028000 08:11 1707740 /lib32/libm-2.13.so f768c000-f768d000 rw-p 00029000 08:11 1707740 /lib32/libm-2.13.so f768d000-f776b000 r-xp 00000000 08:11 405030 /usr/lib32/libstdc++.so.6.0.16 f776b000-f776c000 ---p 000de000 08:11 405030 /usr/lib32/libstdc++.so.6.0.16 f776c000-f7770000 r--p 000de000 08:11 405030 /usr/lib32/libstdc++.so.6.0.16 f7770000-f7771000 rw-p 000e2000 08:11 405030 /usr/lib32/libstdc++.so.6.0.16 f7771000-f7779000 rw-p 00000000 00:00 0 f7779000-f777c000 r-xp 00000000 08:11 1707727 /lib32/libdl-2.13.so f777c000-f777d000 r--p 00002000 08:11 1707727 /lib32/libdl-2.13.so f777d000-f777e000 rw-p 00003000 08:11 1707727 /lib32/libdl-2.13.so f777e000-f7795000 r-xp 00000000 08:11 1707737 /lib32/libpthread-2.13.so f7795000-f7796000 r--p 00016000 08:11 1707737 /lib32/libpthread-2.13.so f7796000-f7797000 rw-p 00017000 08:11 1707737 /lib32/libpthread-2.13.so f7797000-f7799000 rw-p 00000000 00:00 0 f77b7000-f77b8000 rw-p 00000000 00:00 0 f77b8000-f77b9000 ---p 00000000 00:00 0 f77b9000-f77bc000 rw-p 00000000 00:00 0 f77bc000-f77be000 rw-s 00000000 08:11 922945 /var/lib/boinc-client/slots/2/boinc_mmap_file f77be000-f77c0000 rw-p 00000000 00:00 0 f77c0000-f77c1000 r-xp 00000000 00:00 0 [vdso] f77c1000-f77df000 r-xp 00000000 08:11 3538992 /lib/i386-linux-gnu/ld-2.13.so f77df000-f77e0000 r--p 0001d000 08:11 3538992 /lib/i386-linux-gnu/ld-2.13.so f77e0000-f77e1000 rw-p 0001e000 08:11 3538992 /lib/i386-linux-gnu/ld-2.13.so ffbc3000-ffc33000 rw-p 00000000 00:00 0 [stack] SIGABRT: abort called Stack trace (19 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xf77c0400] [0xf77c0425] /lib32/libc.so.6(gsignal+0x4f)[0xf74f8c4f] /lib32/libc.so.6(abort+0x175)[0xf74fc175] /lib32/libc.so.6(+0x63e8c)[0xf752ee8c] /lib32/libc.so.6(+0x6eb72)[0xf7539b72] /lib32/libc.so.6(+0x6f812)[0xf753a812] /lib32/libc.so.6(cfree+0x6d)[0xf753d8cd] /usr/lib32/libstdc++.so.6(_ZdlPv+0x1f)[0xf773abaf] /usr/lib32/libstdc++.so.6(_ZdaPv+0x1b)[0xf773ac0b] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib32/libc.so.6(__libc_start_main+0xf3)[0xf74e40f3] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... </stderr_txt> ]]> |
| Latest Trickles Received | ||||||
|---|---|---|---|---|---|---|
| Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
| 25 Nov 2011 19:15:01 | 1137791 | 13561786 | hadcm3n_yftf_1900_40_007523474_2 | 777,600 | 1,765,584 | 2.2706 |
| 24 Nov 2011 13:22:54 | 1137791 | 13561786 | hadcm3n_yftf_1900_40_007523474_2 | 751,680 | 1,707,724 | 2.2719 |
| 23 Nov 2011 16:04:01 | 1137791 | 13561786 | hadcm3n_yftf_1900_40_007523474_2 | 725,760 | 1,650,051 | 2.2735 |
| 22 Nov 2011 18:01:16 | 1137791 | 13561786 | hadcm3n_yftf_1900_40_007523474_2 | 699,840 | 1,592,484 | 2.2755 |
| 21 Nov 2011 18:01:01 | 1137791 | 13561786 | hadcm3n_yftf_1900_40_007523474_2 | 673,920 | 1,534,756 | 2.2774 |
| 20 Nov 2011 16:09:29 | 1137791 | 13561786 | hadcm3n_yftf_1900_40_007523474_2 | 648,000 | 1,476,993 | 2.2793 |
| 19 Nov 2011 18:47:51 | 1137791 | 13561786 | hadcm3n_yftf_1900_40_007523474_2 | 622,080 | 1,419,353 | 2.2816 |
| 18 Nov 2011 19:39:15 | 1137791 | 13561786 | hadcm3n_yftf_1900_40_007523474_2 | 596,160 | 1,361,592 | 2.2839 |
| 17 Nov 2011 21:50:56 | 1137791 | 13561786 | hadcm3n_yftf_1900_40_007523474_2 | 570,240 | 1,303,975 | 2.2867 |
| 17 Nov 2011 04:27:14 | 1137791 | 13561786 | hadcm3n_yftf_1900_40_007523474_2 | 544,320 | 1,246,254 | 2.2896 |
| 16 Nov 2011 06:43:48 | 1137791 | 13561786 | hadcm3n_yftf_1900_40_007523474_2 | 518,400 | 1,187,380 | 2.2905 |
| 15 Nov 2011 16:48:41 | 1137791 | 13561786 | hadcm3n_yftf_1900_40_007523474_2 | 492,480 | 1,128,799 | 2.2921 |
| 15 Nov 2011 16:48:40 | 1137791 | 13561786 | hadcm3n_yftf_1900_40_007523474_2 | 466,560 | 1,070,431 | 2.2943 |
| 15 Nov 2011 16:48:38 | 1137791 | 13561786 | hadcm3n_yftf_1900_40_007523474_2 | 440,640 | 1,011,986 | 2.2966 |
| 15 Nov 2011 16:48:39 | 1137791 | 13561786 | hadcm3n_yftf_1900_40_007523474_2 | 414,720 | 953,458 | 2.2990 |
| 15 Nov 2011 16:48:39 | 1137791 | 13561786 | hadcm3n_yftf_1900_40_007523474_2 | 388,800 | 894,928 | 2.3018 |
| 15 Nov 2011 16:48:38 | 1137791 | 13561786 | hadcm3n_yftf_1900_40_007523474_2 | 362,880 | 837,104 | 2.3068 |
| 10 Nov 2011 04:09:18 | 1137791 | 13561786 | hadcm3n_yftf_1900_40_007523474_2 | 336,960 | 779,196 | 2.3124 |
| 08 Nov 2011 20:44:38 | 1137791 | 13561786 | hadcm3n_yftf_1900_40_007523474_2 | 311,040 | 721,303 | 2.3190 |
| 07 Nov 2011 16:36:16 | 1137791 | 13561786 | hadcm3n_yftf_1900_40_007523474_2 | 285,120 | 663,591 | 2.3274 |
©2024 climateprediction.net