Task 15614153

Name	hadcm3n_zcor_1880_40_008245864_4
Workunit	8400988
Created	20 Feb 2013, 5:06:01 UTC
Sent	20 Feb 2013, 5:06:02 UTC
Report deadline	22 May 2013, 12:33:13 UTC
Received	11 Apr 2013, 2:59:14 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1180505
Run time	13 days 18 hours 47 min 20 sec
CPU time	11 days 11 hours 35 min 37 sec
Validate state	Invalid
Credit	8,398.08
Device peak FLOPS	2.92 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.0.31</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:52:38 (352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(15638,0xaca0f2c0) malloc: * error for object 0x10b7404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(15638,0xaca0f2c0) malloc: * error for object 0x10b7400: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(15638,0xaca0f2c0) malloc: * error for object 0x10b7404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(15638,0xaca0f2c0) malloc: * error for object 0x10b7400: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(15638,0xaca0f2c0) malloc: * error for object 0x10b7404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(15638,0xaca0f2c0) malloc: * error for object 0x10b7400: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(15638,0xaca0f2c0) malloc: * error for object 0x10b7404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(15638,0xaca0f2c0) malloc: * error for object 0x10b7400: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(15638,0xaca0f2c0) malloc: * error for object 0x10b7404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(15638,0xaca0f2c0) malloc: * error for object 0x10b7400: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(15638,0xaca0f2c0) malloc: * error for object 0x10b7404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(15638,0xaca0f2c0) malloc: * error for object 0x10b7400: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 01:44:06 (2723): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:17:08 (58549): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:18:00 (59024): No heartbeat from core client for 30 sec - exiting 20:18:01 (59024): No heartbeat from core client for 30 sec - exiting 20:18:02 (59024): No heartbeat from core client for 30 sec - exiting 20:18:03 (59024): No heartbeat from core client for 30 sec - exiting 20:18:04 (59024): No heartbeat from core client for 30 sec - exiting 20:18:05 (59024): No heartbeat from core client for 30 sec - exiting 20:18:06 (59024): No heartbeat from core client for 30 sec - exiting 20:18:07 (59024): No heartbeat from core client for 30 sec - exiting 20:18:08 (59024): No heartbeat from core client for 30 sec - exiting 20:18:09 (59024): No heartbeat from core client for 30 sec - exiting 20:18:10 (59024): No heartbeat from core client for 30 sec - exiting 20:18:11 (59024): No heartbeat from core client for 30 sec - exiting 20:18:12 (59024): No heartbeat from core client for 30 sec - exiting 20:18:13 (59024): No heartbeat from core client for 30 sec - exiting 20:18:14 (59024): No heartbeat from core client for 30 sec - exiting 20:18:15 (59024): No heartbeat from core client for 30 sec - exiting 20:18:16 (59024): No heartbeat from core client for 30 sec - exiting 20:18:17 (59024): No heartbeat from core client for 30 sec - exiting 20:18:18 (59024): No heartbeat from core client for 30 sec - exiting 20:18:19 (59024): No heartbeat from core client for 30 sec - exiting 20:18:20 (59024): No heartbeat from core client for 30 sec - exiting 20:18:21 (59024): No heartbeat from core client for 30 sec - exiting 20:18:22 (59024): No heartbeat from core client for 30 sec - exiting 20:18:23 (59024): No heartbeat from core client for 30 sec - exiting 20:18:24 (59024): No heartbeat from core client for 30 sec - exiting 20:18:25 (59024): No heartbeat from core client for 30 sec - exiting 20:18:26 (59024): No heartbeat from core client for 30 sec - exiting 20:18:27 (59024): No heartbeat from core client for 30 sec - exiting 20:18:28 (59024): No heartbeat from core client for 30 sec - exiting 20:18:29 (59024): No heartbeat from core client for 30 sec - exiting 20:18:30 (59024): No heartbeat from core client for 30 sec - exiting 20:18:31 (59024): No heartbeat from core client for 30 sec - exiting 20:18:32 (59024): No heartbeat from core client for 30 sec - exiting 20:18:33 (59024): No heartbeat from core client for 30 sec - exiting 20:18:34 (59024): No heartbeat from core client for 30 sec - exiting 20:18:35 (59024): No heartbeat from core client for 30 sec - exiting 20:18:36 (59024): No heartbeat from core client for 30 sec - exiting 20:18:37 (59024): No heartbeat from core client for 30 sec - exiting 20:18:38 (59024): No heartbeat from core client for 30 sec - exiting 20:18:39 (59024): No heartbeat from core client for 30 sec - exiting 20:18:40 (59024): No heartbeat from core client for 30 sec - exiting 20:18:41 (59024): No heartbeat from core client for 30 sec - exiting 20:18:42 (59024): No heartbeat from core client for 30 sec - exiting 20:18:43 (59024): No heartbeat from core client for 30 sec - exiting 20:18:44 (59024): No heartbeat from core client for 30 sec - exiting 20:18:45 (59024): No heartbeat from core client for 30 sec - exiting 20:18:46 (59024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:18:47 (59024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:55:23 (10559): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:56:12 (12866): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:00:16 (12870): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:00:56 (12870): No heartbeat from core client for 30 sec - exiting 20:00:57 (12870): No heartbeat from core client for 30 sec - exiting 20:00:58 (12870): No heartbeat from core client for 30 sec - exiting 20:00:59 (12870): No heartbeat from core client for 30 sec - exiting 20:01:00 (12870): No heartbeat from core client for 30 sec - exiting 20:01:01 (12870): No heartbeat from core client for 30 sec - exiting 20:01:02 (12870): No heartbeat from core client for 30 sec - exiting 20:01:03 (12870): No heartbeat from core client for 30 sec - exiting 20:01:04 (12870): No heartbeat from core client for 30 sec - exiting 20:01:05 (12870): No heartbeat from core client for 30 sec - exiting 20:01:06 (12870): No heartbeat from core client for 30 sec - exiting 20:01:07 (12870): No heartbeat from core client for 30 sec - exiting 20:01:08 (12870): No heartbeat from core client for 30 sec - exiting 20:01:09 (12870): No heartbeat from core client for 30 sec - exiting 20:01:10 (12870): No heartbeat from core client for 30 sec - exiting 20:01:11 (12870): No heartbeat from core client for 30 sec - exiting 20:01:12 (12870): No heartbeat from core client for 30 sec - exiting 20:01:13 (12870): No heartbeat from core client for 30 sec - exiting 20:01:14 (12870): No heartbeat from core client for 30 sec - exiting 20:01:15 (12870): No heartbeat from core client for 30 sec - exiting 20:01:16 (12870): No heartbeat from core client for 30 sec - exiting 20:01:17 (12870): No heartbeat from core client for 30 sec - exiting 20:01:18 (12870): No heartbeat from core client for 30 sec - exiting 20:01:19 (12870): No heartbeat from core client for 30 sec - exiting 20:01:20 (12870): No heartbeat from core client for 30 sec - exiting 20:01:21 (12870): No heartbeat from core client for 30 sec - exiting 20:01:22 (12870): No heartbeat from core client for 30 sec - exiting 20:01:23 (12870): No heartbeat from core client for 30 sec - exiting 20:01:24 (12870): No heartbeat from core client for 30 sec - exiting 20:01:25 (12870): No heartbeat from core client for 30 sec - exiting 20:01:26 (12870): No heartbeat from core client for 30 sec - exiting 20:01:27 (12870): No heartbeat from core client for 30 sec - exiting 20:01:28 (12870): No heartbeat from core client for 30 sec - exiting 20:01:29 (12870): No heartbeat from core client for 30 sec - exiting 20:01:30 (12870): No heartbeat from core client for 30 sec - exiting 20:01:31 (12870): No heartbeat from core client for 30 sec - exiting 20:01:32 (12870): No heartbeat from core client for 30 sec - exiting 20:01:33 (12870): No heartbeat from core client for 30 sec - exiting 20:01:34 (12870): No heartbeat from core client for 30 sec - exiting 20:01:35 (12870): No heartbeat from core client for 30 sec - exiting 20:01:36 (12870): No heartbeat from core client for 30 sec - exiting 20:01:37 (12870): No heartbeat from core client for 30 sec - exiting 20:01:38 (12870): No heartbeat from core client for 30 sec - exiting 20:01:39 (12870): No heartbeat from core client for 30 sec - exiting 20:01:40 (12870): No heartbeat from core client for 30 sec - exiting 20:01:41 (12870): No heartbeat from core client for 30 sec - exiting 20:01:42 (12870): No heartbeat from core client for 30 sec - exiting 20:01:43 (12870): No heartbeat from core client for 30 sec - exiting 20:01:44 (12870): No heartbeat from core client for 30 sec - exiting 20:01:45 (12870): No heartbeat from core client for 30 sec - exiting 20:01:46 (12870): No heartbeat from core client for 30 sec - exiting 20:01:47 (12870): No heartbeat from core client for 30 sec - exiting 20:01:48 (12870): No heartbeat from core client for 30 sec - exiting 20:01:49 (12870): No heartbeat from core client for 30 sec - exiting 21:21:12 (12907): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_zcor_1880_40_008245864/dataout/stdout_um.txt Image PC Routine Line Source hadcm3n_um_6.07_i 0042E622 Unknown Unknown Unknown hadcm3n_um_6.07_i 0042D0EB Unknown Unknown Unknown hadcm3n_um_6.07_i 003EF4DD Unknown Unknown Unknown hadcm3n_um_6.07_i 003A66F5 Unknown Unknown Unknown hadcm3n_um_6.07_i 003A5E57 Unknown Unknown Unknown hadcm3n_um_6.07_i 003E1BF7 Unknown Unknown Unknown hadcm3n_um_6.07_i 000D93BE Unknown Unknown Unknown hadcm3n_um_6.07_i 0015826A Unknown Unknown Unknown hadcm3n_um_6.07_i 00163AB7 Unknown Unknown Unknown hadcm3n_um_6.07_i 0038CE7D Unknown Unknown Unknown hadcm3n_um_6.07_i 0038A1ED Unknown Unknown Unknown hadcm3n_um_6.07_i 000048E1 Unknown Unknown Unknown hadcm3n_um_6.07_i 000027FB Unknown Unknown Unknown hadcm3n_um_6.07_i 00002729 Unknown Unknown Unknown Unknown 00000003 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2911, iMonCtr=1 Model crash detected, will try to restart... forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_zcor_1880_40_008245864/dataout/stdout_um.txt Image PC Routine Line Source hadcm3n_um_6.07_i 0042E622 Unknown Unknown Unknown hadcm3n_um_6.07_i 0042D0EB Unknown Unknown Unknown hadcm3n_um_6.07_i 003EF4DD Unknown Unknown Unknown hadcm3n_um_6.07_i 003A66F5 Unknown Unknown Unknown hadcm3n_um_6.07_i 003A5E57 Unknown Unknown Unknown hadcm3n_um_6.07_i 003E1E32 Unknown Unknown Unknown hadcm3n_um_6.07_i 003DF4CC Unknown Unknown Unknown hadcm3n_um_6.07_i 003651C5 Unknown Unknown Unknown hadcm3n_um_6.07_i 002DE95F Unknown Unknown Unknown hadcm3n_um_6.07_i 00389F90 Unknown Unknown Unknown hadcm3n_um_6.07_i 000048E1 Unknown Unknown Unknown hadcm3n_um_6.07_i 000027FB Unknown Unknown Unknown hadcm3n_um_6.07_i 00002729 Unknown Unknown Unknown Unknown 00000003 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2911, iMonCtr=1 Model crash detected, will try to restart... forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_zcor_1880_40_008245864/dataout/stdout_um.txt Image PC Routine Line Source hadcm3n_um_6.07_i 0042E622 Unknown Unknown Unknown hadcm3n_um_6.07_i 0042D0EB Unknown Unknown Unknown hadcm3n_um_6.07_i 003EF4DD Unknown Unknown Unknown hadcm3n_um_6.07_i 003A66F5 Unknown Unknown Unknown hadcm3n_um_6.07_i 003A5E57 Unknown Unknown UnknCalled boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
07 Apr 2013 04:01:22	1180505	15614153	hadcm3n_zcor_1880_40_008245864_4	699,840	960,900	1.3730
05 Apr 2013 16:47:24	1180505	15614153	hadcm3n_zcor_1880_40_008245864_4	673,920	924,577	1.3719
05 Apr 2013 04:22:55	1180505	15614153	hadcm3n_zcor_1880_40_008245864_4	648,000	889,258	1.3723
04 Apr 2013 07:45:14	1180505	15614153	hadcm3n_zcor_1880_40_008245864_4	622,080	853,669	1.3723
03 Apr 2013 19:58:59	1180505	15614153	hadcm3n_zcor_1880_40_008245864_4	596,160	817,372	1.3711
02 Apr 2013 20:19:22	1180505	15614153	hadcm3n_zcor_1880_40_008245864_4	570,240	781,567	1.3706
01 Apr 2013 08:09:14	1180505	15614153	hadcm3n_zcor_1880_40_008245864_4	544,320	746,076	1.3707
29 Mar 2013 10:36:35	1180505	15614153	hadcm3n_zcor_1880_40_008245864_4	518,400	710,866	1.3713
27 Mar 2013 17:21:25	1180505	15614153	hadcm3n_zcor_1880_40_008245864_4	492,480	675,662	1.3720
26 Mar 2013 08:46:08	1180505	15614153	hadcm3n_zcor_1880_40_008245864_4	466,560	640,146	1.3721
25 Mar 2013 16:16:45	1180505	15614153	hadcm3n_zcor_1880_40_008245864_4	440,640	604,449	1.3718
24 Mar 2013 16:50:44	1180505	15614153	hadcm3n_zcor_1880_40_008245864_4	414,720	569,252	1.3726
23 Mar 2013 01:45:11	1180505	15614153	hadcm3n_zcor_1880_40_008245864_4	388,800	533,667	1.3726
18 Mar 2013 06:48:09	1180505	15614153	hadcm3n_zcor_1880_40_008245864_4	362,880	497,532	1.3711
13 Mar 2013 18:16:42	1180505	15614153	hadcm3n_zcor_1880_40_008245864_4	336,960	461,732	1.3703
11 Mar 2013 13:40:19	1180505	15614153	hadcm3n_zcor_1880_40_008245864_4	311,040	426,205	1.3703
08 Mar 2013 13:45:16	1180505	15614153	hadcm3n_zcor_1880_40_008245864_4	285,120	391,323	1.3725
06 Mar 2013 14:00:15	1180505	15614153	hadcm3n_zcor_1880_40_008245864_4	259,200	356,171	1.3741
05 Mar 2013 21:02:07	1180505	15614153	hadcm3n_zcor_1880_40_008245864_4	233,280	320,831	1.3753
05 Mar 2013 10:51:27	1180505	15614153	hadcm3n_zcor_1880_40_008245864_4	207,360	285,761	1.3781