Task 15822088

Name	hadcm3n_o536_1980_40_008386496_0
Workunit	8537355
Created	3 Jun 2013, 6:44:58 UTC
Sent	8 Jun 2013, 11:45:34 UTC
Report deadline	7 Sep 2013, 19:12:45 UTC
Received	24 Jun 2013, 18:47:56 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1181249
Run time	6 days 22 hours 9 min 31 sec
CPU time	6 days 11 hours 16 min 32 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.03 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 04:08:42 (5748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:45:31 (5364): No heartbeat from core client for 30 sec - exiting 04:45:38 (5364): No heartbeat from core client for 30 sec - exiting 04:45:39 (5364): No heartbeat from core client for 30 sec - exiting 04:46:48 (5364): No heartbeat from core client for 30 sec - exiting 04:46:49 (5364): No heartbeat from core client for 30 sec - exiting 04:46:50 (5364): No heartbeat from core client for 30 sec - exiting 04:46:51 (5364): No heartbeat from core client for 30 sec - exiting 04:46:52 (5364): No heartbeat from core client for 30 sec - exiting 04:46:55 (5364): No heartbeat from core client for 30 sec - exiting 04:46:56 (5364): No heartbeat from core client for 30 sec - exiting 04:46:57 (5364): No heartbeat from core client for 30 sec - exiting 04:46:58 (5364): No heartbeat from core client for 30 sec - exiting 04:46:59 (5364): No heartbeat from core client for 30 sec - exiting 04:47:00 (5364): No heartbeat from core client for 30 sec - exiting 04:47:01 (5364): No heartbeat from core client for 30 sec - exiting 04:47:02 (5364): No heartbeat from core client for 30 sec - exiting 04:47:03 (5364): No heartbeat from core client for 30 sec - exiting 04:47:04 (5364): No heartbeat from core client for 30 sec - exiting 04:47:05 (5364): No heartbeat from core client for 30 sec - exiting 04:47:06 (5364): No heartbeat from core client for 30 sec - exiting 04:47:07 (5364): No heartbeat from core client for 30 sec - exiting 04:47:08 (5364): No heartbeat from core client for 30 sec - exiting 04:47:09 (5364): No heartbeat from core client for 30 sec - exiting 04:47:10 (5364): No heartbeat from core client for 30 sec - exiting 04:47:11 (5364): No heartbeat from core client for 30 sec - exiting 04:47:12 (5364): No heartbeat from core client for 30 sec - exiting 04:47:13 (5364): No heartbeat from core client for 30 sec - exiting 04:47:14 (5364): No heartbeat from core client for 30 sec - exiting 04:47:15 (5364): No heartbeat from core client for 30 sec - exiting 04:47:16 (5364): No heartbeat from core client for 30 sec - exiting 04:47:17 (5364): No heartbeat from core client for 30 sec - exiting 04:47:18 (5364): No heartbeat from core client for 30 sec - exiting 04:47:19 (5364): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:21:02 (6316): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:09:49 (7760): No heartbeat from core client for 30 sec - exiting 19:09:50 (7760): No heartbeat from core client for 30 sec - exiting 19:09:51 (7760): No heartbeat from core client for 30 sec - exiting 19:09:52 (7760): No heartbeat from core client for 30 sec - exiting 19:09:53 (7760): No heartbeat from core client for 30 sec - exiting 19:09:54 (7760): No heartbeat from core client for 30 sec - exiting 19:09:55 (7760): No heartbeat from core client for 30 sec - exiting 19:09:56 (7760): No heartbeat from core client for 30 sec - exiting 19:09:57 (7760): No heartbeat from core client for 30 sec - exiting 19:09:58 (7760): No heartbeat from core client for 30 sec - exiting 19:09:59 (7760): No heartbeat from core client for 30 sec - exiting 19:10:00 (7760): No heartbeat from core client for 30 sec - exiting 19:10:01 (7760): No heartbeat from core client for 30 sec - exiting 19:10:02 (7760): No heartbeat from core client for 30 sec - exiting 19:10:03 (7760): No heartbeat from core client for 30 sec - exiting 19:10:04 (7760): No heartbeat from core client for 30 sec - exiting 19:10:05 (7760): No heartbeat from core client for 30 sec - exiting 19:10:06 (7760): No heartbeat from core client for 30 sec - exiting 19:10:07 (7760): No heartbeat from core client for 30 sec - exiting 19:10:08 (7760): No heartbeat from core client for 30 sec - exiting 19:10:09 (7760): No heartbeat from core client for 30 sec - exiting 19:10:10 (7760): No heartbeat from core client for 30 sec - exiting 19:10:11 (7760): No heartbeat from core client for 30 sec - exiting 19:10:12 (7760): No heartbeat from core client for 30 sec - exiting 19:10:13 (7760): No heartbeat from core client for 30 sec - exiting 19:10:14 (7760): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:29:36 (9596): No heartbeat from core client for 30 sec - exiting 10:29:37 (9596): No heartbeat from core client for 30 sec - exiting 10:29:38 (9596): No heartbeat from core client for 30 sec - exiting 10:29:39 (9596): No heartbeat from core client for 30 sec - exiting 10:29:40 (9596): No heartbeat from core client for 30 sec - exiting 10:29:41 (9596): No heartbeat from core client for 30 sec - exiting 10:29:42 (9596): No heartbeat from core client for 30 sec - exiting 10:29:43 (9596): No heartbeat from core client for 30 sec - exiting 10:29:44 (9596): No heartbeat from core client for 30 sec - exiting 10:29:45 (9596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 09:48:05 (10112): No heartbeat from core client for 30 sec - exiting 09:48:07 (10112): No heartbeat from core client for 30 sec - exiting 09:48:08 (10112): No heartbeat from core client for 30 sec - exiting 09:48:09 (10112): No heartbeat from core client for 30 sec - exiting 09:48:10 (10112): No heartbeat from core client for 30 sec - exiting 09:48:11 (10112): No heartbeat from core client for 30 sec - exiting 09:48:12 (10112): No heartbeat from core client for 30 sec - exiting 09:48:13 (10112): No heartbeat from core client for 30 sec - exiting 09:48:14 (10112): No heartbeat from core client for 30 sec - exiting 09:48:15 (10112): No heartbeat from core client for 30 sec - exiting 09:48:16 (10112): No heartbeat from core client for 30 sec - exiting 09:48:17 (10112): No heartbeat from core client for 30 sec - exiting 09:48:18 (10112): No heartbeat from core client for 30 sec - exiting 09:48:19 (10112): No heartbeat from core client for 30 sec - exiting 09:48:20 (10112): No heartbeat from core client for 30 sec - exiting 09:48:21 (10112): No heartbeat from core client for 30 sec - exiting 09:48:22 (10112): No heartbeat from core client for 30 sec - exiting 09:48:23 (10112): No heartbeat from core client for 30 sec - exiting 09:48:24 (10112): No heartbeat from core client for 30 sec - exiting 09:48:25 (10112): No heartbeat from core client for 30 sec - exiting 09:48:26 (10112): No heartbeat from core client for 30 sec - exiting 09:48:27 (10112): No heartbeat from core client for 30 sec - exiting 09:48:28 (10112): No heartbeat from core client for 30 sec - exiting 09:48:29 (10112): No heartbeat from core client for 30 sec - exiting 09:48:30 (10112): No heartbeat from core client for 30 sec - exiting 09:48:31 (10112): No heartbeat from core client for 30 sec - exiting 09:48:32 (10112): No heartbeat from core client for 30 sec - exiting 09:48:33 (10112): No heartbeat from core client for 30 sec - exiting 09:48:34 (10112): No heartbeat from core client for 30 sec - exiting 09:48:35 (10112): No heartbeat from core client for 30 sec - exiting 09:48:36 (10112): No heartbeat from core client for 30 sec - exiting 09:48:37 (10112): No heartbeat from core client for 30 sec - exiting 09:48:38 (10112): No heartbeat from core client for 30 sec - exiting 09:48:39 (10112): No heartbeat from core client for 30 sec - exiting 09:48:40 (10112): No heartbeat from core client for 30 sec - exiting 09:48:41 (10112): No heartbeat from core client for 30 sec - exiting 09:48:42 (10112): No heartbeat from core client for 30 sec - exiting 09:48:43 (10112): No heartbeat from core client for 30 sec - exiting 09:48:44 (10112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 04:33:04 (1628): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:33:05 (1628): No heartbeat from core client for 30 sec - exiting 04:33:06 (1628): No heartbeat from core client for 30 sec - exiting 04:33:07 (1628): No heartbeat from core client for 30 sec - exiting 04:43:33 (6656): No heartbeat from core client for 30 sec - exiting 04:43:34 (6656): No heartbeat from core client for 30 sec - exiting 04:43:35 (6656): No heartbeat from core client for 30 sec - exiting 04:43:36 (6656): No heartbeat from core client for 30 sec - exiting 04:43:37 (6656): No heartbeat from core client for 30 sec - exiting 04:43:38 (6656): No heartbeat from core client for 30 sec - exiting 04:43:39 (6656): No heartbeat from core client for 30 sec - exiting 04:43:40 (6656): No heartbeat from core client for 30 sec - exiting 04:43:41 (6656): No heartbeat from core client for 30 sec - exiting 04:43:42 (6656): No heartbeat from core client for 30 sec - exiting 04:43:43 (6656): No heartbeat from core client for 30 sec - exiting 04:43:44 (6656): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x76F13232 read attempt to address 0x40CFF120 Engaging BOINC Windows Runtime Debugger... Cannot serialize file D:\Boinc/projects/climateprediction.net/hadcm3n_o536_1980_40_008386496/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
24 Jun 2013 17:51:44	1181249	15822088	hadcm3n_o536_1980_40_008386496_0	259,200	558,989	2.1566
23 Jun 2013 20:37:24	1181249	15822088	hadcm3n_o536_1980_40_008386496_0	233,280	502,360	2.1535
23 Jun 2013 02:24:00	1181249	15822088	hadcm3n_o536_1980_40_008386496_0	207,360	445,745	2.1496
20 Jun 2013 20:45:50	1181249	15822088	hadcm3n_o536_1980_40_008386496_0	181,440	389,621	2.1474
18 Jun 2013 11:11:40	1181249	15822088	hadcm3n_o536_1980_40_008386496_0	155,520	333,669	2.1455
17 Jun 2013 03:43:53	1181249	15822088	hadcm3n_o536_1980_40_008386496_0	129,600	278,442	2.1485
11 Jun 2013 17:12:27	1181249	15822088	hadcm3n_o536_1980_40_008386496_0	103,680	222,458	2.1456
10 Jun 2013 22:23:13	1181249	15822088	hadcm3n_o536_1980_40_008386496_0	77,760	167,439	2.1533
10 Jun 2013 03:10:28	1181249	15822088	hadcm3n_o536_1980_40_008386496_0	51,840	112,042	2.1613
09 Jun 2013 08:29:35	1181249	15822088	hadcm3n_o536_1980_40_008386496_0	25,920	56,018	2.1612