Task 16072026

Name	hadcm3n_ocon_1900_40_008471562_1
Workunit	8622401
Created	22 Oct 2013, 10:00:51 UTC
Sent	22 Oct 2013, 10:01:31 UTC
Report deadline	21 Jan 2014, 17:28:42 UTC
Received	27 Dec 2013, 21:51:28 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1297857
Run time	35 days 6 hours 30 min 41 sec
CPU time	26 days 23 hours 2 min
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	1.95 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6428, iMonCtr=1 Model crash detected, will try to restart... 06:27:05 (5456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2268, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4240, iMonCtr=1 Model crash detected, will try to restart... 09:21:14 (4820): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:21:15 (4820): No heartbeat from core client for 30 sec - exiting 09:21:16 (4820): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 14:25:16 (5708): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:25:17 (5708): No heartbeat from core client for 30 sec - exiting 14:25:18 (5708): No heartbeat from core client for 30 sec - exiting 14:25:19 (5708): No heartbeat from core client for 30 sec - exiting 14:25:20 (5708): No heartbeat from core client for 30 sec - exiting 14:25:21 (5708): No heartbeat from core client for 30 sec - exiting 14:25:22 (5708): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:31:28 (4828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:31:29 (4828): No heartbeat from core client for 30 sec - exiting 08:31:30 (4828): No heartbeat from core client for 30 sec - exiting 08:31:31 (4828): No heartbeat from core client for 30 sec - exiting 08:31:32 (4828): No heartbeat from core client for 30 sec - exiting 08:31:33 (4828): No heartbeat from core client for 30 sec - exiting 08:31:34 (4828): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 19:52:30 (7812): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 19:52:32 (7812): No heartbeat from core client for 30 sec - exiting 19:52:33 (7812): No heartbeat from core client for 30 sec - exiting 19:52:34 (7812): No heartbeat from core client for 30 sec - exiting 19:52:35 (7812): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 21:58:50 (6072): No heartbeat from core client for 30 sec - exiting 21:58:52 (6072): No heartbeat from core client for 30 sec - exiting 21:58:53 (6072): No heartbeat from core client for 30 sec - exiting 21:58:55 (6072): No heartbeat from core client for 30 sec - exiting 21:58:56 (6072): No heartbeat from core client for 30 sec - exiting 21:58:57 (6072): No heartbeat from core client for 30 sec - exiting 21:58:58 (6072): No heartbeat from core client for 30 sec - exiting 21:59:00 (6072): No heartbeat from core client for 30 sec - exiting 21:59:01 (6072): No heartbeat from core client for 30 sec - exiting 21:59:03 (6072): No heartbeat from core client for 30 sec - exiting 21:59:04 (6072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:00:57 (5428): No heartbeat from core client for 30 sec - exiting 22:01:18 (5428): No heartbeat from core client for 30 sec - exiting 22:01:37 (5428): No heartbeat from core client for 30 sec - exiting 22:01:54 (5428): No heartbeat from core client for 30 sec - exiting 22:02:16 (5428): No heartbeat from core client for 30 sec - exiting 22:02:19 (5428): No heartbeat from core client for 30 sec - exiting 22:02:24 (5428): No heartbeat from core client for 30 sec - exiting 22:02:33 (5428): No heartbeat from core client for 30 sec - exiting 22:02:35 (5428): No heartbeat from core client for 30 sec - exiting 22:02:38 (5428): No heartbeat from core client for 30 sec - exiting 22:02:42 (5428): No heartbeat from core client for 30 sec - exiting 22:02:47 (5428): No heartbeat from core client for 30 sec - exiting 22:02:50 (5428): No heartbeat from core client for 30 sec - exiting 22:02:52 (5428): No heartbeat from core client for 30 sec - exiting 22:02:55 (5428): No heartbeat from core client for 30 sec - exiting 22:02:56 (5428): No heartbeat from core client for 30 sec - exiting 22:02:57 (5428): No heartbeat from core client for 30 sec - exiting 22:03:02 (5428): No heartbeat from core client for 30 sec - exiting 22:03:08 (5428): No heartbeat from core client for 30 sec - exiting 22:03:33 (5428): No heartbeat from core client for 30 sec - exiting 22:03:44 (5428): No heartbeat from core client for 30 sec - exiting 22:03:52 (5428): No heartbeat from core client for 30 sec - exiting 22:04:02 (5428): No heartbeat from core client for 30 sec - exiting 22:04:15 (5428): No heartbeat from core client for 30 sec - exiting 22:04:24 (5428): No heartbeat from core client for 30 sec - exiting 22:04:33 (5428): No heartbeat from core client for 30 sec - exiting 22:04:36 (5428): No heartbeat from core client for 30 sec - exiting 22:04:39 (5428): No heartbeat from core client for 30 sec - exiting 22:04:45 (5428): No heartbeat from core client for 30 sec - exiting 22:04:52 (5428): No heartbeat from core client for 30 sec - exiting 22:04:55 (5428): No heartbeat from core client for 30 sec - exiting 22:05:01 (5428): No heartbeat from core client for 30 sec - exiting 22:05:09 (5428): No heartbeat from core client for 30 sec - exiting 22:05:19 (5428): No heartbeat from core client for 30 sec - exiting 22:05:30 (5428): No heartbeat from core client for 30 sec - exiting 22:05:44 (5428): No heartbeat from core client for 30 sec - exiting 22:05:52 (5428): No heartbeat from core client for 30 sec - exiting 22:05:59 (5428): No heartbeat from core client for 30 sec - exiting 22:06:10 (5428): No heartbeat from core client for 30 sec - exiting 22:06:22 (5428): No heartbeat from core client for 30 sec - exiting 22:06:33 (5428): No heartbeat from core client for 30 sec - exiting 22:06:44 (5428): No heartbeat from core client for 30 sec - exiting 22:06:53 (5428): No heartbeat from core client for 30 sec - exiting 22:07:06 (5428): No heartbeat from core client for 30 sec - exiting 22:07:22 (5428): No heartbeat from core client for 30 sec - exiting 22:07:33 (5428): No heartbeat from core client for 30 sec - exiting 22:07:37 (5428): No heartbeat from core client for 30 sec - exiting 22:07:42 (5428): No heartbeat from core client for 30 sec - exiting 22:07:47 (5428): No heartbeat from core client for 30 sec - exiting 22:07:56 (5428): No heartbeat from core client for 30 sec - exiting 22:08:03 (5428): No heartbeat from core client for 30 sec - exiting 22:08:10 (5428): No heartbeat from core client for 30 sec - exiting 22:08:14 (5428): No heartbeat from core client for 30 sec - exiting 22:08:21 (5428): No heartbeat from core client for 30 sec - exiting 22:08:25 (5428): No heartbeat from core client for 30 sec - exiting 22:08:32 (5428): No heartbeat from core client for 30 sec - exiting 22:08:38 (5428): No heartbeat from core client for 30 sec - exiting 22:08:43 (5428): No heartbeat from core client for 30 sec - exiting 22:08:46 (5428): No heartbeat from core client for 30 sec - exiting 22:08:50 (5428): No heartbeat from core client for 30 sec - exiting 22:08:57 (5428): No heartbeat from core client for 30 sec - exiting 22:09:03 (5428): No heartbeat from core client for 30 sec - exiting 22:09:39 (5428): No heartbeat from core client for 30 sec - exiting 22:09:49 (5428): No heartbeat from core client for 30 sec - exiting 22:09:57 (5428): No heartbeat from core client for 30 sec - exiting 22:10:04 (5428): No heartbeat from core client for 30 sec - exiting 22:10:12 (5428): No heartbeat from core client for 30 sec - exiting 22:10:18 (5428): No heartbeat from core client for 30 sec - exiting 22:10:32 (5428): No heartbeat from core client for 30 sec - exiting 22:10:41 (5428): No heartbeat from core client for 30 sec - exiting 22:10:54 (5428): No heartbeat from core client for 30 sec - exiting 22:11:04 (5428): No heartbeat from core client for 30 sec - exiting 22:11:10 (5428): No heartbeat from core client for 30 sec - exiting 22:11:20 (5428): No heartbeat from core client for 30 sec - exiting 22:11:24 (5428): No heartbeat from core client for 30 sec - exiting 22:11:30 (5428): No heartbeat from core client for 30 sec - exiting 22:11:36 (5428): No heartbeat from core client for 30 sec - exiting 22:11:47 (5428): No heartbeat from core client for 30 sec - exiting 22:11:57 (5428): No heartbeat from core client for 30 sec - exiting 22:12:10 (5428): No heartbeat from core client for 30 sec - exiting 22:12:22 (5428): No heartbeat from core client for 30 sec - exiting 22:12:28 (5428): No heartbeat from core client for 30 sec - exiting 22:12:37 (5428): No heartbeat from core client for 30 sec - exiting 22:12:42 (5428): No heartbeat from core client for 30 sec - exiting 22:12:53 (5428): No heartbeat from core client for 30 sec - exiting 22:13:03 (5428): No heartbeat from core client for 30 sec - exiting 22:13:06 (5428): No heartbeat from core client for 30 sec - exiting 22:13:13 (5428): No heartbeat from core client for 30 sec - exiting 22:13:18 (5428): No heartbeat from core client for 30 sec - exiting 22:13:23 (5428): No heartbeat from core client for 30 sec - exiting 22:13:32 (5428): No heartbeat from core client for 30 sec - exiting 22:13:39 (5428): No heartbeat from core client for 30 sec - exiting 22:13:43 (5428): No heartbeat from core client for 30 sec - exiting 22:13:49 (5428): No heartbeat from core client for 30 sec - exiting 22:13:56 (5428): No heartbeat from core client for 30 sec - exiting 22:14:02 (5428): No heartbeat from core client for 30 sec - exiting 22:14:06 (5428): No heartbeat from core client for 30 sec - exiting 22:14:14 (5428): No heartbeat from core client for 30 sec - exiting 22:14:21 (5428): No heartbeat from core client for 30 sec - exiting 22:14:26 (5428): No heartbeat from core client for 30 sec - exiting 22:14:31 (5428): No heartbeat from core client for 30 sec - exiting 22:14:37 (5428): No heartbeat from core client for 30 sec - exiting 22:14:46 (5428): No heartbeat from core client for 30 sec - exiting 22:14:55 (5428): No heartbeat from core client for 30 sec - exiting 22:15:01 (5428): No heartbeat from core client for 30 sec - exiting 22:15:08 (5428): No heartbeat from core client for 30 sec - exiting 22:15:17 (5428): No heartbeat from core client for 30 sec - exiting 22:15:25 (5428): No heartbeat from core client for 30 sec - exiting 22:15:30 (5428): No heartbeat from core client for 30 sec - exiting 22:15:34 (5428): No heartbeat from core client for 30 sec - exiting 22:15:39 (5428): No heartbeat from core client for 30 sec - exiting 22:15:48 (5428): No heartbeat from core client for 30 sec - exiting 22:15:57 (5428): No heartbeat from core client for 30 sec - exiting 22:16:06 (5428): No heartbeat from core client for 30 sec - exiting 22:16:15 (5428): No heartbeat from core client for 30 sec - exiting 22:16:18 (5428): No heartbeat from core client for 30 sec - exiting 22:16:24 (5428): No heartbeat from core client for 30 sec - exiting 22:16:25 (5428): No heartbeat from core client for 30 sec - exiting 22:16:30 (5428): No heartbeat from core client for 30 sec - exiting 22:16:33 (5428): No heartbeat from core client for 30 sec - exiting 22:16:36 (5428): No heartbeat from core client for 30 sec - exiting 22:16:39 (5428): No heartbeat from core client for 30 sec - exiting 22:16:42 (5428): No heartbeat from core client for 30 sec - exiting 22:16:47 (5428): No heartbeat from core client for 30 sec - exiting 22:16:54 (5428): No heartbeat from core client for 30 sec - exiting 22:17:01 (5428): No heartbeat from core client for 30 sec - exiting 22:17:08 (5428): No heartbeat from core client for 30 sec - exiting 22:17:16 (5428): No heartbeat from core client for 30 sec - exiting 22:17:24 (5428): No heartbeat from core client for 30 sec - exiting 22:17:31 (5428): No heartbeat from core client for 30 sec - exiting 22:17:38 (5428): No heartbeat from core client for 30 sec - exiting 22:17:44 (5428): No heartbeat from core client for 30 sec - exiting 22:17:52 (5428): No heartbeat from core client for 30 sec - exiting 22:18:00 (5428): No heartbeat from core client for 30 sec - exiting 22:18:06 (5428): No heartbeat from core client for 30 sec - exiting 22:18:14 (5428): No heartbeat from core client for 30 sec - exiting 22:18:23 (5428): No heartbeat from core client for 30 sec - exiting 22:18:30 (5428): No heartbeat from core client for 30 sec - exiting 22:18:39 (5428): No heartbeat from core client for 30 sec - exiting 22:18:48 (5428): No heartbeat from core client for 30 sec - exiting 22:18:56 (5428): No heartbeat from core client for 30 sec - exiting 22:19:04 (5428): No heartbeat from core client for 30 sec - exiting 22:19:11 (5428): No heartbeat from core client for 30 sec - exiting 22:19:19 (5428): No heartbeat from core client for 30 sec - exiting 22:19:27 (5428): No heartbeat from core client for 30 sec - exiting 22:19:35 (5428): No heartbeat from core client for 30 sec - exiting 22:19:42 (5428): No heartbeat from core client for 30 sec - exiting 22:19:49 (5428): No heartbeat from core client for 30 sec - exiting 22:19:55 (5428): No heartbeat from core client for 30 sec - exiting 22:20:02 (5428): No heartbeat from core client for 30 sec - exiting 22:20:10 (5428): No heartbeat from core client for 30 sec - exiting 22:20:18 (5428): No heartbeat from core client for 30 sec - exiting 22:20:26 (5428): No heartbeat from core client for 30 sec - exiting 22:20:37 (5428): No heartbeat from core client for 30 sec - exiting 22:20:41 (5428): No heartbeat from core client for 30 sec - exiting 22:20:46 (5428): No heartbeat from core client for 30 sec - exiting 22:20:52 (5428): No heartbeat from core client for 30 sec - exiting 22:20:56 (5428): No heartbeat from core client for 30 sec - exiting 22:21:03 (5428): No heartbeat from core client for 30 sec - exiting 22:21:11 (5428): No heartbeat from core client for 30 sec - exiting 22:21:13 (5428): No heartbeat from core client for 30 sec - exiting 22:21:15 (5428): No heartbeat from core client for 30 sec - exiting 22:21:16 (5428): No heartbeat from core client for 30 sec - exiting 22:21:17 (5428): No heartbeat from core client for 30 sec - exiting 22:21:18 (5428): No heartbeat from core client for 30 sec - exiting 22:21:20 (5428): No heartbeat from core client for 30 sec - exiting 22:21:21 (5428): No heartbeat from core client for 30 sec - exiting 22:21:23 (5428): No heartbeat from core client for 30 sec - exiting 22:21:24 (5428): No heartbeat from core client for 30 sec - exiting 22:21:25 (5428): No heartbeat from core client for 30 sec - exiting 22:21:27 (5428): No heartbeat from core client for 30 sec - exiting 22:21:28 (5428): No heartbeat from core client for 30 sec - exiting 22:21:29 (5428): No heartbeat from core client for 30 sec - exiting 22:21:31 (5428): No heartbeat from core client for 30 sec - exiting 22:21:32 (5428): No heartbeat from core client for 30 sec - exiting 22:21:33 (5428): No heartbeat from core client for 30 sec - exiting 22:21:34 (5428): No heartbeat from core client for 30 sec - exiting 22:21:36 (5428): No heartbeat from core client for 30 sec - exiting 22:21:43 (5428): No heartbeat from core client for 30 sec - exiting 22:21:53 (5428): No heartbeat from core client for 30 sec - exiting 22:22:06 (5428): No heartbeat from core client for 30 sec - exiting 22:22:20 (5428): No heartbeat from core client for 30 sec - exiting 22:22:30 (5428): No heartbeat from core client for 30 sec - exiting 22:22:41 (5428): No heartbeat from core client for 30 sec - exiting 22:22:52 (5428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:33:12 (5344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:33:14 (5344): No heartbeat from core client for 30 sec - exiting 23:38:20 (3864): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:38:25 (3864): No heartbeat from core client for 30 sec - exiting 23:39:50 (5548): No heartbeat from core client for 30 sec - exiting 23:40:05 (5548): No heartbeat from core client for 30 sec - exiting 23:40:11 (5548): No heartbeat from core client for 30 sec - exiting 23:40:13 (5548): No heartbeat from core client for 30 sec - exiting 23:40:14 (5548): No heartbeat from core client for 30 sec - exiting 23:40:15 (5548): No heartbeat from core client for 30 sec - exiting 23:40:16 (5548): No heartbeat from core client for 30 sec - exiting 23:40:17 (5548): No heartbeat from core client for 30 sec - exiting 23:40:19 (5548): No heartbeat from core client for 30 sec - exiting 23:40:20 (5548): No heartbeat from core client for 30 sec - exiting 23:40:21 (5548): No heartbeat from core client for 30 sec - exiting 23:40:22 (5548): No heartbeat from core client for 30 sec - exiting 23:40:24 (5548): No heartbeat from core client for 30 sec - exiting 23:40:25 (5548): No heartbeat from core client for 30 sec - exiting 23:40:26 (5548): No heartbeat from core client for 30 sec - exiting 23:40:57 (5548): No heartbeat from core client for 30 sec - exiting 23:40:59 (5548): No heartbeat from core client for 30 sec - exiting 23:41:00 (5548): No heartbeat from core client for 30 sec - exiting 23:41:34 (5548): No heartbeat from core client for 30 sec - exiting 23:41:44 (5548): No heartbeat from core client for 30 sec - exiting 23:41:45 (5548): No heartbeat from core client for 30 sec - exiting 23:41:47 (5548): No heartbeat from core client for 30 sec - exiting 23:41:48 (5548): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:10:01 (4600): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:10:02 (4600): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 14:43:30 (3084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:58:03 (5692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6056, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4924, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:27:57 (5112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:27:58 (5112): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4520, iMonCtr=1 Model crash detected, will try to restart... 04:48:06 (3304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:48:08 (3304): No heartbeat from core client for 30 sec - exiting 04:48:09 (3304): No heartbeat from core client for 30 sec - exiting 04:48:10 (3304): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1096, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 21:26:59 (6048): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:57:46 (672): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 08:57:50 (672): No heartbeat from core client for 30 sec - exiting 08:57:51 (672): No heartbeat from core client for 30 sec - exiting 08:57:52 (672): No heartbeat from core client for 30 sec - exiting 08:57:53 (672): No heartbeat from core client for 30 sec - exiting 08:57:54 (672): No heartbeat from core client for 30 sec - exiting 08:57:55 (672): No heartbeat from core client for 30 sec - exiting 08:57:56 (672): No heartbeat from core client for 30 sec - exiting 08:57:57 (672): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:45:38 (5100): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 03:45:39 (5100): No heartbeat from core client for 30 sec - exiting 03:45:40 (5100): No heartbeat from core client for 30 sec - exiting 03:45:41 (5100): No heartbeat from core client for 30 sec - exiting 03:45:42 (5100): No heartbeat from core client for 30 sec - exiting 03:45:43 (5100): No heartbeat from core client for 30 sec - exiting 03:45:44 (5100): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
26 Dec 2013 11:34:26	1297857	16072026	hadcm3n_ocon_1900_40_008471562_1	1,036,800	2,328,500	2.2459
21 Dec 2013 12:35:54	1297857	16072026	hadcm3n_ocon_1900_40_008471562_1	1,010,880	2,272,184	2.2477
18 Dec 2013 19:01:11	1297857	16072026	hadcm3n_ocon_1900_40_008471562_1	984,960	2,214,963	2.2488
15 Dec 2013 17:39:04	1297857	16072026	hadcm3n_ocon_1900_40_008471562_1	959,040	2,158,614	2.2508
05 Dec 2013 21:02:25	1297857	16072026	hadcm3n_ocon_1900_40_008471562_1	933,120	2,099,315	2.2498
05 Dec 2013 03:46:05	1297857	16072026	hadcm3n_ocon_1900_40_008471562_1	907,200	2,039,111	2.2477
04 Dec 2013 09:55:33	1297857	16072026	hadcm3n_ocon_1900_40_008471562_1	881,280	1,978,653	2.2452
03 Dec 2013 16:34:06	1297857	16072026	hadcm3n_ocon_1900_40_008471562_1	855,360	1,918,368	2.2428
02 Dec 2013 23:16:37	1297857	16072026	hadcm3n_ocon_1900_40_008471562_1	829,440	1,858,173	2.2403
02 Dec 2013 05:18:00	1297857	16072026	hadcm3n_ocon_1900_40_008471562_1	803,520	1,797,366	2.2369
01 Dec 2013 11:53:55	1297857	16072026	hadcm3n_ocon_1900_40_008471562_1	777,600	1,737,076	2.2339
30 Nov 2013 18:30:12	1297857	16072026	hadcm3n_ocon_1900_40_008471562_1	751,680	1,676,827	2.2308
30 Nov 2013 01:12:47	1297857	16072026	hadcm3n_ocon_1900_40_008471562_1	725,760	1,616,670	2.2276
29 Nov 2013 07:43:52	1297857	16072026	hadcm3n_ocon_1900_40_008471562_1	699,840	1,556,378	2.2239
28 Nov 2013 13:48:37	1297857	16072026	hadcm3n_ocon_1900_40_008471562_1	673,920	1,496,222	2.2202
27 Nov 2013 20:14:27	1297857	16072026	hadcm3n_ocon_1900_40_008471562_1	648,000	1,436,076	2.2162
27 Nov 2013 02:44:14	1297857	16072026	hadcm3n_ocon_1900_40_008471562_1	622,080	1,376,023	2.2120
26 Nov 2013 08:35:14	1297857	16072026	hadcm3n_ocon_1900_40_008471562_1	596,160	1,315,144	2.2060
22 Nov 2013 22:04:18	1297857	16072026	hadcm3n_ocon_1900_40_008471562_1	570,240	1,253,592	2.1984
22 Nov 2013 04:54:44	1297857	16072026	hadcm3n_ocon_1900_40_008471562_1	544,320	1,193,753	2.1931