Task 19210934

Name	hadam3p_anz_k1oe_201212_12_306_010265319_0
Workunit	10265319
Created	25 Jan 2016, 12:50:26 UTC
Sent	25 Jan 2016, 12:51:00 UTC
Report deadline	6 Jan 2017, 18:11:00 UTC
Received	2 Feb 2016, 4:03:15 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1333674
Run time	5 days 3 hours 39 min 14 sec
CPU time	5 days 1 hours 7 min 32 sec
Validate state	Invalid
Credit	4,484.28
Device peak FLOPS	4.22 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Australia New Zealand v6.10 windows_intelx86
Stderr	<core_client_version>7.6.22</core_client_version> <![CDATA[ <stderr_txt> diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 07:54:28 (12696): start_timer_thread(): CreateThread() failed, errno 0 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 07:54:30 (6596): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 20:05:15 (6300): start_timer_thread(): CreateThread() failed, errno 0 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 20:05:17 (4064): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 06:59:46 (15996): start_timer_thread(): CreateThread() failed, errno 0 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 06:59:48 (956): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 18:08:53 (7996): start_timer_thread(): CreateThread() failed, errno 0 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 18:08:55 (13852): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 11:57:45 (16344): start_timer_thread(): CreateThread() failed, errno 0 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 11:57:47 (12456): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 22:34:47 (15596): start_timer_thread(): CreateThread() failed, errno 0 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 22:34:49 (3840): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3840, selfPID=7928, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... 12:59:10 (6732): No heartbeat from core client for 30 sec - exiting 12:59:11 (6732): No heartbeat from core client for 30 sec - exiting 12:59:12 (6732): No heartbeat from core client for 30 sec - exiting 12:59:13 (6732): No heartbeat from core client for 30 sec - exiting 12:59:14 (6732): No heartbeat from core client for 30 sec - exiting 12:59:15 (6732): No heartbeat from core client for 30 sec - exiting 12:59:16 (6732): No heartbeat from core client for 30 sec - exiting 12:59:17 (6732): No heartbeat from core client for 30 sec - exiting 12:59:18 (6732): No heartbeat from core client for 30 sec - exiting 12:59:19 (6732): No heartbeat from core client for 30 sec - exiting 12:59:20 (6732): No heartbeat from core client for 30 sec - exiting 12:59:21 (6732): No heartbeat from core client for 30 sec - exiting 12:59:22 (6732): No heartbeat from core client for 30 sec - exiting 12:59:23 (6732): No heartbeat from core client for 30 sec - exiting 12:59:24 (6732): No heartbeat from core client for 30 sec - exiting 12:59:25 (6732): No heartbeat from core client for 30 sec - exiting 12:59:26 (6732): No heartbeat from core client for 30 sec - exiting 12:59:27 (6732): No heartbeat from core client for 30 sec - exiting 12:59:28 (6732): No heartbeat from core client for 30 sec - exiting 12:59:29 (6732): No heartbeat from core client for 30 sec - exiting 12:59:30 (6732): No heartbeat from core client for 30 sec - exiting 12:59:31 (6732): No heartbeat from core client for 30 sec - exiting 12:59:32 (6732): No heartbeat from core client for 30 sec - exiting 12:59:33 (6732): No heartbeat from core client for 30 sec - exiting 12:59:34 (6732): No heartbeat from core client for 30 sec - exiting 12:59:35 (6732): No heartbeat from core client for 30 sec - exiting 12:59:36 (6732): No heartbeat from core client for 30 sec - exiting 12:59:37 (6732): No heartbeat from core client for 30 sec - exiting 12:59:38 (6732): No heartbeat from core client for 30 sec - exiting 12:59:39 (6732): No heartbeat from core client for 30 sec - exiting 12:59:40 (6732): No heartbeat from core client for 30 sec - exiting 12:59:41 (6732): No heartbeat from core client for 30 sec - exiting 12:59:42 (6732): No heartbeat from core client for 30 sec - exiting diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 12:59:42 (12208): start_timer_thread(): CreateThread() failed, errno 0 12:59:43 (6732): No heartbeat from core client for 30 sec - exiting diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 12:59:43 (12396): start_timer_thread(): CreateThread() failed, errno 0 12:59:44 (6732): No heartbeat from core client for 30 sec - exiting 12:59:45 (6732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 12:59:59 (7324): start_timer_thread(): CreateThread() failed, errno 0 diagnostics_init_unhandled_exception_monitor(): Creating hExceptionMonitorThread failed, errno 12 WARNING: BOINC Windows Runtime Debugger has been disabled. 13:00:00 (4412): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 06:49:35 (9740): start_timer_thread(): CreateThread() failed, errno 0 06:49:36 (2096): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 07:34:14 (7868): start_timer_thread(): CreateThread() failed, errno 0 07:34:16 (7900): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 16:43:00 (9960): start_timer_thread(): CreateThread() failed, errno 0 16:43:01 (5544): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 18:30:29 (6192): start_timer_thread(): CreateThread() failed, errno 0 18:30:30 (12296): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 23:14:06 (6956): start_timer_thread(): CreateThread() failed, errno 0 23:14:08 (8308): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 08:18:41 (10804): start_timer_thread(): CreateThread() failed, errno 0 08:18:43 (12184): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 11:44:16 (10536): start_timer_thread(): CreateThread() failed, errno 0 11:44:17 (7484): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 13:45:18 (9956): start_timer_thread(): CreateThread() failed, errno 0 13:45:20 (10360): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 18:32:39 (10168): start_timer_thread(): CreateThread() failed, errno 0 18:32:41 (1424): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 19:17:26 (6352): start_timer_thread(): CreateThread() failed, errno 0 19:17:28 (10220): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 22:44:27 (9344): start_timer_thread(): CreateThread() failed, errno 0 22:44:29 (11828): start_timer_thread(): CreateThread() failed, errno 0 CPDN Monitor - Quit request from BOINC... 23:07:40 (10612): start_timer_thread(): CreateThread() failed, errno 0 23:07:41 (11096): start_timer_thread(): CreateThread() failed, errno 0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 22:02:03 (5948): start_timer_thread(): CreateThread() failed, errno 0 Global Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=5948, iMonCtr=1 Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=12720, selfPID=12720, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=12720, selfPID=1332, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Called boinc_finish </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadam3p_anz_k1oe_201212_12_306_010265319_0_10.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_k1oe_201212_12_306_010265319_0_11.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadam3p_anz_k1oe_201212_12_306_010265319_0_12.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
01 Feb 2016 09:31:30	1333674	19210934	hadam3p_anz_k1oe_201212_12_306_010265319_0	103,979	424,246	4.0801
31 Jan 2016 13:14:04	1333674	19210934	hadam3p_anz_k1oe_201212_12_306_010265319_0	92,459	376,167	4.0685
30 Jan 2016 23:19:33	1333674	19210934	hadam3p_anz_k1oe_201212_12_306_010265319_0	80,939	327,471	4.0459
30 Jan 2016 08:15:47	1333674	19210934	hadam3p_anz_k1oe_201212_12_306_010265319_0	69,419	279,610	4.0279
29 Jan 2016 14:36:24	1333674	19210934	hadam3p_anz_k1oe_201212_12_306_010265319_0	57,899	231,736	4.0024
28 Jan 2016 23:07:40	1333674	19210934	hadam3p_anz_k1oe_201212_12_306_010265319_0	46,379	182,675	3.9387
27 Jan 2016 20:28:12	1333674	19210934	hadam3p_anz_k1oe_201212_12_306_010265319_0	34,859	136,300	3.9100
27 Jan 2016 06:48:31	1333674	19210934	hadam3p_anz_k1oe_201212_12_306_010265319_0	23,339	91,680	3.9282
26 Jan 2016 13:11:56	1333674	19210934	hadam3p_anz_k1oe_201212_12_306_010265319_0	11,819	46,368	3.9232