Task 18641276

Name	hadam3p_pnw_pm3v_2013_1_009976347_0
Workunit	9982705
Created	29 Jun 2015, 17:56:59 UTC
Sent	5 Jul 2015, 4:19:36 UTC
Report deadline	16 Jun 2016, 9:39:36 UTC
Received	5 Nov 2015, 14:03:36 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	197 (0x000000C5) EXIT_TIME_LIMIT_EXCEEDED
Computer ID	1277339
Run time	53 days 15 hours 19 min 29 sec
CPU time	1 days 2 hours 13 min 38 sec
Validate state	Invalid
Credit	1,007.76
Device peak FLOPS	3.09 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Pacific North West v7.27 windows_intelx86
Stderr	<core_client_version>7.6.9</core_client_version> <![CDATA[ <message> exceeded elapsed time limit 4634356.08 (14343090.00G/3.09G) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:46:52 (33648): Can't acquire lockfile (32) - waiting 35s 20:47:28 (33648): Can't acquire lockfile (32) - exiting 20:47:28 (33648): Error: The process cannot access the file because it is being used by another process. (0x20) 05:35:46 (36936): Can't acquire lockfile (32) - waiting 35s 05:36:21 (36936): Can't acquire lockfile (32) - exiting 05:36:21 (36936): Error: The process cannot access the file because it is being used by another process. (0x20) 05:47:05 (38316): Can't acquire lockfile (32) - waiting 35s 05:47:40 (38316): Can't acquire lockfile (32) - exiting 05:47:40 (38316): Error: The process cannot access the file because it is being used by another process. (0x20) 05:58:18 (38204): Can't acquire lockfile (32) - waiting 35s 05:58:54 (38204): Can't acquire lockfile (32) - exiting 05:58:54 (38204): Error: The process cannot access the file because it is being used by another process. (0x20) 15:27:21 (37740): Can't acquire lockfile (32) - waiting 35s 15:27:56 (37740): Can't acquire lockfile (32) - exiting 15:27:56 (37740): Error: The process cannot access the file because it is being used by another process. (0x20) 15:38:04 (36884): Can't acquire lockfile (32) - waiting 35s 15:38:39 (36884): Can't acquire lockfile (32) - exiting 15:38:39 (36884): Error: The process cannot access the file because it is being used by another process. (0x20) 15:48:44 (40004): Can't acquire lockfile (32) - waiting 35s 15:49:20 (40004): Can't acquire lockfile (32) - exiting 15:49:20 (40004): Error: The process cannot access the file because it is being used by another process. (0x20) 15:59:25 (34416): Can't acquire lockfile (32) - waiting 35s 16:00:01 (34416): Can't acquire lockfile (32) - exiting 16:00:01 (34416): Error: The process cannot access the file because it is being used by another process. (0x20) 17:29:43 (38620): Can't acquire lockfile (32) - waiting 35s 17:30:19 (38620): Can't acquire lockfile (32) - exiting 17:30:19 (38620): Error: The process cannot access the file because it is being used by another process. (0x20) 17:40:28 (41132): Can't acquire lockfile (32) - waiting 35s 17:41:04 (41132): Can't acquire lockfile (32) - exiting 17:41:04 (41132): Error: The process cannot access the file because it is being used by another process. (0x20) 17:51:21 (40668): Can't acquire lockfile (32) - waiting 35s 17:51:56 (40668): Can't acquire lockfile (32) - exiting 17:51:56 (40668): Error: The process cannot access the file because it is being used by another process. (0x20) 18:02:28 (37752): Can't acquire lockfile (32) - waiting 35s 18:03:03 (37752): Can't acquire lockfile (32) - exiting 18:03:03 (37752): Error: The process cannot access the file because it is being used by another process. (0x20) 05:24:19 (44260): Can't acquire lockfile (32) - waiting 35s 05:24:55 (44260): Can't acquire lockfile (32) - exiting 05:24:55 (44260): Error: The process cannot access the file because it is being used by another process. (0x20) 05:34:58 (43284): Can't acquire lockfile (32) - waiting 35s 05:35:34 (43284): Can't acquire lockfile (32) - exiting 05:35:34 (43284): Error: The process cannot access the file because it is being used by another process. (0x20) 05:45:36 (43956): Can't acquire lockfile (32) - waiting 35s 05:46:12 (43956): Can't acquire lockfile (32) - exiting 05:46:12 (43956): Error: The process cannot access the file because it is being used by another process. (0x20) 05:56:13 (42956): Can't acquire lockfile (32) - waiting 35s 05:56:48 (42956): Can't acquire lockfile (32) - exiting 05:56:48 (42956): Error: The process cannot access the file because it is being used by another process. (0x20) 14:34:47 (393604): Can't acquire lockfile (32) - waiting 35s 14:35:22 (393604): Can't acquire lockfile (32) - exiting 14:35:22 (393604): Error: The process cannot access the file because it is being used by another process. (0x20) 14:45:38 (421856): Can't acquire lockfile (32) - waiting 35s 14:46:14 (421856): Can't acquire lockfile (32) - exiting 14:46:14 (421856): Error: The process cannot access the file because it is being used by another process. (0x20) 14:56:18 (282600): Can't acquire lockfile (32) - waiting 35s 14:56:54 (282600): Can't acquire lockfile (32) - exiting 14:56:54 (282600): Error: The process cannot access the file because it is being used by another process. (0x20) 15:06:59 (420984): Can't acquire lockfile (32) - waiting 35s 15:07:34 (420984): Can't acquire lockfile (32) - exiting 15:07:34 (420984): Error: The process cannot access the file because it is being used by another process. (0x20) 15:17:39 (282616): Can't acquire lockfile (32) - waiting 35s 15:18:15 (282616): Can't acquire lockfile (32) - exiting 15:18:15 (282616): Error: The process cannot access the file because it is being used by another process. (0x20) 15:28:19 (406260): Can't acquire lockfile (32) - waiting 35s 15:28:54 (406260): Can't acquire lockfile (32) - exiting 15:28:54 (406260): Error: The process cannot access the file because it is being used by another process. (0x20) 15:38:58 (421180): Can't acquire lockfile (32) - waiting 35s 15:39:33 (421180): Can't acquire lockfile (32) - exiting 15:39:33 (421180): Error: The process cannot access the file because it is being used by another process. (0x20) 15:49:39 (421368): Can't acquire lockfile (32) - waiting 35s 15:50:15 (421368): Can't acquire lockfile (32) - exiting 15:50:15 (421368): Error: The process cannot access the file because it is being used by another process. (0x20) Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6988, selfPID=7348, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Abort request from BOINC... 03:10:11 (4796): called boinc_finish(10) </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
05 Nov 2015 14:17:39	1277339	18641276	hadam3p_pnw_pm3v_2013_1_009976347_0	46,379	92,940	2.0039
24 Aug 2015 14:26:21	1277339	18641276	hadam3p_pnw_pm3v_2013_1_009976347_0	34,859	68,140	1.9547