Task 18650669

Name	hadam3p_pnw_ptlt_2013_1_009985669_0
Workunit	9992027
Created	29 Jun 2015, 19:09:56 UTC
Sent	5 Jul 2015, 17:45:32 UTC
Report deadline	16 Jun 2016, 23:05:32 UTC
Received	28 Jan 2016, 22:04:11 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-226 (0xFFFFFF1E) ERR_TOO_MANY_EXITS
Computer ID	1272151
Run time	13 days 20 hours 0 min 39 sec
CPU time	12 days 0 hours 11 min 57 sec
Validate state	Invalid
Credit	4,011.55
Device peak FLOPS	1.94 GFLOPS
Application version	UK Met Office HadAM3P-HadRM3P Pacific North West v7.27 windows_intelx86
Stderr	<core_client_version>7.4.42</core_client_version> <![CDATA[ <message> too many exit(0)s </message> <stderr_txt> CreateFile() error 32 when trying set file time CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=13416, selfPID=4724, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11460, selfPID=11460, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:56:48 (7756): BOINC client no longer exists - exiting 15:56:48 (7756): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... GCPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4448, selfPID=4448, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3860, selfPID=3860, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=12332, selfPID=12332, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=15580, selfPID=15580, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6960, selfPID=6960, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=17188, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... Gontroller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=984, iMonCtr=2 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:41:21 (13312): No heartbeat from client for 30 sec - exiting 16:41:21 (13312): timer handler: client dead, exiting 16:41:22 (13312): No heartbeat from client for 30 sec - exiting 16:41:22 (13312): timer handler: client dead, exiting 16:41:23 (13312): No heartbeat from client for 30 sec - exiting 16:41:23 (13312): timer handler: client dead, exiting 16:41:24 (13312): No heartbeat from client for 30 sec - exiting 16:41:24 (13312): timer handler: client dead, exiting 16:41:25 (13312): No heartbeat from client for 30 sec - exiting 16:41:25 (13312): timer handler: client dead, exiting 16:41:26 (13312): No heartbeat from client for 30 sec - exiting 16:41:26 (13312): timer handler: client dead, exiting 16:41:27 (13312): No heartbeat from client for 30 sec - exiting 16:41:27 (13312): timer handler: client dead, exiting 16:41:28 (13312): No heartbeat from client for 30 sec - exiting 16:41:28 (13312): timer handler: client dead, exiting 16:41:29 (13312): No heartbeat from client for 30 sec - exiting 16:41:29 (13312): timer handler: client dead, exiting 16:41:30 (13312): No heartbeat from client for 30 sec - exiting 16:41:30 (13312): timer handler: client dead, exiting 16:41:31 (13312): No heartbeat from client for 30 sec - exiting 16:41:31 (13312): timer handler: client dead, exiting 16:41:32 (13312): No heartbeat from client for 30 sec - exiting 16:41:32 (13312): timer handler: client dead, exiting 16:41:33 (13312): No heartbeat from client for 30 sec - exiting 16:41:33 (13312): timer handler: client dead, exiting 16:41:34 (13312): No heartbeat from client for 30 sec - exiting 16:41:34 (13312): timer handler: client dead, exiting 16:41:36 (13312): No heartbeat from client for 30 sec - exiting 16:41:36 (13312): timer handler: client dead, exiting 16:41:37 (13312): No heartbeat from client for 30 sec - exiting 16:41:37 (13312): timer handler: client dead, exiting 16:41:38 (13312): No heartbeat from client for 30 sec - exiting 16:41:38 (13312): timer handler: client dead, exiting 16:41:39 (13312): No heartbeat from client for 30 sec - exiting 16:41:39 (13312): timer handler: client dead, exiting 16:41:40 (13312): No heartbeat from client for 30 sec - exiting 16:41:40 (13312): timer handler: client dead, exiting 16:41:41 (13312): No heartbeat from client for 30 sec - exiting 16:41:41 (13312): timer handler: client dead, exiting 16:41:42 (13312): No heartbeat from client for 30 sec - exiting 16:41:42 (13312): timer handler: client dead, exiting 16:41:43 (13312): No heartbeat from client for 30 sec - exiting 16:41:43 (13312): timer handler: client dead, exiting 16:41:44 (13312): No heartbeat from client for 30 sec - exiting 16:41:44 (13312): timer handler: client dead, exiting 16:41:45 (13312): No heartbeat from client for 30 sec - exiting 16:41:45 (13312): timer handler: client dead, exiting 16:41:46 (13312): No heartbeat from client for 30 sec - exiting 16:41:46 (13312): timer handler: client dead, exiting 16:41:47 (13312): No heartbeat from client for 30 sec - exiting 16:41:47 (13312): timer handler: client dead, exiting 16:41:48 (13312): No heartbeat from client for 30 sec - exiting 16:41:48 (13312): timer handler: client dead, exiting 16:41:49 (13312): No heartbeat from client for 30 sec - exiting 16:41:49 (13312): timer handler: client dead, exiting 16:41:50 (13312): No heartbeat from client for 30 sec - exiting 16:41:50 (13312): timer handler: client dead, exiting 16:41:51 (13312): No heartbeat from client for 30 sec - exiting 16:41:51 (13312): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=13564, selfPID=13564, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... GCPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7392, selfPID=7392, iMonCtr=2 ConCPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6040, selfPID=6040, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7896, selfPID=7896, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9212, selfPID=9212, iMonCtr=2 CPDN Monitor - Quit request from BOINC... 17:37:56 (10052): BOINC client no longer exists - exiting 17:37:56 (10052): timer handler: client dead, exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7044, selfPID=7044, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4116, selfPID=4116, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=12784, selfPID=12300, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=272, selfPID=272, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=392, selfPID=5628, iMonCtr=1 Model crash detected, will try to restart... Leaving CPDN_Main::Monitor... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=12072, selfPID=12072, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=444, selfPID=444, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... GController:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2864, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=20412, selfPID=20412, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Global Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=18140, iMonCtr=2 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=14532, iMonCtr=2 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=21312, selfPID=21312, iMonCtr=2 CPDN Monitor - Quit request from BOINC... Regional Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=23872, selfPID=23872, iMonCtr=2 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... RCPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
27 Jan 2016 20:18:10	1272151	18650669	hadam3p_pnw_ptlt_2013_1_009985669_0	184,619	1,017,279	5.5102
14 Jan 2016 21:34:34	1272151	18650669	hadam3p_pnw_ptlt_2013_1_009985669_0	173,099	953,058	5.5059
21 Dec 2015 12:44:17	1272151	18650669	hadam3p_pnw_ptlt_2013_1_009985669_0	161,579	890,061	5.5085
11 Dec 2015 09:57:23	1272151	18650669	hadam3p_pnw_ptlt_2013_1_009985669_0	150,059	826,932	5.5107
02 Dec 2015 11:13:47	1272151	18650669	hadam3p_pnw_ptlt_2013_1_009985669_0	138,539	764,677	5.5196
12 Nov 2015 16:37:03	1272151	18650669	hadam3p_pnw_ptlt_2013_1_009985669_0	127,019	702,261	5.5288
05 Nov 2015 14:13:46	1272151	18650669	hadam3p_pnw_ptlt_2013_1_009985669_0	115,499	637,391	5.5186
26 Oct 2015 17:48:56	1272151	18650669	hadam3p_pnw_ptlt_2013_1_009985669_0	103,979	574,081	5.5211
26 Oct 2015 17:48:56	1272151	18650669	hadam3p_pnw_ptlt_2013_1_009985669_0	92,459	509,945	5.5154
04 Oct 2015 17:32:37	1272151	18650669	hadam3p_pnw_ptlt_2013_1_009985669_0	80,939	447,287	5.5262
08 Sep 2015 08:54:43	1272151	18650669	hadam3p_pnw_ptlt_2013_1_009985669_0	69,419	383,285	5.5213
24 Aug 2015 12:55:53	1272151	18650669	hadam3p_pnw_ptlt_2013_1_009985669_0	57,899	319,230	5.5136
21 Aug 2015 08:29:01	1272151	18650669	hadam3p_pnw_ptlt_2013_1_009985669_0	46,379	256,884	5.5388
04 Aug 2015 19:08:19	1272151	18650669	hadam3p_pnw_ptlt_2013_1_009985669_0	34,859	191,555	5.4951
13 Jul 2015 19:10:45	1272151	18650669	hadam3p_pnw_ptlt_2013_1_009985669_0	23,339	128,283	5.4965
09 Jul 2015 15:06:00	1272151	18650669	hadam3p_pnw_ptlt_2013_1_009985669_0	11,819	65,853	5.5718