Task 13271598

Name	hadcm3n_yn3f_1940_40_007414396_1
Workunit	7612026
Created	17 Aug 2011, 12:50:40 UTC
Sent	17 Aug 2011, 13:15:09 UTC
Report deadline	16 Nov 2011, 20:42:20 UTC
Received	29 Aug 2011, 18:48:13 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1092189
Run time	2 days 14 hours 34 min 14 sec
CPU time	2 days 8 hours 57 min 51 sec
Validate state	Invalid
Credit	1,555.20
Device peak FLOPS	2.96 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> Il dispositivo non riconosce il comando. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... forrtl: Accesso negato. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5116, iMonCtr=1 Model crash detected, will try to restart... forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6436, selfPID=6436, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1056, selfPID=1056, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6796, selfPID=6796, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5960, selfPID=5960, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10228, selfPID=10228, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=12200, selfPID=12200, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7508, selfPID=7508, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8040, selfPID=8040, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7484, selfPID=7484, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=17288, selfPID=17288, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=19320, selfPID=19320, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=21348, selfPID=21348, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4884, selfPID=4884, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2476, selfPID=2476, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7476, selfPID=7476, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6544, selfPID=6544, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7280, selfPID=7280, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8604, selfPID=8604, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9156, selfPID=9156, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4112, selfPID=4112, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9344, selfPID=9344, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5264, selfPID=5264, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11904, selfPID=11904, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8928, selfPID=8928, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=14072, selfPID=14072, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9980, selfPID=9980, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9396, selfPID=9396, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8336, selfPID=8336, iMonCtr=1 CPDN Monitor - Quit request from BOINC... 23:52:18 (11120): Can't acquire lockfile (32) - waiting 35s No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=12064, selfPID=12064, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10256, selfPID=10256, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10552, selfPID=10552, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=12672, selfPID=12672, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=13108, selfPID=13108, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9556, selfPID=9556, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2960, selfPID=2960, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=12648, selfPID=12648, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7444, selfPID=7444, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=14900, selfPID=14900, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6860, selfPID=6860, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4696, selfPID=4696, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1596, selfPID=1596, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1820, selfPID=1820, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=696, selfPID=696, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6176, selfPID=6176, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1648, selfPID=1648, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9160, selfPID=9160, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=19128, selfPID=19128, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=18572, selfPID=18572, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=16552, selfPID=16552, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=19188, selfPID=19188, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7684, selfPID=7684, iMonCtr=1 CPDN Monitor - Quit request from BOINC... 00:20:33 (21680): Can't acquire lockfile (32) - waiting 35s No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5664, selfPID=5664, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=24108, selfPID=24108, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=22480, selfPID=22480, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 CPDN Monitor - Quit request from BOINC... forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=30288, selfPID=30288, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8376, selfPID=8376, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=15536, selfPID=15536, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=16604, selfPID=16604, iMonCtr=1 CPDN Monitor - Quit request from BOINC... 15:54:16 (6060): Can't acquire lockfile (32) - waiting 35s No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4436, selfPID=4436, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4020, selfPID=4020, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=21204, selfPID=21204, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=23964, selfPID=23964, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8872, selfPID=8872, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8592, selfPID=8592, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=15460, selfPID=15460, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11384, selfPID=11384, iMonCtr=1 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=19160, selfPID=19160, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=21756, selfPID=21756, iMonCtr=1 CPDN Monitor - Quit request from BOINC... 21:54:11 (29632): Can't acquire lockfile (32) - waiting 35s No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=23720, selfPID=23720, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=27716, selfPID=27716, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=25968, selfPID=25968, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10420, selfPID=10420, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2560, selfPID=2560, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=25148, selfPID=25148, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11452, selfPID=11452, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5824, selfPID=5824, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5760, selfPID=5760, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5340, selfPID=5340, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6156, selfPID=6156, iMonCtr=1 10:45:07 (4816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:45:09 (4816): No heartbeat from core client for 30 sec - exiting 10:45:10 (4816): No heartbeat from core client for 30 sec - exiting 10:45:11 (4816): No heartbeat from core client for 30 sec - exiting 10:45:12 (4816): No heartbeat from core client for 30 sec - exiting 10:45:13 (4816): No heartbeat from core client for 30 sec - exiting 10:45:14 (4816): No heartbeat from core client for 30 sec - exiting 10:45:15 (4816): No heartbeat from core client for 30 sec - exiting 10:45:16 (4816): No heartbeat from core client for 30 sec - exiting 10:45:17 (4816): No heartbeat from core client for 30 sec - exiting Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5060, selfPID=5060, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3552, selfPID=3552, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2428, selfPID=2428, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=776, selfPID=776, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6924, selfPID=6924, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5684, selfPID=5684, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4676, selfPID=4676, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5364, selfPID=5364, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5620, selfPID=5620, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3584, selfPID=3584, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9260, selfPID=9260, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2728, selfPID=2728, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=28116, selfPID=28116, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=45612, selfPID=45612, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=53268, selfPID=53268, iMonCtr=1 CPDN Monitor - Quit request from BOINC... 22:02:35 (28004): Can't acquire lockfile (32) - waiting 35s No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=13724, selfPID=13724, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=25772, selfPID=25772, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=46740, selfPID=46740, iMonCtr=1 CPDN Monitor - Quit request from BOINC... 22:42:30 (48812): Can't acquire lockfile (32) - waiting 35s No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=39420, selfPID=39420, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=60200, selfPID=60200, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=55692, selfPID=55692, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8848, selfPID=8848, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=53072, selfPID=53072, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=49796, selfPID=49796, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=43300, selfPID=43300, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=23960, selfPID=23960, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: Accesso negato. Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8796, selfPID=8796, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4676, selfPID=4676, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8872, selfPID=8872, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5708, selfPID=5708, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5972, selfPID=5972, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4400, selfPID=4400, iMonCtr=1 CPDN Monitor - Quit request from BOINC... 20:01:18 (5096): Can't acquire lockfile (32) - waiting 35s Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5564, selfPID=5564, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7572, selfPID=7572, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8616, selfPID=8616, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=14584, selfPID=14584, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=14564, selfPID=14564, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=21208, selfPID=21208, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=22472, selfPID=22472, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9672, selfPID=9672, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5712, selfPID=5712, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8364, selfPID=8364, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=13704, selfPID=13704, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7520, selfPID=7520, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=14724, selfPID=14724, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6792, selfPID=6792, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8460, selfPID=8460, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8212, selfPID=8212, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=376, selfPID=376, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8992, selfPID=8992, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7520, selfPID=7520, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7708, selfPID=7708, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6688, selfPID=6688, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6112, selfPID=6112, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5812, selfPID=5812, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5132, selfPID=5132, iMonCtr=1 CPDN Monitor - Quit request from BOINC... 15:22:00 (5276): Can't acquire lockfile (32) - waiting 35s No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4240, selfPID=4240, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7900, selfPID=7900, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8484, selfPID=8484, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6444, selfPID=6444, iMonCtr=1 forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8080, selfPID=8080, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8420, selfPID=8420, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4292, selfPID=4292, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5188, selfPID=5188, iMonCtr=1 CPDN Monitor - Quit request from BOINC... 21:03:42 (8092): Can't acquire lockfile (32) - waiting 35s No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1844, selfPID=1844, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=9392, selfPID=9392, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=10400, selfPID=10400, iMonCtr=1 21:17:48 (4664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:17:49 (4664): No heartbeat from core client for 30 sec - exiting 21:17:50 (4664): No heartbeat from core client for 30 sec - exiting 21:17:51 (4664): No heartbeat from core client for 30 sec - exiting 21:17:52 (4664): No heartbeat from core client for 30 sec - exiting 21:17:53 (4664): No heartbeat from core client for 30 sec - exiting 21:17:54 (4664): No heartbeat from core client for 30 sec - exiting 21:17:56 (4664): No heartbeat from core client for 30 sec - exiting 21:17:57 (4664): No heartbeat from core client for 30 sec - exiting 21:17:58 (4664): No heartbeat from core client for 30 sec - exiting 21:17:59 (4664): No heartbeat from core client for 30 sec - exiting Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11244, selfPID=11244, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=11364, selfPID=11364, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=12508, selfPID=12508, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=13708, selfPID=13708, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6016, selfPID=6016, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5196, selfPID=5196, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... 11:15:21 (4136): Can't acquire lockfile (32) - waiting 35s No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=896, selfPID=896, iMonCtr=1 CPDN Monitor - Quit request from BOINC... 11:17:09 (5044): Can't acquire lockfile (32) - waiting 35s No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6820, selfPID=6820, iMonCtr=1 CPDN Monitor - Quit request from BOINC... 11:20:10 (7140): Can't acquire lockfile (32) - waiting 35s No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7004, selfPID=7004, iMonCtr=1 CPDN Monitor - Quit request from BOINC... 11:21:10 (6336): Can't acquire lockfile (32) - waiting 35s No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6776, selfPID=6776, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6368, selfPID=6368, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5340, selfPID=5340, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6576, selfPID=6576, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6576, selfPID=6576, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5044, selfPID=5044, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6100, selfPID=6100, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3660, selfPID=3660, iMonCtr=1 CPDN Monitor - Quit request from BOINC... 12:53:23 (6200): Can't acquire lockfile (32) - waiting 35s No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6084, selfPID=6084, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6216, selfPID=6216, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7000, selfPID=7000, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5548, selfPID=5548, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1180, selfPID=1180, iMonCtr=1 CPDN Monitor - Quit request from BOINC... 12:58:35 (5248): Can't acquire lockfile (32) - waiting 35s Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4884, selfPID=4884, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4736, selfPID=4736, iMonCtr=1 CPDN Monitor - Quit request from BOINC... 13:02:15 (6440): Can't acquire lockfile (32) - waiting 35s No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7024, selfPID=7024, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6268, selfPID=6268, iMonCtr=1 Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2260, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2260, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2260, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2260, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2260, iMonCtr=1 Model crash detected, will try to restart... 16:10:52 (2260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:10:54 (2260): No heartbeat from core client for 30 sec - exiting 16:10:55 (2260): No heartbeat from core client for 30 sec - exiting 16:10:56 (2260): No heartbeat from core client for 30 sec - exiting 16:10:57 (2260): No heartbeat from core client for 30 sec - exiting 16:10:58 (2260): No heartbeat from core client for 30 sec - exiting 16:10:59 (2260): No heartbeat from core client for 30 sec - exiting 16:11:00 (2260): No heartbeat from core client for 30 sec - exiting 16:11:01 (2260): No heartbeat from core client for 30 sec - exiting 16:11:02 (2260): No heartbeat from core client for 30 sec - exiting 16:11:03 (2260): No heartbeat from core client for 30 sec - exiting Signal 22 received, exiting... Called boinc_finish forrtl: Accesso negato. CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish forrtl: Accesso negato. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5440, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
26 Aug 2011 10:37:00	1092189	13271598	hadcm3n_yn3f_1940_40_007414396_1	129,600	173,122	1.3358
23 Aug 2011 23:19:20	1092189	13271598	hadcm3n_yn3f_1940_40_007414396_1	103,680	138,220	1.3331
23 Aug 2011 09:00:12	1092189	13271598	hadcm3n_yn3f_1940_40_007414396_1	77,760	103,507	1.3311
21 Aug 2011 14:36:21	1092189	13271598	hadcm3n_yn3f_1940_40_007414396_1	51,840	69,111	1.3332
18 Aug 2011 16:49:43	1092189	13271598	hadcm3n_yn3f_1940_40_007414396_1	25,920	34,510	1.3314