Task 12017730

Name	hadcm3igeo_w1cw_2000_80_06761297_2
Workunit	6964613
Created	12 Nov 2010, 15:32:15 UTC
Sent	13 Nov 2010, 22:52:53 UTC
Report deadline	14 May 2012, 19:36:02 UTC
Received	8 Jan 2011, 19:08:34 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1096714
Run time	50 days 19 hours 54 min 57 sec
CPU time	49 days 9 hours 27 min 38 sec
Validate state	Invalid
Credit	24,820.99
Device peak FLOPS	2.78 GFLOPS
Application version	HadCM3L Coupled Model Experiment Optimised File I/O v6.04 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> scan: 09rj_00_2000_atmos.gz scan: 09rj_00_2000_ocean.gz scan: 1042_flux_corr.anc.gz scan: boinc_lockfile scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fnrohdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w1cw_2000_80_06761297.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volcanic_geocontrol.gz scan: 09rj_00_2000_atmos.gz scan: 09rj_00_2000_ocean.gz scan: 1042_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fnrohdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w1cw_2000_80_06761297.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volcanic_geocontrol.gz No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... scan: 09rj_00_2000_atmos.gz scan: 09rj_00_2000_ocean.gz scan: 1042_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fnrohdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w1cw_2000_80_06761297.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volcanic_geocontrol.gz scan: 09rj_00_2000_atmos.gz scan: 09rj_00_2000_ocean.gz scan: 1042_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: boinc_ufs_cpdnout1.zip scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fnrohdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w1cw_2000_80_06761297.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volcanic_geocontrol.gz CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4876, iMonCtr=1 Model crash detected, will try to restart... scan: 09rj_00_2000_atmos.gz scan: 09rj_00_2000_ocean.gz scan: 1042_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: boinc_ufs_cpdnout1.zip scan: boinc_ufs_cpdnout2.zip scan: boinc_ufs_cpdnout3.zip scan: boinc_ufs_cpdnout4.zip scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fnrohdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w1cw_2000_80_06761297.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volcanic_geocontrol.gz No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... scan: 09rj_00_2000_atmos.gz scan: 09rj_00_2000_ocean.gz scan: 1042_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: boinc_ufs_cpdnout1.zip scan: boinc_ufs_cpdnout2.zip scan: boinc_ufs_cpdnout3.zip scan: boinc_ufs_cpdnout4.zip scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fnrohdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w1cw_2000_80_06761297.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volcanic_geocontrol.gz scan: 09rj_00_2000_atmos.gz scan: 09rj_00_2000_ocean.gz scan: 1042_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: boinc_ufs_cpdnout1.zip scan: boinc_ufs_cpdnout2.zip scan: boinc_ufs_cpdnout3.zip scan: boinc_ufs_cpdnout4.zip scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fnrohdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w1cw_2000_80_06761297.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volcanic_geocontrol.gz No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... scan: 09rj_00_2000_atmos.gz scan: 09rj_00_2000_ocean.gz scan: 1042_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: boinc_ufs_cpdnout1.zip scan: boinc_ufs_cpdnout2.zip scan: boinc_ufs_cpdnout3.zip scan: boinc_ufs_cpdnout4.zip scan: boinc_ufs_cpdnout5.zip scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: fnrohdck_0208_nickfluxcorr.anc.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w1cw_2000_80_06761297.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volcanic_geocontrol.gz Model crashed: Model crashed: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2744, iMonCtr=1 Model crash detected, will try to restart... CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2744, iMonCtr=1 Model crash detected, will try to restart... CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2744, iMonCtr=1 Model crash detected, will try to restart... CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2744, iMonCtr=1 Model crash detected, will try to restart... CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2744, iMonCtr=1 Model crash detected, will try to restart... CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2744, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
08 Jan 2011 18:42:45	1096714	12017730	hadcm3igeo_w1cw_2000_80_06761297_2	1,969,920	4,263,229	2.1642
08 Jan 2011 12:21:32	1096714	12017730	hadcm3igeo_w1cw_2000_80_06761297_2	1,944,000	4,206,208	2.1637
07 Jan 2011 11:39:26	1096714	12017730	hadcm3igeo_w1cw_2000_80_06761297_2	1,918,080	4,148,951	2.1631
06 Jan 2011 18:36:59	1096714	12017730	hadcm3igeo_w1cw_2000_80_06761297_2	1,892,160	4,092,241	2.1627
06 Jan 2011 10:44:39	1096714	12017730	hadcm3igeo_w1cw_2000_80_06761297_2	1,866,240	4,035,800	2.1625
05 Jan 2011 10:22:23	1096714	12017730	hadcm3igeo_w1cw_2000_80_06761297_2	1,840,320	3,978,920	2.1621
04 Jan 2011 18:46:29	1096714	12017730	hadcm3igeo_w1cw_2000_80_06761297_2	1,814,400	3,923,037	2.1622
04 Jan 2011 02:51:58	1096714	12017730	hadcm3igeo_w1cw_2000_80_06761297_2	1,788,480	3,866,610	2.1620
03 Jan 2011 11:14:45	1096714	12017730	hadcm3igeo_w1cw_2000_80_06761297_2	1,762,560	3,810,726	2.1620
02 Jan 2011 20:04:08	1096714	12017730	hadcm3igeo_w1cw_2000_80_06761297_2	1,736,640	3,754,001	2.1616
02 Jan 2011 03:50:08	1096714	12017730	hadcm3igeo_w1cw_2000_80_06761297_2	1,710,720	3,698,373	2.1619
01 Jan 2011 12:05:16	1096714	12017730	hadcm3igeo_w1cw_2000_80_06761297_2	1,684,800	3,642,475	2.1620
31 Dec 2010 20:09:53	1096714	12017730	hadcm3igeo_w1cw_2000_80_06761297_2	1,658,880	3,586,353	2.1619
31 Dec 2010 04:26:50	1096714	12017730	hadcm3igeo_w1cw_2000_80_06761297_2	1,632,960	3,530,555	2.1621
30 Dec 2010 12:37:35	1096714	12017730	hadcm3igeo_w1cw_2000_80_06761297_2	1,607,040	3,474,323	2.1619
29 Dec 2010 20:46:25	1096714	12017730	hadcm3igeo_w1cw_2000_80_06761297_2	1,581,120	3,417,832	2.1617
29 Dec 2010 04:56:56	1096714	12017730	hadcm3igeo_w1cw_2000_80_06761297_2	1,555,200	3,361,595	2.1615
28 Dec 2010 15:20:09	1096714	12017730	hadcm3igeo_w1cw_2000_80_06761297_2	1,529,280	3,304,537	2.1608
27 Dec 2010 19:43:13	1096714	12017730	hadcm3igeo_w1cw_2000_80_06761297_2	1,503,360	3,247,276	2.1600
27 Dec 2010 12:35:07	1096714	12017730	hadcm3igeo_w1cw_2000_80_06761297_2	1,477,440	3,189,313	2.1587