Task 12011920

Name	hadcm3igeo_w295_2000_80_06760136_2
Workunit	6963452
Created	12 Nov 2010, 15:30:03 UTC
Sent	16 Nov 2010, 11:17:45 UTC
Report deadline	17 May 2012, 8:00:54 UTC
Received	24 Nov 2010, 9:38:20 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1103240
Run time	6 days 3 hours 54 min 16 sec
CPU time	5 days 17 hours 51 min 5 sec
Validate state	Invalid
Credit	3,265.92
Device peak FLOPS	2.47 GFLOPS
Application version	HadCM3L Coupled Model Experiment Optimised File I/O v6.04 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> scan: 0g5r_03_2000_atmos.gz scan: 0g5r_03_2000_ocean.gz scan: 1029_flux_corr.anc.gz scan: boinc_lockfile scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w295_2000_80_06760136.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: he16hdck_0208_nickfluxcorr.anc.gz scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volc_geo2_7.gz scan: 0g5r_03_2000_atmos.gz scan: 0g5r_03_2000_ocean.gz scan: 1029_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w295_2000_80_06760136.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: he16hdck_0208_nickfluxcorr.anc.gz scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volc_geo2_7.gz scan: 0g5r_03_2000_atmos.gz scan: 0g5r_03_2000_ocean.gz scan: 1029_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w295_2000_80_06760136.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: he16hdck_0208_nickfluxcorr.anc.gz scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volc_geo2_7.gz scan: 0g5r_03_2000_atmos.gz scan: 0g5r_03_2000_ocean.gz scan: 1029_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w295_2000_80_06760136.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: he16hdck_0208_nickfluxcorr.anc.gz scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volc_geo2_7.gz scan: 0g5r_03_2000_atmos.gz scan: 0g5r_03_2000_ocean.gz scan: 1029_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w295_2000_80_06760136.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: he16hdck_0208_nickfluxcorr.anc.gz scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volc_geo2_7.gz CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5016, iMonCtr=1 Model crash detected, will try to restart... CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5016, iMonCtr=1 Model crash detected, will try to restart... scan: 0g5r_03_2000_atmos.gz scan: 0g5r_03_2000_ocean.gz scan: 1029_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w295_2000_80_06760136.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: he16hdck_0208_nickfluxcorr.anc.gz scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: stderrgfx.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volc_geo2_7.gz scan: 0g5r_03_2000_atmos.gz scan: 0g5r_03_2000_ocean.gz scan: 1029_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w295_2000_80_06760136.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: he16hdck_0208_nickfluxcorr.anc.gz scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: stderrgfx.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volc_geo2_7.gz scan: 0g5r_03_2000_atmos.gz scan: 0g5r_03_2000_ocean.gz scan: 1029_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w295_2000_80_06760136.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: he16hdck_0208_nickfluxcorr.anc.gz scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: stderrgfx.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volc_geo2_7.gz scan: 0g5r_03_2000_atmos.gz scan: 0g5r_03_2000_ocean.gz scan: 1029_flux_corr.anc.gz scan: boinc_lockfile scan: boinc_task_state.xml scan: cpdnout1.zip scan: cpdnout2.zip scan: cpdnout3.zip scan: cpdnout4.zip scan: cpdnout5.zip scan: cpdnout6.zip scan: cpdnout7.zip scan: cpdnout8.zip scan: DMSNH3SO2A1B_19182082N.gz scan: DMSNH3SO2A1B_19182082N.mod.gz scan: ghg_a1b.gz scan: graphics_app scan: hadcm3igeo_w295_2000_80_06760136.zip scan: hadcm3trans_6.04_windows_intelx86.exe scan: hadcm3trans_data_6.04_windows_intelx86.zip scan: hadcm3trans_se_6.04_windows_intelx86.zip scan: hadcm3trans_um_6.04_windows_intelx86.zip scan: he16hdck_0208_nickfluxcorr.anc.gz scan: init_data.xml scan: NAT_VOLC.gz scan: ozone_hadcm3_1919_2082.gz scan: ozone_hadcm3_1919_2082.mod.gz scan: solar_v01.gz scan: spec3a_lw_3_asol2c_hadcm3.gz scan: spec3a_sw_3_asol2b_hadcm3.gz scan: stderr.txt scan: stderrgfx.txt scan: SULPC_OXIDANTS_19_A2_1990.gz scan: SULPC_OXIDANTS_19_A2_1990.mod.gz scan: volc_geo2_7.gz error: cannot delete old hadcm3igeo_w295_2000_80_06760136/jobs/yafbg.PRESM_O error: cannot delete old hadcm3igeo_w295_2000_80_06760136/datain/ancil/basin.index error: cannot delete old hadcm3igeo_w295_2000_80_06760136/jobs/yafbg.namelists CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3igeo_w295_2000_80_06760136/dataout/atmos_restart.day cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3igeo_w295_2000_80_06760136/dataout/ocean_restart.day Model crashed: cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3igeo_w295_2000_80_06760136/dataout/atmos_restart.day cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3igeo_w295_2000_80_06760136/dataout/ocean_restart.day Model crashed: cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3igeo_w295_2000_80_06760136/dataout/atmos_restart.day cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3igeo_w295_2000_80_06760136/dataout/ocean_restart.day Model crashed: cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3igeo_w295_2000_80_06760136/dataout/atmos_restart.day cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3igeo_w295_2000_80_06760136/dataout/ocean_restart.day Model crashed: cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3igeo_w295_2000_80_06760136/dataout/atmos_restart.day cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3igeo_w295_2000_80_06760136/dataout/ocean_restart.day Model crashed: cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3igeo_w295_2000_80_06760136/dataout/atmos_restart.day cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3igeo_w295_2000_80_06760136/dataout/ocean_restart.day Model crashed: Sorry, too many model crashes! :-( called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
24 Nov 2010 09:43:12	1103240	12011920	hadcm3igeo_w295_2000_80_06760136_2	259,200	496,265	1.9146
23 Nov 2010 15:17:28	1103240	12011920	hadcm3igeo_w295_2000_80_06760136_2	233,280	446,603	1.9145
22 Nov 2010 21:28:24	1103240	12011920	hadcm3igeo_w295_2000_80_06760136_2	207,360	397,314	1.9161
22 Nov 2010 06:08:05	1103240	12011920	hadcm3igeo_w295_2000_80_06760136_2	181,440	348,182	1.9190
21 Nov 2010 07:08:07	1103240	12011920	hadcm3igeo_w295_2000_80_06760136_2	155,520	298,099	1.9168
20 Nov 2010 12:07:27	1103240	12011920	hadcm3igeo_w295_2000_80_06760136_2	129,600	247,812	1.9121
19 Nov 2010 16:13:43	1103240	12011920	hadcm3igeo_w295_2000_80_06760136_2	103,680	198,241	1.9120
18 Nov 2010 22:10:48	1103240	12011920	hadcm3igeo_w295_2000_80_06760136_2	77,760	149,074	1.9171
18 Nov 2010 06:06:08	1103240	12011920	hadcm3igeo_w295_2000_80_06760136_2	51,840	99,517	1.9197
17 Nov 2010 08:55:06	1103240	12011920	hadcm3igeo_w295_2000_80_06760136_2	25,920	49,776	1.9204