climateprediction.net home page
Task 22314142

Task 22314142

Name oifs_43r3_bl_a3cx_2016092300_15_991_12211140_1
Workunit 12211140
Created 17 Feb 2023, 2:39:42 UTC
Sent 17 Feb 2023, 2:40:24 UTC
Report deadline 18 Apr 2023, 2:40:24 UTC
Received 17 Feb 2023, 15:42:49 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1517479
Run time 11 hours 47 min 52 sec
CPU time 11 hours 38 min 15 sec
Validate state Valid
Credit 1,813.14
Device peak FLOPS 4.36 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.11
x86_64-pc-linux-gnu
Peak working set size 5,076.09 MB
Peak swap size 5,930.78 MB
Peak disk usage 1,294.66 MB
Stderr
<core_client_version>7.9.3</core_client_version>
<![CDATA[
<stderr_txt>
000864
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+000876
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+000876
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+000876
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+000888
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+000888
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+000888
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+000900
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+000900
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+000900
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+000912
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+000912
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+000912
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+000924
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+000924
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+000924
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+000936
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+000936
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+000936
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+000948
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+000948
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+000948
Zipping up the intermediate file: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_a3cx_2016092300_15_991_12211140_1_r972307138_9.zip
Uploading the intermediate file: upload_file_9.zip
  02:28:24 STEP  961 H= 240:15 +CPU= 27.457
Uploading trickle at timestep: 864000
  02:28:53 STEP  962 H= 240:30 +CPU= 28.725
  02:29:23 STEP  963 H= 240:45 +CPU= 29.336
  02:29:53 STEP  964 H= 241:00 +CPU= 28.902
  02:30:23 STEP  965 H= 241:15 +CPU= 28.359
  02:30:52 STEP  966 H= 241:30 +CPU= 28.230
  02:31:21 STEP  967 H= 241:45 +CPU= 28.169
  02:31:50 STEP  968 H= 242:00 +CPU= 28.957
  02:32:19 STEP  969 H= 242:15 +CPU= 27.803
  02:32:48 STEP  970 H= 242:30 +CPU= 28.552
  02:33:18 STEP  971 H= 242:45 +CPU= 29.093
  02:33:55 STEP  972 H= 243:00 +CPU= 35.616
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+000972
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+000972
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+000972
  02:34:24 STEP  973 H= 243:15 +CPU= 27.854
  02:34:53 STEP  974 H= 243:30 +CPU= 28.053
  02:35:20 STEP  975 H= 243:45 +CPU= 27.557
  02:35:49 STEP  976 H= 244:00 +CPU= 28.434
  02:36:18 STEP  977 H= 244:15 +CPU= 27.811
  02:36:48 STEP  978 H= 244:30 +CPU= 28.680
  02:37:16 STEP  979 H= 244:45 +CPU= 27.702
  02:37:46 STEP  980 H= 245:00 +CPU= 28.844
  02:38:13 STEP  981 H= 245:15 +CPU= 27.042
  02:38:43 STEP  982 H= 245:30 +CPU= 28.634
  02:39:12 STEP  983 H= 245:45 +CPU= 29.287
  02:39:48 STEP  984 H= 246:00 +CPU= 35.264
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+000984
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+000984
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+000984
  02:40:18 STEP  985 H= 246:15 +CPU= 28.590
  02:40:47 STEP  986 H= 246:30 +CPU= 28.908
  02:41:16 STEP  987 H= 246:45 +CPU= 28.017
  02:41:45 STEP  988 H= 247:00 +CPU= 28.717
  02:42:16 STEP  989 H= 247:15 +CPU= 29.932
  02:42:45 STEP  990 H= 247:30 +CPU= 28.628
  02:43:13 STEP  991 H= 247:45 +CPU= 27.313
  02:43:42 STEP  992 H= 248:00 +CPU= 28.165
  02:44:11 STEP  993 H= 248:15 +CPU= 28.460
  02:44:39 STEP  994 H= 248:30 +CPU= 27.757
  02:45:07 STEP  995 H= 248:45 +CPU= 27.261
  02:45:42 STEP  996 H= 249:00 +CPU= 34.309
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+000996
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+000996
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+000996
  02:46:11 STEP  997 H= 249:15 +CPU= 28.389
  02:46:38 STEP  998 H= 249:30 +CPU= 26.870
  02:47:08 STEP  999 H= 249:45 +CPU= 28.754
  02:47:38 STEP 1000 H= 250:00 +CPU= 28.348
  02:48:07 STEP 1001 H= 250:15 +CPU= 27.731
  02:48:37 STEP 1002 H= 250:30 +CPU= 29.641
  02:49:05 STEP 1003 H= 250:45 +CPU= 27.548
  02:49:34 STEP 1004 H= 251:00 +CPU= 27.894
  02:50:02 STEP 1005 H= 251:15 +CPU= 27.626
  02:50:32 STEP 1006 H= 251:30 +CPU= 28.703
  02:51:03 STEP 1007 H= 251:45 +CPU= 30.256
  02:51:44 STEP 1008 H= 252:00 +CPU= 39.953
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001008
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001008
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001008
  02:52:12 STEP 1009 H= 252:15 +CPU= 28.444
  02:52:40 STEP 1010 H= 252:30 +CPU= 27.639
  02:53:09 STEP 1011 H= 252:45 +CPU= 28.215
  02:53:37 STEP 1012 H= 253:00 +CPU= 28.187
  02:54:08 STEP 1013 H= 253:15 +CPU= 29.358
  02:54:36 STEP 1014 H= 253:30 +CPU= 27.291
  02:55:05 STEP 1015 H= 253:45 +CPU= 27.732
  02:55:35 STEP 1016 H= 254:00 +CPU= 28.976
  02:56:05 STEP 1017 H= 254:15 +CPU= 28.123
  02:56:34 STEP 1018 H= 254:30 +CPU= 27.616
  02:57:03 STEP 1019 H= 254:45 +CPU= 28.371
  02:57:39 STEP 1020 H= 255:00 +CPU= 35.255
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001020
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001020
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001020
  02:58:08 STEP 1021 H= 255:15 +CPU= 28.411
  02:58:37 STEP 1022 H= 255:30 +CPU= 28.680
  02:59:06 STEP 1023 H= 255:45 +CPU= 28.098
  02:59:34 STEP 1024 H= 256:00 +CPU= 28.351
  03:00:04 STEP 1025 H= 256:15 +CPU= 28.054
  03:00:31 STEP 1026 H= 256:30 +CPU= 27.399
  03:00:59 STEP 1027 H= 256:45 +CPU= 27.092
  03:01:29 STEP 1028 H= 257:00 +CPU= 28.342
  03:01:58 STEP 1029 H= 257:15 +CPU= 28.226
  03:02:26 STEP 1030 H= 257:30 +CPU= 27.503
  03:02:54 STEP 1031 H= 257:45 +CPU= 27.462
  03:03:32 STEP 1032 H= 258:00 +CPU= 36.739
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001032
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001032
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001032
  03:04:01 STEP 1033 H= 258:15 +CPU= 28.250
  03:04:30 STEP 1034 H= 258:30 +CPU= 27.598
  03:05:00 STEP 1035 H= 258:45 +CPU= 28.641
  03:05:30 STEP 1036 H= 259:00 +CPU= 29.393
  03:06:00 STEP 1037 H= 259:15 +CPU= 28.340
  03:06:29 STEP 1038 H= 259:30 +CPU= 28.213
  03:07:00 STEP 1039 H= 259:45 +CPU= 28.455
  03:07:29 STEP 1040 H= 260:00 +CPU= 28.257
  03:07:57 STEP 1041 H= 260:15 +CPU= 27.302
  03:08:25 STEP 1042 H= 260:30 +CPU= 27.479
  03:08:54 STEP 1043 H= 260:45 +CPU= 28.749
  03:09:30 STEP 1044 H= 261:00 +CPU= 34.256
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001044
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001044
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001044
  03:09:59 STEP 1045 H= 261:15 +CPU= 27.569
  03:10:27 STEP 1046 H= 261:30 +CPU= 27.671
  03:10:55 STEP 1047 H= 261:45 +CPU= 27.611
  03:11:26 STEP 1048 H= 262:00 +CPU= 29.930
  03:11:54 STEP 1049 H= 262:15 +CPU= 27.965
  03:12:24 STEP 1050 H= 262:30 +CPU= 28.490
  03:12:52 STEP 1051 H= 262:45 +CPU= 28.322
  03:13:22 STEP 1052 H= 263:00 +CPU= 28.699
  03:13:52 STEP 1053 H= 263:15 +CPU= 29.150
  03:14:22 STEP 1054 H= 263:30 +CPU= 28.328
  03:14:50 STEP 1055 H= 263:45 +CPU= 28.172
  03:15:31 STEP 1056 H= 264:00 +CPU= 38.744
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001056
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001056
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001056
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+000960
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+000960
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+000960
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+000972
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+000972
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+000972
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+000984
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+000984
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+000984
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+000996
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+000996
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+000996
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+001008
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+001008
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+001008
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+001020
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+001020
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+001020
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+001032
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+001032
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+001032
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+001044
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+001044
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+001044
Zipping up the intermediate file: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_a3cx_2016092300_15_991_12211140_1_r972307138_10.zip
Uploading the intermediate file: upload_file_10.zip
  03:16:00 STEP 1057 H= 264:15 +CPU= 27.943
  03:16:29 STEP 1058 H= 264:30 +CPU= 28.526
  03:16:58 STEP 1059 H= 264:45 +CPU= 27.905
  03:17:28 STEP 1060 H= 265:00 +CPU= 28.132
  03:17:58 STEP 1061 H= 265:15 +CPU= 28.166
  03:18:27 STEP 1062 H= 265:30 +CPU= 27.928
  03:18:56 STEP 1063 H= 265:45 +CPU= 27.659
  03:19:26 STEP 1064 H= 266:00 +CPU= 28.677
  03:19:56 STEP 1065 H= 266:15 +CPU= 29.078
  03:20:25 STEP 1066 H= 266:30 +CPU= 28.070
  03:20:56 STEP 1067 H= 266:45 +CPU= 29.505
  03:21:32 STEP 1068 H= 267:00 +CPU= 34.773
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001068
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001068
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001068
  03:22:02 STEP 1069 H= 267:15 +CPU= 28.360
  03:22:32 STEP 1070 H= 267:30 +CPU= 28.292
  03:23:03 STEP 1071 H= 267:45 +CPU= 29.877
  03:23:32 STEP 1072 H= 268:00 +CPU= 28.050
  03:24:01 STEP 1073 H= 268:15 +CPU= 27.911
  03:24:30 STEP 1074 H= 268:30 +CPU= 27.523
  03:24:58 STEP 1075 H= 268:45 +CPU= 27.725
  03:25:28 STEP 1076 H= 269:00 +CPU= 28.504
  03:25:57 STEP 1077 H= 269:15 +CPU= 27.828
  03:26:26 STEP 1078 H= 269:30 +CPU= 28.254
  03:26:56 STEP 1079 H= 269:45 +CPU= 29.281
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.11_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a3cx
(argv4) batchid: 991
(argv5) wuid: 12211140
(argv6) fclen: 15
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /mydisks/a/boinc_lib/slots/1
Project directory is: /mydisks/a/boinc_lib/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.11
Location of temp folder: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140
..mkdir for temp folder for results failed
Copying: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip to: /mydisks/a/boinc_lib/slots/1/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /mydisks/a/boinc_lib/slots/1/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_abf4dc20b4b6d28a2ee0c3c4a77988a5 to: /mydisks/a/boinc_lib/slots/1/oifs_43r3_bl_a3cx_2016092300_15_991_12211140.zip
Unzipping the namelist zip file: /mydisks/a/boinc_lib/slots/1/oifs_43r3_bl_a3cx_2016092300_15_991_12211140.zip
ic_ancil_file: ic_ancil_12211140
ifsdata_file: ifsdata_12211140
climate_data_file: clim_data_12211140
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /mydisks/a/boinc_lib/slots/1/ic_ancil_12211140.zip
Unzipping the IC ancils zip file: /mydisks/a/boinc_lib/slots/1/ic_ancil_12211140.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_e1c856c9f2109d4420f0b716b5fc679c to: /mydisks/a/boinc_lib/slots/1/ifsdata/ifsdata_12211140.zip
Unzipping the ifsdata_zip file: /mydisks/a/boinc_lib/slots/1/ifsdata/ifsdata_12211140.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /mydisks/a/boinc_lib/slots/1/159l_2/clim_data_12211140.zip
Unzipping the climate data zip file: /mydisks/a/boinc_lib/slots/1/159l_2/clim_data_12211140.zip
Checking for progress XML file: /mydisks/a/boinc_lib/slots/1/progress_file_12211140.xml
Opened progress file ok : /mydisks/a/boinc_lib/slots/1/progress_file_12211140.xml
-- Model is restarting --
Adjusting last_iter, 1079, to previous model restart step.
Creating progress file: /mydisks/a/boinc_lib/slots/1/progress_file_12211140.xml
last_cpu_time: 30827
upload_file_number: 11
last_iter: 1057
last_upload: 950400
model_completed: 0
total_length_of_simulation: 1296000
result_base_name: oifs_43r3_bl_a3cx_2016092300_15_991_12211140_1_r972307138
The child process has been launched with process id: 20298
Executing the command: /mydisks/a/boinc_lib/slots/1/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:amigos:1:1:20298:20298] [20230217:033053:1676633453:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2091] fp = 0x30e4a00
[EC_DRHOOK:amigos:1:1:20298:20298] [20230217:033053:1676633453:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:amigos:1:1:20298:20298] [20230217:033053:1676633453:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:amigos:1:1:20298:20298] [20230217:033053:1676633453:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:amigos:1:1:20298:20298] [20230217:033053:1676633453:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:amigos:1:1:20298:20298] [20230217:033053:1676633453:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:amigos:1:1:20298:20298] [20230217:033053:1676633453:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:amigos:1:1:20298:20298] [20230217:033053:1676633453:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:amigos:1:1:20298:20298] [20230217:033053:1676633453:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:amigos:1:1:20298:20298] [20230217:033053:1676633453:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:amigos:1:1:20298:20298] [20230217:033053:1676633453:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:amigos:1:1:20298:20298] [20230217:033053:1676633453:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:amigos:1:1:20298:20298] [20230217:033053:1676633453:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:amigos:1:1:20298:20298] [20230217:033053:1676633453:0.000] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:amigos:1:1:20298:20298] [20230217:033053:1676633453:0.000] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:amigos:1:1:20298:20298] [20230217:033053:1676633453:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:amigos:1:1:20298:20298] [20230217:033053:1676633453:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f520 (old at (nil))
[EC_DRHOOK:amigos:1:1:20298:20298] [20230217:033053:1676633453:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:amigos:1:1:20298:20298] [20230217:033053:1676633453:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f520 (old at (nil))
[EC_DRHOOK:amigos:1:1:20298:20298] [20230217:033053:1676633453:0.000] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:amigos:1:1:20298:20298] [20230217:033053:1676633453:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:amigos:1:1:20298:20298] [20230217:033053:1676633453:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:amigos:1:1:20298:20298] [20230217:033053:1676633453:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f520 (old at (nil))
[EC_DRHOOK:amigos:1:1:20298:20298] [20230217:033053:1676633453:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:amigos:1:1:20298:20298] [20230217:033053:1676633453:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:amigos:1:1:20298:20298] [20230217:033053:1676633453:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:amigos:1:1:20298:20298] [20230217:033053:1676633453:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f520 (old at (nil))
[EC_DRHOOK:amigos:1:1:20298:20298] [20230217:033053:1676633453:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f520 (old at (nil))
[EC_DRHOOK:amigos:1:1:20298:20298] [20230217:033053:1676633453:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f520 (old at (nil))
[EC_DRHOOK:amigos:1:1:20298:20298] [20230217:033053:1676633453:0.000] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  03:31:21 STEP 1056 H= 264:00 +CPU= 24.044
  03:31:41 STEP 1057 H= 264:15 +CPU= 19.798
  03:32:05 STEP 1058 H= 264:30 +CPU= 24.221
  03:32:32 STEP 1059 H= 264:45 +CPU= 26.572
  03:33:00 STEP 1060 H= 265:00 +CPU= 27.811
  03:33:30 STEP 1061 H= 265:15 +CPU= 28.778
  03:33:58 STEP 1062 H= 265:30 +CPU= 27.171
  03:34:26 STEP 1063 H= 265:45 +CPU= 27.710
  03:34:55 STEP 1064 H= 266:00 +CPU= 28.412
  03:35:23 STEP 1065 H= 266:15 +CPU= 27.704
  03:35:51 STEP 1066 H= 266:30 +CPU= 27.345
  03:36:19 STEP 1067 H= 266:45 +CPU= 27.101
  03:36:56 STEP 1068 H= 267:00 +CPU= 35.477
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001068
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001068
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001068
  03:37:24 STEP 1069 H= 267:15 +CPU= 26.909
  03:37:53 STEP 1070 H= 267:30 +CPU= 28.044
  03:38:24 STEP 1071 H= 267:45 +CPU= 29.206
  03:38:51 STEP 1072 H= 268:00 +CPU= 27.841
  03:39:20 STEP 1073 H= 268:15 +CPU= 27.624
  03:39:50 STEP 1074 H= 268:30 +CPU= 28.576
  03:40:18 STEP 1075 H= 268:45 +CPU= 27.450
  03:40:46 STEP 1076 H= 269:00 +CPU= 28.192
  03:41:16 STEP 1077 H= 269:15 +CPU= 29.376
  03:41:45 STEP 1078 H= 269:30 +CPU= 28.481
  03:42:15 STEP 1079 H= 269:45 +CPU= 28.413
  03:42:51 STEP 1080 H= 270:00 +CPU= 34.689
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001080
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001080
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001080
  03:43:19 STEP 1081 H= 270:15 +CPU= 27.804
  03:43:47 STEP 1082 H= 270:30 +CPU= 27.248
  03:44:15 STEP 1083 H= 270:45 +CPU= 26.951
  03:44:44 STEP 1084 H= 271:00 +CPU= 28.985
  03:45:13 STEP 1085 H= 271:15 +CPU= 28.133
  03:45:43 STEP 1086 H= 271:30 +CPU= 28.426
  03:46:12 STEP 1087 H= 271:45 +CPU= 27.677
  03:46:42 STEP 1088 H= 272:00 +CPU= 29.165
  03:47:13 STEP 1089 H= 272:15 +CPU= 28.216
  03:47:41 STEP 1090 H= 272:30 +CPU= 27.770
  03:48:10 STEP 1091 H= 272:45 +CPU= 28.926
  03:48:46 STEP 1092 H= 273:00 +CPU= 34.787
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001092
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001092
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001092
  03:49:15 STEP 1093 H= 273:15 +CPU= 28.126
  03:49:44 STEP 1094 H= 273:30 +CPU= 28.344
  03:50:13 STEP 1095 H= 273:45 +CPU= 28.383
  03:50:42 STEP 1096 H= 274:00 +CPU= 28.169
  03:51:11 STEP 1097 H= 274:15 +CPU= 27.930
  03:51:39 STEP 1098 H= 274:30 +CPU= 27.511
  03:52:07 STEP 1099 H= 274:45 +CPU= 27.128
  03:52:37 STEP 1100 H= 275:00 +CPU= 29.379
  03:53:07 STEP 1101 H= 275:15 +CPU= 29.050
  03:53:35 STEP 1102 H= 275:30 +CPU= 27.025
  03:54:03 STEP 1103 H= 275:45 +CPU= 27.407
  03:54:43 STEP 1104 H= 276:00 +CPU= 37.992
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001104
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001104
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001104
  03:55:11 STEP 1105 H= 276:15 +CPU= 27.034
  03:55:39 STEP 1106 H= 276:30 +CPU= 27.959
  03:56:08 STEP 1107 H= 276:45 +CPU= 28.106
  03:56:37 STEP 1108 H= 277:00 +CPU= 27.728
  03:57:05 STEP 1109 H= 277:15 +CPU= 26.897
  03:57:34 STEP 1110 H= 277:30 +CPU= 28.393
  03:58:04 STEP 1111 H= 277:45 +CPU= 28.880
  03:58:32 STEP 1112 H= 278:00 +CPU= 27.781
  03:59:00 STEP 1113 H= 278:15 +CPU= 27.857
  03:59:29 STEP 1114 H= 278:30 +CPU= 27.566
  03:59:58 STEP 1115 H= 278:45 +CPU= 27.931
  04:00:35 STEP 1116 H= 279:00 +CPU= 35.721
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001116
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001116
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001116
  04:01:04 STEP 1117 H= 279:15 +CPU= 29.052
  04:01:32 STEP 1118 H= 279:30 +CPU= 27.306
  04:02:02 STEP 1119 H= 279:45 +CPU= 28.434
  04:02:32 STEP 1120 H= 280:00 +CPU= 29.062
  04:03:00 STEP 1121 H= 280:15 +CPU= 26.945
  04:03:28 STEP 1122 H= 280:30 +CPU= 27.394
  04:03:56 STEP 1123 H= 280:45 +CPU= 28.413
  04:04:26 STEP 1124 H= 281:00 +CPU= 29.300
  04:04:55 STEP 1125 H= 281:15 +CPU= 28.012
  04:05:23 STEP 1126 H= 281:30 +CPU= 27.495
  04:05:52 STEP 1127 H= 281:45 +CPU= 27.969
  04:06:28 STEP 1128 H= 282:00 +CPU= 34.612
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001128
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001128
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001128
  04:06:58 STEP 1129 H= 282:15 +CPU= 28.560
  04:07:26 STEP 1130 H= 282:30 +CPU= 28.336
  04:07:55 STEP 1131 H= 282:45 +CPU= 27.364
  04:08:25 STEP 1132 H= 283:00 +CPU= 28.250
  04:08:54 STEP 1133 H= 283:15 +CPU= 27.853
  04:09:22 STEP 1134 H= 283:30 +CPU= 28.159
  04:09:51 STEP 1135 H= 283:45 +CPU= 27.878
  04:10:19 STEP 1136 H= 284:00 +CPU= 27.993
  04:10:48 STEP 1137 H= 284:15 +CPU= 28.008
  04:11:17 STEP 1138 H= 284:30 +CPU= 27.606
  04:11:46 STEP 1139 H= 284:45 +CPU= 28.182
  04:12:24 STEP 1140 H= 285:00 +CPU= 36.800
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001140
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001140
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001140
  04:12:53 STEP 1141 H= 285:15 +CPU= 28.574
  04:13:22 STEP 1142 H= 285:30 +CPU= 27.477
  04:13:50 STEP 1143 H= 285:45 +CPU= 27.320
  04:14:18 STEP 1144 H= 286:00 +CPU= 28.480
  04:14:46 STEP 1145 H= 286:15 +CPU= 27.473
  04:15:16 STEP 1146 H= 286:30 +CPU= 28.182
  04:15:45 STEP 1147 H= 286:45 +CPU= 27.434
  04:16:15 STEP 1148 H= 287:00 +CPU= 28.824
  04:16:43 STEP 1149 H= 287:15 +CPU= 27.365
  04:17:10 STEP 1150 H= 287:30 +CPU= 27.141
  04:17:38 STEP 1151 H= 287:45 +CPU= 27.205
  04:18:17 STEP 1152 H= 288:00 +CPU= 37.910
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001152
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001152
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001152
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+001056
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+001056
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+001056
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+001068
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+001068
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+001068
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+001080
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+001080
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+001080
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+001092
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+001092
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+001092
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+001104
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+001104
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+001104
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+001116
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+001116
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+001116
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+001128
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+001128
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+001128
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+001140
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+001140
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+001140
Zipping up the intermediate file: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_a3cx_2016092300_15_991_12211140_1_r972307138_11.zip
Uploading the intermediate file: upload_file_11.zip
  04:18:46 STEP 1153 H= 288:15 +CPU= 28.223
  04:19:14 STEP 1154 H= 288:30 +CPU= 27.144
  04:19:41 STEP 1155 H= 288:45 +CPU= 26.873
  04:20:10 STEP 1156 H= 289:00 +CPU= 27.920
  04:20:38 STEP 1157 H= 289:15 +CPU= 27.481
  04:21:06 STEP 1158 H= 289:30 +CPU= 27.197
  04:21:34 STEP 1159 H= 289:45 +CPU= 27.623
  04:22:03 STEP 1160 H= 290:00 +CPU= 28.797
  04:22:31 STEP 1161 H= 290:15 +CPU= 27.240
  04:23:01 STEP 1162 H= 290:30 +CPU= 28.638
  04:23:30 STEP 1163 H= 290:45 +CPU= 27.891
  04:24:08 STEP 1164 H= 291:00 +CPU= 36.062
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001164
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001164
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001164
  04:24:37 STEP 1165 H= 291:15 +CPU= 28.151
  04:25:05 STEP 1166 H= 291:30 +CPU= 27.322
  04:25:34 STEP 1167 H= 291:45 +CPU= 27.817
  04:26:03 STEP 1168 H= 292:00 +CPU= 28.846
  04:26:33 STEP 1169 H= 292:15 +CPU= 27.953
  04:27:02 STEP 1170 H= 292:30 +CPU= 28.075
  04:27:31 STEP 1171 H= 292:45 +CPU= 28.279
  04:28:00 STEP 1172 H= 293:00 +CPU= 28.535
  04:28:28 STEP 1173 H= 293:15 +CPU= 27.357
  04:28:58 STEP 1174 H= 293:30 +CPU= 27.788
  04:29:27 STEP 1175 H= 293:45 +CPU= 28.740
  04:30:04 STEP 1176 H= 294:00 +CPU= 35.331
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001176
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001176
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001176
  04:30:32 STEP 1177 H= 294:15 +CPU= 27.750
  04:31:01 STEP 1178 H= 294:30 +CPU= 28.205
  04:31:29 STEP 1179 H= 294:45 +CPU= 27.386
  04:31:57 STEP 1180 H= 295:00 +CPU= 27.763
  04:32:25 STEP 1181 H= 295:15 +CPU= 27.692
  04:32:55 STEP 1182 H= 295:30 +CPU= 27.879
  04:33:23 STEP 1183 H= 295:45 +CPU= 28.045
  04:33:53 STEP 1184 H= 296:00 +CPU= 28.853
  04:34:23 STEP 1185 H= 296:15 +CPU= 29.221
  04:34:52 STEP 1186 H= 296:30 +CPU= 28.391
  04:35:21 STEP 1187 H= 296:45 +CPU= 27.520
  04:35:57 STEP 1188 H= 297:00 +CPU= 35.935
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001188
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001188
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001188
  04:36:25 STEP 1189 H= 297:15 +CPU= 27.475
  04:36:54 STEP 1190 H= 297:30 +CPU= 28.495
  04:37:25 STEP 1191 H= 297:45 +CPU= 29.757
  04:37:55 STEP 1192 H= 298:00 +CPU= 29.674
  04:38:23 STEP 1193 H= 298:15 +CPU= 27.367
  04:38:52 STEP 1194 H= 298:30 +CPU= 27.544
  04:39:21 STEP 1195 H= 298:45 +CPU= 28.529
  04:39:51 STEP 1196 H= 299:00 +CPU= 29.958
  04:40:20 STEP 1197 H= 299:15 +CPU= 27.586
  04:40:48 STEP 1198 H= 299:30 +CPU= 27.896
  04:41:17 STEP 1199 H= 299:45 +CPU= 28.261
  04:41:56 STEP 1200 H= 300:00 +CPU= 37.746
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001200
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001200
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001200
  04:42:24 STEP 1201 H= 300:15 +CPU= 27.115
  04:42:51 STEP 1202 H= 300:30 +CPU= 27.128
  04:43:19 STEP 1203 H= 300:45 +CPU= 27.791
  04:43:47 STEP 1204 H= 301:00 +CPU= 27.703
  04:44:16 STEP 1205 H= 301:15 +CPU= 27.795
  04:44:45 STEP 1206 H= 301:30 +CPU= 27.750
  04:45:15 STEP 1207 H= 301:45 +CPU= 28.521
  04:45:44 STEP 1208 H= 302:00 +CPU= 28.093
  04:46:11 STEP 1209 H= 302:15 +CPU= 27.318
  04:46:40 STEP 1210 H= 302:30 +CPU= 27.655
  04:47:09 STEP 1211 H= 302:45 +CPU= 28.077
  04:47:45 STEP 1212 H= 303:00 +CPU= 35.274
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001212
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001212
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001212
  04:48:14 STEP 1213 H= 303:15 +CPU= 28.320
  04:48:42 STEP 1214 H= 303:30 +CPU= 28.601
  04:49:11 STEP 1215 H= 303:45 +CPU= 27.855
  04:49:39 STEP 1216 H= 304:00 +CPU= 27.942
  04:50:08 STEP 1217 H= 304:15 +CPU= 28.167
  04:50:40 STEP 1218 H= 304:30 +CPU= 29.366
  04:51:09 STEP 1219 H= 304:45 +CPU= 28.108
  04:51:39 STEP 1220 H= 305:00 +CPU= 29.490
  04:52:08 STEP 1221 H= 305:15 +CPU= 28.096
  04:52:35 STEP 1222 H= 305:30 +CPU= 27.279
  04:53:03 STEP 1223 H= 305:45 +CPU= 27.636
  04:53:39 STEP 1224 H= 306:00 +CPU= 34.948
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001224
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001224
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001224
  04:54:08 STEP 1225 H= 306:15 +CPU= 28.704
  04:54:37 STEP 1226 H= 306:30 +CPU= 28.184
  04:55:07 STEP 1227 H= 306:45 +CPU= 29.651
  04:55:37 STEP 1228 H= 307:00 +CPU= 29.702
  04:56:05 STEP 1229 H= 307:15 +CPU= 27.667
  04:56:35 STEP 1230 H= 307:30 +CPU= 29.437
  04:57:04 STEP 1231 H= 307:45 +CPU= 28.100
  04:57:32 STEP 1232 H= 308:00 +CPU= 28.453
  04:58:01 STEP 1233 H= 308:15 +CPU= 28.088
  04:58:30 STEP 1234 H= 308:30 +CPU= 27.204
  04:58:58 STEP 1235 H= 308:45 +CPU= 27.789
  04:59:34 STEP 1236 H= 309:00 +CPU= 35.358
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001236
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001236
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001236
  05:00:04 STEP 1237 H= 309:15 +CPU= 29.022
  05:00:34 STEP 1238 H= 309:30 +CPU= 28.661
  05:01:03 STEP 1239 H= 309:45 +CPU= 28.176
  05:01:32 STEP 1240 H= 310:00 +CPU= 28.449
  05:02:01 STEP 1241 H= 310:15 +CPU= 27.542
  05:02:29 STEP 1242 H= 310:30 +CPU= 27.802
  05:02:59 STEP 1243 H= 310:45 +CPU= 28.632
  05:03:27 STEP 1244 H= 311:00 +CPU= 27.662
  05:03:57 STEP 1245 H= 311:15 +CPU= 28.675
  05:04:25 STEP 1246 H= 311:30 +CPU= 28.247
  05:04:54 STEP 1247 H= 311:45 +CPU= 27.952
  05:05:33 STEP 1248 H= 312:00 +CPU= 39.301
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001248
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001248
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001248
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+001152
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+001152
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+001152
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+001164
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+001164
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+001164
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+001176
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+001176
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+001176
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+001188
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+001188
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+001188
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+001200
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+001200
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+001200
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+001212
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+001212
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+001212
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+001224
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+001224
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+001224
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+001236
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+001236
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+001236
Zipping up the intermediate file: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_a3cx_2016092300_15_991_12211140_1_r972307138_12.zip
Uploading the intermediate file: upload_file_12.zip
  05:06:04 STEP 1249 H= 312:15 +CPU= 28.986
  05:06:32 STEP 1250 H= 312:30 +CPU= 27.967
  05:07:00 STEP 1251 H= 312:45 +CPU= 27.488
  05:07:29 STEP 1252 H= 313:00 +CPU= 27.975
  05:07:56 STEP 1253 H= 313:15 +CPU= 27.272
  05:08:25 STEP 1254 H= 313:30 +CPU= 28.218
  05:08:54 STEP 1255 H= 313:45 +CPU= 28.416
  05:09:23 STEP 1256 H= 314:00 +CPU= 28.126
  05:09:50 STEP 1257 H= 314:15 +CPU= 27.378
  05:10:20 STEP 1258 H= 314:30 +CPU= 29.276
  05:10:51 STEP 1259 H= 314:45 +CPU= 29.757
  05:11:27 STEP 1260 H= 315:00 +CPU= 35.292
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001260
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001260
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001260
  05:11:56 STEP 1261 H= 315:15 +CPU= 28.142
  05:12:26 STEP 1262 H= 315:30 +CPU= 29.254
  05:12:54 STEP 1263 H= 315:45 +CPU= 27.740
  05:13:24 STEP 1264 H= 316:00 +CPU= 28.551
  05:13:53 STEP 1265 H= 316:15 +CPU= 28.612
  05:14:22 STEP 1266 H= 316:30 +CPU= 28.799
  05:14:50 STEP 1267 H= 316:45 +CPU= 27.387
  05:15:19 STEP 1268 H= 317:00 +CPU= 27.814
  05:15:48 STEP 1269 H= 317:15 +CPU= 27.508
  05:16:17 STEP 1270 H= 317:30 +CPU= 27.627
  05:16:47 STEP 1271 H= 317:45 +CPU= 28.725
  05:17:23 STEP 1272 H= 318:00 +CPU= 34.768
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001272
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001272
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001272
  05:17:51 STEP 1273 H= 318:15 +CPU= 27.363
  05:18:19 STEP 1274 H= 318:30 +CPU= 27.449
  05:18:48 STEP 1275 H= 318:45 +CPU= 28.645
  05:19:17 STEP 1276 H= 319:00 +CPU= 27.914
  05:19:46 STEP 1277 H= 319:15 +CPU= 28.126
  05:20:15 STEP 1278 H= 319:30 +CPU= 28.181
  05:20:43 STEP 1279 H= 319:45 +CPU= 27.084
  05:21:15 STEP 1280 H= 320:00 +CPU= 29.744
  05:21:43 STEP 1281 H= 320:15 +CPU= 27.690
  05:22:13 STEP 1282 H= 320:30 +CPU= 27.986
  05:22:42 STEP 1283 H= 320:45 +CPU= 28.685
  05:23:18 STEP 1284 H= 321:00 +CPU= 34.773
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001284
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001284
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001284
  05:23:47 STEP 1285 H= 321:15 +CPU= 29.255
  05:24:15 STEP 1286 H= 321:30 +CPU= 27.229
  05:24:45 STEP 1287 H= 321:45 +CPU= 29.166
  05:25:12 STEP 1288 H= 322:00 +CPU= 27.450
  05:25:40 STEP 1289 H= 322:15 +CPU= 27.151
  05:26:09 STEP 1290 H= 322:30 +CPU= 28.678
  05:26:39 STEP 1291 H= 322:45 +CPU= 29.676
  05:27:09 STEP 1292 H= 323:00 +CPU= 29.269
  05:27:38 STEP 1293 H= 323:15 +CPU= 27.722
  05:28:07 STEP 1294 H= 323:30 +CPU= 27.198
  05:28:37 STEP 1295 H= 323:45 +CPU= 29.178
  05:29:18 STEP 1296 H= 324:00 +CPU= 39.506
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001296
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001296
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001296
  05:29:47 STEP 1297 H= 324:15 +CPU= 27.898
  05:30:16 STEP 1298 H= 324:30 +CPU= 29.407
  05:30:45 STEP 1299 H= 324:45 +CPU= 27.879
  05:31:15 STEP 1300 H= 325:00 +CPU= 29.313
  05:31:45 STEP 1301 H= 325:15 +CPU= 29.287
  05:32:14 STEP 1302 H= 325:30 +CPU= 26.976
  05:32:43 STEP 1303 H= 325:45 +CPU= 27.508
  05:33:11 STEP 1304 H= 326:00 +CPU= 28.234
  05:33:40 STEP 1305 H= 326:15 +CPU= 27.501
  05:34:09 STEP 1306 H= 326:30 +CPU= 27.960
  05:34:40 STEP 1307 H= 326:45 +CPU= 29.482
  05:35:16 STEP 1308 H= 327:00 +CPU= 34.719
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001308
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001308
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001308
  05:35:45 STEP 1309 H= 327:15 +CPU= 27.183
  05:36:14 STEP 1310 H= 327:30 +CPU= 28.134
  05:36:43 STEP 1311 H= 327:45 +CPU= 27.979
  05:37:12 STEP 1312 H= 328:00 +CPU= 28.331
  05:37:41 STEP 1313 H= 328:15 +CPU= 28.015
  05:38:10 STEP 1314 H= 328:30 +CPU= 28.172
  05:38:38 STEP 1315 H= 328:45 +CPU= 26.961
  05:39:07 STEP 1316 H= 329:00 +CPU= 27.852
  05:39:36 STEP 1317 H= 329:15 +CPU= 27.743
  05:40:05 STEP 1318 H= 329:30 +CPU= 27.671
  05:40:35 STEP 1319 H= 329:45 +CPU= 29.354
  05:41:10 STEP 1320 H= 330:00 +CPU= 33.821
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001320
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001320
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001320
  05:41:39 STEP 1321 H= 330:15 +CPU= 27.871
  05:42:08 STEP 1322 H= 330:30 +CPU= 28.612
  05:42:38 STEP 1323 H= 330:45 +CPU= 29.279
  05:43:07 STEP 1324 H= 331:00 +CPU= 28.217
  05:43:36 STEP 1325 H= 331:15 +CPU= 28.062
  05:44:05 STEP 1326 H= 331:30 +CPU= 28.550
  05:44:34 STEP 1327 H= 331:45 +CPU= 28.013
  05:45:03 STEP 1328 H= 332:00 +CPU= 29.045
  05:45:33 STEP 1329 H= 332:15 +CPU= 28.761
  05:46:01 STEP 1330 H= 332:30 +CPU= 28.361
  05:46:29 STEP 1331 H= 332:45 +CPU= 27.437
  05:47:05 STEP 1332 H= 333:00 +CPU= 34.940
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001332
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001332
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001332
  05:47:35 STEP 1333 H= 333:15 +CPU= 28.825
  05:48:04 STEP 1334 H= 333:30 +CPU= 28.054
  05:48:33 STEP 1335 H= 333:45 +CPU= 28.605
  05:49:01 STEP 1336 H= 334:00 +CPU= 27.856
  05:49:31 STEP 1337 H= 334:15 +CPU= 28.940
  05:50:00 STEP 1338 H= 334:30 +CPU= 29.412
  05:50:28 STEP 1339 H= 334:45 +CPU= 27.445
  05:50:57 STEP 1340 H= 335:00 +CPU= 27.708
  05:51:26 STEP 1341 H= 335:15 +CPU= 28.693
  05:51:54 STEP 1342 H= 335:30 +CPU= 27.585
  05:52:23 STEP 1343 H= 335:45 +CPU= 27.401
  05:53:02 STEP 1344 H= 336:00 +CPU= 39.424
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001344
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001344
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001344
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+001248
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+001248
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+001248
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+001260
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+001260
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+001260
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+001272
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+001272
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+001272
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+001284
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+001284
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+001284
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+001296
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+001296
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+001296
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+001308
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+001308
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+001308
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+001320
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+001320
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+001320
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+001332
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+001332
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+001332
Zipping up the intermediate file: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_a3cx_2016092300_15_991_12211140_1_r972307138_13.zip
Uploading the intermediate file: upload_file_13.zip
  05:53:30 STEP 1345 H= 336:15 +CPU= 27.701
  05:53:59 STEP 1346 H= 336:30 +CPU= 27.869
  05:54:29 STEP 1347 H= 336:45 +CPU= 28.428
  05:54:59 STEP 1348 H= 337:00 +CPU= 29.841
  05:55:28 STEP 1349 H= 337:15 +CPU= 27.987
  05:55:56 STEP 1350 H= 337:30 +CPU= 27.003
  05:56:25 STEP 1351 H= 337:45 +CPU= 28.290
  05:56:54 STEP 1352 H= 338:00 +CPU= 28.717
  05:57:22 STEP 1353 H= 338:15 +CPU= 27.423
  05:57:50 STEP 1354 H= 338:30 +CPU= 27.506
  05:58:19 STEP 1355 H= 338:45 +CPU= 29.268
  05:58:57 STEP 1356 H= 339:00 +CPU= 36.192
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001356
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001356
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001356
  05:59:25 STEP 1357 H= 339:15 +CPU= 27.387
  05:59:53 STEP 1358 H= 339:30 +CPU= 27.734
  06:00:22 STEP 1359 H= 339:45 +CPU= 28.135
  06:00:53 STEP 1360 H= 340:00 +CPU= 29.754
  06:01:21 STEP 1361 H= 340:15 +CPU= 27.946
  06:01:50 STEP 1362 H= 340:30 +CPU= 27.730
  06:02:17 STEP 1363 H= 340:45 +CPU= 27.008
  06:02:46 STEP 1364 H= 341:00 +CPU= 28.294
  06:03:15 STEP 1365 H= 341:15 +CPU= 27.569
  06:03:44 STEP 1366 H= 341:30 +CPU= 27.412
  06:04:12 STEP 1367 H= 341:45 +CPU= 27.847
  06:04:48 STEP 1368 H= 342:00 +CPU= 34.728
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001368
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001368
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001368
  06:05:16 STEP 1369 H= 342:15 +CPU= 27.833
  06:05:45 STEP 1370 H= 342:30 +CPU= 27.340
  06:06:13 STEP 1371 H= 342:45 +CPU= 27.030
  06:06:42 STEP 1372 H= 343:00 +CPU= 28.298
  06:07:10 STEP 1373 H= 343:15 +CPU= 27.597
  06:07:39 STEP 1374 H= 343:30 +CPU= 28.140
  06:08:08 STEP 1375 H= 343:45 +CPU= 28.251
  06:08:38 STEP 1376 H= 344:00 +CPU= 30.032
  06:09:06 STEP 1377 H= 344:15 +CPU= 27.287
  06:09:33 STEP 1378 H= 344:30 +CPU= 27.255
  06:10:03 STEP 1379 H= 344:45 +CPU= 28.283
  06:10:38 STEP 1380 H= 345:00 +CPU= 34.782
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001380
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001380
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001380
  06:11:08 STEP 1381 H= 345:15 +CPU= 28.509
  06:11:37 STEP 1382 H= 345:30 +CPU= 27.903
  06:12:05 STEP 1383 H= 345:45 +CPU= 27.216
  06:12:33 STEP 1384 H= 346:00 +CPU= 28.195
  06:13:04 STEP 1385 H= 346:15 +CPU= 27.920
  06:13:34 STEP 1386 H= 346:30 +CPU= 28.125
  06:14:02 STEP 1387 H= 346:45 +CPU= 27.781
  06:14:32 STEP 1388 H= 347:00 +CPU= 29.429
  06:15:01 STEP 1389 H= 347:15 +CPU= 27.613
  06:15:32 STEP 1390 H= 347:30 +CPU= 27.946
  06:16:03 STEP 1391 H= 347:45 +CPU= 29.887
  06:16:42 STEP 1392 H= 348:00 +CPU= 37.819
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001392
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001392
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001392
  06:17:11 STEP 1393 H= 348:15 +CPU= 27.761
  06:17:38 STEP 1394 H= 348:30 +CPU= 27.362
  06:18:06 STEP 1395 H= 348:45 +CPU= 27.586
  06:18:37 STEP 1396 H= 349:00 +CPU= 30.906
  06:19:06 STEP 1397 H= 349:15 +CPU= 27.835
  06:19:35 STEP 1398 H= 349:30 +CPU= 27.542
  06:20:04 STEP 1399 H= 349:45 +CPU= 27.816
  06:20:33 STEP 1400 H= 350:00 +CPU= 28.189
  06:21:00 STEP 1401 H= 350:15 +CPU= 27.365
  06:21:29 STEP 1402 H= 350:30 +CPU= 27.978
  06:21:58 STEP 1403 H= 350:45 +CPU= 28.329
  06:22:34 STEP 1404 H= 351:00 +CPU= 34.322
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001404
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001404
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001404
  06:23:04 STEP 1405 H= 351:15 +CPU= 30.080
  06:23:34 STEP 1406 H= 351:30 +CPU= 29.289
  06:24:03 STEP 1407 H= 351:45 +CPU= 27.829
  06:24:34 STEP 1408 H= 352:00 +CPU= 29.854
  06:25:03 STEP 1409 H= 352:15 +CPU= 27.432
  06:25:33 STEP 1410 H= 352:30 +CPU= 29.513
  06:26:01 STEP 1411 H= 352:45 +CPU= 27.413
  06:26:30 STEP 1412 H= 353:00 +CPU= 28.337
  06:26:59 STEP 1413 H= 353:15 +CPU= 27.991
  06:27:29 STEP 1414 H= 353:30 +CPU= 28.475
  06:27:57 STEP 1415 H= 353:45 +CPU= 27.048
  06:28:33 STEP 1416 H= 354:00 +CPU= 35.460
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001416
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001416
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001416
  06:29:03 STEP 1417 H= 354:15 +CPU= 28.987
  06:29:33 STEP 1418 H= 354:30 +CPU= 28.731
  06:30:02 STEP 1419 H= 354:45 +CPU= 28.723
  06:30:32 STEP 1420 H= 355:00 +CPU= 28.922
  06:31:00 STEP 1421 H= 355:15 +CPU= 27.585
  06:31:28 STEP 1422 H= 355:30 +CPU= 27.423
  06:31:58 STEP 1423 H= 355:45 +CPU= 29.957
  06:32:29 STEP 1424 H= 356:00 +CPU= 30.036
  06:32:59 STEP 1425 H= 356:15 +CPU= 28.096
  06:33:27 STEP 1426 H= 356:30 +CPU= 27.549
  06:33:57 STEP 1427 H= 356:45 +CPU= 28.378
  06:34:33 STEP 1428 H= 357:00 +CPU= 35.626
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001428
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001428
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001428
  06:35:03 STEP 1429 H= 357:15 +CPU= 28.892
  06:35:33 STEP 1430 H= 357:30 +CPU= 28.583
  06:36:03 STEP 1431 H= 357:45 +CPU= 28.168
  06:36:31 STEP 1432 H= 358:00 +CPU= 28.192
  06:37:00 STEP 1433 H= 358:15 +CPU= 28.970
  06:37:27 STEP 1434 H= 358:30 +CPU= 27.449
  06:37:56 STEP 1435 H= 358:45 +CPU= 28.306
  06:38:25 STEP 1436 H= 359:00 +CPU= 28.821
  06:38:53 STEP 1437 H= 359:15 +CPU= 27.906
  06:39:23 STEP 1438 H= 359:30 +CPU= 28.982
  06:39:52 STEP 1439 H= 359:45 +CPU= 28.013
  06:40:29 STEP 1440 H= 360:00 +CPU= 35.384
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  360.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I     -0.8084152624E-21      I       0.1302292536E-05      I
ICloud fractionI      0.0000000000E+00      I       0.1190449902E-01      I
I  Relat. Hum. I      0.2967107587E+00      I       0.3465624018E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.2366460169E+03      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1440 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001440 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001440
 MPI-TASK:   1 -    312595889 BYTES IN      7 RECORDS TRANSFERRED IN    0.0003 SECONDS******** Mbytes/s, TOTAL TIME=    0.0108( 2.8%)
 06:40:29 STEP 1440 H= 360:00 +CPU= 35.384
 END CNT3
 NSTEP =  1440 CNT0   000000000  25.997  25.997  27.090   0.960               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           34/  1069125708 WRITTEN        11344/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1    203    0       34      1044068    11344 ************    19.3     1.6    17.7     2.3    41.4
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 ********* *********    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 ********* *********     99.97      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  417    4726.6    4726.6     17.33      0.00
   9 SPCM     - SPECTRAL COMP.                     384     881.1     881.1      2.97      0.00
  10 SCAN2M - PHYSICS                              385   13234.6   13234.6     44.79      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 32     849.2     849.2      0.24      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 17      59.1      59.1      0.01      0.00
  17 GRIDFPOS IN CNT4                               32      32.8      32.8      0.01      0.00
  27 RESTART FILES - WRITING                         8    3396.7    3396.7      0.24      0.00
  28 RESTART FILES - READING                         1    2947.0    2947.0      0.03      0.00
  29 SU4FPOS IN CNT4                                32       0.1       0.1      0.00      0.00
  30 DYNFPOS IN CNT4                                32    6450.0    6450.0      1.81      0.00
  31 POSDDH IN STEPO                                 2       1.1       1.1      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                    385    6666.1    6666.1     22.56      0.00
  39 SU0YOMB                                         1     345.0     345.0      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                    385      25.6      25.6      0.09      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              384     375.0     375.0      1.27      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              384      67.9      67.9      0.23      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 17       0.2       0.2      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM      836     493.0     493.0      3.62      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM       800     290.4     290.4      2.04      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM        800     103.5     103.5      0.73      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM       836     153.4     153.4      1.13      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      37.7      37.7      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION            836      58.0      58.0      0.43      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION            800      31.0      31.0      0.22      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION            836      88.7      88.7      0.65      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION            800      21.0      21.0      0.15      0.00
 400 GSTATS                                      45376       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 42168       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME    11375.5 CPU TIME   11068.2 VECTOR TIME    11068.2


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY      114.0
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMGGh7zg+001440
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMSHh7zg+001440
Moving to projects directory: /mydisks/a/boinc_lib/slots/1/ICMUAh7zg+001440
Adding to the zip: /mydisks/a/boinc_lib/slots/1/NODE.001_01
Adding to the zip: /mydisks/a/boinc_lib/slots/1/ifs.stat
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+001344
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+001344
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+001344
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+001356
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+001356
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+001356
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+001368
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+001368
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+001368
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+001380
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+001380
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+001380
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+001392
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+001392
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+001392
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+001404
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+001404
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+001404
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+001416
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+001416
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+001416
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+001428
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+001428
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+001428
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMGGh7zg+001440
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMSHh7zg+001440
Adding to the zip: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_12211140/ICMUAh7zg+001440
Zipping up the final file: /mydisks/a/boinc_lib/projects/climateprediction.net/oifs_43r3_bl_a3cx_2016092300_15_991_12211140_1_r972307138_14.zip
Uploading the final file: upload_file_14.zip
Uploading trickle at timestep: 1295100
06:44:00 (20293): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:52:04 1517479 22314142 oifs_43r3_bl_a3cx_2016092300_15_991_12211140_1 1,295,100 41,895 0.0323
09 May 2023 16:52:03 1517479 22314142 oifs_43r3_bl_a3cx_2016092300_15_991_12211140_1 864,000 27,399 0.0317


©2024 climateprediction.net