climateprediction.net home page
Task 22440132

Task 22440132

Name oifs_43r3_bl_a37y_2016092300_20_1017_12286001_1
Workunit 12286001
Created 7 Jun 2024, 23:34:15 UTC
Sent 8 Jun 2024, 1:04:54 UTC
Report deadline 7 Aug 2024, 1:04:54 UTC
Received 8 Jun 2024, 19:18:44 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 0 (0x00000000)
Computer ID 1507812
Run time 14 hours 31 min 56 sec
CPU time 12 hours 49 min 58 sec
Validate state Invalid
Credit 1,976.32
Device peak FLOPS 5.39 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,499.65 MB
Peak swap size 5,980.81 MB
Peak disk usage 1,582.01 MB
Stderr
<core_client_version>7.24.1</core_client_version>
<![CDATA[
<stderr_txt>
03 STEP  944 H= 236:00 +CPU= 31.700
  15:33:40 STEP  945 H= 236:15 +CPU= 31.683
  15:34:15 STEP  946 H= 236:30 +CPU= 30.137
  15:34:48 STEP  947 H= 236:45 +CPU= 28.840
  15:35:32 STEP  948 H= 237:00 +CPU= 37.252
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+000948
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+000948
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+000948
  15:36:05 STEP  949 H= 237:15 +CPU= 29.064
  15:36:40 STEP  950 H= 237:30 +CPU= 28.999
  15:37:15 STEP  951 H= 237:45 +CPU= 28.832
  15:37:49 STEP  952 H= 238:00 +CPU= 29.771
  15:38:24 STEP  953 H= 238:15 +CPU= 30.323
  15:39:01 STEP  954 H= 238:30 +CPU= 31.717
  15:39:34 STEP  955 H= 238:45 +CPU= 29.637
  15:40:07 STEP  956 H= 239:00 +CPU= 30.380
  15:40:42 STEP  957 H= 239:15 +CPU= 29.298
  15:41:15 STEP  958 H= 239:30 +CPU= 28.553
  15:41:48 STEP  959 H= 239:45 +CPU= 28.026
  15:42:59 STEP  960 H= 240:00 +CPU= 36.070
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+000960
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+000960
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+000960
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+000864
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+000864
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+000864
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+000876
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+000876
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+000876
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+000888
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+000888
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+000888
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+000900
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+000900
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+000900
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+000912
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+000912
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+000912
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+000924
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+000924
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+000924
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+000936
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+000936
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+000936
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+000948
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+000948
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+000948
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a37y_2016092300_20_1017_12286001_1_r77867385_9.zip
Uploading the intermediate file: upload_file_9.zip
  15:43:36 STEP  961 H= 240:15 +CPU= 27.783
  15:44:08 STEP  962 H= 240:30 +CPU= 28.312
  15:44:41 STEP  963 H= 240:45 +CPU= 28.782
  15:45:16 STEP  964 H= 241:00 +CPU= 29.606
  15:45:50 STEP  965 H= 241:15 +CPU= 29.709
  15:46:26 STEP  966 H= 241:30 +CPU= 29.636
  15:47:02 STEP  967 H= 241:45 +CPU= 30.156
  15:47:38 STEP  968 H= 242:00 +CPU= 29.719
  15:48:10 STEP  969 H= 242:15 +CPU= 27.895
  15:48:42 STEP  970 H= 242:30 +CPU= 28.427
  15:49:19 STEP  971 H= 242:45 +CPU= 31.015
  15:50:07 STEP  972 H= 243:00 +CPU= 39.735
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+000972
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+000972
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+000972
  15:50:44 STEP  973 H= 243:15 +CPU= 32.058
  15:51:14 STEP  974 H= 243:30 +CPU= 27.108
  15:51:45 STEP  975 H= 243:45 +CPU= 27.659
  15:52:19 STEP  976 H= 244:00 +CPU= 30.239
  15:52:56 STEP  977 H= 244:15 +CPU= 32.047
  15:53:36 STEP  978 H= 244:30 +CPU= 33.471
  15:54:14 STEP  979 H= 244:45 +CPU= 32.229
  15:54:46 STEP  980 H= 245:00 +CPU= 29.648
  15:55:20 STEP  981 H= 245:15 +CPU= 29.595
  15:55:51 STEP  982 H= 245:30 +CPU= 28.085
  15:56:25 STEP  983 H= 245:45 +CPU= 28.374
  15:57:07 STEP  984 H= 246:00 +CPU= 35.574
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+000984
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+000984
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+000984
  15:57:40 STEP  985 H= 246:15 +CPU= 28.364
  15:58:14 STEP  986 H= 246:30 +CPU= 28.612
  15:58:49 STEP  987 H= 246:45 +CPU= 29.639
  15:59:24 STEP  988 H= 247:00 +CPU= 30.262
  16:00:00 STEP  989 H= 247:15 +CPU= 30.339
  16:00:35 STEP  990 H= 247:30 +CPU= 29.396
  16:01:10 STEP  991 H= 247:45 +CPU= 28.625
  16:01:45 STEP  992 H= 248:00 +CPU= 30.338
  16:02:17 STEP  993 H= 248:15 +CPU= 28.315
  16:02:51 STEP  994 H= 248:30 +CPU= 27.997
  16:03:26 STEP  995 H= 248:45 +CPU= 29.020
  16:04:09 STEP  996 H= 249:00 +CPU= 35.341
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+000996
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+000996
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+000996
  16:04:43 STEP  997 H= 249:15 +CPU= 27.876
  16:05:16 STEP  998 H= 249:30 +CPU= 28.370
  16:05:49 STEP  999 H= 249:45 +CPU= 28.662
  16:06:25 STEP 1000 H= 250:00 +CPU= 29.318
  16:07:02 STEP 1001 H= 250:15 +CPU= 29.502
  16:07:36 STEP 1002 H= 250:30 +CPU= 28.556
  16:08:11 STEP 1003 H= 250:45 +CPU= 28.900
  16:08:46 STEP 1004 H= 251:00 +CPU= 29.004
  16:09:20 STEP 1005 H= 251:15 +CPU= 29.078
  16:09:53 STEP 1006 H= 251:30 +CPU= 28.528
  16:10:26 STEP 1007 H= 251:45 +CPU= 27.435
  16:11:54 STEP 1008 H= 252:00 +CPU= 38.655
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001008
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001008
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001008
  16:12:31 STEP 1009 H= 252:15 +CPU= 30.258
  16:13:07 STEP 1010 H= 252:30 +CPU= 30.314
  16:13:41 STEP 1011 H= 252:45 +CPU= 28.879
  16:14:16 STEP 1012 H= 253:00 +CPU= 28.889
  16:14:49 STEP 1013 H= 253:15 +CPU= 27.840
  16:15:22 STEP 1014 H= 253:30 +CPU= 28.245
  16:15:55 STEP 1015 H= 253:45 +CPU= 28.414
  16:16:28 STEP 1016 H= 254:00 +CPU= 28.922
  16:17:02 STEP 1017 H= 254:15 +CPU= 28.921
  16:17:36 STEP 1018 H= 254:30 +CPU= 28.519
  16:18:09 STEP 1019 H= 254:45 +CPU= 28.737
  16:18:52 STEP 1020 H= 255:00 +CPU= 36.727
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001020
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001020
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001020
  16:19:28 STEP 1021 H= 255:15 +CPU= 30.165
  16:20:04 STEP 1022 H= 255:30 +CPU= 29.651
  16:20:42 STEP 1023 H= 255:45 +CPU= 30.471
  16:21:19 STEP 1024 H= 256:00 +CPU= 30.118
  16:21:54 STEP 1025 H= 256:15 +CPU= 29.444
  16:22:29 STEP 1026 H= 256:30 +CPU= 29.696
  16:23:03 STEP 1027 H= 256:45 +CPU= 29.102
  16:23:38 STEP 1028 H= 257:00 +CPU= 29.137
  16:24:12 STEP 1029 H= 257:15 +CPU= 28.950
  16:24:48 STEP 1030 H= 257:30 +CPU= 29.669
  16:25:23 STEP 1031 H= 257:45 +CPU= 29.838
  16:26:06 STEP 1032 H= 258:00 +CPU= 36.189
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001032
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001032
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001032
  16:26:43 STEP 1033 H= 258:15 +CPU= 30.194
  16:27:18 STEP 1034 H= 258:30 +CPU= 29.333
  16:27:54 STEP 1035 H= 258:45 +CPU= 29.358
  16:28:29 STEP 1036 H= 259:00 +CPU= 29.772
  16:29:05 STEP 1037 H= 259:15 +CPU= 29.385
  16:29:40 STEP 1038 H= 259:30 +CPU= 29.461
  16:30:15 STEP 1039 H= 259:45 +CPU= 29.393
  16:30:51 STEP 1040 H= 260:00 +CPU= 29.934
  16:31:27 STEP 1041 H= 260:15 +CPU= 29.645
  16:32:02 STEP 1042 H= 260:30 +CPU= 30.026
  16:32:38 STEP 1043 H= 260:45 +CPU= 30.797
  16:33:20 STEP 1044 H= 261:00 +CPU= 36.359
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001044
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001044
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001044
  16:33:57 STEP 1045 H= 261:15 +CPU= 30.698
  16:34:29 STEP 1046 H= 261:30 +CPU= 28.131
  16:35:02 STEP 1047 H= 261:45 +CPU= 28.369
  16:35:37 STEP 1048 H= 262:00 +CPU= 29.971
  16:36:12 STEP 1049 H= 262:15 +CPU= 29.685
  16:36:45 STEP 1050 H= 262:30 +CPU= 28.146
  16:37:18 STEP 1051 H= 262:45 +CPU= 28.043
  16:37:54 STEP 1052 H= 263:00 +CPU= 30.547
  16:38:26 STEP 1053 H= 263:15 +CPU= 28.345
  16:39:01 STEP 1054 H= 263:30 +CPU= 29.776
  16:39:34 STEP 1055 H= 263:45 +CPU= 28.415
  16:41:01 STEP 1056 H= 264:00 +CPU= 38.569
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001056
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001056
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+000960
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+000960
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+000960
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+000972
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+000972
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+000972
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+000984
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+000984
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+000984
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+000996
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+000996
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+000996
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+001008
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+001008
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+001008
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+001020
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+001020
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+001020
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+001032
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+001032
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+001032
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+001044
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+001044
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+001044
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a37y_2016092300_20_1017_12286001_1_r77867385_10.zip
Uploading the intermediate file: upload_file_10.zip
  16:41:36 STEP 1057 H= 264:15 +CPU= 28.892
  16:42:10 STEP 1058 H= 264:30 +CPU= 29.884
  16:42:44 STEP 1059 H= 264:45 +CPU= 29.815
  16:43:18 STEP 1060 H= 265:00 +CPU= 30.009
  16:43:51 STEP 1061 H= 265:15 +CPU= 29.836
  16:44:25 STEP 1062 H= 265:30 +CPU= 30.083
  16:44:59 STEP 1063 H= 265:45 +CPU= 30.256
  16:45:33 STEP 1064 H= 266:00 +CPU= 29.676
  16:46:08 STEP 1065 H= 266:15 +CPU= 30.511
  16:46:42 STEP 1066 H= 266:30 +CPU= 29.490
  16:47:16 STEP 1067 H= 266:45 +CPU= 29.205
  16:47:55 STEP 1068 H= 267:00 +CPU= 34.828
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001068
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001068
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001068
  16:48:28 STEP 1069 H= 267:15 +CPU= 29.082
  16:49:02 STEP 1070 H= 267:30 +CPU= 29.230
  16:49:36 STEP 1071 H= 267:45 +CPU= 29.432
  16:50:11 STEP 1072 H= 268:00 +CPU= 29.867
  16:50:45 STEP 1073 H= 268:15 +CPU= 29.337
  16:51:19 STEP 1074 H= 268:30 +CPU= 29.772
  16:51:53 STEP 1075 H= 268:45 +CPU= 29.814
  16:52:28 STEP 1076 H= 269:00 +CPU= 30.158
  16:53:01 STEP 1077 H= 269:15 +CPU= 29.549
  16:53:34 STEP 1078 H= 269:30 +CPU= 29.113
  16:54:07 STEP 1079 H= 269:45 +CPU= 29.135
  16:54:49 STEP 1080 H= 270:00 +CPU= 36.533
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001080
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001080
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001080
  16:55:22 STEP 1081 H= 270:15 +CPU= 29.359
  16:55:56 STEP 1082 H= 270:30 +CPU= 29.080
  16:56:30 STEP 1083 H= 270:45 +CPU= 28.590
  16:57:05 STEP 1084 H= 271:00 +CPU= 29.641
  16:57:39 STEP 1085 H= 271:15 +CPU= 29.786
  16:58:13 STEP 1086 H= 271:30 +CPU= 29.470
  16:58:47 STEP 1087 H= 271:45 +CPU= 29.729
  16:59:23 STEP 1088 H= 272:00 +CPU= 30.285
  16:59:57 STEP 1089 H= 272:15 +CPU= 29.671
  17:00:30 STEP 1090 H= 272:30 +CPU= 29.137
  17:01:03 STEP 1091 H= 272:45 +CPU= 29.309
  17:01:46 STEP 1092 H= 273:00 +CPU= 36.549
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001092
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001092
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001092
  17:02:21 STEP 1093 H= 273:15 +CPU= 29.335
  17:02:55 STEP 1094 H= 273:30 +CPU= 29.640
  17:03:29 STEP 1095 H= 273:45 +CPU= 29.536
  17:04:03 STEP 1096 H= 274:00 +CPU= 30.117
  17:04:39 STEP 1097 H= 274:15 +CPU= 30.112
  17:05:16 STEP 1098 H= 274:30 +CPU= 30.762
  17:05:50 STEP 1099 H= 274:45 +CPU= 29.872
  17:06:23 STEP 1100 H= 275:00 +CPU= 28.613
  17:06:58 STEP 1101 H= 275:15 +CPU= 29.458
  17:07:32 STEP 1102 H= 275:30 +CPU= 28.527
  17:08:04 STEP 1103 H= 275:45 +CPU= 27.891
  17:09:08 STEP 1104 H= 276:00 +CPU= 38.638
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001104
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001104
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001104
  17:09:41 STEP 1105 H= 276:15 +CPU= 29.043
  17:10:14 STEP 1106 H= 276:30 +CPU= 29.503
  17:10:48 STEP 1107 H= 276:45 +CPU= 29.266
  17:11:23 STEP 1108 H= 277:00 +CPU= 30.067
  17:11:59 STEP 1109 H= 277:15 +CPU= 30.150
  17:12:35 STEP 1110 H= 277:30 +CPU= 30.413
  17:13:10 STEP 1111 H= 277:45 +CPU= 29.726
  17:13:44 STEP 1112 H= 278:00 +CPU= 29.867
  17:14:19 STEP 1113 H= 278:15 +CPU= 29.587
  17:14:55 STEP 1114 H= 278:30 +CPU= 29.766
  17:15:30 STEP 1115 H= 278:45 +CPU= 29.329
  17:16:09 STEP 1116 H= 279:00 +CPU= 34.120
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001116
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001116
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001116
  17:16:44 STEP 1117 H= 279:15 +CPU= 28.861
  17:17:21 STEP 1118 H= 279:30 +CPU= 29.104
  17:17:57 STEP 1119 H= 279:45 +CPU= 29.659
  17:18:33 STEP 1120 H= 280:00 +CPU= 30.382
  17:19:08 STEP 1121 H= 280:15 +CPU= 29.955
  17:19:44 STEP 1122 H= 280:30 +CPU= 29.535
  17:20:20 STEP 1123 H= 280:45 +CPU= 29.413
  17:20:55 STEP 1124 H= 281:00 +CPU= 29.958
  17:21:28 STEP 1125 H= 281:15 +CPU= 29.384
  17:22:03 STEP 1126 H= 281:30 +CPU= 29.614
  17:22:39 STEP 1127 H= 281:45 +CPU= 29.253
  17:23:24 STEP 1128 H= 282:00 +CPU= 36.719
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001128
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001128
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001128
  17:23:59 STEP 1129 H= 282:15 +CPU= 29.557
  17:24:32 STEP 1130 H= 282:30 +CPU= 29.922
  17:25:08 STEP 1131 H= 282:45 +CPU= 30.253
  17:25:45 STEP 1132 H= 283:00 +CPU= 31.082
  17:26:21 STEP 1133 H= 283:15 +CPU= 30.153
  17:26:54 STEP 1134 H= 283:30 +CPU= 29.519
  17:27:29 STEP 1135 H= 283:45 +CPU= 29.370
  17:28:05 STEP 1136 H= 284:00 +CPU= 30.108
  17:28:41 STEP 1137 H= 284:15 +CPU= 29.892
  17:29:16 STEP 1138 H= 284:30 +CPU= 29.431
  17:29:52 STEP 1139 H= 284:45 +CPU= 29.458
  17:30:38 STEP 1140 H= 285:00 +CPU= 37.123
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001140
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001140
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001140
  17:31:13 STEP 1141 H= 285:15 +CPU= 29.837
  17:31:49 STEP 1142 H= 285:30 +CPU= 30.410
  17:32:26 STEP 1143 H= 285:45 +CPU= 30.518
  17:33:03 STEP 1144 H= 286:00 +CPU= 30.692
  17:33:38 STEP 1145 H= 286:15 +CPU= 29.065
  17:34:14 STEP 1146 H= 286:30 +CPU= 29.656
  17:34:47 STEP 1147 H= 286:45 +CPU= 28.203
  17:35:23 STEP 1148 H= 287:00 +CPU= 29.457
  17:36:00 STEP 1149 H= 287:15 +CPU= 29.573
  17:36:32 STEP 1150 H= 287:30 +CPU= 27.651
  17:37:05 STEP 1151 H= 287:45 +CPU= 28.044
  17:38:21 STEP 1152 H= 288:00 +CPU= 40.136
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001152
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001152
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+001056
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+001056
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+001068
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+001068
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+001068
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+001080
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+001080
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+001080
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+001092
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+001092
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+001092
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+001104
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+001104
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+001104
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+001116
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+001116
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+001116
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+001128
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+001128
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+001128
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+001140
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+001140
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+001140
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a37y_2016092300_20_1017_12286001_1_r77867385_11.zip
Uploading the intermediate file: upload_file_11.zip
  17:38:59 STEP 1153 H= 288:15 +CPU= 29.976
  17:39:37 STEP 1154 H= 288:30 +CPU= 30.707
  17:40:13 STEP 1155 H= 288:45 +CPU= 30.084
  17:40:51 STEP 1156 H= 289:00 +CPU= 30.558
  17:41:26 STEP 1157 H= 289:15 +CPU= 29.854
  17:42:00 STEP 1158 H= 289:30 +CPU= 29.566
  17:42:35 STEP 1159 H= 289:45 +CPU= 29.425
  17:43:11 STEP 1160 H= 290:00 +CPU= 30.049
  17:43:47 STEP 1161 H= 290:15 +CPU= 29.867
  17:44:22 STEP 1162 H= 290:30 +CPU= 29.515
  17:44:59 STEP 1163 H= 290:45 +CPU= 30.307
  17:45:47 STEP 1164 H= 291:00 +CPU= 36.610
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001164
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001164
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001164
  17:46:23 STEP 1165 H= 291:15 +CPU= 30.072
  17:46:57 STEP 1166 H= 291:30 +CPU= 29.241
  17:47:31 STEP 1167 H= 291:45 +CPU= 29.305
  17:48:05 STEP 1168 H= 292:00 +CPU= 29.551
  17:48:40 STEP 1169 H= 292:15 +CPU= 29.314
  17:49:15 STEP 1170 H= 292:30 +CPU= 29.378
  17:49:50 STEP 1171 H= 292:45 +CPU= 29.516
  17:50:26 STEP 1172 H= 293:00 +CPU= 30.184
  17:51:00 STEP 1173 H= 293:15 +CPU= 29.871
  17:51:36 STEP 1174 H= 293:30 +CPU= 29.944
  17:52:12 STEP 1175 H= 293:45 +CPU= 30.484
  17:52:58 STEP 1176 H= 294:00 +CPU= 38.336
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001176
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001176
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001176
  17:53:33 STEP 1177 H= 294:15 +CPU= 29.660
  17:54:08 STEP 1178 H= 294:30 +CPU= 29.591
  17:54:43 STEP 1179 H= 294:45 +CPU= 29.601
  17:55:19 STEP 1180 H= 295:00 +CPU= 29.964
  17:55:55 STEP 1181 H= 295:15 +CPU= 29.628
  17:56:30 STEP 1182 H= 295:30 +CPU= 29.700
  17:57:06 STEP 1183 H= 295:45 +CPU= 29.713
  17:57:42 STEP 1184 H= 296:00 +CPU= 30.550
  17:58:19 STEP 1185 H= 296:15 +CPU= 30.254
  17:58:56 STEP 1186 H= 296:30 +CPU= 30.671
  17:59:33 STEP 1187 H= 296:45 +CPU= 30.739
  18:00:20 STEP 1188 H= 297:00 +CPU= 37.670
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001188
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001188
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001188
  18:00:56 STEP 1189 H= 297:15 +CPU= 29.639
  18:01:33 STEP 1190 H= 297:30 +CPU= 29.803
  18:02:09 STEP 1191 H= 297:45 +CPU= 29.535
  18:02:46 STEP 1192 H= 298:00 +CPU= 30.358
  18:03:23 STEP 1193 H= 298:15 +CPU= 29.546
  18:03:58 STEP 1194 H= 298:30 +CPU= 29.168
  18:04:33 STEP 1195 H= 298:45 +CPU= 29.122
  18:05:10 STEP 1196 H= 299:00 +CPU= 30.343
  18:05:45 STEP 1197 H= 299:15 +CPU= 29.001
  18:06:21 STEP 1198 H= 299:30 +CPU= 30.231
  18:06:56 STEP 1199 H= 299:45 +CPU= 29.319
  18:08:00 STEP 1200 H= 300:00 +CPU= 38.894
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001200
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001200
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001200
  18:08:36 STEP 1201 H= 300:15 +CPU= 30.033
  18:09:12 STEP 1202 H= 300:30 +CPU= 29.754
  18:09:48 STEP 1203 H= 300:45 +CPU= 29.867
  18:10:25 STEP 1204 H= 301:00 +CPU= 30.349
  18:11:01 STEP 1205 H= 301:15 +CPU= 30.099
  18:11:39 STEP 1206 H= 301:30 +CPU= 30.284
  18:12:15 STEP 1207 H= 301:45 +CPU= 30.204
  18:12:53 STEP 1208 H= 302:00 +CPU= 31.213
  18:13:31 STEP 1209 H= 302:15 +CPU= 31.043
  18:14:10 STEP 1210 H= 302:30 +CPU= 30.236
  18:14:49 STEP 1211 H= 302:45 +CPU= 30.059
  18:15:43 STEP 1212 H= 303:00 +CPU= 36.089
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001212
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001212
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001212
  18:16:20 STEP 1213 H= 303:15 +CPU= 29.298
  18:16:57 STEP 1214 H= 303:30 +CPU= 29.678
  18:17:35 STEP 1215 H= 303:45 +CPU= 29.896
  18:18:12 STEP 1216 H= 304:00 +CPU= 30.568
  18:18:48 STEP 1217 H= 304:15 +CPU= 30.040
  18:19:24 STEP 1218 H= 304:30 +CPU= 30.249
  18:19:59 STEP 1219 H= 304:45 +CPU= 30.469
  18:20:35 STEP 1220 H= 305:00 +CPU= 31.263
  18:21:11 STEP 1221 H= 305:15 +CPU= 30.044
  18:21:46 STEP 1222 H= 305:30 +CPU= 29.882
  18:22:21 STEP 1223 H= 305:45 +CPU= 29.860
  18:23:03 STEP 1224 H= 306:00 +CPU= 36.857
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001224
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001224
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001224
  18:23:36 STEP 1225 H= 306:15 +CPU= 29.429
  18:24:09 STEP 1226 H= 306:30 +CPU= 29.583
  18:24:44 STEP 1227 H= 306:45 +CPU= 29.778
  18:25:18 STEP 1228 H= 307:00 +CPU= 30.350
  18:25:52 STEP 1229 H= 307:15 +CPU= 30.134
  18:26:26 STEP 1230 H= 307:30 +CPU= 30.074
  18:27:03 STEP 1231 H= 307:45 +CPU= 30.470
  18:27:39 STEP 1232 H= 308:00 +CPU= 30.179
  18:28:13 STEP 1233 H= 308:15 +CPU= 29.623
  18:28:48 STEP 1234 H= 308:30 +CPU= 29.831
  18:29:23 STEP 1235 H= 308:45 +CPU= 29.923
  18:30:05 STEP 1236 H= 309:00 +CPU= 36.654
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001236
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001236
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001236
  18:30:39 STEP 1237 H= 309:15 +CPU= 29.480
  18:31:13 STEP 1238 H= 309:30 +CPU= 29.792
  18:31:48 STEP 1239 H= 309:45 +CPU= 29.783
  18:32:24 STEP 1240 H= 310:00 +CPU= 30.846
  18:32:59 STEP 1241 H= 310:15 +CPU= 29.888
  18:33:32 STEP 1242 H= 310:30 +CPU= 29.240
  18:34:06 STEP 1243 H= 310:45 +CPU= 29.774
  18:34:42 STEP 1244 H= 311:00 +CPU= 29.246
  18:35:15 STEP 1245 H= 311:15 +CPU= 28.925
  18:35:50 STEP 1246 H= 311:30 +CPU= 28.991
  18:36:23 STEP 1247 H= 311:45 +CPU= 28.243
  18:37:22 STEP 1248 H= 312:00 +CPU= 40.336
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001248
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001248
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001248
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+001152
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+001152
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+001164
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+001164
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+001164
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+001176
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+001176
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+001176
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+001188
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+001188
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+001188
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+001200
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+001200
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+001200
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+001212
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+001212
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+001212
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+001224
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+001224
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+001224
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+001236
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+001236
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+001236
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a37y_2016092300_20_1017_12286001_1_r77867385_12.zip
Uploading the intermediate file: upload_file_12.zip
  18:37:58 STEP 1249 H= 312:15 +CPU= 29.562
  18:38:33 STEP 1250 H= 312:30 +CPU= 30.077
  18:39:09 STEP 1251 H= 312:45 +CPU= 30.264
  18:39:46 STEP 1252 H= 313:00 +CPU= 30.960
  18:40:22 STEP 1253 H= 313:15 +CPU= 30.544
  18:40:56 STEP 1254 H= 313:30 +CPU= 29.405
  18:41:31 STEP 1255 H= 313:45 +CPU= 29.665
  18:42:05 STEP 1256 H= 314:00 +CPU= 30.008
  18:42:39 STEP 1257 H= 314:15 +CPU= 29.580
  18:43:14 STEP 1258 H= 314:30 +CPU= 29.793
  18:43:48 STEP 1259 H= 314:45 +CPU= 29.354
  18:44:50 STEP 1260 H= 315:00 +CPU= 36.700
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001260
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001260
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001260
  18:45:26 STEP 1261 H= 315:15 +CPU= 30.201
  18:46:02 STEP 1262 H= 315:30 +CPU= 30.542
  18:46:39 STEP 1263 H= 315:45 +CPU= 30.748
  18:47:15 STEP 1264 H= 316:00 +CPU= 30.248
  18:47:50 STEP 1265 H= 316:15 +CPU= 29.788
  18:48:24 STEP 1266 H= 316:30 +CPU= 29.557
  18:48:58 STEP 1267 H= 316:45 +CPU= 29.651
  18:49:33 STEP 1268 H= 317:00 +CPU= 30.193
  18:50:08 STEP 1269 H= 317:15 +CPU= 29.989
  18:50:43 STEP 1270 H= 317:30 +CPU= 29.925
  18:51:17 STEP 1271 H= 317:45 +CPU= 29.861
  18:52:02 STEP 1272 H= 318:00 +CPU= 38.187
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001272
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001272
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001272
  18:52:37 STEP 1273 H= 318:15 +CPU= 30.471
  18:53:14 STEP 1274 H= 318:30 +CPU= 31.023
  18:53:48 STEP 1275 H= 318:45 +CPU= 29.986
  18:54:23 STEP 1276 H= 319:00 +CPU= 30.028
  18:54:56 STEP 1277 H= 319:15 +CPU= 29.161
  18:55:30 STEP 1278 H= 319:30 +CPU= 29.394
  18:56:04 STEP 1279 H= 319:45 +CPU= 29.703
  18:56:39 STEP 1280 H= 320:00 +CPU= 29.645
  18:57:14 STEP 1281 H= 320:15 +CPU= 29.693
  18:57:48 STEP 1282 H= 320:30 +CPU= 29.842
  18:58:23 STEP 1283 H= 320:45 +CPU= 30.049
  18:59:08 STEP 1284 H= 321:00 +CPU= 38.288
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001284
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001284
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001284
  18:59:45 STEP 1285 H= 321:15 +CPU= 31.009
  19:00:20 STEP 1286 H= 321:30 +CPU= 29.674
  19:00:55 STEP 1287 H= 321:45 +CPU= 29.618
  19:01:29 STEP 1288 H= 322:00 +CPU= 29.463
  19:02:01 STEP 1289 H= 322:15 +CPU= 28.139
  19:02:35 STEP 1290 H= 322:30 +CPU= 29.057
  19:03:09 STEP 1291 H= 322:45 +CPU= 29.475
  19:03:41 STEP 1292 H= 323:00 +CPU= 28.562
  19:04:14 STEP 1293 H= 323:15 +CPU= 29.430
  19:04:43 STEP 1294 H= 323:30 +CPU= 27.117
  19:05:16 STEP 1295 H= 323:45 +CPU= 28.817
  19:06:26 STEP 1296 H= 324:00 +CPU= 43.092
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001296
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001296
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001296
  19:07:04 STEP 1297 H= 324:15 +CPU= 31.722
  19:07:43 STEP 1298 H= 324:30 +CPU= 31.922
  19:08:16 STEP 1299 H= 324:45 +CPU= 30.090
  19:08:49 STEP 1300 H= 325:00 +CPU= 30.163
  19:09:23 STEP 1301 H= 325:15 +CPU= 29.444
  19:09:56 STEP 1302 H= 325:30 +CPU= 30.314
  19:10:35 STEP 1303 H= 325:45 +CPU= 32.460
  19:11:14 STEP 1304 H= 326:00 +CPU= 32.953
  19:11:49 STEP 1305 H= 326:15 +CPU= 31.481
  19:12:23 STEP 1306 H= 326:30 +CPU= 30.779
  19:12:56 STEP 1307 H= 326:45 +CPU= 29.109
  19:13:47 STEP 1308 H= 327:00 +CPU= 35.434
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001308
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001308
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001308
  19:14:17 STEP 1309 H= 327:15 +CPU= 27.309
  19:14:45 STEP 1310 H= 327:30 +CPU= 25.633
  19:15:15 STEP 1311 H= 327:45 +CPU= 25.805
  19:15:43 STEP 1312 H= 328:00 +CPU= 25.741
  19:16:11 STEP 1313 H= 328:15 +CPU= 25.492
  19:16:40 STEP 1314 H= 328:30 +CPU= 26.168
  19:17:09 STEP 1315 H= 328:45 +CPU= 26.489
  19:17:39 STEP 1316 H= 329:00 +CPU= 26.858
  19:18:10 STEP 1317 H= 329:15 +CPU= 26.604
  19:18:43 STEP 1318 H= 329:30 +CPU= 28.612
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a37y
(argv4) batchid: 1017
(argv5) wuid: 12286001
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/2
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/2/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/2/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_5b323ef1cdf7e4736f415119507e0a62 to: /var/lib/boinc/slots/2/oifs_43r3_bl_a37y_2016092300_20_1017_12286001.zip
Unzipping the namelist zip file: /var/lib/boinc/slots/2/oifs_43r3_bl_a37y_2016092300_20_1017_12286001.zip
ic_ancil_file: ic_ancil_12286001
ifsdata_file: ifsdata_12286001
climate_data_file: clim_data_12286001
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc/slots/2/ic_ancil_12286001.zip
Unzipping the IC ancils zip file: /var/lib/boinc/slots/2/ic_ancil_12286001.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_392df65a0a6fd9fa155b59973138f8c8 to: /var/lib/boinc/slots/2/ifsdata/ifsdata_12286001.zip
Unzipping the ifsdata_zip file: /var/lib/boinc/slots/2/ifsdata/ifsdata_12286001.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/2/159l_2/clim_data_12286001.zip
Unzipping the climate data zip file: /var/lib/boinc/slots/2/159l_2/clim_data_12286001.zip
Checking for progress XML file: /var/lib/boinc/slots/2/progress_file_12286001.xml
Opened progress file ok : /var/lib/boinc/slots/2/progress_file_12286001.xml
-- Model is restarting --
Adjusting last_iter, 1319, to previous model restart step.
Creating progress file: /var/lib/boinc/slots/2/progress_file_12286001.xml
last_cpu_time: 41760
upload_file_number: 13
last_iter: 1297
last_upload: 1123200
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a37y_2016092300_20_1017_12286001_1_r77867385
The child process has been launched with process id: 1905218
Executing the command: /var/lib/boinc/slots/2/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:zion.faskatech.net:1:1:1905218:1905218] [20240608:192045:1717867245:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:zion.faskatech.net:1:1:1905218:1905218] [20240608:192045:1717867245:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:zion.faskatech.net:1:1:1905218:1905218] [20240608:192045:1717867245:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:zion.faskatech.net:1:1:1905218:1905218] [20240608:192045:1717867245:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:zion.faskatech.net:1:1:1905218:1905218] [20240608:192045:1717867245:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:zion.faskatech.net:1:1:1905218:1905218] [20240608:192045:1717867245:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:zion.faskatech.net:1:1:1905218:1905218] [20240608:192045:1717867245:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:zion.faskatech.net:1:1:1905218:1905218] [20240608:192045:1717867245:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:zion.faskatech.net:1:1:1905218:1905218] [20240608:192045:1717867245:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:zion.faskatech.net:1:1:1905218:1905218] [20240608:192045:1717867245:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:zion.faskatech.net:1:1:1905218:1905218] [20240608:192045:1717867245:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:zion.faskatech.net:1:1:1905218:1905218] [20240608:192045:1717867245:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:zion.faskatech.net:1:1:1905218:1905218] [20240608:192045:1717867245:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:zion.faskatech.net:1:1:1905218:1905218] [20240608:192045:1717867245:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:zion.faskatech.net:1:1:1905218:1905218] [20240608:192045:1717867245:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:zion.faskatech.net:1:1:1905218:1905218] [20240608:192045:1717867245:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:zion.faskatech.net:1:1:1905218:1905218] [20240608:192045:1717867245:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:zion.faskatech.net:1:1:1905218:1905218] [20240608:192045:1717867245:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:zion.faskatech.net:1:1:1905218:1905218] [20240608:192045:1717867245:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:zion.faskatech.net:1:1:1905218:1905218] [20240608:192045:1717867245:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:zion.faskatech.net:1:1:1905218:1905218] [20240608:192045:1717867245:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:zion.faskatech.net:1:1:1905218:1905218] [20240608:192045:1717867245:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:zion.faskatech.net:1:1:1905218:1905218] [20240608:192045:1717867245:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:zion.faskatech.net:1:1:1905218:1905218] [20240608:192045:1717867245:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:zion.faskatech.net:1:1:1905218:1905218] [20240608:192045:1717867245:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:zion.faskatech.net:1:1:1905218:1905218] [20240608:192045:1717867245:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:zion.faskatech.net:1:1:1905218:1905218] [20240608:192045:1717867245:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:zion.faskatech.net:1:1:1905218:1905218] [20240608:192045:1717867245:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:zion.faskatech.net:1:1:1905218:1905218] [20240608:192045:1717867245:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:zion.faskatech.net:1:1:1905218:1905218] [20240608:192045:1717867245:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  19:21:58 STEP 1296 H= 324:00 +CPU= 29.268
  19:22:32 STEP 1297 H= 324:15 +CPU= 30.048
  19:23:07 STEP 1298 H= 324:30 +CPU= 30.488
  19:23:42 STEP 1299 H= 324:45 +CPU= 30.264
  19:24:16 STEP 1300 H= 325:00 +CPU= 30.497
  19:24:50 STEP 1301 H= 325:15 +CPU= 29.567
  19:25:22 STEP 1302 H= 325:30 +CPU= 29.076
  19:25:56 STEP 1303 H= 325:45 +CPU= 29.491
  19:26:30 STEP 1304 H= 326:00 +CPU= 29.479
  19:27:03 STEP 1305 H= 326:15 +CPU= 28.982
  19:27:39 STEP 1306 H= 326:30 +CPU= 29.847
  19:28:13 STEP 1307 H= 326:45 +CPU= 30.241
  19:28:56 STEP 1308 H= 327:00 +CPU= 36.950
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001308
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001308
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001308
  19:29:31 STEP 1309 H= 327:15 +CPU= 30.183
  19:30:06 STEP 1310 H= 327:30 +CPU= 30.653
  19:30:40 STEP 1311 H= 327:45 +CPU= 30.135
  19:31:14 STEP 1312 H= 328:00 +CPU= 29.968
  19:31:48 STEP 1313 H= 328:15 +CPU= 29.254
  19:32:21 STEP 1314 H= 328:30 +CPU= 29.427
  19:32:55 STEP 1315 H= 328:45 +CPU= 29.789
  19:33:30 STEP 1316 H= 329:00 +CPU= 30.073
  19:34:03 STEP 1317 H= 329:15 +CPU= 29.884
  19:34:38 STEP 1318 H= 329:30 +CPU= 30.250
  19:35:12 STEP 1319 H= 329:45 +CPU= 30.530
  19:35:56 STEP 1320 H= 330:00 +CPU= 37.827
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001320
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001320
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001320
  19:36:31 STEP 1321 H= 330:15 +CPU= 30.405
  19:37:05 STEP 1322 H= 330:30 +CPU= 29.960
  19:37:40 STEP 1323 H= 330:45 +CPU= 29.586
  19:38:14 STEP 1324 H= 331:00 +CPU= 30.160
  19:38:48 STEP 1325 H= 331:15 +CPU= 29.774
  19:39:22 STEP 1326 H= 331:30 +CPU= 29.811
  19:39:57 STEP 1327 H= 331:45 +CPU= 29.937
  19:40:32 STEP 1328 H= 332:00 +CPU= 30.437
  19:41:05 STEP 1329 H= 332:15 +CPU= 29.785
  19:41:41 STEP 1330 H= 332:30 +CPU= 30.288
  19:42:16 STEP 1331 H= 332:45 +CPU= 30.380
  19:43:00 STEP 1332 H= 333:00 +CPU= 38.179
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001332
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001332
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001332
  19:43:35 STEP 1333 H= 333:15 +CPU= 30.170
  19:44:09 STEP 1334 H= 333:30 +CPU= 29.333
  19:44:42 STEP 1335 H= 333:45 +CPU= 29.348
  19:45:15 STEP 1336 H= 334:00 +CPU= 29.434
  19:45:49 STEP 1337 H= 334:15 +CPU= 29.145
  19:46:23 STEP 1338 H= 334:30 +CPU= 29.631
  19:46:57 STEP 1339 H= 334:45 +CPU= 29.603
  19:47:32 STEP 1340 H= 335:00 +CPU= 30.510
  19:48:04 STEP 1341 H= 335:15 +CPU= 28.457
  19:48:38 STEP 1342 H= 335:30 +CPU= 29.407
  19:49:11 STEP 1343 H= 335:45 +CPU= 28.787
  19:50:40 STEP 1344 H= 336:00 +CPU= 38.838
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001344
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001344
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+001248
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+001248
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+001248
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+001260
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+001260
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+001260
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+001272
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+001272
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+001272
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+001284
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+001284
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+001284
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+001296
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+001296
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+001296
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+001308
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+001308
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+001308
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+001320
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+001320
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+001320
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+001332
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+001332
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+001332
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a37y_2016092300_20_1017_12286001_1_r77867385_13.zip
Uploading the intermediate file: upload_file_13.zip
  19:51:17 STEP 1345 H= 336:15 +CPU= 29.008
  19:51:53 STEP 1346 H= 336:30 +CPU= 29.854
  19:52:28 STEP 1347 H= 336:45 +CPU= 29.984
  19:53:03 STEP 1348 H= 337:00 +CPU= 30.350
  19:53:38 STEP 1349 H= 337:15 +CPU= 29.935
  19:54:12 STEP 1350 H= 337:30 +CPU= 29.698
  19:54:48 STEP 1351 H= 337:45 +CPU= 30.027
  19:55:24 STEP 1352 H= 338:00 +CPU= 31.099
  19:56:01 STEP 1353 H= 338:15 +CPU= 31.280
  19:56:38 STEP 1354 H= 338:30 +CPU= 30.471
  19:57:14 STEP 1355 H= 338:45 +CPU= 29.954
  19:57:59 STEP 1356 H= 339:00 +CPU= 37.289
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001356
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001356
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001356
  19:58:34 STEP 1357 H= 339:15 +CPU= 30.133
  19:59:09 STEP 1358 H= 339:30 +CPU= 29.946
  19:59:43 STEP 1359 H= 339:45 +CPU= 29.944
  20:00:19 STEP 1360 H= 340:00 +CPU= 30.624
  20:00:54 STEP 1361 H= 340:15 +CPU= 30.094
  20:01:29 STEP 1362 H= 340:30 +CPU= 30.633
  20:02:05 STEP 1363 H= 340:45 +CPU= 30.910
  20:02:42 STEP 1364 H= 341:00 +CPU= 31.568
  20:03:18 STEP 1365 H= 341:15 +CPU= 30.512
  20:03:55 STEP 1366 H= 341:30 +CPU= 30.159
  20:04:30 STEP 1367 H= 341:45 +CPU= 29.427
  20:05:13 STEP 1368 H= 342:00 +CPU= 37.365
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001368
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001368
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001368
  20:05:50 STEP 1369 H= 342:15 +CPU= 30.179
  20:06:25 STEP 1370 H= 342:30 +CPU= 30.164
  20:07:00 STEP 1371 H= 342:45 +CPU= 29.952
  20:07:36 STEP 1372 H= 343:00 +CPU= 30.800
  20:08:12 STEP 1373 H= 343:15 +CPU= 30.741
  20:08:47 STEP 1374 H= 343:30 +CPU= 30.519
  20:09:24 STEP 1375 H= 343:45 +CPU= 31.034
  20:09:59 STEP 1376 H= 344:00 +CPU= 30.527
  20:10:33 STEP 1377 H= 344:15 +CPU= 29.773
  20:11:07 STEP 1378 H= 344:30 +CPU= 29.837
  20:11:42 STEP 1379 H= 344:45 +CPU= 29.937
  20:12:24 STEP 1380 H= 345:00 +CPU= 36.803
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001380
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001380
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001380
  20:12:58 STEP 1381 H= 345:15 +CPU= 29.866
  20:13:33 STEP 1382 H= 345:30 +CPU= 29.958
  20:14:07 STEP 1383 H= 345:45 +CPU= 30.129
  20:14:43 STEP 1384 H= 346:00 +CPU= 30.937
  20:15:18 STEP 1385 H= 346:15 +CPU= 30.894
  20:15:55 STEP 1386 H= 346:30 +CPU= 31.354
  20:16:30 STEP 1387 H= 346:45 +CPU= 30.158
  20:17:00 STEP 1388 H= 347:00 +CPU= 27.527
  20:17:33 STEP 1389 H= 347:15 +CPU= 28.374
  20:18:04 STEP 1390 H= 347:30 +CPU= 26.808
  20:18:36 STEP 1391 H= 347:45 +CPU= 28.673
  20:19:46 STEP 1392 H= 348:00 +CPU= 40.201
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001392
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001392
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001392
  20:20:21 STEP 1393 H= 348:15 +CPU= 30.165
  20:20:57 STEP 1394 H= 348:30 +CPU= 30.457
  20:21:33 STEP 1395 H= 348:45 +CPU= 30.815
  20:22:10 STEP 1396 H= 349:00 +CPU= 31.517
  20:22:46 STEP 1397 H= 349:15 +CPU= 30.893
  20:23:20 STEP 1398 H= 349:30 +CPU= 29.804
  20:23:55 STEP 1399 H= 349:45 +CPU= 30.260
  20:24:31 STEP 1400 H= 350:00 +CPU= 30.572
  20:25:05 STEP 1401 H= 350:15 +CPU= 30.223
  20:25:40 STEP 1402 H= 350:30 +CPU= 29.834
  20:26:15 STEP 1403 H= 350:45 +CPU= 30.125
  20:27:00 STEP 1404 H= 351:00 +CPU= 37.903
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001404
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001404
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001404
  20:27:37 STEP 1405 H= 351:15 +CPU= 30.480
  20:28:15 STEP 1406 H= 351:30 +CPU= 31.023
  20:28:51 STEP 1407 H= 351:45 +CPU= 31.186
  20:29:27 STEP 1408 H= 352:00 +CPU= 31.186
  20:30:02 STEP 1409 H= 352:15 +CPU= 30.111
  20:30:36 STEP 1410 H= 352:30 +CPU= 29.966
  20:31:11 STEP 1411 H= 352:45 +CPU= 30.014
  20:31:45 STEP 1412 H= 353:00 +CPU= 30.640
  20:32:20 STEP 1413 H= 353:15 +CPU= 30.280
  20:32:54 STEP 1414 H= 353:30 +CPU= 30.094
  20:33:30 STEP 1415 H= 353:45 +CPU= 30.399
  20:34:13 STEP 1416 H= 354:00 +CPU= 38.033
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001416
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001416
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001416
  20:34:49 STEP 1417 H= 354:15 +CPU= 30.889
  20:35:24 STEP 1418 H= 354:30 +CPU= 31.131
  20:35:59 STEP 1419 H= 354:45 +CPU= 30.291
  20:36:33 STEP 1420 H= 355:00 +CPU= 30.093
  20:37:06 STEP 1421 H= 355:15 +CPU= 29.669
  20:37:41 STEP 1422 H= 355:30 +CPU= 29.654
  20:38:16 STEP 1423 H= 355:45 +CPU= 30.011
  20:38:50 STEP 1424 H= 356:00 +CPU= 30.547
  20:39:25 STEP 1425 H= 356:15 +CPU= 30.136
  20:40:00 STEP 1426 H= 356:30 +CPU= 29.977
  20:40:36 STEP 1427 H= 356:45 +CPU= 30.741
  20:41:20 STEP 1428 H= 357:00 +CPU= 38.372
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001428
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001428
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001428
  20:41:57 STEP 1429 H= 357:15 +CPU= 31.144
  20:42:32 STEP 1430 H= 357:30 +CPU= 30.374
  20:43:06 STEP 1431 H= 357:45 +CPU= 29.918
  20:43:41 STEP 1432 H= 358:00 +CPU= 30.279
  20:44:17 STEP 1433 H= 358:15 +CPU= 29.917
  20:44:52 STEP 1434 H= 358:30 +CPU= 30.043
  20:45:27 STEP 1435 H= 358:45 +CPU= 29.990
  20:46:01 STEP 1436 H= 359:00 +CPU= 29.345
  20:46:30 STEP 1437 H= 359:15 +CPU= 26.741
  20:46:55 STEP 1438 H= 359:30 +CPU= 23.145
  20:47:29 STEP 1439 H= 359:45 +CPU= 28.942
  20:48:30 STEP 1440 H= 360:00 +CPU= 28.812
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  360.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I      0.1047676801E-20      I       0.6229053782E-07      I
ICloud fractionI      0.0000000000E+00      I       0.1909840886E-01      I
I  Relat. Hum. I      0.3746467889E+00      I       0.4073030069E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.6430911634E+02      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1440 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001440 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001440
 MPI-TASK:   1 -    723280692 BYTES IN      7 RECORDS TRANSFERRED IN    0.0002 SECONDS******** Mbytes/s, TOTAL TIME=    0.0105( 1.9%)
 20:48:30 STEP 1440 H= 360:00 +CPU= 28.812
 END CNT3
 NSTEP =  1440 CNT0   000000000  22.008  22.008  24.571   0.896               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           46/  1069125708 WRITTEN         4290/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1     78    0       46      1044068     4290 ************   103.8    38.2    65.5    38.7    92.3
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 5266065.0 5266065.0    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 5226125.7 5226125.7     99.24      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  157    6112.4    6112.4     18.22      0.00
   9 SPCM     - SPECTRAL COMP.                     144    1099.4    1099.4      3.01      0.00
  10 SCAN2M - PHYSICS                              145   15452.1   15452.1     42.55      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 12    7339.3    7339.3      1.67      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                  7      36.8      36.8      0.00      0.00
  17 GRIDFPOS IN CNT4                               12      35.7      35.7      0.01      0.00
  27 RESTART FILES - WRITING                         3   29285.4   29285.4      1.67      0.00
  28 RESTART FILES - READING                         1   39209.4   39209.4      0.74      0.00
  29 SU4FPOS IN CNT4                                12       0.1       0.1      0.00      0.00
  30 DYNFPOS IN CNT4                                12    8910.9    8910.9      2.03      0.00
  31 POSDDH IN STEPO                                 1     274.1     274.1      0.01      0.00
  37 CPGLAG   - SL COMPUTATIONS                    145    7841.5    7841.5     21.59      0.00
  39 SU0YOMB                                         1     187.4     187.4      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                    145      34.3      34.3      0.09      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              144     503.1     503.1      1.38      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              144     147.0     147.0      0.40      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                  7       0.3       0.3      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM      316     558.2     558.2      3.35      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM       300     359.6     359.6      2.05      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM        300     109.6     109.6      0.62      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM       316     180.3     180.3      1.08      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      37.9      37.9      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION            316      80.2      80.2      0.48      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION            300      41.1      41.1      0.23      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION            316      67.8      67.8      0.41      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION            300      29.7      29.7      0.17      0.00
 400 GSTATS                                      17763       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 16473       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME     5266.1 CPU TIME    4437.9 VECTOR TIME     4437.9


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY      248.0
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc/slots/2/ICMGGh7zg+001440
Moving to projects directory: /var/lib/boinc/slots/2/ICMSHh7zg+001440
Moving to projects directory: /var/lib/boinc/slots/2/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc/slots/2/NODE.001_01
Adding to the zip: /var/lib/boinc/slots/2/ifs.stat
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+001356
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+001356
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+001356
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+001368
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+001368
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+001368
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+001380
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+001380
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+001380
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+001392
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+001392
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+001392
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+001404
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+001404
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+001404
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+001416
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+001416
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+001416
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+001428
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+001428
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+001428
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMGGh7zg+001440
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMSHh7zg+001440
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286001/ICMUAh7zg+001440
Zipping up the final file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a37y_2016092300_20_1017_12286001_1_r77867385_14.zip
Uploading the final file: upload_file_14.zip
Uploading trickle at timestep: 1295100
20:52:13 (1905152): called boinc_finish(0)

</stderr_txt>
<message>
upload failure: <file_xfer_error>
  <file_name>oifs_43r3_bl_a37y_2016092300_20_1017_12286001_1_r77867385_15.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a37y_2016092300_20_1017_12286001_1_r77867385_16.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a37y_2016092300_20_1017_12286001_1_r77867385_17.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a37y_2016092300_20_1017_12286001_1_r77867385_18.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a37y_2016092300_20_1017_12286001_1_r77867385_19.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
</message>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
08 Jun 2024 19:19:53 1507812 22440132 oifs_43r3_bl_a37y_2016092300_20_1017_12286001_1 1,295,100 46,198 0.0357


©2024 climateprediction.net