Name | oifs_43r3_ps_1554_2021050100_123_946_12164643_1 |
Workunit | 12164643 |
Created | 30 Nov 2022, 22:44:32 UTC |
Sent | 30 Nov 2022, 22:45:40 UTC |
Report deadline | 30 Dec 2022, 22:45:40 UTC |
Received | 1 Dec 2022, 12:30:51 UTC |
Server state | Over |
Outcome | Computation error |
Client state | Compute error |
Exit status | 9 (0x00000009) Unknown error code |
Computer ID | 1527784 |
Run time | 3 hours 0 min 52 sec |
CPU time | 1 hours 52 min 21 sec |
Validate state | Invalid |
Credit | 3,024.00 |
Device peak FLOPS | 6.65 GFLOPS |
Application version | OpenIFS 43r3 Perturbed Surface v1.01 x86_64-pc-linux-gnu |
Peak working set size | 4,418.29 MB |
Peak swap size | 4,978.06 MB |
Peak disk usage | 2,126.66 MB |
Stderr | <core_client_version>7.18.1</core_client_version> <![CDATA[ <message> process exited with code 9 (0x9, -247)</message> <stderr_txt> ermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_1554_2021050100_123_946_12164643_1_r92312708_6.zip Uploading the intermediate file: upload_file_6.zip 23:30:29 STEP 169 H= 169:00 +CPU= 12.930 23:30:43 STEP 170 H= 170:00 +CPU= 13.317 Uploading trickle at timestep: 604800 23:31:03 STEP 171 H= 171:00 +CPU= 20.304 23:31:17 STEP 172 H= 172:00 +CPU= 13.351 23:31:30 STEP 173 H= 173:00 +CPU= 12.991 23:31:51 STEP 174 H= 174:00 +CPU= 20.751 23:32:04 STEP 175 H= 175:00 +CPU= 12.967 23:32:17 STEP 176 H= 176:00 +CPU= 13.377 23:32:38 STEP 177 H= 177:00 +CPU= 20.334 23:32:51 STEP 178 H= 178:00 +CPU= 13.316 23:33:04 STEP 179 H= 179:00 +CPU= 12.993 23:33:27 STEP 180 H= 180:00 +CPU= 22.538 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000180 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000180 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000180 23:33:40 STEP 181 H= 181:00 +CPU= 12.994 23:33:53 STEP 182 H= 182:00 +CPU= 13.379 23:34:14 STEP 183 H= 183:00 +CPU= 20.350 23:34:27 STEP 184 H= 184:00 +CPU= 13.344 23:34:40 STEP 185 H= 185:00 +CPU= 12.965 23:35:01 STEP 186 H= 186:00 +CPU= 20.722 23:35:14 STEP 187 H= 187:00 +CPU= 12.975 23:35:27 STEP 188 H= 188:00 +CPU= 13.348 23:35:48 STEP 189 H= 189:00 +CPU= 20.293 23:36:01 STEP 190 H= 190:00 +CPU= 13.359 23:36:15 STEP 191 H= 191:00 +CPU= 13.359 23:36:39 STEP 192 H= 192:00 +CPU= 24.233 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000192 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000192 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000192 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000168 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000168 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000168 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000180 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000180 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000180 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_1554_2021050100_123_946_12164643_1_r92312708_7.zip Uploading the intermediate file: upload_file_7.zip 23:36:52 STEP 193 H= 193:00 +CPU= 12.959 23:37:05 STEP 194 H= 194:00 +CPU= 13.310 Uploading trickle at timestep: 691200 23:37:26 STEP 195 H= 195:00 +CPU= 20.306 23:37:39 STEP 196 H= 196:00 +CPU= 13.326 23:37:52 STEP 197 H= 197:00 +CPU= 12.966 23:38:13 STEP 198 H= 198:00 +CPU= 20.752 23:38:26 STEP 199 H= 199:00 +CPU= 12.974 23:38:40 STEP 200 H= 200:00 +CPU= 13.356 23:39:00 STEP 201 H= 201:00 +CPU= 20.379 23:39:13 STEP 202 H= 202:00 +CPU= 13.383 23:39:27 STEP 203 H= 203:00 +CPU= 13.039 23:39:49 STEP 204 H= 204:00 +CPU= 22.463 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000204 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000204 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000204 23:40:02 STEP 205 H= 205:00 +CPU= 12.966 23:40:16 STEP 206 H= 206:00 +CPU= 13.355 23:40:36 STEP 207 H= 207:00 +CPU= 20.348 23:40:49 STEP 208 H= 208:00 +CPU= 13.364 23:41:02 STEP 209 H= 209:00 +CPU= 12.997 23:41:23 STEP 210 H= 210:00 +CPU= 20.785 23:41:36 STEP 211 H= 211:00 +CPU= 12.963 23:41:50 STEP 212 H= 212:00 +CPU= 13.303 23:42:10 STEP 213 H= 213:00 +CPU= 20.299 23:42:24 STEP 214 H= 214:00 +CPU= 13.321 23:42:38 STEP 215 H= 215:00 +CPU= 13.360 23:43:04 STEP 216 H= 216:00 +CPU= 24.509 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000216 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000216 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000216 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000192 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000192 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000192 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000204 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000204 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000204 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_1554_2021050100_123_946_12164643_1_r92312708_8.zip Uploading the intermediate file: upload_file_8.zip 23:43:18 STEP 217 H= 217:00 +CPU= 13.124 23:43:31 STEP 218 H= 218:00 +CPU= 13.379 Uploading trickle at timestep: 777600 23:43:52 STEP 219 H= 219:00 +CPU= 20.415 23:44:06 STEP 220 H= 220:00 +CPU= 13.390 23:44:19 STEP 221 H= 221:00 +CPU= 13.096 23:44:41 STEP 222 H= 222:00 +CPU= 21.013 23:44:55 STEP 223 H= 223:00 +CPU= 13.143 23:45:09 STEP 224 H= 224:00 +CPU= 13.501 23:45:29 STEP 225 H= 225:00 +CPU= 20.284 23:45:43 STEP 226 H= 226:00 +CPU= 13.396 23:45:56 STEP 227 H= 227:00 +CPU= 13.065 23:46:18 STEP 228 H= 228:00 +CPU= 22.422 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000228 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000228 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000228 23:46:31 STEP 229 H= 229:00 +CPU= 12.995 23:46:45 STEP 230 H= 230:00 +CPU= 13.444 23:47:05 STEP 231 H= 231:00 +CPU= 20.416 23:47:19 STEP 232 H= 232:00 +CPU= 13.342 23:47:32 STEP 233 H= 233:00 +CPU= 13.098 23:47:53 STEP 234 H= 234:00 +CPU= 20.783 23:48:07 STEP 235 H= 235:00 +CPU= 12.989 23:48:20 STEP 236 H= 236:00 +CPU= 13.370 23:48:42 STEP 237 H= 237:00 +CPU= 20.564 23:48:56 STEP 238 H= 238:00 +CPU= 13.370 Suspend request received from the BOINC client, suspending the child process [EC_DRHOOK:antony-OptiPlex-3060:1:1:45012:45012] [20221130:234914:1669852154:3794.638] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 4745MB (heap), 4415MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1 [EC_DRHOOK:antony-OptiPlex-3060:1:1:45012:45012] [20221130:234914:1669852154:3794.638] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1 [EC_DRHOOK:antony-OptiPlex-3060:1:1:45012:45012] [20221130:234914:1669852154:3794.638] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil)) [EC_DRHOOK:antony-OptiPlex-3060:1:1:45012:45012] [20221130:234914:1669852154:3794.638] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1 [EC_DRHOOK:antony-OptiPlex-3060:1:1:45012:45012] [20221130:234914:1669852154:3794.638] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1 [EC_DRHOOK:antony-OptiPlex-3060:1:1:45012:45012] [20221130:234914:1669852154:3794.638] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 4745 MB (maxheap), 4415 MB (maxrss), 0 MB (maxstack) [EC_DRHOOK:antony-OptiPlex-3060:1:1:45012:45012] [20221130:234914:1669852154:3794.638] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : MASTER [EC_DRHOOK:antony-OptiPlex-3060:1:1:45012:45012] [20221130:234914:1669852154:3794.638] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT0 [EC_DRHOOK:antony-OptiPlex-3060:1:1:45012:45012] [20221130:234914:1669852154:3794.638] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT1 [EC_DRHOOK:antony-OptiPlex-3060:1:1:45012:45012] [20221130:234914:1669852154:3794.638] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT2 [EC_DRHOOK:antony-OptiPlex-3060:1:1:45012:45012] [20221130:234914:1669852154:3794.638] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT3 [EC_DRHOOK:antony-OptiPlex-3060:1:1:45012:45012] [20221130:234914:1669852154:3794.638] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT4 [EC_DRHOOK:antony-OptiPlex-3060:1:1:45012:45012] [20221130:234914:1669852154:3794.638] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : STEPO [EC_DRHOOK:antony-OptiPlex-3060:1:1:45012:45012] [20221130:234914:1669852154:3794.638] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : SCAN2M [EC_DRHOOK:antony-OptiPlex-3060:1:1:45012:45012] [20221130:234914:1669852154:3794.638] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL_HEAP [EC_DRHOOK:antony-OptiPlex-3060:1:1:45012:45012] [20221130:234914:1669852154:3794.638] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL [EC_DRHOOK:antony-OptiPlex-3060:1:1:45012:45012] [20221130:234914:1669852154:3794.638] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CALL_SL_HEAP [EC_DRHOOK:antony-OptiPlex-3060:1:1:45012:45012] [20221130:234914:1669852154:3794.638] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CALL_SL [EC_DRHOOK:antony-OptiPlex-3060:1:1:45012:45012] [20221130:234914:1669852154:3794.638] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : >OMP-CALL_SL 1 (1004) [EC_DRHOOK:antony-OptiPlex-3060:1:1:45012:45012] [20221130:234914:1669852154:3794.638] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : LAPINEA [EC_DRHOOK:antony-OptiPlex-3060:1:1:45012:45012] [20221130:234914:1669852154:3794.638] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : LARMES [EC_DRHOOK:antony-OptiPlex-3060:1:1:45012:45012] [20221130:234914:1669852154:3794.638] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : LARCINA [EC_DRHOOK:antony-OptiPlex-3060:1:1:45012:45012] [20221130:234914:1669852154:3794.638] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : LASCAW [EC_DRHOOK:antony-OptiPlex-3060:1:1:45012:45012] [20221130:234915:1669852155:3795.638] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1 [EC_DRHOOK:antony-OptiPlex-3060:1:1:45012:45012] [20221130:234915:1669852155:3795.638] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1 forrtl: error (78): process killed (SIGTERM) Image PC Routine Line Source master.exe 0000000001CE916B Unknown Unknown Unknown master.exe 000000000081FFF1 Unknown Unknown Unknown master.exe 0000000001DC9090 Unknown Unknown Unknown master.exe 0000000001D53DA4 Unknown Unknown Unknown master.exe 0000000000DF1251 lascaw_ 470 lascaw.F90 master.exe 0000000000DED525 larcina_ 319 larcina.F90 master.exe 0000000000DEABF8 larmes_ 603 larmes.F90 master.exe 0000000000DE50FD lapinea_ 270 lapinea.F90 master.exe 0000000000D7A4EC call_sl_ 300 call_sl.F90 master.exe 0000000000D784FD call_sl_heap_ 94 call_sl_heap.F90 master.exe 0000000000BF9958 gp_model_ 533 gp_model.F90 master.exe 00000000012E9482 gp_model_heap_ 74 gp_model_heap.F90 master.exe 0000000000BBC1CE scan2m_ 535 scan2m.F90 master.exe 000000000057A664 stepo_ 327 stepo.F90 master.exe 000000000055DEEC cnt4_ 1133 cnt4.F90 master.exe 00000000005471C9 cnt3_ 267 cnt3.F90 master.exe 0000000000546412 cnt2_ 88 cnt2.F90 master.exe 0000000000545E28 cnt1_ 92 cnt1.F90 master.exe 000000000040708D cnt0_ 146 cnt0.F90 master.exe 000000000040220F MAIN__ 96 master.F90 master.exe 00000000004021A2 Unknown Unknown Unknown master.exe 0000000001DCA390 Unknown Unknown Unknown master.exe 000000000040206E Unknown Unknown Unknown (argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.01_x86_64-pc-linux-gnu (argv1) start_date: 2021050100 (argv2) exptid: hpi1 (argv3) unique_member_id: 1554 (argv4) batchid: 946 (argv5) wuid: 12164643 (argv6) fclen: 123 (argv7) app_name: oifs_43r3_ps (argv8) nthreads: 1 Working directory is: /var/lib/boinc-client/slots/6 Project directory is: /var/lib/boinc-client/projects/climateprediction.net/ app name: oifs_43r3_ps version: 1.01 Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643 ..mkdir for temp folder for results failed Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/6/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip Unzipping the app zip file: /var/lib/boinc-client/slots/6/oifs_43r3_ps_app_1.01_x86_64-pc-linux-gnu.zip Copying the namelist files from: ../../projects/climateprediction.net/jf_7efd10e034b9da91f94cb5de87cadaeb to: /var/lib/boinc-client/slots/6/oifs_43r3_ps_1554_2021050100_123_946_12164643.zip Unzipping the namelist zip file: /var/lib/boinc-client/slots/6/oifs_43r3_ps_1554_2021050100_123_946_12164643.zip ic_ancil_file: ic_ancil_12164643 ifsdata_file: ifsdata_12164643 climate_data_file: clim_data_12164643 horiz_resolution: 159 vert_resolution: 60 grid_type: l_2 upload_interval: 24 utstep: 3600 Copying IC ancils from: ../../projects/climateprediction.net/jf_9255127d70349c16abe4c44712e090a6 to: /var/lib/boinc-client/slots/6/ic_ancil_12164643.zip Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/6/ic_ancil_12164643.zip ..mkdir for ifsdata folder failed Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_f4094c751d09804c28bd45de9b9d17e8 to: /var/lib/boinc-client/slots/6/ifsdata/ifsdata_12164643.zip Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/6/ifsdata/ifsdata_12164643.zip ..mkdir for the climate data folder failed Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/6/159l_2/clim_data_12164643.zip Unzipping the climate data zip file: /var/lib/boinc-client/slots/6/159l_2/clim_data_12164643.zip total_length_of_simulation: 10627200 result_base_name: oifs_43r3_ps_1554_2021050100_123_946_12164643_1_r92312708 The child process has been launched with process id: 14278 Executing the command: /var/lib/boinc-client/slots/6/./master.exe [EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1 [EC_DRHOOK:antony-OptiPlex-3060:1:1:14278:14278] [20221201:102206:1669890126:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0 [EC_DRHOOK:antony-OptiPlex-3060:1:1:14278:14278] [20221201:102206:1669890126:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1 [EC_DRHOOK:antony-OptiPlex-3060:1:1:14278:14278] [20221201:102206:1669890126:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0) [EC_DRHOOK:antony-OptiPlex-3060:1:1:14278:14278] [20221201:102206:1669890126:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1 [EC_DRHOOK:antony-OptiPlex-3060:1:1:14278:14278] [20221201:102206:1669890126:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000 [EC_DRHOOK:antony-OptiPlex-3060:1:1:14278:14278] [20221201:102206:1669890126:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647) [EC_DRHOOK:antony-OptiPlex-3060:1:1:14278:14278] [20221201:102206:1669890126:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16 [EC_DRHOOK:antony-OptiPlex-3060:1:1:14278:14278] [20221201:102206:1669890126:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0 [EC_DRHOOK:antony-OptiPlex-3060:1:1:14278:14278] [20221201:102206:1669890126:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500 [EC_DRHOOK:antony-OptiPlex-3060:1:1:14278:14278] [20221201:102206:1669890126:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1 [EC_DRHOOK:antony-OptiPlex-3060:1:1:14278:14278] [20221201:102206:1669890126:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1 [EC_DRHOOK:antony-OptiPlex-3060:1:1:14278:14278] [20221201:102206:1669890126:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1 [EC_DRHOOK:antony-OptiPlex-3060:1:1:14278:14278] [20221201:102206:1669890126:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1 [EC_DRHOOK:antony-OptiPlex-3060:1:1:14278:14278] [20221201:102206:1669890126:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef> [EC_DRHOOK:antony-OptiPlex-3060:1:1:14278:14278] [20221201:102206:1669890126:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef> [EC_DRHOOK:antony-OptiPlex-3060:1:1:14278:14278] [20221201:102206:1669890126:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:antony-OptiPlex-3060:1:1:14278:14278] [20221201:102206:1669890126:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil)) [EC_DRHOOK:antony-OptiPlex-3060:1:1:14278:14278] [20221201:102206:1669890126:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:antony-OptiPlex-3060:1:1:14278:14278] [20221201:102206:1669890126:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil)) [EC_DRHOOK:antony-OptiPlex-3060:1:1:14278:14278] [20221201:102206:1669890126:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1 [EC_DRHOOK:antony-OptiPlex-3060:1:1:14278:14278] [20221201:102206:1669890126:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:antony-OptiPlex-3060:1:1:14278:14278] [20221201:102206:1669890126:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:antony-OptiPlex-3060:1:1:14278:14278] [20221201:102206:1669890126:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil)) [EC_DRHOOK:antony-OptiPlex-3060:1:1:14278:14278] [20221201:102206:1669890126:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:antony-OptiPlex-3060:1:1:14278:14278] [20221201:102206:1669890126:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:antony-OptiPlex-3060:1:1:14278:14278] [20221201:102206:1669890126:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0) [EC_DRHOOK:antony-OptiPlex-3060:1:1:14278:14278] [20221201:102206:1669890126:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil)) [EC_DRHOOK:antony-OptiPlex-3060:1:1:14278:14278] [20221201:102206:1669890126:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil)) [EC_DRHOOK:antony-OptiPlex-3060:1:1:14278:14278] [20221201:102206:1669890126:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil)) [EC_DRHOOK:antony-OptiPlex-3060:1:1:14278:14278] [20221201:102206:1669890126:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef> 10:22:33 STEP 216 H= 216:00 +CPU= 23.379 10:22:47 STEP 217 H= 217:00 +CPU= 13.241 10:23:01 STEP 218 H= 218:00 +CPU= 13.569 10:23:22 STEP 219 H= 219:00 +CPU= 20.692 10:23:35 STEP 220 H= 220:00 +CPU= 13.517 10:23:49 STEP 221 H= 221:00 +CPU= 13.303 10:24:11 STEP 222 H= 222:00 +CPU= 21.228 10:24:25 STEP 223 H= 223:00 +CPU= 13.413 10:24:39 STEP 224 H= 224:00 +CPU= 13.627 10:25:00 STEP 225 H= 225:00 +CPU= 20.691 10:25:14 STEP 226 H= 226:00 +CPU= 13.693 10:25:27 STEP 227 H= 227:00 +CPU= 13.212 10:25:50 STEP 228 H= 228:00 +CPU= 22.779 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000228 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000228 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000228 10:26:04 STEP 229 H= 229:00 +CPU= 13.214 10:26:18 STEP 230 H= 230:00 +CPU= 13.674 10:26:38 STEP 231 H= 231:00 +CPU= 20.632 10:26:52 STEP 232 H= 232:00 +CPU= 13.562 10:27:06 STEP 233 H= 233:00 +CPU= 13.182 10:27:27 STEP 234 H= 234:00 +CPU= 21.064 10:27:40 STEP 235 H= 235:00 +CPU= 13.239 10:27:54 STEP 236 H= 236:00 +CPU= 13.613 10:28:15 STEP 237 H= 237:00 +CPU= 20.632 10:28:29 STEP 238 H= 238:00 +CPU= 13.552 10:28:43 STEP 239 H= 239:00 +CPU= 13.617 10:29:07 STEP 240 H= 240:00 +CPU= 24.432 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000240 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000240 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000240 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000216 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000216 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000216 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000228 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000228 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000228 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_1554_2021050100_123_946_12164643_1_r92312708_1.zip Uploading the intermediate file: upload_file_1.zip 10:29:21 STEP 241 H= 241:00 +CPU= 13.203 Uploading trickle at timestep: 864000 10:29:35 STEP 242 H= 242:00 +CPU= 13.545 10:29:55 STEP 243 H= 243:00 +CPU= 20.658 10:30:09 STEP 244 H= 244:00 +CPU= 13.562 10:30:22 STEP 245 H= 245:00 +CPU= 13.207 10:30:44 STEP 246 H= 246:00 +CPU= 21.083 10:30:57 STEP 247 H= 247:00 +CPU= 13.231 10:31:11 STEP 248 H= 248:00 +CPU= 13.625 10:31:32 STEP 249 H= 249:00 +CPU= 20.669 10:31:45 STEP 250 H= 250:00 +CPU= 13.626 10:31:59 STEP 251 H= 251:00 +CPU= 13.240 10:32:22 STEP 252 H= 252:00 +CPU= 22.848 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000252 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000252 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000252 10:32:35 STEP 253 H= 253:00 +CPU= 13.239 10:32:49 STEP 254 H= 254:00 +CPU= 13.613 10:33:10 STEP 255 H= 255:00 +CPU= 20.657 10:33:24 STEP 256 H= 256:00 +CPU= 13.589 10:33:37 STEP 257 H= 257:00 +CPU= 13.240 10:33:58 STEP 258 H= 258:00 +CPU= 21.026 10:34:12 STEP 259 H= 259:00 +CPU= 13.248 10:34:25 STEP 260 H= 260:00 +CPU= 13.565 10:34:46 STEP 261 H= 261:00 +CPU= 20.634 10:35:00 STEP 262 H= 262:00 +CPU= 13.619 10:35:14 STEP 263 H= 263:00 +CPU= 13.748 10:35:40 STEP 264 H= 264:00 +CPU= 24.838 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000264 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000264 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000264 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000240 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000240 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000240 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000252 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000252 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000252 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_1554_2021050100_123_946_12164643_1_r92312708_2.zip Uploading the intermediate file: upload_file_2.zip 10:35:54 STEP 265 H= 265:00 +CPU= 13.312 10:36:08 STEP 266 H= 266:00 +CPU= 13.684 Uploading trickle at timestep: 950400 10:36:29 STEP 267 H= 267:00 +CPU= 20.731 10:36:43 STEP 268 H= 268:00 +CPU= 13.628 10:36:57 STEP 269 H= 269:00 +CPU= 13.277 10:37:19 STEP 270 H= 270:00 +CPU= 21.184 10:37:32 STEP 271 H= 271:00 +CPU= 13.307 10:37:46 STEP 272 H= 272:00 +CPU= 13.701 10:38:08 STEP 273 H= 273:00 +CPU= 20.757 10:38:22 STEP 274 H= 274:00 +CPU= 13.657 10:38:35 STEP 275 H= 275:00 +CPU= 13.320 10:38:59 STEP 276 H= 276:00 +CPU= 22.982 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000276 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000276 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000276 10:39:13 STEP 277 H= 277:00 +CPU= 13.293 10:39:26 STEP 278 H= 278:00 +CPU= 13.662 10:39:48 STEP 279 H= 279:00 +CPU= 20.801 10:40:02 STEP 280 H= 280:00 +CPU= 13.703 10:40:15 STEP 281 H= 281:00 +CPU= 13.216 10:40:37 STEP 282 H= 282:00 +CPU= 21.286 10:40:51 STEP 283 H= 283:00 +CPU= 13.353 10:41:07 STEP 284 H= 284:00 +CPU= 14.196 10:41:29 STEP 285 H= 285:00 +CPU= 21.121 10:41:43 STEP 286 H= 286:00 +CPU= 13.964 10:41:58 STEP 287 H= 287:00 +CPU= 14.125 10:42:24 STEP 288 H= 288:00 +CPU= 25.077 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000288 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000288 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000288 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000264 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000264 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000264 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000276 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000276 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000276 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_1554_2021050100_123_946_12164643_1_r92312708_3.zip Uploading the intermediate file: upload_file_3.zip 10:42:38 STEP 289 H= 289:00 +CPU= 13.618 10:42:53 STEP 290 H= 290:00 +CPU= 14.019 Uploading trickle at timestep: 1036800 10:43:14 STEP 291 H= 291:00 +CPU= 21.003 10:43:28 STEP 292 H= 292:00 +CPU= 13.853 10:43:42 STEP 293 H= 293:00 +CPU= 13.521 10:44:04 STEP 294 H= 294:00 +CPU= 21.436 10:44:17 STEP 295 H= 295:00 +CPU= 13.501 10:44:31 STEP 296 H= 296:00 +CPU= 13.839 10:44:53 STEP 297 H= 297:00 +CPU= 20.999 10:45:07 STEP 298 H= 298:00 +CPU= 13.845 10:45:20 STEP 299 H= 299:00 +CPU= 13.484 10:45:44 STEP 300 H= 300:00 +CPU= 23.153 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000300 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000300 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000300 10:45:58 STEP 301 H= 301:00 +CPU= 13.496 10:46:12 STEP 302 H= 302:00 +CPU= 13.941 10:46:33 STEP 303 H= 303:00 +CPU= 21.018 10:46:47 STEP 304 H= 304:00 +CPU= 13.855 10:47:01 STEP 305 H= 305:00 +CPU= 13.536 10:47:23 STEP 306 H= 306:00 +CPU= 21.466 10:47:37 STEP 307 H= 307:00 +CPU= 13.511 10:47:51 STEP 308 H= 308:00 +CPU= 13.868 10:48:12 STEP 309 H= 309:00 +CPU= 20.956 10:48:26 STEP 310 H= 310:00 +CPU= 13.824 10:48:40 STEP 311 H= 311:00 +CPU= 13.908 10:49:05 STEP 312 H= 312:00 +CPU= 24.968 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000312 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000312 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000312 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000288 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000288 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000288 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000300 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000300 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000300 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_1554_2021050100_123_946_12164643_1_r92312708_4.zip Uploading the intermediate file: upload_file_4.zip 10:49:19 STEP 313 H= 313:00 +CPU= 13.479 10:49:33 STEP 314 H= 314:00 +CPU= 13.893 Uploading trickle at timestep: 1123200 10:49:54 STEP 315 H= 315:00 +CPU= 21.027 10:50:08 STEP 316 H= 316:00 +CPU= 13.904 10:50:22 STEP 317 H= 317:00 +CPU= 13.517 10:50:44 STEP 318 H= 318:00 +CPU= 21.453 10:50:58 STEP 319 H= 319:00 +CPU= 13.569 10:51:12 STEP 320 H= 320:00 +CPU= 13.916 10:51:33 STEP 321 H= 321:00 +CPU= 21.040 10:51:47 STEP 322 H= 322:00 +CPU= 13.907 10:52:01 STEP 323 H= 323:00 +CPU= 13.584 10:52:25 STEP 324 H= 324:00 +CPU= 23.290 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000324 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000324 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000324 10:52:38 STEP 325 H= 325:00 +CPU= 13.533 10:52:53 STEP 326 H= 326:00 +CPU= 14.116 10:53:15 STEP 327 H= 327:00 +CPU= 21.189 10:53:29 STEP 328 H= 328:00 +CPU= 13.909 10:53:43 STEP 329 H= 329:00 +CPU= 13.622 10:54:05 STEP 330 H= 330:00 +CPU= 21.627 10:54:18 STEP 331 H= 331:00 +CPU= 13.550 10:54:33 STEP 332 H= 332:00 +CPU= 14.069 10:54:55 STEP 333 H= 333:00 +CPU= 21.249 10:55:09 STEP 334 H= 334:00 +CPU= 13.995 10:55:24 STEP 335 H= 335:00 +CPU= 14.059 10:55:49 STEP 336 H= 336:00 +CPU= 25.121 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000336 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000336 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000336 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000312 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000312 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000312 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000324 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000324 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000324 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_1554_2021050100_123_946_12164643_1_r92312708_5.zip Uploading the intermediate file: upload_file_5.zip 10:56:03 STEP 337 H= 337:00 +CPU= 13.529 10:56:18 STEP 338 H= 338:00 +CPU= 13.940 Uploading trickle at timestep: 1209600 10:56:39 STEP 339 H= 339:00 +CPU= 20.970 10:56:53 STEP 340 H= 340:00 +CPU= 13.922 10:57:06 STEP 341 H= 341:00 +CPU= 13.557 10:57:28 STEP 342 H= 342:00 +CPU= 21.409 10:57:41 STEP 343 H= 343:00 +CPU= 13.490 10:57:55 STEP 344 H= 344:00 +CPU= 13.881 10:58:16 STEP 345 H= 345:00 +CPU= 20.993 10:58:30 STEP 346 H= 346:00 +CPU= 13.899 10:58:44 STEP 347 H= 347:00 +CPU= 13.559 10:59:07 STEP 348 H= 348:00 +CPU= 23.183 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000348 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000348 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000348 10:59:21 STEP 349 H= 349:00 +CPU= 13.533 10:59:35 STEP 350 H= 350:00 +CPU= 13.897 10:59:56 STEP 351 H= 351:00 +CPU= 21.069 11:00:10 STEP 352 H= 352:00 +CPU= 13.885 11:00:24 STEP 353 H= 353:00 +CPU= 13.479 11:00:45 STEP 354 H= 354:00 +CPU= 21.427 11:00:59 STEP 355 H= 355:00 +CPU= 13.451 11:01:13 STEP 356 H= 356:00 +CPU= 13.824 11:01:34 STEP 357 H= 357:00 +CPU= 21.015 11:01:48 STEP 358 H= 358:00 +CPU= 13.810 11:02:02 STEP 359 H= 359:00 +CPU= 13.874 11:02:27 STEP 360 H= 360:00 +CPU= 25.033 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000360 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000360 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000360 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000336 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000336 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000336 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000348 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000348 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000348 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_1554_2021050100_123_946_12164643_1_r92312708_6.zip Uploading the intermediate file: upload_file_6.zip 11:02:41 STEP 361 H= 361:00 +CPU= 13.488 Uploading trickle at timestep: 1296000 11:02:54 STEP 362 H= 362:00 +CPU= 13.818 11:03:15 STEP 363 H= 363:00 +CPU= 20.901 11:03:29 STEP 364 H= 364:00 +CPU= 13.817 11:03:43 STEP 365 H= 365:00 +CPU= 13.473 11:04:05 STEP 366 H= 366:00 +CPU= 21.569 11:04:19 STEP 367 H= 367:00 +CPU= 13.604 11:04:33 STEP 368 H= 368:00 +CPU= 14.083 11:04:55 STEP 369 H= 369:00 +CPU= 21.388 11:05:10 STEP 370 H= 370:00 +CPU= 14.032 11:05:24 STEP 371 H= 371:00 +CPU= 13.650 11:05:48 STEP 372 H= 372:00 +CPU= 23.534 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000372 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000372 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000372 11:06:02 STEP 373 H= 373:00 +CPU= 13.773 11:06:17 STEP 374 H= 374:00 +CPU= 14.185 11:06:40 STEP 375 H= 375:00 +CPU= 21.254 11:06:54 STEP 376 H= 376:00 +CPU= 13.927 11:07:07 STEP 377 H= 377:00 +CPU= 13.507 11:07:29 STEP 378 H= 378:00 +CPU= 21.461 11:07:43 STEP 379 H= 379:00 +CPU= 13.527 11:07:57 STEP 380 H= 380:00 +CPU= 13.876 11:08:18 STEP 381 H= 381:00 +CPU= 21.035 11:08:32 STEP 382 H= 382:00 +CPU= 13.900 11:08:47 STEP 383 H= 383:00 +CPU= 13.983 11:09:12 STEP 384 H= 384:00 +CPU= 25.063 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000384 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000384 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000384 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000360 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000360 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000360 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000372 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000372 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000372 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_1554_2021050100_123_946_12164643_1_r92312708_7.zip Uploading the intermediate file: upload_file_7.zip 11:09:26 STEP 385 H= 385:00 +CPU= 13.530 Uploading trickle at timestep: 1382400 11:09:40 STEP 386 H= 386:00 +CPU= 13.905 11:10:01 STEP 387 H= 387:00 +CPU= 21.051 11:10:15 STEP 388 H= 388:00 +CPU= 13.882 11:10:29 STEP 389 H= 389:00 +CPU= 13.510 11:10:51 STEP 390 H= 390:00 +CPU= 21.467 11:11:04 STEP 391 H= 391:00 +CPU= 13.495 11:11:19 STEP 392 H= 392:00 +CPU= 13.902 11:11:40 STEP 393 H= 393:00 +CPU= 21.092 11:11:54 STEP 394 H= 394:00 +CPU= 13.876 11:12:08 STEP 395 H= 395:00 +CPU= 13.530 11:12:31 STEP 396 H= 396:00 +CPU= 23.254 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000396 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000396 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000396 11:12:45 STEP 397 H= 397:00 +CPU= 13.592 11:12:59 STEP 398 H= 398:00 +CPU= 13.921 11:13:20 STEP 399 H= 399:00 +CPU= 20.990 11:13:34 STEP 400 H= 400:00 +CPU= 13.864 11:13:48 STEP 401 H= 401:00 +CPU= 13.515 11:14:10 STEP 402 H= 402:00 +CPU= 21.459 11:14:23 STEP 403 H= 403:00 +CPU= 13.514 11:14:37 STEP 404 H= 404:00 +CPU= 13.860 11:14:59 STEP 405 H= 405:00 +CPU= 21.004 11:15:13 STEP 406 H= 406:00 +CPU= 13.848 11:15:27 STEP 407 H= 407:00 +CPU= 13.959 11:15:52 STEP 408 H= 408:00 +CPU= 25.038 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000408 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000408 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000408 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000384 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000384 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000384 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000396 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000396 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000396 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_1554_2021050100_123_946_12164643_1_r92312708_8.zip Uploading the intermediate file: upload_file_8.zip 11:16:06 STEP 409 H= 409:00 +CPU= 13.537 Uploading trickle at timestep: 1468800 11:16:20 STEP 410 H= 410:00 +CPU= 13.872 11:16:42 STEP 411 H= 411:00 +CPU= 20.986 11:16:56 STEP 412 H= 412:00 +CPU= 13.869 11:17:09 STEP 413 H= 413:00 +CPU= 13.460 11:17:31 STEP 414 H= 414:00 +CPU= 21.429 11:17:44 STEP 415 H= 415:00 +CPU= 13.541 11:17:58 STEP 416 H= 416:00 +CPU= 13.930 11:18:20 STEP 417 H= 417:00 +CPU= 21.055 11:18:34 STEP 418 H= 418:00 +CPU= 13.860 11:18:48 STEP 419 H= 419:00 +CPU= 13.474 11:19:11 STEP 420 H= 420:00 +CPU= 23.201 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000420 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000420 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000420 11:19:25 STEP 421 H= 421:00 +CPU= 13.484 11:19:39 STEP 422 H= 422:00 +CPU= 13.851 11:20:00 STEP 423 H= 423:00 +CPU= 20.975 11:20:14 STEP 424 H= 424:00 +CPU= 13.828 11:20:28 STEP 425 H= 425:00 +CPU= 13.467 11:20:49 STEP 426 H= 426:00 +CPU= 21.338 11:21:03 STEP 427 H= 427:00 +CPU= 13.404 11:21:17 STEP 428 H= 428:00 +CPU= 13.844 11:21:38 STEP 429 H= 429:00 +CPU= 20.934 11:21:52 STEP 430 H= 430:00 +CPU= 13.822 11:22:06 STEP 431 H= 431:00 +CPU= 13.896 11:22:31 STEP 432 H= 432:00 +CPU= 24.986 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000432 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000432 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000432 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000408 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000408 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000408 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000420 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000420 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000420 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_1554_2021050100_123_946_12164643_1_r92312708_9.zip Uploading the intermediate file: upload_file_9.zip 11:22:45 STEP 433 H= 433:00 +CPU= 13.487 Uploading trickle at timestep: 1555200 11:22:59 STEP 434 H= 434:00 +CPU= 13.797 11:23:20 STEP 435 H= 435:00 +CPU= 20.989 11:23:34 STEP 436 H= 436:00 +CPU= 13.853 11:23:48 STEP 437 H= 437:00 +CPU= 13.444 11:24:10 STEP 438 H= 438:00 +CPU= 21.379 11:24:24 STEP 439 H= 439:00 +CPU= 13.501 11:24:38 STEP 440 H= 440:00 +CPU= 13.858 11:24:59 STEP 441 H= 441:00 +CPU= 20.898 11:25:13 STEP 442 H= 442:00 +CPU= 13.866 11:25:26 STEP 443 H= 443:00 +CPU= 13.481 11:25:50 STEP 444 H= 444:00 +CPU= 23.103 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000444 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000444 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000444 11:26:03 STEP 445 H= 445:00 +CPU= 13.477 11:26:17 STEP 446 H= 446:00 +CPU= 13.806 11:26:38 STEP 447 H= 447:00 +CPU= 20.951 11:26:52 STEP 448 H= 448:00 +CPU= 13.792 11:27:06 STEP 449 H= 449:00 +CPU= 13.452 11:27:28 STEP 450 H= 450:00 +CPU= 21.387 11:27:41 STEP 451 H= 451:00 +CPU= 13.468 11:27:55 STEP 452 H= 452:00 +CPU= 13.855 11:28:16 STEP 453 H= 453:00 +CPU= 20.947 11:28:30 STEP 454 H= 454:00 +CPU= 13.815 11:28:44 STEP 455 H= 455:00 +CPU= 13.831 11:29:10 STEP 456 H= 456:00 +CPU= 25.007 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000456 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000456 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000456 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000432 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000432 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000432 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000444 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000444 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000444 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_1554_2021050100_123_946_12164643_1_r92312708_10.zip Uploading the intermediate file: upload_file_10.zip 11:29:25 STEP 457 H= 457:00 +CPU= 13.711 Uploading trickle at timestep: 1641600 11:29:39 STEP 458 H= 458:00 +CPU= 13.824 11:30:00 STEP 459 H= 459:00 +CPU= 21.024 11:30:14 STEP 460 H= 460:00 +CPU= 13.824 11:30:28 STEP 461 H= 461:00 +CPU= 13.558 11:30:51 STEP 462 H= 462:00 +CPU= 21.690 11:31:05 STEP 463 H= 463:00 +CPU= 13.548 11:31:19 STEP 464 H= 464:00 +CPU= 13.886 11:31:41 STEP 465 H= 465:00 +CPU= 21.183 11:31:56 STEP 466 H= 466:00 +CPU= 14.116 11:32:10 STEP 467 H= 467:00 +CPU= 13.482 11:32:33 STEP 468 H= 468:00 +CPU= 22.859 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000468 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000468 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000468 11:32:47 STEP 469 H= 469:00 +CPU= 13.309 11:33:01 STEP 470 H= 470:00 +CPU= 13.683 11:33:22 STEP 471 H= 471:00 +CPU= 20.770 11:33:35 STEP 472 H= 472:00 +CPU= 13.625 11:33:49 STEP 473 H= 473:00 +CPU= 13.268 11:34:12 STEP 474 H= 474:00 +CPU= 21.386 11:34:26 STEP 475 H= 475:00 +CPU= 13.541 11:34:40 STEP 476 H= 476:00 +CPU= 13.772 11:35:02 STEP 477 H= 477:00 +CPU= 20.903 11:35:16 STEP 478 H= 478:00 +CPU= 13.696 11:35:30 STEP 479 H= 479:00 +CPU= 13.671 11:35:55 STEP 480 H= 480:00 +CPU= 24.751 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000480 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000480 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000480 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000456 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000456 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000456 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000468 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000468 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000468 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_1554_2021050100_123_946_12164643_1_r92312708_11.zip Uploading the intermediate file: upload_file_11.zip 11:36:09 STEP 481 H= 481:00 +CPU= 13.370 11:36:23 STEP 482 H= 482:00 +CPU= 13.666 Uploading trickle at timestep: 1728000 11:36:44 STEP 483 H= 483:00 +CPU= 20.837 11:36:58 STEP 484 H= 484:00 +CPU= 13.683 11:37:11 STEP 485 H= 485:00 +CPU= 13.322 11:37:33 STEP 486 H= 486:00 +CPU= 21.178 11:37:46 STEP 487 H= 487:00 +CPU= 13.276 11:38:00 STEP 488 H= 488:00 +CPU= 13.632 11:38:21 STEP 489 H= 489:00 +CPU= 20.788 11:38:35 STEP 490 H= 490:00 +CPU= 13.669 11:38:48 STEP 491 H= 491:00 +CPU= 13.157 11:39:12 STEP 492 H= 492:00 +CPU= 22.767 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000492 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000492 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000492 11:39:25 STEP 493 H= 493:00 +CPU= 13.221 11:39:39 STEP 494 H= 494:00 +CPU= 13.605 11:40:00 STEP 495 H= 495:00 +CPU= 20.832 11:40:14 STEP 496 H= 496:00 +CPU= 13.790 11:40:28 STEP 497 H= 497:00 +CPU= 13.347 11:40:49 STEP 498 H= 498:00 +CPU= 21.134 11:41:03 STEP 499 H= 499:00 +CPU= 13.255 11:41:18 STEP 500 H= 500:00 +CPU= 13.856 11:41:40 STEP 501 H= 501:00 +CPU= 21.024 11:41:54 STEP 502 H= 502:00 +CPU= 13.771 11:42:09 STEP 503 H= 503:00 +CPU= 13.803 11:42:35 STEP 504 H= 504:00 +CPU= 24.897 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000504 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000504 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000504 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000480 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000480 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000480 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000492 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000492 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000492 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_1554_2021050100_123_946_12164643_1_r92312708_12.zip Uploading the intermediate file: upload_file_12.zip 11:42:49 STEP 505 H= 505:00 +CPU= 13.434 11:43:03 STEP 506 H= 506:00 +CPU= 13.718 Uploading trickle at timestep: 1814400 11:43:25 STEP 507 H= 507:00 +CPU= 20.914 11:43:39 STEP 508 H= 508:00 +CPU= 13.766 11:43:53 STEP 509 H= 509:00 +CPU= 13.478 11:44:15 STEP 510 H= 510:00 +CPU= 21.346 11:44:29 STEP 511 H= 511:00 +CPU= 13.387 11:44:43 STEP 512 H= 512:00 +CPU= 13.758 11:45:04 STEP 513 H= 513:00 +CPU= 20.920 11:45:19 STEP 514 H= 514:00 +CPU= 13.782 11:45:33 STEP 515 H= 515:00 +CPU= 13.478 11:45:56 STEP 516 H= 516:00 +CPU= 22.927 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000516 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000516 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000516 11:46:10 STEP 517 H= 517:00 +CPU= 13.311 11:46:23 STEP 518 H= 518:00 +CPU= 13.672 11:46:45 STEP 519 H= 519:00 +CPU= 20.743 11:46:58 STEP 520 H= 520:00 +CPU= 13.604 11:47:12 STEP 521 H= 521:00 +CPU= 13.267 11:47:33 STEP 522 H= 522:00 +CPU= 21.229 11:47:47 STEP 523 H= 523:00 +CPU= 13.293 11:48:01 STEP 524 H= 524:00 +CPU= 13.632 11:48:22 STEP 525 H= 525:00 +CPU= 20.774 11:48:36 STEP 526 H= 526:00 +CPU= 13.618 11:48:49 STEP 527 H= 527:00 +CPU= 13.662 11:49:14 STEP 528 H= 528:00 +CPU= 24.663 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000528 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000528 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000528 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000504 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000504 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000504 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000516 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000516 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000516 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_1554_2021050100_123_946_12164643_1_r92312708_13.zip Uploading the intermediate file: upload_file_13.zip 11:49:28 STEP 529 H= 529:00 +CPU= 13.216 Uploading trickle at timestep: 1900800 11:49:42 STEP 530 H= 530:00 +CPU= 13.619 11:50:02 STEP 531 H= 531:00 +CPU= 20.654 11:50:16 STEP 532 H= 532:00 +CPU= 13.574 11:50:30 STEP 533 H= 533:00 +CPU= 13.237 11:50:51 STEP 534 H= 534:00 +CPU= 21.158 11:51:05 STEP 535 H= 535:00 +CPU= 13.310 11:51:18 STEP 536 H= 536:00 +CPU= 13.615 11:51:39 STEP 537 H= 537:00 +CPU= 20.729 11:51:53 STEP 538 H= 538:00 +CPU= 13.634 11:52:07 STEP 539 H= 539:00 +CPU= 13.250 11:52:30 STEP 540 H= 540:00 +CPU= 22.914 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000540 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000540 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000540 11:52:43 STEP 541 H= 541:00 +CPU= 13.292 11:52:57 STEP 542 H= 542:00 +CPU= 13.695 11:53:18 STEP 543 H= 543:00 +CPU= 20.776 11:53:32 STEP 544 H= 544:00 +CPU= 13.626 11:53:46 STEP 545 H= 545:00 +CPU= 13.263 11:54:07 STEP 546 H= 546:00 +CPU= 21.280 11:54:21 STEP 547 H= 547:00 +CPU= 13.402 11:54:36 STEP 548 H= 548:00 +CPU= 13.738 11:54:57 STEP 549 H= 549:00 +CPU= 20.912 11:55:11 STEP 550 H= 550:00 +CPU= 13.734 11:55:26 STEP 551 H= 551:00 +CPU= 13.838 11:55:51 STEP 552 H= 552:00 +CPU= 25.011 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000552 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000552 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000552 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000528 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000528 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000528 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000540 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000540 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000540 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_1554_2021050100_123_946_12164643_1_r92312708_14.zip Uploading the intermediate file: upload_file_14.zip 11:56:05 STEP 553 H= 553:00 +CPU= 13.470 11:56:20 STEP 554 H= 554:00 +CPU= 13.807 Uploading trickle at timestep: 1987200 11:56:41 STEP 555 H= 555:00 +CPU= 20.908 11:56:55 STEP 556 H= 556:00 +CPU= 13.711 11:57:09 STEP 557 H= 557:00 +CPU= 13.374 11:57:30 STEP 558 H= 558:00 +CPU= 21.328 11:57:44 STEP 559 H= 559:00 +CPU= 13.311 11:57:58 STEP 560 H= 560:00 +CPU= 13.736 11:58:19 STEP 561 H= 561:00 +CPU= 20.852 11:58:33 STEP 562 H= 562:00 +CPU= 13.738 11:58:46 STEP 563 H= 563:00 +CPU= 13.306 11:59:09 STEP 564 H= 564:00 +CPU= 22.974 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000564 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000564 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000564 11:59:23 STEP 565 H= 565:00 +CPU= 13.319 11:59:37 STEP 566 H= 566:00 +CPU= 13.730 11:59:58 STEP 567 H= 567:00 +CPU= 20.845 12:00:14 STEP 568 H= 568:00 +CPU= 13.756 12:00:30 STEP 569 H= 569:00 +CPU= 13.632 12:00:56 STEP 570 H= 570:00 +CPU= 21.496 12:01:12 STEP 571 H= 571:00 +CPU= 13.448 12:01:29 STEP 572 H= 572:00 +CPU= 13.870 12:01:55 STEP 573 H= 573:00 +CPU= 21.051 12:02:12 STEP 574 H= 574:00 +CPU= 13.885 12:02:29 STEP 575 H= 575:00 +CPU= 13.883 12:02:59 STEP 576 H= 576:00 +CPU= 25.083 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000576 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000576 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000576 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000552 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000552 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000552 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000564 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000564 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000564 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_1554_2021050100_123_946_12164643_1_r92312708_15.zip Uploading the intermediate file: upload_file_15.zip 12:03:15 STEP 577 H= 577:00 +CPU= 13.473 Uploading trickle at timestep: 2073600 12:03:32 STEP 578 H= 578:00 +CPU= 13.940 12:03:58 STEP 579 H= 579:00 +CPU= 21.146 12:04:15 STEP 580 H= 580:00 +CPU= 13.938 12:04:31 STEP 581 H= 581:00 +CPU= 13.565 12:04:57 STEP 582 H= 582:00 +CPU= 21.611 12:05:13 STEP 583 H= 583:00 +CPU= 13.586 12:05:30 STEP 584 H= 584:00 +CPU= 13.968 12:05:56 STEP 585 H= 585:00 +CPU= 21.197 12:06:13 STEP 586 H= 586:00 +CPU= 13.949 12:06:30 STEP 587 H= 587:00 +CPU= 13.540 12:06:59 STEP 588 H= 588:00 +CPU= 23.323 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000588 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000588 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000588 12:07:16 STEP 589 H= 589:00 +CPU= 13.734 12:07:33 STEP 590 H= 590:00 +CPU= 14.154 12:08:01 STEP 591 H= 591:00 +CPU= 21.421 12:08:18 STEP 592 H= 592:00 +CPU= 14.147 12:08:35 STEP 593 H= 593:00 +CPU= 13.673 12:09:02 STEP 594 H= 594:00 +CPU= 21.759 12:09:20 STEP 595 H= 595:00 +CPU= 13.732 12:09:37 STEP 596 H= 596:00 +CPU= 14.025 12:10:03 STEP 597 H= 597:00 +CPU= 21.245 12:10:20 STEP 598 H= 598:00 +CPU= 13.987 12:10:37 STEP 599 H= 599:00 +CPU= 13.818 12:11:08 STEP 600 H= 600:00 +CPU= 25.297 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000600 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000600 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000600 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000576 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000576 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000576 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000588 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000588 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000588 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_1554_2021050100_123_946_12164643_1_r92312708_16.zip Uploading the intermediate file: upload_file_16.zip 12:11:24 STEP 601 H= 601:00 +CPU= 13.670 Uploading trickle at timestep: 2160000 12:11:41 STEP 602 H= 602:00 +CPU= 14.036 12:12:08 STEP 603 H= 603:00 +CPU= 21.346 12:12:26 STEP 604 H= 604:00 +CPU= 14.107 12:12:44 STEP 605 H= 605:00 +CPU= 13.655 12:13:11 STEP 606 H= 606:00 +CPU= 21.582 12:13:28 STEP 607 H= 607:00 +CPU= 13.600 12:13:45 STEP 608 H= 608:00 +CPU= 14.018 12:14:11 STEP 609 H= 609:00 +CPU= 21.214 12:14:29 STEP 610 H= 610:00 +CPU= 14.105 12:14:46 STEP 611 H= 611:00 +CPU= 13.640 12:15:15 STEP 612 H= 612:00 +CPU= 23.252 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000612 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000612 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000612 12:15:31 STEP 613 H= 613:00 +CPU= 13.506 12:15:48 STEP 614 H= 614:00 +CPU= 13.893 12:16:15 STEP 615 H= 615:00 +CPU= 21.079 12:16:32 STEP 616 H= 616:00 +CPU= 13.849 12:16:49 STEP 617 H= 617:00 +CPU= 13.506 12:17:15 STEP 618 H= 618:00 +CPU= 21.471 12:17:32 STEP 619 H= 619:00 +CPU= 13.502 12:17:50 STEP 620 H= 620:00 +CPU= 13.844 12:18:16 STEP 621 H= 621:00 +CPU= 21.020 12:18:33 STEP 622 H= 622:00 +CPU= 13.837 12:18:50 STEP 623 H= 623:00 +CPU= 13.915 12:19:18 STEP 624 H= 624:00 +CPU= 24.786 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhpi1+000624 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhpi1+000624 Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhpi1+000624 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000600 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000600 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000600 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMGGhpi1+000612 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMSHhpi1+000612 Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12164643/ICMUAhpi1+000612 Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_1554_2021050100_123_946_12164643_1_r92312708_17.zip Uploading the intermediate file: upload_file_17.zip 12:19:31 STEP 625 H= 625:00 +CPU= 13.174 12:19:45 STEP 626 H= 626:00 +CPU= 13.522 Uploading trickle at timestep: 2246400 double free or corruption (out) </stderr_txt> ]]> |
Latest Trickles Received | ||||||
---|---|---|---|---|---|---|
Time Sent (UTC) | Host ID | Result ID | Result Name | Timestep | CPU Time (sec) | Average (sec/TS) |
09 May 2023 15:21:41 | 1527784 | 22248712 | oifs_43r3_ps_1554_2021050100_123_946_12164643_1 | 2,246,400 | 6,714 | 0.0030 |
09 May 2023 15:21:41 | 1527784 | 22248712 | oifs_43r3_ps_1554_2021050100_123_946_12164643_1 | 2,160,000 | 6,314 | 0.0029 |
09 May 2023 15:21:41 | 1527784 | 22248712 | oifs_43r3_ps_1554_2021050100_123_946_12164643_1 | 2,073,600 | 5,913 | 0.0029 |
09 May 2023 15:21:41 | 1527784 | 22248712 | oifs_43r3_ps_1554_2021050100_123_946_12164643_1 | 1,987,200 | 5,526 | 0.0028 |
09 May 2023 15:21:41 | 1527784 | 22248712 | oifs_43r3_ps_1554_2021050100_123_946_12164643_1 | 1,900,800 | 5,131 | 0.0027 |
09 May 2023 15:21:41 | 1527784 | 22248712 | oifs_43r3_ps_1554_2021050100_123_946_12164643_1 | 1,814,400 | 4,748 | 0.0026 |
09 May 2023 15:21:41 | 1527784 | 22248712 | oifs_43r3_ps_1554_2021050100_123_946_12164643_1 | 1,728,000 | 4,355 | 0.0025 |
09 May 2023 15:21:24 | 1527784 | 22248712 | oifs_43r3_ps_1554_2021050100_123_946_12164643_1 | 1,641,600 | 3,956 | 0.0024 |
09 May 2023 15:21:24 | 1527784 | 22248712 | oifs_43r3_ps_1554_2021050100_123_946_12164643_1 | 1,555,200 | 3,568 | 0.0023 |
09 May 2023 15:21:24 | 1527784 | 22248712 | oifs_43r3_ps_1554_2021050100_123_946_12164643_1 | 1,468,800 | 3,170 | 0.0022 |
09 May 2023 15:21:24 | 1527784 | 22248712 | oifs_43r3_ps_1554_2021050100_123_946_12164643_1 | 1,382,400 | 2,773 | 0.0020 |
09 May 2023 15:21:24 | 1527784 | 22248712 | oifs_43r3_ps_1554_2021050100_123_946_12164643_1 | 1,296,000 | 2,379 | 0.0018 |
09 May 2023 15:21:24 | 1527784 | 22248712 | oifs_43r3_ps_1554_2021050100_123_946_12164643_1 | 1,209,600 | 1,989 | 0.0016 |
09 May 2023 15:21:24 | 1527784 | 22248712 | oifs_43r3_ps_1554_2021050100_123_946_12164643_1 | 1,123,200 | 1,594 | 0.0014 |
09 May 2023 15:21:24 | 1527784 | 22248712 | oifs_43r3_ps_1554_2021050100_123_946_12164643_1 | 1,036,800 | 1,198 | 0.0012 |
09 May 2023 15:21:24 | 1527784 | 22248712 | oifs_43r3_ps_1554_2021050100_123_946_12164643_1 | 950,400 | 808 | 0.0009 |
09 May 2023 15:21:09 | 1527784 | 22248712 | oifs_43r3_ps_1554_2021050100_123_946_12164643_1 | 864,000 | 411 | 0.0005 |
09 May 2023 15:17:35 | 1527784 | 22248712 | oifs_43r3_ps_1554_2021050100_123_946_12164643_1 | 777,600 | 3,378 | 0.0043 |
09 May 2023 15:17:35 | 1527784 | 22248712 | oifs_43r3_ps_1554_2021050100_123_946_12164643_1 | 691,200 | 3,000 | 0.0043 |
09 May 2023 15:17:35 | 1527784 | 22248712 | oifs_43r3_ps_1554_2021050100_123_946_12164643_1 | 604,800 | 2,619 | 0.0043 |
©2024 climateprediction.net