climateprediction.net home page
Task 22256243

Task 22256243

Name oifs_43r3_ps_0215_1986050100_123_955_12171859_0
Workunit 12171859
Created 21 Dec 2022, 8:03:15 UTC
Sent 21 Dec 2022, 8:15:27 UTC
Report deadline 20 Jan 2023, 8:15:27 UTC
Received 23 Dec 2022, 21:42:14 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1532181
Run time 20 hours 39 min 33 sec
CPU time 14 hours 32 min 20 sec
Validate state Valid
Credit 14,873.04
Device peak FLOPS 8.81 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,775.97 MB
Peak swap size 4,981.06 MB
Peak disk usage 3,647.76 MB
Stderr
<core_client_version>7.20.3</core_client_version>
<![CDATA[
<stderr_txt>
6:47:24 STEP 2398 H=2398:00 +CPU= 12.723
  06:47:38 STEP 2399 H=2399:00 +CPU= 13.419
  06:48:03 STEP 2400 H=2400:00 +CPU= 24.952
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002400
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002400
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002400
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002376
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002376
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002376
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002388
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002388
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002388
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0215_1986050100_123_955_12171859_0_r1605739543_99.zip
Uploading the intermediate file: upload_file_99.zip
  06:48:16 STEP 2401 H=2401:00 +CPU= 13.479
  06:48:31 STEP 2402 H=2402:00 +CPU= 14.213
  06:48:51 STEP 2403 H=2403:00 +CPU= 20.531
  06:49:05 STEP 2404 H=2404:00 +CPU= 13.178
  06:49:18 STEP 2405 H=2405:00 +CPU= 13.590
  06:49:40 STEP 2406 H=2406:00 +CPU= 21.424
  06:49:53 STEP 2407 H=2407:00 +CPU= 13.222
  06:50:08 STEP 2408 H=2408:00 +CPU= 14.699
  06:50:28 STEP 2409 H=2409:00 +CPU= 20.831
  06:50:43 STEP 2410 H=2410:00 +CPU= 13.954
  06:50:56 STEP 2411 H=2411:00 +CPU= 13.554
  06:51:19 STEP 2412 H=2412:00 +CPU= 22.586
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002412
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002412
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002412
  06:51:32 STEP 2413 H=2413:00 +CPU= 13.168
  06:51:46 STEP 2414 H=2414:00 +CPU= 13.469
  06:52:06 STEP 2415 H=2415:00 +CPU= 20.848
  06:52:20 STEP 2416 H=2416:00 +CPU= 13.700
  06:52:34 STEP 2417 H=2417:00 +CPU= 13.305
  06:52:55 STEP 2418 H=2418:00 +CPU= 21.176
  06:53:09 STEP 2419 H=2419:00 +CPU= 13.754
  06:53:22 STEP 2420 H=2420:00 +CPU= 13.402
  06:53:43 STEP 2421 H=2421:00 +CPU= 20.863
  06:53:56 STEP 2422 H=2422:00 +CPU= 13.534
  06:54:10 STEP 2423 H=2423:00 +CPU= 13.952
  06:54:35 STEP 2424 H=2424:00 +CPU= 24.895
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002424
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002424
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002424
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002400
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002400
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002400
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002412
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002412
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002412
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0215_1986050100_123_955_12171859_0_r1605739543_100.zip
Uploading the intermediate file: upload_file_100.zip
  06:54:49 STEP 2425 H=2425:00 +CPU= 13.656
  06:55:03 STEP 2426 H=2426:00 +CPU= 13.783
  06:55:24 STEP 2427 H=2427:00 +CPU= 20.954
  06:55:38 STEP 2428 H=2428:00 +CPU= 13.969
  06:55:52 STEP 2429 H=2429:00 +CPU= 13.924
  06:56:13 STEP 2430 H=2430:00 +CPU= 21.261
  06:56:28 STEP 2431 H=2431:00 +CPU= 15.016
  06:56:42 STEP 2432 H=2432:00 +CPU= 13.886
  06:57:03 STEP 2433 H=2433:00 +CPU= 20.329
  06:57:16 STEP 2434 H=2434:00 +CPU= 13.164
  06:57:29 STEP 2435 H=2435:00 +CPU= 12.636
  06:57:51 STEP 2436 H=2436:00 +CPU= 22.835
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002436
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002436
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002436
  06:58:04 STEP 2437 H=2437:00 +CPU= 12.614
  06:58:17 STEP 2438 H=2438:00 +CPU= 13.112
  06:58:37 STEP 2439 H=2439:00 +CPU= 20.193
  06:58:50 STEP 2440 H=2440:00 +CPU= 12.995
  06:59:03 STEP 2441 H=2441:00 +CPU= 12.841
  06:59:24 STEP 2442 H=2442:00 +CPU= 20.304
  06:59:36 STEP 2443 H=2443:00 +CPU= 12.726
  06:59:50 STEP 2444 H=2444:00 +CPU= 13.147
  07:00:10 STEP 2445 H=2445:00 +CPU= 20.490
  07:00:24 STEP 2446 H=2446:00 +CPU= 13.951
  07:00:38 STEP 2447 H=2447:00 +CPU= 14.132
  07:01:03 STEP 2448 H=2448:00 +CPU= 24.029
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002448
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002448
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002448
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002424
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002424
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002424
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002436
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002436
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002436
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0215_1986050100_123_955_12171859_0_r1605739543_101.zip
Uploading the intermediate file: upload_file_101.zip
  07:01:16 STEP 2449 H=2449:00 +CPU= 13.187
  07:01:30 STEP 2450 H=2450:00 +CPU= 13.859
  07:01:50 STEP 2451 H=2451:00 +CPU= 20.133
  07:02:04 STEP 2452 H=2452:00 +CPU= 13.713
  07:02:17 STEP 2453 H=2453:00 +CPU= 13.033
  07:02:38 STEP 2454 H=2454:00 +CPU= 21.367
  07:02:52 STEP 2455 H=2455:00 +CPU= 13.319
  07:03:05 STEP 2456 H=2456:00 +CPU= 13.752
  07:03:26 STEP 2457 H=2457:00 +CPU= 20.773
  07:03:40 STEP 2458 H=2458:00 +CPU= 13.441
  07:03:53 STEP 2459 H=2459:00 +CPU= 13.630
  07:04:16 STEP 2460 H=2460:00 +CPU= 22.944
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002460
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002460
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002460
  07:04:30 STEP 2461 H=2461:00 +CPU= 13.647
  07:04:43 STEP 2462 H=2462:00 +CPU= 13.232
  07:05:04 STEP 2463 H=2463:00 +CPU= 20.321
  07:05:18 STEP 2464 H=2464:00 +CPU= 14.542
  07:05:32 STEP 2465 H=2465:00 +CPU= 13.758
  07:05:54 STEP 2466 H=2466:00 +CPU= 21.785
  07:06:08 STEP 2467 H=2467:00 +CPU= 13.509
  07:06:21 STEP 2468 H=2468:00 +CPU= 13.355
  07:06:42 STEP 2469 H=2469:00 +CPU= 20.744
  07:06:55 STEP 2470 H=2470:00 +CPU= 13.733
  07:07:09 STEP 2471 H=2471:00 +CPU= 13.752
  07:07:34 STEP 2472 H=2472:00 +CPU= 24.329
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002472
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002472
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002472
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002448
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002448
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002448
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002460
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002460
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002460
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0215_1986050100_123_955_12171859_0_r1605739543_102.zip
Uploading the intermediate file: upload_file_102.zip
  07:07:47 STEP 2473 H=2473:00 +CPU= 13.009
  07:08:00 STEP 2474 H=2474:00 +CPU= 13.043
Uploading trickle at timestep: 8899200
  07:08:21 STEP 2475 H=2475:00 +CPU= 21.216
  07:08:34 STEP 2476 H=2476:00 +CPU= 13.321
  07:08:48 STEP 2477 H=2477:00 +CPU= 13.083
  07:09:09 STEP 2478 H=2478:00 +CPU= 21.594
  07:09:22 STEP 2479 H=2479:00 +CPU= 13.104
  07:09:36 STEP 2480 H=2480:00 +CPU= 13.572
  07:09:57 STEP 2481 H=2481:00 +CPU= 21.237
  07:10:11 STEP 2482 H=2482:00 +CPU= 13.715
  07:10:24 STEP 2483 H=2483:00 +CPU= 13.280
  07:10:48 STEP 2484 H=2484:00 +CPU= 23.510
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002484
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002484
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002484
  07:11:01 STEP 2485 H=2485:00 +CPU= 13.026
  07:11:15 STEP 2486 H=2486:00 +CPU= 13.717
  07:11:36 STEP 2487 H=2487:00 +CPU= 21.128
  07:11:50 STEP 2488 H=2488:00 +CPU= 13.556
  07:12:03 STEP 2489 H=2489:00 +CPU= 12.931
  07:12:24 STEP 2490 H=2490:00 +CPU= 21.239
  07:12:37 STEP 2491 H=2491:00 +CPU= 12.960
  07:12:51 STEP 2492 H=2492:00 +CPU= 14.323
  07:13:13 STEP 2493 H=2493:00 +CPU= 22.089
  07:13:27 STEP 2494 H=2494:00 +CPU= 13.502
  07:13:40 STEP 2495 H=2495:00 +CPU= 13.265
  07:14:05 STEP 2496 H=2496:00 +CPU= 24.498
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002496
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002496
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002496
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002472
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002472
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002472
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002484
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002484
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002484
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0215_1986050100_123_955_12171859_0_r1605739543_103.zip
Uploading the intermediate file: upload_file_103.zip
  07:14:18 STEP 2497 H=2497:00 +CPU= 13.175
  07:14:31 STEP 2498 H=2498:00 +CPU= 13.101
  07:14:52 STEP 2499 H=2499:00 +CPU= 20.540
  07:15:05 STEP 2500 H=2500:00 +CPU= 13.581
  07:15:18 STEP 2501 H=2501:00 +CPU= 12.530
  07:15:40 STEP 2502 H=2502:00 +CPU= 22.028
  07:15:54 STEP 2503 H=2503:00 +CPU= 13.925
  07:16:09 STEP 2504 H=2504:00 +CPU= 14.492
  07:16:30 STEP 2505 H=2505:00 +CPU= 21.278
  07:16:44 STEP 2506 H=2506:00 +CPU= 13.826
  07:16:58 STEP 2507 H=2507:00 +CPU= 13.685
  07:17:21 STEP 2508 H=2508:00 +CPU= 23.457
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002508
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002508
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002508
  07:17:34 STEP 2509 H=2509:00 +CPU= 12.963
  07:17:48 STEP 2510 H=2510:00 +CPU= 13.524
  07:18:08 STEP 2511 H=2511:00 +CPU= 20.613
  07:18:22 STEP 2512 H=2512:00 +CPU= 13.941
  07:18:36 STEP 2513 H=2513:00 +CPU= 13.265
  07:18:57 STEP 2514 H=2514:00 +CPU= 20.844
  07:19:09 STEP 2515 H=2515:00 +CPU= 12.739
  07:19:23 STEP 2516 H=2516:00 +CPU= 13.674
  07:19:46 STEP 2517 H=2517:00 +CPU= 22.443
  07:19:59 STEP 2518 H=2518:00 +CPU= 13.022
  07:20:12 STEP 2519 H=2519:00 +CPU= 13.736
  07:20:37 STEP 2520 H=2520:00 +CPU= 24.550
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002520
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002520
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002520
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002496
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002496
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002496
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002508
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002508
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002508
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0215_1986050100_123_955_12171859_0_r1605739543_104.zip
Uploading the intermediate file: upload_file_104.zip
  07:20:51 STEP 2521 H=2521:00 +CPU= 14.126
  07:21:06 STEP 2522 H=2522:00 +CPU= 14.858
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1986050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0215
(argv4) batchid: 955
(argv5) wuid: 12171859
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/14
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/14/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/14/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_e13dd7f2874c95e0873bad31b82393fe to: /var/lib/boinc/slots/14/oifs_43r3_ps_0215_1986050100_123_955_12171859.zip
Unzipping the namelist zip file: /var/lib/boinc/slots/14/oifs_43r3_ps_0215_1986050100_123_955_12171859.zip
ic_ancil_file: ic_ancil_12171859
ifsdata_file: ifsdata_12171859
climate_data_file: clim_data_12171859
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_ed8406aa2b79c29c53f4546374b4ac68 to: /var/lib/boinc/slots/14/ic_ancil_12171859.zip
Unzipping the IC ancils zip file: /var/lib/boinc/slots/14/ic_ancil_12171859.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_cad7f03667f8a041ddc5160a12f01945 to: /var/lib/boinc/slots/14/ifsdata/ifsdata_12171859.zip
Unzipping the ifsdata_zip file: /var/lib/boinc/slots/14/ifsdata/ifsdata_12171859.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/14/159l_2/clim_data_12171859.zip
Unzipping the climate data zip file: /var/lib/boinc/slots/14/159l_2/clim_data_12171859.zip
Checking for progress XML file: /var/lib/boinc/slots/14/progress_file_12171859.xml
Opened progress file ok : /var/lib/boinc/slots/14/progress_file_12171859.xml
-- Model is restarting --
Adjusting last_iter, 2523, to previous model restart step.
Creating progress file: /var/lib/boinc/slots/14/progress_file_12171859.xml
last_cpu_time: 44591
upload_file_number: 105
last_iter: 2519
last_upload: 9072000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0215_1986050100_123_955_12171859_0_r1605739543
The child process has been launched with process id: 2817282
Executing the command: /var/lib/boinc/slots/14/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:IKVAK:1:1:2817282:2817282] [20221223:201439:1671822879:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:IKVAK:1:1:2817282:2817282] [20221223:201439:1671822879:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:IKVAK:1:1:2817282:2817282] [20221223:201439:1671822879:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:IKVAK:1:1:2817282:2817282] [20221223:201439:1671822879:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:IKVAK:1:1:2817282:2817282] [20221223:201439:1671822879:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:IKVAK:1:1:2817282:2817282] [20221223:201439:1671822879:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:IKVAK:1:1:2817282:2817282] [20221223:201439:1671822879:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:IKVAK:1:1:2817282:2817282] [20221223:201439:1671822879:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:IKVAK:1:1:2817282:2817282] [20221223:201439:1671822879:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:IKVAK:1:1:2817282:2817282] [20221223:201439:1671822879:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:IKVAK:1:1:2817282:2817282] [20221223:201439:1671822879:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:IKVAK:1:1:2817282:2817282] [20221223:201439:1671822879:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:IKVAK:1:1:2817282:2817282] [20221223:201439:1671822879:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:IKVAK:1:1:2817282:2817282] [20221223:201439:1671822879:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:IKVAK:1:1:2817282:2817282] [20221223:201439:1671822879:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:IKVAK:1:1:2817282:2817282] [20221223:201439:1671822879:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:IKVAK:1:1:2817282:2817282] [20221223:201439:1671822879:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:IKVAK:1:1:2817282:2817282] [20221223:201439:1671822879:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:IKVAK:1:1:2817282:2817282] [20221223:201439:1671822879:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:IKVAK:1:1:2817282:2817282] [20221223:201439:1671822879:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:IKVAK:1:1:2817282:2817282] [20221223:201439:1671822879:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:IKVAK:1:1:2817282:2817282] [20221223:201439:1671822879:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:IKVAK:1:1:2817282:2817282] [20221223:201439:1671822879:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:IKVAK:1:1:2817282:2817282] [20221223:201439:1671822879:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:IKVAK:1:1:2817282:2817282] [20221223:201439:1671822879:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:IKVAK:1:1:2817282:2817282] [20221223:201439:1671822879:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:IKVAK:1:1:2817282:2817282] [20221223:201439:1671822879:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:IKVAK:1:1:2817282:2817282] [20221223:201439:1671822879:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:IKVAK:1:1:2817282:2817282] [20221223:201439:1671822879:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:IKVAK:1:1:2817282:2817282] [20221223:201439:1671822879:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  20:15:02 STEP 2520 H=2520:00 +CPU= 21.323
  20:15:14 STEP 2521 H=2521:00 +CPU= 12.261
  20:15:28 STEP 2522 H=2522:00 +CPU= 13.519
  20:15:50 STEP 2523 H=2523:00 +CPU= 21.654
  20:16:03 STEP 2524 H=2524:00 +CPU= 13.149
  20:16:16 STEP 2525 H=2525:00 +CPU= 13.188
  20:16:38 STEP 2526 H=2526:00 +CPU= 21.680
  20:16:51 STEP 2527 H=2527:00 +CPU= 13.261
  20:17:05 STEP 2528 H=2528:00 +CPU= 13.525
  20:17:26 STEP 2529 H=2529:00 +CPU= 21.608
  20:17:40 STEP 2530 H=2530:00 +CPU= 13.262
  20:17:54 STEP 2531 H=2531:00 +CPU= 13.871
  20:18:18 STEP 2532 H=2532:00 +CPU= 24.343
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002532
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002532
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002532
  20:18:31 STEP 2533 H=2533:00 +CPU= 12.722
  20:18:45 STEP 2534 H=2534:00 +CPU= 13.757
  20:19:08 STEP 2535 H=2535:00 +CPU= 22.779
  20:19:21 STEP 2536 H=2536:00 +CPU= 13.464
  20:19:37 STEP 2537 H=2537:00 +CPU= 15.645
  20:20:00 STEP 2538 H=2538:00 +CPU= 23.355
  20:20:16 STEP 2539 H=2539:00 +CPU= 15.581
  20:20:31 STEP 2540 H=2540:00 +CPU= 15.438
  20:20:55 STEP 2541 H=2541:00 +CPU= 23.074
  20:21:11 STEP 2542 H=2542:00 +CPU= 15.915
  20:21:27 STEP 2543 H=2543:00 +CPU= 16.010
  20:21:55 STEP 2544 H=2544:00 +CPU= 28.487
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002544
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002544
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002520
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002520
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002520
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002532
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002532
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002532
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0215_1986050100_123_955_12171859_0_r1605739543_105.zip
Uploading the intermediate file: upload_file_105.zip
  20:22:09 STEP 2545 H=2545:00 +CPU= 13.348
  20:22:24 STEP 2546 H=2546:00 +CPU= 15.636
  20:22:50 STEP 2547 H=2547:00 +CPU= 25.579
  20:23:05 STEP 2548 H=2548:00 +CPU= 14.774
  20:23:20 STEP 2549 H=2549:00 +CPU= 14.865
  20:23:43 STEP 2550 H=2550:00 +CPU= 22.990
  20:23:56 STEP 2551 H=2551:00 +CPU= 13.413
  20:24:13 STEP 2552 H=2552:00 +CPU= 16.244
  20:24:35 STEP 2553 H=2553:00 +CPU= 22.454
  20:24:51 STEP 2554 H=2554:00 +CPU= 15.373
  20:25:05 STEP 2555 H=2555:00 +CPU= 14.123
  20:25:30 STEP 2556 H=2556:00 +CPU= 24.910
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002556
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002556
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002556
  20:25:43 STEP 2557 H=2557:00 +CPU= 12.733
  20:25:59 STEP 2558 H=2558:00 +CPU= 16.315
  20:26:23 STEP 2559 H=2559:00 +CPU= 24.060
  20:26:39 STEP 2560 H=2560:00 +CPU= 15.519
  20:26:53 STEP 2561 H=2561:00 +CPU= 14.064
  20:27:18 STEP 2562 H=2562:00 +CPU= 25.698
  20:27:32 STEP 2563 H=2563:00 +CPU= 13.735
  20:27:48 STEP 2564 H=2564:00 +CPU= 15.854
  20:28:11 STEP 2565 H=2565:00 +CPU= 22.522
  20:28:24 STEP 2566 H=2566:00 +CPU= 13.550
  20:28:39 STEP 2567 H=2567:00 +CPU= 15.045
  20:29:07 STEP 2568 H=2568:00 +CPU= 27.746
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002568
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002568
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002544
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002544
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002556
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002556
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002556
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0215_1986050100_123_955_12171859_0_r1605739543_106.zip
Uploading the intermediate file: upload_file_106.zip
  20:29:22 STEP 2569 H=2569:00 +CPU= 14.444
  20:29:37 STEP 2570 H=2570:00 +CPU= 15.470
  20:30:01 STEP 2571 H=2571:00 +CPU= 23.213
  20:30:17 STEP 2572 H=2572:00 +CPU= 16.682
  20:30:32 STEP 2573 H=2573:00 +CPU= 14.951
  20:30:56 STEP 2574 H=2574:00 +CPU= 23.347
  20:31:10 STEP 2575 H=2575:00 +CPU= 14.313
  20:31:25 STEP 2576 H=2576:00 +CPU= 14.595
  20:31:49 STEP 2577 H=2577:00 +CPU= 24.489
  20:32:06 STEP 2578 H=2578:00 +CPU= 16.053
  20:32:21 STEP 2579 H=2579:00 +CPU= 14.990
  20:32:48 STEP 2580 H=2580:00 +CPU= 26.776
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002580
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002580
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002580
  20:33:02 STEP 2581 H=2581:00 +CPU= 14.125
  20:33:17 STEP 2582 H=2582:00 +CPU= 15.532
  20:33:41 STEP 2583 H=2583:00 +CPU= 23.610
  20:33:57 STEP 2584 H=2584:00 +CPU= 15.957
  20:34:11 STEP 2585 H=2585:00 +CPU= 13.519
  20:34:36 STEP 2586 H=2586:00 +CPU= 25.431
  20:34:51 STEP 2587 H=2587:00 +CPU= 15.032
  20:35:08 STEP 2588 H=2588:00 +CPU= 16.369
  20:35:31 STEP 2589 H=2589:00 +CPU= 22.852
  20:35:45 STEP 2590 H=2590:00 +CPU= 14.771
  20:36:01 STEP 2591 H=2591:00 +CPU= 15.269
  20:36:30 STEP 2592 H=2592:00 +CPU= 29.089
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002592
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002592
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002592
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002568
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002568
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002580
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002580
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002580
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0215_1986050100_123_955_12171859_0_r1605739543_107.zip
Uploading the intermediate file: upload_file_107.zip
  20:36:45 STEP 2593 H=2593:00 +CPU= 14.740
  20:37:00 STEP 2594 H=2594:00 +CPU= 15.122
  20:37:23 STEP 2595 H=2595:00 +CPU= 23.299
  20:37:37 STEP 2596 H=2596:00 +CPU= 14.126
  20:37:51 STEP 2597 H=2597:00 +CPU= 13.844
  20:38:15 STEP 2598 H=2598:00 +CPU= 23.687
  20:38:33 STEP 2599 H=2599:00 +CPU= 17.656
  20:38:47 STEP 2600 H=2600:00 +CPU= 14.461
  20:39:12 STEP 2601 H=2601:00 +CPU= 24.511
  20:39:28 STEP 2602 H=2602:00 +CPU= 15.835
  20:39:45 STEP 2603 H=2603:00 +CPU= 17.098
  20:40:10 STEP 2604 H=2604:00 +CPU= 25.249
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002604
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002604
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002604
  20:40:26 STEP 2605 H=2605:00 +CPU= 15.783
  20:40:40 STEP 2606 H=2606:00 +CPU= 14.155
  20:41:05 STEP 2607 H=2607:00 +CPU= 24.559
  20:41:21 STEP 2608 H=2608:00 +CPU= 16.163
  20:41:38 STEP 2609 H=2609:00 +CPU= 16.373
  20:42:02 STEP 2610 H=2610:00 +CPU= 24.328
  20:42:17 STEP 2611 H=2611:00 +CPU= 14.594
  20:42:32 STEP 2612 H=2612:00 +CPU= 15.261
  20:42:55 STEP 2613 H=2613:00 +CPU= 23.101
  20:43:10 STEP 2614 H=2614:00 +CPU= 14.690
  20:43:25 STEP 2615 H=2615:00 +CPU= 15.337
  20:43:52 STEP 2616 H=2616:00 +CPU= 26.833
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002616
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002616
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002616
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002592
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002592
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002592
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002604
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002604
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002604
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0215_1986050100_123_955_12171859_0_r1605739543_108.zip
Uploading the intermediate file: upload_file_108.zip
  20:44:08 STEP 2617 H=2617:00 +CPU= 15.994
  20:44:23 STEP 2618 H=2618:00 +CPU= 14.246
  20:44:47 STEP 2619 H=2619:00 +CPU= 24.377
  20:45:03 STEP 2620 H=2620:00 +CPU= 15.907
  20:45:20 STEP 2621 H=2621:00 +CPU= 16.587
  20:45:43 STEP 2622 H=2622:00 +CPU= 23.021
  20:45:58 STEP 2623 H=2623:00 +CPU= 14.809
  20:46:14 STEP 2624 H=2624:00 +CPU= 16.155
  20:46:38 STEP 2625 H=2625:00 +CPU= 23.784
  20:46:51 STEP 2626 H=2626:00 +CPU= 13.351
  20:47:06 STEP 2627 H=2627:00 +CPU= 14.685
  20:47:33 STEP 2628 H=2628:00 +CPU= 26.573
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002628
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002628
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002628
  20:47:48 STEP 2629 H=2629:00 +CPU= 14.784
  20:48:01 STEP 2630 H=2630:00 +CPU= 13.442
  20:48:25 STEP 2631 H=2631:00 +CPU= 23.571
  20:48:40 STEP 2632 H=2632:00 +CPU= 15.534
  20:48:57 STEP 2633 H=2633:00 +CPU= 16.318
  20:49:20 STEP 2634 H=2634:00 +CPU= 23.253
  20:49:35 STEP 2635 H=2635:00 +CPU= 14.579
  20:49:50 STEP 2636 H=2636:00 +CPU= 14.820
  20:50:14 STEP 2637 H=2637:00 +CPU= 24.686
  20:50:28 STEP 2638 H=2638:00 +CPU= 13.780
  20:50:45 STEP 2639 H=2639:00 +CPU= 16.582
  20:51:13 STEP 2640 H=2640:00 +CPU= 27.488
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002640
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002640
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002640
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002616
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002616
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002616
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002628
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002628
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002628
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0215_1986050100_123_955_12171859_0_r1605739543_109.zip
Uploading the intermediate file: upload_file_109.zip
  20:51:28 STEP 2641 H=2641:00 +CPU= 14.931
  20:51:42 STEP 2642 H=2642:00 +CPU= 13.987
  20:52:05 STEP 2643 H=2643:00 +CPU= 23.637
  20:52:21 STEP 2644 H=2644:00 +CPU= 15.427
  20:52:37 STEP 2645 H=2645:00 +CPU= 15.721
  20:52:59 STEP 2646 H=2646:00 +CPU= 22.251
  20:53:13 STEP 2647 H=2647:00 +CPU= 13.992
  20:53:28 STEP 2648 H=2648:00 +CPU= 15.446
  20:53:53 STEP 2649 H=2649:00 +CPU= 23.946
  20:54:08 STEP 2650 H=2650:00 +CPU= 15.375
  20:54:23 STEP 2651 H=2651:00 +CPU= 15.118
  20:54:49 STEP 2652 H=2652:00 +CPU= 25.882
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002652
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002652
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002652
  20:55:03 STEP 2653 H=2653:00 +CPU= 14.132
  20:55:18 STEP 2654 H=2654:00 +CPU= 14.567
  20:55:40 STEP 2655 H=2655:00 +CPU= 22.284
  20:55:56 STEP 2656 H=2656:00 +CPU= 15.715
  20:56:11 STEP 2657 H=2657:00 +CPU= 15.406
  20:56:35 STEP 2658 H=2658:00 +CPU= 23.613
  20:56:48 STEP 2659 H=2659:00 +CPU= 12.668
  20:57:02 STEP 2660 H=2660:00 +CPU= 14.319
  20:57:26 STEP 2661 H=2661:00 +CPU= 23.599
  20:57:41 STEP 2662 H=2662:00 +CPU= 15.012
  20:57:54 STEP 2663 H=2663:00 +CPU= 13.497
  20:58:21 STEP 2664 H=2664:00 +CPU= 26.437
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002664
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002664
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002664
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002640
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002640
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002640
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002652
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002652
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002652
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0215_1986050100_123_955_12171859_0_r1605739543_110.zip
Uploading the intermediate file: upload_file_110.zip
  20:58:36 STEP 2665 H=2665:00 +CPU= 14.498
  20:58:51 STEP 2666 H=2666:00 +CPU= 15.857
  20:59:14 STEP 2667 H=2667:00 +CPU= 22.273
  20:59:30 STEP 2668 H=2668:00 +CPU= 16.011
  20:59:44 STEP 2669 H=2669:00 +CPU= 13.897
  21:00:07 STEP 2670 H=2670:00 +CPU= 23.406
  21:00:21 STEP 2671 H=2671:00 +CPU= 13.983
  21:00:40 STEP 2672 H=2672:00 +CPU= 18.371
  21:01:05 STEP 2673 H=2673:00 +CPU= 25.423
  21:01:21 STEP 2674 H=2674:00 +CPU= 15.886
  21:01:35 STEP 2675 H=2675:00 +CPU= 14.080
  21:02:02 STEP 2676 H=2676:00 +CPU= 26.835
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002676
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002676
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002676
  21:02:18 STEP 2677 H=2677:00 +CPU= 15.654
  21:02:33 STEP 2678 H=2678:00 +CPU= 15.136
  21:02:55 STEP 2679 H=2679:00 +CPU= 21.654
  21:03:11 STEP 2680 H=2680:00 +CPU= 16.235
  21:03:27 STEP 2681 H=2681:00 +CPU= 15.924
  21:03:51 STEP 2682 H=2682:00 +CPU= 23.544
  21:04:04 STEP 2683 H=2683:00 +CPU= 12.959
  21:04:20 STEP 2684 H=2684:00 +CPU= 16.108
  21:04:44 STEP 2685 H=2685:00 +CPU= 24.082
  21:05:00 STEP 2686 H=2686:00 +CPU= 15.762
  21:05:14 STEP 2687 H=2687:00 +CPU= 14.442
  21:05:42 STEP 2688 H=2688:00 +CPU= 27.723
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002688
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002688
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002688
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002664
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002664
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002664
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002676
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002676
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002676
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0215_1986050100_123_955_12171859_0_r1605739543_111.zip
Uploading the intermediate file: upload_file_111.zip
  21:05:58 STEP 2689 H=2689:00 +CPU= 15.231
  21:06:14 STEP 2690 H=2690:00 +CPU= 15.998
  21:06:36 STEP 2691 H=2691:00 +CPU= 21.938
  21:06:52 STEP 2692 H=2692:00 +CPU= 15.994
  21:07:08 STEP 2693 H=2693:00 +CPU= 15.885
  21:07:32 STEP 2694 H=2694:00 +CPU= 24.500
  21:07:46 STEP 2695 H=2695:00 +CPU= 13.343
  21:08:01 STEP 2696 H=2696:00 +CPU= 15.742
  21:08:28 STEP 2697 H=2697:00 +CPU= 26.350
  21:08:43 STEP 2698 H=2698:00 +CPU= 15.064
  21:08:57 STEP 2699 H=2699:00 +CPU= 13.656
  21:09:23 STEP 2700 H=2700:00 +CPU= 25.884
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002700
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002700
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002700
  21:09:37 STEP 2701 H=2701:00 +CPU= 14.577
  21:09:53 STEP 2702 H=2702:00 +CPU= 15.602
  21:10:16 STEP 2703 H=2703:00 +CPU= 23.346
  21:10:31 STEP 2704 H=2704:00 +CPU= 15.019
  21:10:45 STEP 2705 H=2705:00 +CPU= 14.095
  21:11:10 STEP 2706 H=2706:00 +CPU= 24.482
  21:11:24 STEP 2707 H=2707:00 +CPU= 13.482
  21:11:41 STEP 2708 H=2708:00 +CPU= 16.924
  21:12:04 STEP 2709 H=2709:00 +CPU= 22.946
  21:12:18 STEP 2710 H=2710:00 +CPU= 14.513
  21:12:33 STEP 2711 H=2711:00 +CPU= 15.147
  21:13:01 STEP 2712 H=2712:00 +CPU= 27.791
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002712
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002712
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002712
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002688
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002688
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002688
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002700
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002700
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002700
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0215_1986050100_123_955_12171859_0_r1605739543_112.zip
Uploading the intermediate file: upload_file_112.zip
  21:13:15 STEP 2713 H=2713:00 +CPU= 14.092
  21:13:30 STEP 2714 H=2714:00 +CPU= 14.381
  21:13:54 STEP 2715 H=2715:00 +CPU= 23.858
  21:14:09 STEP 2716 H=2716:00 +CPU= 15.241
  21:14:24 STEP 2717 H=2717:00 +CPU= 14.455
  21:14:50 STEP 2718 H=2718:00 +CPU= 26.271
  21:15:04 STEP 2719 H=2719:00 +CPU= 14.130
  21:15:19 STEP 2720 H=2720:00 +CPU= 15.033
  21:15:45 STEP 2721 H=2721:00 +CPU= 25.519
  21:15:59 STEP 2722 H=2722:00 +CPU= 13.809
  21:16:14 STEP 2723 H=2723:00 +CPU= 15.079
  21:16:41 STEP 2724 H=2724:00 +CPU= 26.682
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002724
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002724
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002724
  21:16:57 STEP 2725 H=2725:00 +CPU= 16.029
  21:17:10 STEP 2726 H=2726:00 +CPU= 13.273
  21:17:34 STEP 2727 H=2727:00 +CPU= 23.998
  21:17:48 STEP 2728 H=2728:00 +CPU= 14.061
  21:18:03 STEP 2729 H=2729:00 +CPU= 14.389
  21:18:27 STEP 2730 H=2730:00 +CPU= 24.361
  21:18:41 STEP 2731 H=2731:00 +CPU= 13.701
  21:18:58 STEP 2732 H=2732:00 +CPU= 16.580
  21:19:22 STEP 2733 H=2733:00 +CPU= 24.555
  21:19:40 STEP 2734 H=2734:00 +CPU= 17.208
  21:19:53 STEP 2735 H=2735:00 +CPU= 13.798
  21:20:22 STEP 2736 H=2736:00 +CPU= 28.044
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002736
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002736
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002736
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002712
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002712
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002712
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002724
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002724
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002724
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0215_1986050100_123_955_12171859_0_r1605739543_113.zip
Uploading the intermediate file: upload_file_113.zip
  21:20:37 STEP 2737 H=2737:00 +CPU= 15.746
  21:20:54 STEP 2738 H=2738:00 +CPU= 16.220
  21:21:16 STEP 2739 H=2739:00 +CPU= 22.197
  21:21:32 STEP 2740 H=2740:00 +CPU= 15.559
  21:21:47 STEP 2741 H=2741:00 +CPU= 15.738
  21:22:12 STEP 2742 H=2742:00 +CPU= 24.269
  21:22:24 STEP 2743 H=2743:00 +CPU= 12.281
  21:22:40 STEP 2744 H=2744:00 +CPU= 15.681
  21:23:04 STEP 2745 H=2745:00 +CPU= 23.527
  21:23:20 STEP 2746 H=2746:00 +CPU= 16.626
  21:23:34 STEP 2747 H=2747:00 +CPU= 13.318
  21:23:59 STEP 2748 H=2748:00 +CPU= 25.385
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002748
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002748
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002748
  21:24:14 STEP 2749 H=2749:00 +CPU= 15.325
  21:24:29 STEP 2750 H=2750:00 +CPU= 14.671
  21:24:52 STEP 2751 H=2751:00 +CPU= 22.461
  21:25:05 STEP 2752 H=2752:00 +CPU= 13.375
  21:25:20 STEP 2753 H=2753:00 +CPU= 14.826
  21:25:45 STEP 2754 H=2754:00 +CPU= 24.980
  21:26:01 STEP 2755 H=2755:00 +CPU= 16.205
  21:26:15 STEP 2756 H=2756:00 +CPU= 14.039
  21:26:38 STEP 2757 H=2757:00 +CPU= 22.547
  21:26:54 STEP 2758 H=2758:00 +CPU= 15.905
  21:27:11 STEP 2759 H=2759:00 +CPU= 17.200
  21:27:38 STEP 2760 H=2760:00 +CPU= 26.632
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002760
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002760
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002760
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002736
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002736
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002736
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002748
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002748
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002748
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0215_1986050100_123_955_12171859_0_r1605739543_114.zip
Uploading the intermediate file: upload_file_114.zip
  21:27:53 STEP 2761 H=2761:00 +CPU= 14.641
Uploading trickle at timestep: 9936000
  21:28:10 STEP 2762 H=2762:00 +CPU= 17.549
  21:28:34 STEP 2763 H=2763:00 +CPU= 23.353
  21:28:47 STEP 2764 H=2764:00 +CPU= 12.981
  21:29:01 STEP 2765 H=2765:00 +CPU= 14.344
  21:29:24 STEP 2766 H=2766:00 +CPU= 22.630
  21:29:41 STEP 2767 H=2767:00 +CPU= 16.790
  21:29:55 STEP 2768 H=2768:00 +CPU= 14.636
  21:30:18 STEP 2769 H=2769:00 +CPU= 22.633
  21:30:33 STEP 2770 H=2770:00 +CPU= 15.147
  21:30:48 STEP 2771 H=2771:00 +CPU= 14.710
  21:31:15 STEP 2772 H=2772:00 +CPU= 26.524
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002772
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002772
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002772
  21:31:29 STEP 2773 H=2773:00 +CPU= 13.802
  21:31:44 STEP 2774 H=2774:00 +CPU= 15.572
  21:32:10 STEP 2775 H=2775:00 +CPU= 25.435
  21:32:25 STEP 2776 H=2776:00 +CPU= 15.420
  21:32:40 STEP 2777 H=2777:00 +CPU= 15.059
  21:33:05 STEP 2778 H=2778:00 +CPU= 24.341
  21:33:19 STEP 2779 H=2779:00 +CPU= 14.564
  21:33:36 STEP 2780 H=2780:00 +CPU= 16.049
  21:33:58 STEP 2781 H=2781:00 +CPU= 22.295
  21:34:15 STEP 2782 H=2782:00 +CPU= 16.617
  21:34:30 STEP 2783 H=2783:00 +CPU= 15.614
  21:34:57 STEP 2784 H=2784:00 +CPU= 26.946
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002784
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002784
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002784
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002760
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002760
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002760
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002772
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002772
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002772
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0215_1986050100_123_955_12171859_0_r1605739543_115.zip
Uploading the intermediate file: upload_file_115.zip
  21:35:12 STEP 2785 H=2785:00 +CPU= 14.282
  21:35:27 STEP 2786 H=2786:00 +CPU= 15.085
  21:35:51 STEP 2787 H=2787:00 +CPU= 23.893
  21:36:06 STEP 2788 H=2788:00 +CPU= 14.829
  21:36:19 STEP 2789 H=2789:00 +CPU= 13.758
  21:36:44 STEP 2790 H=2790:00 +CPU= 24.387
  21:36:59 STEP 2791 H=2791:00 +CPU= 14.754
  21:37:14 STEP 2792 H=2792:00 +CPU= 15.626
  21:37:38 STEP 2793 H=2793:00 +CPU= 23.868
  21:37:52 STEP 2794 H=2794:00 +CPU= 14.170
  21:38:08 STEP 2795 H=2795:00 +CPU= 15.124
  21:38:34 STEP 2796 H=2796:00 +CPU= 25.948
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002796
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002796
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002796
  21:38:48 STEP 2797 H=2797:00 +CPU= 14.580
  21:39:03 STEP 2798 H=2798:00 +CPU= 14.412
  21:39:28 STEP 2799 H=2799:00 +CPU= 24.789
  21:39:44 STEP 2800 H=2800:00 +CPU= 16.479
  21:40:00 STEP 2801 H=2801:00 +CPU= 15.429
  21:40:25 STEP 2802 H=2802:00 +CPU= 24.973
  21:40:40 STEP 2803 H=2803:00 +CPU= 14.919
  21:40:55 STEP 2804 H=2804:00 +CPU= 15.310
  21:41:18 STEP 2805 H=2805:00 +CPU= 22.642
  21:41:33 STEP 2806 H=2806:00 +CPU= 15.235
  21:41:48 STEP 2807 H=2807:00 +CPU= 14.801
  21:42:15 STEP 2808 H=2808:00 +CPU= 26.620
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002808
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002808
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002808
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002784
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002784
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002784
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002796
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002796
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002796
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0215_1986050100_123_955_12171859_0_r1605739543_116.zip
Uploading the intermediate file: upload_file_116.zip
  21:42:29 STEP 2809 H=2809:00 +CPU= 13.995
  21:42:43 STEP 2810 H=2810:00 +CPU= 14.664
  21:43:08 STEP 2811 H=2811:00 +CPU= 24.742
  21:43:23 STEP 2812 H=2812:00 +CPU= 14.981
  21:43:36 STEP 2813 H=2813:00 +CPU= 13.206
  21:44:01 STEP 2814 H=2814:00 +CPU= 24.621
  21:44:16 STEP 2815 H=2815:00 +CPU= 15.141
  21:44:31 STEP 2816 H=2816:00 +CPU= 15.093
  21:44:53 STEP 2817 H=2817:00 +CPU= 21.873
  21:45:09 STEP 2818 H=2818:00 +CPU= 15.797
  21:45:24 STEP 2819 H=2819:00 +CPU= 15.006
  21:45:50 STEP 2820 H=2820:00 +CPU= 25.813
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002820
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002820
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002820
  21:46:04 STEP 2821 H=2821:00 +CPU= 13.886
  21:46:19 STEP 2822 H=2822:00 +CPU= 14.464
  21:46:44 STEP 2823 H=2823:00 +CPU= 24.943
  21:47:00 STEP 2824 H=2824:00 +CPU= 15.827
  21:47:16 STEP 2825 H=2825:00 +CPU= 16.093
  21:47:37 STEP 2826 H=2826:00 +CPU= 21.375
  21:47:53 STEP 2827 H=2827:00 +CPU= 15.432
  21:48:07 STEP 2828 H=2828:00 +CPU= 14.612
  21:48:31 STEP 2829 H=2829:00 +CPU= 23.512
  21:48:44 STEP 2830 H=2830:00 +CPU= 13.596
  21:48:59 STEP 2831 H=2831:00 +CPU= 14.740
  21:49:27 STEP 2832 H=2832:00 +CPU= 27.459
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002832
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002832
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002832
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002808
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002808
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002808
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002820
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002820
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002820
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0215_1986050100_123_955_12171859_0_r1605739543_117.zip
Uploading the intermediate file: upload_file_117.zip
  21:49:40 STEP 2833 H=2833:00 +CPU= 13.510
  21:49:55 STEP 2834 H=2834:00 +CPU= 14.932
  21:50:20 STEP 2835 H=2835:00 +CPU= 24.247
  21:50:34 STEP 2836 H=2836:00 +CPU= 14.747
  21:50:48 STEP 2837 H=2837:00 +CPU= 13.847
  21:51:13 STEP 2838 H=2838:00 +CPU= 24.204
  21:51:28 STEP 2839 H=2839:00 +CPU= 15.503
  21:51:43 STEP 2840 H=2840:00 +CPU= 14.781
  21:52:05 STEP 2841 H=2841:00 +CPU= 21.598
  21:52:20 STEP 2842 H=2842:00 +CPU= 15.244
  21:52:35 STEP 2843 H=2843:00 +CPU= 14.646
  21:53:01 STEP 2844 H=2844:00 +CPU= 26.069
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002844
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002844
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002844
  21:53:14 STEP 2845 H=2845:00 +CPU= 13.130
  21:53:30 STEP 2846 H=2846:00 +CPU= 15.440
  21:53:54 STEP 2847 H=2847:00 +CPU= 24.393
  21:54:10 STEP 2848 H=2848:00 +CPU= 16.375
  21:54:27 STEP 2849 H=2849:00 +CPU= 16.707
  21:54:51 STEP 2850 H=2850:00 +CPU= 23.301
  21:55:06 STEP 2851 H=2851:00 +CPU= 15.241
  21:55:19 STEP 2852 H=2852:00 +CPU= 13.348
  21:55:44 STEP 2853 H=2853:00 +CPU= 25.057
  21:56:00 STEP 2854 H=2854:00 +CPU= 15.125
  21:56:16 STEP 2855 H=2855:00 +CPU= 16.308
  21:56:41 STEP 2856 H=2856:00 +CPU= 24.758
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002856
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002856
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002856
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002832
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002832
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002832
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002844
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002844
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002844
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0215_1986050100_123_955_12171859_0_r1605739543_118.zip
Uploading the intermediate file: upload_file_118.zip
  21:56:56 STEP 2857 H=2857:00 +CPU= 14.651
  21:57:12 STEP 2858 H=2858:00 +CPU= 16.196
  21:57:36 STEP 2859 H=2859:00 +CPU= 24.071
  21:57:49 STEP 2860 H=2860:00 +CPU= 13.454
  21:58:02 STEP 2861 H=2861:00 +CPU= 12.197
  21:58:24 STEP 2862 H=2862:00 +CPU= 22.578
  21:58:40 STEP 2863 H=2863:00 +CPU= 16.037
  21:58:53 STEP 2864 H=2864:00 +CPU= 12.831
  21:59:14 STEP 2865 H=2865:00 +CPU= 20.905
  21:59:29 STEP 2866 H=2866:00 +CPU= 14.770
  21:59:44 STEP 2867 H=2867:00 +CPU= 15.186
  22:00:12 STEP 2868 H=2868:00 +CPU= 27.055
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002868
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002868
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002868
  22:00:24 STEP 2869 H=2869:00 +CPU= 12.529
  22:00:38 STEP 2870 H=2870:00 +CPU= 13.616
  22:01:01 STEP 2871 H=2871:00 +CPU= 23.259
  22:01:15 STEP 2872 H=2872:00 +CPU= 14.268
  22:01:31 STEP 2873 H=2873:00 +CPU= 15.985
  22:01:53 STEP 2874 H=2874:00 +CPU= 21.477
  22:02:06 STEP 2875 H=2875:00 +CPU= 12.884
  22:02:21 STEP 2876 H=2876:00 +CPU= 14.694
  22:02:46 STEP 2877 H=2877:00 +CPU= 24.776
  22:03:00 STEP 2878 H=2878:00 +CPU= 14.413
  22:03:13 STEP 2879 H=2879:00 +CPU= 12.655
  22:03:40 STEP 2880 H=2880:00 +CPU= 26.919
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002880
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002880
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002880
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002856
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002856
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002856
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002868
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002868
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002868
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0215_1986050100_123_955_12171859_0_r1605739543_119.zip
Uploading the intermediate file: upload_file_119.zip
  22:03:54 STEP 2881 H=2881:00 +CPU= 14.600
  22:04:08 STEP 2882 H=2882:00 +CPU= 14.047
  22:04:30 STEP 2883 H=2883:00 +CPU= 21.452
  22:04:45 STEP 2884 H=2884:00 +CPU= 14.535
  22:04:59 STEP 2885 H=2885:00 +CPU= 14.207
  22:05:23 STEP 2886 H=2886:00 +CPU= 24.119
  22:05:36 STEP 2887 H=2887:00 +CPU= 12.650
  22:05:49 STEP 2888 H=2888:00 +CPU= 12.782
  22:06:11 STEP 2889 H=2889:00 +CPU= 22.664
  22:06:25 STEP 2890 H=2890:00 +CPU= 13.927
  22:06:40 STEP 2891 H=2891:00 +CPU= 14.786
  22:07:01 STEP 2892 H=2892:00 +CPU= 21.261
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002892
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002892
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002892
  22:07:17 STEP 2893 H=2893:00 +CPU= 15.226
  22:07:31 STEP 2894 H=2894:00 +CPU= 14.747
  22:07:56 STEP 2895 H=2895:00 +CPU= 24.146
  22:08:09 STEP 2896 H=2896:00 +CPU= 12.831
  22:08:21 STEP 2897 H=2897:00 +CPU= 12.455
  22:08:44 STEP 2898 H=2898:00 +CPU= 23.292
  22:08:59 STEP 2899 H=2899:00 +CPU= 14.587
  22:09:15 STEP 2900 H=2900:00 +CPU= 15.715
  22:09:34 STEP 2901 H=2901:00 +CPU= 18.771
  22:09:48 STEP 2902 H=2902:00 +CPU= 14.387
  22:10:04 STEP 2903 H=2903:00 +CPU= 15.541
  22:10:31 STEP 2904 H=2904:00 +CPU= 27.030
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002904
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002904
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002904
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002880
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002880
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002880
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002892
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002892
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002892
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0215_1986050100_123_955_12171859_0_r1605739543_120.zip
Uploading the intermediate file: upload_file_120.zip
  22:10:43 STEP 2905 H=2905:00 +CPU= 11.794
  22:10:55 STEP 2906 H=2906:00 +CPU= 12.148
  22:11:18 STEP 2907 H=2907:00 +CPU= 23.071
  22:11:33 STEP 2908 H=2908:00 +CPU= 15.373
  22:11:48 STEP 2909 H=2909:00 +CPU= 14.420
  22:12:09 STEP 2910 H=2910:00 +CPU= 21.567
  22:12:25 STEP 2911 H=2911:00 +CPU= 15.982
  22:12:40 STEP 2912 H=2912:00 +CPU= 14.415
  22:13:02 STEP 2913 H=2913:00 +CPU= 22.452
  22:13:15 STEP 2914 H=2914:00 +CPU= 12.097
  22:13:29 STEP 2915 H=2915:00 +CPU= 14.470
  22:13:54 STEP 2916 H=2916:00 +CPU= 24.959
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002916
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002916
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002916
  22:14:08 STEP 2917 H=2917:00 +CPU= 14.336
  22:14:22 STEP 2918 H=2918:00 +CPU= 13.344
  22:14:44 STEP 2919 H=2919:00 +CPU= 22.105
  22:15:00 STEP 2920 H=2920:00 +CPU= 16.002
  22:15:15 STEP 2921 H=2921:00 +CPU= 15.208
  22:15:37 STEP 2922 H=2922:00 +CPU= 21.908
  22:15:52 STEP 2923 H=2923:00 +CPU= 14.695
  22:16:07 STEP 2924 H=2924:00 +CPU= 15.385
  22:16:32 STEP 2925 H=2925:00 +CPU= 24.359
  22:16:47 STEP 2926 H=2926:00 +CPU= 14.976
  22:16:59 STEP 2927 H=2927:00 +CPU= 12.388
  22:17:23 STEP 2928 H=2928:00 +CPU= 23.118
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002928
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002928
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002928
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002904
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002904
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002904
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002916
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002916
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002916
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0215_1986050100_123_955_12171859_0_r1605739543_121.zip
Uploading the intermediate file: upload_file_121.zip
  22:17:39 STEP 2929 H=2929:00 +CPU= 16.552
  22:17:55 STEP 2930 H=2930:00 +CPU= 15.575
  22:18:16 STEP 2931 H=2931:00 +CPU= 20.705
  22:18:29 STEP 2932 H=2932:00 +CPU= 13.267
  22:18:44 STEP 2933 H=2933:00 +CPU= 15.103
  22:19:08 STEP 2934 H=2934:00 +CPU= 23.650
  22:19:22 STEP 2935 H=2935:00 +CPU= 14.337
  22:19:36 STEP 2936 H=2936:00 +CPU= 14.147
  22:19:57 STEP 2937 H=2937:00 +CPU= 20.588
  22:20:12 STEP 2938 H=2938:00 +CPU= 14.683
  22:20:27 STEP 2939 H=2939:00 +CPU= 14.806
  22:20:50 STEP 2940 H=2940:00 +CPU= 23.333
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002940
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002940
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002940
  22:21:03 STEP 2941 H=2941:00 +CPU= 12.588
  22:21:16 STEP 2942 H=2942:00 +CPU= 13.063
  22:21:39 STEP 2943 H=2943:00 +CPU= 22.777
  22:21:53 STEP 2944 H=2944:00 +CPU= 14.740
  22:22:08 STEP 2945 H=2945:00 +CPU= 14.488
  22:22:30 STEP 2946 H=2946:00 +CPU= 22.226
  22:22:46 STEP 2947 H=2947:00 +CPU= 15.934
  22:23:01 STEP 2948 H=2948:00 +CPU= 14.191
  22:23:22 STEP 2949 H=2949:00 +CPU= 21.503
  22:23:35 STEP 2950 H=2950:00 +CPU= 12.549
  22:23:50 STEP 2951 H=2951:00 +CPU= 15.267
  22:24:17 STEP 2952 H=2952:00 +CPU= 26.639
The child process terminated with status: 0
Moving to projects directory: /var/lib/boinc/slots/14/ICMGGhq0f+002952
Moving to projects directory: /var/lib/boinc/slots/14/ICMSHhq0f+002952
Moving to projects directory: /var/lib/boinc/slots/14/ICMUAhq0f+002952
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002928
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002928
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002928
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002940
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002940
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002940
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMGGhq0f+002952
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMSHhq0f+002952
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12171859/ICMUAhq0f+002952
Zipping up the final file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0215_1986050100_123_955_12171859_0_r1605739543_122.zip
Uploading the final file: upload_file_122.zip
Uploading trickle at timestep: 10623600
22:27:39 (2817275): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 15:51:00 1532181 22256243 oifs_43r3_ps_0215_1986050100_123_955_12171859_0 10,623,600 52,340 0.0049
09 May 2023 15:50:55 1532181 22256243 oifs_43r3_ps_0215_1986050100_123_955_12171859_0 9,936,000 48,954 0.0049
09 May 2023 15:49:39 1532181 22256243 oifs_43r3_ps_0215_1986050100_123_955_12171859_0 8,899,200 43,781 0.0049
09 May 2023 15:49:28 1532181 22256243 oifs_43r3_ps_0215_1986050100_123_955_12171859_0 8,035,200 39,882 0.0050
09 May 2023 15:49:23 1532181 22256243 oifs_43r3_ps_0215_1986050100_123_955_12171859_0 7,171,200 35,655 0.0050
09 May 2023 15:49:17 1532181 22256243 oifs_43r3_ps_0215_1986050100_123_955_12171859_0 6,307,200 31,084 0.0049
09 May 2023 15:49:06 1532181 22256243 oifs_43r3_ps_0215_1986050100_123_955_12171859_0 5,270,400 25,387 0.0048
09 May 2023 15:49:00 1532181 22256243 oifs_43r3_ps_0215_1986050100_123_955_12171859_0 4,406,400 21,161 0.0048
09 May 2023 15:48:55 1532181 22256243 oifs_43r3_ps_0215_1986050100_123_955_12171859_0 3,542,400 17,113 0.0048
09 May 2023 15:48:49 1532181 22256243 oifs_43r3_ps_0215_1986050100_123_955_12171859_0 2,678,400 13,078 0.0049
09 May 2023 15:48:43 1532181 22256243 oifs_43r3_ps_0215_1986050100_123_955_12171859_0 1,814,400 9,332 0.0051
09 May 2023 15:47:58 1532181 22256243 oifs_43r3_ps_0215_1986050100_123_955_12171859_0 950,400 5,546 0.0058


©2024 climateprediction.net