climateprediction.net home page
Task 22302976

Task 22302976

Name oifs_43r3_ps_0252_2004050100_123_973_12189896_1
Workunit 12189896
Created 24 Jan 2023, 15:19:58 UTC
Sent 24 Jan 2023, 15:55:05 UTC
Report deadline 25 Mar 2023, 15:55:05 UTC
Received 1 Feb 2023, 15:28:00 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1536027
Run time 13 hours 59 min 52 sec
CPU time 13 hours 43 min 24 sec
Validate state Valid
Credit 14,873.04
Device peak FLOPS 7.20 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,798.48 MB
Peak swap size 4,978.07 MB
Peak disk usage 2,873.74 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<stderr_txt>
:18 STEP 2468 H=2468:00 +CPU= 15.295
  21:52:39 STEP 2469 H=2469:00 +CPU= 21.459
  21:52:54 STEP 2470 H=2470:00 +CPU= 14.807
  21:53:09 STEP 2471 H=2471:00 +CPU= 14.882
  21:53:35 STEP 2472 H=2472:00 +CPU= 25.540
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002472
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002472
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002460
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002460
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002460
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0252_2004050100_123_973_12189896_1_r329579438_102.zip
Uploading the intermediate file: upload_file_102.zip
  21:53:50 STEP 2473 H=2473:00 +CPU= 14.916
  21:54:05 STEP 2474 H=2474:00 +CPU= 14.444
  21:54:26 STEP 2475 H=2475:00 +CPU= 21.254
  21:54:43 STEP 2476 H=2476:00 +CPU= 16.207
  21:54:57 STEP 2477 H=2477:00 +CPU= 14.575
  21:55:20 STEP 2478 H=2478:00 +CPU= 22.416
  21:55:35 STEP 2479 H=2479:00 +CPU= 14.768
  21:55:52 STEP 2480 H=2480:00 +CPU= 16.554
  21:56:14 STEP 2481 H=2481:00 +CPU= 21.674
  21:56:30 STEP 2482 H=2482:00 +CPU= 15.176
  21:56:43 STEP 2483 H=2483:00 +CPU= 13.531
  21:57:07 STEP 2484 H=2484:00 +CPU= 23.119
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002484
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002484
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002484
  21:57:21 STEP 2485 H=2485:00 +CPU= 14.072
  21:57:35 STEP 2486 H=2486:00 +CPU= 14.130
  21:57:56 STEP 2487 H=2487:00 +CPU= 20.871
  21:58:11 STEP 2488 H=2488:00 +CPU= 14.283
  21:58:25 STEP 2489 H=2489:00 +CPU= 13.822
  21:58:46 STEP 2490 H=2490:00 +CPU= 21.208
  21:59:00 STEP 2491 H=2491:00 +CPU= 13.511
  21:59:13 STEP 2492 H=2492:00 +CPU= 13.468
  21:59:35 STEP 2493 H=2493:00 +CPU= 21.024
  21:59:49 STEP 2494 H=2494:00 +CPU= 14.284
  22:00:04 STEP 2495 H=2495:00 +CPU= 14.645
  22:00:29 STEP 2496 H=2496:00 +CPU= 24.564
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002496
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002496
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002484
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002484
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002484
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0252_2004050100_123_973_12189896_1_r329579438_103.zip
Uploading the intermediate file: upload_file_103.zip
  22:00:44 STEP 2497 H=2497:00 +CPU= 14.385
  22:00:57 STEP 2498 H=2498:00 +CPU= 13.280
  22:01:18 STEP 2499 H=2499:00 +CPU= 20.732
  22:01:33 STEP 2500 H=2500:00 +CPU= 14.216
  22:01:46 STEP 2501 H=2501:00 +CPU= 13.524
  22:02:08 STEP 2502 H=2502:00 +CPU= 21.768
  22:02:23 STEP 2503 H=2503:00 +CPU= 13.958
  22:02:36 STEP 2504 H=2504:00 +CPU= 13.716
  22:02:57 STEP 2505 H=2505:00 +CPU= 19.990
  22:03:11 STEP 2506 H=2506:00 +CPU= 14.021
  22:03:25 STEP 2507 H=2507:00 +CPU= 14.004
  22:03:48 STEP 2508 H=2508:00 +CPU= 23.153
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002508
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002508
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002508
  22:04:02 STEP 2509 H=2509:00 +CPU= 13.639
  22:04:16 STEP 2510 H=2510:00 +CPU= 14.160
  22:04:38 STEP 2511 H=2511:00 +CPU= 21.691
  22:04:53 STEP 2512 H=2512:00 +CPU= 14.548
  22:05:07 STEP 2513 H=2513:00 +CPU= 13.975
  22:05:30 STEP 2514 H=2514:00 +CPU= 22.413
  22:05:45 STEP 2515 H=2515:00 +CPU= 14.852
  22:06:00 STEP 2516 H=2516:00 +CPU= 15.481
  22:06:22 STEP 2517 H=2517:00 +CPU= 21.642
  22:06:38 STEP 2518 H=2518:00 +CPU= 15.383
  22:06:53 STEP 2519 H=2519:00 +CPU= 14.429
  22:07:18 STEP 2520 H=2520:00 +CPU= 24.983
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002520
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002520
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002508
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002508
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002508
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0252_2004050100_123_973_12189896_1_r329579438_104.zip
Uploading the intermediate file: upload_file_104.zip
  22:07:32 STEP 2521 H=2521:00 +CPU= 13.743
  22:07:46 STEP 2522 H=2522:00 +CPU= 13.943
  22:08:09 STEP 2523 H=2523:00 +CPU= 22.522
  22:08:23 STEP 2524 H=2524:00 +CPU= 14.049
  22:08:37 STEP 2525 H=2525:00 +CPU= 13.925
  22:09:00 STEP 2526 H=2526:00 +CPU= 22.794
  22:09:15 STEP 2527 H=2527:00 +CPU= 14.410
  22:09:31 STEP 2528 H=2528:00 +CPU= 16.049
  22:09:53 STEP 2529 H=2529:00 +CPU= 21.487
  22:10:07 STEP 2530 H=2530:00 +CPU= 14.146
  22:10:22 STEP 2531 H=2531:00 +CPU= 14.342
  22:10:45 STEP 2532 H=2532:00 +CPU= 23.258
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002532
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002532
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002532
  22:10:59 STEP 2533 H=2533:00 +CPU= 13.615
  22:11:13 STEP 2534 H=2534:00 +CPU= 13.981
  22:11:35 STEP 2535 H=2535:00 +CPU= 21.239
  22:11:49 STEP 2536 H=2536:00 +CPU= 14.602
  22:12:03 STEP 2537 H=2537:00 +CPU= 13.674
  22:12:25 STEP 2538 H=2538:00 +CPU= 21.985
  22:12:39 STEP 2539 H=2539:00 +CPU= 14.072
  22:12:53 STEP 2540 H=2540:00 +CPU= 13.173
  22:13:11 STEP 2541 H=2541:00 +CPU= 18.391
  22:13:24 STEP 2542 H=2542:00 +CPU= 12.317
  22:13:35 STEP 2543 H=2543:00 +CPU= 11.911
  22:13:55 STEP 2544 H=2544:00 +CPU= 19.950
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002544
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002544
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002532
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002532
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002532
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0252_2004050100_123_973_12189896_1_r329579438_105.zip
Uploading the intermediate file: upload_file_105.zip
  22:14:06 STEP 2545 H=2545:00 +CPU= 10.130
  22:14:17 STEP 2546 H=2546:00 +CPU= 11.347
  22:14:34 STEP 2547 H=2547:00 +CPU= 17.437
  22:14:45 STEP 2548 H=2548:00 +CPU= 10.856
  22:14:56 STEP 2549 H=2549:00 +CPU= 10.816
  22:15:13 STEP 2550 H=2550:00 +CPU= 17.355
  22:15:25 STEP 2551 H=2551:00 +CPU= 11.077
  22:15:36 STEP 2552 H=2552:00 +CPU= 11.046
  22:15:56 STEP 2553 H=2553:00 +CPU= 20.829
  22:16:09 STEP 2554 H=2554:00 +CPU= 12.324
  22:16:20 STEP 2555 H=2555:00 +CPU= 11.676
  22:16:41 STEP 2556 H=2556:00 +CPU= 20.852
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002556
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002556
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002556
  22:16:55 STEP 2557 H=2557:00 +CPU= 13.107
  22:17:09 STEP 2558 H=2558:00 +CPU= 14.452
  22:17:33 STEP 2559 H=2559:00 +CPU= 21.676
  22:17:48 STEP 2560 H=2560:00 +CPU= 15.272
  22:18:04 STEP 2561 H=2561:00 +CPU= 14.721
  22:18:27 STEP 2562 H=2562:00 +CPU= 22.588
  22:18:41 STEP 2563 H=2563:00 +CPU= 14.222
  22:18:56 STEP 2564 H=2564:00 +CPU= 14.723
  22:19:18 STEP 2565 H=2565:00 +CPU= 21.549
  22:19:33 STEP 2566 H=2566:00 +CPU= 14.579
  22:19:48 STEP 2567 H=2567:00 +CPU= 14.773
  22:20:14 STEP 2568 H=2568:00 +CPU= 25.671
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002568
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002568
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002556
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002556
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002556
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0252_2004050100_123_973_12189896_1_r329579438_106.zip
Uploading the intermediate file: upload_file_106.zip
  22:20:27 STEP 2569 H=2569:00 +CPU= 13.548
  22:20:44 STEP 2570 H=2570:00 +CPU= 16.502
  22:21:07 STEP 2571 H=2571:00 +CPU= 22.862
  22:21:23 STEP 2572 H=2572:00 +CPU= 15.418
  22:21:38 STEP 2573 H=2573:00 +CPU= 14.458
  22:22:01 STEP 2574 H=2574:00 +CPU= 22.967
  22:22:17 STEP 2575 H=2575:00 +CPU= 15.259
  22:22:33 STEP 2576 H=2576:00 +CPU= 15.140
  22:22:56 STEP 2577 H=2577:00 +CPU= 22.987
  22:23:11 STEP 2578 H=2578:00 +CPU= 15.217
  22:23:27 STEP 2579 H=2579:00 +CPU= 15.365
  22:23:52 STEP 2580 H=2580:00 +CPU= 24.717
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002580
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002580
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002580
  22:24:07 STEP 2581 H=2581:00 +CPU= 15.185
  22:24:22 STEP 2582 H=2582:00 +CPU= 14.992
  22:24:45 STEP 2583 H=2583:00 +CPU= 21.898
  22:25:00 STEP 2584 H=2584:00 +CPU= 14.755
  22:25:15 STEP 2585 H=2585:00 +CPU= 14.448
  22:25:38 STEP 2586 H=2586:00 +CPU= 22.977
  22:25:53 STEP 2587 H=2587:00 +CPU= 15.111
  22:26:08 STEP 2588 H=2588:00 +CPU= 14.724
  22:26:30 STEP 2589 H=2589:00 +CPU= 21.639
  22:26:46 STEP 2590 H=2590:00 +CPU= 15.807
  22:27:01 STEP 2591 H=2591:00 +CPU= 15.103
  22:27:28 STEP 2592 H=2592:00 +CPU= 25.733
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002592
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002592
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002580
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002580
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002580
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0252_2004050100_123_973_12189896_1_r329579438_107.zip
Uploading the intermediate file: upload_file_107.zip
  22:27:42 STEP 2593 H=2593:00 +CPU= 14.569
  22:27:57 STEP 2594 H=2594:00 +CPU= 14.451
  22:28:19 STEP 2595 H=2595:00 +CPU= 21.464
  22:28:33 STEP 2596 H=2596:00 +CPU= 14.340
  22:28:47 STEP 2597 H=2597:00 +CPU= 13.849
  22:29:10 STEP 2598 H=2598:00 +CPU= 22.008
  22:29:24 STEP 2599 H=2599:00 +CPU= 14.523
  22:29:39 STEP 2600 H=2600:00 +CPU= 14.781
  22:30:03 STEP 2601 H=2601:00 +CPU= 22.685
  22:30:17 STEP 2602 H=2602:00 +CPU= 14.611
  22:30:32 STEP 2603 H=2603:00 +CPU= 14.227
  22:30:56 STEP 2604 H=2604:00 +CPU= 23.660
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002604
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002604
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002604
  22:31:11 STEP 2605 H=2605:00 +CPU= 15.099
  22:31:26 STEP 2606 H=2606:00 +CPU= 14.558
  22:31:48 STEP 2607 H=2607:00 +CPU= 21.767
  22:32:03 STEP 2608 H=2608:00 +CPU= 14.422
  22:32:17 STEP 2609 H=2609:00 +CPU= 14.529
  22:32:39 STEP 2610 H=2610:00 +CPU= 21.778
  22:32:54 STEP 2611 H=2611:00 +CPU= 13.908
  22:33:08 STEP 2612 H=2612:00 +CPU= 14.717
  22:33:31 STEP 2613 H=2613:00 +CPU= 22.154
  22:33:45 STEP 2614 H=2614:00 +CPU= 14.015
  22:34:00 STEP 2615 H=2615:00 +CPU= 15.359
  22:34:25 STEP 2616 H=2616:00 +CPU= 24.612
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002616
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002616
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002604
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002604
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002604
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0252_2004050100_123_973_12189896_1_r329579438_108.zip
Uploading the intermediate file: upload_file_108.zip
  22:34:39 STEP 2617 H=2617:00 +CPU= 13.864
  22:34:54 STEP 2618 H=2618:00 +CPU= 14.866
  22:35:16 STEP 2619 H=2619:00 +CPU= 21.463
  22:35:31 STEP 2620 H=2620:00 +CPU= 14.609
  22:35:46 STEP 2621 H=2621:00 +CPU= 14.833
  22:36:08 STEP 2622 H=2622:00 +CPU= 21.916
  22:36:22 STEP 2623 H=2623:00 +CPU= 13.574
  22:36:37 STEP 2624 H=2624:00 +CPU= 14.709
  22:36:59 STEP 2625 H=2625:00 +CPU= 21.747
  22:37:14 STEP 2626 H=2626:00 +CPU= 14.469
  22:37:28 STEP 2627 H=2627:00 +CPU= 14.162
  22:37:52 STEP 2628 H=2628:00 +CPU= 23.361
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002628
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002628
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002628
  22:38:06 STEP 2629 H=2629:00 +CPU= 13.876
  22:38:21 STEP 2630 H=2630:00 +CPU= 14.672
  22:38:42 STEP 2631 H=2631:00 +CPU= 21.486
  22:38:57 STEP 2632 H=2632:00 +CPU= 14.293
  22:39:11 STEP 2633 H=2633:00 +CPU= 14.296
  22:39:34 STEP 2634 H=2634:00 +CPU= 22.088
  22:39:48 STEP 2635 H=2635:00 +CPU= 13.605
  22:40:03 STEP 2636 H=2636:00 +CPU= 14.996
  22:40:24 STEP 2637 H=2637:00 +CPU= 21.189
  22:40:38 STEP 2638 H=2638:00 +CPU= 13.940
  22:40:54 STEP 2639 H=2639:00 +CPU= 15.245
  22:41:19 STEP 2640 H=2640:00 +CPU= 24.766
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002640
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002640
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002628
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002628
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002628
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0252_2004050100_123_973_12189896_1_r329579438_109.zip
Uploading the intermediate file: upload_file_109.zip
  22:41:33 STEP 2641 H=2641:00 +CPU= 14.143
Uploading trickle at timestep: 9504000
  22:41:48 STEP 2642 H=2642:00 +CPU= 15.000
  22:42:09 STEP 2643 H=2643:00 +CPU= 21.032
  22:42:24 STEP 2644 H=2644:00 +CPU= 14.059
  22:42:39 STEP 2645 H=2645:00 +CPU= 15.119
  22:43:01 STEP 2646 H=2646:00 +CPU= 21.996
  22:43:15 STEP 2647 H=2647:00 +CPU= 14.007
  22:43:30 STEP 2648 H=2648:00 +CPU= 14.673
  22:43:52 STEP 2649 H=2649:00 +CPU= 21.096
  22:44:06 STEP 2650 H=2650:00 +CPU= 14.646
  22:44:21 STEP 2651 H=2651:00 +CPU= 14.442
  22:44:45 STEP 2652 H=2652:00 +CPU= 23.569
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002652
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002652
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002652
  22:44:59 STEP 2653 H=2653:00 +CPU= 13.961
  22:45:14 STEP 2654 H=2654:00 +CPU= 14.689
  22:45:35 STEP 2655 H=2655:00 +CPU= 21.266
  22:45:50 STEP 2656 H=2656:00 +CPU= 14.382
  22:46:04 STEP 2657 H=2657:00 +CPU= 14.084
  22:46:27 STEP 2658 H=2658:00 +CPU= 22.298
  22:46:41 STEP 2659 H=2659:00 +CPU= 13.745
  22:46:56 STEP 2660 H=2660:00 +CPU= 14.781
  22:47:17 STEP 2661 H=2661:00 +CPU= 20.867
  22:47:32 STEP 2662 H=2662:00 +CPU= 14.694
  22:47:46 STEP 2663 H=2663:00 +CPU= 14.231
  22:48:12 STEP 2664 H=2664:00 +CPU= 25.450
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002664
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002664
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002652
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002652
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002652
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0252_2004050100_123_973_12189896_1_r329579438_110.zip
Uploading the intermediate file: upload_file_110.zip
  22:48:26 STEP 2665 H=2665:00 +CPU= 14.175
  22:48:41 STEP 2666 H=2666:00 +CPU= 14.887
  22:49:02 STEP 2667 H=2667:00 +CPU= 21.108
  22:49:17 STEP 2668 H=2668:00 +CPU= 14.287
  22:49:32 STEP 2669 H=2669:00 +CPU= 14.707
  22:49:53 STEP 2670 H=2670:00 +CPU= 21.342
  22:50:08 STEP 2671 H=2671:00 +CPU= 14.441
  22:50:23 STEP 2672 H=2672:00 +CPU= 14.618
  22:50:45 STEP 2673 H=2673:00 +CPU= 21.636
  22:50:59 STEP 2674 H=2674:00 +CPU= 14.058
  22:51:13 STEP 2675 H=2675:00 +CPU= 14.201
  22:51:38 STEP 2676 H=2676:00 +CPU= 23.745
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002676
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002676
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002676
  22:51:52 STEP 2677 H=2677:00 +CPU= 14.049
  22:52:07 STEP 2678 H=2678:00 +CPU= 14.652
  22:52:28 STEP 2679 H=2679:00 +CPU= 20.974
  22:52:43 STEP 2680 H=2680:00 +CPU= 14.745
  22:52:57 STEP 2681 H=2681:00 +CPU= 14.270
  22:53:19 STEP 2682 H=2682:00 +CPU= 21.811
  22:53:33 STEP 2683 H=2683:00 +CPU= 13.641
  22:53:48 STEP 2684 H=2684:00 +CPU= 14.589
  22:54:09 STEP 2685 H=2685:00 +CPU= 21.075
  22:54:25 STEP 2686 H=2686:00 +CPU= 15.165
  22:54:40 STEP 2687 H=2687:00 +CPU= 14.950
  22:55:05 STEP 2688 H=2688:00 +CPU= 24.929
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002688
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002688
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002676
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002676
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002676
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0252_2004050100_123_973_12189896_1_r329579438_111.zip
Uploading the intermediate file: upload_file_111.zip
  22:55:19 STEP 2689 H=2689:00 +CPU= 14.063
  22:55:34 STEP 2690 H=2690:00 +CPU= 14.490
  22:55:56 STEP 2691 H=2691:00 +CPU= 21.422
  22:56:11 STEP 2692 H=2692:00 +CPU= 14.423
  22:56:25 STEP 2693 H=2693:00 +CPU= 14.439
  22:56:47 STEP 2694 H=2694:00 +CPU= 21.465
  22:57:01 STEP 2695 H=2695:00 +CPU= 14.409
  22:57:17 STEP 2696 H=2696:00 +CPU= 14.630
  22:57:38 STEP 2697 H=2697:00 +CPU= 21.403
  22:57:53 STEP 2698 H=2698:00 +CPU= 14.563
  22:58:08 STEP 2699 H=2699:00 +CPU= 14.360
  22:58:32 STEP 2700 H=2700:00 +CPU= 23.480
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002700
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002700
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002700
  22:58:46 STEP 2701 H=2701:00 +CPU= 14.130
  22:59:00 STEP 2702 H=2702:00 +CPU= 14.243
  22:59:22 STEP 2703 H=2703:00 +CPU= 21.359
  22:59:37 STEP 2704 H=2704:00 +CPU= 14.869
  22:59:52 STEP 2705 H=2705:00 +CPU= 14.441
  23:00:14 STEP 2706 H=2706:00 +CPU= 22.184
  23:00:29 STEP 2707 H=2707:00 +CPU= 14.043
  23:00:44 STEP 2708 H=2708:00 +CPU= 15.006
  23:01:05 STEP 2709 H=2709:00 +CPU= 21.363
  23:01:19 STEP 2710 H=2710:00 +CPU= 13.919
  23:01:35 STEP 2711 H=2711:00 +CPU= 15.327
  23:02:00 STEP 2712 H=2712:00 +CPU= 24.730
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002712
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002712
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002700
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002700
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002700
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0252_2004050100_123_973_12189896_1_r329579438_112.zip
Uploading the intermediate file: upload_file_112.zip
  23:02:14 STEP 2713 H=2713:00 +CPU= 14.065
  23:02:29 STEP 2714 H=2714:00 +CPU= 14.782
  23:02:51 STEP 2715 H=2715:00 +CPU= 21.529
  23:03:06 STEP 2716 H=2716:00 +CPU= 14.677
  23:03:20 STEP 2717 H=2717:00 +CPU= 14.010
  23:03:42 STEP 2718 H=2718:00 +CPU= 21.962
  23:03:56 STEP 2719 H=2719:00 +CPU= 13.940
  23:04:11 STEP 2720 H=2720:00 +CPU= 14.517
  23:04:33 STEP 2721 H=2721:00 +CPU= 21.883
  23:04:48 STEP 2722 H=2722:00 +CPU= 14.501
  23:05:02 STEP 2723 H=2723:00 +CPU= 14.030
  23:05:26 STEP 2724 H=2724:00 +CPU= 23.483
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002724
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002724
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002724
  23:05:39 STEP 2725 H=2725:00 +CPU= 13.494
  23:05:54 STEP 2726 H=2726:00 +CPU= 14.735
  23:06:16 STEP 2727 H=2727:00 +CPU= 21.361
  23:06:30 STEP 2728 H=2728:00 +CPU= 14.465
  23:06:45 STEP 2729 H=2729:00 +CPU= 14.359
  23:07:07 STEP 2730 H=2730:00 +CPU= 21.427
  23:07:21 STEP 2731 H=2731:00 +CPU= 13.856
  23:07:36 STEP 2732 H=2732:00 +CPU= 14.746
  23:07:57 STEP 2733 H=2733:00 +CPU= 20.737
  23:08:11 STEP 2734 H=2734:00 +CPU= 14.162
  23:08:26 STEP 2735 H=2735:00 +CPU= 14.883
  23:08:52 STEP 2736 H=2736:00 +CPU= 25.279
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002736
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002736
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002724
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002724
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002724
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0252_2004050100_123_973_12189896_1_r329579438_113.zip
Uploading the intermediate file: upload_file_113.zip
  23:09:06 STEP 2737 H=2737:00 +CPU= 13.714
  23:09:20 STEP 2738 H=2738:00 +CPU= 14.537
  23:09:42 STEP 2739 H=2739:00 +CPU= 20.967
  23:09:57 STEP 2740 H=2740:00 +CPU= 14.473
  23:10:11 STEP 2741 H=2741:00 +CPU= 13.636
  23:10:33 STEP 2742 H=2742:00 +CPU= 22.729
  23:10:48 STEP 2743 H=2743:00 +CPU= 14.155
  23:11:03 STEP 2744 H=2744:00 +CPU= 14.643
  23:11:24 STEP 2745 H=2745:00 +CPU= 21.566
  23:11:39 STEP 2746 H=2746:00 +CPU= 14.242
  23:11:54 STEP 2747 H=2747:00 +CPU= 14.682
  23:12:17 STEP 2748 H=2748:00 +CPU= 23.341
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002748
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002748
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002748
  23:12:32 STEP 2749 H=2749:00 +CPU= 14.209
  23:12:47 STEP 2750 H=2750:00 +CPU= 14.624
  23:13:08 STEP 2751 H=2751:00 +CPU= 20.991
  23:13:22 STEP 2752 H=2752:00 +CPU= 14.192
  23:13:37 STEP 2753 H=2753:00 +CPU= 14.421
  23:13:58 STEP 2754 H=2754:00 +CPU= 21.486
  23:14:13 STEP 2755 H=2755:00 +CPU= 13.943
  23:14:26 STEP 2756 H=2756:00 +CPU= 13.770
  23:14:49 STEP 2757 H=2757:00 +CPU= 21.963
  23:15:03 STEP 2758 H=2758:00 +CPU= 14.675
  23:15:18 STEP 2759 H=2759:00 +CPU= 14.255
  23:15:42 STEP 2760 H=2760:00 +CPU= 24.435
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002760
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002760
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002748
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002748
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002748
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0252_2004050100_123_973_12189896_1_r329579438_114.zip
Uploading the intermediate file: upload_file_114.zip
  23:15:57 STEP 2761 H=2761:00 +CPU= 14.463
  23:16:12 STEP 2762 H=2762:00 +CPU= 14.596
  23:16:33 STEP 2763 H=2763:00 +CPU= 20.956
  23:16:48 STEP 2764 H=2764:00 +CPU= 14.583
  23:17:02 STEP 2765 H=2765:00 +CPU= 13.711
  23:17:24 STEP 2766 H=2766:00 +CPU= 21.838
  23:17:38 STEP 2767 H=2767:00 +CPU= 14.129
  23:17:53 STEP 2768 H=2768:00 +CPU= 14.481
  23:18:15 STEP 2769 H=2769:00 +CPU= 21.634
  23:18:30 STEP 2770 H=2770:00 +CPU= 14.810
  23:18:44 STEP 2771 H=2771:00 +CPU= 13.980
  23:19:08 STEP 2772 H=2772:00 +CPU= 23.593
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002772
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002772
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002772
  23:19:22 STEP 2773 H=2773:00 +CPU= 14.514
  23:19:37 STEP 2774 H=2774:00 +CPU= 14.444
  23:19:59 STEP 2775 H=2775:00 +CPU= 21.239
  23:20:13 STEP 2776 H=2776:00 +CPU= 14.555
  23:20:27 STEP 2777 H=2777:00 +CPU= 13.909
  23:20:50 STEP 2778 H=2778:00 +CPU= 21.598
  23:21:05 STEP 2779 H=2779:00 +CPU= 14.911
  23:21:19 STEP 2780 H=2780:00 +CPU= 14.519
  23:21:41 STEP 2781 H=2781:00 +CPU= 21.663
  23:21:56 STEP 2782 H=2782:00 +CPU= 14.854
  23:22:11 STEP 2783 H=2783:00 +CPU= 14.568
  23:22:37 STEP 2784 H=2784:00 +CPU= 25.602
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002784
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002784
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002772
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002772
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002772
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0252_2004050100_123_973_12189896_1_r329579438_115.zip
Uploading the intermediate file: upload_file_115.zip
  23:22:51 STEP 2785 H=2785:00 +CPU= 14.154
  23:23:06 STEP 2786 H=2786:00 +CPU= 14.530
  23:23:28 STEP 2787 H=2787:00 +CPU= 21.504
  23:23:43 STEP 2788 H=2788:00 +CPU= 14.899
  23:23:57 STEP 2789 H=2789:00 +CPU= 14.043
  23:24:19 STEP 2790 H=2790:00 +CPU= 21.721
  23:24:34 STEP 2791 H=2791:00 +CPU= 14.507
  23:24:49 STEP 2792 H=2792:00 +CPU= 14.617
  23:25:10 STEP 2793 H=2793:00 +CPU= 21.485
  23:25:26 STEP 2794 H=2794:00 +CPU= 15.143
  23:25:40 STEP 2795 H=2795:00 +CPU= 13.794
  23:26:04 STEP 2796 H=2796:00 +CPU= 23.663
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002796
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002796
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002796
  23:26:19 STEP 2797 H=2797:00 +CPU= 14.513
  23:26:34 STEP 2798 H=2798:00 +CPU= 14.656
  23:26:56 STEP 2799 H=2799:00 +CPU= 21.497
  23:27:11 STEP 2800 H=2800:00 +CPU= 14.861
  23:27:25 STEP 2801 H=2801:00 +CPU= 14.486
  23:27:47 STEP 2802 H=2802:00 +CPU= 21.640
  23:28:02 STEP 2803 H=2803:00 +CPU= 14.428
  23:28:17 STEP 2804 H=2804:00 +CPU= 14.913
  23:28:39 STEP 2805 H=2805:00 +CPU= 21.274
  23:28:53 STEP 2806 H=2806:00 +CPU= 14.462
  23:29:08 STEP 2807 H=2807:00 +CPU= 14.912
  23:29:34 STEP 2808 H=2808:00 +CPU= 24.861
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002808
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002808
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002796
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002796
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002796
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0252_2004050100_123_973_12189896_1_r329579438_116.zip
Uploading the intermediate file: upload_file_116.zip
  23:29:48 STEP 2809 H=2809:00 +CPU= 14.587
  23:30:03 STEP 2810 H=2810:00 +CPU= 14.524
  23:30:25 STEP 2811 H=2811:00 +CPU= 21.367
  23:30:40 STEP 2812 H=2812:00 +CPU= 14.775
  23:30:55 STEP 2813 H=2813:00 +CPU= 14.413
  23:31:17 STEP 2814 H=2814:00 +CPU= 22.169
  23:31:32 STEP 2815 H=2815:00 +CPU= 14.515
  23:31:47 STEP 2816 H=2816:00 +CPU= 15.169
  23:32:09 STEP 2817 H=2817:00 +CPU= 21.680
  23:32:24 STEP 2818 H=2818:00 +CPU= 14.794
  23:32:38 STEP 2819 H=2819:00 +CPU= 14.158
  23:33:03 STEP 2820 H=2820:00 +CPU= 23.940
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002820
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002820
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002820
  23:33:17 STEP 2821 H=2821:00 +CPU= 14.106
  23:33:31 STEP 2822 H=2822:00 +CPU= 14.223
  23:33:53 STEP 2823 H=2823:00 +CPU= 21.645
  23:34:08 STEP 2824 H=2824:00 +CPU= 14.606
  23:34:23 STEP 2825 H=2825:00 +CPU= 14.549
  23:34:44 STEP 2826 H=2826:00 +CPU= 21.188
  23:34:59 STEP 2827 H=2827:00 +CPU= 14.861
  23:35:14 STEP 2828 H=2828:00 +CPU= 14.665
  23:35:35 STEP 2829 H=2829:00 +CPU= 20.627
  23:35:50 STEP 2830 H=2830:00 +CPU= 14.609
  23:36:05 STEP 2831 H=2831:00 +CPU= 15.018
  23:36:30 STEP 2832 H=2832:00 +CPU= 25.226
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002832
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002832
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002820
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002820
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002820
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0252_2004050100_123_973_12189896_1_r329579438_117.zip
Uploading the intermediate file: upload_file_117.zip
  23:36:45 STEP 2833 H=2833:00 +CPU= 14.757
  23:37:00 STEP 2834 H=2834:00 +CPU= 14.623
  23:37:21 STEP 2835 H=2835:00 +CPU= 20.911
  23:37:37 STEP 2836 H=2836:00 +CPU= 15.245
  23:37:52 STEP 2837 H=2837:00 +CPU= 15.055
  23:38:14 STEP 2838 H=2838:00 +CPU= 21.992
  23:38:28 STEP 2839 H=2839:00 +CPU= 14.271
  23:38:43 STEP 2840 H=2840:00 +CPU= 14.933
  23:39:05 STEP 2841 H=2841:00 +CPU= 21.012
  23:39:19 STEP 2842 H=2842:00 +CPU= 14.622
  23:39:34 STEP 2843 H=2843:00 +CPU= 14.292
  23:39:57 STEP 2844 H=2844:00 +CPU= 23.192
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002844
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002844
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002844
  23:40:12 STEP 2845 H=2845:00 +CPU= 14.697
  23:40:27 STEP 2846 H=2846:00 +CPU= 14.670
  23:40:48 STEP 2847 H=2847:00 +CPU= 21.190
  23:41:03 STEP 2848 H=2848:00 +CPU= 14.828
  23:41:18 STEP 2849 H=2849:00 +CPU= 14.761
  23:41:40 STEP 2850 H=2850:00 +CPU= 21.388
  23:41:54 STEP 2851 H=2851:00 +CPU= 14.454
  23:42:10 STEP 2852 H=2852:00 +CPU= 15.091
  23:42:32 STEP 2853 H=2853:00 +CPU= 22.299
  23:42:47 STEP 2854 H=2854:00 +CPU= 14.235
  23:43:02 STEP 2855 H=2855:00 +CPU= 15.411
  23:43:27 STEP 2856 H=2856:00 +CPU= 25.033
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002856
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002856
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002844
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002844
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002844
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0252_2004050100_123_973_12189896_1_r329579438_118.zip
Uploading the intermediate file: upload_file_118.zip
  23:43:42 STEP 2857 H=2857:00 +CPU= 14.685
  23:43:57 STEP 2858 H=2858:00 +CPU= 14.812
  23:44:18 STEP 2859 H=2859:00 +CPU= 21.189
  23:44:34 STEP 2860 H=2860:00 +CPU= 14.883
  23:44:48 STEP 2861 H=2861:00 +CPU= 14.068
  23:45:11 STEP 2862 H=2862:00 +CPU= 22.336
  23:45:26 STEP 2863 H=2863:00 +CPU= 14.659
  23:45:41 STEP 2864 H=2864:00 +CPU= 14.655
  23:46:02 STEP 2865 H=2865:00 +CPU= 20.754
  23:46:17 STEP 2866 H=2866:00 +CPU= 14.674
  23:46:31 STEP 2867 H=2867:00 +CPU= 14.195
  23:46:56 STEP 2868 H=2868:00 +CPU= 24.936
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002868
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002868
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002868
  23:47:11 STEP 2869 H=2869:00 +CPU= 14.685
  23:47:27 STEP 2870 H=2870:00 +CPU= 15.411
  23:47:48 STEP 2871 H=2871:00 +CPU= 21.349
  23:48:04 STEP 2872 H=2872:00 +CPU= 15.371
  23:48:18 STEP 2873 H=2873:00 +CPU= 14.181
  23:48:41 STEP 2874 H=2874:00 +CPU= 22.124
  23:48:55 STEP 2875 H=2875:00 +CPU= 14.220
  23:49:09 STEP 2876 H=2876:00 +CPU= 14.204
  23:49:31 STEP 2877 H=2877:00 +CPU= 21.201
  23:49:45 STEP 2878 H=2878:00 +CPU= 14.457
  23:50:00 STEP 2879 H=2879:00 +CPU= 14.398
  23:50:25 STEP 2880 H=2880:00 +CPU= 24.326
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002880
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002880
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002868
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002868
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002868
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0252_2004050100_123_973_12189896_1_r329579438_119.zip
Uploading the intermediate file: upload_file_119.zip
  23:50:39 STEP 2881 H=2881:00 +CPU= 14.284
Uploading trickle at timestep: 10368000
  23:50:55 STEP 2882 H=2882:00 +CPU= 15.245
  23:51:16 STEP 2883 H=2883:00 +CPU= 20.787
  23:51:30 STEP 2884 H=2884:00 +CPU= 14.009
  23:51:43 STEP 2885 H=2885:00 +CPU= 13.446
  23:52:07 STEP 2886 H=2886:00 +CPU= 23.390
  23:52:23 STEP 2887 H=2887:00 +CPU= 15.841
  23:52:39 STEP 2888 H=2888:00 +CPU= 16.181
  23:53:02 STEP 2889 H=2889:00 +CPU= 22.999
  23:53:18 STEP 2890 H=2890:00 +CPU= 15.475
  23:53:33 STEP 2891 H=2891:00 +CPU= 14.899
  23:53:58 STEP 2892 H=2892:00 +CPU= 24.277
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002892
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002892
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002892
  23:54:12 STEP 2893 H=2893:00 +CPU= 14.015
  23:54:27 STEP 2894 H=2894:00 +CPU= 15.147
  23:54:49 STEP 2895 H=2895:00 +CPU= 21.389
  23:55:04 STEP 2896 H=2896:00 +CPU= 15.098
  23:55:19 STEP 2897 H=2897:00 +CPU= 14.952
  23:55:43 STEP 2898 H=2898:00 +CPU= 23.364
  23:55:58 STEP 2899 H=2899:00 +CPU= 14.547
  23:56:13 STEP 2900 H=2900:00 +CPU= 15.071
  23:56:37 STEP 2901 H=2901:00 +CPU= 22.851
  23:56:52 STEP 2902 H=2902:00 +CPU= 15.800
  23:57:09 STEP 2903 H=2903:00 +CPU= 15.898
  23:57:34 STEP 2904 H=2904:00 +CPU= 25.548
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002904
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002904
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002880
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002892
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002892
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002892
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0252_2004050100_123_973_12189896_1_r329579438_120.zip
Uploading the intermediate file: upload_file_120.zip
  23:57:49 STEP 2905 H=2905:00 +CPU= 13.962
  23:58:03 STEP 2906 H=2906:00 +CPU= 14.633
  23:58:25 STEP 2907 H=2907:00 +CPU= 21.114
  23:58:41 STEP 2908 H=2908:00 +CPU= 16.064
  23:58:56 STEP 2909 H=2909:00 +CPU= 14.125
Suspend request received from the BOINC client, suspending the child process
Quit request received from the BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2004050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0252
(argv4) batchid: 973
(argv5) wuid: 12189896
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/5
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/5/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/5/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_1c2573cd94f0de954b97d51ab76d06f1 to: /var/lib/boinc-client/slots/5/oifs_43r3_ps_0252_2004050100_123_973_12189896.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/5/oifs_43r3_ps_0252_2004050100_123_973_12189896.zip
ic_ancil_file: ic_ancil_12189896
ifsdata_file: ifsdata_12189896
climate_data_file: clim_data_12189896
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_4129db8f98966c3b241b9cb48d581251 to: /var/lib/boinc-client/slots/5/ic_ancil_12189896.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/5/ic_ancil_12189896.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_4869f6ee36e3907670b9c27b5a23a9f5 to: /var/lib/boinc-client/slots/5/ifsdata/ifsdata_12189896.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/5/ifsdata/ifsdata_12189896.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/5/159l_2/clim_data_12189896.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/5/159l_2/clim_data_12189896.zip
Checking for progress XML file: /var/lib/boinc-client/slots/5/progress_file_12189896.xml
Opened progress file ok : /var/lib/boinc-client/slots/5/progress_file_12189896.xml
-- Model is restarting --
Adjusting last_iter, 2910, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/5/progress_file_12189896.xml
last_cpu_time: 48554
upload_file_number: 121
last_iter: 2905
last_upload: 10454400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0252_2004050100_123_973_12189896_1_r329579438
The child process has been launched with process id: 527
Executing the command: /var/lib/boinc-client/slots/5/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:Biggles:1:1:527:527] [20230201:141241:1675260761:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:Biggles:1:1:527:527] [20230201:141241:1675260761:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:Biggles:1:1:527:527] [20230201:141241:1675260761:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:Biggles:1:1:527:527] [20230201:141241:1675260761:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:Biggles:1:1:527:527] [20230201:141241:1675260761:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:Biggles:1:1:527:527] [20230201:141241:1675260761:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:Biggles:1:1:527:527] [20230201:141241:1675260761:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:Biggles:1:1:527:527] [20230201:141241:1675260761:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:Biggles:1:1:527:527] [20230201:141241:1675260761:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:Biggles:1:1:527:527] [20230201:141241:1675260761:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:Biggles:1:1:527:527] [20230201:141241:1675260761:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:Biggles:1:1:527:527] [20230201:141241:1675260761:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:Biggles:1:1:527:527] [20230201:141241:1675260761:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:Biggles:1:1:527:527] [20230201:141241:1675260761:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:Biggles:1:1:527:527] [20230201:141241:1675260761:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:Biggles:1:1:527:527] [20230201:141241:1675260761:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Biggles:1:1:527:527] [20230201:141241:1675260761:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Biggles:1:1:527:527] [20230201:141241:1675260761:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Biggles:1:1:527:527] [20230201:141241:1675260761:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:Biggles:1:1:527:527] [20230201:141241:1675260761:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:Biggles:1:1:527:527] [20230201:141241:1675260761:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Biggles:1:1:527:527] [20230201:141241:1675260761:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Biggles:1:1:527:527] [20230201:141241:1675260761:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:Biggles:1:1:527:527] [20230201:141241:1675260761:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Biggles:1:1:527:527] [20230201:141241:1675260761:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Biggles:1:1:527:527] [20230201:141241:1675260761:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:Biggles:1:1:527:527] [20230201:141241:1675260761:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:Biggles:1:1:527:527] [20230201:141241:1675260761:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:Biggles:1:1:527:527] [20230201:141241:1675260761:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:Biggles:1:1:527:527] [20230201:141241:1675260761:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  14:13:10 STEP 2904 H=2904:00 +CPU= 26.496
  14:13:26 STEP 2905 H=2905:00 +CPU= 15.062
  14:13:41 STEP 2906 H=2906:00 +CPU= 15.109
  14:14:03 STEP 2907 H=2907:00 +CPU= 21.508
  14:14:18 STEP 2908 H=2908:00 +CPU= 15.084
  14:14:33 STEP 2909 H=2909:00 +CPU= 14.783
  14:14:55 STEP 2910 H=2910:00 +CPU= 21.468
  14:15:09 STEP 2911 H=2911:00 +CPU= 13.869
  14:15:25 STEP 2912 H=2912:00 +CPU= 14.643
  14:15:45 STEP 2913 H=2913:00 +CPU= 20.414
  14:16:00 STEP 2914 H=2914:00 +CPU= 14.250
  14:16:15 STEP 2915 H=2915:00 +CPU= 14.826
  14:16:39 STEP 2916 H=2916:00 +CPU= 23.510
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002916
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002916
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002916
  14:16:54 STEP 2917 H=2917:00 +CPU= 14.593
  14:17:08 STEP 2918 H=2918:00 +CPU= 13.855
  14:17:29 STEP 2919 H=2919:00 +CPU= 20.977
  14:17:44 STEP 2920 H=2920:00 +CPU= 14.613
  14:17:59 STEP 2921 H=2921:00 +CPU= 14.915
  14:18:21 STEP 2922 H=2922:00 +CPU= 20.933
  14:18:34 STEP 2923 H=2923:00 +CPU= 13.729
  14:18:49 STEP 2924 H=2924:00 +CPU= 14.497
  14:19:09 STEP 2925 H=2925:00 +CPU= 19.904
  14:19:24 STEP 2926 H=2926:00 +CPU= 14.297
  14:19:39 STEP 2927 H=2927:00 +CPU= 14.590
  14:20:04 STEP 2928 H=2928:00 +CPU= 24.949
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002928
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002928
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002904
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002916
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002916
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002916
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0252_2004050100_123_973_12189896_1_r329579438_121.zip
Uploading the intermediate file: upload_file_121.zip
  14:20:19 STEP 2929 H=2929:00 +CPU= 14.000
  14:20:33 STEP 2930 H=2930:00 +CPU= 14.130
  14:20:53 STEP 2931 H=2931:00 +CPU= 20.355
  14:21:08 STEP 2932 H=2932:00 +CPU= 14.323
  14:21:23 STEP 2933 H=2933:00 +CPU= 14.640
  14:21:44 STEP 2934 H=2934:00 +CPU= 20.658
  14:21:58 STEP 2935 H=2935:00 +CPU= 14.345
  14:22:12 STEP 2936 H=2936:00 +CPU= 14.103
  14:22:34 STEP 2937 H=2937:00 +CPU= 21.467
  14:22:49 STEP 2938 H=2938:00 +CPU= 14.466
  14:23:07 STEP 2939 H=2939:00 +CPU= 18.096
  14:23:37 STEP 2940 H=2940:00 +CPU= 29.495
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002940
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002940
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002940
  14:23:54 STEP 2941 H=2941:00 +CPU= 17.352
  14:24:09 STEP 2942 H=2942:00 +CPU= 14.490
  14:24:31 STEP 2943 H=2943:00 +CPU= 21.185
  14:24:45 STEP 2944 H=2944:00 +CPU= 14.436
  14:25:00 STEP 2945 H=2945:00 +CPU= 14.636
  14:25:21 STEP 2946 H=2946:00 +CPU= 21.040
  14:25:35 STEP 2947 H=2947:00 +CPU= 14.106
  14:25:50 STEP 2948 H=2948:00 +CPU= 14.367
  14:26:11 STEP 2949 H=2949:00 +CPU= 21.220
  14:26:26 STEP 2950 H=2950:00 +CPU= 14.505
  14:26:40 STEP 2951 H=2951:00 +CPU= 14.284
  14:27:04 STEP 2952 H=2952:00 +CPU= 23.366
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20040831233000  !!!!!!!!!!!! 

   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20040831234500  !!!!!!!!!!!! 

   !!!!!!!!!!!!!! WAVE FIELDS INTEGRATED.  DATE IS: 20040901000000  !!!!!!!!!!!! 
  
  GLOBAL NORM OF FIELDS RETURNED TO IFS :
  Charnock     1.331793565382952E-002  7.500100767539640E-003
  7.059929248038460E-002       26160
              HEX: 3F8B466F10AADD6E  3F7EB86CF8366CB2  3FB212CB9452F10C
  U-Stokes     5.740637227643339E-003 -0.269112499622984     
  0.438652071290645            26160
              HEX: 3F77837E926A33BD  BFD13923A234D572  3FDC12E0232105CC
  V-Stokes     3.479455842680009E-003 -0.275420664336830     
  0.232923178271688            26160
              HEX: 3F6C80F2A1AB8391  BFD1A07DFE7E08A6  3FCDD06D3C9421AC
  
  
 END OF WAVE MODEL CALL 
 ----------------------------------------------------------------------- 
 14:27:04 STEP 2952 H=2952:00 +CPU= 23.366
 END CNT3
 NSTEP =  2952 CNT0   000000000  20.241  20.241  20.553   0.985               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           35/   804992476 WRITTEN          662/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1     29    0       35       786125      662 ************     2.8     1.2     1.6     1.7     2.9
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1  863347.9  863347.9    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1  860789.5  860789.5     99.70      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                   53    1576.8    1576.8      9.68      0.00
   9 SPCM     - SPECTRAL COMP.                      48     263.7     263.7      1.47      0.00
  10 SCAN2M - PHYSICS                               49    7348.0    7348.0     41.70      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                  4     638.4     638.4      0.30      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 49      12.9      12.9      0.07      0.00
  13 SCAN2M - RADIATION CALC.                       17    7394.6    7394.6     14.56      0.00
  17 GRIDFPOS IN CNT4                                4      11.1      11.1      0.01      0.00
  27 RESTART FILES - WRITING                         2    1276.7    1276.7      0.30      0.00
  28 RESTART FILES - READING                         1    2321.3    2321.3      0.27      0.00
  29 SU4FPOS IN CNT4                                 4       0.1       0.1      0.00      0.00
  30 DYNFPOS IN CNT4                                 4    1850.8    1850.8      0.86      0.00
  37 CPGLAG   - SL COMPUTATIONS                     49    2462.1    2462.1     13.97      0.00
  38 WAM      - TOTAL COST OF WAVE MODEL            49    1830.7    1830.7     10.39      0.00
  39 SU0YOMB                                         1     158.2     158.2      0.02      0.00
  51 SCAN2M   - SL COMM. PART 1                     49      18.0      18.0      0.10      0.00
  54 SPCM     - M TO S/S TO M TRANSP.               48     222.4     222.4      1.24      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.               48      36.9      36.9      0.21      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 49       0.1       0.1      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM      170      89.9      89.9      1.77      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM       104      83.3      83.3      1.00      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM        104      31.4      31.4      0.38      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM       170      34.9      34.9      0.69      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            2      10.0      10.0      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION            170      11.4      11.4      0.22      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION            104      10.6      10.6      0.13      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION            170      16.7      16.7      0.33      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION            104      10.8      10.8      0.13      0.00
 400 GSTATS                                      12634       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 12063       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME      863.3 CPU TIME     850.2 VECTOR TIME      850.2


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY    12345.9
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+002952
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+002952
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+002952
Adding to the zip: /var/lib/boinc-client/slots/5/NODE.001_01
Adding to the zip: /var/lib/boinc-client/slots/5/ifs.stat
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002952
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002940
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002952
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002940
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMUAhq0f+002928
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMGGhq0f+002952
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002940
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189896/ICMSHhq0f+002928
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0252_2004050100_123_973_12189896_1_r329579438_122.zip
Uploading the final file: upload_file_122.zip
Uploading trickle at timestep: 10623600
14:30:26 (507): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:46:31 1536027 22302976 oifs_43r3_ps_0252_2004050100_123_973_12189896_1 10,623,600 49,404 0.0047
09 May 2023 16:46:30 1536027 22302976 oifs_43r3_ps_0252_2004050100_123_973_12189896_1 10,368,000 48,038 0.0046
09 May 2023 16:46:12 1536027 22302976 oifs_43r3_ps_0252_2004050100_123_973_12189896_1 9,504,000 43,943 0.0046
09 May 2023 16:46:11 1536027 22302976 oifs_43r3_ps_0252_2004050100_123_973_12189896_1 8,640,000 39,899 0.0046
09 May 2023 16:46:11 1536027 22302976 oifs_43r3_ps_0252_2004050100_123_973_12189896_1 8,640,000 39,899 0.0046
09 May 2023 16:46:10 1536027 22302976 oifs_43r3_ps_0252_2004050100_123_973_12189896_1 7,776,000 35,928 0.0046
09 May 2023 16:46:08 1536027 22302976 oifs_43r3_ps_0252_2004050100_123_973_12189896_1 6,912,000 31,928 0.0046
09 May 2023 16:46:07 1536027 22302976 oifs_43r3_ps_0252_2004050100_123_973_12189896_1 6,048,000 27,940 0.0046
09 May 2023 16:46:07 1536027 22302976 oifs_43r3_ps_0252_2004050100_123_973_12189896_1 6,048,000 27,940 0.0046
09 May 2023 16:46:05 1536027 22302976 oifs_43r3_ps_0252_2004050100_123_973_12189896_1 5,184,000 24,038 0.0046
09 May 2023 16:46:04 1536027 22302976 oifs_43r3_ps_0252_2004050100_123_973_12189896_1 4,320,000 20,152 0.0047
09 May 2023 16:46:03 1536027 22302976 oifs_43r3_ps_0252_2004050100_123_973_12189896_1 3,456,000 16,193 0.0047
09 May 2023 16:46:02 1536027 22302976 oifs_43r3_ps_0252_2004050100_123_973_12189896_1 2,592,000 12,089 0.0047
09 May 2023 16:46:00 1536027 22302976 oifs_43r3_ps_0252_2004050100_123_973_12189896_1 1,728,000 8,028 0.0046
09 May 2023 16:46:00 1536027 22302976 oifs_43r3_ps_0252_2004050100_123_973_12189896_1 1,728,000 8,028 0.0046
09 May 2023 16:45:59 1536027 22302976 oifs_43r3_ps_0252_2004050100_123_973_12189896_1 864,000 4,001 0.0046
09 May 2023 16:45:59 1536027 22302976 oifs_43r3_ps_0252_2004050100_123_973_12189896_1 864,000 4,001 0.0046


©2024 climateprediction.net