<core_client_version>7.18.1</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
s_43r3_ps_12189883/ICMUAhq0f+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001572
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0239_2004050100_123_973_12189883_0_r2084701587_65.zip
Uploading the intermediate file: upload_file_65.zip
09:21:55 STEP 1585 H=1585:00 +CPU= 15.028
09:22:16 STEP 1586 H=1586:00 +CPU= 19.519
09:22:47 STEP 1587 H=1587:00 +CPU= 29.807
09:23:08 STEP 1588 H=1588:00 +CPU= 19.952
09:23:29 STEP 1589 H=1589:00 +CPU= 20.129
09:24:02 STEP 1590 H=1590:00 +CPU= 31.360
09:24:24 STEP 1591 H=1591:00 +CPU= 19.893
09:24:44 STEP 1592 H=1592:00 +CPU= 20.048
09:25:18 STEP 1593 H=1593:00 +CPU= 30.927
09:25:35 STEP 1594 H=1594:00 +CPU= 17.353
09:25:56 STEP 1595 H=1595:00 +CPU= 19.402
09:26:30 STEP 1596 H=1596:00 +CPU= 32.005
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001596
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001596
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001596
09:26:50 STEP 1597 H=1597:00 +CPU= 18.907
09:27:11 STEP 1598 H=1598:00 +CPU= 20.400
09:27:43 STEP 1599 H=1599:00 +CPU= 30.377
09:28:04 STEP 1600 H=1600:00 +CPU= 20.022
09:28:24 STEP 1601 H=1601:00 +CPU= 18.946
09:28:56 STEP 1602 H=1602:00 +CPU= 30.224
09:29:17 STEP 1603 H=1603:00 +CPU= 19.287
09:29:39 STEP 1604 H=1604:00 +CPU= 20.032
09:30:11 STEP 1605 H=1605:00 +CPU= 29.679
09:30:32 STEP 1606 H=1606:00 +CPU= 20.315
09:30:54 STEP 1607 H=1607:00 +CPU= 20.360
09:31:31 STEP 1608 H=1608:00 +CPU= 35.113
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001608
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001608
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001596
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0239_2004050100_123_973_12189883_0_r2084701587_66.zip
Uploading the intermediate file: upload_file_66.zip
09:31:49 STEP 1609 H=1609:00 +CPU= 17.628
09:32:08 STEP 1610 H=1610:00 +CPU= 18.363
09:32:35 STEP 1611 H=1611:00 +CPU= 26.869
09:32:50 STEP 1612 H=1612:00 +CPU= 14.275
09:33:06 STEP 1613 H=1613:00 +CPU= 15.747
09:33:38 STEP 1614 H=1614:00 +CPU= 30.080
09:33:58 STEP 1615 H=1615:00 +CPU= 19.117
09:34:19 STEP 1616 H=1616:00 +CPU= 20.042
09:34:51 STEP 1617 H=1617:00 +CPU= 29.121
09:35:12 STEP 1618 H=1618:00 +CPU= 19.551
09:35:32 STEP 1619 H=1619:00 +CPU= 18.737
09:36:07 STEP 1620 H=1620:00 +CPU= 32.848
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001620
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001620
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001620
09:36:28 STEP 1621 H=1621:00 +CPU= 19.300
09:36:49 STEP 1622 H=1622:00 +CPU= 21.233
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
09:39:52 STEP 1623 H=1623:00 +CPU= 30.544
09:40:13 STEP 1624 H=1624:00 +CPU= 19.857
09:40:32 STEP 1625 H=1625:00 +CPU= 18.716
09:41:04 STEP 1626 H=1626:00 +CPU= 29.975
09:41:25 STEP 1627 H=1627:00 +CPU= 19.134
09:41:47 STEP 1628 H=1628:00 +CPU= 19.674
09:42:17 STEP 1629 H=1629:00 +CPU= 27.480
09:42:38 STEP 1630 H=1630:00 +CPU= 20.194
09:43:00 STEP 1631 H=1631:00 +CPU= 19.965
09:43:39 STEP 1632 H=1632:00 +CPU= 36.041
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001632
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001632
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001620
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0239_2004050100_123_973_12189883_0_r2084701587_67.zip
Uploading the intermediate file: upload_file_67.zip
09:44:01 STEP 1633 H=1633:00 +CPU= 20.176
09:44:24 STEP 1634 H=1634:00 +CPU= 20.495
09:44:56 STEP 1635 H=1635:00 +CPU= 30.000
09:45:18 STEP 1636 H=1636:00 +CPU= 19.883
09:45:38 STEP 1637 H=1637:00 +CPU= 19.431
09:46:13 STEP 1638 H=1638:00 +CPU= 32.330
09:46:35 STEP 1639 H=1639:00 +CPU= 20.002
09:46:58 STEP 1640 H=1640:00 +CPU= 20.940
09:47:31 STEP 1641 H=1641:00 +CPU= 30.535
09:47:53 STEP 1642 H=1642:00 +CPU= 20.298
09:48:14 STEP 1643 H=1643:00 +CPU= 19.493
09:48:51 STEP 1644 H=1644:00 +CPU= 33.898
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001644
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001644
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001644
09:49:13 STEP 1645 H=1645:00 +CPU= 19.694
09:49:35 STEP 1646 H=1646:00 +CPU= 20.335
09:50:08 STEP 1647 H=1647:00 +CPU= 30.402
09:50:29 STEP 1648 H=1648:00 +CPU= 19.927
09:50:52 STEP 1649 H=1649:00 +CPU= 19.863
09:51:25 STEP 1650 H=1650:00 +CPU= 31.395
09:51:46 STEP 1651 H=1651:00 +CPU= 19.001
09:52:08 STEP 1652 H=1652:00 +CPU= 20.819
09:52:39 STEP 1653 H=1653:00 +CPU= 29.948
09:53:02 STEP 1654 H=1654:00 +CPU= 21.663
09:53:26 STEP 1655 H=1655:00 +CPU= 21.998
09:54:06 STEP 1656 H=1656:00 +CPU= 36.688
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001656
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001656
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001644
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0239_2004050100_123_973_12189883_0_r2084701587_68.zip
Uploading the intermediate file: upload_file_68.zip
09:54:28 STEP 1657 H=1657:00 +CPU= 19.861
09:54:50 STEP 1658 H=1658:00 +CPU= 20.491
09:55:24 STEP 1659 H=1659:00 +CPU= 30.869
09:55:46 STEP 1660 H=1660:00 +CPU= 21.117
09:56:06 STEP 1661 H=1661:00 +CPU= 19.388
09:56:40 STEP 1662 H=1662:00 +CPU= 31.647
09:56:58 STEP 1663 H=1663:00 +CPU= 16.695
09:57:22 STEP 1664 H=1664:00 +CPU= 21.202
09:57:57 STEP 1665 H=1665:00 +CPU= 32.591
09:58:22 STEP 1666 H=1666:00 +CPU= 21.450
09:58:40 STEP 1667 H=1667:00 +CPU= 18.292
09:59:14 STEP 1668 H=1668:00 +CPU= 31.063
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001668
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001668
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001668
09:59:31 STEP 1669 H=1669:00 +CPU= 17.466
09:59:52 STEP 1670 H=1670:00 +CPU= 19.865
10:00:25 STEP 1671 H=1671:00 +CPU= 31.375
10:00:47 STEP 1672 H=1672:00 +CPU= 20.026
10:01:07 STEP 1673 H=1673:00 +CPU= 18.986
10:01:39 STEP 1674 H=1674:00 +CPU= 30.409
10:02:01 STEP 1675 H=1675:00 +CPU= 20.624
10:02:24 STEP 1676 H=1676:00 +CPU= 21.886
10:02:56 STEP 1677 H=1677:00 +CPU= 30.335
10:03:18 STEP 1678 H=1678:00 +CPU= 19.973
10:03:41 STEP 1679 H=1679:00 +CPU= 21.121
10:04:21 STEP 1680 H=1680:00 +CPU= 37.479
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001680
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001680
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001668
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0239_2004050100_123_973_12189883_0_r2084701587_69.zip
Uploading the intermediate file: upload_file_69.zip
10:04:44 STEP 1681 H=1681:00 +CPU= 20.844
10:05:06 STEP 1682 H=1682:00 +CPU= 20.915
10:05:41 STEP 1683 H=1683:00 +CPU= 32.385
10:06:04 STEP 1684 H=1684:00 +CPU= 21.055
10:06:26 STEP 1685 H=1685:00 +CPU= 20.410
10:06:59 STEP 1686 H=1686:00 +CPU= 30.566
10:07:21 STEP 1687 H=1687:00 +CPU= 21.164
10:07:44 STEP 1688 H=1688:00 +CPU= 21.389
10:08:17 STEP 1689 H=1689:00 +CPU= 31.509
10:08:40 STEP 1690 H=1690:00 +CPU= 20.796
10:09:02 STEP 1691 H=1691:00 +CPU= 20.426
10:09:42 STEP 1692 H=1692:00 +CPU= 35.452
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001692
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001692
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001692
10:10:00 STEP 1693 H=1693:00 +CPU= 16.978
10:10:25 STEP 1694 H=1694:00 +CPU= 22.382
10:10:58 STEP 1695 H=1695:00 +CPU= 31.836
10:11:20 STEP 1696 H=1696:00 +CPU= 20.511
10:11:42 STEP 1697 H=1697:00 +CPU= 20.504
10:12:17 STEP 1698 H=1698:00 +CPU= 32.628
10:12:38 STEP 1699 H=1699:00 +CPU= 19.754
10:12:58 STEP 1700 H=1700:00 +CPU= 19.109
10:13:27 STEP 1701 H=1701:00 +CPU= 27.271
10:13:48 STEP 1702 H=1702:00 +CPU= 19.915
10:14:08 STEP 1703 H=1703:00 +CPU= 19.094
10:14:48 STEP 1704 H=1704:00 +CPU= 36.116
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001704
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001704
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001692
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0239_2004050100_123_973_12189883_0_r2084701587_70.zip
Uploading the intermediate file: upload_file_70.zip
10:15:09 STEP 1705 H=1705:00 +CPU= 19.544
10:15:31 STEP 1706 H=1706:00 +CPU= 20.341
10:16:05 STEP 1707 H=1707:00 +CPU= 31.118
10:16:28 STEP 1708 H=1708:00 +CPU= 20.892
10:16:50 STEP 1709 H=1709:00 +CPU= 20.355
10:17:25 STEP 1710 H=1710:00 +CPU= 32.143
10:17:47 STEP 1711 H=1711:00 +CPU= 20.306
10:18:10 STEP 1712 H=1712:00 +CPU= 20.710
10:18:42 STEP 1713 H=1713:00 +CPU= 30.327
10:19:06 STEP 1714 H=1714:00 +CPU= 20.688
10:19:28 STEP 1715 H=1715:00 +CPU= 19.785
10:20:04 STEP 1716 H=1716:00 +CPU= 33.546
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001716
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001716
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001716
10:20:25 STEP 1717 H=1717:00 +CPU= 19.528
10:20:48 STEP 1718 H=1718:00 +CPU= 20.659
10:21:17 STEP 1719 H=1719:00 +CPU= 27.681
10:21:39 STEP 1720 H=1720:00 +CPU= 21.248
10:22:00 STEP 1721 H=1721:00 +CPU= 20.033
10:22:34 STEP 1722 H=1722:00 +CPU= 30.980
10:22:55 STEP 1723 H=1723:00 +CPU= 19.642
10:23:17 STEP 1724 H=1724:00 +CPU= 20.083
10:23:49 STEP 1725 H=1725:00 +CPU= 29.942
10:24:11 STEP 1726 H=1726:00 +CPU= 19.998
10:24:32 STEP 1727 H=1727:00 +CPU= 19.858
10:25:11 STEP 1728 H=1728:00 +CPU= 35.759
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001728
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001728
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001716
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0239_2004050100_123_973_12189883_0_r2084701587_71.zip
Uploading the intermediate file: upload_file_71.zip
10:25:32 STEP 1729 H=1729:00 +CPU= 19.062
Uploading trickle at timestep: 6220800
10:25:53 STEP 1730 H=1730:00 +CPU= 19.671
10:26:27 STEP 1731 H=1731:00 +CPU= 29.808
10:26:48 STEP 1732 H=1732:00 +CPU= 19.564
10:27:09 STEP 1733 H=1733:00 +CPU= 19.219
10:27:44 STEP 1734 H=1734:00 +CPU= 31.264
10:28:06 STEP 1735 H=1735:00 +CPU= 19.515
10:28:27 STEP 1736 H=1736:00 +CPU= 19.879
10:29:00 STEP 1737 H=1737:00 +CPU= 30.808
10:29:22 STEP 1738 H=1738:00 +CPU= 20.599
10:29:43 STEP 1739 H=1739:00 +CPU= 19.364
10:30:20 STEP 1740 H=1740:00 +CPU= 33.287
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001740
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001740
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001740
10:30:41 STEP 1741 H=1741:00 +CPU= 19.239
10:31:02 STEP 1742 H=1742:00 +CPU= 19.397
10:31:29 STEP 1743 H=1743:00 +CPU= 26.091
10:31:46 STEP 1744 H=1744:00 +CPU= 17.347
10:32:04 STEP 1745 H=1745:00 +CPU= 17.087
10:32:45 STEP 1746 H=1746:00 +CPU= 32.899
10:33:07 STEP 1747 H=1747:00 +CPU= 20.573
10:33:29 STEP 1748 H=1748:00 +CPU= 21.307
10:34:03 STEP 1749 H=1749:00 +CPU= 31.719
10:34:27 STEP 1750 H=1750:00 +CPU= 22.154
10:34:51 STEP 1751 H=1751:00 +CPU= 21.692
10:35:31 STEP 1752 H=1752:00 +CPU= 37.461
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001752
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001752
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001740
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0239_2004050100_123_973_12189883_0_r2084701587_72.zip
Uploading the intermediate file: upload_file_72.zip
10:35:54 STEP 1753 H=1753:00 +CPU= 20.489
10:36:16 STEP 1754 H=1754:00 +CPU= 20.729
10:36:50 STEP 1755 H=1755:00 +CPU= 30.834
10:37:13 STEP 1756 H=1756:00 +CPU= 21.435
10:37:37 STEP 1757 H=1757:00 +CPU= 21.273
10:38:14 STEP 1758 H=1758:00 +CPU= 33.208
10:38:36 STEP 1759 H=1759:00 +CPU= 20.765
10:38:59 STEP 1760 H=1760:00 +CPU= 21.010
10:39:35 STEP 1761 H=1761:00 +CPU= 32.196
10:39:58 STEP 1762 H=1762:00 +CPU= 22.238
10:40:20 STEP 1763 H=1763:00 +CPU= 20.275
10:40:57 STEP 1764 H=1764:00 +CPU= 35.332
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001764
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001764
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001764
10:41:19 STEP 1765 H=1765:00 +CPU= 20.523
10:41:42 STEP 1766 H=1766:00 +CPU= 20.700
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2004050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0239
(argv4) batchid: 973
(argv5) wuid: 12189883
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/6
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/6/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/6/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_834c954b7040b6c2eba35307e2f220c3 to: /var/lib/boinc-client/slots/6/oifs_43r3_ps_0239_2004050100_123_973_12189883.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/6/oifs_43r3_ps_0239_2004050100_123_973_12189883.zip
ic_ancil_file: ic_ancil_12189883
ifsdata_file: ifsdata_12189883
climate_data_file: clim_data_12189883
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_a3ffaf10fb1b456d1b6a7b2bbb19ecc4 to: /var/lib/boinc-client/slots/6/ic_ancil_12189883.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/6/ic_ancil_12189883.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_7767775e9b853d6358b36622de3e4f82 to: /var/lib/boinc-client/slots/6/ifsdata/ifsdata_12189883.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/6/ifsdata/ifsdata_12189883.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/6/159l_2/clim_data_12189883.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/6/159l_2/clim_data_12189883.zip
Checking for progress XML file: /var/lib/boinc-client/slots/6/progress_file_12189883.xml
Opened progress file ok : /var/lib/boinc-client/slots/6/progress_file_12189883.xml
-- Model is restarting --
Adjusting last_iter, 1767, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/6/progress_file_12189883.xml
last_cpu_time: 42170
upload_file_number: 73
last_iter: 1751
last_upload: 6307200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0239_2004050100_123_973_12189883_0_r2084701587
The child process has been launched with process id: 674170
Executing the command: /var/lib/boinc-client/slots/6/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:674170:674170] [20221224:104335:1671839015:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
10:44:17 STEP 1752 H=1752:00 +CPU= 35.661
10:44:39 STEP 1753 H=1753:00 +CPU= 20.533
10:45:02 STEP 1754 H=1754:00 +CPU= 21.158
10:45:36 STEP 1755 H=1755:00 +CPU= 31.821
10:46:00 STEP 1756 H=1756:00 +CPU= 21.834
10:46:23 STEP 1757 H=1757:00 +CPU= 20.724
10:46:58 STEP 1758 H=1758:00 +CPU= 32.225
10:47:20 STEP 1759 H=1759:00 +CPU= 20.415
10:47:43 STEP 1760 H=1760:00 +CPU= 20.590
10:48:16 STEP 1761 H=1761:00 +CPU= 30.136
10:48:38 STEP 1762 H=1762:00 +CPU= 19.973
10:48:58 STEP 1763 H=1763:00 +CPU= 19.255
10:49:29 STEP 1764 H=1764:00 +CPU= 29.111
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001764
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001764
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001764
10:49:51 STEP 1765 H=1765:00 +CPU= 19.913
10:50:13 STEP 1766 H=1766:00 +CPU= 19.990
10:50:46 STEP 1767 H=1767:00 +CPU= 30.319
10:51:03 STEP 1768 H=1768:00 +CPU= 16.523
10:51:25 STEP 1769 H=1769:00 +CPU= 20.201
10:51:55 STEP 1770 H=1770:00 +CPU= 28.789
10:52:13 STEP 1771 H=1771:00 +CPU= 17.045
10:52:35 STEP 1772 H=1772:00 +CPU= 20.218
10:53:09 STEP 1773 H=1773:00 +CPU= 31.001
10:53:32 STEP 1774 H=1774:00 +CPU= 20.840
10:53:53 STEP 1775 H=1775:00 +CPU= 19.825
10:54:34 STEP 1776 H=1776:00 +CPU= 35.903
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001776
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001776
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001764
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0239_2004050100_123_973_12189883_0_r2084701587_73.zip
Uploading the intermediate file: upload_file_73.zip
10:54:55 STEP 1777 H=1777:00 +CPU= 19.046
10:55:17 STEP 1778 H=1778:00 +CPU= 20.056
10:55:51 STEP 1779 H=1779:00 +CPU= 30.779
10:56:13 STEP 1780 H=1780:00 +CPU= 19.883
10:56:35 STEP 1781 H=1781:00 +CPU= 19.696
10:57:09 STEP 1782 H=1782:00 +CPU= 31.065
10:57:27 STEP 1783 H=1783:00 +CPU= 17.457
10:57:49 STEP 1784 H=1784:00 +CPU= 20.325
10:58:22 STEP 1785 H=1785:00 +CPU= 30.866
10:58:44 STEP 1786 H=1786:00 +CPU= 20.033
10:59:04 STEP 1787 H=1787:00 +CPU= 19.359
10:59:40 STEP 1788 H=1788:00 +CPU= 34.024
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001788
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001788
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001788
11:00:01 STEP 1789 H=1789:00 +CPU= 19.388
11:00:20 STEP 1790 H=1790:00 +CPU= 17.990
11:00:53 STEP 1791 H=1791:00 +CPU= 30.847
11:01:15 STEP 1792 H=1792:00 +CPU= 20.359
11:01:33 STEP 1793 H=1793:00 +CPU= 17.003
11:02:04 STEP 1794 H=1794:00 +CPU= 30.077
11:02:27 STEP 1795 H=1795:00 +CPU= 20.136
11:02:49 STEP 1796 H=1796:00 +CPU= 20.363
11:03:20 STEP 1797 H=1797:00 +CPU= 30.201
11:03:39 STEP 1798 H=1798:00 +CPU= 17.606
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2004050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0239
(argv4) batchid: 973
(argv5) wuid: 12189883
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/6
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/6/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/6/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_834c954b7040b6c2eba35307e2f220c3 to: /var/lib/boinc-client/slots/6/oifs_43r3_ps_0239_2004050100_123_973_12189883.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/6/oifs_43r3_ps_0239_2004050100_123_973_12189883.zip
ic_ancil_file: ic_ancil_12189883
ifsdata_file: ifsdata_12189883
climate_data_file: clim_data_12189883
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_a3ffaf10fb1b456d1b6a7b2bbb19ecc4 to: /var/lib/boinc-client/slots/6/ic_ancil_12189883.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/6/ic_ancil_12189883.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_7767775e9b853d6358b36622de3e4f82 to: /var/lib/boinc-client/slots/6/ifsdata/ifsdata_12189883.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/6/ifsdata/ifsdata_12189883.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/6/159l_2/clim_data_12189883.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/6/159l_2/clim_data_12189883.zip
Checking for progress XML file: /var/lib/boinc-client/slots/6/progress_file_12189883.xml
Opened progress file ok : /var/lib/boinc-client/slots/6/progress_file_12189883.xml
-- Model is restarting --
Adjusting last_iter, 1799, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/6/progress_file_12189883.xml
last_cpu_time: 43293
upload_file_number: 74
last_iter: 1775
last_upload: 6393600
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0239_2004050100_123_973_12189883_0_r2084701587
The child process has been launched with process id: 690803
Executing the command: /var/lib/boinc-client/slots/6/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:690803:690803] [20221224:135941:1671850781:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
14:00:22 STEP 1776 H=1776:00 +CPU= 35.259
14:00:44 STEP 1777 H=1777:00 +CPU= 21.047
14:01:06 STEP 1778 H=1778:00 +CPU= 21.100
14:01:39 STEP 1779 H=1779:00 +CPU= 31.945
14:01:59 STEP 1780 H=1780:00 +CPU= 19.588
14:02:19 STEP 1781 H=1781:00 +CPU= 19.234
14:02:51 STEP 1782 H=1782:00 +CPU= 30.088
14:03:08 STEP 1783 H=1783:00 +CPU= 16.839
14:03:26 STEP 1784 H=1784:00 +CPU= 18.065
14:03:51 STEP 1785 H=1785:00 +CPU= 23.989
14:04:04 STEP 1786 H=1786:00 +CPU= 12.621
14:04:16 STEP 1787 H=1787:00 +CPU= 12.374
14:04:38 STEP 1788 H=1788:00 +CPU= 21.406
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001788
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001788
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001788
14:04:56 STEP 1789 H=1789:00 +CPU= 16.966
14:05:15 STEP 1790 H=1790:00 +CPU= 19.090
14:05:47 STEP 1791 H=1791:00 +CPU= 30.480
14:06:07 STEP 1792 H=1792:00 +CPU= 19.973
14:06:25 STEP 1793 H=1793:00 +CPU= 17.286
14:06:59 STEP 1794 H=1794:00 +CPU= 32.078
14:07:18 STEP 1795 H=1795:00 +CPU= 18.925
14:07:38 STEP 1796 H=1796:00 +CPU= 19.377
14:08:10 STEP 1797 H=1797:00 +CPU= 29.466
14:08:31 STEP 1798 H=1798:00 +CPU= 19.540
14:08:52 STEP 1799 H=1799:00 +CPU= 19.583
14:09:30 STEP 1800 H=1800:00 +CPU= 34.814
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001800
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001800
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMGGhq0f+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMSHhq0f+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883/ICMUAhq0f+001788
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0239_2004050100_123_973_12189883_0_r2084701587_74.zip
Uploading the intermediate file: upload_file_74.zip
14:09:41 STEP 1801 H=1801:00 +CPU= 11.200
14:09:50 STEP 1802 H=1802:00 +CPU= 9.302
Suspend request received from the BOINC client, suspending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2004050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0239
(argv4) batchid: 973
(argv5) wuid: 12189883
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/6
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/6/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/6/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_834c954b7040b6c2eba35307e2f220c3 to: /var/lib/boinc-client/slots/6/oifs_43r3_ps_0239_2004050100_123_973_12189883.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/6/oifs_43r3_ps_0239_2004050100_123_973_12189883.zip
ic_ancil_file: ic_ancil_12189883
ifsdata_file: ifsdata_12189883
climate_data_file: clim_data_12189883
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_a3ffaf10fb1b456d1b6a7b2bbb19ecc4 to: /var/lib/boinc-client/slots/6/ic_ancil_12189883.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/6/ic_ancil_12189883.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_7767775e9b853d6358b36622de3e4f82 to: /var/lib/boinc-client/slots/6/ifsdata/ifsdata_12189883.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/6/ifsdata/ifsdata_12189883.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/6/159l_2/clim_data_12189883.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/6/159l_2/clim_data_12189883.zip
Checking for progress XML file: /var/lib/boinc-client/slots/6/progress_file_12189883.xml
Opened progress file ok : /var/lib/boinc-client/slots/6/progress_file_12189883.xml
-- Model is restarting --
Adjusting last_iter, 1801, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/6/progress_file_12189883.xml
last_cpu_time: 43860
upload_file_number: 75
last_iter: 1799
last_upload: 6480000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0239_2004050100_123_973_12189883_0_r2084701587
The child process has been launched with process id: 9824
Executing the command: /var/lib/boinc-client/slots/6/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:9824:9824] [20221224:161054:1671858654:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
16:11:43 STEP 1800 H=1800:00 +CPU= 40.175
16:12:05 STEP 1801 H=1801:00 +CPU= 20.500
16:12:26 STEP 1802 H=1802:00 +CPU= 20.026
16:12:58 STEP 1803 H=1803:00 +CPU= 30.809
16:13:19 STEP 1804 H=1804:00 +CPU= 20.347
16:13:39 STEP 1805 H=1805:00 +CPU= 19.897
16:14:11 STEP 1806 H=1806:00 +CPU= 30.219
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2004050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0239
(argv4) batchid: 973
(argv5) wuid: 12189883
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/6
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189883
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/6/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/6/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_834c954b7040b6c2eba35307e2f220c3 to: /var/lib/boinc-client/slots/6/oifs_43r3_ps_0239_2004050100_123_973_12189883.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/6/oifs_43r3_ps_0239_2004050100_123_973_12189883.zip
ic_ancil_file: ic_ancil_12189883
ifsdata_file: ifsdata_12189883
climate_data_file: clim_data_12189883
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_a3ffaf10fb1b456d1b6a7b2bbb19ecc4 to: /var/lib/boinc-client/slots/6/ic_ancil_12189883.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/6/ic_ancil_12189883.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_7767775e9b853d6358b36622de3e4f82 to: /var/lib/boinc-client/slots/6/ifsdata/ifsdata_12189883.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/6/ifsdata/ifsdata_12189883.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/6/159l_2/clim_data_12189883.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/6/159l_2/clim_data_12189883.zip
Checking for progress XML file: /var/lib/boinc-client/slots/6/progress_file_12189883.xml
Opened progress file ok : /var/lib/boinc-client/slots/6/progress_file_12189883.xml
-- Model is restarting --
Adjusting last_iter, 1807, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/6/progress_file_12189883.xml
last_cpu_time: 44060
upload_file_number: 75
last_iter: 1799
last_upload: 6480000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0239_2004050100_123_973_12189883_0_r2084701587
The child process has been launched with process id: 10080
Executing the command: /var/lib/boinc-client/slots/6/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:10080:10080] [20221224:161532:1671858932:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
16:16:09 STEP 1800 H=1800:00 +CPU= 33.404
16:16:28 STEP 1801 H=1801:00 +CPU= 18.157
16:16:50 STEP 1802 H=1802:00 +CPU= 20.955
16:17:21 STEP 1803 H=1803:00 +CPU= 30.149
16:17:40 STEP 1804 H=1804:00 +CPU= 18.587
16:18:03 STEP 1805 H=1805:00 +CPU= 20.694
16:18:37 STEP 1806 H=1806:00 +CPU= 33.125
16:18:57 STEP 1807 H=1807:00 +CPU= 18.589
16:19:20 STEP 1808 H=1808:00 +CPU= 22.840
16:19:52 STEP 1809 H=1809:00 +CPU= 30.987
16:20:12 STEP 1810 H=1810:00 +CPU= 19.286
16:20:33 STEP 1811 H=1811:00 +CPU= 19.727
16:21:09 STEP 1812 H=1812:00 +CPU= 34.428
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMGGhq0f+001812
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMSHhq0f+001812
Moving to projects directory: /var/lib/boinc-client/slots/6/ICMUAhq0f+001812
16:21:29 STEP 1813 H=1813:00 +CPU= 18.801
16:21:50 STEP 1814 H=1814:00 +CPU= 19.232
16:22:21 STEP 1815 H=1815:00 +CPU= 29.261
16:22:40 STEP 1816 H=1816:00 +CPU= 19.228
16:23:00 STEP 1817 H=1817:00 +CPU= 19.091
16:23:33 STEP 1818 H=1818:00 +CPU= 31.284
16:23:53 STEP 1819 H=1819:00 +CPU= 19.052
16:24:14 STEP 1820 H=1820:00 +CPU= 19.827
16:24:49 STEP 1821 H=1821:00 +CPU= 30.755
16:25:09 STEP 1822 H=1822:00 +CPU= 19.224
16:25:32 STEP 1823 H=1823:00 +CPU= 20.471
Abort request received from BOINC client, ending the child process
..Failed, model did not complete successfully
</stderr_txt>
]]>
