climateprediction.net home page
Task 22268800

Task 22268800

Name oifs_43r3_ps_0965_1997050100_123_966_12183609_0
Workunit 12183609
Created 22 Dec 2022, 14:57:37 UTC
Sent 5 Jan 2023, 7:05:49 UTC
Report deadline 4 Feb 2023, 7:05:49 UTC
Received 12 Jan 2023, 6:47:24 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1454366
Run time 13 hours 10 min
CPU time 12 hours 23 min 56 sec
Validate state Invalid
Credit 7,499.52
Device peak FLOPS 5.04 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,416.43 MB
Peak swap size 4,978.07 MB
Peak disk usage 7,185.33 MB
Stderr
<core_client_version>7.16.16</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
0.171
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:10:30 STEP 1589 H=1589:00 +CPU= 19.653
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:11:05 STEP 1590 H=1590:00 +CPU= 31.415
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:11:27 STEP 1591 H=1591:00 +CPU= 19.667
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:11:49 STEP 1592 H=1592:00 +CPU= 20.225
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:12:22 STEP 1593 H=1593:00 +CPU= 30.789
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:12:46 STEP 1594 H=1594:00 +CPU= 20.174
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:13:08 STEP 1595 H=1595:00 +CPU= 19.657
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:13:46 STEP 1596 H=1596:00 +CPU= 34.014
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+001596
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+001596
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+001596
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:14:08 STEP 1597 H=1597:00 +CPU= 19.631
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:14:30 STEP 1598 H=1598:00 +CPU= 20.199
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:15:05 STEP 1599 H=1599:00 +CPU= 30.807
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:15:27 STEP 1600 H=1600:00 +CPU= 20.153
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:15:49 STEP 1601 H=1601:00 +CPU= 19.602
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:16:25 STEP 1602 H=1602:00 +CPU= 31.494
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:16:46 STEP 1603 H=1603:00 +CPU= 19.613
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:17:09 STEP 1604 H=1604:00 +CPU= 20.118
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:17:41 STEP 1605 H=1605:00 +CPU= 30.711
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:18:06 STEP 1606 H=1606:00 +CPU= 20.024
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:18:28 STEP 1607 H=1607:00 +CPU= 20.156
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:19:08 STEP 1608 H=1608:00 +CPU= 36.027
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+001608
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+001608
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMGGhq0f+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMSHhq0f+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMUAhq0f+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMGGhq0f+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMSHhq0f+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMUAhq0f+001596
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0965_1997050100_123_966_12183609_0_r1695271824_66.zip
Uploading the intermediate file: upload_file_66.zip
  05:19:28 STEP 1609 H=1609:00 +CPU= 19.556
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:19:50 STEP 1610 H=1610:00 +CPU= 20.121
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:20:25 STEP 1611 H=1611:00 +CPU= 30.805
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:20:47 STEP 1612 H=1612:00 +CPU= 20.079
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:21:09 STEP 1613 H=1613:00 +CPU= 19.568
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:21:42 STEP 1614 H=1614:00 +CPU= 31.502
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:22:06 STEP 1615 H=1615:00 +CPU= 19.566
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:22:28 STEP 1616 H=1616:00 +CPU= 20.057
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:23:01 STEP 1617 H=1617:00 +CPU= 30.838
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:23:25 STEP 1618 H=1618:00 +CPU= 20.245
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:23:47 STEP 1619 H=1619:00 +CPU= 19.547
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:24:25 STEP 1620 H=1620:00 +CPU= 34.065
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+001620
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+001620
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+001620
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:24:47 STEP 1621 H=1621:00 +CPU= 19.653
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:25:09 STEP 1622 H=1622:00 +CPU= 20.210
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:25:42 STEP 1623 H=1623:00 +CPU= 30.865
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:26:06 STEP 1624 H=1624:00 +CPU= 20.108
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:26:28 STEP 1625 H=1625:00 +CPU= 19.621
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:27:02 STEP 1626 H=1626:00 +CPU= 31.503
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:27:25 STEP 1627 H=1627:00 +CPU= 19.547
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:27:48 STEP 1628 H=1628:00 +CPU= 20.181
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:28:21 STEP 1629 H=1629:00 +CPU= 30.854
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:28:45 STEP 1630 H=1630:00 +CPU= 20.099
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:29:07 STEP 1631 H=1631:00 +CPU= 20.176
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:29:47 STEP 1632 H=1632:00 +CPU= 36.078
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+001632
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+001632
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMGGhq0f+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMSHhq0f+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMUAhq0f+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMGGhq0f+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMSHhq0f+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMUAhq0f+001620
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0965_1997050100_123_966_12183609_0_r1695271824_67.zip
Uploading the intermediate file: upload_file_67.zip
  05:30:07 STEP 1633 H=1633:00 +CPU= 19.400
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:30:29 STEP 1634 H=1634:00 +CPU= 20.162
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:31:02 STEP 1635 H=1635:00 +CPU= 30.783
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:31:26 STEP 1636 H=1636:00 +CPU= 20.090
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:31:48 STEP 1637 H=1637:00 +CPU= 19.619
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:32:22 STEP 1638 H=1638:00 +CPU= 31.497
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:32:45 STEP 1639 H=1639:00 +CPU= 19.550
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:33:08 STEP 1640 H=1640:00 +CPU= 20.208
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:33:40 STEP 1641 H=1641:00 +CPU= 30.846
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:34:03 STEP 1642 H=1642:00 +CPU= 20.077
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:34:26 STEP 1643 H=1643:00 +CPU= 19.686
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:35:03 STEP 1644 H=1644:00 +CPU= 34.003
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+001644
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+001644
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+001644
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:35:26 STEP 1645 H=1645:00 +CPU= 19.623
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:35:48 STEP 1646 H=1646:00 +CPU= 20.138
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:36:21 STEP 1647 H=1647:00 +CPU= 30.803
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:36:46 STEP 1648 H=1648:00 +CPU= 20.107
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:37:07 STEP 1649 H=1649:00 +CPU= 19.512
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:37:41 STEP 1650 H=1650:00 +CPU= 31.394
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:38:04 STEP 1651 H=1651:00 +CPU= 19.620
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:38:27 STEP 1652 H=1652:00 +CPU= 20.193
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:39:00 STEP 1653 H=1653:00 +CPU= 30.813
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:39:22 STEP 1654 H=1654:00 +CPU= 20.162
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:39:46 STEP 1655 H=1655:00 +CPU= 20.198
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:40:26 STEP 1656 H=1656:00 +CPU= 35.994
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+001656
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+001656
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMGGhq0f+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMSHhq0f+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMUAhq0f+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMGGhq0f+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMSHhq0f+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMUAhq0f+001644
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0965_1997050100_123_966_12183609_0_r1695271824_68.zip
Uploading the intermediate file: upload_file_68.zip
  05:40:46 STEP 1657 H=1657:00 +CPU= 19.513
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:41:08 STEP 1658 H=1658:00 +CPU= 20.231
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:41:41 STEP 1659 H=1659:00 +CPU= 30.837
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:42:06 STEP 1660 H=1660:00 +CPU= 20.270
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:42:27 STEP 1661 H=1661:00 +CPU= 19.631
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:43:01 STEP 1662 H=1662:00 +CPU= 31.416
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:43:25 STEP 1663 H=1663:00 +CPU= 19.689
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:43:47 STEP 1664 H=1664:00 +CPU= 20.210
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:44:20 STEP 1665 H=1665:00 +CPU= 30.841
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:44:42 STEP 1666 H=1666:00 +CPU= 20.231
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:45:06 STEP 1667 H=1667:00 +CPU= 19.595
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:45:42 STEP 1668 H=1668:00 +CPU= 34.057
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+001668
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+001668
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+001668
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:46:06 STEP 1669 H=1669:00 +CPU= 19.630
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:46:28 STEP 1670 H=1670:00 +CPU= 20.195
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:47:01 STEP 1671 H=1671:00 +CPU= 30.865
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:47:25 STEP 1672 H=1672:00 +CPU= 20.119
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:47:47 STEP 1673 H=1673:00 +CPU= 19.603
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:48:20 STEP 1674 H=1674:00 +CPU= 31.499
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:48:42 STEP 1675 H=1675:00 +CPU= 19.570
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:49:06 STEP 1676 H=1676:00 +CPU= 20.005
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:49:39 STEP 1677 H=1677:00 +CPU= 30.869
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:50:01 STEP 1678 H=1678:00 +CPU= 20.184
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:50:26 STEP 1679 H=1679:00 +CPU= 20.002
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:51:06 STEP 1680 H=1680:00 +CPU= 36.009
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+001680
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+001680
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMGGhq0f+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMSHhq0f+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMUAhq0f+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMGGhq0f+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMSHhq0f+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMUAhq0f+001668
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0965_1997050100_123_966_12183609_0_r1695271824_69.zip
Uploading the intermediate file: upload_file_69.zip
  05:51:25 STEP 1681 H=1681:00 +CPU= 19.470
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:51:48 STEP 1682 H=1682:00 +CPU= 20.124
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:52:21 STEP 1683 H=1683:00 +CPU= 30.854
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:52:45 STEP 1684 H=1684:00 +CPU= 20.423
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:53:08 STEP 1685 H=1685:00 +CPU= 20.759
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:53:45 STEP 1686 H=1686:00 +CPU= 33.055
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:54:08 STEP 1687 H=1687:00 +CPU= 20.873
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:54:30 STEP 1688 H=1688:00 +CPU= 20.272
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:55:05 STEP 1689 H=1689:00 +CPU= 30.904
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:55:28 STEP 1690 H=1690:00 +CPU= 20.225
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:55:49 STEP 1691 H=1691:00 +CPU= 19.559
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:56:28 STEP 1692 H=1692:00 +CPU= 34.132
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+001692
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+001692
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+001692
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:56:49 STEP 1693 H=1693:00 +CPU= 19.607
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:57:12 STEP 1694 H=1694:00 +CPU= 20.254
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:57:47 STEP 1695 H=1695:00 +CPU= 30.863
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:58:09 STEP 1696 H=1696:00 +CPU= 20.109
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:58:31 STEP 1697 H=1697:00 +CPU= 19.667
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:59:06 STEP 1698 H=1698:00 +CPU= 31.461
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:59:28 STEP 1699 H=1699:00 +CPU= 19.575
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:59:50 STEP 1700 H=1700:00 +CPU= 20.197
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:00:25 STEP 1701 H=1701:00 +CPU= 30.836
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:00:47 STEP 1702 H=1702:00 +CPU= 19.987
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:01:09 STEP 1703 H=1703:00 +CPU= 20.253
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:01:50 STEP 1704 H=1704:00 +CPU= 36.022
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+001704
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+001704
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMGGhq0f+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMSHhq0f+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMUAhq0f+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMGGhq0f+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMSHhq0f+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMUAhq0f+001692
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0965_1997050100_123_966_12183609_0_r1695271824_70.zip
Uploading the intermediate file: upload_file_70.zip
  06:02:09 STEP 1705 H=1705:00 +CPU= 19.513
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:02:32 STEP 1706 H=1706:00 +CPU= 20.167
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:03:07 STEP 1707 H=1707:00 +CPU= 30.874
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:03:29 STEP 1708 H=1708:00 +CPU= 20.272
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:03:51 STEP 1709 H=1709:00 +CPU= 19.741
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:04:27 STEP 1710 H=1710:00 +CPU= 31.575
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1997050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0965
(argv4) batchid: 966
(argv5) wuid: 12183609
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/1
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_8c74c206b27ce2a30ac8e399ef56d744 to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0965_1997050100_123_966_12183609.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0965_1997050100_123_966_12183609.zip
ic_ancil_file: ic_ancil_12183609
ifsdata_file: ifsdata_12183609
climate_data_file: clim_data_12183609
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_c328814daba69aaa1621bb1d6df3825f to: /var/lib/boinc-client/slots/1/ic_ancil_12183609.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/1/ic_ancil_12183609.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_64bf5757b85325b22de978bb6ea79f9c to: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12183609.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12183609.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/1/159l_2/clim_data_12183609.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/1/159l_2/clim_data_12183609.zip
Checking for progress XML file: /var/lib/boinc-client/slots/1/progress_file_12183609.xml
Opened progress file ok : /var/lib/boinc-client/slots/1/progress_file_12183609.xml
-- Model is restarting --
Adjusting last_iter, 1711, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/1/progress_file_12183609.xml
last_cpu_time: 42839
upload_file_number: 71
last_iter: 1703
last_upload: 6134400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0965_1997050100_123_966_12183609_0_r1695271824
The child process has been launched with process id: 600121
Executing the command: /var/lib/boinc-client/slots/1/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:charm:1:1:600121:600121] [20230112:071116:1673500276:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:charm:1:1:600121:600121] [20230112:071116:1673500276:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:charm:1:1:600121:600121] [20230112:071116:1673500276:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:charm:1:1:600121:600121] [20230112:071116:1673500276:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:charm:1:1:600121:600121] [20230112:071116:1673500276:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:charm:1:1:600121:600121] [20230112:071116:1673500276:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:charm:1:1:600121:600121] [20230112:071116:1673500276:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:charm:1:1:600121:600121] [20230112:071116:1673500276:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:charm:1:1:600121:600121] [20230112:071116:1673500276:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:charm:1:1:600121:600121] [20230112:071116:1673500276:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:charm:1:1:600121:600121] [20230112:071116:1673500276:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:charm:1:1:600121:600121] [20230112:071116:1673500276:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:charm:1:1:600121:600121] [20230112:071116:1673500276:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:charm:1:1:600121:600121] [20230112:071116:1673500276:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:charm:1:1:600121:600121] [20230112:071116:1673500276:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:charm:1:1:600121:600121] [20230112:071116:1673500276:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:charm:1:1:600121:600121] [20230112:071116:1673500276:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:charm:1:1:600121:600121] [20230112:071116:1673500276:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:charm:1:1:600121:600121] [20230112:071116:1673500276:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:charm:1:1:600121:600121] [20230112:071116:1673500276:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:charm:1:1:600121:600121] [20230112:071116:1673500276:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:charm:1:1:600121:600121] [20230112:071116:1673500276:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:charm:1:1:600121:600121] [20230112:071116:1673500276:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:charm:1:1:600121:600121] [20230112:071116:1673500276:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:charm:1:1:600121:600121] [20230112:071116:1673500276:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:charm:1:1:600121:600121] [20230112:071116:1673500276:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:charm:1:1:600121:600121] [20230112:071116:1673500276:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:charm:1:1:600121:600121] [20230112:071116:1673500276:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:charm:1:1:600121:600121] [20230112:071116:1673500276:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:charm:1:1:600121:600121] [20230112:071116:1673500276:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:11:58 STEP 1704 H=1704:00 +CPU= 35.163
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:12:20 STEP 1705 H=1705:00 +CPU= 19.735
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:12:45 STEP 1706 H=1706:00 +CPU= 20.305
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:13:18 STEP 1707 H=1707:00 +CPU= 31.001
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:13:41 STEP 1708 H=1708:00 +CPU= 20.293
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:14:03 STEP 1709 H=1709:00 +CPU= 19.751
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:14:38 STEP 1710 H=1710:00 +CPU= 31.600
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:15:00 STEP 1711 H=1711:00 +CPU= 19.763
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:15:25 STEP 1712 H=1712:00 +CPU= 20.191
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:15:58 STEP 1713 H=1713:00 +CPU= 31.093
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:16:21 STEP 1714 H=1714:00 +CPU= 20.341
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:16:43 STEP 1715 H=1715:00 +CPU= 19.592
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:17:21 STEP 1716 H=1716:00 +CPU= 34.315
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+001716
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+001716
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+001716
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:17:45 STEP 1717 H=1717:00 +CPU= 19.788
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:18:08 STEP 1718 H=1718:00 +CPU= 20.255
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:18:41 STEP 1719 H=1719:00 +CPU= 31.035
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:19:06 STEP 1720 H=1720:00 +CPU= 20.364
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:19:27 STEP 1721 H=1721:00 +CPU= 19.561
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:20:02 STEP 1722 H=1722:00 +CPU= 31.712
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:20:25 STEP 1723 H=1723:00 +CPU= 19.577
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:20:48 STEP 1724 H=1724:00 +CPU= 20.173
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:21:21 STEP 1725 H=1725:00 +CPU= 31.117
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:21:46 STEP 1726 H=1726:00 +CPU= 20.375
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:22:09 STEP 1727 H=1727:00 +CPU= 20.423
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:22:50 STEP 1728 H=1728:00 +CPU= 36.364
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+001728
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+001728
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMGGhq0f+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMSHhq0f+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMUAhq0f+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMGGhq0f+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMSHhq0f+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMUAhq0f+001716
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0965_1997050100_123_966_12183609_0_r1695271824_71.zip
Uploading the intermediate file: upload_file_71.zip
  07:23:10 STEP 1729 H=1729:00 +CPU= 19.690
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:23:33 STEP 1730 H=1730:00 +CPU= 20.433
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:24:08 STEP 1731 H=1731:00 +CPU= 31.224
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:24:31 STEP 1732 H=1732:00 +CPU= 20.339
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:24:53 STEP 1733 H=1733:00 +CPU= 19.858
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:25:29 STEP 1734 H=1734:00 +CPU= 31.894
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:25:52 STEP 1735 H=1735:00 +CPU= 19.810
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:26:14 STEP 1736 H=1736:00 +CPU= 20.444
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:26:50 STEP 1737 H=1737:00 +CPU= 31.212
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:27:13 STEP 1738 H=1738:00 +CPU= 20.459
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:27:35 STEP 1739 H=1739:00 +CPU= 19.894
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:28:14 STEP 1740 H=1740:00 +CPU= 34.396
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+001740
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+001740
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+001740
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:28:36 STEP 1741 H=1741:00 +CPU= 19.833
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:28:59 STEP 1742 H=1742:00 +CPU= 20.418
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:29:34 STEP 1743 H=1743:00 +CPU= 31.124
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:29:57 STEP 1744 H=1744:00 +CPU= 20.411
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:30:19 STEP 1745 H=1745:00 +CPU= 19.761
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:30:56 STEP 1746 H=1746:00 +CPU= 31.746
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:31:18 STEP 1747 H=1747:00 +CPU= 19.783
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:31:40 STEP 1748 H=1748:00 +CPU= 20.206
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:32:15 STEP 1749 H=1749:00 +CPU= 30.859
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1997050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0965
(argv4) batchid: 966
(argv5) wuid: 12183609
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/1
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_8c74c206b27ce2a30ac8e399ef56d744 to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0965_1997050100_123_966_12183609.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0965_1997050100_123_966_12183609.zip
ic_ancil_file: ic_ancil_12183609
ifsdata_file: ifsdata_12183609
climate_data_file: clim_data_12183609
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_c328814daba69aaa1621bb1d6df3825f to: /var/lib/boinc-client/slots/1/ic_ancil_12183609.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/1/ic_ancil_12183609.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_64bf5757b85325b22de978bb6ea79f9c to: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12183609.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12183609.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/1/159l_2/clim_data_12183609.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/1/159l_2/clim_data_12183609.zip
Checking for progress XML file: /var/lib/boinc-client/slots/1/progress_file_12183609.xml
Opened progress file ok : /var/lib/boinc-client/slots/1/progress_file_12183609.xml
-- Model is restarting --
Adjusting last_iter, 1749, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/1/progress_file_12183609.xml
last_cpu_time: 43955
upload_file_number: 72
last_iter: 1727
last_upload: 6220800
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0965_1997050100_123_966_12183609_0_r1695271824
The child process has been launched with process id: 600219
Executing the command: /var/lib/boinc-client/slots/1/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:charm:1:1:600219:600219] [20230112:073345:1673501625:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:charm:1:1:600219:600219] [20230112:073345:1673501625:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:charm:1:1:600219:600219] [20230112:073345:1673501625:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:charm:1:1:600219:600219] [20230112:073345:1673501625:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:charm:1:1:600219:600219] [20230112:073345:1673501625:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:charm:1:1:600219:600219] [20230112:073345:1673501625:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:charm:1:1:600219:600219] [20230112:073345:1673501625:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:charm:1:1:600219:600219] [20230112:073345:1673501625:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:charm:1:1:600219:600219] [20230112:073345:1673501625:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:charm:1:1:600219:600219] [20230112:073345:1673501625:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:charm:1:1:600219:600219] [20230112:073345:1673501625:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:charm:1:1:600219:600219] [20230112:073345:1673501625:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:charm:1:1:600219:600219] [20230112:073345:1673501625:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:charm:1:1:600219:600219] [20230112:073345:1673501625:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:charm:1:1:600219:600219] [20230112:073345:1673501625:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:charm:1:1:600219:600219] [20230112:073345:1673501625:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:charm:1:1:600219:600219] [20230112:073345:1673501625:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:charm:1:1:600219:600219] [20230112:073345:1673501625:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:charm:1:1:600219:600219] [20230112:073345:1673501625:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:charm:1:1:600219:600219] [20230112:073345:1673501625:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:charm:1:1:600219:600219] [20230112:073345:1673501625:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:charm:1:1:600219:600219] [20230112:073345:1673501625:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:charm:1:1:600219:600219] [20230112:073345:1673501625:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:charm:1:1:600219:600219] [20230112:073345:1673501625:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:charm:1:1:600219:600219] [20230112:073345:1673501625:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:charm:1:1:600219:600219] [20230112:073345:1673501625:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:charm:1:1:600219:600219] [20230112:073345:1673501625:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:charm:1:1:600219:600219] [20230112:073345:1673501625:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:charm:1:1:600219:600219] [20230112:073345:1673501625:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:charm:1:1:600219:600219] [20230112:073345:1673501625:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:34:26 STEP 1728 H=1728:00 +CPU= 34.886
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:34:48 STEP 1729 H=1729:00 +CPU= 19.674
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:35:11 STEP 1730 H=1730:00 +CPU= 20.297
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:35:46 STEP 1731 H=1731:00 +CPU= 30.967
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:36:08 STEP 1732 H=1732:00 +CPU= 20.294
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:36:30 STEP 1733 H=1733:00 +CPU= 19.728
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:37:05 STEP 1734 H=1734:00 +CPU= 31.959
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:37:28 STEP 1735 H=1735:00 +CPU= 20.280
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:37:51 STEP 1736 H=1736:00 +CPU= 20.743
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:38:28 STEP 1737 H=1737:00 +CPU= 31.737
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:38:51 STEP 1738 H=1738:00 +CPU= 21.013
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:39:15 STEP 1739 H=1739:00 +CPU= 20.438
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:39:55 STEP 1740 H=1740:00 +CPU= 35.275
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+001740
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+001740
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+001740
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:40:18 STEP 1741 H=1741:00 +CPU= 20.351
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:40:42 STEP 1742 H=1742:00 +CPU= 20.987
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:41:19 STEP 1743 H=1743:00 +CPU= 32.105
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:41:46 STEP 1744 H=1744:00 +CPU= 21.091
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:42:09 STEP 1745 H=1745:00 +CPU= 20.522
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:42:46 STEP 1746 H=1746:00 +CPU= 32.632
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:43:09 STEP 1747 H=1747:00 +CPU= 20.369
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:43:32 STEP 1748 H=1748:00 +CPU= 20.758
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:44:08 STEP 1749 H=1749:00 +CPU= 31.864
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:44:32 STEP 1750 H=1750:00 +CPU= 21.013
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:44:59 STEP 1751 H=1751:00 +CPU= 21.461
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:46:07 STEP 1752 H=1752:00 +CPU= 39.857
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+001752
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+001752
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMGGhq0f+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMSHhq0f+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMUAhq0f+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMGGhq0f+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMSHhq0f+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12183609/ICMUAhq0f+001740
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0965_1997050100_123_966_12183609_0_r1695271824_72.zip
Uploading the intermediate file: upload_file_72.zip
  07:46:33 STEP 1753 H=1753:00 +CPU= 20.577
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:47:13 STEP 1754 H=1754:00 +CPU= 22.510
Abort request received from BOINC client, ending the child process
..Failed, model did not complete successfully

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:16:11 1454366 22268800 oifs_43r3_ps_0965_1997050100_123_966_12183609_0 5,356,800 37,500 0.0070
09 May 2023 16:15:40 1454366 22268800 oifs_43r3_ps_0965_1997050100_123_966_12183609_0 3,456,000 23,202 0.0067
09 May 2023 16:15:37 1454366 22268800 oifs_43r3_ps_0965_1997050100_123_966_12183609_0 2,592,000 17,409 0.0067
09 May 2023 16:15:32 1454366 22268800 oifs_43r3_ps_0965_1997050100_123_966_12183609_0 1,728,000 11,618 0.0067
09 May 2023 16:15:28 1454366 22268800 oifs_43r3_ps_0965_1997050100_123_966_12183609_0 864,000 5,820 0.0067


©2024 climateprediction.net