climateprediction.net home page
Task 22250313

Task 22250313

Name oifs_43r3_bl_a00b_2016092300_15_949_12166405_0
Workunit 12166405
Created 14 Dec 2022, 13:59:01 UTC
Sent 14 Dec 2022, 13:59:30 UTC
Report deadline 13 Jan 2023, 13:59:30 UTC
Received 15 Dec 2022, 8:24:07 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1520317
Run time 11 hours 37 min
CPU time 11 hours 3 min 32 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 8.61 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.07
x86_64-pc-linux-gnu
Peak working set size 5,051.95 MB
Peak swap size 5,930.81 MB
Peak disk usage 6,467.95 MB
Stderr 
<core_client_version>7.18.1</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
316:1671058396:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:138671:138671] [20221215:095316:1671058396:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:138671:138671] [20221215:095316:1671058396:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:138671:138671] [20221215:095316:1671058396:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:138671:138671] [20221215:095316:1671058396:0.000] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:roo:1:1:138671:138671] [20221215:095316:1671058396:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:138671:138671] [20221215:095316:1671058396:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:138671:138671] [20221215:095316:1671058396:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:138671:138671] [20221215:095316:1671058396:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:138671:138671] [20221215:095316:1671058396:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:138671:138671] [20221215:095316:1671058396:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:138671:138671] [20221215:095316:1671058396:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:138671:138671] [20221215:095316:1671058396:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:138671:138671] [20221215:095316:1671058396:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:138671:138671] [20221215:095316:1671058396:0.000] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  09:54:01 STEP  720 H= 180:00 +CPU= 35.729
  09:54:37 STEP  721 H= 180:15 +CPU= 34.167
  09:55:15 STEP  722 H= 180:30 +CPU= 35.425
  09:55:52 STEP  723 H= 180:45 +CPU= 35.351
  09:56:40 STEP  724 H= 181:00 +CPU= 44.564
  09:57:24 STEP  725 H= 181:15 +CPU= 41.528
  09:58:04 STEP  726 H= 181:30 +CPU= 38.635
  09:58:43 STEP  727 H= 181:45 +CPU= 36.689
  09:59:26 STEP  728 H= 182:00 +CPU= 39.989
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:06:15 STEP  729 H= 182:15 +CPU= 40.716
  10:06:51 STEP  730 H= 182:30 +CPU= 35.051
  10:07:29 STEP  731 H= 182:45 +CPU= 36.674
  10:08:22 STEP  732 H= 183:00 +CPU= 50.929
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+000732
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+000732
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+000732
  10:09:11 STEP  733 H= 183:15 +CPU= 45.907
  10:09:55 STEP  734 H= 183:30 +CPU= 40.715
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:46:18 STEP  735 H= 183:45 +CPU= 43.629
  10:47:08 STEP  736 H= 184:00 +CPU= 43.327
  10:47:50 STEP  737 H= 184:15 +CPU= 37.192
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:09:25 STEP  738 H= 184:30 +CPU= 36.727
  11:10:10 STEP  739 H= 184:45 +CPU= 41.229
  11:10:50 STEP  740 H= 185:00 +CPU= 37.643
  11:11:35 STEP  741 H= 185:15 +CPU= 40.695
  11:12:18 STEP  742 H= 185:30 +CPU= 40.216
  11:13:04 STEP  743 H= 185:45 +CPU= 41.813
  11:13:53 STEP  744 H= 186:00 +CPU= 48.962
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+000744
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+000744
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+000744
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:29:36 STEP  745 H= 186:15 +CPU= 44.708
  11:30:24 STEP  746 H= 186:30 +CPU= 41.706
  11:31:15 STEP  747 H= 186:45 +CPU= 44.279
  11:32:04 STEP  748 H= 187:00 +CPU= 43.176
  11:32:54 STEP  749 H= 187:15 +CPU= 43.584
  11:33:45 STEP  750 H= 187:30 +CPU= 45.126
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:43:23 STEP  751 H= 187:45 +CPU= 43.601
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:54:19 STEP  752 H= 188:00 +CPU= 40.116
  12:54:59 STEP  753 H= 188:15 +CPU= 36.194
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:14:34 STEP  754 H= 188:30 +CPU= 36.563
  13:15:18 STEP  755 H= 188:45 +CPU= 41.210
  13:16:15 STEP  756 H= 189:00 +CPU= 52.219
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+000756
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+000756
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+000756
  13:16:58 STEP  757 H= 189:15 +CPU= 39.412
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:20:19 STEP  758 H= 189:30 +CPU= 34.309
  13:20:59 STEP  759 H= 189:45 +CPU= 37.874
  13:21:41 STEP  760 H= 190:00 +CPU= 38.382
  13:22:16 STEP  761 H= 190:15 +CPU= 33.128
  13:22:57 STEP  762 H= 190:30 +CPU= 36.278
  13:23:39 STEP  763 H= 190:45 +CPU= 38.721
  13:24:17 STEP  764 H= 191:00 +CPU= 35.336
  13:24:59 STEP  765 H= 191:15 +CPU= 38.152
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:36:12 STEP  766 H= 191:30 +CPU= 30.915
  13:36:50 STEP  767 H= 191:45 +CPU= 36.216
  13:37:39 STEP  768 H= 192:00 +CPU= 45.737
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+000768
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+000768
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMGGh7zg+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMSHh7zg+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMUAh7zg+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMGGh7zg+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMSHh7zg+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMUAh7zg+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMGGh7zg+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMSHh7zg+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMUAh7zg+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMGGh7zg+000708
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMSHh7zg+000708
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMUAh7zg+000708
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMGGh7zg+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMSHh7zg+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMUAh7zg+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMGGh7zg+000732
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMSHh7zg+000732
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMUAh7zg+000732
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMGGh7zg+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMSHh7zg+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMUAh7zg+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMGGh7zg+000756
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMSHh7zg+000756
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMUAh7zg+000756
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a00b_2016092300_15_949_12166405_0_r978999605_7.zip
Uploading the intermediate file: upload_file_7.zip
  13:38:17 STEP  769 H= 192:15 +CPU= 35.699
  13:38:55 STEP  770 H= 192:30 +CPU= 35.084
  13:39:34 STEP  771 H= 192:45 +CPU= 36.217
  13:40:11 STEP  772 H= 193:00 +CPU= 34.124
  13:40:47 STEP  773 H= 193:15 +CPU= 33.573
  13:41:13 STEP  774 H= 193:30 +CPU= 25.324
  13:41:45 STEP  775 H= 193:45 +CPU= 30.272
  13:42:17 STEP  776 H= 194:00 +CPU= 29.839
  13:42:53 STEP  777 H= 194:15 +CPU= 32.484
  13:43:28 STEP  778 H= 194:30 +CPU= 34.056
  13:44:03 STEP  779 H= 194:45 +CPU= 32.140
  13:44:46 STEP  780 H= 195:00 +CPU= 39.672
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+000780
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+000780
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+000780
  13:45:21 STEP  781 H= 195:15 +CPU= 32.136
  13:45:58 STEP  782 H= 195:30 +CPU= 34.252
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:48:33 STEP  783 H= 195:45 +CPU= 32.013
  13:49:07 STEP  784 H= 196:00 +CPU= 31.754
  13:49:39 STEP  785 H= 196:15 +CPU= 29.942
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:52:08 STEP  786 H= 196:30 +CPU= 26.348
  13:52:41 STEP  787 H= 196:45 +CPU= 30.416
  13:53:12 STEP  788 H= 197:00 +CPU= 29.521
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:56:18 STEP  789 H= 197:15 +CPU= 26.405
  13:56:47 STEP  790 H= 197:30 +CPU= 26.951
  13:57:16 STEP  791 H= 197:45 +CPU= 26.610
  13:57:50 STEP  792 H= 198:00 +CPU= 32.581
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+000792
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+000792
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+000792
  13:58:21 STEP  793 H= 198:15 +CPU= 28.319
  13:58:45 STEP  794 H= 198:30 +CPU= 23.152
  13:59:11 STEP  795 H= 198:45 +CPU= 24.891
  13:59:37 STEP  796 H= 199:00 +CPU= 25.211
  14:00:04 STEP  797 H= 199:15 +CPU= 24.912
  14:00:30 STEP  798 H= 199:30 +CPU= 24.166
  14:00:55 STEP  799 H= 199:45 +CPU= 24.380
  14:01:18 STEP  800 H= 200:00 +CPU= 22.921
  14:01:48 STEP  801 H= 200:15 +CPU= 26.883
  14:02:13 STEP  802 H= 200:30 +CPU= 24.244
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:04:37 STEP  803 H= 200:45 +CPU= 20.277
  14:05:11 STEP  804 H= 201:00 +CPU= 30.593
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+000804
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+000804
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+000804
  14:05:37 STEP  805 H= 201:15 +CPU= 23.248
  14:06:01 STEP  806 H= 201:30 +CPU= 23.324
  14:06:26 STEP  807 H= 201:45 +CPU= 23.859
  14:06:52 STEP  808 H= 202:00 +CPU= 24.545
  14:07:17 STEP  809 H= 202:15 +CPU= 23.581
  14:07:40 STEP  810 H= 202:30 +CPU= 22.849
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:19:39 STEP  811 H= 202:45 +CPU= 21.813
  14:20:03 STEP  812 H= 203:00 +CPU= 23.308
  14:20:30 STEP  813 H= 203:15 +CPU= 25.257
  14:20:55 STEP  814 H= 203:30 +CPU= 24.090
  14:21:22 STEP  815 H= 203:45 +CPU= 24.720
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:23:57 STEP  816 H= 204:00 +CPU= 32.407
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+000816
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+000816
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+000816
  14:24:21 STEP  817 H= 204:15 +CPU= 22.775
  14:24:47 STEP  818 H= 204:30 +CPU= 24.236
  14:25:14 STEP  819 H= 204:45 +CPU= 25.524
  14:25:38 STEP  820 H= 205:00 +CPU= 24.192
  14:26:05 STEP  821 H= 205:15 +CPU= 24.485
  14:26:30 STEP  822 H= 205:30 +CPU= 24.102
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:28:54 STEP  823 H= 205:45 +CPU= 22.126
  14:29:19 STEP  824 H= 206:00 +CPU= 24.609
  14:29:43 STEP  825 H= 206:15 +CPU= 22.524
  14:30:07 STEP  826 H= 206:30 +CPU= 22.697
  14:30:31 STEP  827 H= 206:45 +CPU= 22.468
  14:31:05 STEP  828 H= 207:00 +CPU= 31.213
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+000828
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+000828
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+000828
  14:31:26 STEP  829 H= 207:15 +CPU= 20.413
  14:31:47 STEP  830 H= 207:30 +CPU= 20.379
  14:32:08 STEP  831 H= 207:45 +CPU= 19.817
  14:32:30 STEP  832 H= 208:00 +CPU= 20.720
  14:32:53 STEP  833 H= 208:15 +CPU= 21.866
  14:33:17 STEP  834 H= 208:30 +CPU= 22.303
  14:33:39 STEP  835 H= 208:45 +CPU= 20.492
  14:34:02 STEP  836 H= 209:00 +CPU= 21.942
  14:34:25 STEP  837 H= 209:15 +CPU= 20.520
  14:34:46 STEP  838 H= 209:30 +CPU= 20.412
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  14:54:11 STEP  839 H= 209:45 +CPU= 20.241
  14:54:39 STEP  840 H= 210:00 +CPU= 26.177
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+000840
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+000840
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+000840
  14:55:01 STEP  841 H= 210:15 +CPU= 20.989
  14:55:23 STEP  842 H= 210:30 +CPU= 20.978
  14:55:45 STEP  843 H= 210:45 +CPU= 21.127
  14:56:07 STEP  844 H= 211:00 +CPU= 20.596
  14:56:28 STEP  845 H= 211:15 +CPU= 20.311
  14:56:52 STEP  846 H= 211:30 +CPU= 21.099
  14:57:14 STEP  847 H= 211:45 +CPU= 20.211
  14:57:37 STEP  848 H= 212:00 +CPU= 21.840
  14:57:59 STEP  849 H= 212:15 +CPU= 20.224
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:00:20 STEP  850 H= 212:30 +CPU= 20.081
  15:00:43 STEP  851 H= 212:45 +CPU= 21.630
  15:01:09 STEP  852 H= 213:00 +CPU= 23.835
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+000852
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+000852
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+000852
  15:01:32 STEP  853 H= 213:15 +CPU= 21.293
  15:01:54 STEP  854 H= 213:30 +CPU= 21.160
  15:02:17 STEP  855 H= 213:45 +CPU= 20.950
  15:02:40 STEP  856 H= 214:00 +CPU= 21.039
  15:03:01 STEP  857 H= 214:15 +CPU= 20.035
  15:03:22 STEP  858 H= 214:30 +CPU= 19.877
  15:03:44 STEP  859 H= 214:45 +CPU= 20.992
  15:04:07 STEP  860 H= 215:00 +CPU= 20.815
  15:04:30 STEP  861 H= 215:15 +CPU= 22.110
  15:04:54 STEP  862 H= 215:30 +CPU= 22.509
  15:05:15 STEP  863 H= 215:45 +CPU= 20.756
  15:05:47 STEP  864 H= 216:00 +CPU= 29.444
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+000864
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+000864
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMGGh7zg+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMSHh7zg+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMUAh7zg+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMGGh7zg+000780
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMSHh7zg+000780
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMUAh7zg+000780
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMGGh7zg+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMSHh7zg+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMUAh7zg+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMGGh7zg+000804
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMSHh7zg+000804
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMUAh7zg+000804
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMGGh7zg+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMSHh7zg+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMUAh7zg+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMGGh7zg+000828
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMSHh7zg+000828
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMUAh7zg+000828
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMGGh7zg+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMSHh7zg+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMUAh7zg+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMGGh7zg+000852
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMSHh7zg+000852
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMUAh7zg+000852
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a00b_2016092300_15_949_12166405_0_r978999605_8.zip
Uploading the intermediate file: upload_file_8.zip
  15:06:09 STEP  865 H= 216:15 +CPU= 20.956
  15:06:30 STEP  866 H= 216:30 +CPU= 20.666
  15:06:49 STEP  867 H= 216:45 +CPU= 19.285
  15:07:12 STEP  868 H= 217:00 +CPU= 21.715
  15:07:34 STEP  869 H= 217:15 +CPU= 21.831
  15:07:57 STEP  870 H= 217:30 +CPU= 22.168
  15:08:19 STEP  871 H= 217:45 +CPU= 21.027
  15:08:40 STEP  872 H= 218:00 +CPU= 20.637
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:11:03 STEP  873 H= 218:15 +CPU= 21.267
  15:11:25 STEP  874 H= 218:30 +CPU= 21.211
  15:11:46 STEP  875 H= 218:45 +CPU= 20.789
  15:12:14 STEP  876 H= 219:00 +CPU= 27.549
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+000876
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+000876
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+000876
  15:12:37 STEP  877 H= 219:15 +CPU= 21.488
  15:12:57 STEP  878 H= 219:30 +CPU= 18.656
  15:13:19 STEP  879 H= 219:45 +CPU= 20.103
  15:13:41 STEP  880 H= 220:00 +CPU= 21.865
  15:14:03 STEP  881 H= 220:15 +CPU= 21.045
  15:14:25 STEP  882 H= 220:30 +CPU= 20.842
  15:14:47 STEP  883 H= 220:45 +CPU= 20.592
  15:15:10 STEP  884 H= 221:00 +CPU= 21.630
  15:15:31 STEP  885 H= 221:15 +CPU= 19.892
  15:15:52 STEP  886 H= 221:30 +CPU= 20.680
  15:16:16 STEP  887 H= 221:45 +CPU= 21.704
  15:16:45 STEP  888 H= 222:00 +CPU= 26.131
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+000888
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+000888
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+000888
  15:17:08 STEP  889 H= 222:15 +CPU= 21.653
  15:17:30 STEP  890 H= 222:30 +CPU= 21.104
  15:17:54 STEP  891 H= 222:45 +CPU= 22.897
  15:18:17 STEP  892 H= 223:00 +CPU= 21.685
  15:18:40 STEP  893 H= 223:15 +CPU= 21.205
  15:19:02 STEP  894 H= 223:30 +CPU= 20.977
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:21:25 STEP  895 H= 223:45 +CPU= 20.990
  15:21:46 STEP  896 H= 224:00 +CPU= 20.166
  15:22:08 STEP  897 H= 224:15 +CPU= 21.391
  15:22:32 STEP  898 H= 224:30 +CPU= 21.683
  15:22:54 STEP  899 H= 224:45 +CPU= 20.297
  15:23:20 STEP  900 H= 225:00 +CPU= 25.134
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+000900
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+000900
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+000900
  15:23:42 STEP  901 H= 225:15 +CPU= 21.104
  15:24:05 STEP  902 H= 225:30 +CPU= 20.905
  15:24:27 STEP  903 H= 225:45 +CPU= 20.487
  15:24:49 STEP  904 H= 226:00 +CPU= 20.756
  15:25:12 STEP  905 H= 226:15 +CPU= 21.084
  15:25:35 STEP  906 H= 226:30 +CPU= 21.101
  15:25:55 STEP  907 H= 226:45 +CPU= 19.354
  15:26:18 STEP  908 H= 227:00 +CPU= 21.711
  15:26:42 STEP  909 H= 227:15 +CPU= 22.671
  15:27:05 STEP  910 H= 227:30 +CPU= 20.857
  15:27:26 STEP  911 H= 227:45 +CPU= 19.876
  15:27:57 STEP  912 H= 228:00 +CPU= 29.327
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+000912
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+000912
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+000912
  15:28:18 STEP  913 H= 228:15 +CPU= 20.235
  15:28:37 STEP  914 H= 228:30 +CPU= 19.028
  15:28:59 STEP  915 H= 228:45 +CPU= 20.460
  15:29:21 STEP  916 H= 229:00 +CPU= 21.123
  15:29:43 STEP  917 H= 229:15 +CPU= 21.246
  15:30:10 STEP  918 H= 229:30 +CPU= 25.295
  15:30:37 STEP  919 H= 229:45 +CPU= 26.101
  15:31:05 STEP  920 H= 230:00 +CPU= 25.623
  15:31:34 STEP  921 H= 230:15 +CPU= 27.993
  15:32:04 STEP  922 H= 230:30 +CPU= 27.998
  15:32:35 STEP  923 H= 230:45 +CPU= 28.860
  15:33:15 STEP  924 H= 231:00 +CPU= 37.728
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+000924
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+000924
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+000924
  15:33:51 STEP  925 H= 231:15 +CPU= 33.536
  15:34:29 STEP  926 H= 231:30 +CPU= 35.649
  15:35:01 STEP  927 H= 231:45 +CPU= 31.313
  15:35:42 STEP  928 H= 232:00 +CPU= 39.093
  15:36:22 STEP  929 H= 232:15 +CPU= 37.751
  15:36:58 STEP  930 H= 232:30 +CPU= 34.084
  15:37:26 STEP  931 H= 232:45 +CPU= 27.684
  15:38:05 STEP  932 H= 233:00 +CPU= 36.977
  15:38:39 STEP  933 H= 233:15 +CPU= 33.107
  15:39:19 STEP  934 H= 233:30 +CPU= 36.597
  15:39:54 STEP  935 H= 233:45 +CPU= 33.020
  15:40:37 STEP  936 H= 234:00 +CPU= 40.344
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+000936
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+000936
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+000936
  15:41:13 STEP  937 H= 234:15 +CPU= 34.315
  15:41:53 STEP  938 H= 234:30 +CPU= 38.960
  15:42:24 STEP  939 H= 234:45 +CPU= 29.116
  15:43:04 STEP  940 H= 235:00 +CPU= 37.794
  15:43:43 STEP  941 H= 235:15 +CPU= 36.717
  15:44:23 STEP  942 H= 235:30 +CPU= 38.498
  15:45:01 STEP  943 H= 235:45 +CPU= 36.414
  15:45:41 STEP  944 H= 236:00 +CPU= 38.051
  15:46:20 STEP  945 H= 236:15 +CPU= 37.968
  15:47:00 STEP  946 H= 236:30 +CPU= 37.698
  15:47:41 STEP  947 H= 236:45 +CPU= 39.738
  15:48:26 STEP  948 H= 237:00 +CPU= 43.790
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+000948
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+000948
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+000948
  15:49:02 STEP  949 H= 237:15 +CPU= 34.448
  15:49:43 STEP  950 H= 237:30 +CPU= 38.478
  15:50:23 STEP  951 H= 237:45 +CPU= 37.690
  15:51:06 STEP  952 H= 238:00 +CPU= 40.031
  15:51:44 STEP  953 H= 238:15 +CPU= 36.941
  15:52:22 STEP  954 H= 238:30 +CPU= 36.933
  15:53:05 STEP  955 H= 238:45 +CPU= 39.798
  15:53:44 STEP  956 H= 239:00 +CPU= 36.306
  15:54:24 STEP  957 H= 239:15 +CPU= 38.018
  15:55:07 STEP  958 H= 239:30 +CPU= 39.852
  15:55:47 STEP  959 H= 239:45 +CPU= 37.216
  15:56:43 STEP  960 H= 240:00 +CPU= 52.015
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+000960
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+000960
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMGGh7zg+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMSHh7zg+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMUAh7zg+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMGGh7zg+000876
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMSHh7zg+000876
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMUAh7zg+000876
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMGGh7zg+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMSHh7zg+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMUAh7zg+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMGGh7zg+000900
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMSHh7zg+000900
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMUAh7zg+000900
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMGGh7zg+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMSHh7zg+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMUAh7zg+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMGGh7zg+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMSHh7zg+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMUAh7zg+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMGGh7zg+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMSHh7zg+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMUAh7zg+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMGGh7zg+000948
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMSHh7zg+000948
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMUAh7zg+000948
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a00b_2016092300_15_949_12166405_0_r978999605_9.zip
Uploading the intermediate file: upload_file_9.zip
  15:57:25 STEP  961 H= 240:15 +CPU= 39.623
  15:58:03 STEP  962 H= 240:30 +CPU= 36.446
  15:58:39 STEP  963 H= 240:45 +CPU= 35.181
  15:59:23 STEP  964 H= 241:00 +CPU= 41.462
  16:00:04 STEP  965 H= 241:15 +CPU= 38.195
  16:00:47 STEP  966 H= 241:30 +CPU= 40.562
  16:01:28 STEP  967 H= 241:45 +CPU= 37.714
  16:02:10 STEP  968 H= 242:00 +CPU= 39.497
  16:02:47 STEP  969 H= 242:15 +CPU= 35.083
  16:03:29 STEP  970 H= 242:30 +CPU= 38.959
  16:04:10 STEP  971 H= 242:45 +CPU= 38.709
  16:05:00 STEP  972 H= 243:00 +CPU= 47.261
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+000972
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+000972
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+000972
  16:05:37 STEP  973 H= 243:15 +CPU= 35.376
  16:06:14 STEP  974 H= 243:30 +CPU= 35.657
  16:06:50 STEP  975 H= 243:45 +CPU= 34.733
  16:07:27 STEP  976 H= 244:00 +CPU= 35.000
  16:08:07 STEP  977 H= 244:15 +CPU= 37.601
  16:08:49 STEP  978 H= 244:30 +CPU= 39.912
  16:09:26 STEP  979 H= 244:45 +CPU= 35.243
  16:10:08 STEP  980 H= 245:00 +CPU= 40.619
  16:10:45 STEP  981 H= 245:15 +CPU= 34.640
  16:11:19 STEP  982 H= 245:30 +CPU= 32.830
  16:12:00 STEP  983 H= 245:45 +CPU= 38.776
  16:12:48 STEP  984 H= 246:00 +CPU= 46.850
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+000984
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+000984
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+000984
  16:13:28 STEP  985 H= 246:15 +CPU= 38.875
  16:14:09 STEP  986 H= 246:30 +CPU= 39.897
  16:14:54 STEP  987 H= 246:45 +CPU= 42.325
  16:15:30 STEP  988 H= 247:00 +CPU= 35.955
  16:16:06 STEP  989 H= 247:15 +CPU= 34.493
  16:16:46 STEP  990 H= 247:30 +CPU= 38.867
  16:17:30 STEP  991 H= 247:45 +CPU= 41.270
  16:18:09 STEP  992 H= 248:00 +CPU= 37.230
  16:18:51 STEP  993 H= 248:15 +CPU= 41.222
  16:19:35 STEP  994 H= 248:30 +CPU= 40.898
  16:20:17 STEP  995 H= 248:45 +CPU= 40.717
  16:21:10 STEP  996 H= 249:00 +CPU= 50.349
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+000996
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+000996
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+000996
  16:21:55 STEP  997 H= 249:15 +CPU= 42.786
  16:22:40 STEP  998 H= 249:30 +CPU= 44.419
  16:23:29 STEP  999 H= 249:45 +CPU= 46.400
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.07_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a00b
(argv4) batchid: 949
(argv5) wuid: 12166405
(argv6) fclen: 15
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/15
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.07
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.07_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/15/oifs_43r3_bl_app_1.07_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/15/oifs_43r3_bl_app_1.07_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_0bd62498ff326d80d9efeda8f90805cd to: /var/lib/boinc-client/slots/15/oifs_43r3_bl_a00b_2016092300_15_949_12166405.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/15/oifs_43r3_bl_a00b_2016092300_15_949_12166405.zip
ic_ancil_file: ic_ancil_12166405
ifsdata_file: ifsdata_12166405
climate_data_file: clim_data_12166405
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/15/ic_ancil_12166405.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/15/ic_ancil_12166405.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_213b11bbe6aa441ae3eceb93d6b741ba to: /var/lib/boinc-client/slots/15/ifsdata/ifsdata_12166405.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/15/ifsdata/ifsdata_12166405.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/15/159l_2/clim_data_12166405.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/15/159l_2/clim_data_12166405.zip
Checking for progress XML file: /var/lib/boinc-client/slots/15/progress_file_12166405.xml
Opened progress file ok : /var/lib/boinc-client/slots/15/progress_file_12166405.xml
-- Model is restarting --
Adjusting last_iter, 1000, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/15/progress_file_12166405.xml
last_cpu_time: 35624
upload_file_number: 10
last_iter: 959
last_upload: 864000
model_completed: 0
total_length_of_simulation: 1296000
result_base_name: oifs_43r3_bl_a00b_2016092300_15_949_12166405_0_r978999605
The child process has been launched with process id: 186520
Executing the command: /var/lib/boinc-client/slots/15/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:roo:1:1:186520:186520] [20221215:162453:1671081893:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2091] fp = 0x30eaa80
[EC_DRHOOK:roo:1:1:186520:186520] [20221215:162453:1671081893:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:roo:1:1:186520:186520] [20221215:162453:1671081893:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:roo:1:1:186520:186520] [20221215:162453:1671081893:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:roo:1:1:186520:186520] [20221215:162453:1671081893:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:roo:1:1:186520:186520] [20221215:162453:1671081893:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:roo:1:1:186520:186520] [20221215:162453:1671081893:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:roo:1:1:186520:186520] [20221215:162453:1671081893:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:roo:1:1:186520:186520] [20221215:162453:1671081893:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:roo:1:1:186520:186520] [20221215:162453:1671081893:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:roo:1:1:186520:186520] [20221215:162453:1671081893:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:roo:1:1:186520:186520] [20221215:162453:1671081893:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:roo:1:1:186520:186520] [20221215:162453:1671081893:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:roo:1:1:186520:186520] [20221215:162453:1671081893:0.000] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:roo:1:1:186520:186520] [20221215:162453:1671081893:0.000] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:roo:1:1:186520:186520] [20221215:162453:1671081893:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:186520:186520] [20221215:162453:1671081893:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:186520:186520] [20221215:162453:1671081893:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:186520:186520] [20221215:162453:1671081893:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:186520:186520] [20221215:162453:1671081893:0.000] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:roo:1:1:186520:186520] [20221215:162453:1671081893:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:186520:186520] [20221215:162453:1671081893:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:186520:186520] [20221215:162453:1671081893:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:186520:186520] [20221215:162453:1671081893:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:186520:186520] [20221215:162453:1671081893:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:186520:186520] [20221215:162453:1671081893:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:186520:186520] [20221215:162453:1671081893:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:186520:186520] [20221215:162453:1671081893:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:186520:186520] [20221215:162453:1671081893:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:186520:186520] [20221215:162453:1671081893:0.000] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  16:25:51 STEP  960 H= 240:00 +CPU= 47.842
  16:26:31 STEP  961 H= 240:15 +CPU= 38.885
  16:27:13 STEP  962 H= 240:30 +CPU= 39.464
  16:27:57 STEP  963 H= 240:45 +CPU= 41.773
  16:28:45 STEP  964 H= 241:00 +CPU= 44.188
  16:29:30 STEP  965 H= 241:15 +CPU= 42.647
  16:30:13 STEP  966 H= 241:30 +CPU= 40.120
  16:30:57 STEP  967 H= 241:45 +CPU= 42.252
  16:31:41 STEP  968 H= 242:00 +CPU= 40.984
  16:32:27 STEP  969 H= 242:15 +CPU= 43.352
  16:33:14 STEP  970 H= 242:30 +CPU= 44.371
  16:34:03 STEP  971 H= 242:45 +CPU= 46.420
  16:35:06 STEP  972 H= 243:00 +CPU= 58.905
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+000972
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+000972
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+000972
  16:36:01 STEP  973 H= 243:15 +CPU= 51.251
  16:36:59 STEP  974 H= 243:30 +CPU= 54.145
  16:37:48 STEP  975 H= 243:45 +CPU= 47.175
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:48:14 STEP  976 H= 244:00 +CPU= 48.931
  16:49:08 STEP  977 H= 244:15 +CPU= 50.249
  16:50:03 STEP  978 H= 244:30 +CPU= 51.056
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:59:20 STEP  979 H= 244:45 +CPU= 51.347
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:12:47 STEP  980 H= 245:00 +CPU= 48.166
  17:13:49 STEP  981 H= 245:15 +CPU= 52.495
  17:14:49 STEP  982 H= 245:30 +CPU= 51.974
  17:15:47 STEP  983 H= 245:45 +CPU= 52.162
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:24:32 STEP  984 H= 246:00 +CPU= 57.119
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+000984
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+000984
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+000984
  17:27:49 STEP  985 H= 246:15 +CPU= 52.118
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:36:34 STEP  986 H= 246:30 +CPU= 36.116
  17:37:12 STEP  987 H= 246:45 +CPU= 32.792
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:49:45 STEP  988 H= 247:00 +CPU= 32.069
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:56:36 STEP  989 H= 247:15 +CPU= 29.889
  17:57:06 STEP  990 H= 247:30 +CPU= 29.128
  17:57:36 STEP  991 H= 247:45 +CPU= 28.771
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:00:07 STEP  992 H= 248:00 +CPU= 28.799
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:02:43 STEP  993 H= 248:15 +CPU= 29.943
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:06:30 STEP  994 H= 248:30 +CPU= 30.451
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:20:44 STEP  995 H= 248:45 +CPU= 32.340
  18:21:24 STEP  996 H= 249:00 +CPU= 37.192
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+000996
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+000996
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+000996
  18:22:01 STEP  997 H= 249:15 +CPU= 32.468
  18:22:37 STEP  998 H= 249:30 +CPU= 32.792
  18:23:12 STEP  999 H= 249:45 +CPU= 31.393
  18:23:47 STEP 1000 H= 250:00 +CPU= 32.523
  18:24:23 STEP 1001 H= 250:15 +CPU= 32.049
  18:25:03 STEP 1002 H= 250:30 +CPU= 36.921
  18:25:46 STEP 1003 H= 250:45 +CPU= 38.271
  18:26:26 STEP 1004 H= 251:00 +CPU= 36.729
  18:27:06 STEP 1005 H= 251:15 +CPU= 37.189
  18:27:46 STEP 1006 H= 251:30 +CPU= 37.203
  18:28:28 STEP 1007 H= 251:45 +CPU= 37.228
  18:29:28 STEP 1008 H= 252:00 +CPU= 51.449
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001008
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001008
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001008
  18:30:11 STEP 1009 H= 252:15 +CPU= 38.076
  18:30:54 STEP 1010 H= 252:30 +CPU= 38.583
  18:31:37 STEP 1011 H= 252:45 +CPU= 38.924
  18:32:18 STEP 1012 H= 253:00 +CPU= 37.256
  18:32:59 STEP 1013 H= 253:15 +CPU= 36.785
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  18:35:48 STEP 1014 H= 253:30 +CPU= 38.512
  18:36:30 STEP 1015 H= 253:45 +CPU= 38.105
  18:37:13 STEP 1016 H= 254:00 +CPU= 38.953
  18:37:56 STEP 1017 H= 254:15 +CPU= 39.126
  18:38:38 STEP 1018 H= 254:30 +CPU= 37.573
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.07_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a00b
(argv4) batchid: 949
(argv5) wuid: 12166405
(argv6) fclen: 15
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/15
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.07
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.07_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/15/oifs_43r3_bl_app_1.07_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/15/oifs_43r3_bl_app_1.07_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_0bd62498ff326d80d9efeda8f90805cd to: /var/lib/boinc-client/slots/15/oifs_43r3_bl_a00b_2016092300_15_949_12166405.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/15/oifs_43r3_bl_a00b_2016092300_15_949_12166405.zip
ic_ancil_file: ic_ancil_12166405
ifsdata_file: ifsdata_12166405
climate_data_file: clim_data_12166405
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/15/ic_ancil_12166405.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/15/ic_ancil_12166405.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_213b11bbe6aa441ae3eceb93d6b741ba to: /var/lib/boinc-client/slots/15/ifsdata/ifsdata_12166405.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/15/ifsdata/ifsdata_12166405.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/15/159l_2/clim_data_12166405.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/15/159l_2/clim_data_12166405.zip
Checking for progress XML file: /var/lib/boinc-client/slots/15/progress_file_12166405.xml
Opened progress file ok : /var/lib/boinc-client/slots/15/progress_file_12166405.xml
-- Model is restarting --
Adjusting last_iter, 1019, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/15/progress_file_12166405.xml
last_cpu_time: 38060
upload_file_number: 10
last_iter: 1007
last_upload: 864000
model_completed: 0
total_length_of_simulation: 1296000
result_base_name: oifs_43r3_bl_a00b_2016092300_15_949_12166405_0_r978999605
The child process has been launched with process id: 218432
Executing the command: /var/lib/boinc-client/slots/15/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:roo:1:1:218432:218432] [20221215:184932:1671090572:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2091] fp = 0x30eaa80
[EC_DRHOOK:roo:1:1:218432:218432] [20221215:184932:1671090572:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:roo:1:1:218432:218432] [20221215:184932:1671090572:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:roo:1:1:218432:218432] [20221215:184932:1671090572:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:roo:1:1:218432:218432] [20221215:184932:1671090572:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:roo:1:1:218432:218432] [20221215:184932:1671090572:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:roo:1:1:218432:218432] [20221215:184932:1671090572:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:roo:1:1:218432:218432] [20221215:184932:1671090572:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:roo:1:1:218432:218432] [20221215:184932:1671090572:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:roo:1:1:218432:218432] [20221215:184932:1671090572:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:roo:1:1:218432:218432] [20221215:184932:1671090572:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:roo:1:1:218432:218432] [20221215:184932:1671090572:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:roo:1:1:218432:218432] [20221215:184932:1671090572:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:roo:1:1:218432:218432] [20221215:184932:1671090572:0.000] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:roo:1:1:218432:218432] [20221215:184932:1671090572:0.000] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:roo:1:1:218432:218432] [20221215:184932:1671090572:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:218432:218432] [20221215:184932:1671090572:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:218432:218432] [20221215:184932:1671090572:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:218432:218432] [20221215:184932:1671090572:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:218432:218432] [20221215:184932:1671090572:0.000] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:roo:1:1:218432:218432] [20221215:184932:1671090572:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:218432:218432] [20221215:184932:1671090572:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:218432:218432] [20221215:184932:1671090572:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:218432:218432] [20221215:184932:1671090572:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:218432:218432] [20221215:184932:1671090572:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:218432:218432] [20221215:184932:1671090572:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:218432:218432] [20221215:184932:1671090572:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:218432:218432] [20221215:184932:1671090572:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:218432:218432] [20221215:184932:1671090572:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:218432:218432] [20221215:184932:1671090572:0.000] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  18:50:25 STEP 1008 H= 252:00 +CPU= 40.989
  18:51:03 STEP 1009 H= 252:15 +CPU= 35.508
  18:51:45 STEP 1010 H= 252:30 +CPU= 39.154
  18:52:23 STEP 1011 H= 252:45 +CPU= 36.414
  18:53:09 STEP 1012 H= 253:00 +CPU= 42.371
  18:53:45 STEP 1013 H= 253:15 +CPU= 33.695
  18:54:26 STEP 1014 H= 253:30 +CPU= 37.822
  18:55:08 STEP 1015 H= 253:45 +CPU= 39.126
  18:55:44 STEP 1016 H= 254:00 +CPU= 33.021
  18:56:24 STEP 1017 H= 254:15 +CPU= 38.198
  18:57:09 STEP 1018 H= 254:30 +CPU= 39.661
  18:57:48 STEP 1019 H= 254:45 +CPU= 35.794
  18:58:35 STEP 1020 H= 255:00 +CPU= 43.807
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001020
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001020
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001020
  18:59:16 STEP 1021 H= 255:15 +CPU= 37.534
  19:00:00 STEP 1022 H= 255:30 +CPU= 41.516
  19:00:48 STEP 1023 H= 255:45 +CPU= 43.711
  19:01:31 STEP 1024 H= 256:00 +CPU= 38.856
  19:02:13 STEP 1025 H= 256:15 +CPU= 38.754
  19:02:52 STEP 1026 H= 256:30 +CPU= 36.763
  19:03:30 STEP 1027 H= 256:45 +CPU= 36.007
  19:04:08 STEP 1028 H= 257:00 +CPU= 34.265
  19:04:50 STEP 1029 H= 257:15 +CPU= 38.856
  19:05:24 STEP 1030 H= 257:30 +CPU= 31.852
  19:06:01 STEP 1031 H= 257:45 +CPU= 34.652
  19:06:41 STEP 1032 H= 258:00 +CPU= 37.815
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001032
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001032
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001032
  19:07:13 STEP 1033 H= 258:15 +CPU= 31.626
  19:07:52 STEP 1034 H= 258:30 +CPU= 36.204
  19:08:23 STEP 1035 H= 258:45 +CPU= 29.919
  19:08:47 STEP 1036 H= 259:00 +CPU= 23.435
  19:09:17 STEP 1037 H= 259:15 +CPU= 28.692
  19:09:52 STEP 1038 H= 259:30 +CPU= 33.469
  19:10:30 STEP 1039 H= 259:45 +CPU= 34.492
  19:11:03 STEP 1040 H= 260:00 +CPU= 30.975
  19:11:38 STEP 1041 H= 260:15 +CPU= 33.120
  19:12:15 STEP 1042 H= 260:30 +CPU= 34.399
  19:12:48 STEP 1043 H= 260:45 +CPU= 31.935
  19:13:30 STEP 1044 H= 261:00 +CPU= 40.032
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001044
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001044
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001044
  19:14:04 STEP 1045 H= 261:15 +CPU= 31.906
  19:14:33 STEP 1046 H= 261:30 +CPU= 26.490
  19:15:00 STEP 1047 H= 261:45 +CPU= 25.796
  19:15:27 STEP 1048 H= 262:00 +CPU= 24.928
  19:15:52 STEP 1049 H= 262:15 +CPU= 24.184
  19:16:15 STEP 1050 H= 262:30 +CPU= 22.566
  19:16:39 STEP 1051 H= 262:45 +CPU= 24.202
  19:17:02 STEP 1052 H= 263:00 +CPU= 21.713
  19:17:29 STEP 1053 H= 263:15 +CPU= 25.806
  19:17:56 STEP 1054 H= 263:30 +CPU= 26.293
  19:18:22 STEP 1055 H= 263:45 +CPU= 24.224
  19:18:59 STEP 1056 H= 264:00 +CPU= 37.425
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMGGh7zg+001056
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMSHh7zg+001056
Moving to projects directory: /var/lib/boinc-client/slots/15/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMGGh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMSHh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMUAh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMGGh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMSHh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMUAh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMGGh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMSHh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMUAh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMGGh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMSHh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMUAh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMGGh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMSHh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMUAh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMGGh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMSHh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMUAh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMGGh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMSHh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMUAh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMGGh7zg+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMSHh7zg+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166405/ICMUAh7zg+001044
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a00b_2016092300_15_949_12166405_0_r978999605_10.zip
Uploading the intermediate file: upload_file_10.zip
  19:19:27 STEP 1057 H= 264:15 +CPU= 26.126
  19:19:55 STEP 1058 H= 264:30 +CPU= 25.516
  19:20:21 STEP 1059 H= 264:45 +CPU= 25.154
Abort request received from BOINC client, ending the child process

</stderr_txt>
]]>
No trickles!


©2024 climateprediction.net