climateprediction.net home page
Task 22437560

Task 22437560

Name oifs_43r3_bl_a36f_2016092300_20_1017_12285946_0
Workunit 12285946
Created 7 Jun 2024, 16:48:02 UTC
Sent 8 Jun 2024, 0:30:56 UTC
Report deadline 7 Aug 2024, 0:30:56 UTC
Received 8 Jun 2024, 7:13:43 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 1536639
Run time 2 hours 20 min 12 sec
CPU time 2 hours 18 min 27 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 6.19 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,554.76 MB
Peak swap size 5,980.80 MB
Peak disk usage 1,268.78 MB
Stderr
<core_client_version>7.24.1</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
ting progress file: /var/lib/boinc/slots/13/progress_file_12285946.xml
last_cpu_time: 0
upload_file_number: 0
last_iter: 0
last_upload: 0
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a36f_2016092300_20_1017_12285946_0_r144752439
The child process has been launched with process id: 163865
Executing the command: /var/lib/boinc/slots/13/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:054947:1717818587:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:054947:1717818587:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:054947:1717818587:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:054947:1717818587:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:054947:1717818587:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:054947:1717818587:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:054947:1717818587:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:054947:1717818587:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:054947:1717818587:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:054947:1717818587:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:054947:1717818587:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:054947:1717818587:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:054947:1717818587:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:054947:1717818587:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:054947:1717818587:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:054947:1717818587:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:054947:1717818587:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:054947:1717818587:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:054947:1717818587:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:054947:1717818587:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:054947:1717818587:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:054947:1717818587:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:054947:1717818587:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:054947:1717818587:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:054947:1717818587:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:054947:1717818587:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:054947:1717818587:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:054947:1717818587:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:054947:1717818587:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:054947:1717818587:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:50:59 STEP    0 H=   0:00 +CPU= 24.250
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+000000
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+000000
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+000000
  05:51:18 STEP    1 H=   0:15 +CPU= 18.728
  05:51:40 STEP    2 H=   0:30 +CPU= 21.875
  05:52:00 STEP    3 H=   0:45 +CPU= 20.238
  05:52:20 STEP    4 H=   1:00 +CPU= 19.855
  05:52:41 STEP    5 H=   1:15 +CPU= 20.260
  05:53:00 STEP    6 H=   1:30 +CPU= 19.028
  05:53:19 STEP    7 H=   1:45 +CPU= 18.904
  05:53:40 STEP    8 H=   2:00 +CPU= 20.564
  05:54:00 STEP    9 H=   2:15 +CPU= 20.640
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  05:54:44 STEP   10 H=   2:30 +CPU= 22.854
  05:55:08 STEP   11 H=   2:45 +CPU= 23.626
  05:55:36 STEP   12 H=   3:00 +CPU= 27.782
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+000012
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+000012
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+000012
  05:55:58 STEP   13 H=   3:15 +CPU= 21.560
  05:56:22 STEP   14 H=   3:30 +CPU= 24.031
  05:56:45 STEP   15 H=   3:45 +CPU= 22.792
  05:57:09 STEP   16 H=   4:00 +CPU= 23.565
  05:57:32 STEP   17 H=   4:15 +CPU= 22.566
  05:57:56 STEP   18 H=   4:30 +CPU= 24.469
  05:58:20 STEP   19 H=   4:45 +CPU= 23.748
  05:58:45 STEP   20 H=   5:00 +CPU= 24.672
  05:59:07 STEP   21 H=   5:15 +CPU= 22.161
  05:59:29 STEP   22 H=   5:30 +CPU= 21.951
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:00:11 STEP   23 H=   5:45 +CPU= 20.569
  06:00:34 STEP   24 H=   6:00 +CPU= 22.960
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+000024
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+000024
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+000024
  06:00:55 STEP   25 H=   6:15 +CPU= 20.415
  06:01:14 STEP   26 H=   6:30 +CPU= 19.271
  06:01:35 STEP   27 H=   6:45 +CPU= 20.231
  06:01:55 STEP   28 H=   7:00 +CPU= 20.222
  06:02:16 STEP   29 H=   7:15 +CPU= 20.778
  06:02:38 STEP   30 H=   7:30 +CPU= 21.409
  06:02:58 STEP   31 H=   7:45 +CPU= 19.583
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:03:39 STEP   32 H=   8:00 +CPU= 20.739
  06:04:03 STEP   33 H=   8:15 +CPU= 23.029
  06:04:26 STEP   34 H=   8:30 +CPU= 22.875
  06:04:49 STEP   35 H=   8:45 +CPU= 22.668
  06:05:19 STEP   36 H=   9:00 +CPU= 30.181
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+000036
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+000036
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+000036
  06:05:42 STEP   37 H=   9:15 +CPU= 22.671
  06:06:07 STEP   38 H=   9:30 +CPU= 24.520
  06:06:31 STEP   39 H=   9:45 +CPU= 24.172
  06:06:54 STEP   40 H=  10:00 +CPU= 23.253
  06:07:20 STEP   41 H=  10:15 +CPU= 25.430
  06:07:44 STEP   42 H=  10:30 +CPU= 23.426
  06:08:06 STEP   43 H=  10:45 +CPU= 22.677
  06:08:31 STEP   44 H=  11:00 +CPU= 24.124
  06:08:55 STEP   45 H=  11:15 +CPU= 23.646
  06:09:18 STEP   46 H=  11:30 +CPU= 23.620
  06:09:43 STEP   47 H=  11:45 +CPU= 24.641
  06:10:14 STEP   48 H=  12:00 +CPU= 30.776
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+000048
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+000048
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+000048
  06:10:38 STEP   49 H=  12:15 +CPU= 23.260
  06:11:02 STEP   50 H=  12:30 +CPU= 23.984
  06:11:27 STEP   51 H=  12:45 +CPU= 24.447
  06:11:51 STEP   52 H=  13:00 +CPU= 24.329
  06:12:16 STEP   53 H=  13:15 +CPU= 24.438
  06:12:39 STEP   54 H=  13:30 +CPU= 23.588
  06:13:03 STEP   55 H=  13:45 +CPU= 22.915
  06:13:27 STEP   56 H=  14:00 +CPU= 24.600
  06:13:51 STEP   57 H=  14:15 +CPU= 22.998
  06:14:15 STEP   58 H=  14:30 +CPU= 24.464
  06:14:41 STEP   59 H=  14:45 +CPU= 25.643
  06:15:11 STEP   60 H=  15:00 +CPU= 29.840
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+000060
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+000060
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+000060
  06:15:36 STEP   61 H=  15:15 +CPU= 24.776
  06:16:00 STEP   62 H=  15:30 +CPU= 24.299
  06:16:26 STEP   63 H=  15:45 +CPU= 25.378
  06:16:50 STEP   64 H=  16:00 +CPU= 23.944
  06:17:14 STEP   65 H=  16:15 +CPU= 23.339
  06:17:38 STEP   66 H=  16:30 +CPU= 24.360
  06:18:02 STEP   67 H=  16:45 +CPU= 23.456
  06:18:25 STEP   68 H=  17:00 +CPU= 22.791
  06:18:48 STEP   69 H=  17:15 +CPU= 22.553
  06:19:12 STEP   70 H=  17:30 +CPU= 23.977
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:19:55 STEP   71 H=  17:45 +CPU= 21.865
  06:20:24 STEP   72 H=  18:00 +CPU= 28.660
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+000072
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+000072
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+000072
  06:20:47 STEP   73 H=  18:15 +CPU= 23.120
  06:21:11 STEP   74 H=  18:30 +CPU= 23.585
  06:21:37 STEP   75 H=  18:45 +CPU= 25.798
  06:22:01 STEP   76 H=  19:00 +CPU= 24.469
  06:22:26 STEP   77 H=  19:15 +CPU= 24.163
  06:22:49 STEP   78 H=  19:30 +CPU= 23.212
  06:23:12 STEP   79 H=  19:45 +CPU= 22.884
  06:23:36 STEP   80 H=  20:00 +CPU= 23.770
  06:23:59 STEP   81 H=  20:15 +CPU= 22.349
  06:24:22 STEP   82 H=  20:30 +CPU= 23.352
  06:24:45 STEP   83 H=  20:45 +CPU= 23.081
  06:25:15 STEP   84 H=  21:00 +CPU= 29.189
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+000084
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+000084
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+000084
  06:25:39 STEP   85 H=  21:15 +CPU= 24.244
  06:26:05 STEP   86 H=  21:30 +CPU= 25.278
  06:26:29 STEP   87 H=  21:45 +CPU= 23.622
  06:26:52 STEP   88 H=  22:00 +CPU= 23.464
  06:27:16 STEP   89 H=  22:15 +CPU= 23.391
  06:27:40 STEP   90 H=  22:30 +CPU= 24.237
  06:28:05 STEP   91 H=  22:45 +CPU= 24.630
  06:28:31 STEP   92 H=  23:00 +CPU= 25.379
  06:28:54 STEP   93 H=  23:15 +CPU= 23.268
  06:29:17 STEP   94 H=  23:30 +CPU= 23.164
  06:29:41 STEP   95 H=  23:45 +CPU= 23.798
  06:30:14 STEP   96 H=  24:00 +CPU= 32.341
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+000096
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+000096
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+000096
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMGGh7zg+000000
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMSHh7zg+000000
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMUAh7zg+000000
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMGGh7zg+000012
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMSHh7zg+000012
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMUAh7zg+000012
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMGGh7zg+000024
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMSHh7zg+000024
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMUAh7zg+000024
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMGGh7zg+000036
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMSHh7zg+000036
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMUAh7zg+000036
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMGGh7zg+000048
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMSHh7zg+000048
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMUAh7zg+000048
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMGGh7zg+000060
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMSHh7zg+000060
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMUAh7zg+000060
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMGGh7zg+000072
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMSHh7zg+000072
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMUAh7zg+000072
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMGGh7zg+000084
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMSHh7zg+000084
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMUAh7zg+000084
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a36f_2016092300_20_1017_12285946_0_r144752439_0.zip
Uploading the intermediate file: upload_file_0.zip
  06:30:38 STEP   97 H=  24:15 +CPU= 23.940
  06:31:02 STEP   98 H=  24:30 +CPU= 23.949
  06:31:26 STEP   99 H=  24:45 +CPU= 23.722
  06:31:51 STEP  100 H=  25:00 +CPU= 24.781
  06:32:16 STEP  101 H=  25:15 +CPU= 24.382
  06:32:39 STEP  102 H=  25:30 +CPU= 23.257
  06:33:04 STEP  103 H=  25:45 +CPU= 24.227
  06:33:28 STEP  104 H=  26:00 +CPU= 24.403
  06:33:52 STEP  105 H=  26:15 +CPU= 23.638
  06:34:17 STEP  106 H=  26:30 +CPU= 25.079
  06:34:42 STEP  107 H=  26:45 +CPU= 24.065
  06:35:12 STEP  108 H=  27:00 +CPU= 29.832
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+000108
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+000108
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+000108
  06:35:36 STEP  109 H=  27:15 +CPU= 24.654
  06:36:00 STEP  110 H=  27:30 +CPU= 23.341
  06:36:23 STEP  111 H=  27:45 +CPU= 23.023
  06:36:48 STEP  112 H=  28:00 +CPU= 24.712
  06:37:12 STEP  113 H=  28:15 +CPU= 24.083
  06:37:35 STEP  114 H=  28:30 +CPU= 22.443
  06:37:58 STEP  115 H=  28:45 +CPU= 22.871
  06:38:25 STEP  116 H=  29:00 +CPU= 26.361
  06:38:49 STEP  117 H=  29:15 +CPU= 24.425
  06:39:13 STEP  118 H=  29:30 +CPU= 23.528
  06:39:37 STEP  119 H=  29:45 +CPU= 23.785
  06:40:07 STEP  120 H=  30:00 +CPU= 29.859
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+000120
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+000120
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+000120
  06:40:32 STEP  121 H=  30:15 +CPU= 25.042
  06:40:55 STEP  122 H=  30:30 +CPU= 22.651
  06:41:20 STEP  123 H=  30:45 +CPU= 25.019
  06:41:45 STEP  124 H=  31:00 +CPU= 25.011
  06:42:10 STEP  125 H=  31:15 +CPU= 24.522
  06:42:33 STEP  126 H=  31:30 +CPU= 22.930
  06:42:57 STEP  127 H=  31:45 +CPU= 23.787
  06:43:22 STEP  128 H=  32:00 +CPU= 24.648
  06:43:47 STEP  129 H=  32:15 +CPU= 25.386
  06:44:12 STEP  130 H=  32:30 +CPU= 24.130
  06:44:36 STEP  131 H=  32:45 +CPU= 23.880
  06:45:05 STEP  132 H=  33:00 +CPU= 29.206
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+000132
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+000132
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+000132
  06:45:29 STEP  133 H=  33:15 +CPU= 23.740
  06:45:54 STEP  134 H=  33:30 +CPU= 24.972
  06:46:19 STEP  135 H=  33:45 +CPU= 24.331
  06:46:43 STEP  136 H=  34:00 +CPU= 23.439
  06:47:07 STEP  137 H=  34:15 +CPU= 23.897
  06:47:32 STEP  138 H=  34:30 +CPU= 24.661
  06:47:56 STEP  139 H=  34:45 +CPU= 23.836
  06:48:20 STEP  140 H=  35:00 +CPU= 24.213
  06:48:44 STEP  141 H=  35:15 +CPU= 23.833
  06:49:09 STEP  142 H=  35:30 +CPU= 24.506
  06:49:33 STEP  143 H=  35:45 +CPU= 24.598
  06:50:06 STEP  144 H=  36:00 +CPU= 32.332
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+000144
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+000144
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+000144
  06:50:31 STEP  145 H=  36:15 +CPU= 24.286
  06:50:56 STEP  146 H=  36:30 +CPU= 25.232
  06:51:20 STEP  147 H=  36:45 +CPU= 24.057
  06:51:46 STEP  148 H=  37:00 +CPU= 25.909
  06:52:10 STEP  149 H=  37:15 +CPU= 23.335
  06:52:34 STEP  150 H=  37:30 +CPU= 23.518
  06:52:57 STEP  151 H=  37:45 +CPU= 22.834
  06:53:21 STEP  152 H=  38:00 +CPU= 24.464
  06:53:46 STEP  153 H=  38:15 +CPU= 24.117
  06:54:09 STEP  154 H=  38:30 +CPU= 23.153
  06:54:34 STEP  155 H=  38:45 +CPU= 24.570
  06:55:03 STEP  156 H=  39:00 +CPU= 28.977
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+000156
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+000156
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+000156
  06:55:28 STEP  157 H=  39:15 +CPU= 24.968
  06:55:51 STEP  158 H=  39:30 +CPU= 23.082
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  06:56:35 STEP  159 H=  39:45 +CPU= 22.203
  06:57:00 STEP  160 H=  40:00 +CPU= 25.588
  06:57:25 STEP  161 H=  40:15 +CPU= 24.730
  06:57:51 STEP  162 H=  40:30 +CPU= 25.692
  06:58:16 STEP  163 H=  40:45 +CPU= 24.355
  06:58:42 STEP  164 H=  41:00 +CPU= 25.984
  06:59:06 STEP  165 H=  41:15 +CPU= 24.061
  06:59:31 STEP  166 H=  41:30 +CPU= 24.742
  06:59:55 STEP  167 H=  41:45 +CPU= 23.630
  07:00:26 STEP  168 H=  42:00 +CPU= 31.101
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+000168
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+000168
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+000168
  07:00:51 STEP  169 H=  42:15 +CPU= 24.882
  07:01:17 STEP  170 H=  42:30 +CPU= 25.268
  07:01:40 STEP  171 H=  42:45 +CPU= 22.788
  07:02:04 STEP  172 H=  43:00 +CPU= 24.465
  07:02:28 STEP  173 H=  43:15 +CPU= 23.761
  07:02:54 STEP  174 H=  43:30 +CPU= 25.182
  07:03:18 STEP  175 H=  43:45 +CPU= 24.232
  07:03:45 STEP  176 H=  44:00 +CPU= 26.520
  07:04:10 STEP  177 H=  44:15 +CPU= 24.779
  07:04:34 STEP  178 H=  44:30 +CPU= 23.700
  07:04:57 STEP  179 H=  44:45 +CPU= 23.643
  07:05:27 STEP  180 H=  45:00 +CPU= 29.577
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+000180
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+000180
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+000180
  07:05:53 STEP  181 H=  45:15 +CPU= 25.435
  07:06:18 STEP  182 H=  45:30 +CPU= 25.132
  07:06:44 STEP  183 H=  45:45 +CPU= 26.130
  07:07:10 STEP  184 H=  46:00 +CPU= 25.034
  07:07:34 STEP  185 H=  46:15 +CPU= 24.493
  07:07:59 STEP  186 H=  46:30 +CPU= 24.212
  07:08:24 STEP  187 H=  46:45 +CPU= 24.590
  07:08:49 STEP  188 H=  47:00 +CPU= 25.525
  07:09:13 STEP  189 H=  47:15 +CPU= 23.703
  07:09:37 STEP  190 H=  47:30 +CPU= 23.704
  07:10:01 STEP  191 H=  47:45 +CPU= 23.861
  07:10:34 STEP  192 H=  48:00 +CPU= 32.074
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+000192
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+000192
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+000192
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMGGh7zg+000096
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMSHh7zg+000096
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMUAh7zg+000096
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMGGh7zg+000108
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMSHh7zg+000108
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMUAh7zg+000108
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMGGh7zg+000120
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMSHh7zg+000120
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMUAh7zg+000120
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMGGh7zg+000132
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMSHh7zg+000132
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMUAh7zg+000132
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMGGh7zg+000144
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMSHh7zg+000144
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMUAh7zg+000144
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMGGh7zg+000156
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMSHh7zg+000156
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMUAh7zg+000156
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMGGh7zg+000168
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMSHh7zg+000168
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMUAh7zg+000168
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMGGh7zg+000180
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMSHh7zg+000180
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMUAh7zg+000180
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a36f_2016092300_20_1017_12285946_0_r144752439_1.zip
Uploading the intermediate file: upload_file_1.zip
  07:10:59 STEP  193 H=  48:15 +CPU= 24.898
  07:11:23 STEP  194 H=  48:30 +CPU= 24.115
  07:11:49 STEP  195 H=  48:45 +CPU= 26.273
  07:12:14 STEP  196 H=  49:00 +CPU= 24.788
  07:12:39 STEP  197 H=  49:15 +CPU= 24.699
  07:13:04 STEP  198 H=  49:30 +CPU= 24.756
  07:13:28 STEP  199 H=  49:45 +CPU= 23.686
  07:13:54 STEP  200 H=  50:00 +CPU= 25.539
  07:14:18 STEP  201 H=  50:15 +CPU= 23.639
  07:14:42 STEP  202 H=  50:30 +CPU= 23.800
  07:15:05 STEP  203 H=  50:45 +CPU= 23.145
  07:15:35 STEP  204 H=  51:00 +CPU= 29.361
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+000204
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+000204
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+000204
  07:16:00 STEP  205 H=  51:15 +CPU= 24.901
  07:16:25 STEP  206 H=  51:30 +CPU= 25.512
  07:16:50 STEP  207 H=  51:45 +CPU= 24.955
  07:17:15 STEP  208 H=  52:00 +CPU= 24.073
  07:17:39 STEP  209 H=  52:15 +CPU= 24.546
  07:18:04 STEP  210 H=  52:30 +CPU= 24.063
  07:18:29 STEP  211 H=  52:45 +CPU= 25.155
  07:18:53 STEP  212 H=  53:00 +CPU= 23.816
  07:19:17 STEP  213 H=  53:15 +CPU= 23.501
  07:19:42 STEP  214 H=  53:30 +CPU= 25.245
  07:20:07 STEP  215 H=  53:45 +CPU= 25.046
  07:20:38 STEP  216 H=  54:00 +CPU= 30.365
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+000216
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+000216
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+000216
  07:21:03 STEP  217 H=  54:15 +CPU= 24.721
  07:21:27 STEP  218 H=  54:30 +CPU= 24.217
  07:21:55 STEP  219 H=  54:45 +CPU= 27.556
  07:22:21 STEP  220 H=  55:00 +CPU= 25.716
  07:22:45 STEP  221 H=  55:15 +CPU= 23.872
  07:23:10 STEP  222 H=  55:30 +CPU= 24.401
  07:23:34 STEP  223 H=  55:45 +CPU= 24.137
  07:24:00 STEP  224 H=  56:00 +CPU= 25.717
  07:24:23 STEP  225 H=  56:15 +CPU= 22.681
  07:24:48 STEP  226 H=  56:30 +CPU= 24.868
  07:25:12 STEP  227 H=  56:45 +CPU= 24.417
  07:25:42 STEP  228 H=  57:00 +CPU= 29.168
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+000228
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+000228
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+000228
  07:26:06 STEP  229 H=  57:15 +CPU= 24.173
  07:26:30 STEP  230 H=  57:30 +CPU= 23.745
  07:26:54 STEP  231 H=  57:45 +CPU= 23.792
  07:27:19 STEP  232 H=  58:00 +CPU= 24.768
  07:27:44 STEP  233 H=  58:15 +CPU= 24.742
  07:28:08 STEP  234 H=  58:30 +CPU= 23.900
  07:28:33 STEP  235 H=  58:45 +CPU= 24.989
  07:28:58 STEP  236 H=  59:00 +CPU= 24.527
  07:29:21 STEP  237 H=  59:15 +CPU= 23.194
  07:29:46 STEP  238 H=  59:30 +CPU= 24.649
  07:30:11 STEP  239 H=  59:45 +CPU= 25.212
  07:30:45 STEP  240 H=  60:00 +CPU= 33.093
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+000240
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+000240
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+000240
  07:31:11 STEP  241 H=  60:15 +CPU= 25.753
  07:31:35 STEP  242 H=  60:30 +CPU= 23.886
  07:32:00 STEP  243 H=  60:45 +CPU= 24.973
  07:32:25 STEP  244 H=  61:00 +CPU= 25.272
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:33:12 STEP  245 H=  61:15 +CPU= 23.980
  07:33:36 STEP  246 H=  61:30 +CPU= 23.973
  07:34:00 STEP  247 H=  61:45 +CPU= 23.931
  07:34:24 STEP  248 H=  62:00 +CPU= 24.369
  07:34:51 STEP  249 H=  62:15 +CPU= 26.107
  07:35:15 STEP  250 H=  62:30 +CPU= 24.451
  07:35:41 STEP  251 H=  62:45 +CPU= 25.119
  07:36:12 STEP  252 H=  63:00 +CPU= 31.378
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+000252
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+000252
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+000252
  07:36:36 STEP  253 H=  63:15 +CPU= 23.990
  07:37:00 STEP  254 H=  63:30 +CPU= 23.958
  07:37:25 STEP  255 H=  63:45 +CPU= 24.649
  07:37:51 STEP  256 H=  64:00 +CPU= 25.468
  07:38:16 STEP  257 H=  64:15 +CPU= 25.123
  07:38:42 STEP  258 H=  64:30 +CPU= 25.348
  07:39:07 STEP  259 H=  64:45 +CPU= 24.538
  07:39:31 STEP  260 H=  65:00 +CPU= 24.318
  07:39:55 STEP  261 H=  65:15 +CPU= 23.328
  07:40:20 STEP  262 H=  65:30 +CPU= 24.834
  07:40:44 STEP  263 H=  65:45 +CPU= 24.655
  07:41:17 STEP  264 H=  66:00 +CPU= 31.844
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+000264
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+000264
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+000264
  07:41:39 STEP  265 H=  66:15 +CPU= 22.773
  07:42:04 STEP  266 H=  66:30 +CPU= 24.546
  07:42:29 STEP  267 H=  66:45 +CPU= 24.719
  07:42:55 STEP  268 H=  67:00 +CPU= 25.719
  07:43:19 STEP  269 H=  67:15 +CPU= 24.277
  07:43:45 STEP  270 H=  67:30 +CPU= 25.575
  07:44:10 STEP  271 H=  67:45 +CPU= 25.073
  07:44:37 STEP  272 H=  68:00 +CPU= 26.284
  07:45:01 STEP  273 H=  68:15 +CPU= 24.193
  07:45:27 STEP  274 H=  68:30 +CPU= 25.658
  07:45:52 STEP  275 H=  68:45 +CPU= 25.032
  07:46:21 STEP  276 H=  69:00 +CPU= 28.430
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+000276
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+000276
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+000276
  07:46:47 STEP  277 H=  69:15 +CPU= 25.457
  07:47:12 STEP  278 H=  69:30 +CPU= 25.502
  07:47:37 STEP  279 H=  69:45 +CPU= 24.543
  07:48:02 STEP  280 H=  70:00 +CPU= 24.531
  07:48:27 STEP  281 H=  70:15 +CPU= 24.538
  07:48:51 STEP  282 H=  70:30 +CPU= 24.575
  07:49:16 STEP  283 H=  70:45 +CPU= 24.779
  07:49:43 STEP  284 H=  71:00 +CPU= 26.099
  07:50:07 STEP  285 H=  71:15 +CPU= 23.769
  07:50:33 STEP  286 H=  71:30 +CPU= 25.967
  07:50:57 STEP  287 H=  71:45 +CPU= 23.617
  07:51:30 STEP  288 H=  72:00 +CPU= 32.810
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+000288
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+000288
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+000288
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMGGh7zg+000192
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMSHh7zg+000192
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMUAh7zg+000192
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMGGh7zg+000204
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMSHh7zg+000204
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMUAh7zg+000204
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMGGh7zg+000216
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMSHh7zg+000216
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMUAh7zg+000216
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMGGh7zg+000228
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMSHh7zg+000228
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMUAh7zg+000228
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMGGh7zg+000240
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMSHh7zg+000240
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMUAh7zg+000240
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMGGh7zg+000252
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMSHh7zg+000252
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMUAh7zg+000252
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMGGh7zg+000264
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMSHh7zg+000264
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMUAh7zg+000264
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMGGh7zg+000276
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMSHh7zg+000276
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12285946/ICMUAh7zg+000276
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a36f_2016092300_20_1017_12285946_0_r144752439_2.zip
Uploading the intermediate file: upload_file_2.zip
  07:51:55 STEP  289 H=  72:15 +CPU= 25.048
  07:52:19 STEP  290 H=  72:30 +CPU= 23.801
  07:52:44 STEP  291 H=  72:45 +CPU= 24.665
  07:53:10 STEP  292 H=  73:00 +CPU= 25.860
  07:53:35 STEP  293 H=  73:15 +CPU= 25.193
  07:53:59 STEP  294 H=  73:30 +CPU= 24.098
  07:54:24 STEP  295 H=  73:45 +CPU= 24.114
  07:54:48 STEP  296 H=  74:00 +CPU= 24.288
  07:55:14 STEP  297 H=  74:15 +CPU= 25.161
  07:55:37 STEP  298 H=  74:30 +CPU= 23.701
  07:56:02 STEP  299 H=  74:45 +CPU= 24.456
  07:56:32 STEP  300 H=  75:00 +CPU= 30.089
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+000300
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+000300
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+000300
  07:56:59 STEP  301 H=  75:15 +CPU= 26.346
  07:57:25 STEP  302 H=  75:30 +CPU= 25.527
  07:57:48 STEP  303 H=  75:45 +CPU= 23.442
  07:58:15 STEP  304 H=  76:00 +CPU= 26.813
  07:58:40 STEP  305 H=  76:15 +CPU= 24.299
  07:59:04 STEP  306 H=  76:30 +CPU= 24.306
  07:59:30 STEP  307 H=  76:45 +CPU= 25.390
  07:59:55 STEP  308 H=  77:00 +CPU= 25.220
  08:00:20 STEP  309 H=  77:15 +CPU= 24.607
  08:00:45 STEP  310 H=  77:30 +CPU= 24.636
  08:01:08 STEP  311 H=  77:45 +CPU= 23.185
  08:01:39 STEP  312 H=  78:00 +CPU= 30.348
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+000312
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+000312
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+000312
  08:02:03 STEP  313 H=  78:15 +CPU= 23.594
  08:02:28 STEP  314 H=  78:30 +CPU= 24.945
  08:02:54 STEP  315 H=  78:45 +CPU= 25.593
  08:03:18 STEP  316 H=  79:00 +CPU= 24.610
  08:03:42 STEP  317 H=  79:15 +CPU= 23.591
  08:04:07 STEP  318 H=  79:30 +CPU= 24.355
  08:04:31 STEP  319 H=  79:45 +CPU= 24.334
  08:04:57 STEP  320 H=  80:00 +CPU= 25.016
  08:05:22 STEP  321 H=  80:15 +CPU= 25.288
  08:05:48 STEP  322 H=  80:30 +CPU= 25.867
  08:06:13 STEP  323 H=  80:45 +CPU= 24.513
  08:06:44 STEP  324 H=  81:00 +CPU= 31.335
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+000324
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+000324
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+000324
  08:07:09 STEP  325 H=  81:15 +CPU= 24.324
  08:07:33 STEP  326 H=  81:30 +CPU= 24.112
  08:07:58 STEP  327 H=  81:45 +CPU= 24.760
  08:08:24 STEP  328 H=  82:00 +CPU= 25.503
  08:08:48 STEP  329 H=  82:15 +CPU= 23.590
  08:09:13 STEP  330 H=  82:30 +CPU= 25.274
  08:09:39 STEP  331 H=  82:45 +CPU= 26.122
  08:10:04 STEP  332 H=  83:00 +CPU= 24.273
  08:10:29 STEP  333 H=  83:15 +CPU= 24.842
  08:10:54 STEP  334 H=  83:30 +CPU= 24.473
  08:11:18 STEP  335 H=  83:45 +CPU= 24.593
  08:11:52 STEP  336 H=  84:00 +CPU= 33.130
Moving to projects directory: /var/lib/boinc/slots/13/ICMGGh7zg+000336
Moving to projects directory: /var/lib/boinc/slots/13/ICMSHh7zg+000336
Moving to projects directory: /var/lib/boinc/slots/13/ICMUAh7zg+000336
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081208:1717827128:8540.920] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1538] Received signal#8 (SIGFPE) :: 4411MB (heap), 5555MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081208:1717827128:8540.920] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081208:1717827128:8540.920] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x820890 (old at (nil))
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081208:1717827128:8540.921] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1617] Signal#8 was caused by floating-point overflow [memaddr=0x1cc6692], nsigs = 1
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081208:1717827128:8540.921] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#8, nsigs = 1
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081208:1717827128:8540.921] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3843] 4411 MB (maxheap), 5555 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081208:1717827128:8540.921] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081208:1717827128:8540.921] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081208:1717827128:8540.921] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081208:1717827128:8540.921] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081208:1717827128:8540.921] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081208:1717827128:8540.921] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081208:1717827128:8540.921] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081208:1717827128:8540.921] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :         SCAN2M 
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081208:1717827128:8540.921] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :          GP_MODEL_HEAP 
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081208:1717827128:8540.921] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :           GP_MODEL 
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081208:1717827128:8540.921] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :            EC_PHYS_DRV 
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081208:1717827128:8540.921] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :             >OMP-PHYSICS              (1001) 
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081208:1717827128:8540.921] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :              EC_PHYS 
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081208:1717827128:8540.921] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :               CALLPAR 
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081208:1717827128:8540.921] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :                TURBULENCE_LAYER 
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081208:1717827128:8540.921] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :                 VDFOUTER 
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081208:1717827128:8540.921] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :                  VDFMAIN 
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081209:1717827129:8541.924] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:240] Backtrace(s) for program '/var/lib/boinc/slots/13/oifs_43r3_model.exe' : sigcontextptr=0x7ffd67315b10
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081209:1717827129:8541.924] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:277] Backtrace (size = 24) with addr2line-cmd
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081209:1717827129:8541.924] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:302] /usr/bin/addr2line -fs -e '/var/lib/boinc/slots/13/oifs_43r3_model.exe' 0x83a8f3 0x83aef9 0x1dd5490 0x1cc6692 0x121fd8b 0x1207e29 0x11fdcb2 0xfac003 0xf5fa6b 0xf4dbc4 0xbd7131 0x12c5002 0xbacbae 0x688153 0x4cd1dc 0x4b5d85 0x4b4f52 0x49d638 0x406fbd 0x401e7f 0x401e12 0x1dcc34a 0x1dcdbe7 0x401cd5
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081209:1717827129:8541.924] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [0]: [0x83a8f3] : ??() at ??:0
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081209:1717827129:8541.924] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [1]: [0x83aef9] : ??() at ??:0
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081209:1717827129:8541.924] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [2]: [0x1dd5490] : ??() at ??:0
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081209:1717827129:8541.924] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [3]: [0x1cc6692] : ??() at ??:0
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081209:1717827129:8541.924] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [4]: [0x121fd8b] : ??() at ??:0
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081209:1717827129:8541.924] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [5]: [0x1207e29] : ??() at ??:0
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081209:1717827129:8541.924] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [6]: [0x11fdcb2] : ??() at ??:0
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081209:1717827129:8541.924] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [7]: [0xfac003] : ??() at ??:0
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081209:1717827129:8541.924] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [8]: [0xf5fa6b] : ??() at ??:0
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081209:1717827129:8541.924] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [9]: [0xf4dbc4] : ??() at ??:0
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081209:1717827129:8541.924] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [10]: [0xbd7131] : ??() at ??:0
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081209:1717827129:8541.924] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [11]: [0x12c5002] : ??() at ??:0
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081209:1717827129:8541.924] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [12]: [0xbacbae] : ??() at ??:0
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081209:1717827129:8541.924] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [13]: [0x688153] : ??() at ??:0
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081209:1717827129:8541.924] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [14]: [0x4cd1dc] : ??() at ??:0
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081209:1717827129:8541.924] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [15]: [0x4b5d85] : ??() at ??:0
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081209:1717827129:8541.924] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [16]: [0x4b4f52] : ??() at ??:0
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081209:1717827129:8541.924] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [17]: [0x49d638] : ??() at ??:0
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081209:1717827129:8541.924] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [18]: [0x406fbd] : ??() at ??:0
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081209:1717827129:8541.924] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [19]: [0x401e7f] : ??() at ??:0
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081209:1717827129:8541.924] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [20]: [0x401e12] : ??() at ??:0
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081209:1717827129:8541.924] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [21]: [0x1dcc34a] : ??() at ??:0
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081209:1717827129:8541.924] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [22]: [0x1dcdbe7] : ??() at ??:0
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081209:1717827129:8541.924] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [23]: [0x401cd5] : ??() at ??:0
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081209:1717827129:8541.924] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:370] End of backtrace(s)
 Process            1  thread            1  calling linux_trbk from intel_trbk()
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:240] Backtrace(s) for program '/var/lib/boinc/slots/13/oifs_43r3_model.exe' : sigcontextptr=0x7ffd67315a20
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:277] Backtrace (size = 25) with addr2line-cmd
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:302] /usr/bin/addr2line -fs -e '/var/lib/boinc/slots/13/oifs_43r3_model.exe' 0x83a8f3 0x83aef9 0x822540 0x1dd5490 0x1cc6692 0x121fd8b 0x1207e29 0x11fdcb2 0xfac003 0xf5fa6b 0xf4dbc4 0xbd7131 0x12c5002 0xbacbae 0x688153 0x4cd1dc 0x4b5d85 0x4b4f52 0x49d638 0x406fbd 0x401e7f 0x401e12 0x1dcc34a 0x1dcdbe7 0x401cd5
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [0]: [0x83a8f3] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [1]: [0x83aef9] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [2]: [0x822540] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [3]: [0x1dd5490] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [4]: [0x1cc6692] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [5]: [0x121fd8b] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [6]: [0x1207e29] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [7]: [0x11fdcb2] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [8]: [0xfac003] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [9]: [0xf5fa6b] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [10]: [0xf4dbc4] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [11]: [0xbd7131] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [12]: [0x12c5002] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [13]: [0xbacbae] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [14]: [0x688153] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [15]: [0x4cd1dc] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [16]: [0x4b5d85] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [17]: [0x4b4f52] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [18]: [0x49d638] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [19]: [0x406fbd] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [20]: [0x401e7f] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [21]: [0x401e12] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [22]: [0x1dcc34a] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [23]: [0x1dcdbe7] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [24]: [0x401cd5] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:370] End of backtrace(s)
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081209:1717827129:8541.943] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#8, nsigs = 1
[EC_DRHOOK:Philip-Laptop:1:1:163865:163865] [20240608:081209:1717827129:8541.943] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1cf8da0 for signal#8, nsigs = 1
forrtl: error (72): floating overflow
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  0000000001CF935B  Unknown               Unknown  Unknown
oifs_43r3_model.e  00000000008214C1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DD5490  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001CC6692  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000121FD8B  vdfmain_                  874  vdfmain.F90
oifs_43r3_model.e  0000000001207E29  vdfouter_                 739  vdfouter.F90
oifs_43r3_model.e  00000000011FDCB2  turbulence_layer_         158  turbulence_layer.F90
oifs_43r3_model.e  0000000000FAC003  callpar_                  869  callpar.F90
oifs_43r3_model.e  0000000000F5FA6B  ec_phys_                  670  ec_phys.F90
oifs_43r3_model.e  0000000000F4DBC4  ec_phys_drv_              599  ec_phys_drv.F90
oifs_43r3_model.e  0000000000BD7131  gp_model_                 613  gp_model.F90
oifs_43r3_model.e  00000000012C5002  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  0000000000BACBAE  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  0000000000688153  stepo_                    327  stepo.F90
oifs_43r3_model.e  00000000004CD1DC  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  00000000004B5D85  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  00000000004B4F52  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  000000000049D638  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  0000000000406FBD  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  0000000000401E7F  MAIN__                     96  master.F90
oifs_43r3_model.e  0000000000401E12  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCC34A  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCDBE7  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000000401CD5  Unknown               Unknown  Unknown
..The child process has been killed with signal: 6
>>> Printing last 70 lines from file: NODE.001_01

 GPNORM  ICE WATER           AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.491097195065679E-05 0.000000000000000E+00 0.514735343563624E-02
 GPNORM SNOW                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.245697280527755E-04 0.000000000000000E+00 0.176182368112595E-01
 GPNORM RAIN                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.486398164160531E-05 0.000000000000000E+00 0.511647454297804E-02
 GPNORM  CLOUD FRACTION      AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.190293053885417E+00 0.000000000000000E+00 0.100000000000000E+01
 GPNORM CLW                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.218426157169619E-05 0.000000000000000E+00 0.438700654217957E-02
 GPNORM CIW                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.495876474661626E-05 0.000000000000000E+00 0.489238866155704E-02
 GPNORM CC                   AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.190228091160198E+00 0.000000000000000E+00 0.100000000000000E+01
 GPNORM RFL                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.180574249123884E-04 0.000000000000000E+00 0.182439165178756E-01
 GPNORM SFL                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.949524446935909E-05 0.000000000000000E+00 0.632500461876504E-02
 GPNORM CRFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.547766136335805E-05 0.000000000000000E+00 0.907078883820386E-02
 GPNORM CSFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.145644696527406E-05 0.000000000000000E+00 0.571132219189897E-02
 GPNORM PFRC                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.106744535543276E+00 0.000000000000000E+00 0.100000000000000E+01
 NSTEP =   336 STEPO   F00000000
 CREATE RESTART FILE ./srf00031200.0001                                                                                                                                                                                                                                              
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./srf00031200.0001 MODE=w
 IO-STREAM CLOSED - ./srf00031200.0001
 MPI-TASK:   1 - ************ BYTES IN     46 RECORDS TRANSFERRED IN    1.2477 SECONDS******** Mbytes/s, TOTAL TIME=    1.7240(72.4%)
 CREATE RESTART CONTROL FILE ./rcf                                                                                                                   
 WRRESF: DELETED PREVIOUS RESTART FILE OK: ./srf00030000.0001
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =   336 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+000336 MODE=w
 IO-STREAM CLOSED - ICMGGh7zg+000336
 MPI-TASK:   1 -   1012286932 BYTES IN      6 RECORDS TRANSFERRED IN    0.0002 SECONDS******** Mbytes/s, TOTAL TIME=    0.0076( 2.6%)
 NSTEP =   336 STEPO   0AA000000
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMSHh7zg+000336 MODE=w
 IO-STREAM CLOSED - ./ICMSHh7zg+000336
 MPI-TASK:   1 -    707399250 BYTES IN    202 RECORDS TRANSFERRED IN    0.0067 SECONDS105582.0 Mbytes/s, TOTAL TIME=    0.2342( 2.9%)
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000336 MODE=w
 IO-STREAM CLOSED - ./ICMUAh7zg+000336
 MPI-TASK:   1 -  16196590912 BYTES IN    125 RECORDS TRANSFERRED IN    0.0038 SECONDS******** Mbytes/s, TOTAL TIME=    0.1741( 2.2%)
 STEPO_FPOS WAS : B00000F00E
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000336 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000336
 MPI-TASK:   1 -           10 BYTES IN      2 RECORDS TRANSFERRED IN    0.0002 SECONDS     0.1 Mbytes/s, TOTAL TIME=    0.0036( 5.6%)
 STEPO_FPOS WAS : V00000000Y
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000336 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000336
 MPI-TASK:   1 -   1003707203 BYTES IN      4 RECORDS TRANSFERRED IN    0.0003 SECONDS******** Mbytes/s, TOTAL TIME=    0.0060( 5.0%)
 STEPO_FPOS WAS : T00000000M
 NSTEP =   336 STEPO   0AAA00AAA
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =   336 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+000336 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+000336
 MPI-TASK:   1 -   1012286932 BYTES IN      7 RECORDS TRANSFERRED IN    0.0005 SECONDS******** Mbytes/s, TOTAL TIME=    0.0118( 4.2%)
 08:11:52 STEP  336 H=  84:00 +CPU= 33.130
 NSTEP =   337 STEPO   0AAA00AAA
------------------------------------------------
CNT0 not found; string returned was: 'STEPO'
>>> Printing last 8 lines from file: ifs.stat

 08:11:19 F00000000 STEPO      336    24.764   24.764   24.952    137:38    141:31 0.27579240877612E-04       2GB       0MB
 08:11:21 0AA000000 STEPO      336     1.975    1.975    1.998    137:40    141:33 0.27579240877612E-04       2GB       0MB
 08:11:22 FULLPOS-B DYNFPOS    336     1.001    1.001    1.010    137:41    141:34 0.27579240877612E-04       2GB       0MB
 08:11:24 FULLPOS-V DYNFPOS    336     2.482    2.482    2.503    137:43    141:37 0.27579240877612E-04       2GB       0MB
 08:11:24 FULLPOS-T DYNFPOS    336     0.296    0.296    0.299    137:43    141:37 0.27579240877612E-04       2GB       0MB
 08:11:25 0AAA00AAA STEPO      336     0.341    0.341    0.345    137:44    141:37 0.27579240877612E-04       2GB       0MB
 08:11:52 0AAA00AAA STEPO      337    26.864   26.864   27.093    138:11    142:04 0.27579240877612E-04       2GB       0MB
------------------------------------------------
>>> Printing last 11 lines from file: rcf

 CSTEP   = "     336",
 CTIME   = "00031200      ",
 NSTEPLPP        =         201,         202,          49,      228226,      228227, 2*0,         324, 2*0,
 IPRGPNSRES      =           1,
 IPRGPEWRES      =           1,
 IPRTRWRES       =           1,
 IPRTRVRES       =           1,
 GMASS0  =  0.000000000000000E+000,
 GMASSI  =  0.000000000000000E+000
 /
------------------------------------------------
>>> Printing last 8 lines from file: /var/lib/boinc/slots/13/progress_file_12285946.xml

<running_values>
  <last_cpu_time>8298.340000</last_cpu_time>
  <upload_file_number>3</upload_file_number>
  <last_iter>337</last_iter>
  <last_upload>259200</last_upload>
  <model_completed>0</model_completed>
</running_values>
------------------------------------------------
..Failed, model did not complete successfully

</stderr_txt>
]]>
No trickles!


©2024 climateprediction.net