<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
:1:67063:67063] [20230114:214006:1673728806:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:67063:67063] [20230114:214006:1673728806:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:67063:67063] [20230114:214006:1673728806:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:67063:67063] [20230114:214006:1673728806:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:67063:67063] [20230114:214006:1673728806:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:67063:67063] [20230114:214006:1673728806:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:67063:67063] [20230114:214006:1673728806:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:67063:67063] [20230114:214006:1673728806:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:67063:67063] [20230114:214006:1673728806:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:67063:67063] [20230114:214006:1673728806:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:67063:67063] [20230114:214006:1673728806:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:67063:67063] [20230114:214006:1673728806:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:67063:67063] [20230114:214006:1673728806:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:67063:67063] [20230114:214006:1673728806:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:67063:67063] [20230114:214006:1673728806:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:67063:67063] [20230114:214006:1673728806:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:42:10 STEP 336 H= 336:00 +CPU= 38.077
21:42:31 STEP 337 H= 337:00 +CPU= 21.552
21:42:54 STEP 338 H= 338:00 +CPU= 22.010
21:43:30 STEP 339 H= 339:00 +CPU= 35.849
21:43:54 STEP 340 H= 340:00 +CPU= 24.149
21:44:19 STEP 341 H= 341:00 +CPU= 24.062
21:45:04 STEP 342 H= 342:00 +CPU= 44.587
21:45:28 STEP 343 H= 343:00 +CPU= 23.527
21:45:53 STEP 344 H= 344:00 +CPU= 25.115
21:46:37 STEP 345 H= 345:00 +CPU= 43.694
21:47:05 STEP 346 H= 346:00 +CPU= 27.154
21:47:30 STEP 347 H= 347:00 +CPU= 25.466
21:48:20 STEP 348 H= 348:00 +CPU= 44.404
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000348
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000348
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000348
21:48:45 STEP 349 H= 349:00 +CPU= 25.118
21:49:16 STEP 350 H= 350:00 +CPU= 29.948
21:49:56 STEP 351 H= 351:00 +CPU= 40.586
21:50:23 STEP 352 H= 352:00 +CPU= 26.509
21:50:53 STEP 353 H= 353:00 +CPU= 25.411
21:51:34 STEP 354 H= 354:00 +CPU= 40.865
21:52:00 STEP 355 H= 355:00 +CPU= 25.674
21:52:27 STEP 356 H= 356:00 +CPU= 27.271
21:53:17 STEP 357 H= 357:00 +CPU= 49.391
21:53:50 STEP 358 H= 358:00 +CPU= 32.506
21:54:33 STEP 359 H= 359:00 +CPU= 28.001
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMGGhq0f+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMSHhq0f+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMUAhq0f+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMGGhq0f+000348
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMSHhq0f+000348
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMUAhq0f+000348
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0499_2007050100_123_976_12193143_0_r555329025_2.zip
Uploading the intermediate file: upload_file_2.zip
21:55:51 STEP 360 H= 360:00 +CPU= 52.591
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000360
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000360
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000360
21:56:21 STEP 361 H= 361:00 +CPU= 27.028
21:56:46 STEP 362 H= 362:00 +CPU= 24.101
21:57:26 STEP 363 H= 363:00 +CPU= 39.578
21:57:51 STEP 364 H= 364:00 +CPU= 25.145
21:58:22 STEP 365 H= 365:00 +CPU= 31.289
21:59:12 STEP 366 H= 366:00 +CPU= 48.594
21:59:38 STEP 367 H= 367:00 +CPU= 26.321
22:00:12 STEP 368 H= 368:00 +CPU= 32.917
22:01:01 STEP 369 H= 369:00 +CPU= 49.075
22:01:36 STEP 370 H= 370:00 +CPU= 34.009
22:02:07 STEP 371 H= 371:00 +CPU= 30.773
22:02:59 STEP 372 H= 372:00 +CPU= 51.489
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000372
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000372
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000372
22:03:27 STEP 373 H= 373:00 +CPU= 27.594
22:03:57 STEP 374 H= 374:00 +CPU= 30.429
22:04:47 STEP 375 H= 375:00 +CPU= 49.670
22:05:20 STEP 376 H= 376:00 +CPU= 32.769
22:05:52 STEP 377 H= 377:00 +CPU= 31.660
22:06:40 STEP 378 H= 378:00 +CPU= 47.307
22:07:11 STEP 379 H= 379:00 +CPU= 31.027
22:07:44 STEP 380 H= 380:00 +CPU= 31.917
22:08:34 STEP 381 H= 381:00 +CPU= 50.163
22:09:07 STEP 382 H= 382:00 +CPU= 32.192
22:09:40 STEP 383 H= 383:00 +CPU= 32.378
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMGGhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMSHhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMUAhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMGGhq0f+000372
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMSHhq0f+000372
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMUAhq0f+000372
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0499_2007050100_123_976_12193143_0_r555329025_3.zip
Uploading the intermediate file: upload_file_3.zip
22:10:42 STEP 384 H= 384:00 +CPU= 56.855
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000384
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000384
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000384
22:11:10 STEP 385 H= 385:00 +CPU= 27.609
22:11:43 STEP 386 H= 386:00 +CPU= 32.554
22:12:33 STEP 387 H= 387:00 +CPU= 49.449
22:13:06 STEP 388 H= 388:00 +CPU= 32.482
22:13:38 STEP 389 H= 389:00 +CPU= 31.067
22:14:29 STEP 390 H= 390:00 +CPU= 51.141
22:14:56 STEP 391 H= 391:00 +CPU= 26.368
22:15:26 STEP 392 H= 392:00 +CPU= 29.889
22:16:17 STEP 393 H= 393:00 +CPU= 50.061
22:16:50 STEP 394 H= 394:00 +CPU= 32.101
22:17:21 STEP 395 H= 395:00 +CPU= 30.789
22:18:16 STEP 396 H= 396:00 +CPU= 54.230
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000396
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000396
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000396
22:18:41 STEP 397 H= 397:00 +CPU= 24.853
22:19:17 STEP 398 H= 398:00 +CPU= 33.451
22:20:07 STEP 399 H= 399:00 +CPU= 49.048
22:20:42 STEP 400 H= 400:00 +CPU= 33.769
22:21:13 STEP 401 H= 401:00 +CPU= 30.714
22:22:04 STEP 402 H= 402:00 +CPU= 50.708
22:22:35 STEP 403 H= 403:00 +CPU= 30.816
22:23:07 STEP 404 H= 404:00 +CPU= 32.149
22:23:59 STEP 405 H= 405:00 +CPU= 50.440
22:24:32 STEP 406 H= 406:00 +CPU= 32.664
22:25:05 STEP 407 H= 407:00 +CPU= 32.973
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMGGhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMSHhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMUAhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMGGhq0f+000396
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMSHhq0f+000396
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMUAhq0f+000396
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0499_2007050100_123_976_12193143_0_r555329025_4.zip
Uploading the intermediate file: upload_file_4.zip
22:25:55 STEP 408 H= 408:00 +CPU= 48.753
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000408
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000408
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000408
22:26:27 STEP 409 H= 409:00 +CPU= 31.865
22:26:59 STEP 410 H= 410:00 +CPU= 31.791
22:27:50 STEP 411 H= 411:00 +CPU= 49.880
22:28:22 STEP 412 H= 412:00 +CPU= 31.872
22:28:55 STEP 413 H= 413:00 +CPU= 32.195
22:29:46 STEP 414 H= 414:00 +CPU= 50.439
22:30:18 STEP 415 H= 415:00 +CPU= 32.280
22:30:52 STEP 416 H= 416:00 +CPU= 33.090
22:31:43 STEP 417 H= 417:00 +CPU= 50.643
22:32:15 STEP 418 H= 418:00 +CPU= 31.749
22:32:48 STEP 419 H= 419:00 +CPU= 33.021
22:33:44 STEP 420 H= 420:00 +CPU= 55.018
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000420
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000420
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000420
22:34:17 STEP 421 H= 421:00 +CPU= 32.577
22:34:50 STEP 422 H= 422:00 +CPU= 31.851
22:35:40 STEP 423 H= 423:00 +CPU= 50.139
22:36:12 STEP 424 H= 424:00 +CPU= 31.772
22:36:46 STEP 425 H= 425:00 +CPU= 31.915
22:37:37 STEP 426 H= 426:00 +CPU= 50.297
22:38:10 STEP 427 H= 427:00 +CPU= 32.800
22:38:43 STEP 428 H= 428:00 +CPU= 32.210
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2007050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0499
(argv4) batchid: 976
(argv5) wuid: 12193143
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/5
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/5/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/5/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_0e3198f9a4c114fec24a653ccfaa7663 to: /var/lib/boinc-client/slots/5/oifs_43r3_ps_0499_2007050100_123_976_12193143.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/5/oifs_43r3_ps_0499_2007050100_123_976_12193143.zip
ic_ancil_file: ic_ancil_12193143
ifsdata_file: ifsdata_12193143
climate_data_file: clim_data_12193143
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_7e60b00cc05949b2b3582335a1df4eb3 to: /var/lib/boinc-client/slots/5/ic_ancil_12193143.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/5/ic_ancil_12193143.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_e693e2bc99207f9f4e2a092cd51668d0 to: /var/lib/boinc-client/slots/5/ifsdata/ifsdata_12193143.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/5/ifsdata/ifsdata_12193143.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/5/159l_2/clim_data_12193143.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/5/159l_2/clim_data_12193143.zip
Checking for progress XML file: /var/lib/boinc-client/slots/5/progress_file_12193143.xml
Opened progress file ok : /var/lib/boinc-client/slots/5/progress_file_12193143.xml
-- Model is restarting --
Adjusting last_iter, 429, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/5/progress_file_12193143.xml
last_cpu_time: 4531
upload_file_number: 5
last_iter: 407
last_upload: 1465200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0499_2007050100_123_976_12193143_0_r555329025
The child process has been launched with process id: 70977
Executing the command: /var/lib/boinc-client/slots/5/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:70977:70977] [20230114:224032:1673732432:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:70977:70977] [20230114:224032:1673732432:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:70977:70977] [20230114:224032:1673732432:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:70977:70977] [20230114:224032:1673732432:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:70977:70977] [20230114:224032:1673732432:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:70977:70977] [20230114:224032:1673732432:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:70977:70977] [20230114:224032:1673732432:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:70977:70977] [20230114:224032:1673732432:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:70977:70977] [20230114:224032:1673732432:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:70977:70977] [20230114:224032:1673732432:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:70977:70977] [20230114:224032:1673732432:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:70977:70977] [20230114:224032:1673732432:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:70977:70977] [20230114:224032:1673732432:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:70977:70977] [20230114:224032:1673732432:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:70977:70977] [20230114:224032:1673732432:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:70977:70977] [20230114:224032:1673732432:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:70977:70977] [20230114:224032:1673732432:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:70977:70977] [20230114:224032:1673732432:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:70977:70977] [20230114:224032:1673732432:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:70977:70977] [20230114:224032:1673732432:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:70977:70977] [20230114:224032:1673732432:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:70977:70977] [20230114:224032:1673732432:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:70977:70977] [20230114:224032:1673732432:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:70977:70977] [20230114:224032:1673732432:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:70977:70977] [20230114:224032:1673732432:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:70977:70977] [20230114:224032:1673732432:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:70977:70977] [20230114:224032:1673732432:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:70977:70977] [20230114:224032:1673732432:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:70977:70977] [20230114:224032:1673732432:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:70977:70977] [20230114:224032:1673732432:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
22:41:51 STEP 408 H= 408:00 +CPU= 56.010
22:42:23 STEP 409 H= 409:00 +CPU= 32.142
22:42:56 STEP 410 H= 410:00 +CPU= 32.349
22:43:45 STEP 411 H= 411:00 +CPU= 49.167
22:44:18 STEP 412 H= 412:00 +CPU= 32.598
22:44:49 STEP 413 H= 413:00 +CPU= 30.783
22:45:42 STEP 414 H= 414:00 +CPU= 52.043
22:46:15 STEP 415 H= 415:00 +CPU= 31.379
22:46:49 STEP 416 H= 416:00 +CPU= 34.052
22:47:40 STEP 417 H= 417:00 +CPU= 49.809
22:48:13 STEP 418 H= 418:00 +CPU= 32.676
22:48:44 STEP 419 H= 419:00 +CPU= 30.902
22:49:40 STEP 420 H= 420:00 +CPU= 55.162
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000420
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000420
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000420
22:50:11 STEP 421 H= 421:00 +CPU= 30.673
22:50:44 STEP 422 H= 422:00 +CPU= 32.730
22:51:34 STEP 423 H= 423:00 +CPU= 50.211
22:52:07 STEP 424 H= 424:00 +CPU= 32.673
22:52:39 STEP 425 H= 425:00 +CPU= 31.109
22:53:31 STEP 426 H= 426:00 +CPU= 51.304
22:54:02 STEP 427 H= 427:00 +CPU= 31.186
22:54:35 STEP 428 H= 428:00 +CPU= 32.459
22:55:25 STEP 429 H= 429:00 +CPU= 49.695
22:55:57 STEP 430 H= 430:00 +CPU= 31.653
22:56:29 STEP 431 H= 431:00 +CPU= 31.991
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMGGhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMSHhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMUAhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMGGhq0f+000420
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMSHhq0f+000420
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMUAhq0f+000420
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0499_2007050100_123_976_12193143_0_r555329025_5.zip
Uploading the intermediate file: upload_file_5.zip
22:57:28 STEP 432 H= 432:00 +CPU= 58.284
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000432
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000432
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000432
22:58:00 STEP 433 H= 433:00 +CPU= 31.023
22:58:32 STEP 434 H= 434:00 +CPU= 32.139
22:59:22 STEP 435 H= 435:00 +CPU= 49.626
22:59:55 STEP 436 H= 436:00 +CPU= 32.237
23:00:27 STEP 437 H= 437:00 +CPU= 31.516
23:01:18 STEP 438 H= 438:00 +CPU= 50.563
23:01:49 STEP 439 H= 439:00 +CPU= 31.544
23:02:22 STEP 440 H= 440:00 +CPU= 32.088
23:03:12 STEP 441 H= 441:00 +CPU= 50.058
23:03:45 STEP 442 H= 442:00 +CPU= 32.410
23:04:16 STEP 443 H= 443:00 +CPU= 31.043
23:05:14 STEP 444 H= 444:00 +CPU= 56.056
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000444
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000444
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000444
23:05:45 STEP 445 H= 445:00 +CPU= 30.676
23:06:18 STEP 446 H= 446:00 +CPU= 32.670
23:07:08 STEP 447 H= 447:00 +CPU= 49.088
23:07:40 STEP 448 H= 448:00 +CPU= 32.061
23:08:11 STEP 449 H= 449:00 +CPU= 30.770
23:09:03 STEP 450 H= 450:00 +CPU= 51.730
23:09:35 STEP 451 H= 451:00 +CPU= 30.889
23:10:08 STEP 452 H= 452:00 +CPU= 32.748
23:10:57 STEP 453 H= 453:00 +CPU= 49.288
23:11:32 STEP 454 H= 454:00 +CPU= 34.139
23:12:04 STEP 455 H= 455:00 +CPU= 31.595
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMGGhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMSHhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMUAhq0f+000432
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMGGhq0f+000444
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMSHhq0f+000444
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMUAhq0f+000444
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0499_2007050100_123_976_12193143_0_r555329025_6.zip
Uploading the intermediate file: upload_file_6.zip
23:12:56 STEP 456 H= 456:00 +CPU= 51.109
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000456
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000456
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000456
23:13:28 STEP 457 H= 457:00 +CPU= 31.200
23:14:00 STEP 458 H= 458:00 +CPU= 31.500
23:14:50 STEP 459 H= 459:00 +CPU= 50.050
23:15:22 STEP 460 H= 460:00 +CPU= 31.684
23:15:54 STEP 461 H= 461:00 +CPU= 31.300
23:16:46 STEP 462 H= 462:00 +CPU= 51.462
23:17:18 STEP 463 H= 463:00 +CPU= 31.596
23:17:51 STEP 464 H= 464:00 +CPU= 32.448
23:18:42 STEP 465 H= 465:00 +CPU= 50.054
23:19:14 STEP 466 H= 466:00 +CPU= 31.754
23:19:45 STEP 467 H= 467:00 +CPU= 31.310
23:20:41 STEP 468 H= 468:00 +CPU= 54.861
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000468
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000468
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000468
23:21:13 STEP 469 H= 469:00 +CPU= 31.754
23:21:46 STEP 470 H= 470:00 +CPU= 32.662
23:22:35 STEP 471 H= 471:00 +CPU= 49.400
23:23:08 STEP 472 H= 472:00 +CPU= 32.631
23:23:40 STEP 473 H= 473:00 +CPU= 31.765
23:24:32 STEP 474 H= 474:00 +CPU= 51.226
23:25:04 STEP 475 H= 475:00 +CPU= 31.274
23:25:36 STEP 476 H= 476:00 +CPU= 32.046
23:26:27 STEP 477 H= 477:00 +CPU= 50.090
23:26:59 STEP 478 H= 478:00 +CPU= 32.044
23:27:32 STEP 479 H= 479:00 +CPU= 32.780
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMGGhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMSHhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMUAhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMGGhq0f+000468
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMSHhq0f+000468
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMUAhq0f+000468
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0499_2007050100_123_976_12193143_0_r555329025_7.zip
Uploading the intermediate file: upload_file_7.zip
23:28:31 STEP 480 H= 480:00 +CPU= 57.572
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000480
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000480
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000480
23:29:02 STEP 481 H= 481:00 +CPU= 30.926
23:29:35 STEP 482 H= 482:00 +CPU= 32.473
23:30:26 STEP 483 H= 483:00 +CPU= 50.121
23:30:58 STEP 484 H= 484:00 +CPU= 32.349
23:31:32 STEP 485 H= 485:00 +CPU= 32.678
23:32:24 STEP 486 H= 486:00 +CPU= 51.782
23:32:55 STEP 487 H= 487:00 +CPU= 30.718
23:33:28 STEP 488 H= 488:00 +CPU= 32.958
23:34:19 STEP 489 H= 489:00 +CPU= 49.936
23:34:51 STEP 490 H= 490:00 +CPU= 32.626
23:35:23 STEP 491 H= 491:00 +CPU= 31.587
23:36:19 STEP 492 H= 492:00 +CPU= 55.166
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000492
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000492
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000492
23:36:50 STEP 493 H= 493:00 +CPU= 30.700
23:37:23 STEP 494 H= 494:00 +CPU= 32.591
23:38:14 STEP 495 H= 495:00 +CPU= 50.198
23:38:46 STEP 496 H= 496:00 +CPU= 32.316
23:39:18 STEP 497 H= 497:00 +CPU= 31.675
23:40:10 STEP 498 H= 498:00 +CPU= 51.003
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2007050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0499
(argv4) batchid: 976
(argv5) wuid: 12193143
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/5
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/5/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/5/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_0e3198f9a4c114fec24a653ccfaa7663 to: /var/lib/boinc-client/slots/5/oifs_43r3_ps_0499_2007050100_123_976_12193143.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/5/oifs_43r3_ps_0499_2007050100_123_976_12193143.zip
ic_ancil_file: ic_ancil_12193143
ifsdata_file: ifsdata_12193143
climate_data_file: clim_data_12193143
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_7e60b00cc05949b2b3582335a1df4eb3 to: /var/lib/boinc-client/slots/5/ic_ancil_12193143.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/5/ic_ancil_12193143.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_e693e2bc99207f9f4e2a092cd51668d0 to: /var/lib/boinc-client/slots/5/ifsdata/ifsdata_12193143.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/5/ifsdata/ifsdata_12193143.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/5/159l_2/clim_data_12193143.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/5/159l_2/clim_data_12193143.zip
Checking for progress XML file: /var/lib/boinc-client/slots/5/progress_file_12193143.xml
Opened progress file ok : /var/lib/boinc-client/slots/5/progress_file_12193143.xml
-- Model is restarting --
Adjusting last_iter, 499, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/5/progress_file_12193143.xml
last_cpu_time: 8066
upload_file_number: 8
last_iter: 479
last_upload: 1724400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0499_2007050100_123_976_12193143_0_r555329025
The child process has been launched with process id: 74535
Executing the command: /var/lib/boinc-client/slots/5/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:74535:74535] [20230114:234143:1673736103:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:74535:74535] [20230114:234143:1673736103:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:74535:74535] [20230114:234143:1673736103:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:74535:74535] [20230114:234143:1673736103:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:74535:74535] [20230114:234143:1673736103:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:74535:74535] [20230114:234143:1673736103:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:74535:74535] [20230114:234143:1673736103:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:74535:74535] [20230114:234143:1673736103:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:74535:74535] [20230114:234143:1673736103:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:74535:74535] [20230114:234143:1673736103:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:74535:74535] [20230114:234143:1673736103:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:74535:74535] [20230114:234143:1673736103:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:74535:74535] [20230114:234143:1673736103:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:74535:74535] [20230114:234143:1673736103:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:74535:74535] [20230114:234143:1673736103:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:74535:74535] [20230114:234143:1673736103:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:74535:74535] [20230114:234143:1673736103:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:74535:74535] [20230114:234143:1673736103:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:74535:74535] [20230114:234143:1673736103:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:74535:74535] [20230114:234143:1673736103:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:74535:74535] [20230114:234143:1673736103:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:74535:74535] [20230114:234143:1673736103:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:74535:74535] [20230114:234143:1673736103:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:74535:74535] [20230114:234143:1673736103:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:74535:74535] [20230114:234143:1673736103:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:74535:74535] [20230114:234143:1673736103:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:74535:74535] [20230114:234143:1673736103:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:74535:74535] [20230114:234143:1673736103:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:74535:74535] [20230114:234143:1673736103:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:74535:74535] [20230114:234143:1673736103:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
23:42:54 STEP 480 H= 480:00 +CPU= 58.525
23:43:25 STEP 481 H= 481:00 +CPU= 31.147
23:43:58 STEP 482 H= 482:00 +CPU= 32.591
23:44:49 STEP 483 H= 483:00 +CPU= 50.221
23:45:21 STEP 484 H= 484:00 +CPU= 31.898
23:45:53 STEP 485 H= 485:00 +CPU= 31.845
23:46:44 STEP 486 H= 486:00 +CPU= 50.841
23:47:16 STEP 487 H= 487:00 +CPU= 31.648
23:47:48 STEP 488 H= 488:00 +CPU= 31.838
23:48:39 STEP 489 H= 489:00 +CPU= 49.898
23:49:11 STEP 490 H= 490:00 +CPU= 31.790
23:49:42 STEP 491 H= 491:00 +CPU= 31.102
23:50:39 STEP 492 H= 492:00 +CPU= 55.859
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000492
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000492
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000492
23:51:11 STEP 493 H= 493:00 +CPU= 31.792
23:51:42 STEP 494 H= 494:00 +CPU= 31.370
23:52:34 STEP 495 H= 495:00 +CPU= 50.834
23:53:04 STEP 496 H= 496:00 +CPU= 30.154
23:53:33 STEP 497 H= 497:00 +CPU= 29.017
23:54:24 STEP 498 H= 498:00 +CPU= 50.763
23:54:57 STEP 499 H= 499:00 +CPU= 31.725
23:55:30 STEP 500 H= 500:00 +CPU= 33.121
23:56:21 STEP 501 H= 501:00 +CPU= 50.519
23:56:55 STEP 502 H= 502:00 +CPU= 33.269
23:57:27 STEP 503 H= 503:00 +CPU= 31.407
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMGGhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMSHhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMUAhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMGGhq0f+000492
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMSHhq0f+000492
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMUAhq0f+000492
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0499_2007050100_123_976_12193143_0_r555329025_8.zip
Uploading the intermediate file: upload_file_8.zip
23:58:26 STEP 504 H= 504:00 +CPU= 58.046
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000504
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000504
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000504
23:58:57 STEP 505 H= 505:00 +CPU= 30.534
23:59:29 STEP 506 H= 506:00 +CPU= 31.468
00:00:19 STEP 507 H= 507:00 +CPU= 49.575
00:00:45 STEP 508 H= 508:00 +CPU= 25.619
00:01:12 STEP 509 H= 509:00 +CPU= 26.396
00:02:02 STEP 510 H= 510:00 +CPU= 49.902
00:02:33 STEP 511 H= 511:00 +CPU= 31.289
00:03:05 STEP 512 H= 512:00 +CPU= 31.678
00:03:55 STEP 513 H= 513:00 +CPU= 49.078
00:04:29 STEP 514 H= 514:00 +CPU= 30.313
00:05:01 STEP 515 H= 515:00 +CPU= 31.747
00:05:53 STEP 516 H= 516:00 +CPU= 49.395
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000516
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000516
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000516
00:06:16 STEP 517 H= 517:00 +CPU= 22.443
00:06:41 STEP 518 H= 518:00 +CPU= 25.439
00:07:28 STEP 519 H= 519:00 +CPU= 46.335
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:08:23 STEP 520 H= 520:00 +CPU= 33.243
00:08:54 STEP 521 H= 521:00 +CPU= 30.482
00:09:46 STEP 522 H= 522:00 +CPU= 51.412
00:10:19 STEP 523 H= 523:00 +CPU= 32.435
00:10:51 STEP 524 H= 524:00 +CPU= 31.510
00:11:42 STEP 525 H= 525:00 +CPU= 51.073
00:12:14 STEP 526 H= 526:00 +CPU= 31.673
00:12:46 STEP 527 H= 527:00 +CPU= 31.337
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMGGhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMSHhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMUAhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMGGhq0f+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMSHhq0f+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMUAhq0f+000516
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0499_2007050100_123_976_12193143_0_r555329025_9.zip
Uploading the intermediate file: upload_file_9.zip
00:13:48 STEP 528 H= 528:00 +CPU= 59.195
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000528
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000528
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000528
00:14:19 STEP 529 H= 529:00 +CPU= 30.198
00:14:51 STEP 530 H= 530:00 +CPU= 32.412
00:15:42 STEP 531 H= 531:00 +CPU= 50.664
00:16:14 STEP 532 H= 532:00 +CPU= 31.518
00:16:47 STEP 533 H= 533:00 +CPU= 31.735
00:17:38 STEP 534 H= 534:00 +CPU= 50.927
00:18:13 STEP 535 H= 535:00 +CPU= 31.823
00:18:44 STEP 536 H= 536:00 +CPU= 31.362
00:19:34 STEP 537 H= 537:00 +CPU= 49.668
00:20:06 STEP 538 H= 538:00 +CPU= 31.637
00:20:38 STEP 539 H= 539:00 +CPU= 30.996
00:21:34 STEP 540 H= 540:00 +CPU= 55.023
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000540
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000540
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000540
00:22:05 STEP 541 H= 541:00 +CPU= 31.211
00:22:38 STEP 542 H= 542:00 +CPU= 32.233
00:23:28 STEP 543 H= 543:00 +CPU= 49.735
00:24:01 STEP 544 H= 544:00 +CPU= 32.504
00:24:32 STEP 545 H= 545:00 +CPU= 30.539
00:25:23 STEP 546 H= 546:00 +CPU= 51.006
00:25:54 STEP 547 H= 547:00 +CPU= 30.044
00:26:27 STEP 548 H= 548:00 +CPU= 33.359
00:27:17 STEP 549 H= 549:00 +CPU= 49.182
00:27:51 STEP 550 H= 550:00 +CPU= 33.354
00:28:23 STEP 551 H= 551:00 +CPU= 31.828
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMGGhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMSHhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMUAhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMGGhq0f+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMSHhq0f+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMUAhq0f+000540
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0499_2007050100_123_976_12193143_0_r555329025_10.zip
Uploading the intermediate file: upload_file_10.zip
00:29:19 STEP 552 H= 552:00 +CPU= 54.406
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000552
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000552
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000552
00:29:50 STEP 553 H= 553:00 +CPU= 30.375
00:30:22 STEP 554 H= 554:00 +CPU= 32.123
00:31:13 STEP 555 H= 555:00 +CPU= 50.627
00:31:45 STEP 556 H= 556:00 +CPU= 31.310
00:32:23 STEP 557 H= 557:00 +CPU= 35.310
00:33:20 STEP 558 H= 558:00 +CPU= 55.965
00:33:52 STEP 559 H= 559:00 +CPU= 31.851
00:35:12 STEP 560 H= 560:00 +CPU= 33.513
00:42:05 STEP 561 H= 561:00 +CPU= 44.242
00:43:22 STEP 562 H= 562:00 +CPU= 31.821
00:44:39 STEP 563 H= 563:00 +CPU= 30.723
00:46:46 STEP 564 H= 564:00 +CPU= 55.681
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000564
00:47:23 STEP 565 H= 565:00 +CPU= 34.294
00:48:25 STEP 566 H= 566:00 +CPU= 32.975
00:49:38 STEP 567 H= 567:00 +CPU= 53.921
00:50:50 STEP 568 H= 568:00 +CPU= 34.726
00:51:30 STEP 569 H= 569:00 +CPU= 33.129
00:52:37 STEP 570 H= 570:00 +CPU= 57.580
00:53:13 STEP 571 H= 571:00 +CPU= 34.860
00:53:51 STEP 572 H= 572:00 +CPU= 36.651
00:55:49 STEP 573 H= 573:00 +CPU= 49.036
00:56:20 STEP 574 H= 574:00 +CPU= 26.134
00:57:05 STEP 575 H= 575:00 +CPU= 28.396
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMGGhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMSHhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMUAhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMGGhq0f+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMSHhq0f+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMUAhq0f+000564
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0499_2007050100_123_976_12193143_0_r555329025_11.zip
Uploading the intermediate file: upload_file_11.zip
00:59:19 STEP 576 H= 576:00 +CPU= 61.160
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000576
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000576
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000576
00:59:54 STEP 577 H= 577:00 +CPU= 34.636
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2007050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0499
(argv4) batchid: 976
(argv5) wuid: 12193143
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/5
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/5/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/5/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_0e3198f9a4c114fec24a653ccfaa7663 to: /var/lib/boinc-client/slots/5/oifs_43r3_ps_0499_2007050100_123_976_12193143.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/5/oifs_43r3_ps_0499_2007050100_123_976_12193143.zip
ic_ancil_file: ic_ancil_12193143
ifsdata_file: ifsdata_12193143
climate_data_file: clim_data_12193143
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_7e60b00cc05949b2b3582335a1df4eb3 to: /var/lib/boinc-client/slots/5/ic_ancil_12193143.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/5/ic_ancil_12193143.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_e693e2bc99207f9f4e2a092cd51668d0 to: /var/lib/boinc-client/slots/5/ifsdata/ifsdata_12193143.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/5/ifsdata/ifsdata_12193143.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/5/159l_2/clim_data_12193143.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/5/159l_2/clim_data_12193143.zip
Checking for progress XML file: /var/lib/boinc-client/slots/5/progress_file_12193143.xml
Opened progress file ok : /var/lib/boinc-client/slots/5/progress_file_12193143.xml
-- Model is restarting --
Adjusting last_iter, 578, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/5/progress_file_12193143.xml
last_cpu_time: 11838
upload_file_number: 12
last_iter: 575
last_upload: 2070000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0499_2007050100_123_976_12193143_0_r555329025
The child process has been launched with process id: 77944
Executing the command: /var/lib/boinc-client/slots/5/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:77944:77944] [20230115:010410:1673741050:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:77944:77944] [20230115:010410:1673741050:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:77944:77944] [20230115:010410:1673741050:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:77944:77944] [20230115:010410:1673741050:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:77944:77944] [20230115:010410:1673741050:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:77944:77944] [20230115:010410:1673741050:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:77944:77944] [20230115:010410:1673741050:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:77944:77944] [20230115:010410:1673741050:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:77944:77944] [20230115:010410:1673741050:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:77944:77944] [20230115:010410:1673741050:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:77944:77944] [20230115:010410:1673741050:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:77944:77944] [20230115:010410:1673741050:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:77944:77944] [20230115:010410:1673741050:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:77944:77944] [20230115:010410:1673741050:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:77944:77944] [20230115:010410:1673741050:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:77944:77944] [20230115:010410:1673741050:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:77944:77944] [20230115:010410:1673741050:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:77944:77944] [20230115:010410:1673741050:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:77944:77944] [20230115:010410:1673741050:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:77944:77944] [20230115:010410:1673741050:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:77944:77944] [20230115:010410:1673741050:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:77944:77944] [20230115:010410:1673741050:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:77944:77944] [20230115:010410:1673741050:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:77944:77944] [20230115:010410:1673741050:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:77944:77944] [20230115:010410:1673741050:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:77944:77944] [20230115:010410:1673741050:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:77944:77944] [20230115:010410:1673741050:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:77944:77944] [20230115:010410:1673741050:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:77944:77944] [20230115:010410:1673741050:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:77944:77944] [20230115:010410:1673741050:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
01:05:36 STEP 576 H= 576:00 +CPU= 58.611
01:06:08 STEP 577 H= 577:00 +CPU= 32.051
01:06:41 STEP 578 H= 578:00 +CPU= 32.073
01:07:33 STEP 579 H= 579:00 +CPU= 52.084
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
01:08:17 STEP 580 H= 580:00 +CPU= 32.397
01:08:48 STEP 581 H= 581:00 +CPU= 30.139
01:09:38 STEP 582 H= 582:00 +CPU= 50.370
01:10:11 STEP 583 H= 583:00 +CPU= 32.431
01:10:43 STEP 584 H= 584:00 +CPU= 31.528
01:11:34 STEP 585 H= 585:00 +CPU= 50.943
01:12:06 STEP 586 H= 586:00 +CPU= 31.631
01:12:39 STEP 587 H= 587:00 +CPU= 32.646
01:13:35 STEP 588 H= 588:00 +CPU= 54.818
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000588
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000588
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000588
01:14:08 STEP 589 H= 589:00 +CPU= 32.685
01:14:40 STEP 590 H= 590:00 +CPU= 31.569
01:15:31 STEP 591 H= 591:00 +CPU= 51.400
01:16:03 STEP 592 H= 592:00 +CPU= 31.345
01:16:34 STEP 593 H= 593:00 +CPU= 30.854
01:17:26 STEP 594 H= 594:00 +CPU= 51.297
01:17:58 STEP 595 H= 595:00 +CPU= 31.744
01:18:30 STEP 596 H= 596:00 +CPU= 31.925
01:19:21 STEP 597 H= 597:00 +CPU= 50.392
01:19:54 STEP 598 H= 598:00 +CPU= 32.444
01:20:26 STEP 599 H= 599:00 +CPU= 32.304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMGGhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMSHhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMUAhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMGGhq0f+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMSHhq0f+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12193143/ICMUAhq0f+000588
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0499_2007050100_123_976_12193143_0_r555329025_12.zip
Uploading the intermediate file: upload_file_12.zip
01:21:28 STEP 600 H= 600:00 +CPU= 57.793
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMGGhq0f+000600
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMSHhq0f+000600
Moving to projects directory: /var/lib/boinc-client/slots/5/ICMUAhq0f+000600
01:21:59 STEP 601 H= 601:00 +CPU= 30.968
01:22:32 STEP 602 H= 602:00 +CPU= 32.093
Abort request received from BOINC client, ending the child process
</stderr_txt>
]]>
