climateprediction.net home page
Task 22444095

Task 22444095

Name oifs_43r3_bl_a1oc_2016092300_20_1018_12290499_0
Workunit 12290499
Created 12 Jun 2024, 23:06:18 UTC
Sent 12 Jun 2024, 23:18:45 UTC
Report deadline 11 Aug 2024, 23:18:45 UTC
Received 16 Jun 2024, 1:04:13 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1535694
Run time 10 hours 27 min 12 sec
CPU time 10 hours 27 min 12 sec
Validate state Valid
Credit 2,635.55
Device peak FLOPS 9.73 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 4,999.15 MB
Peak swap size 5,981.07 MB
Peak disk usage 2,334.62 MB
Stderr
<core_client_version>8.0.2</core_client_version>
<![CDATA[
<stderr_txt>
H= 346:00 +CPU= 20.406
  13:53:49 STEP 1385 H= 346:15 +CPU= 19.555
  13:54:08 STEP 1386 H= 346:30 +CPU= 19.112
  13:54:27 STEP 1387 H= 346:45 +CPU= 18.900
  13:54:47 STEP 1388 H= 347:00 +CPU= 20.077
  13:55:06 STEP 1389 H= 347:15 +CPU= 19.109
  13:55:26 STEP 1390 H= 347:30 +CPU= 19.832
  13:55:46 STEP 1391 H= 347:45 +CPU= 19.613
  13:56:14 STEP 1392 H= 348:00 +CPU= 28.219
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001392
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001392
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001392
  13:56:34 STEP 1393 H= 348:15 +CPU= 20.009
  13:56:54 STEP 1394 H= 348:30 +CPU= 19.287
  13:57:13 STEP 1395 H= 348:45 +CPU= 19.532
  13:57:33 STEP 1396 H= 349:00 +CPU= 19.813
  13:57:52 STEP 1397 H= 349:15 +CPU= 19.399
  13:58:12 STEP 1398 H= 349:30 +CPU= 19.615
  13:58:32 STEP 1399 H= 349:45 +CPU= 19.477
  13:58:51 STEP 1400 H= 350:00 +CPU= 19.546
  13:59:11 STEP 1401 H= 350:15 +CPU= 19.474
  13:59:30 STEP 1402 H= 350:30 +CPU= 18.851
  13:59:50 STEP 1403 H= 350:45 +CPU= 19.894
  14:00:13 STEP 1404 H= 351:00 +CPU= 23.806
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001404
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001404
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001404
  14:00:34 STEP 1405 H= 351:15 +CPU= 19.968
  14:00:54 STEP 1406 H= 351:30 +CPU= 20.055
  14:01:13 STEP 1407 H= 351:45 +CPU= 19.643
  14:01:33 STEP 1408 H= 352:00 +CPU= 19.335
  14:01:51 STEP 1409 H= 352:15 +CPU= 18.081
  14:02:09 STEP 1410 H= 352:30 +CPU= 18.123
  14:02:27 STEP 1411 H= 352:45 +CPU= 18.112
  14:02:46 STEP 1412 H= 353:00 +CPU= 19.070
  14:03:05 STEP 1413 H= 353:15 +CPU= 18.379
  14:03:23 STEP 1414 H= 353:30 +CPU= 18.439
  14:03:41 STEP 1415 H= 353:45 +CPU= 18.182
  14:04:04 STEP 1416 H= 354:00 +CPU= 22.859
  14:04:22 STEP 1417 H= 354:15 +CPU= 18.393
  14:04:41 STEP 1418 H= 354:30 +CPU= 18.370
  14:04:59 STEP 1419 H= 354:45 +CPU= 18.369
  14:05:18 STEP 1420 H= 355:00 +CPU= 18.715
  14:05:36 STEP 1421 H= 355:15 +CPU= 18.453
  14:05:55 STEP 1422 H= 355:30 +CPU= 18.397
  14:06:13 STEP 1423 H= 355:45 +CPU= 18.585
  14:06:32 STEP 1424 H= 356:00 +CPU= 18.868
  14:06:51 STEP 1425 H= 356:15 +CPU= 18.643
  14:07:09 STEP 1426 H= 356:30 +CPU= 18.428
  14:07:28 STEP 1427 H= 356:45 +CPU= 18.538
  14:07:50 STEP 1428 H= 357:00 +CPU= 22.450
  14:08:09 STEP 1429 H= 357:15 +CPU= 18.292
  14:08:27 STEP 1430 H= 357:30 +CPU= 18.454
  14:08:45 STEP 1431 H= 357:45 +CPU= 18.178
  14:09:04 STEP 1432 H= 358:00 +CPU= 18.891
  14:09:23 STEP 1433 H= 358:15 +CPU= 18.474
  14:09:41 STEP 1434 H= 358:30 +CPU= 18.526
  14:09:59 STEP 1435 H= 358:45 +CPU= 18.372
  14:10:18 STEP 1436 H= 359:00 +CPU= 18.727
  14:10:37 STEP 1437 H= 359:15 +CPU= 18.701
  14:10:55 STEP 1438 H= 359:30 +CPU= 18.538
  14:11:14 STEP 1439 H= 359:45 +CPU= 18.526
  14:11:40 STEP 1440 H= 360:00 +CPU= 26.088
  14:11:58 STEP 1441 H= 360:15 +CPU= 18.357
  14:12:17 STEP 1442 H= 360:30 +CPU= 18.317
  14:12:35 STEP 1443 H= 360:45 +CPU= 18.225
  14:12:54 STEP 1444 H= 361:00 +CPU= 18.692
  14:13:12 STEP 1445 H= 361:15 +CPU= 18.348
  14:13:30 STEP 1446 H= 361:30 +CPU= 18.309
  14:13:49 STEP 1447 H= 361:45 +CPU= 18.451
  14:14:07 STEP 1448 H= 362:00 +CPU= 18.795
  14:14:26 STEP 1449 H= 362:15 +CPU= 18.499
  14:14:44 STEP 1450 H= 362:30 +CPU= 18.438
  14:15:03 STEP 1451 H= 362:45 +CPU= 18.525
  14:15:27 STEP 1452 H= 363:00 +CPU= 23.994
  14:15:46 STEP 1453 H= 363:15 +CPU= 18.962
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001452
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001452
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001404
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a1oc_2016092300_20_1018_12290499_0_r1220820879_14.zip
Uploading the intermediate file: upload_file_14.zip
  14:16:05 STEP 1454 H= 363:30 +CPU= 19.058
Uploading trickle at timestep: 1306800
  14:16:24 STEP 1455 H= 363:45 +CPU= 19.097
  14:16:46 STEP 1456 H= 364:00 +CPU= 21.604
  14:17:12 STEP 1457 H= 364:15 +CPU= 25.622
  14:17:34 STEP 1458 H= 364:30 +CPU= 21.925
  14:17:52 STEP 1459 H= 364:45 +CPU= 18.309
  14:18:11 STEP 1460 H= 365:00 +CPU= 18.872
  14:18:30 STEP 1461 H= 365:15 +CPU= 18.551
  14:18:48 STEP 1462 H= 365:30 +CPU= 18.605
  14:19:07 STEP 1463 H= 365:45 +CPU= 18.782
  14:19:30 STEP 1464 H= 366:00 +CPU= 23.404
  14:19:49 STEP 1465 H= 366:15 +CPU= 18.463
  14:20:08 STEP 1466 H= 366:30 +CPU= 19.047
  14:20:28 STEP 1467 H= 366:45 +CPU= 20.361
  14:20:49 STEP 1468 H= 367:00 +CPU= 20.735
  14:21:10 STEP 1469 H= 367:15 +CPU= 20.288
  14:21:29 STEP 1470 H= 367:30 +CPU= 19.595
  14:21:50 STEP 1471 H= 367:45 +CPU= 20.303
  14:22:10 STEP 1472 H= 368:00 +CPU= 20.035
  14:22:30 STEP 1473 H= 368:15 +CPU= 19.745
  14:22:49 STEP 1474 H= 368:30 +CPU= 19.784
  14:23:09 STEP 1475 H= 368:45 +CPU= 19.847
  14:23:34 STEP 1476 H= 369:00 +CPU= 24.228
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001476
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001476
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001476
  14:23:53 STEP 1477 H= 369:15 +CPU= 19.806
  14:24:14 STEP 1478 H= 369:30 +CPU= 20.442
  14:24:33 STEP 1479 H= 369:45 +CPU= 19.439
  14:24:53 STEP 1480 H= 370:00 +CPU= 19.565
  14:25:13 STEP 1481 H= 370:15 +CPU= 19.489
  14:25:33 STEP 1482 H= 370:30 +CPU= 20.746
  14:25:54 STEP 1483 H= 370:45 +CPU= 20.217
  14:26:14 STEP 1484 H= 371:00 +CPU= 20.597
  14:26:34 STEP 1485 H= 371:15 +CPU= 19.978
  14:26:54 STEP 1486 H= 371:30 +CPU= 19.777
  14:27:14 STEP 1487 H= 371:45 +CPU= 19.894
  14:27:43 STEP 1488 H= 372:00 +CPU= 28.427
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001488
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001488
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001488
  14:28:02 STEP 1489 H= 372:15 +CPU= 19.590
  14:28:23 STEP 1490 H= 372:30 +CPU= 20.262
  14:28:43 STEP 1491 H= 372:45 +CPU= 20.129
  14:29:04 STEP 1492 H= 373:00 +CPU= 21.208
  14:29:25 STEP 1493 H= 373:15 +CPU= 20.290
  14:29:45 STEP 1494 H= 373:30 +CPU= 20.547
  14:30:05 STEP 1495 H= 373:45 +CPU= 19.949
  14:30:25 STEP 1496 H= 374:00 +CPU= 20.043
  14:30:45 STEP 1497 H= 374:15 +CPU= 19.702
  14:31:06 STEP 1498 H= 374:30 +CPU= 20.689
  14:31:25 STEP 1499 H= 374:45 +CPU= 19.423
  14:31:50 STEP 1500 H= 375:00 +CPU= 24.376
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001500
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001500
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001500
  14:32:10 STEP 1501 H= 375:15 +CPU= 20.259
  14:32:30 STEP 1502 H= 375:30 +CPU= 19.835
  14:32:50 STEP 1503 H= 375:45 +CPU= 19.791
  14:33:10 STEP 1504 H= 376:00 +CPU= 20.023
  14:33:29 STEP 1505 H= 376:15 +CPU= 19.101
  14:33:49 STEP 1506 H= 376:30 +CPU= 19.785
  14:34:09 STEP 1507 H= 376:45 +CPU= 19.422
  14:34:29 STEP 1508 H= 377:00 +CPU= 20.391
  14:34:49 STEP 1509 H= 377:15 +CPU= 19.561
  14:35:09 STEP 1510 H= 377:30 +CPU= 20.006
  14:35:29 STEP 1511 H= 377:45 +CPU= 19.976
  14:35:53 STEP 1512 H= 378:00 +CPU= 24.704
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001512
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001512
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001512
  14:36:14 STEP 1513 H= 378:15 +CPU= 20.635
  14:36:35 STEP 1514 H= 378:30 +CPU= 20.362
  14:36:54 STEP 1515 H= 378:45 +CPU= 19.564
  14:37:15 STEP 1516 H= 379:00 +CPU= 20.585
  14:37:34 STEP 1517 H= 379:15 +CPU= 19.458
  14:37:55 STEP 1518 H= 379:30 +CPU= 20.150
  14:38:14 STEP 1519 H= 379:45 +CPU= 19.329
  14:38:34 STEP 1520 H= 380:00 +CPU= 20.168
  14:38:54 STEP 1521 H= 380:15 +CPU= 19.545
  14:39:14 STEP 1522 H= 380:30 +CPU= 19.668
  14:39:33 STEP 1523 H= 380:45 +CPU= 19.176
  14:39:58 STEP 1524 H= 381:00 +CPU= 25.018
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001524
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001524
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001524
  14:40:19 STEP 1525 H= 381:15 +CPU= 20.977
  14:40:39 STEP 1526 H= 381:30 +CPU= 20.104
  14:40:59 STEP 1527 H= 381:45 +CPU= 19.392
  14:41:18 STEP 1528 H= 382:00 +CPU= 19.521
  14:41:38 STEP 1529 H= 382:15 +CPU= 20.320
  14:41:58 STEP 1530 H= 382:30 +CPU= 19.105
  14:42:17 STEP 1531 H= 382:45 +CPU= 19.423
  14:42:37 STEP 1532 H= 383:00 +CPU= 20.273
  14:42:57 STEP 1533 H= 383:15 +CPU= 19.534
  14:43:17 STEP 1534 H= 383:30 +CPU= 20.306
  14:43:37 STEP 1535 H= 383:45 +CPU= 20.018
  14:44:06 STEP 1536 H= 384:00 +CPU= 28.677
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001536
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001536
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001536
  14:44:26 STEP 1537 H= 384:15 +CPU= 19.977
  14:44:47 STEP 1538 H= 384:30 +CPU= 20.607
  14:45:07 STEP 1539 H= 384:45 +CPU= 20.142
  14:45:28 STEP 1540 H= 385:00 +CPU= 20.376
  14:45:49 STEP 1541 H= 385:15 +CPU= 21.018
  14:46:10 STEP 1542 H= 385:30 +CPU= 20.693
  14:46:30 STEP 1543 H= 385:45 +CPU= 20.606
  14:46:51 STEP 1544 H= 386:00 +CPU= 20.222
  14:47:11 STEP 1545 H= 386:15 +CPU= 20.297
  14:47:36 STEP 1546 H= 386:30 +CPU= 24.766
  14:47:58 STEP 1547 H= 386:45 +CPU= 22.141
  14:48:23 STEP 1548 H= 387:00 +CPU= 25.010
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001548
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001548
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001536
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a1oc_2016092300_20_1018_12290499_0_r1220820879_15.zip
Uploading the intermediate file: upload_file_15.zip
  14:48:43 STEP 1549 H= 387:15 +CPU= 19.745
  14:49:03 STEP 1550 H= 387:30 +CPU= 19.871
  14:49:28 STEP 1551 H= 387:45 +CPU= 24.575
  14:49:52 STEP 1552 H= 388:00 +CPU= 24.193
  14:50:12 STEP 1553 H= 388:15 +CPU= 19.906
  14:50:32 STEP 1554 H= 388:30 +CPU= 19.990
  14:50:52 STEP 1555 H= 388:45 +CPU= 19.757
  14:51:13 STEP 1556 H= 389:00 +CPU= 20.360
  14:51:32 STEP 1557 H= 389:15 +CPU= 19.647
  14:51:53 STEP 1558 H= 389:30 +CPU= 20.132
  14:52:13 STEP 1559 H= 389:45 +CPU= 20.373
  14:52:37 STEP 1560 H= 390:00 +CPU= 24.210
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001560
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001560
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001560
  14:52:58 STEP 1561 H= 390:15 +CPU= 20.311
  14:53:17 STEP 1562 H= 390:30 +CPU= 19.777
  14:53:38 STEP 1563 H= 390:45 +CPU= 20.252
  14:53:58 STEP 1564 H= 391:00 +CPU= 20.474
  14:54:18 STEP 1565 H= 391:15 +CPU= 19.553
  14:54:38 STEP 1566 H= 391:30 +CPU= 19.555
  14:54:57 STEP 1567 H= 391:45 +CPU= 19.499
  14:55:18 STEP 1568 H= 392:00 +CPU= 20.877
  14:55:38 STEP 1569 H= 392:15 +CPU= 19.856
  14:55:58 STEP 1570 H= 392:30 +CPU= 19.941
  14:56:19 STEP 1571 H= 392:45 +CPU= 20.813
  14:56:44 STEP 1572 H= 393:00 +CPU= 25.217
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001572
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001572
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001572
  14:57:05 STEP 1573 H= 393:15 +CPU= 20.666
  14:57:25 STEP 1574 H= 393:30 +CPU= 20.245
  14:57:46 STEP 1575 H= 393:45 +CPU= 20.259
  14:58:06 STEP 1576 H= 394:00 +CPU= 20.160
  14:58:25 STEP 1577 H= 394:15 +CPU= 19.512
  14:58:46 STEP 1578 H= 394:30 +CPU= 20.211
  14:59:05 STEP 1579 H= 394:45 +CPU= 19.486
  14:59:25 STEP 1580 H= 395:00 +CPU= 19.627
  14:59:45 STEP 1581 H= 395:15 +CPU= 19.849
  15:00:05 STEP 1582 H= 395:30 +CPU= 20.385
  15:00:25 STEP 1583 H= 395:45 +CPU= 19.500
  15:00:53 STEP 1584 H= 396:00 +CPU= 28.440
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001584
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001584
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001584
  15:01:15 STEP 1585 H= 396:15 +CPU= 21.868
  15:01:35 STEP 1586 H= 396:30 +CPU= 19.482
  15:01:55 STEP 1587 H= 396:45 +CPU= 20.077
  15:02:15 STEP 1588 H= 397:00 +CPU= 20.055
  15:02:35 STEP 1589 H= 397:15 +CPU= 19.940
  15:02:55 STEP 1590 H= 397:30 +CPU= 19.876
  15:03:15 STEP 1591 H= 397:45 +CPU= 19.560
  15:03:35 STEP 1592 H= 398:00 +CPU= 20.319
  15:03:55 STEP 1593 H= 398:15 +CPU= 20.056
  15:04:15 STEP 1594 H= 398:30 +CPU= 20.068
  15:04:35 STEP 1595 H= 398:45 +CPU= 19.366
  15:04:59 STEP 1596 H= 399:00 +CPU= 23.840
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001596
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001596
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001596
  15:05:18 STEP 1597 H= 399:15 +CPU= 19.350
  15:05:38 STEP 1598 H= 399:30 +CPU= 20.133
  15:05:57 STEP 1599 H= 399:45 +CPU= 18.925
  15:06:17 STEP 1600 H= 400:00 +CPU= 20.042
  15:06:37 STEP 1601 H= 400:15 +CPU= 19.627
  15:06:57 STEP 1602 H= 400:30 +CPU= 19.573
  15:07:17 STEP 1603 H= 400:45 +CPU= 19.833
  15:07:36 STEP 1604 H= 401:00 +CPU= 19.772
  15:07:56 STEP 1605 H= 401:15 +CPU= 19.330
  15:08:16 STEP 1606 H= 401:30 +CPU= 19.602
  15:08:35 STEP 1607 H= 401:45 +CPU= 19.849
  15:08:59 STEP 1608 H= 402:00 +CPU= 23.739
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001608
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001608
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001608
  15:09:18 STEP 1609 H= 402:15 +CPU= 19.152
  15:09:38 STEP 1610 H= 402:30 +CPU= 19.342
  15:09:58 STEP 1611 H= 402:45 +CPU= 19.715
  15:10:18 STEP 1612 H= 403:00 +CPU= 20.108
  15:10:37 STEP 1613 H= 403:15 +CPU= 19.115
  15:10:56 STEP 1614 H= 403:30 +CPU= 19.433
  15:11:16 STEP 1615 H= 403:45 +CPU= 19.552
  15:11:36 STEP 1616 H= 404:00 +CPU= 20.108
  15:11:56 STEP 1617 H= 404:15 +CPU= 19.440
  15:12:15 STEP 1618 H= 404:30 +CPU= 19.680
  15:12:35 STEP 1619 H= 404:45 +CPU= 19.674
  15:12:59 STEP 1620 H= 405:00 +CPU= 23.821
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001620
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001620
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001620
  15:13:18 STEP 1621 H= 405:15 +CPU= 19.244
  15:13:39 STEP 1622 H= 405:30 +CPU= 20.848
  15:14:00 STEP 1623 H= 405:45 +CPU= 20.382
  15:14:20 STEP 1624 H= 406:00 +CPU= 19.674
  15:14:40 STEP 1625 H= 406:15 +CPU= 20.206
  15:15:00 STEP 1626 H= 406:30 +CPU= 20.062
  15:15:19 STEP 1627 H= 406:45 +CPU= 19.113
  15:15:39 STEP 1628 H= 407:00 +CPU= 19.450
  15:15:58 STEP 1629 H= 407:15 +CPU= 19.562
  15:16:18 STEP 1630 H= 407:30 +CPU= 19.240
  15:16:37 STEP 1631 H= 407:45 +CPU= 19.327
  15:17:05 STEP 1632 H= 408:00 +CPU= 27.903
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001632
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001632
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001632
  15:17:25 STEP 1633 H= 408:15 +CPU= 19.597
  15:17:45 STEP 1634 H= 408:30 +CPU= 19.821
  15:18:04 STEP 1635 H= 408:45 +CPU= 19.724
  15:18:27 STEP 1636 H= 409:00 +CPU= 21.947
  15:18:48 STEP 1637 H= 409:15 +CPU= 21.703
  15:19:07 STEP 1638 H= 409:30 +CPU= 18.463
  15:19:25 STEP 1639 H= 409:45 +CPU= 18.400
  15:19:44 STEP 1640 H= 410:00 +CPU= 18.639
  15:20:02 STEP 1641 H= 410:15 +CPU= 18.383
  15:20:21 STEP 1642 H= 410:30 +CPU= 18.312
  15:20:39 STEP 1643 H= 410:45 +CPU= 18.110
  15:21:01 STEP 1644 H= 411:00 +CPU= 22.452
  15:21:19 STEP 1645 H= 411:15 +CPU= 17.999
  15:21:44 STEP 1646 H= 411:30 +CPU= 21.746
  15:22:02 STEP 1647 H= 411:45 +CPU= 17.784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001632
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a1oc_2016092300_20_1018_12290499_0_r1220820879_16.zip
Uploading the intermediate file: upload_file_16.zip
  15:22:19 STEP 1648 H= 412:00 +CPU= 16.771
  15:22:35 STEP 1649 H= 412:15 +CPU= 15.896
  15:22:53 STEP 1650 H= 412:30 +CPU= 17.869
  15:23:17 STEP 1651 H= 412:45 +CPU= 23.202
  15:23:40 STEP 1652 H= 413:00 +CPU= 23.298
  15:23:58 STEP 1653 H= 413:15 +CPU= 17.953
  15:24:16 STEP 1654 H= 413:30 +CPU= 17.649
  15:24:34 STEP 1655 H= 413:45 +CPU= 18.529
  15:24:57 STEP 1656 H= 414:00 +CPU= 22.233
  15:25:14 STEP 1657 H= 414:15 +CPU= 17.643
  15:25:33 STEP 1658 H= 414:30 +CPU= 18.215
  15:25:50 STEP 1659 H= 414:45 +CPU= 17.499
  15:26:08 STEP 1660 H= 415:00 +CPU= 18.155
  15:26:26 STEP 1661 H= 415:15 +CPU= 17.856
  15:26:44 STEP 1662 H= 415:30 +CPU= 17.639
  15:27:01 STEP 1663 H= 415:45 +CPU= 17.270
  15:27:20 STEP 1664 H= 416:00 +CPU= 18.185
  15:27:38 STEP 1665 H= 416:15 +CPU= 18.306
  15:27:57 STEP 1666 H= 416:30 +CPU= 19.332
  15:28:16 STEP 1667 H= 416:45 +CPU= 18.632
  15:28:39 STEP 1668 H= 417:00 +CPU= 23.312
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001668
  15:28:58 STEP 1669 H= 417:15 +CPU= 18.616
  15:29:17 STEP 1670 H= 417:30 +CPU= 18.656
  15:29:36 STEP 1671 H= 417:45 +CPU= 18.875
  15:29:55 STEP 1672 H= 418:00 +CPU= 19.643
Suspend request received from the BOINC client, suspending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a1oc
(argv4) batchid: 1018
(argv5) wuid: 12290499
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/0
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/0/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/0/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_e350ee2ca30cd91731ff3818a259d53b to: /var/lib/boinc-client/slots/0/oifs_43r3_bl_a1oc_2016092300_20_1018_12290499.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/0/oifs_43r3_bl_a1oc_2016092300_20_1018_12290499.zip
ic_ancil_file: ic_ancil_12290499
ifsdata_file: ifsdata_12290499
climate_data_file: clim_data_12290499
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/0/ic_ancil_12290499.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/0/ic_ancil_12290499.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_4afecf8ff39c54e890ac838b4364e22d to: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12290499.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/0/ifsdata/ifsdata_12290499.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/0/159l_2/clim_data_12290499.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/0/159l_2/clim_data_12290499.zip
Checking for progress XML file: /var/lib/boinc-client/slots/0/progress_file_12290499.xml
Opened progress file ok : /var/lib/boinc-client/slots/0/progress_file_12290499.xml
-- Model is restarting --
Adjusting last_iter, 1673, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/0/progress_file_12290499.xml
last_cpu_time: 32359
upload_file_number: 17
last_iter: 1633
last_upload: 1481400
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a1oc_2016092300_20_1018_12290499_0_r1220820879
The child process has been launched with process id: 4416
Executing the command: /var/lib/boinc-client/slots/0/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:gp-VirtualBox:1:1:4416:4416] [20240616:090008:1718492408:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:gp-VirtualBox:1:1:4416:4416] [20240616:090008:1718492408:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:gp-VirtualBox:1:1:4416:4416] [20240616:090008:1718492408:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:gp-VirtualBox:1:1:4416:4416] [20240616:090008:1718492408:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:gp-VirtualBox:1:1:4416:4416] [20240616:090008:1718492408:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:gp-VirtualBox:1:1:4416:4416] [20240616:090008:1718492408:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:gp-VirtualBox:1:1:4416:4416] [20240616:090008:1718492408:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:gp-VirtualBox:1:1:4416:4416] [20240616:090008:1718492408:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:gp-VirtualBox:1:1:4416:4416] [20240616:090008:1718492408:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:gp-VirtualBox:1:1:4416:4416] [20240616:090008:1718492408:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:gp-VirtualBox:1:1:4416:4416] [20240616:090008:1718492408:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:gp-VirtualBox:1:1:4416:4416] [20240616:090008:1718492408:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:gp-VirtualBox:1:1:4416:4416] [20240616:090008:1718492408:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:gp-VirtualBox:1:1:4416:4416] [20240616:090008:1718492408:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:gp-VirtualBox:1:1:4416:4416] [20240616:090008:1718492408:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:gp-VirtualBox:1:1:4416:4416] [20240616:090008:1718492408:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:gp-VirtualBox:1:1:4416:4416] [20240616:090008:1718492408:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:gp-VirtualBox:1:1:4416:4416] [20240616:090008:1718492408:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:gp-VirtualBox:1:1:4416:4416] [20240616:090008:1718492408:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:gp-VirtualBox:1:1:4416:4416] [20240616:090008:1718492408:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:gp-VirtualBox:1:1:4416:4416] [20240616:090008:1718492408:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:gp-VirtualBox:1:1:4416:4416] [20240616:090008:1718492408:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:gp-VirtualBox:1:1:4416:4416] [20240616:090008:1718492408:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:gp-VirtualBox:1:1:4416:4416] [20240616:090008:1718492408:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:gp-VirtualBox:1:1:4416:4416] [20240616:090008:1718492408:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:gp-VirtualBox:1:1:4416:4416] [20240616:090008:1718492408:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:gp-VirtualBox:1:1:4416:4416] [20240616:090008:1718492408:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:gp-VirtualBox:1:1:4416:4416] [20240616:090008:1718492408:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:gp-VirtualBox:1:1:4416:4416] [20240616:090008:1718492408:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:gp-VirtualBox:1:1:4416:4416] [20240616:090008:1718492408:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  09:01:12 STEP 1632 H= 408:00 +CPU= 43.070
  09:01:32 STEP 1633 H= 408:15 +CPU= 19.451
  09:01:50 STEP 1634 H= 408:30 +CPU= 18.748
  09:02:10 STEP 1635 H= 408:45 +CPU= 19.832
  09:02:30 STEP 1636 H= 409:00 +CPU= 19.806
  09:02:50 STEP 1637 H= 409:15 +CPU= 20.114
  09:03:10 STEP 1638 H= 409:30 +CPU= 19.164
  09:03:28 STEP 1639 H= 409:45 +CPU= 18.734
  09:03:49 STEP 1640 H= 410:00 +CPU= 20.246
  09:04:08 STEP 1641 H= 410:15 +CPU= 19.322
  09:04:28 STEP 1642 H= 410:30 +CPU= 19.837
  09:04:48 STEP 1643 H= 410:45 +CPU= 19.766
  09:05:12 STEP 1644 H= 411:00 +CPU= 23.608
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001644
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001644
  09:05:30 STEP 1645 H= 411:15 +CPU= 18.343
  09:05:49 STEP 1646 H= 411:30 +CPU= 18.392
  09:06:08 STEP 1647 H= 411:45 +CPU= 18.963
  09:06:26 STEP 1648 H= 412:00 +CPU= 18.805
  09:06:45 STEP 1649 H= 412:15 +CPU= 18.218
  09:07:03 STEP 1650 H= 412:30 +CPU= 17.966
  09:07:21 STEP 1651 H= 412:45 +CPU= 17.953
  09:07:40 STEP 1652 H= 413:00 +CPU= 18.881
  09:07:58 STEP 1653 H= 413:15 +CPU= 18.222
  09:08:16 STEP 1654 H= 413:30 +CPU= 18.032
  09:08:35 STEP 1655 H= 413:45 +CPU= 18.618
  09:08:57 STEP 1656 H= 414:00 +CPU= 21.735
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001656
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001656
  09:09:15 STEP 1657 H= 414:15 +CPU= 18.269
  09:09:34 STEP 1658 H= 414:30 +CPU= 18.848
  09:09:52 STEP 1659 H= 414:45 +CPU= 17.939
  09:10:11 STEP 1660 H= 415:00 +CPU= 18.537
  09:10:29 STEP 1661 H= 415:15 +CPU= 18.128
  09:10:47 STEP 1662 H= 415:30 +CPU= 18.221
  09:11:05 STEP 1663 H= 415:45 +CPU= 18.092
  09:11:25 STEP 1664 H= 416:00 +CPU= 20.073
  09:11:43 STEP 1665 H= 416:15 +CPU= 17.767
  09:12:02 STEP 1666 H= 416:30 +CPU= 18.543
  09:12:20 STEP 1667 H= 416:45 +CPU= 17.881
  09:12:41 STEP 1668 H= 417:00 +CPU= 21.103
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001668
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001668
  09:12:59 STEP 1669 H= 417:15 +CPU= 17.633
  09:13:16 STEP 1670 H= 417:30 +CPU= 16.987
  09:13:33 STEP 1671 H= 417:45 +CPU= 17.712
  09:13:51 STEP 1672 H= 418:00 +CPU= 17.487
  09:14:08 STEP 1673 H= 418:15 +CPU= 17.065
  09:14:26 STEP 1674 H= 418:30 +CPU= 17.784
  09:14:43 STEP 1675 H= 418:45 +CPU= 16.806
  09:15:00 STEP 1676 H= 419:00 +CPU= 17.361
  09:15:19 STEP 1677 H= 419:15 +CPU= 18.311
  09:15:36 STEP 1678 H= 419:30 +CPU= 17.664
  09:15:54 STEP 1679 H= 419:45 +CPU= 17.772
  09:16:22 STEP 1680 H= 420:00 +CPU= 27.639
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001680
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001680
  09:16:41 STEP 1681 H= 420:15 +CPU= 19.294
  09:17:01 STEP 1682 H= 420:30 +CPU= 19.572
  09:17:20 STEP 1683 H= 420:45 +CPU= 19.109
  09:17:41 STEP 1684 H= 421:00 +CPU= 20.430
  09:18:00 STEP 1685 H= 421:15 +CPU= 19.496
  09:18:21 STEP 1686 H= 421:30 +CPU= 20.154
  09:18:40 STEP 1687 H= 421:45 +CPU= 19.050
  09:18:58 STEP 1688 H= 422:00 +CPU= 18.618
  09:19:16 STEP 1689 H= 422:15 +CPU= 17.714
  09:19:34 STEP 1690 H= 422:30 +CPU= 18.242
  09:19:52 STEP 1691 H= 422:45 +CPU= 17.420
  09:20:15 STEP 1692 H= 423:00 +CPU= 22.860
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001692
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001692
  09:20:34 STEP 1693 H= 423:15 +CPU= 18.700
  09:20:52 STEP 1694 H= 423:30 +CPU= 18.606
  09:21:11 STEP 1695 H= 423:45 +CPU= 18.280
  09:21:29 STEP 1696 H= 424:00 +CPU= 18.092
  09:21:46 STEP 1697 H= 424:15 +CPU= 17.515
  09:22:05 STEP 1698 H= 424:30 +CPU= 18.619
  09:22:24 STEP 1699 H= 424:45 +CPU= 18.619
  09:22:43 STEP 1700 H= 425:00 +CPU= 19.219
  09:23:01 STEP 1701 H= 425:15 +CPU= 18.390
  09:23:19 STEP 1702 H= 425:30 +CPU= 17.696
  09:23:36 STEP 1703 H= 425:45 +CPU= 16.653
  09:23:57 STEP 1704 H= 426:00 +CPU= 21.186
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001704
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001704
  09:24:15 STEP 1705 H= 426:15 +CPU= 17.653
  09:24:32 STEP 1706 H= 426:30 +CPU= 17.308
  09:24:50 STEP 1707 H= 426:45 +CPU= 17.569
  09:25:07 STEP 1708 H= 427:00 +CPU= 17.342
  09:25:25 STEP 1709 H= 427:15 +CPU= 17.563
  09:25:42 STEP 1710 H= 427:30 +CPU= 17.451
  09:26:00 STEP 1711 H= 427:45 +CPU= 17.366
  09:26:18 STEP 1712 H= 428:00 +CPU= 17.596
  09:26:35 STEP 1713 H= 428:15 +CPU= 17.624
  09:26:53 STEP 1714 H= 428:30 +CPU= 17.300
  09:27:10 STEP 1715 H= 428:45 +CPU= 17.208
  09:27:31 STEP 1716 H= 429:00 +CPU= 21.568
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001716
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001716
  09:27:48 STEP 1717 H= 429:15 +CPU= 16.769
  09:28:06 STEP 1718 H= 429:30 +CPU= 17.621
  09:28:23 STEP 1719 H= 429:45 +CPU= 17.330
  09:28:41 STEP 1720 H= 430:00 +CPU= 17.619
  09:28:57 STEP 1721 H= 430:15 +CPU= 16.292
  09:29:14 STEP 1722 H= 430:30 +CPU= 16.985
  09:29:33 STEP 1723 H= 430:45 +CPU= 18.030
  09:29:50 STEP 1724 H= 431:00 +CPU= 17.049
  09:30:07 STEP 1725 H= 431:15 +CPU= 17.481
  09:30:29 STEP 1726 H= 431:30 +CPU= 21.518
  09:30:45 STEP 1727 H= 431:45 +CPU= 16.524
  09:31:11 STEP 1728 H= 432:00 +CPU= 25.591
  09:31:26 STEP 1729 H= 432:15 +CPU= 14.271
  09:31:40 STEP 1730 H= 432:30 +CPU= 14.828
  09:31:55 STEP 1731 H= 432:45 +CPU= 15.021
  09:32:11 STEP 1732 H= 433:00 +CPU= 15.079
  09:32:26 STEP 1733 H= 433:15 +CPU= 15.080
  09:32:41 STEP 1734 H= 433:30 +CPU= 15.147
  09:32:56 STEP 1735 H= 433:45 +CPU= 14.905
  09:33:11 STEP 1736 H= 434:00 +CPU= 15.549
  09:33:26 STEP 1737 H= 434:15 +CPU= 15.095
  09:33:41 STEP 1738 H= 434:30 +CPU= 15.046
  09:33:56 STEP 1739 H= 434:45 +CPU= 15.106
  09:34:15 STEP 1740 H= 435:00 +CPU= 18.732
  09:34:31 STEP 1741 H= 435:15 +CPU= 15.346
  09:34:47 STEP 1742 H= 435:30 +CPU= 16.198
  09:35:02 STEP 1743 H= 435:45 +CPU= 15.641
  09:35:21 STEP 1744 H= 436:00 +CPU= 18.220
  09:35:39 STEP 1745 H= 436:15 +CPU= 18.440
  09:35:55 STEP 1746 H= 436:30 +CPU= 15.816
  09:36:11 STEP 1747 H= 436:45 +CPU= 15.692
  09:36:27 STEP 1748 H= 437:00 +CPU= 16.196
  09:36:43 STEP 1749 H= 437:15 +CPU= 15.805
  09:36:58 STEP 1750 H= 437:30 +CPU= 15.550
  09:37:14 STEP 1751 H= 437:45 +CPU= 15.551
  09:37:33 STEP 1752 H= 438:00 +CPU= 19.282
  09:37:49 STEP 1753 H= 438:15 +CPU= 15.560
  09:38:04 STEP 1754 H= 438:30 +CPU= 15.528
  09:38:20 STEP 1755 H= 438:45 +CPU= 15.649
  09:38:36 STEP 1756 H= 439:00 +CPU= 15.971
  09:38:51 STEP 1757 H= 439:15 +CPU= 15.644
  09:39:07 STEP 1758 H= 439:30 +CPU= 15.501
  09:39:23 STEP 1759 H= 439:45 +CPU= 15.705
  09:39:38 STEP 1760 H= 440:00 +CPU= 15.968
  09:39:54 STEP 1761 H= 440:15 +CPU= 15.774
  09:40:10 STEP 1762 H= 440:30 +CPU= 15.577
  09:40:26 STEP 1763 H= 440:45 +CPU= 15.854
  09:40:45 STEP 1764 H= 441:00 +CPU= 19.213
  09:41:00 STEP 1765 H= 441:15 +CPU= 15.605
  09:41:16 STEP 1766 H= 441:30 +CPU= 15.675
  09:41:32 STEP 1767 H= 441:45 +CPU= 15.646
  09:41:48 STEP 1768 H= 442:00 +CPU= 15.885
  09:42:03 STEP 1769 H= 442:15 +CPU= 15.125
  09:42:17 STEP 1770 H= 442:30 +CPU= 14.490
  09:42:31 STEP 1771 H= 442:45 +CPU= 14.169
  09:42:46 STEP 1772 H= 443:00 +CPU= 14.456
  09:43:00 STEP 1773 H= 443:15 +CPU= 13.743
  09:43:14 STEP 1774 H= 443:30 +CPU= 14.334
  09:43:28 STEP 1775 H= 443:45 +CPU= 14.229
  10:04:02 STEP 1776 H= 444:00 +CPU= 30.922
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001776
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001776
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001668
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001716
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a1oc_2016092300_20_1018_12290499_0_r1220820879_17.zip
Uploading the intermediate file: upload_file_17.zip
  10:04:21 STEP 1777 H= 444:15 +CPU= 18.421
  10:04:38 STEP 1778 H= 444:30 +CPU= 17.393
  10:04:54 STEP 1779 H= 444:45 +CPU= 16.350
  10:05:14 STEP 1780 H= 445:00 +CPU= 19.736
  10:05:37 STEP 1781 H= 445:15 +CPU= 22.939
  10:05:56 STEP 1782 H= 445:30 +CPU= 18.757
  10:06:14 STEP 1783 H= 445:45 +CPU= 17.808
  10:06:33 STEP 1784 H= 446:00 +CPU= 19.045
  10:06:53 STEP 1785 H= 446:15 +CPU= 19.537
  10:07:11 STEP 1786 H= 446:30 +CPU= 17.953
  10:07:29 STEP 1787 H= 446:45 +CPU= 18.501
  10:07:53 STEP 1788 H= 447:00 +CPU= 23.662
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001788
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001788
  10:08:11 STEP 1789 H= 447:15 +CPU= 18.211
  10:08:29 STEP 1790 H= 447:30 +CPU= 18.033
  10:08:48 STEP 1791 H= 447:45 +CPU= 18.783
  10:09:07 STEP 1792 H= 448:00 +CPU= 18.613
  10:09:25 STEP 1793 H= 448:15 +CPU= 18.023
  10:09:43 STEP 1794 H= 448:30 +CPU= 17.811
  10:10:02 STEP 1795 H= 448:45 +CPU= 18.680
  10:10:22 STEP 1796 H= 449:00 +CPU= 19.849
  10:10:41 STEP 1797 H= 449:15 +CPU= 19.611
  10:11:01 STEP 1798 H= 449:30 +CPU= 19.356
  10:11:19 STEP 1799 H= 449:45 +CPU= 18.320
  10:11:42 STEP 1800 H= 450:00 +CPU= 22.743
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001800
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001800
  10:12:01 STEP 1801 H= 450:15 +CPU= 18.582
  10:12:20 STEP 1802 H= 450:30 +CPU= 19.053
  10:12:39 STEP 1803 H= 450:45 +CPU= 18.675
  10:12:58 STEP 1804 H= 451:00 +CPU= 18.965
  10:13:16 STEP 1805 H= 451:15 +CPU= 18.324
  10:13:34 STEP 1806 H= 451:30 +CPU= 18.261
  10:13:53 STEP 1807 H= 451:45 +CPU= 18.555
  10:14:12 STEP 1808 H= 452:00 +CPU= 19.253
  10:14:31 STEP 1809 H= 452:15 +CPU= 18.439
  10:14:49 STEP 1810 H= 452:30 +CPU= 18.250
  10:15:09 STEP 1811 H= 452:45 +CPU= 19.543
  10:15:32 STEP 1812 H= 453:00 +CPU= 22.869
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001812
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001812
  10:15:51 STEP 1813 H= 453:15 +CPU= 18.744
  10:16:09 STEP 1814 H= 453:30 +CPU= 18.551
  10:16:29 STEP 1815 H= 453:45 +CPU= 19.552
  10:16:48 STEP 1816 H= 454:00 +CPU= 18.886
  10:17:07 STEP 1817 H= 454:15 +CPU= 19.341
  10:17:26 STEP 1818 H= 454:30 +CPU= 19.032
  10:17:45 STEP 1819 H= 454:45 +CPU= 18.445
  10:18:04 STEP 1820 H= 455:00 +CPU= 19.384
  10:18:24 STEP 1821 H= 455:15 +CPU= 19.484
  10:18:42 STEP 1822 H= 455:30 +CPU= 18.064
  10:18:59 STEP 1823 H= 455:45 +CPU= 16.771
  10:19:24 STEP 1824 H= 456:00 +CPU= 24.577
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001824
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001824
  10:19:40 STEP 1825 H= 456:15 +CPU= 16.736
  10:19:57 STEP 1826 H= 456:30 +CPU= 16.816
  10:20:17 STEP 1827 H= 456:45 +CPU= 20.002
  10:20:34 STEP 1828 H= 457:00 +CPU= 17.071
  10:20:51 STEP 1829 H= 457:15 +CPU= 16.814
  10:21:08 STEP 1830 H= 457:30 +CPU= 16.807
  10:21:25 STEP 1831 H= 457:45 +CPU= 16.938
  10:21:42 STEP 1832 H= 458:00 +CPU= 17.288
  10:21:59 STEP 1833 H= 458:15 +CPU= 16.821
  10:22:16 STEP 1834 H= 458:30 +CPU= 16.712
  10:22:33 STEP 1835 H= 458:45 +CPU= 16.869
  10:22:53 STEP 1836 H= 459:00 +CPU= 20.862
  10:23:10 STEP 1837 H= 459:15 +CPU= 16.697
  10:23:27 STEP 1838 H= 459:30 +CPU= 17.050
  10:23:44 STEP 1839 H= 459:45 +CPU= 16.872
  10:24:01 STEP 1840 H= 460:00 +CPU= 17.213
  10:24:21 STEP 1841 H= 460:15 +CPU= 18.019
  10:24:38 STEP 1842 H= 460:30 +CPU= 16.449
  10:24:58 STEP 1843 H= 460:45 +CPU= 19.201
  10:25:15 STEP 1844 H= 461:00 +CPU= 17.126
  10:25:31 STEP 1845 H= 461:15 +CPU= 16.424
  10:25:48 STEP 1846 H= 461:30 +CPU= 16.289
  10:26:04 STEP 1847 H= 461:45 +CPU= 16.638
  10:26:25 STEP 1848 H= 462:00 +CPU= 20.453
  10:26:40 STEP 1849 H= 462:15 +CPU= 15.508
  10:26:56 STEP 1850 H= 462:30 +CPU= 15.656
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001848
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001848
  10:27:12 STEP 1851 H= 462:45 +CPU= 16.417
  10:27:30 STEP 1852 H= 463:00 +CPU= 17.267
  10:27:47 STEP 1853 H= 463:15 +CPU= 17.119
  10:28:04 STEP 1854 H= 463:30 +CPU= 16.896
  10:28:20 STEP 1855 H= 463:45 +CPU= 16.178
  10:28:37 STEP 1856 H= 464:00 +CPU= 17.211
  10:28:54 STEP 1857 H= 464:15 +CPU= 17.211
  10:29:12 STEP 1858 H= 464:30 +CPU= 17.562
  10:29:30 STEP 1859 H= 464:45 +CPU= 17.970
  10:29:52 STEP 1860 H= 465:00 +CPU= 21.710
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001860
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001860
  10:30:09 STEP 1861 H= 465:15 +CPU= 17.092
  10:30:27 STEP 1862 H= 465:30 +CPU= 17.756
  10:30:44 STEP 1863 H= 465:45 +CPU= 17.443
  10:31:02 STEP 1864 H= 466:00 +CPU= 17.772
  10:31:20 STEP 1865 H= 466:15 +CPU= 17.584
  10:31:38 STEP 1866 H= 466:30 +CPU= 17.736
  10:31:56 STEP 1867 H= 466:45 +CPU= 17.973
  10:32:14 STEP 1868 H= 467:00 +CPU= 18.164
  10:32:31 STEP 1869 H= 467:15 +CPU= 17.519
  10:32:49 STEP 1870 H= 467:30 +CPU= 17.953
  10:33:08 STEP 1871 H= 467:45 +CPU= 18.256
  10:33:35 STEP 1872 H= 468:00 +CPU= 27.275
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001872
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001860
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a1oc_2016092300_20_1018_12290499_0_r1220820879_18.zip
Uploading the intermediate file: upload_file_18.zip
  10:33:54 STEP 1873 H= 468:15 +CPU= 18.332
  10:34:11 STEP 1874 H= 468:30 +CPU= 17.476
  10:34:32 STEP 1875 H= 468:45 +CPU= 21.125
  10:34:56 STEP 1876 H= 469:00 +CPU= 22.924
  10:35:14 STEP 1877 H= 469:15 +CPU= 18.305
  10:35:31 STEP 1878 H= 469:30 +CPU= 17.341
  10:35:49 STEP 1879 H= 469:45 +CPU= 18.052
  10:36:08 STEP 1880 H= 470:00 +CPU= 18.200
  10:36:25 STEP 1881 H= 470:15 +CPU= 17.732
  10:36:42 STEP 1882 H= 470:30 +CPU= 16.813
  10:37:01 STEP 1883 H= 470:45 +CPU= 18.195
  10:37:23 STEP 1884 H= 471:00 +CPU= 21.983
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001884
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001884
  10:37:41 STEP 1885 H= 471:15 +CPU= 18.192
  10:37:59 STEP 1886 H= 471:30 +CPU= 17.936
  10:38:17 STEP 1887 H= 471:45 +CPU= 18.421
  10:38:36 STEP 1888 H= 472:00 +CPU= 18.382
  10:38:54 STEP 1889 H= 472:15 +CPU= 17.755
  10:39:11 STEP 1890 H= 472:30 +CPU= 17.466
  10:39:29 STEP 1891 H= 472:45 +CPU= 18.021
  10:39:47 STEP 1892 H= 473:00 +CPU= 17.596
  10:40:05 STEP 1893 H= 473:15 +CPU= 18.304
  10:40:23 STEP 1894 H= 473:30 +CPU= 17.750
  10:40:40 STEP 1895 H= 473:45 +CPU= 17.474
  10:41:03 STEP 1896 H= 474:00 +CPU= 22.364
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001896
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001896
  10:41:21 STEP 1897 H= 474:15 +CPU= 18.319
  10:41:39 STEP 1898 H= 474:30 +CPU= 17.728
  10:41:57 STEP 1899 H= 474:45 +CPU= 17.456
  10:42:15 STEP 1900 H= 475:00 +CPU= 18.492
  10:42:33 STEP 1901 H= 475:15 +CPU= 17.510
  10:42:50 STEP 1902 H= 475:30 +CPU= 17.048
  10:43:08 STEP 1903 H= 475:45 +CPU= 17.880
  10:43:26 STEP 1904 H= 476:00 +CPU= 18.034
  10:43:44 STEP 1905 H= 476:15 +CPU= 18.616
  10:44:03 STEP 1906 H= 476:30 +CPU= 18.121
  10:44:21 STEP 1907 H= 476:45 +CPU= 18.081
  10:44:43 STEP 1908 H= 477:00 +CPU= 21.899
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001908
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001908
  10:45:01 STEP 1909 H= 477:15 +CPU= 17.785
  10:45:18 STEP 1910 H= 477:30 +CPU= 17.481
  10:45:36 STEP 1911 H= 477:45 +CPU= 17.446
  10:45:54 STEP 1912 H= 478:00 +CPU= 17.939
  10:46:12 STEP 1913 H= 478:15 +CPU= 17.892
  10:46:29 STEP 1914 H= 478:30 +CPU= 17.824
  10:46:47 STEP 1915 H= 478:45 +CPU= 17.413
  10:47:06 STEP 1916 H= 479:00 +CPU= 18.720
  10:47:24 STEP 1917 H= 479:15 +CPU= 17.839
  10:47:41 STEP 1918 H= 479:30 +CPU= 17.280
  10:47:59 STEP 1919 H= 479:45 +CPU= 18.065
  10:48:24 STEP 1920 H= 480:00 +CPU= 24.580
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  480.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I      0.5445963899E-21      I       0.3148298444E-05      I
ICloud fractionI      0.0000000000E+00      I       0.4782506317E-01      I
I  Relat. Hum. I      0.1360828022E+00      I       0.3378949289E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.9080728500E+03      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1920 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001920 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001920
 MPI-TASK:   1 -    312413876 BYTES IN      7 RECORDS TRANSFERRED IN    0.0006 SECONDS520689.8 Mbytes/s, TOTAL TIME=    0.0074( 8.1%)
 10:48:24 STEP 1920 H= 480:00 +CPU= 24.580
 END CNT3
 NSTEP =  1920 CNT0   000000000  17.380  17.380  17.487   0.994               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           46/  1069125708 WRITTEN         8580/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1    153    0       46      1044068     8580 ************  1229.8    10.8  1219.0    15.3  1231.1
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 6495392.0 6495392.0    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 6474902.8 6474902.8     99.68      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  313    3064.6    3064.6     14.77      0.00
   9 SPCM     - SPECTRAL COMP.                     288     391.8     391.8      1.74      0.00
  10 SCAN2M - PHYSICS                              289    8375.6    8375.6     37.27      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 24   51060.0   51060.0     18.87      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 13      18.7      18.7      0.00      0.00
  17 GRIDFPOS IN CNT4                               24      22.7      22.7      0.01      0.00
  27 RESTART FILES - WRITING                         6  204239.8  204239.8     18.87      0.00
  28 RESTART FILES - READING                         1   17185.6   17185.6      0.26      0.00
  29 SU4FPOS IN CNT4                                24       0.0       0.0      0.00      0.00
  30 DYNFPOS IN CNT4                                24    3861.4    3861.4      1.43      0.00
  31 POSDDH IN STEPO                                 2       1.0       1.0      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                    289    4155.9    4155.9     18.49      0.00
  39 SU0YOMB                                         1    2816.7    2816.7      0.04      0.00
  51 SCAN2M   - SL COMM. PART 1                    289      15.4      15.4      0.07      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              288      90.9      90.9      0.40      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              288      43.4      43.4      0.19      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 13       0.1       0.1      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM      628     277.5     277.5      2.68      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM       600     149.6     149.6      1.38      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM        600      63.7      63.7      0.59      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM       628      73.4      73.4      0.71      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1     104.0     104.0      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION            628      64.9      64.9      0.63      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION            600      29.9      29.9      0.28      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION            628      26.3      26.3      0.25      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION            600      14.9      14.9      0.14      0.00
 400 GSTATS                                      34412       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 31964       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME     6495.4 CPU TIME    5273.4 VECTOR TIME     5273.4


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY      266.9
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMGGh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMSHh7zg+001920
Moving to projects directory: /var/lib/boinc-client/slots/0/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/slots/0/NODE.001_01
Adding to the zip: /var/lib/boinc-client/slots/0/ifs.stat
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMGGh7zg+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMUAh7zg+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12290499/ICMSHh7zg+001908
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a1oc_2016092300_20_1018_12290499_0_r1220820879_19.zip
Uploading the final file: upload_file_19.zip
Uploading trickle at timestep: 1727100
10:51:51 (4406): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
16 Jun 2024 01:05:52 1535694 22444095 oifs_43r3_bl_a1oc_2016092300_20_1018_12290499_0 1,727,100 37,632 0.0218
15 Jun 2024 04:48:24 1535694 22444095 oifs_43r3_bl_a1oc_2016092300_20_1018_12290499_0 1,306,800 27,933 0.0214


©2024 climateprediction.net