climateprediction.net home page
Task 16007355

Task 16007355

Name hadcm3n_o1g1_1980_40_008386445_2
Workunit 8537304
Created 7 Sep 2013, 21:49:42 UTC
Sent 7 Sep 2013, 21:49:45 UTC
Report deadline 8 Dec 2013, 5:16:56 UTC
Received 9 Oct 2013, 22:45:49 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1376550
Run time 29 days 11 hours 35 min 21 sec
CPU time 28 days 1 hours 8 min 22 sec
Validate state Invalid
Credit 9,642.24
Device peak FLOPS 1.65 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
windows_intelx86
Stderr 
<core_client_version>7.0.64</core_client_version>
<![CDATA[
<message>
The device does not recognize the command.
 (0x16) - exit code 22 (0x16)
</message>
<stderr_txt>
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3668, iMonCtr=1
Model crash detected, will try to restart...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
01:08:21 (5384): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:08:49 (5384): No heartbeat from core client for 30 sec - exiting
01:08:50 (5384): No heartbeat from core client for 30 sec - exiting
01:08:51 (5384): No heartbeat from core client for 30 sec - exiting
01:08:52 (5384): No heartbeat from core client for 30 sec - exiting
01:08:53 (5384): No heartbeat from core client for 30 sec - exiting
01:08:55 (5384): No heartbeat from core client for 30 sec - exiting
01:08:56 (5384): No heartbeat from core client for 30 sec - exiting
01:08:57 (5384): No heartbeat from core client for 30 sec - exiting
01:08:58 (5384): No heartbeat from core client for 30 sec - exiting
01:08:59 (5384): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
02:41:42 (4036): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:42:10 (4036): No heartbeat from core client for 30 sec - exiting
02:42:11 (4036): No heartbeat from core client for 30 sec - exiting
02:42:13 (4036): No heartbeat from core client for 30 sec - exiting
02:42:14 (4036): No heartbeat from core client for 30 sec - exiting
02:42:15 (4036): No heartbeat from core client for 30 sec - exiting
02:42:16 (4036): No heartbeat from core client for 30 sec - exiting
02:42:17 (4036): No heartbeat from core client for 30 sec - exiting
02:42:18 (4036): No heartbeat from core client for 30 sec - exiting
02:42:19 (4036): No heartbeat from core client for 30 sec - exiting
02:42:20 (4036): No heartbeat from core client for 30 sec - exiting
05:55:20 (8280): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:21:06 (8472): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:22:04 (8748): No heartbeat from core client for 30 sec - exiting
19:22:31 (8748): No heartbeat from core client for 30 sec - exiting
19:22:32 (8748): No heartbeat from core client for 30 sec - exiting
19:22:34 (8748): No heartbeat from core client for 30 sec - exiting
19:22:35 (8748): No heartbeat from core client for 30 sec - exiting
19:22:36 (8748): No heartbeat from core client for 30 sec - exiting
19:22:37 (8748): No heartbeat from core client for 30 sec - exiting
19:22:38 (8748): No heartbeat from core client for 30 sec - exiting
19:22:39 (8748): No heartbeat from core client for 30 sec - exiting
19:22:40 (8748): No heartbeat from core client for 30 sec - exiting
19:22:41 (8748): No heartbeat from core client for 30 sec - exiting
19:22:42 (8748): No heartbeat from core client for 30 sec - exiting
19:22:43 (8748): No heartbeat from core client for 30 sec - exiting
19:22:44 (8748): No heartbeat from core client for 30 sec - exiting
19:22:46 (8748): No heartbeat from core client for 30 sec - exiting
19:22:47 (8748): No heartbeat from core client for 30 sec - exiting
19:22:48 (8748): No heartbeat from core client for 30 sec - exiting
19:22:49 (8748): No heartbeat from core client for 30 sec - exiting
19:22:50 (8748): No heartbeat from core client for 30 sec - exiting
19:22:51 (8748): No heartbeat from core client for 30 sec - exiting
19:22:52 (8748): No heartbeat from core client for 30 sec - exiting
19:22:53 (8748): No heartbeat from core client for 30 sec - exiting
19:22:54 (8748): No heartbeat from core client for 30 sec - exiting
19:22:55 (8748): No heartbeat from core client for 30 sec - exiting
19:22:56 (8748): No heartbeat from core client for 30 sec - exiting
19:22:58 (8748): No heartbeat from core client for 30 sec - exiting
19:22:59 (8748): No heartbeat from core client for 30 sec - exiting
19:23:00 (8748): No heartbeat from core client for 30 sec - exiting
19:23:01 (8748): No heartbeat from core client for 30 sec - exiting
19:23:02 (8748): No heartbeat from core client for 30 sec - exiting
19:23:03 (8748): No heartbeat from core client for 30 sec - exiting
19:23:04 (8748): No heartbeat from core client for 30 sec - exiting
19:23:05 (8748): No heartbeat from core client for 30 sec - exiting
19:23:06 (8748): No heartbeat from core client for 30 sec - exiting
19:23:07 (8748): No heartbeat from core client for 30 sec - exiting
19:23:08 (8748): No heartbeat from core client for 30 sec - exiting
19:23:10 (8748): No heartbeat from core client for 30 sec - exiting
19:23:11 (8748): No heartbeat from core client for 30 sec - exiting
19:23:12 (8748): No heartbeat from core client for 30 sec - exiting
19:23:13 (8748): No heartbeat from core client for 30 sec - exiting
19:23:14 (8748): No heartbeat from core client for 30 sec - exiting
19:23:15 (8748): No heartbeat from core client for 30 sec - exiting
19:23:16 (8748): No heartbeat from core client for 30 sec - exiting
19:23:17 (8748): No heartbeat from core client for 30 sec - exiting
19:23:18 (8748): No heartbeat from core client for 30 sec - exiting
19:23:19 (8748): No heartbeat from core client for 30 sec - exiting
19:23:21 (8748): No heartbeat from core client for 30 sec - exiting
19:23:22 (8748): No heartbeat from core client for 30 sec - exiting
19:23:23 (8748): No heartbeat from core client for 30 sec - exiting
19:23:24 (8748): No heartbeat from core client for 30 sec - exiting
19:23:25 (8748): No heartbeat from core client for 30 sec - exiting
19:23:26 (8748): No heartbeat from core client for 30 sec - exiting
19:23:27 (8748): No heartbeat from core client for 30 sec - exiting
19:23:28 (8748): No heartbeat from core client for 30 sec - exiting
19:23:29 (8748): No heartbeat from core client for 30 sec - exiting
19:23:30 (8748): No heartbeat from core client for 30 sec - exiting
19:23:31 (8748): No heartbeat from core client for 30 sec - exiting
19:23:33 (8748): No heartbeat from core client for 30 sec - exiting
19:23:34 (8748): No heartbeat from core client for 30 sec - exiting
19:23:35 (8748): No heartbeat from core client for 30 sec - exiting
19:23:36 (8748): No heartbeat from core client for 30 sec - exiting
19:23:37 (8748): No heartbeat from core client for 30 sec - exiting
19:23:38 (8748): No heartbeat from core client for 30 sec - exiting
19:23:39 (8748): No heartbeat from core client for 30 sec - exiting
19:23:40 (8748): No heartbeat from core client for 30 sec - exiting
19:23:41 (8748): No heartbeat from core client for 30 sec - exiting
19:23:42 (8748): No heartbeat from core client for 30 sec - exiting
19:23:43 (8748): No heartbeat from core client for 30 sec - exiting
19:23:45 (8748): No heartbeat from core client for 30 sec - exiting
19:23:46 (8748): No heartbeat from core client for 30 sec - exiting
19:23:47 (8748): No heartbeat from core client for 30 sec - exiting
19:23:48 (8748): No heartbeat from core client for 30 sec - exiting
19:23:49 (8748): No heartbeat from core client for 30 sec - exiting
19:23:50 (8748): No heartbeat from core client for 30 sec - exiting
19:23:51 (8748): No heartbeat from core client for 30 sec - exiting
19:23:52 (8748): No heartbeat from core client for 30 sec - exiting
19:23:53 (8748): No heartbeat from core client for 30 sec - exiting
19:23:54 (8748): No heartbeat from core client for 30 sec - exiting
19:23:55 (8748): No heartbeat from core client for 30 sec - exiting
19:23:57 (8748): No heartbeat from core client for 30 sec - exiting
19:23:58 (8748): No heartbeat from core client for 30 sec - exiting
19:23:59 (8748): No heartbeat from core client for 30 sec - exiting
19:24:00 (8748): No heartbeat from core client for 30 sec - exiting
19:24:01 (8748): No heartbeat from core client for 30 sec - exiting
19:24:02 (8748): No heartbeat from core client for 30 sec - exiting
19:24:03 (8748): No heartbeat from core client for 30 sec - exiting
19:24:04 (8748): No heartbeat from core client for 30 sec - exiting
19:24:05 (8748): No heartbeat from core client for 30 sec - exiting
19:24:06 (8748): No heartbeat from core client for 30 sec - exiting
19:24:07 (8748): No heartbeat from core client for 30 sec - exiting
19:24:09 (8748): No heartbeat from core client for 30 sec - exiting
19:24:10 (8748): No heartbeat from core client for 30 sec - exiting
19:24:11 (8748): No heartbeat from core client for 30 sec - exiting
19:24:12 (8748): No heartbeat from core client for 30 sec - exiting
19:24:13 (8748): No heartbeat from core client for 30 sec - exiting
19:24:14 (8748): No heartbeat from core client for 30 sec - exiting
19:24:15 (8748): No heartbeat from core client for 30 sec - exiting
19:24:16 (8748): No heartbeat from core client for 30 sec - exiting
19:24:17 (8748): No heartbeat from core client for 30 sec - exiting
19:24:18 (8748): No heartbeat from core client for 30 sec - exiting
19:24:19 (8748): No heartbeat from core client for 30 sec - exiting
19:24:21 (8748): No heartbeat from core client for 30 sec - exiting
19:24:22 (8748): No heartbeat from core client for 30 sec - exiting
19:24:23 (8748): No heartbeat from core client for 30 sec - exiting
19:24:24 (8748): No heartbeat from core client for 30 sec - exiting
19:24:25 (8748): No heartbeat from core client for 30 sec - exiting
19:24:26 (8748): No heartbeat from core client for 30 sec - exiting
19:24:27 (8748): No heartbeat from core client for 30 sec - exiting
19:24:28 (8748): No heartbeat from core client for 30 sec - exiting
19:24:29 (8748): No heartbeat from core client for 30 sec - exiting
19:24:30 (8748): No heartbeat from core client for 30 sec - exiting
19:24:31 (8748): No heartbeat from core client for 30 sec - exiting
19:24:33 (8748): No heartbeat from core client for 30 sec - exiting
19:24:34 (8748): No heartbeat from core client for 30 sec - exiting
19:24:35 (8748): No heartbeat from core client for 30 sec - exiting
19:24:36 (8748): No heartbeat from core client for 30 sec - exiting
19:24:37 (8748): No heartbeat from core client for 30 sec - exiting
19:24:38 (8748): No heartbeat from core client for 30 sec - exiting
19:24:39 (8748): No heartbeat from core client for 30 sec - exiting
19:24:40 (8748): No heartbeat from core client for 30 sec - exiting
19:24:41 (8748): No heartbeat from core client for 30 sec - exiting
19:24:42 (8748): No heartbeat from core client for 30 sec - exiting
19:24:43 (8748): No heartbeat from core client for 30 sec - exiting
19:24:45 (8748): No heartbeat from core client for 30 sec - exiting
19:24:46 (8748): No heartbeat from core client for 30 sec - exiting
19:24:47 (8748): No heartbeat from core client for 30 sec - exiting
19:24:48 (8748): No heartbeat from core client for 30 sec - exiting
19:24:49 (8748): No heartbeat from core client for 30 sec - exiting
19:24:50 (8748): No heartbeat from core client for 30 sec - exiting
19:24:51 (8748): No heartbeat from core client for 30 sec - exiting
19:24:52 (8748): No heartbeat from core client for 30 sec - exiting
19:24:53 (8748): No heartbeat from core client for 30 sec - exiting
19:24:54 (8748): No heartbeat from core client for 30 sec - exiting
19:24:56 (8748): No heartbeat from core client for 30 sec - exiting
19:24:57 (8748): No heartbeat from core client for 30 sec - exiting
19:24:58 (8748): No heartbeat from core client for 30 sec - exiting
19:24:59 (8748): No heartbeat from core client for 30 sec - exiting
19:25:00 (8748): No heartbeat from core client for 30 sec - exiting
19:25:01 (8748): No heartbeat from core client for 30 sec - exiting
19:25:02 (8748): No heartbeat from core client for 30 sec - exiting
19:25:03 (8748): No heartbeat from core client for 30 sec - exiting
19:25:04 (8748): No heartbeat from core client for 30 sec - exiting
19:25:05 (8748): No heartbeat from core client for 30 sec - exiting
19:25:06 (8748): No heartbeat from core client for 30 sec - exiting
19:25:08 (8748): No heartbeat from core client for 30 sec - exiting
19:25:09 (8748): No heartbeat from core client for 30 sec - exiting
19:25:10 (8748): No heartbeat from core client for 30 sec - exiting
19:25:11 (8748): No heartbeat from core client for 30 sec - exiting
19:25:12 (8748): No heartbeat from core client for 30 sec - exiting
19:25:13 (8748): No heartbeat from core client for 30 sec - exiting
19:25:14 (8748): No heartbeat from core client for 30 sec - exiting
19:25:15 (8748): No heartbeat from core client for 30 sec - exiting
19:25:16 (8748): No heartbeat from core client for 30 sec - exiting
19:25:17 (8748): No heartbeat from core client for 30 sec - exiting
19:25:18 (8748): No heartbeat from core client for 30 sec - exiting
19:25:20 (8748): No heartbeat from core client for 30 sec - exiting
19:25:21 (8748): No heartbeat from core client for 30 sec - exiting
19:25:22 (8748): No heartbeat from core client for 30 sec - exiting
19:25:23 (8748): No heartbeat from core client for 30 sec - exiting
19:25:24 (8748): No heartbeat from core client for 30 sec - exiting
19:25:25 (8748): No heartbeat from core client for 30 sec - exiting
19:25:26 (8748): No heartbeat from core client for 30 sec - exiting
19:25:27 (8748): No heartbeat from core client for 30 sec - exiting
19:25:28 (8748): No heartbeat from core client for 30 sec - exiting
19:25:29 (8748): No heartbeat from core client for 30 sec - exiting
19:25:30 (8748): No heartbeat from core client for 30 sec - exiting
19:25:32 (8748): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:36:15 (7492): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
02:40:11 (5520): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4328, iMonCtr=1
Model crash detected, will try to restart...
Suspended CPDN Monitor - Suspend request from BOINC...
21:41:07 (2548): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Atmos Hold Restart file rename failed on atmos_restart.hold

Model crashed: INITDUMP: Wrong no of ocean prognostic fields                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    

Model crashed: INITDUMP: Wrong no of ocean prognostic fields                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    

Model crashed: INITDUMP: Wrong no of ocean prognostic fields                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    

Model crashed: INITDUMP: Wrong no of ocean prognostic fields                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    

Model crashed: INITDUMP: Wrong no of ocean prognostic fields                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    

Model crashed: INITDUMP: Wrong no of ocean prognostic fields                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
08 Oct 2013 16:23:54 1274848 16007355 hadcm3n_o1g1_1980_40_008386445_2 803,520 2,381,202 2.9635
07 Oct 2013 18:39:36 1274848 16007355 hadcm3n_o1g1_1980_40_008386445_2 777,600 2,303,598 2.9624
06 Oct 2013 20:57:11 1274848 16007355 hadcm3n_o1g1_1980_40_008386445_2 751,680 2,225,972 2.9613
05 Oct 2013 23:04:26 1274848 16007355 hadcm3n_o1g1_1980_40_008386445_2 725,760 2,148,081 2.9598
05 Oct 2013 01:25:52 1274848 16007355 hadcm3n_o1g1_1980_40_008386445_2 699,840 2,070,543 2.9586
04 Oct 2013 03:53:45 1274848 16007355 hadcm3n_o1g1_1980_40_008386445_2 673,920 1,993,358 2.9579
02 Oct 2013 05:32:33 1274848 16007355 hadcm3n_o1g1_1980_40_008386445_2 648,000 1,916,233 2.9571
01 Oct 2013 03:13:18 1274848 16007355 hadcm3n_o1g1_1980_40_008386445_2 622,080 1,836,698 2.9525
30 Sep 2013 01:06:45 1274848 16007355 hadcm3n_o1g1_1980_40_008386445_2 596,160 1,758,451 2.9496
28 Sep 2013 23:15:48 1274848 16007355 hadcm3n_o1g1_1980_40_008386445_2 570,240 1,679,717 2.9456
27 Sep 2013 21:35:01 1274848 16007355 hadcm3n_o1g1_1980_40_008386445_2 544,320 1,600,259 2.9399
26 Sep 2013 20:08:04 1274848 16007355 hadcm3n_o1g1_1980_40_008386445_2 518,400 1,536,469 2.9639
25 Sep 2013 21:07:34 1274848 16007355 hadcm3n_o1g1_1980_40_008386445_2 492,480 1,460,724 2.9661
25 Sep 2013 09:25:21 1274848 16007355 hadcm3n_o1g1_1980_40_008386445_2 466,560 1,384,675 2.9678
24 Sep 2013 00:24:45 1274848 16007355 hadcm3n_o1g1_1980_40_008386445_2 440,640 1,309,273 2.9713
23 Sep 2013 10:45:45 1274848 16007355 hadcm3n_o1g1_1980_40_008386445_2 414,720 1,233,561 2.9744
23 Sep 2013 10:45:45 1274848 16007355 hadcm3n_o1g1_1980_40_008386445_2 388,800 1,156,462 2.9744
21 Sep 2013 06:29:48 1274848 16007355 hadcm3n_o1g1_1980_40_008386445_2 362,880 1,079,044 2.9736
20 Sep 2013 08:09:06 1274848 16007355 hadcm3n_o1g1_1980_40_008386445_2 336,960 1,000,913 2.9704
19 Sep 2013 09:37:33 1274848 16007355 hadcm3n_o1g1_1980_40_008386445_2 311,040 922,114 2.9646


©2024 climateprediction.net