climateprediction.net home page
Task 16162571

Task 16162571

Name hadcm3n_ofaf_1900_40_008474938_1
Workunit 8625777
Created 28 Dec 2013, 3:29:39 UTC
Sent 28 Dec 2013, 3:29:46 UTC
Report deadline 29 Mar 2014, 10:56:57 UTC
Received 31 Dec 2013, 20:06:58 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1297364
Run time 2 days 15 hours 50 min 34 sec
CPU time 2 days 14 hours 38 min 21 sec
Validate state Invalid
Credit 933.12
Device peak FLOPS 2.27 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr 
<core_client_version>7.1.0</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
00:37:19 (2903): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:38:40 (3354): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:38:41 (3354): No heartbeat from core client for 30 sec - exiting
00:38:42 (3354): No heartbeat from core client for 30 sec - exiting
00:43:08 (3375): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:43:17 (3404): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:43:29 (3776): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:49:32 (3987): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
11:03:54 (5069): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:05:56 (5256): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:15:11 (5284): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:18:57 (5907): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:19:53 (5933): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:21:44 (5951): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
20:57:51 (6827): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:08:17 (8626): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
23:21:35 (9304): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:30:45 (9970): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:46:32 (10612): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:47:04 (11315): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:47:05 (11315): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
08:07:00 (12134): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:07:55 (12491): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:12:59 (12514): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:24:11 (12820): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:25:41 (12820): No heartbeat from core client for 30 sec - exiting
10:26:37 (13222): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:29:43 (13247): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
12:28:37 (13583): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
12:30:07 (13908): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:57:29 (13947): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:59:12 (15252): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:03:35 (15319): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:04:18 (15319): No heartbeat from core client for 30 sec - exiting
16:04:19 (15319): No heartbeat from core client for 30 sec - exiting
16:05:17 (15487): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Atmos Hold Restart file rename failed on atmos_restart.hold
CPDN Monitor - Quit request from BOINC...
00:48:04 (16868): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:48:20 (16868): No heartbeat from core client for 30 sec - exiting
Atmos Hold Restart file rename failed on atmos_restart.hold
01:57:11 (17218): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:59:41 (17663): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:10:57 (17684): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:14:35 (18245): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:14:36 (18245): No heartbeat from core client for 30 sec - exiting
04:14:37 (18245): No heartbeat from core client for 30 sec - exiting
04:14:38 (18245): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
Atmos Hold Restart file rename failed on atmos_restart.hold
05:28:21 (18665): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:28:27 (18665): No heartbeat from core client for 30 sec - exiting
05:28:28 (18665): No heartbeat from core client for 30 sec - exiting
05:29:52 (18826): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Atmos Hold Restart file rename failed on atmos_restart.hold
06:28:26 (18847): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:30:37 (19033): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
08:46:37 (19289): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:51:36 (19875): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
13:09:46 (20524): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:13:49 (20814): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:18:43 (21082): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:42:45 (21465): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:51:47 (22638): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:53:27 (22638): No heartbeat from core client for 30 sec - exiting
19:53:28 (22638): No heartbeat from core client for 30 sec - exiting
19:53:29 (22638): No heartbeat from core client for 30 sec - exiting
19:53:30 (22638): No heartbeat from core client for 30 sec - exiting
19:53:31 (22638): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
22:44:14 (23569): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
23:49:46 (23770): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:51:58 (24220): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:52:30 (24488): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:56:32 (24513): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:04:43 (24871): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:14:11 (25295): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:16:24 (25295): No heartbeat from core client for 30 sec - exiting
07:18:31 (25813): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:19:52 (26061): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:20:14 (26061): No heartbeat from core client for 30 sec - exiting
07:20:15 (26061): No heartbeat from core client for 30 sec - exiting
07:20:16 (26061): No heartbeat from core client for 30 sec - exiting
07:20:17 (26061): No heartbeat from core client for 30 sec - exiting
07:20:18 (26061): No heartbeat from core client for 30 sec - exiting
07:20:19 (26061): No heartbeat from core client for 30 sec - exiting
07:20:20 (26061): No heartbeat from core client for 30 sec - exiting
07:20:21 (26061): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
13:51:05 (26945): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:51:06 (26945): No heartbeat from core client for 30 sec - exiting
SIGABRT: abort called
Stack trace (9 frames):
/usr/local/ondrejch/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf7714400]
[0xf7714430]
/usr/lib/libc.so.6(gsignal+0x46)[0xf751f936]
/usr/lib/libc.so.6(abort+0x143)[0xf7521173]
/usr/local/ondrejch/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/usr/local/ondrejch/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/usr/local/ondrejch/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/usr/lib/libc.so.6(__libc_start_main+0xf3)[0xf750a963]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=27232, iMonCtr=1
Model crash detected, will try to restart...
13:52:00 (27232): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
SIGABRT: abort called
Stack trace (9 frames):
/usr/local/ondrejch/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf77bb400]
[0xf77bb430]
/usr/lib/libc.so.6(gsignal+0x46)[0xf75c6936]
/usr/lib/libc.so.6(abort+0x143)[0xf75c8173]
/usr/local/ondrejch/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/usr/local/ondrejch/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/usr/local/ondrejch/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/usr/lib/libc.so.6(__libc_start_main+0xf3)[0xf75b1963]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=27259, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/usr/local/ondrejch/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf7763400]
[0xf7763430]
/usr/lib/libc.so.6(gsignal+0x46)[0xf756e936]
/usr/lib/libc.so.6(abort+0x143)[0xf7570173]
/usr/local/ondrejch/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/usr/local/ondrejch/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/usr/local/ondrejch/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/usr/lib/libc.so.6(__libc_start_main+0xf3)[0xf7559963]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=27259, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/usr/local/ondrejch/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf77ca400]
[0xf77ca430]
/usr/lib/libc.so.6(gsignal+0x46)[0xf75d5936]
/usr/lib/libc.so.6(abort+0x143)[0xf75d7173]
/usr/local/ondrejch/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/usr/local/ondrejch/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/usr/local/ondrejch/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/usr/lib/libc.so.6(__libc_start_main+0xf3)[0xf75c0963]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=27259, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/usr/local/ondrejch/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf77c3400]
[0xf77c3430]
/usr/lib/libc.so.6(gsignal+0x46)[0xf75ce936]
/usr/lib/libc.so.6(abort+0x143)[0xf75d0173]
/usr/local/ondrejch/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/usr/local/ondrejch/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/usr/local/ondrejch/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/usr/lib/libc.so.6(__libc_start_main+0xf3)[0xf75b9963]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=27259, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/usr/local/ondrejch/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf7774400]
[0xf7774430]
/usr/lib/libc.so.6(gsignal+0x46)[0xf757f936]
/usr/lib/libc.so.6(abort+0x143)[0xf7581173]
/usr/local/ondrejch/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/usr/local/ondrejch/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/usr/local/ondrejch/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/usr/lib/libc.so.6(__libc_start_main+0xf3)[0xf756a963]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=27259, iMonCtr=1
Model crash detected, will try to restart...
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
30 Dec 2013 23:07:44 1297364 16162571 hadcm3n_ofaf_1900_40_008474938_1 77,760 181,130 2.3293
29 Dec 2013 21:59:24 1297364 16162571 hadcm3n_ofaf_1900_40_008474938_1 51,840 121,563 2.3450
29 Dec 2013 03:36:25 1297364 16162571 hadcm3n_ofaf_1900_40_008474938_1 25,920 62,232 2.4009


©2024 climateprediction.net