Task 15820198

Name	hadcm3n_o7l8_1980_40_008385044_0
Workunit	8535903
Created	3 Jun 2013, 1:50:57 UTC
Sent	3 Jun 2013, 1:59:01 UTC
Report deadline	2 Sep 2013, 9:26:12 UTC
Received	10 Sep 2013, 12:00:05 UTC
Server state	Over
Outcome	Success
Client state	Done
Exit status	0 (0x00000000)
Computer ID	1195874
Run time	70 days 16 hours 30 min 54 sec
CPU time	60 days 23 hours 14 min 30 sec
Validate state	Valid
Credit	12,441.60
Device peak FLOPS	0.79 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.17</core_client_version> <![CDATA[ <stderr_txt> CPDN Monitor - Quit request from BOINC... 10:54:55 (5033): No heartbeat from core client for 30 sec - exiting 10:55:07 (5033): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:55:09 (5033): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:19:29 (1529): No heartbeat from core client for 30 sec - exiting 13:19:59 (1529): No heartbeat from core client for 30 sec - exiting 13:20:00 (1529): No heartbeat from core client for 30 sec - exiting 13:20:01 (1529): No heartbeat from core client for 30 sec - exiting 13:20:02 (1529): No heartbeat from core client for 30 sec - exiting 13:20:03 (1529): No heartbeat from core client for 30 sec - exiting 13:20:04 (1529): No heartbeat from core client for 30 sec - exiting 13:20:05 (1529): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:20:06 (1529): No heartbeat from core client for 30 sec - exiting 13:20:07 (1529): No heartbeat from core client for 30 sec - exiting 13:20:08 (1529): No heartbeat from core client for 30 sec - exiting 13:20:09 (1529): No heartbeat from core client for 30 sec - exiting 13:20:10 (1529): No heartbeat from core client for 30 sec - exiting 13:21:15 (2765): No heartbeat from core client for 30 sec - exiting 13:21:16 (2765): No heartbeat from core client for 30 sec - exiting 13:21:17 (2765): No heartbeat from core client for 30 sec - exiting 13:21:18 (2765): No heartbeat from core client for 30 sec - exiting 13:21:19 (2765): No heartbeat from core client for 30 sec - exiting 13:21:20 (2765): No heartbeat from core client for 30 sec - exiting 13:21:21 (2765): No heartbeat from core client for 30 sec - exiting 13:21:22 (2765): No heartbeat from core client for 30 sec - exiting 13:21:23 (2765): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 15 received, exiting... Called boinc_finish 15:25:43 (1512): No heartbeat from core client for 30 sec - exiting 15:25:44 (1512): No heartbeat from core client for 30 sec - exiting 15:25:45 (1512): No heartbeat from core client for 30 sec - exiting 15:25:46 (1512): No heartbeat from core client for 30 sec - exiting 15:25:47 (1512): No heartbeat from core client for 30 sec - exiting 15:25:48 (1512): No heartbeat from core client for 30 sec - exiting 15:25:49 (1512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 08:31:46 (4683): No heartbeat from core client for 30 sec - exiting Signal 15 received, exiting... Called boinc_finish 08:32:14 (4683): No heartbeat from core client for 30 sec - exiting 08:32:15 (4683): No heartbeat from core client for 30 sec - exiting 02:02:20 (1446): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:04:46 (1446): No heartbeat from core client for 30 sec - exiting 02:04:47 (1446): No heartbeat from core client for 30 sec - exiting 02:04:48 (1446): No heartbeat from core client for 30 sec - exiting 07:47:52 (8700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:48:28 (8700): No heartbeat from core client for 30 sec - exiting 07:48:29 (8700): No heartbeat from core client for 30 sec - exiting 07:48:30 (8700): No heartbeat from core client for 30 sec - exiting 07:48:31 (8700): No heartbeat from core client for 30 sec - exiting 07:48:32 (8700): No heartbeat from core client for 30 sec - exiting 07:48:33 (8700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1501, iMonCtr=1 Model crash detected, will try to restart... Signal 15 received, exiting... Called boinc_finish SIGSEGV: segmentation violation Stack trace (7 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xc9d400] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805067e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xd15bd6] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... 04:17:49 (1160): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:17:50 (1160): No heartbeat from core client for 30 sec - exiting 07:56:44 (4829): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... 07:20:32 (4121):07:22:04 (1499): No heartbeat from core client for 30 sec - exiting 07:22:05 (1499): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:17:40 (1749): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:17:45 (1749): No heartbeat from core client for 30 sec - exiting 14:17:46 (1749): No heartbeat from core client for 30 sec - exiting 14:17:47 (1749): No heartbeat from core client for 30 sec - exiting 14:17:48 (1749): No heartbeat from core client for 30 sec - exiting 14:17:49 (1749): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:32:35 (2686): No heartbeat from core client for 30 sec - exiting 11:32:36 (2686): No heartbeat from core client for 30 sec - exiting 11:32:37 (2686): No heartbeat from core client for 30 sec - exiting 11:32:38 (2686): No heartbeat from core client for 30 sec - exiting 11:32:39 (2686): No heartbeat from core client for 30 sec - exiting 11:32:40 (2686): No heartbeat from core client for 30 sec - exiting 11:32:41 (2686): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:08:57 (2694): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:25:49 (4372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:26:03 (4372): No heartbeat from core client for 30 sec - exiting 05:26:04 (4372): No heartbeat from core client for 30 sec - exiting 05:26:05 (4372): No heartbeat from core client for 30 sec - exiting 05:26:06 (4372): No heartbeat from core client for 30 sec - exiting 05:26:48 (5594): No heartbeat from core client for 30 sec - exiting 05:26:53 (5594): No heartbeat from core client for 30 sec - exiting 05:26:54 (5594): No heartbeat from core client for 30 sec - exiting 05:26:55 (5594): No heartbeat from core client for 30 sec - exiting 05:26:56 (5594): No heartbeat from core client for 30 sec - exiting 05:26:57 (5594): No heartbeat from core client for 30 sec - exiting 05:26:58 (5594): No heartbeat from core client for 30 sec - exiting 05:26:59 (5594): No heartbeat from core client for 30 sec - exiting 05:27:00 (5594): No heartbeat from core client for 30 sec - exiting 05:27:01 (5594): No heartbeat from core client for 30 sec - exiting 05:27:02 (5594): No heartbeat from core client for 30 sec - exiting 05:27:03 (5594): No heartbeat from core client for 30 sec - exiting 05:27:04 (5594): No heartbeat from core client for 30 sec - exiting 05:27:05 (5594): No heartbeat from core client for 30 sec - exiting 05:27:06 (5594): No heartbeat from core client for 30 sec - exiting 05:27:07 (5594): No heartbeat from core client for 30 sec - exiting 05:27:08 (5594): No heartbeat from core client for 30 sec - exiting 05:27:09 (5594): No heartbeat from core client for 30 sec - exiting 05:27:10 (5594): No heartbeat from core client for 30 sec - exiting 05:27:11 (5594): No heartbeat from core client for 30 sec - exiting 05:27:12 (5594): No heartbeat from core client for 30 sec - exiting 05:27:13 (5594): No heartbeat from core client for 30 sec - exiting 05:27:14 (5594): No heartbeat from core client for 30 sec - exiting 05:27:15 (5594): No heartbeat from core client for 30 sec - exiting 05:27:16 (5594): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish 08:06:55 (1514): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:06:56 (1514): No heartbeat from core client for 30 sec - exiting 08:06:57 (1514): No heartbeat from core client for 30 sec - exiting 08:06:58 (1514): No heartbeat from core client for 30 sec - exiting 08:06:59 (1514): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 10:08:24 (1529): No heartbeat from core client for 30 sec - exiting 10:08:28 (1529): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:08:29 (1529): No heartbeat from core client for 30 sec - exiting 10:08:30 (1529): No heartbeat from core client for 30 sec - exiting 10:08:31 (1529): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1566, iMonCtr=1 Model crash detected, will try to restart... Signal 15 received, exiting... Called boinc_finish SIGSEGV: segmentation violation Stack trace (7 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0x720400] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805067e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xce3bd6] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... 08:45:54 (1502): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:45:55 (1502): No heartbeat from core client for 30 sec - exiting Signal 15 received, exiting... Called boinc_finish 01:48:46 (1527): No heartbeat from core client for 30 sec - exiting 01:49:19 (1527): No heartbeat from core client for 30 sec - exiting 01:49:20 (1527): No heartbeat from core client for 30 sec - exiting 01:49:21 (1527): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:49:22 (1527): No heartbeat from core client for 30 sec - exiting 01:49:23 (1527): No heartbeat from core client for 30 sec - exiting 01:49:24 (1527): No heartbeat from core client for 30 sec - exiting 01:49:25 (1527): No heartbeat from core client for 30 sec - exiting 01:49:26 (1527): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 06:20:38 (1511): No heartbeat from core client for 30 sec - exiting 06:20:39 (1511): No heartbeat from core client for 30 sec - exiting 06:20:40 (1511): No heartbeat from core client for 30 sec - exiting 06:20:41 (1511): No heartbeat from core client for 30 sec - exiting 06:20:42 (1511): No heartbeat from core client for 30 sec - exiting 06:20:43 (1511): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:42:42 (1786): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:42:44 (1786): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 11:35:40 (10801): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:35:42 (10801): No heartbeat from core client for 30 sec - exiting 11:35:43 (10801): No heartbeat from core client for 30 sec - exiting 11:35:44 (10801): No heartbeat from core client for 30 sec - exiting 11:35:45 (10801): No heartbeat from core client for 30 sec - exiting 11:40:41 (10814): No heartbeat from core client for 30 sec - exiting 11:40:55 (10814): No heartbeat from core client for 30 sec - exiting 11:40:56 (10814): No heartbeat from core client for 30 sec - exiting 11:40:57 (10814): No heartbeat from core client for 30 sec - exiting 11:40:58 (10814): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:40:59 (10814): No heartbeat from core client for 30 sec - exiting 11:41:00 (10814): No heartbeat from core client for 30 sec - exiting 11:41:01 (10814): No heartbeat from core client for 30 sec - exiting 11:41:02 (10814): No heartbeat from core client for 30 sec - exiting 11:41:03 (10814): No heartbeat from core client for 30 sec - exiting 11:41:04 (10814): No heartbeat from core client for 30 sec - exiting 11:42:18 (10835): No heartbeat from core client for 30 sec - exiting 11:42:19 (10835): No heartbeat from core client for 30 sec - exiting 11:42:20 (10835): No heartbeat from core client for 30 sec - exiting 11:42:21 (10835): No heartbeat from core client for 30 sec - exiting 11:42:22 (10835): No heartbeat from core client for 30 sec - exiting 11:42:23 (10835): No heartbeat from core client for 30 sec - exiting 11:42:24 (10835): No heartbeat from core client for 30 sec - exiting 11:42:25 (10835): No heartbeat from core client for 30 sec - exiting 11:42:26 (10835): No heartbeat from core client for 30 sec - exiting 11:42:27 (10835): No heartbeat from core client for 30 sec - exiting 11:42:28 (10835): No heartbeat from core client for 30 sec - exiting 11:42:29 (10835): No heartbeat from core client for 30 sec - exiting 11:42:30 (10835): No heartbeat from core client for 30 sec - exiting 11:42:31 (10835): No heartbeat from core client for 30 sec - exiting 11:42:32 (10835): No heartbeat from core client for 30 sec - exiting 11:42:33 (10835): No heartbeat from core client for 30 sec - exiting 11:42:34 (10835): No heartbeat from core client for 30 sec - exiting 11:42:35 (10835): No heartbeat from core client for 30 sec - exiting 11:42:36 (10835): No heartbeat from core client for 30 sec - exiting 11:42:37 (10835): No heartbeat from core client for 30 sec - exiting 11:42:38 (10835): No heartbeat from core client for 30 sec - exiting 11:42:39 (10835): No heartbeat from core client for 30 sec - exiting 11:42:40 (10835): No heartbeat from core client for 30 sec - exiting 11:42:41 (10835): No heartbeat from core client for 30 sec - exiting 11:42:42 (10835): No heartbeat from core client for 30 sec - exiting 11:42:43 (10835): No heartbeat from core client for 30 sec - exiting 11:42:44 (10835): No heartbeat from core client for 30 sec - exiting 11:42:45 (10835): No heartbeat from core client for 30 sec - exiting 11:42:46 (10835): No heartbeat from core client for 30 sec - exiting 11:42:47 (10835): No heartbeat from core client for 30 sec - exiting 11:42:48 (10835): No heartbeat from core client for 30 sec - exiting 11:42:49 (10835): No heartbeat from core client for 30 sec - exiting 11:42:50 (10835): No heartbeat from core client for 30 sec - exiting 11:42:51 (10835): No heartbeat from core client for 30 sec - exiting 11:42:52 (10835): No heartbeat from core client for 30 sec - exiting 11:42:53 (10835): No heartbeat from core client for 30 sec - exiting 11:42:54 (10835): No heartbeat from core client for 30 sec - exiting 11:42:55 (10835): No heartbeat from core client for 30 sec - exiting 11:42:56 (10835): No heartbeat from core client for 30 sec - exiting 11:42:57 (10835): No heartbeat from core client for 30 sec - exiting 11:42:58 (10835): No heartbeat from core client for 30 sec - exiting 11:42:59 (10835): No heartbeat from core client for 30 sec - exiting 11:43:00 (10835): No heartbeat from core client for 30 sec - exiting 11:43:01 (10835): No heartbeat from core client for 30 sec - exiting 11:43:02 (10835): No heartbeat from core client for 30 sec - exiting 11:43:03 (10835): No heartbeat from core client for 30 sec - exiting 11:43:04 (10835): No heartbeat from core client for 30 sec - exiting 11:43:05 (10835): No heartbeat from core client for 30 sec - exiting 11:43:07 (10835): No heartbeat from core client for 30 sec - exiting 11:43:08 (10835): No heartbeat from core client for 30 sec - exiting 11:43:09 (10835): No heartbeat from core client for 30 sec - exiting 11:43:10 (10835): No heartbeat from core client for 30 sec - exiting 11:43:11 (10835): No heartbeat from core client for 30 sec - exiting 11:43:12 (10835): No heartbeat from core client for 30 sec - exiting 11:43:13 (10835): No heartbeat from core client for 30 sec - exiting 11:43:14 (10835): No heartbeat from core client for 30 sec - exiting 11:43:15 (10835): No heartbeat from core client for 30 sec - exiting 11:43:16 (10835): No heartbeat from core client for 30 sec - exiting 11:43:17 (10835): No heartbeat from core client for 30 sec - exiting 11:43:18 (10835): No heartbeat from core client for 30 sec - exiting 11:43:19 (10835): No heartbeat from core client for 30 sec - exiting 11:43:20 (10835): No heartbeat from core client for 30 sec - exiting 11:43:21 (10835): No heartbeat from core client for 30 sec - exiting 11:43:22 (10835): No heartbeat from core client for 30 sec - exiting 11:43:23 (10835): No heartbeat from core client for 30 sec - exiting 11:43:24 (10835): No heartbeat from core client for 30 sec - exiting 11:43:25 (10835): No heartbeat from core client for 30 sec - exiting 11:43:26 (10835): No heartbeat from core client for 30 sec - exiting 11:43:27 (10835): No heartbeat from core client for 30 sec - exiting 11:43:28 (10835): No heartbeat from core client for 30 sec - exiting 11:43:29 (10835): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:48:45 (10844): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:48:49 (10844): No heartbeat from core client for 30 sec - exiting 11:48:50 (10844): No heartbeat from core client for 30 sec - exiting 11:48:51 (10844): No heartbeat from core client for 30 sec - exiting 11:48:52 (10844): No heartbeat from core client for 30 sec - exiting 11:48:53 (10844): No heartbeat from core client for 30 sec - exiting 11:48:54 (10844): No heartbeat from core client for 30 sec - exiting 11:48:55 (10844): No heartbeat from core client for 30 sec - exiting 11:48:56 (10844): No heartbeat from core client for 30 sec - exiting 11:50:38 (10857): No heartbeat from core client for 30 sec - exiting 11:50:39 (10857): No heartbeat from core client for 30 sec - exiting 11:50:40 (10857): No heartbeat from core client for 30 sec - exiting 11:50:41 (10857): No heartbeat from core client for 30 sec - exiting 11:50:42 (10857): No heartbeat from core client for 30 sec - exiting 11:50:43 (10857): No heartbeat from core client for 30 sec - exiting 11:50:44 (10857): No heartbeat from core client for 30 sec - exiting 11:50:45 (10857): No heartbeat from core client for 30 sec - exiting 11:50:46 (10857): No heartbeat from core client for 30 sec - exiting 11:50:47 (10857): No heartbeat from core client for 30 sec - exiting 11:50:48 (10857): No heartbeat from core client for 30 sec - exiting 11:50:49 (10857): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10863, iMonCtr=1 Model crash detected, will try to restart... Signal 15 received, exiting... Called boinc_finish SIGSEGV: segmentation violation Stack trace (7 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xede400] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805067e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xcaebd6] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... 04:52:02 (1502): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:52:15 (1502): No heartbeat from core client for 30 sec - exiting 04:52:16 (1502): No heartbeat from core client for 30 sec - exiting 04:52:17 (1502): No heartbeat from core client for 30 sec - exiting 04:52:18 (1502): No heartbeat from core client for 30 sec - exiting 04:52:19 (1502): No heartbeat from core client for 30 sec - exiting 06:54:24 (3858): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 07:58:35 (12916): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:58:43 (12916): No heartbeat from core client for 30 sec - exiting Signal 15 received, exiting... Called boinc_finish SIGSEGV: segmentation violation Stack trace (7 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xf3c400] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805067e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0x3afbd6] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5117, iMonCtr=1 Model crash detected, will try to restart... Signal 15 received, exiting... Called boinc_finish SIGSEGV: segmentation violation Stack trace (6 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xfbd400] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8051147] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0x14cbd6] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish 08:00:39 (1516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4965, iMonCtr=1 Model crash detected, will try to restart... Signal 15 received, exiting... Called boinc_finish SIGSEGV: segmentation violation Stack trace (7 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0x803400] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805067e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0x21bbd6] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... 08:47:29 (1493): No heartbeat from core client for 30 sec - exiting 08:48:20 (1493): No heartbeat from core client for 30 sec - exiting 08:48:21 (1493): No heartbeat from core client for 30 sec - exiting 08:48:22 (1493): No heartbeat from core client for 30 sec - exiting 08:48:23 (1493): No heartbeat from core client for 30 sec - exiting 08:48:24 (1493): No heartbeat from core client for 30 sec - exiting 08:48:26 (1493): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:48:27 (1493): No heartbeat from core client for 30 sec - exiting 08:48:28 (1493): No heartbeat from core client for 30 sec - exiting 08:48:29 (1493): No heartbeat from core client for 30 sec - exiting 08:48:30 (1493): No heartbeat from core client for 30 sec - exiting 08:48:32 (1493): No heartbeat from core client for 30 sec - exiting 08:48:43 (1493): No heartbeat from core client for 30 sec - exiting 08:48:45 (1493): No heartbeat from core client for 30 sec - exiting 08:48:46 (1493): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish 17:45:19 (1315):Signal 15 received, exiting... Called boinc_finish No heartbeat from core client for 30 sec - exiting 17:45:21 (1315): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 12:38:54 (8693):Signal 15 received, exiting... Called boinc_finish No heartbeat from core client for 30 sec - exiting 12:38:55 (8693): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish 08:23:02 (1530): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:23:04 (1530): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish 08:32:03 (1146): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 22:25:01 (2797): No heartbeat from core client for 30 sec - exiting 22:25:04 (2797): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:25:05 (2797): No heartbeat from core client for 30 sec - exiting 22:25:06 (2797): No heartbeat from core client for 30 sec - exiting 22:25:07 (2797): No heartbeat from core client for 30 sec - exiting 22:25:08 (2797): No heartbeat from core client for 30 sec - exiting 22:25:09 (2797): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 07:43:20 (1508): No heartbeat from core client for 30 sec - exiting 07:44:15 (1508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 15 received, exiting... Called boinc_finish 07:44:16 (1508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2488, iMonCtr=1 Model crash detected, will try to restart... Signal 15 received, exiting... Called boinc_finish SIGSEGV: segmentation violation Stack trace (6 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xf84400] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8051147] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0x20bbd6] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6272, iMonCtr=1 Model crash detected, will try to restart... Signal 15 received, exiting... Called boinc_finish SIGSEGV: segmentation violation Stack trace (6 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0x6a1400] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8051147] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xa12bd6] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish Signal 15 received, exiting... Called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
10 Sep 2013 11:00:05	1195874	15820198	hadcm3n_o7l8_1980_40_008385044_0	1,036,800	5,265,485	5.0786
08 Sep 2013 15:13:58	1195874	15820198	hadcm3n_o7l8_1980_40_008385044_0	1,010,880	5,128,057	5.0729
06 Sep 2013 20:25:52	1195874	15820198	hadcm3n_o7l8_1980_40_008385044_0	984,960	4,991,372	5.0676
05 Sep 2013 00:01:05	1195874	15820198	hadcm3n_o7l8_1980_40_008385044_0	959,040	4,851,946	5.0592
03 Sep 2013 03:38:19	1195874	15820198	hadcm3n_o7l8_1980_40_008385044_0	933,120	4,715,512	5.0535
01 Sep 2013 06:28:16	1195874	15820198	hadcm3n_o7l8_1980_40_008385044_0	907,200	4,578,863	5.0472
30 Aug 2013 09:48:32	1195874	15820198	hadcm3n_o7l8_1980_40_008385044_0	881,280	4,441,610	5.0400
28 Aug 2013 12:49:32	1195874	15820198	hadcm3n_o7l8_1980_40_008385044_0	855,360	4,304,626	5.0325
26 Aug 2013 15:00:41	1195874	15820198	hadcm3n_o7l8_1980_40_008385044_0	829,440	4,168,168	5.0253
24 Aug 2013 17:35:11	1195874	15820198	hadcm3n_o7l8_1980_40_008385044_0	803,520	4,036,927	5.0241
23 Aug 2013 10:11:52	1195874	15820198	hadcm3n_o7l8_1980_40_008385044_0	777,600	3,899,774	5.0151
20 Aug 2013 23:48:21	1195874	15820198	hadcm3n_o7l8_1980_40_008385044_0	751,680	3,762,861	5.0059
18 Aug 2013 15:11:06	1195874	15820198	hadcm3n_o7l8_1980_40_008385044_0	725,760	3,626,158	4.9964
16 Aug 2013 19:11:47	1195874	15820198	hadcm3n_o7l8_1980_40_008385044_0	699,840	3,489,449	4.9861
14 Aug 2013 23:01:51	1195874	15820198	hadcm3n_o7l8_1980_40_008385044_0	673,920	3,352,749	4.9750
14 Aug 2013 16:12:05	1195874	15820198	hadcm3n_o7l8_1980_40_008385044_0	648,000	3,215,516	4.9622
14 Aug 2013 16:12:05	1195874	15820198	hadcm3n_o7l8_1980_40_008385044_0	622,080	3,077,996	4.9479
14 Aug 2013 16:12:05	1195874	15820198	hadcm3n_o7l8_1980_40_008385044_0	596,160	2,941,238	4.9336
14 Aug 2013 16:12:05	1195874	15820198	hadcm3n_o7l8_1980_40_008385044_0	570,240	2,812,551	4.9322
14 Aug 2013 16:12:05	1195874	15820198	hadcm3n_o7l8_1980_40_008385044_0	544,320	2,677,126	4.9183