Task 13924643

Name	hadcm3n_ym1g_1940_40_007682552_2
Workunit	7837639
Created	16 Jan 2012, 1:03:59 UTC
Sent	16 Jan 2012, 1:04:05 UTC
Report deadline	16 Apr 2012, 8:31:16 UTC
Received	9 Feb 2012, 4:47:44 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1145674
Run time	20 days 5 hours 29 min 19 sec
CPU time	19 days 21 hours 24 min 14 sec
Validate state	Invalid
Credit	11,819.52
Device peak FLOPS	3.04 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.12.33</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:31:56 (6718): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:10:01 (15730): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:48:26 (18260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:00:09 (20734): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:42:51 (21519): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:02:51 (24343): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:10:33 (25709): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:22:34 (30021): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... SIGABRT: abort called Stack trace (9 frames): /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xf76ea400] [0xf76ea425] /lib32/libc.so.6(gsignal+0x4f)[0xf752fc4f] /lib32/libc.so.6(abort+0x175)[0xf7533175] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib32/libc.so.6(__libc_start_main+0xf3)[0xf751b0f3] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2474, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (9 frames): /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xf7759400] [0xf7759425] /lib32/libc.so.6(gsignal+0x4f)[0xf759ec4f] /lib32/libc.so.6(abort+0x175)[0xf75a2175] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib32/libc.so.6(__libc_start_main+0xf3)[0xf758a0f3] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2474, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (9 frames): /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xf7713400] [0xf7713425] /lib32/libc.so.6(gsignal+0x4f)[0xf7558c4f] /lib32/libc.so.6(abort+0x175)[0xf755c175] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib32/libc.so.6(__libc_start_main+0xf3)[0xf75440f3] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2474, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (9 frames): /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xf777e400] [0xf777e425] /lib32/libc.so.6(gsignal+0x4f)[0xf75c3c4f] /lib32/libc.so.6(abort+0x175)[0xf75c7175] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib32/libc.so.6(__libc_start_main+0xf3)[0xf75af0f3] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2474, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (9 frames): /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xf77b5400] [0xf77b5425] /lib32/libc.so.6(gsignal+0x4f)[0xf75fac4f] /lib32/libc.so.6(abort+0x175)[0xf75fe175] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib32/libc.so.6(__libc_start_main+0xf3)[0xf75e60f3] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2474, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Stack trace (9 frames): /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xf7798400] [0xf7798425] /lib32/libc.so.6(gsignal+0x4f)[0xf75ddc4f] /lib32/libc.so.6(abort+0x175)[0xf75e1175] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395] /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib32/libc.so.6(__libc_start_main+0xf3)[0xf75c90f3] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2474, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
09 Feb 2012 01:07:33	1145674	13924643	hadcm3n_ym1g_1940_40_007682552_2	984,960	1,708,337	1.7344
05 Feb 2012 11:01:50	1145674	13924643	hadcm3n_ym1g_1940_40_007682552_2	959,040	1,666,416	1.7376
04 Feb 2012 23:30:03	1145674	13924643	hadcm3n_ym1g_1940_40_007682552_2	933,120	1,625,219	1.7417
04 Feb 2012 11:59:19	1145674	13924643	hadcm3n_ym1g_1940_40_007682552_2	907,200	1,583,834	1.7458
04 Feb 2012 00:22:42	1145674	13924643	hadcm3n_ym1g_1940_40_007682552_2	881,280	1,542,313	1.7501
03 Feb 2012 12:48:14	1145674	13924643	hadcm3n_ym1g_1940_40_007682552_2	855,360	1,500,826	1.7546
03 Feb 2012 01:40:21	1145674	13924643	hadcm3n_ym1g_1940_40_007682552_2	829,440	1,461,040	1.7615
02 Feb 2012 14:34:11	1145674	13924643	hadcm3n_ym1g_1940_40_007682552_2	803,520	1,421,256	1.7688
02 Feb 2012 03:16:31	1145674	13924643	hadcm3n_ym1g_1940_40_007682552_2	777,600	1,380,856	1.7758
01 Feb 2012 15:37:40	1145674	13924643	hadcm3n_ym1g_1940_40_007682552_2	751,680	1,339,089	1.7815
01 Feb 2012 03:19:07	1145674	13924643	hadcm3n_ym1g_1940_40_007682552_2	725,760	1,294,845	1.7841
31 Jan 2012 13:26:11	1145674	13924643	hadcm3n_ym1g_1940_40_007682552_2	699,840	1,252,089	1.7891
31 Jan 2012 02:00:12	1145674	13924643	hadcm3n_ym1g_1940_40_007682552_2	673,920	1,211,127	1.7971
30 Jan 2012 13:33:24	1145674	13924643	hadcm3n_ym1g_1940_40_007682552_2	648,000	1,166,763	1.8006
30 Jan 2012 01:10:24	1145674	13924643	hadcm3n_ym1g_1940_40_007682552_2	622,080	1,122,410	1.8043
29 Jan 2012 11:51:11	1145674	13924643	hadcm3n_ym1g_1940_40_007682552_2	596,160	1,075,344	1.8038
28 Jan 2012 22:31:29	1145674	13924643	hadcm3n_ym1g_1940_40_007682552_2	570,240	1,024,951	1.7974
28 Jan 2012 08:02:06	1145674	13924643	hadcm3n_ym1g_1940_40_007682552_2	544,320	974,401	1.7901
27 Jan 2012 16:33:10	1145674	13924643	hadcm3n_ym1g_1940_40_007682552_2	518,400	922,376	1.7793
27 Jan 2012 01:15:14	1145674	13924643	hadcm3n_ym1g_1940_40_007682552_2	492,480	868,857	1.7642