|
Message boards :
Number crunching :
Hardware requirements for upcoming models
Message board moderation
Author | Message |
---|---|
Send message Joined: 5 Sep 04 Posts: 7629 Credit: 24,240,330 RAC: 0 |
Processor speed Memory amount Hard disk space etc |
![]() Send message Joined: 15 May 09 Posts: 4444 Credit: 17,929,339 RAC: 29,974 |
etc should probably include internet connection speed and allowance for any still on a metered connection. |
Send message Joined: 18 Jul 13 Posts: 438 Credit: 25,201,050 RAC: 13,119 |
I would also include power consumption under others. It seems more and more important as electricity prices soar while people are planning their upgrades. |
Send message Joined: 9 Oct 20 Posts: 690 Credit: 4,391,754 RAC: 6,918 |
Sign up to gridcoin, it helps pay for the power. https://gridcoin.us/ Unfortunately they only take projects with regular work: https://gridcoin.us/guides/whitelist.htm |
![]() Send message Joined: 15 May 09 Posts: 4444 Credit: 17,929,339 RAC: 29,974 |
I went with gridcoin for a short while. I prefer to have full control over my computer. They are not as bad as SU but I wouldn't go there again. |
Send message Joined: 9 Oct 20 Posts: 690 Credit: 4,391,754 RAC: 6,918 |
I went with gridcoin for a short while. I prefer to have full control over my computer. They are not as bad as SU but I wouldn't go there again.You do have full control. You use Boinc as usual, but Gridcoin gives you coins when you do work for their whitelisted projects. You can still do the others, you just won't get any coins. You don't need to bother with a pool anymore, since they've added a function to withdraw coins without investing. Seriously, I don't change my Boinc habits one bit for Gridcoin. It just sits there in the background and I don't have to touch it. |
![]() Send message Joined: 15 May 09 Posts: 4444 Credit: 17,929,339 RAC: 29,974 |
Even though they do give out coins to CPDN users occasionally, seeing it as a worthwhile project, as I don't run anything else most of the time and CPDN is not whitelisted, I doubt it is worth my while joining. |
Send message Joined: 9 Oct 20 Posts: 690 Credit: 4,391,754 RAC: 6,918 |
It would only get whitelisted if it has regular work. And I'm guessing that means for Windows aswell. |
Send message Joined: 29 Oct 17 Posts: 944 Credit: 15,193,000 RAC: 23,306 |
Hi Les, Just browsing the forums and came across this message. I'm probably not well known on the forums. I'm retired as of last year but I worked at ECMWF and developed OpenIFS@home with Andy & Sarah. OpenIFS is a very memory intensive model. That's in part because it's very effficient, it's designed to run as fast as possible for weather forecasting applications. At the moment we've only run OpenIFS as a single-threaded application and at low resolutions that are not (yet) scientifically all that useful. I've been testing a multi-core version as we'd like to increase the model resolution available for OpenIFS but this implies significant increases in peak RAM used by OpenIFS. Since we are bottlenecked on RAM, we might as well make use of additional cores to speed up the model. Processor speed is not that relevant here, what is though is available core count. IFS (and OpenIFS) is a highly parallel model. If I enable threading, there is a 2x speedup with 2 cores, 3.5x with 4 cores etc. I plan to release some tests on the dev site which will try out multi-core, higher model resolution (and hence memory) after the summer with assistance from the CPDN team. Since network upload is the bottleneck (and maybe upload server capacity at the remote end), available local disk space is not really a big deal. Cheers, Glenn |
![]() Send message Joined: 15 May 09 Posts: 4444 Credit: 17,929,339 RAC: 29,974 |
Looking forward to these! |
Send message Joined: 15 Jan 06 Posts: 637 Credit: 26,751,529 RAC: 653 |
Thank you. We go from knowing almost nothing, to quite a bit. I can do lots of cores, and have a good cable modem, so send them out. And 64GB is no problem, and I can do 128GB if you need it. It is just sitting in the drawer anyway. |
Send message Joined: 29 Oct 17 Posts: 944 Credit: 15,193,000 RAC: 23,306 |
Andy B looked at available machines signed to CPDN with > 16Gb RAM assigned to BOINC and there were over 1000 which is a good number for higher resolution experiments. I also forgot to add I agreed with one of the other posts that mentioned energy costs too. We need to do some tests to find a happy place between model resolution, wall-clock time and model output size for running in the CPDN framework. BTW, does anyone know how to get alerts or emails when a new reply/post appears to a thread I am subscribed to? It doesn't seem to be working for me, I have to go to the forum page to check. Thx. |
Send message Joined: 9 Oct 20 Posts: 690 Credit: 4,391,754 RAC: 6,918 |
BTW, does anyone know how to get alerts or emails when a new reply/post appears to a thread I am subscribed to? It doesn't seem to be working for me, I have to go to the forum page to check.They're working for me. Have you got "immediately by email" ticked in here? https://www.cpdn.org/edit_forum_preferences_form.php If you have, whitelist cpdn.org in your spam filter or ISP's spam filter. I had to do that for another project. |
Send message Joined: 5 Sep 04 Posts: 7629 Credit: 24,240,330 RAC: 0 |
Hi Glenn That's info's good to know. Time to get ready for them. I remember when the first ones appeared, at the time of the announcement on our front page: Oxford e-Research Centre teams up with ECMWF to enable scientists to run the OpenIFS weather model on thousands of people’s home computers. I only got one task from that, which ran for about 18 hours. Les |
Send message Joined: 29 Oct 17 Posts: 944 Credit: 15,193,000 RAC: 23,306 |
BTW, does anyone know how to get alerts or emails when a new reply/post appears to a thread I am subscribed to? It doesn't seem to be working for me, I have to go to the forum page to check.They're working for me. Have you got "immediately by email" ticked in here? https://www.cpdn.org/edit_forum_preferences_form.php Yes, that's probably it. I had no email set. We'll see how much of a good idea that was... :) |
Send message Joined: 29 Oct 17 Posts: 944 Credit: 15,193,000 RAC: 23,306 |
Yes, that was a short forecast (about 10 days I think). There are some seasonal length runs coming which will take much longer, hence the push for multiple cores. |
Send message Joined: 9 Oct 20 Posts: 690 Credit: 4,391,754 RAC: 6,918 |
Yes, that was a short forecast (about 10 days I think). There are some seasonal length runs coming which will take much longer, hence the push for multiple cores.I suppose it's too much to ask for GPU support :-) |
![]() Send message Joined: 15 May 09 Posts: 4444 Credit: 17,929,339 RAC: 29,974 |
And worth remembering that OpenIFS is Linux and Mac only. I am not aware of any plans to add Windows to the mix.Yes, that was a short forecast (about 10 days I think). There are some seasonal length runs coming which will take much longer, hence the push for multiple cores.I suppose it's too much to ask for GPU support :-) |
Send message Joined: 29 Oct 17 Posts: 944 Credit: 15,193,000 RAC: 23,306 |
ECMWF and external colleagues have been testing GPUs in IFS for many years, since they first looked potentially useful for speeding up the forecast. The problem with GPUs is the relative cost of moving data on & off them. i.e. they perform best when a lot of computation is needed on large data arrays. Unfortunately because of the algorithms used in IFS this limits their use to select parts of the model.I suppose it's too much to ask for GPU support :-)And worth remembering that OpenIFS is Linux and Mac only. I am not aware of any plans to add Windows to the mix. There is currently no GPU part in the operational IFS model and since OpenIFS is based on the operational model (not the 'in-development' model) it will be sometime yet, assuming IFS does eventually use them for operational forecasts. To be honest though, I'd rather not have to manage GPU use in a CPDN context with the myriad of graphics cards out there. The 'low-hanging fruit' is to make use of the highly parallel nature of IFS and all the spare cores available on home machines. As for Windows, I've talked with the CPDN team about creating a virtual machine for OpenIFS that will run on Windows, but that's lower down the priority list at the moment. As for macs, I'm afraid I don't have access to a mac with the new chips, so we'd probably go with a VM for that as well. If you are interested there are technical reports here (apologies, I only had a quick google search and couldn't find anything more of an easy read): https://www.ecmwf.int/en/elibrary/17320-openacc-gpu-adaptation-ifs-cloud-microphysics-scheme https://www.ecmwf.int/en/elibrary/18628-accelerating-weather-prediction-nvidia-gpus https://www.ecmwf.int/sites/default/files/elibrary/2018/18628-accelerating-weather-prediction-nvidia-gpus.pdf https://events.ecmwf.int/event/169/contributions/2767/attachments/1460/2636/HPC-WS_Lapillonne.pdf ECMWF's new supercomputer in Bologna includes GPUs on each node for application development, see: https://www.ecmwf.int/en/about/media-centre/news/2022/taking-ecmwfs-new-high-performance-computing-facility-operation Hope that's useful, happy to answer any questions. |
Send message Joined: 29 Oct 17 Posts: 944 Credit: 15,193,000 RAC: 23,306 |
Quick question, out of interest. How many cores on your machines to you make available to CPDN/BOINC? I ask because we considering what to create for OpenIFS; thinking 2 & 4 core versions but maybe people on this thread have some thoughts? Thanks. |
©2024 climateprediction.net