|
Questions and Answers :
Unix/Linux :
Help requested - using new hard disk under Linux Mint 21 [SOLVED]
Message board moderation
Author | Message |
---|---|
Send message Joined: 1 Jan 07 Posts: 1014 Credit: 35,789,563 RAC: 25,282 |
Followers of number crunching will know that I have two Linux Mint machines - one Mint 20.3, and one Mint 21 (Ubuntu 22.04). This is about the second. Followers of number crunching will also know that I'm running out of disk space, and I've purchased a second SSD to move data onto. I've installed it, and cabled it, and so far I've got to: richard# fdisk -l Disk /dev/nvme0n1: 476.94 GiB, 512110190592 bytes, 1000215216 sectors Disk model: ADATA SX8200PNP Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: gpt Disk identifier: BFFDD46F-C75B-45F9-AC13-A42D58BC37DC Device Start End Sectors Size Type /dev/nvme0n1p1 2048 1050623 1048576 512M EFI System /dev/nvme0n1p2 1050624 1000214527 999163904 476.4G Linux filesystem Disk /dev/sda: 1.82 TiB, 2000398934016 bytes, 3907029168 sectors Disk model: CT2000MX500SSD1 Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disklabel type: gpt Disk identifier: 174E3A15-CC2D-47DE-8C5A-AB698A1E8AAF Device Start End Sectors Size Type /dev/sda1 2048 3907028991 3907026944 1.8T Linux filesystemSo it's seen by the hardware, partitioned, formatted, has a device name, and all seems good. The next step seems to be adding it to fstab. So I did that. And it bricked the system. A slightly panicked session with a recovery disk got me back to square 1 - all is safe, BOINC is running, I can breath again. The lines I added to fstab were: # new data drive UUID=174E3A15-CC2D-47DE-8C5A-AB698A1E8AAF /hdd ext4 errors=remount-ro 0 2- I need a better idea, please. |
![]() Send message Joined: 15 May 09 Posts: 4444 Credit: 17,930,168 RAC: 29,872 |
The way I would do it is run BOINC out of work, then using gparted, set the new disk up with a mount point of /var/lib/boinc-client having first renamed the directory with the same location, then move everything across. |
![]() Send message Joined: 29 Nov 17 Posts: 55 Credit: 8,254,259 RAC: 50,830 |
Mount point /hdd already created and got the right permissions ? |
Send message Joined: 1 Jan 07 Posts: 1014 Credit: 35,789,563 RAC: 25,282 |
Something like that. My plan was to have the new disk, named something like 'data', visible automatically at boot time. Then create a folder structure like /var/lib/boinc-client (or simply /boinc-client/), copy across, and either use a folder redirect or a modified systemd so BOINC looks at the new location. Then, I'd have spare space for other purposes, like compiling software, if I decide to go down that path. But with so much Windows history cluttering up my brain, the Linux pathway for setting that up always seems to be missing a vital signpost. I'm hoping it can be delayed until the uploads have cleared, but I thought I'd start on the process before that got underway. |
Send message Joined: 1 Jan 07 Posts: 1014 Credit: 35,789,563 RAC: 25,282 |
Mount point /hdd already created and got the right permissions ?I'd followed point 2.1, 'Create a mount point', from https://askubuntu.com/questions/125257/how-do-i-add-an-additional-hard-drive - but that feels sort of skinny. I'd expect two parameters, mkdir what where? |
![]() Send message Joined: 29 Nov 17 Posts: 55 Credit: 8,254,259 RAC: 50,830 |
From that command in the link you are making a directory called hdd at the top level so if you cd to / you'd see everything in the top level including your hdd directory. You can call it ssd if you wanted to, the /hdd isn't part of the command just the name (of the mount point) to be used. |
Send message Joined: 1 Jan 07 Posts: 1014 Credit: 35,789,563 RAC: 25,282 |
Well, he's saying "make the target named directory, then use it in fstab at step 2.2". But there's something missing that he's assumed, and left unstated. |
![]() Send message Joined: 29 Nov 17 Posts: 55 Credit: 8,254,259 RAC: 50,830 |
I'm assuming you read all the comments on the page as you are using the UUID format. When did it brick the system when you did the mount command or when you rebooted ? Did you check the fstab file by doing "sudo mount -av" before rebooting ? What are the permissions and ownership set to for the /hdd directory ? |
Send message Joined: 1 Jan 07 Posts: 1014 Credit: 35,789,563 RAC: 25,282 |
At the moment, with the new hardware present but not active, the machine looks like this: ![]() ADATA is the original M.2 boot+storage SSD (512 GB) CT2000MX is the new SATA3 SSD, which I labelled 'Data' during formatting. I can mount it manually after boot, and I've created a test folder called boinc-client. I can't show you the permissions for a 'device', but I can for the folder on it: ![]() Now I look at it, that needs some work, but I need to get a clearer understanding of what belongs to what. I can't find the label 'hdd' anywhere, and I have no idea where 'File system' lives - but it's got lots of bytes in/on it. |
![]() Send message Joined: 29 Nov 17 Posts: 55 Credit: 8,254,259 RAC: 50,830 |
At the moment, with the new hardware present but not active, the machine looks like this: The 'hdd' isn't a label, it is a directory you created in the root of the ADATA drive (using the 'sudo mkdir /hdd', you could make another directory called for example wibble, 'sudo mkdir /wibble'). I'm assuming it is still there if your restoration session just involved editing the ftsab file. The fstab entry is using that /hdd (or /wibble) as the mount point where all the disk space and files can be put/found. Good to know that the drive is found and works when you mount it manually. |
Send message Joined: 1 Jan 07 Posts: 1014 Credit: 35,789,563 RAC: 25,282 |
Ah. I see it: it shows as a folder inside 'File system', but it's only got 42 GB free - that's the problem that started me down this road in the first place. I think we'd better call that 'end of round 1', and call it quits for the day. That machine's in a brute of a HAF case - there's plenty of room to work inside it (once you've removed the GPUs, which obscure the SATA connectors...), but it weighs a ton when you try to get it out from under the desk. 'nuff for one day. Thanks for your help. |
![]() Send message Joined: 29 Nov 17 Posts: 55 Credit: 8,254,259 RAC: 50,830 |
The 42Gb free is what's left to use on the ADATA drive. If the fstab mount request worked then everything created/appearing below /hdd (or /wibble) would be using the space on the new SSD. It's like a symlink that points to somewhere else, in this case a completely separate drive. |
Send message Joined: 5 Aug 04 Posts: 1106 Credit: 17,025,459 RAC: 5,652 |
- I need a better idea, please. I do not have an idea but this is what I have and perhaps you can compare with what you have and see what is different. This is my entire /etc/fstab file. DId you read what it says near the very top? # After editing this file, run 'systemctl daemon-reload' to update systemd You can ignore the stuff I added. /dev/sda and /dev/sdb that are for my spinning drives. Everything before them came with my Red Hat Enterprise Linux release 8.0 (Ootpa) computer when I got it; I am now running 8.7.. That stuff is an SSD. [/etc]$ cat fstab # # /etc/fstab # Created by anaconda on Mon Oct 26 16:05:08 2020 # # Accessible filesystems, by reference, are maintained under '/dev/disk/'. # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. # # After editing this file, run 'systemctl daemon-reload' to update systemd # units generated from this file. # /dev/mapper/rhel-root / xfs defaults 0 0 UUID=3af1d664-9271-4c74-8693-c9948daf0939 /boot xfs defaults 0 0 UUID=BF2A-AC12 /boot/efi vfat umask=0077,shortname=winnt 0 2 /dev/mapper/rhel-home /home xfs defaults 0 0 /dev/mapper/rhel-swap swap swap defaults 0 0 # Added below by jeandavid8 # Four Terabyte 5400 rpm hard drive (/dev/sda) UUID=e6c2ff29-e09f-4973-98b3-8e609387219f /home/jeandavid8/Videos xfs defaults 1 2 UUID=cda6c62a-c343-43db-b71c-2f69dcc947a6 /home/jeandavid8/Sound xfs defaults 1 2 UUID=4ba0402d-3bfa-4855-8307-79cd570b1658 /var/lib/boinc xfs defaults 1 2 # Four Terabyte 7200 rpm hard drive (/dev/sdb) UUID=998e2c0c-5a09-4519-8de2-1e9b14031cff /home/margaret xfs defaults 0 0 UUID=d9870066-3fea-471b-a7f7-3c0c897780d6 /D3P1 xfs defaults 1 2 UUID=545c03ed-fcfd-4eb0-a0ff-53b9eaa7fc39 /D3P2 xfs defaults 1 2 UUID=b4e9957a-8880-4220-9dd1-4431f70d93d0 /D3P3 xfs defaults 1 2 And this is what fstab -l has to say: [/etc]# fdisk -l Disk /dev/nvme0n1: 477 GiB, 512110190592 bytes, 1000215216 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: gpt Disk identifier: 1DD24E00-67C1-4E74-BBDB-A27B4908568F Device Start End Sectors Size Type /dev/nvme0n1p1 2048 1230847 1228800 600M EFI System /dev/nvme0n1p2 1230848 3327999 2097152 1G Linux filesystem /dev/nvme0n1p3 3328000 1000214527 996886528 475.4G Linux LVM Disk /dev/sdb: 3.7 TiB, 4000787030016 bytes, 7814037168 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disklabel type: gpt Disk identifier: B9402EBE-2458-4F92-A570-4F48C89DA595 Device Start End Sectors Size Type /dev/sdb1 2048 409602047 409600000 195.3G Linux filesystem /dev/sdb2 409602048 819202047 409600000 195.3G Linux filesystem /dev/sdb3 819202048 1228802047 409600000 195.3G Linux filesystem /dev/sdb4 1228802048 2048002047 819200000 390.6G Linux filesystem Disk /dev/sda: 3.7 TiB, 4000787030016 bytes, 7814037168 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disklabel type: gpt Disk identifier: F47722E7-982E-4946-8F9B-D3B629246CF0 Device Start End Sectors Size Type /dev/sda1 2048 1024002047 1024000000 488.3G Linux filesystem /dev/sda2 1024002048 1228802047 204800000 97.7G Linux filesystem /dev/sda3 1228802048 2252802047 1024000000 488.3G Linux filesystem Disk /dev/mapper/rhel-root: 50 GiB, 53687091200 bytes, 104857600 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk /dev/mapper/rhel-swap: 15.6 GiB, 16768827392 bytes, 32751616 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk /dev/mapper/rhel-home: 409.8 GiB, 439948935168 bytes, 859275264 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk /dev/sdd: 1.8 TiB, 2000365289472 bytes, 3906963456 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: gpt Disk identifier: 17136D15-4671-47C9-A8AD-FCE806835965 Device Start End Sectors Size Type /dev/sdd1 2048 3906961407 3906959360 1.8T Microsoft basic data localhost:root[/etc]# ![]() |
Send message Joined: 1 Jan 07 Posts: 1014 Credit: 35,789,563 RAC: 25,282 |
OK, so here's the plan for today. My current batch of 5 IFS tasks will finish around 17:00. I'm not going to experiment with anything until they've reached a clean finish. By then, we should have some sort of idea how the work on the upload server is progressing. I'll spend some of the time today re-reading the comments already posted, and the man pages for fstab(5). I might prepare another version of fstab to try later, now that I know the recovery process is available. Someone asked for details of the failure. The machine starts as normal with a show of the BIOS screen. It moves on to show some brief text-mode messages from Mint (normal), and then switches to graphics mode: normally, my regular desktop is shown within seconds. Yesterday, it showed the Mint symbol only, for 90 seconds. Then, after what seems to be a pre-set delay, it switch to an emergency text-mode display, and offered various options like trying again, or starting a recovery session. I couldn't get that to work. If the upload server has restarted, and seems to be making good progress, I'll concentrate on getting the science back to the researchers. If it still seems to be struggling, I'll maybe drop back to one of two tasks at a time, just to keep the upload retries tickled, and maybe restart one or two of my other projects (I'm keeping an eye on the free space - currently down below 28 GB, but IFS fills it quickly). And if uploads are still borked, I'll switch completely to other work overnight and retry IFS each day until it's ready. I also see I have an upgrade to Mint 21.1 available, but I'll delay that until things have settled down. As before, all suggestions are welcome. |
Send message Joined: 7 Jun 17 Posts: 23 Credit: 44,434,789 RAC: 2,600,991 |
As before, all suggestions are welcome. Hi. I have had to do this quite often with boinc. This is what I do. /dev/sdX is your new drive You'll need to adapt this for systemd. I used parted to make the /dev/sdX partition and # mkfs -t ext4 /dev/sdX before this procedure. # service boinc-client stop # mkdir /tmp/bc # mount /dev/sdX /tmp/bc # rsync -a /var/lib/boinc-client/ /tmp/bc (note trailing /) # mv /var/lib/boinc-client /path/to/backup/directory/for/safe/keeping # blkid /dev/sdX >> /etc/fstab (NOTE double arrows!!!) # vi /etc/fstab (edit last entry to mount /var/lib/boinc-client on /dev/sdX) # umount /tmp/bc # mkdir /var/lib/boinc-client # chown boinc:boinc /var/lib/boinc-client # mount -a # service boinc-client start *Only* when you are confident that the data transfer has worked properly, remove the backup. Before doing this, please wait for a couple of confirmations that these instructions are good to go. I've copied them from a bash history, so I can confirm that they worked on that machine. YMMV Good luck... fraser |
Send message Joined: 1 Jan 07 Posts: 1014 Credit: 35,789,563 RAC: 25,282 |
Thanks - I think I can follow that, making substitutions as needed and adapting to systemd (I stop and disable BOINC before every maintenance update, and restart it after the reboot, because I need to allow time for the GPU drivers to become active before BOINC starts). Can you confirm whether those changes are persistent - i.e. will the new disk become the active BOINC data directory for subsequent restarts? Or, if not, can it be scripted? I'd prefer not to have to go through it after every restart. For the time being, I'm going to put this upgrade project on hold. I tried a slightly simply mod to fstab - leaving out the optional bits - but it's still not booting. I'd prefer to do that rsync after CPDN has collected all the data I'm holding for it - safer that way. In the meantime, I've returned the machine to its previous role of mainly GPU projects, and I'll restart CPDN gently once the uploads have started, and I'll use maybe one task at a time to keep the uploads from going into backoff. |
Send message Joined: 7 Jun 17 Posts: 23 Credit: 44,434,789 RAC: 2,600,991 |
Provided that the fstab entry for the disk and mountpoint are correctly entered and saved, the changes should persist between reboots. At least, they do on my sysvinit hosts; it might be worth checking the situation with someone who understands systemd.mount, although there doesn't seem to be any conflict. https://unix.stackexchange.com/questions/90723/is-there-any-reason-to-move-away-from-fstab-on-a-systemd-system Don't forget to confirm that you've backed up the old fstab before unleashing the blkid >> best fraser |
Send message Joined: 1 Jan 07 Posts: 1014 Credit: 35,789,563 RAC: 25,282 |
ooh - er... BOINC has crashed three times since yesterday evening. I think the first two may have been caused by a particular application from another BOINC project - I'll take it up with them. The third seems to have been triggered by timeshift. The good news is that timeshift can see the new drive, so I've made a full new snapshot on that. And I'm going to stop tinkering until I've transferred responsibility for the CPDN files I've already created - over 100 tasks now. |
Send message Joined: 12 Apr 21 Posts: 286 Credit: 13,557,282 RAC: 31,430 |
over 100 tasks now. Yeah, got to get rid of those files, man. We're processing volunteers, not storage ones after all. We don't get credit for storing stuff, come on! We should be talking about GigaHz of speed and GigaBytes of memory, not what to do with GigaHeaps of weather paraphernalia on our disks! Chances are my SSD will die before I ever get it this full again. And why did they pick Jasmin, because the name sounded pretty? The performance has been anything but. Come on, Jasmin, let's jazz this thing up! :-D |
Send message Joined: 1 Jan 07 Posts: 1014 Credit: 35,789,563 RAC: 25,282 |
LOL - that cheered me up! |
©2024 climateprediction.net