Splitting a Linux install across several disks

[Jason]

My main Linux desktop with Kubuntu was acting up bizarrely. Every 10 or so boots, I couldn't shut it down without actually powering off the PC.

So I reinstalled Kubuntu from scratch. When I did so, this time, I put /home on my 4 TB RAID array which looks like a single disk to the OS. The main drive tree was still installed on the SSD (i.e. root or / ).

I did it so that programs are on the fast drive which only has 250 GB and data is on the slow drive but has 4 TB. Before that, I used the RAID array to just put extra stuff on it like my OS backup through Timeshift and huge files like ISOs and VMs.

Does anyone else do this with their Linux install? And by that, I meant having /home on a separate drive. Btw, you can do this after you do an install, too, if you later buy a drive you want to move /home to (or any folder).

[ssfc72]

Thanks for the info on your use of a Raid system, Jason.

I mainly have notebook computers, so using a Raid is not really practical for me, since adding external hardrives through the usb port would be cumbersome and slow at transferring the Data.

How do you go about separating the Data from your Distro, for storing on another Drive/Partition?  Also, what part of the Linux Distro is considered as Data?  I realize that stuff such as photos, music, movies, documents would be Data.

[buster]

Jason, are raid arrays used as much as they used to be, or have duplicate discs taken over that role, with constant backups and the ability to jump in if the main hd fails?

Never used either system ever. Probably never will, but I am curious.

[Jason]

I should have been more clear. My question wasn't meant to be about whether anyone else was using RAID. It was about storing /home on a separate drive, SDD or HDD or anything else you like.

[Jason]

I mainly have notebook computers, so using a Raid is not really practical for me, since adding external hardrives through the usb port would be cumbersome and slow at transferring the Data.

Yeah, it wouldn't be the type of thing you'd have with a notebook. But you could still put your data on an external drive unless you're transferring GBs worth of data. Which is really what I meant.

I'm going to answer your questions out of order. As one leads to the other. First,

...what part of the Linux Distro is considered as Data?  I realize that stuff such as photos, music, movies, documents would be Data.

I'm using the term data the way you are, to refer to files that other programs load such as a document that LibreOffice opens. Of course, in the purest sense, everything is data but here I mean user-created data. If you're using Linux as a desktop, the data is stored in the /home directory with each user having their own sub-directory to that (e.g. /home/bill). Programs often store settings and other configuration data there. The individual settings of your desktop environment are also stored there, as well. If you're using Linux as a server, it's more complicated.

How do you go about separating the Data from your Distro, for storing on another Drive/Partition?

In my case, I separated it during the install. Under the customized partitioning, where you do it yourself, you don't let the installer automatically configure partitions, you choose the other drive and mount it at /home.

Did you mean how do you move /home to a separate drive after you already have an operating system with everyone on one drive? That's more involved and not something beginners would want to do, but we're smarter than the average bear. Here's how you do it using the command-line interface (CLI). IMHO, that's the right way to do it. However, you could do some of the steps using a graphical program. For example, partitioning the drive which will become the new location for /home. No matter how you do it, it basically involves these steps:

1. Identify the drive you want to put /home on.
2. Mount it (the file manager should be able to see it) and partition it to a Linux filesystem
3. Copy the data from your existing /home over to the drive (under /home, not including /home)
4. Change your /etc/fstab file so the OS will see the new drive as /home on boot.
5. Reboot and you should now see that /home is on a different device than / (root filesystem)
6. Check to make sure your files are on the new home (you can do this after copying, too)

In step 3, when you move your data to the new /home, you're going to copy not just your /home but anything under /home. For example, under my /home, I have the directories called jason, lost+found, and timeshift. I might not want to transfer them all but if I have any data at all, I should at leave move the jason directory to the new drive. If I did just that, my new drive will have just jason in it. The wrong way to do this would be to copy only the data under jason to the new /home. Linux isn't going to expect data in the /home directory, but in a user directory under it.

On the last step, you can use your file manager to see that they're on different devices, at least I can in Kubuntu. Partitions can also appear as devices so, on my install, my Windows partition appears as its own device. I attached screenshots of the graphical way to do the check as well as the CLI way to look. In the file manager, the first screenshot shows the root filesystem device (note the highlighted text in bottom-left) and the second shows the /home device.

If you're averse to doing a reinstall of Linux, you could just move your /home data (again, everything under home) to the new drive (with a Linux-recognizable filesystem) and then when you do the reinstall, choose that drive for /home. It'll be under partitioning. Just make sure to not have the format option for /home when you do so.

[Jason]

Jason, are raid arrays used as much as they used to be, or have duplicate discs taken over that role, with constant backups and the ability to jump in if the main hd fails?

Other than the backup part, which you should still do, you just described RAID. RAID stands for Redundant Array of Inexpensive Disks.

The first thing you need to know is that RAID takes a bunch of physical drives and creates one logical drive out of them. It does this for several reasons: data redundancy (it's in the name!), performance improvement or both. Data redundancy means if you have multiple drives and one fails, you can put in a new drive (without even turning off the machine) and the array will be rebuilt. That is, you won't lose any data.

There are many RAID levels depending on what you're trying to achieve and you can have many more drives than I do (4). I use RAID 10, which is a combination of RAID 1 and RAID 0. Basically, it gives me mirroring and performance. Mirroring decreases performance (RAID 1) but striping increases it (RAID 0) so I use the two together. There are 4 x 2 TB drives in my array. Other RAID levels use a few more techniques aimed at higher redundancy levels (you can lose multiple drives at the same time).

So, to answer your question finally, I'm pretty sure RAID is still used extensively because it does what you mentioned. It allows you to recover from losing one or more drives by using mirroring (or parity drives). If the drive(s) fail, you can hot-swap replacement drives and the array is rebuilt so no data is lost. And you can use multiple drives for better performance. RAID 0 does only this but it's often said that RAID 0 isn't truly RAID because there isn't any redundancy. It would be great in applications that write a lot of data to storage, for example, video or animation work.

[ssfc72]

Thanks Jason, for the detailed info.
I think I will take one of my non critical notebook computers and try to move the /Home directory onto a new partition on the drive, for a test to see how it is done.

Yeah, it wouldn't be the type of thing you'd have with a notebook. But you could still put your data on an external drive unless you're transferring GBs worth of data. Which is really what I meant.



If you're averse to doing a reinstall of Linux, you could just move your /home data (again, everything under home) to the new drive (with a Linux-recognizable filesystem) and then when you do the reinstall, choose that drive for /home. It'll be under partitioning. Just make sure to not have the format option for /home when you do so.

[Jason]

Have fun!

[buster]

" I'm pretty sure RAID is still used extensively because it does what you mentioned."

I would think some drives would have to be off site - military, research centres, Democrats - and can that be done so sabotage at one spot in the world does not knock out the system? And would the whole system just keep chugging along?

[Jason]

I would think some drives would have to be off site - military, research centres, Democrats - and can that be done so sabotage at one spot in the world does not knock out the system? And would the whole system just keep chugging along?

AFAIK, RAID doesn't extend across multiple locations like that. But you can have backups servers that can jump in when the primary one goes down. RAID isn't designed to deal with sabotage, it's designed to deal with drive failure first and performance second. If the sabotage involved someone physically damaging one or more drives than yes, maybe a RAID-based system might recover. But I would expect anyone doing something like that would make sure all the drives were destroyed.

[Jason]

Back in the ole' days, Buster and Bill know what I'm talking about, we used to install a distro and purposely put /home on a different partition. That way when we re-installed a distro or even installed a different one (not that we ever did such a thing), we wouldn't have to back up our files first and then put them back on later. Of course, we should have backed up anyway, but we were young and crazy back then.

Then SSDs came out. They were very fast but didn't have much storage unless you were rich. So for a while, it made sense to have the OS/programs on a fast SSD and data on a cheaper HDD, especially if you were a collector of movies and such. I suppose you could put a VMs on the HDD too if you're a collector of VMs like one of us here. For most people, it makes less sense to do this with even 1 TB SSDs not being all that expensive.

For me, I used to, ahem, download movies from less than kosher sources so it still made sense. I no longer do that or at least trying not to. So my setup isn't really necessary, but I guess it still has the benefit of providing more protection for my data and a quicker reinstall, like in the olden days of yore, back when dragons roamed the skies and wizards lived in their high towers.

One issue I have is that if I use VMs, and put them on /home as I usually do, they're going to forced to operate at a slower speed. So I'm going to have to create another directory in the root tree ( i.e. / ) for the VMs and try not to go crazy with them like, um, certain people.

For no reason at all, here's my desktop setup. I just like to show off the variety of wallpapers you get with the Picture of the Day feature in Plasma.