Boost Your Hard Drive with Bcache

Made by Albert Westra using with MyPaint and Gimp/Gmic

A few weeks ago I converted my friend’s Chromebook from its stock ChromeOS to Ubuntu. As payment, he let me keep the Kingston 16GB M.2 SSD which we replaced with a bigger 240GB SSD. My laptop has two mSATA ports, which means I can put that 16GB SSD in my Laptop as long as I get the correct adapter, but 16GB is pretty small for a heavy user like me. So what can a guy like me do with this small drive?

Well how about using it as cache drive? Luckily for us, as linux users, we do have quite a few options to accomplish this task, but the one that stood out to me was Bcache.

Bcache is, for all intents and purposes, a Kernel module that allows a SSD to cache all writes if writeback mode is enabled, or caching blocks read from the main backend drive; which ,in my case , would be my 750GB HDD. In a nutshell, it allows a cheap large storage drive to have marginally the same performance as a SSD.

On the downside, however, the Ubuntu installer does not have install support for Bcache. We will have to enable it ourselves on a fresh install. There is a tool that does this for you without needing to do a fresh install, but me personally, I like doing things on a clean slate.

Things to Know Before We Get Started

Before we get started, I would highly suggest that you read the Bcache documentation for a better understanding of how it works. Additionally, make sure you read this article to end before you get started. Believe me, you will not regret it.

I am doing this guide on an Ubuntu distribution, specifically on Ubuntu Gnome 15.10. So if you are on a different distribution outside of the Ubuntu/Debian realm, I would highly suggest you use this as a reference and not as an all-purpose guide. Also, there are quirks to my system which make it refuse to boot from the SSD so my boot/efi partition will be on my HDD.

Benchmarks: Before

I did take some benchmarks with Gnome’s built in Disks Utility of both drives to get a performance baseline. At the end of the article, I’ll benchmark the Bcache drive and compare it with the data I took now to see where it improves and where it doesn’t.

2015-1103-Benchmarking_Before
Benchmark comparing my HDD on the right and SSD on the left.

Now according to the graph above my 750GB HDD on the right has average read speeds of 81MB/s, average write speeds of 68MB/s, and the access time at about 16 msecs. Now, with the 16GB M.2 SSD on the left has read speeds of 486MB/s, average write speeds of 384MB/s and access time at around 0.05msecs. Also as a side note, according to the graph the SSD performance did not degrade over time like the HDD did. So you can see the potential Disk-IO improvements we could have if we enable Bcache on these two drives.

Okay, let’s get started.

Partitioning and Initial Install

Before you click on that Install Icon when you boot from the liveUSB thumb drive, first open up Gparted or your favorite partitioner. Since we are creating our own partition scheme, the EFI partition needs to be formatted and flags raised before you install. I learned this the hard way, so I’ll help you avoid the problem I had. The other four partitions I added included a temporary root partition, a boot partition, and two cleared partitions. In this case my HDD is /dev/sda and my SSD is /dev/sdb.

The Hard Drive on /dev/sda

  • A 64MB partition formatted to fat32 with the flags esp and boot raised. This will be the EFI partition.
  • A 16GB partition formatted as ext4. This will be the temporary root partition where we will initially install Ubuntu to. After Bcache is configured, this will become my SWAP partition. Be sure to size it according to the SWAP size you need while still keeping in mind the installer’s minimal space requirement.
  • Finally, the rest of the space is formatted as cleared and left empty. This will be the backend of Bcache.
2015-1104-Partition_Example_01
My partition scheme layout in the end for /dev/sda.

The Solid State Drive on /dev/sdb

  • A 200MB partition formatted to ext4. This will be the Boot Partition where grub will be installed.
  • The rest of the SSD space will be formatted as cleared and left empty as well. This will be the cache part of Bcache.
2015-1104-Partition_Example_02
My partition scheme layout in the end for /dev/sdb.

Once you have your initial partition scheme set up, open the installer and proceed to install your distribution. When you get to the Partition Editor, set your boot partition and your root partition and point your grub installer to the device that has the boot partition. In my case, it’s /dev/sdb. Leave the two cleared partitions alone. Since the partitions are already formatted, you do not have to click on the format box but that is completely up to you. You will get warnings about no swap and a couple of other warnings but you should be able to ignore them and continue the installation. Once the installation is complete, restart and boot into your fresh temporary install.

Installing and Initializing Bcache

Now that you have a freshly installed system, we need to install bcache-tools. These tools will enable us to use those two cleared partitions as a Bcache drive. There is a ppa for bcache-tools, but on Ubuntu Gnome 15.10 the latest versions are available. All of the commands will need to be done as root. The tool to use to create the Bcache drive is make-bcache. First with the -C option give it the cache drive location(In my case /dev/sdb2 on my SSD). Then with the -B option give it the backing drive location(In my case /dev/sda3 on my HDD). Once the Bcache is created, we’ll need to format this newly created drive to the ext4 filesystem. Here are the commands I used on my system.

$ sudo bash
# apt-get install bcache-tools
# make-bcache -C /dev/sdb2 -B /dev/sda3
# mkfs.ext4 /dev/bcache0

Move all the Things

Now that your bache drive is created and initialized, we can transfer all of our temporary root system to its permanent home in /dev/bcache0. We’ll do this with a bit of mounting and rsync. In the same terminal you’ll need to create two new directories, OLD and NEW. Then mount the temporary root to OLD (in my case /dev/sda2) and mount /dev/bcache0 to NEW. Then we’ll use rsync with the -a option to copy OLD to NEW. Here are the commands I used:

# mkdir OLD NEW
# mount /dev/sda2 OLD
# mount /dev/bcache0 NEW
# rsync -a OLD/ NEW/    # Don’t forget to add the slash at the end of each directory!

Redirecting Grub

Once rsync finishes its magic, we need to redirect grub to boot from the new and permanent root directory. To do that, we will need to mount the correct folders and point them to the new locations on /dev/bcache0.

# mount /dev/sdb1 NEW/boot
# mount /dev/sda1 NEW/boot/efi
# mount -o bind /dev NEW/dev
# mount -t proc none NEW/proc
# mount -t sysfs none NEW/sys

Then we’ll need to chroot into /dev/bcache0 and pull the UUID of both /dev/bcache0 and the temporary root partition (in this case it’s /dev/sda2) with ls and grep. UUID is the unique string of numbers and letters that allows Linux to identify that partition. Basically a partition’s call sign. Keep note of them because we will need them in the next step.

# chroot NEW
# ls -l /dev/bcache/by-uuid | grep bcache0
# ls -l /dev/disk/by-uuid | grep sda2

In a new terminal, you’ll need to run gedit with sudoedit to open and edit two files, fstab and grub.cfg. Why not just use sudo gedit? Becausea as one reader pointed out to me, using sudo gedit can create file permissions so that no other gnome program settings can be stored for that userid. In other words, it could create a file permission conundrum and sudoedit helps avoid that. With sudoedit, we are  essentially replacing the UUID of the temporary root with the UUID of /dev/bcache in both files. Use the find and replace tool in gedit since there are a lot of UUIDs you need to replace (especially in grub.cfg).

$ export EDITOR=/usr/bin/gedit
$ sudoedit /NEW/etc/fstab
$ sudoedit /NEW/boot/grub/grub.cfg
2015-1104-Gedit_Example
Replacing sda2 UUID with bcache0’s UUID.

After you are done there, close that terminal and go back to the original terminal that is chroot-ed into your new file-system. Re-install grub so that it will detect the new root partition. Once again, be sure to point grub to the device that has the boot partition. In my case, it’s at /dev/sdb. Reboot your system, and it should boot into your Bcache root instead of the temporary root partition. If it doesn’t, hold down shift [Or ESC] while it’s booting and it should bring up the grub menu. Select the correct root partition and it should boot into it.

# grub-install /dev/sdb

Converted the Temporary Root to SWAP

All that is left to do is to convert the temporary root as SWAP, and add its UUID to fstab. After that your are essentially done.

$ sudo bash
# mkswap /dev/sda2 #Be sure to copy the UUID it generates.
# swapon /dev/sda2
# echo “UUID=PASTE_UUID_HERE none swap defaults 0 0” >> /etc/fstab

I would highly suggest that you create a disk image of both your HDD and SSD to avoid the hour and a half headache of redoing all of this. It’s is built into Gnome Disks Utility if you are wondering. Unless you like to mentally abuse yourself. Don’t worry-I did… 4 times…

Optional Things to do

Here are two things you can do if you want, but will risk data loss on your system if you suddenly lose power. Consider yourself warned. First, enable the writeback cache_mode option. This will enable it to cache writes as well. You will notice the speedup when you install packages. Lastly if you want trim support for your SSD you will need to enable the discard option. Be sure to replace the UUID to reflect your /dev/bcache, otherwise you will get an error. That can easily be done with the cd and ls commands.

$ sudo bash
# echo writeback > /sys/block/bcache0/bcache/bcache_mode
# echo 1 > /sys/fs/bcache/848af769-ebe2-479b-944d-54adc4ec20ac/cache/discard

Benchmarks: After

After a few boot ups you’ll probably notice that your system starts up faster, applications may load faster and overall it may feel snappier. However, what do the numbers say? How much of an improvement compared to the Kingston SSD initial benchmarks? Well after a week of using it, I booted into my liveUSB thumb drive and benchmarked bcache0. Here are the results:

Benchmark comparing /dev/bcahce0 on the right and the Kingston 16GB SSD on the Left.
Benchmark comparing /dev/bcahce0 on the right and the Kingston 16GB SSD on the Left.

Impressed, eh? Okay, perhaps just a little bit then? Well okay maybe the Bcache system is not as fast as the Kingston initial benchmarks. However, the read speed did improved by about 10MB/s give or take, and access speed did drop from 16msec to 0.22msec which probably explains my overall snappy experience. On the downside, Write speeds didn’t improve at all on the Bcache drive even with write-back enabled. It actually decreased a bit, which is disappointing.

By comparison with the benchmarks from before, I started to notice that Read and Write speeds didn’t improve that much. However, access time did improve. This pleased me since SSDs are good for their random-IO access.

From a cost perspective, the Kingston 16GB SSD or something like it costs around $30 and only improved the access time in my case. For some, that is good enough because it gives an overall better experience. However, the steps to accomplish this can be a bit daunting for less experienced users. If you were to spend a $130 more for a 500GB Samsung EVO 850 SSD not only would it increase the access time, but also read and write speeds as well. Also for SSD longevity reasons, TRIM support is on by default in Ubuntu and its distributions, unlike having to manually enable it on the Bcache drive. For comparison purposes I did benchmark the Samsung SSD drive I have. Here are the results.

 

Benchmark comparing /dev/bcache0 on the right and a Samsung EVO 850 Drive on the left.

Conclusion

I wouldn’t recommend Bcache for the average Linux user just due to the complexity of how difficult it is to set up. However, if you were to do this, I would recommend you spend the extra money for a SSD that is 120GB or larger and use some of the SSD’s space to cache your HDD. Not only will your system Disk-IO improve, but accessing your personal files on your HDD will be a better experience.

Like mentioned in the beginning, there are other alternatives when it comes to using SSDs to speed up your spinning disks. Two such examples are EnhanceIO and External Journaling. They are both built into the ext4 filesystem which can be implemented without having to do a fresh install.

So what are your thoughts? Does Bcache seem like it’s worth it to you? If you have any questions about Bcache and how to use it, leave a comment down below. Or if you prefer, you can go to our contact page and fill out the form there. You can also click that email icon on my about section below if you have any questions for me, and/or potential people to interview. Well, as our great RMS (Richard Stallman) would say, “Happy Hacking!



UPDATE 11-16: Thanks to DJ-Pfulio who emailed me about the file permission issues that gedit had when used with sudo. The article has been updated to use sudoedit instead as suggested.


.

Why Open Source Software?

Fox Character Shruggin at the Reader

Hi, my name is Albert Westra. Back when I was around 12 years old, my best friend introduced me to the operating system we all know and love (and grieve for) Mandrake. Back then, I wasn’t really all that interested in open source software. Heck, all I really cared about Mandrake was the fact that I could change the wallpaper on every single virtual desktop, plus mess around with how my desktop looked and felt. Additionally, I was completely overwhelmed by the amount of disks that were required to install it.

A few months later I ended up erasing my parents IBM tower by accident, and that mistake began my curiosity with computers. It wasn’t until I was 15 that I built my first computer, and became interested in computer graphics. It was during this period of time that I was introduced to the wonderful world of Adobe software, specifically Photoshop. I was very intrigued by what I could do with it, but I wasn’t really willing to pony up the money for it after the trial period was up.Then my best-friend showed me Ubuntu 6.10 which had a piece of software pre-installed known as (the) GIMP.

Over the next three to four years I bounced between Gimp/Ubuntu and Photoshop/Windows, mainly because each title had something the other couldn’t do. Fast forward to today, you wouldn’t see me using Windows or Photoshop even if my life depended on it. Especially on Windows 10.

So why go open source? Some chose it because they value their privacy above all else. Especially after the public revelations of a certain NSA whistleblower, and a recent website hack conundrum that has left many a men disappointed, and couch ridden. For others, it is and ever will be a matter of principle. Software to them is a form of free speech which should remain in the open and stay in the open. For me and probably a lot of others switching to open source, the big initial reason was ultimately the cost.

Granted, just because a project is open source and free to download, doesn’t mean it’s free as in beer. It still requires developers to write the code, and when it comes to graphically focused projects, a lot of math. A “shit ton” of math. Enough math to make someone have a hangover from all the math they did the night before. Like the hangover you’d get if you had a few too many and woke up the next morning newly married to a member of a biker gang. And yet… these writers of code make it freely available on the inter-webs for people like you and I to use to our heart’s content. While my initial reasons for using open source software was because I had a big gaping hole in my wallet, it doesn’t mean I don’t own them anything. In fact, I owe these developers a great deal. If it wasn’t for them, I probably would have given up on drawing and pursued something else. I do give back when I can, but it will never amount to what I truly owe them.

Now my next reason for using open source software does kinda flow with the first one, but it’s a little more of an incentive reason. I may not be give as much as I wish I could to projects, however I’m able to focus where that money goes into a particular project. Take the Krita project for example. So far they have launched two Kickstarter campaigns and both have successfully been funded.  What made their Kickstarter work was not only the swag (though we do like swag), but that those who contributed money got to choose the direction the development would go. Granted that list was predetermined before the Kickstarter campaign began, but it was features that had already been requested by the community of Krita. Ten years ago, it was pretty hard to fund the projects you wanted to help, but today you are only 2 to 5 clicks from sending money anywhere in the world. It can also be said that you could use the same money you would have spent on proprietary software to learn a bit of code and fix a bug or add a new feature to the software of your choice. So while most open source software does not cost anything upfront, it does give you  the opportunity to give back financially or even with your personal time.

The biggest reason why I “go open source” and use it daily is ultimately the community. Call me a sucker, but there are some really amazing people in the open source community, and it’s not just the Developers. It’s the writers, artists, engineers, scientists, and others who use the software on a daily basis. If it wasn’t for open source software, we probably wouldn’t be as technologically advanced and knowledgeable as we are now. In fact, the majority of the Internet we use as a communication tool, runs on one of the biggest code collaborations in the world. This collaboration is know as the Linux Kernel. Obviously if it wasn’t for this particular community, we probably wouldn’t be having this conversation right now. Another example of this is the MyPaint community which I’m a part of. I not only enjoy reading about what is going on in the code, but I also enjoy talking to some of the people in the community about it. Plus every once in a while I get to submit a bug fix. Quite simply, I like the interaction.

Is the open source community perfect? I would like it to be perfect, but it is far from it. Just imagine, the person who is the overseer of the Linux Kernel is not a benevolent angel. He can be or is for the lack of a better word a dick. Especially if you yourself have experienced his wrath or been associated with a company that he has flipped off. Now, I’m far from perfect and have made a few mistakes which have affected those around me. But despite our individual challenges and the behavior of a few, this chaos of people from around the world manages to get along and create amazing projects. So all-in-all, it is a beautiful mess. My reasons for going Open Source are probably different than your reasons, and that’s okay. We are people who can not only think as a group, but are individuals as well and we will have different opinions on any subject. One thing is for sure, when we get together and collaborate, we can achieve great things, even if it’s just on the software level.

We have great open source software that can rival its proprietary counterparts, however, don’t forget about the people behind the scenes. One of my main goals here on Freedom Penguin, besides software reviews and technical tutorials, is that I want to interview the people behind the code. Though not just the developers, but also its community. If any of you readers out there have someone that you want interviewed, be sure to contact one of us here at Freedom Penguin. Most of my articles will be focused towards the artist community, because it’s what I love to do. However don’t be afraid to ask me any questions. If I don’t know the answer, we’ll figure it out together. I’m honored to be part of the community Matt has assembled and I hope you will not only read, but enjoy our content.