Lessons from the Linux Mint Hack

Lessons from the Linux Mint Hack

POSTED 3:43PM PST, Sunday — Unless you’re completely unplugged from the Linux news media, by now you’ve heard about the exploit that affected both the Linux Mint WordPress site and the Linux Mint 17.3 Cinnamon edition.

What you need to know:

  • Softpedia provides a solid account and breakdown of events. However, they did miss something…more on that later. If you’re into screen shots and the details of the event, check it out.
  • ISO Torrents were not affected.
  • SSL wouldn’t have protected squat. Don’t misunderstand, it does protect against OTHER potential attacks, but the initial point of entry was WordPress. Remember the entry point of attack was WordPress, so for this specific attack, Clem’s statement below is correct. However, offering anything for download without SSL in play is a special kind of dangerous. Let’s hope they keep the site offline until SSL is implemented.

nizzle Says:
February 21st, 2016 at 2:46 am
Doesn’t do much good to post hashes on a site that’s not served over TLS.
When will *.linuxmint.com go https only?
Edit by Clem: It’s planned and I’m hoping it’ll happen soon. Please note that this wouldn’t have helped here though. You’d be served the exact same hacked information via HTTPs.

  • Checksums simply don’t cut it, however, end users won’t mess with OpenPGP secret keys….unless you force them to. Most people simply don’t understand the importance of using them or simply don’t care.
  • twitterThis isn’t a new issue. Mind the date of this tweet…clearly Mint’s web team needs training or some assistance with security. And then a month later (note the date again) we see things finally being addressed.
  • Critical: Mint doesn’t release security advisories to trusted alert sites at this time.
  • Fixed/irrelevant?: Seven hours ago, I received an unsubstantiated claim that their blogging site was running WordPress 4.2.2. I can’t speak to this, however I can tell you their site as of the time of this post runs WordPress 4.4.2. I’d file this under rumor-mill for now. Pinguy OS‘ Antoni Norman has confirmed that this rumor is not true after checking through the cache of the site.

How to protect ourselves

As a community, casual Linux users (myself included) are generally pretty complacent. We blindly download anything and everything from PPAs to AUR packages, assuming that nothing has slipped through the cracks. Granted, I believe the AUR receives greater scrutny than the current PPA system, but that’s beside the point.

The key things we can do to protect ourselves as end users are:

  • Only download ISOs from trusted sources. This means distribution providers need to make sure “trusted download” options are provided. Adding a SSL cert isn’t a fix, there needs to be additional measures put into place. Whether or not this means OpenPGP is the solution remains to be seen. I honestly don’t think it is.
  • Get to know the MD5 signature(s). Before you install a downloaded ISO, verify the hash sum for that ISO. Far from a great solution, at least as an end user you did SOMETHING to protect yourself.
  • Locking down your distro locally (for end users). Arch documentation has a solid write up that you should familiarize yourself with. This can help minimize the damage that a exploit can do to your system. So instead of taking the scan and pray approach, use these techniques to prevent the available ground an attack can exploit. Pay special attention to the firewall and root control.
  • Keep spare (trusted) ISOs handy locally. This borders on the “security through obscurity” philosophy, but it also means you have safe/functional ISOs available no matter what.

There is no spoon

The fact of the matter is if you can read/write to a system, there is an opportunity to exploit it. All we can do is control the attack surface available and minimize it as much as possible. Remember: Lock it down, keep it patched and pay attention. This is the best advice anyone could hope for coming from the end user perspective.

On the distribution and server side of things, I think this provides us with an opportunity to re-examine how we’re distributing Linux ISOs. Yes, minding the IP sources and hash sums are a “fair” place to start. I for one, think we can do better and I would like to see some new ideas. Fact of the matter is, this won’t be the first time this happens. And thankfully, due to Clem’s rapid response, this event was addressed very quickly. Let’s hope some tough lessons were learned here to prevent another event in the future.

For the sake of research, I saved a cached copy of the download page for Linux Mint 17.3 Cinnamon edition. Every single download for each country is pointing to the malicious IP address. Worse, the mere act of clicking on any of the download links instantly starts the download process – no browsing of the directory. Folks, the current method for downloading ISOs is in need of something a bit more secure.

Why this bitter pill is a good thing – long term

At the end of the day, I see this as a positive. First, I believe that Linux Mint will come out of this stronger than ever. Second, this will force others to take ISO security more seriously. This also provides end users with a stronger reason to pay closer attention to what they’re doing.

I don’t know how all of this will play out. But two things I do know for sure. The Linux Mint team did an outstanding job dealing with this right away. Also, downloading ISO images from randomly linked mirrors might not be the most secure way to distribute today’s modern Linux distributions.

Matt Hartley
Freedom Penguin’s founder & talking head – Matt has over a decade working with Linux desktops, his operating system experience consists of both Windows and Linux operating platforms. In addition to writing articles on Linux and open source technology for Datamation.com and OpenLogic.com/wazi, Matt also once served as a co-host for a popular Linux-centric podcast.

Matt has written about various software titles, such as Moodle, Joomla, WordPress, openCRX, Alfresco, Liferay and more. He also has additional Linux experience working with Debian based distributions, openSUSE, CentOS, and Arch Linux.

4 thoughts on “Lessons from the Linux Mint Hack”

  1. It seems to me there are 3 things that need implementing as broadly across the board as possible:

    1/ GnuPG tooling must be made easy from the GUI and CLI – manage sources of public keys in one operation; add/remove keys in a second operation, verify target files as a third operation. No more, no less, make it easy.

    2/ Decentralize the public key servers such that they are able to sync public keys to eachother, but that a breach and key-replace of one server does not topple over to the other servers. A client should be querying multiple key servers for redundant security.

    3/ ISO signing (and indeed, any software signing) needs to happen off-network. Take your binary, put it on a USB stick. Scan stick in another system using a different OS (BSD, Solaris, Haiku, wahtever) that is offline, perform signing on a third machine which is also offline. It’s more cumbersome, but with core software like this, it’s likely worth it.

  2. A culture of blind trust is another issue – there’s this very worrying tendency to want the latest and greatest – which leads to (as you point out) adding PPAs ad-hoc, using the AUR without second thought, and simply running git clone ; make ; make install … and vagrant up, running docker images, etc etc

    This comes from developers wanting new releases to build with (buffer overflows etc there?), users wanting latest software (backdoors there?), rookie sysadmins wanting to make their lives easier (hoo hoo get hold of a sysadmin’s account, now we’re talking!).

    We’re no better than getting a re-pimped gimp from Sourceforge when we do this.

    Why not stay with slow-movers like Debian, CentOS or a BSD? Because you can’t guarantee that all packages are going to receive security updates for their entire lifetime. Case in point the ownCloud debacle, where they said “don’t use the versions from the repos, we’re not maintaining them.” OK fine, let’s add a PPA, with continually updated code. Whoops, back to square one.

    I use the repos pricesly because I am counting on that to be a base web of trust. If I can’t trust the repos, and I can’t trust ad-hoc PPAs without deeper work, what are my options??

    We’re far from out of the woods…

  3. One issue is that Mint makes you backup, nuke and pave whenever there is a major release. Because the vast majority of users will turn to mint’s web page to download the .iso, this creates a potential vulnerability that just doesn’t exist in distros that don’t do upgrades this way.

    I agree with earlier posters who called for more user-friendly GPG verification. If it’s not easy to use, it’s just not going to be used.

Leave a Comment