The X First Aid Kit For Linux
If you spend some time on Linux-based social media sites or forums, you inevitably see posts where a user is having trouble logging into their desktop. Either the boot finishes and they’re left looking at a black screen, or they can’t seem to get past the display manager login.
“I enter my user name and password, the screen goes black for a second, then it goes back to the login screen.”
One of the most common ‘fixes’ I see is the suggestion to reinstall the distro, or various parts of it. I cringe every time I see this. There’s really no reason to reinstall the OS unless one has been screwing around in the system to the point they’ve already trashed it beyond repair, or the installation ISO was bad to begin with. In which case reinstalling won’t fix anything when you’re using the same ISO to do it. In the second instance, here’s a suggested Rule of Thumb: always check the hash signature of a downloaded ISO against the one posted on the distro website before trying to install. Most distros put the signature right on the download page, but not all. With some you might actually have to search for elsewhere, but take the time to find and check it. Because it can save hours of headaches later.
Another common scenario occurs after an update of a perfectly working system, when the X server, display manager or desktop refuses to start. In either case, whether a new install or an update has gone bad, reinstalling, uninstalling or rolling-back shouldn’t be the first course of action. You can probably fix the problem yourself with a few tools and a little time.
Let me state here though, if you have a phobia or complete aversion to the command-line (which I will refer to with the acronym ‘CLI’ from here on), you might as well stop reading this now. Same goes for those who have no real interest in how their system works or trying to fix it themselves.
One of the beauties of Linux is that it was designed from the start to be a multi-user environment. My first install of Linux back in the Dark Ages didn’t have a GUI, but I quickly learned that if I pressed CTRL+ALT+ I could log into another session and have a whole new screen to work in. Pressing CTRL+ALT and a function key allowed me to switch back and forth between the different sessions where I would be doing different things, much like the virtual desktops in a GUI. A bit crude perhaps by today’s standards, but it worked and was instrumental in hooking me on Linux. Fortunately for those whose GUI has failed them, this still works on any Linux system. Even now, while you’re reading this, you should be able to press CTRL+ALT+F1 and switch to a command console with a prompt for you to log in. Most modern distros hook the X session to F5 or F7 to switch back, you’ll probably have to try it to see which one of the function keys brings you back to the GUI.
A WORD OF WARNING: on some distros when using the proprietary drivers (nVidia or ATI/AMD) switching to a console can cause the driver to unload, destroying your X session. If you use one of the proprietary drivers, don’t try switching to a console, especially if you have anything running in the GUI that you can’t or don’t want to lose. If you want to try it, start from a fresh, empty session. If you find you can’t switch back to the GUI, or X has died, do a ‘sudo reboot’ from the console to restart the system. Yes, there are ways of restarting the X server, but they can differ depending on the distro, so you would have to check with your particular one to find out how to do it. A reboot should work everywhere.
Now, we’re not going to discuss how to fix a broken X or display server here. There’s simply too many differences in how it gets started in the various distros, because what works in one may not in another. What we are going to cover is some tools almost all distros will have in their repositories. Tools I recommend installing that can help in the event of a failed GUI They’ll not only help you find and fix the problem, but more importantly some allow you to connect with the outside world to get help and information. They’re CLI programs that are the first things I add to a new install. After years of trying tons of different ones, I’ve chosen these due to the fact you don’t need to be an uber-geek to use them. They’re all fairly simple to understand and work with.
First thing I recommend is to learn how to use the disto’s CLI package manager. All distros have a command-line utility for installing, removing, updating or repairing packages. Whether it’s apt-get, pacman, zypper, urpmi or whatever they call it, it’s there and installed by default. Learning the basics can make all the difference when you find yourself stuck in console mode. Often a problem stems from a broken package or missing dependency, so learning how to make at least basic use of the CLI package manager is well worth the effort. With some distros it’s the only way of working with packages, since they have no GUI tool. For others, they wrap it in a pretty GUI and many users never touch the CLI version. Try it, in fact, try installing some or all of the following CLI applications with it. You might find you prefer it to the GUI one, and if not, at least you’ll know how to use it in an emergency.
Top of my list is Midnight Commander, otherwise known as ‘mc’. This is a dual-pane file manager on steroids. There are plenty of others, you may want to try some of them to compare for yourself. I chose mc because it’s intuitive and most operations can be done via menus or function keys. The most common file operation function keys are shown right on the bottom of the screen. It has a built-in help system, so it’s easy to find out how to do something that’s not in the menus you might need to do. It has a built-in text editor, and although most distros will install a text editor like nano, vi or vim (or all of them) by default, I find being able to navigate to a file with mc then edit it all within the same application fast and convenient. The built-in editor might not have all the power and options of the others, but for simple script editing it’s more than adequate. Like the file manager itself, the most common operations people do on text are mapped to the function keys listed at the bottom of the screen, while others can be found via the help screens.
When fixing a bug in startups, it involves editing a script or configuration file, and mc’s editor will do that easily. You’ll likely find yourself looking though log files to track down a problem, again, mc makes this so much easier than having to remember a bunch of terminal commands. Another nice feature is if you need to issue a terminal command while in mc, you can do that too and view the output, all without having to quit mc.
As with all the applications I’ll suggest here, or any you might choose to use instead, take some time to learn the basics of it’s use. After installing, pop it open in a terminal on your desktop and try it out. Get to know how to navigate around with it, open a file for editing, view a file, search for one, etc. A little time spent with it once in a while will help tremendously if and when you actually need it. Who knows, you might just like it enough to use it regularly. I do. I find myself using mc in a terminal just as much, if not more than my GUI file manager. Simply because in many things it’s faster and more powerful than the GUI one. Also, I tend to be keyboard-centric, keeping my hands on the keyboard, and GUI file managers require a lot of mouse work. With mc it’s pop-open a terminal, type ‘mc’ at the prompt, hit ENTER and go to work.
The next CLI application I install is htop. I think every Linux distro on the planet installs ‘top’ by default, and it is a great tool for seeing what processes are running (or not running) and much more. It’s truly a Swiss Army Knife of process investigation tools. There’s tons of things it can do, but it’s not very intuitive or user-friendly. And while there’s plenty of other process-viewing applications around, I find htop the easiest to read and most intuitive to use. Again, like mc it has common operations bound to function keys listed on the bottom of the window. Often when starting to diagnose a problem, one of the first things to do is find out what is running, and what isn’t. With a glance you can see if the X server is running, if the display manager is, or any other process you’re interested in. Using htop’s tree view you can see the parent/child relationships of processes, what process spawned what other, getting an understanding of how they all connect. With it’s sort command you can sort them by PID to see what order they started in, as the order of starting up can make a difference with things that are dependent on something else. You can sort by CPU usage to see if one’s hogging the system, and many other ways to see what you want to find out. You can see their status, whether they’re running, sleeping or “zombied”, and much more. While htop can’t fix anything, it’s a great tool that’s easy to use for getting an idea of the current state of the system, which can help in diagnosing what the problem is.
For many users, the computer they’re having problems with is the only computer they own. If X or the display manager or desktop fail to start, they have no access to their best resource for help, their web browser. While one can use a smart phone to seek help, if you’ve ever tried to read something like the Arch Wiki or Ask Ubuntu on a phone, it’s not all that easy. Fortunately, there are several very good CLI browsers available. My personal choice is w3m, mostly because it’s fairly easy to use, and it can display graphics. Lynx is another popular CLI browser. No CLI browser will give you the pretty laid-out view of a web page that a GUI browser will, but when we’re trying to find information, text is really all we need. w3m has the added benefit that if a web page has a graphic, like a screen-shot showing something as an example, w3m can display it. As you can see in the screen-shot here, a CLI browser is pretty basic in rendering a web page, but will get the job done. To invoke w3m at a console (or the GUI terminal) type ‘w3m web address’, “web address” being the web site you want. In the example, I entered ‘w3m https://google.com’. When the page came up, I typed in my search term (in this case it was ‘systemd’). The page displayed is the result. Arrow keys, PGUP & PGDWN all work to move around the page, TAB works to move from one link to another, and ENTER to go to that link. Admittedly it’s not the easiest way to browse the web, but CLI browsers are fast, light, and will get the job done. Again, it helps to spend at least a little time using the browser in a terminal window from your desktop, just to get familiar with it’s basic use. That way you’re not faced with trying to figure out how to work it when you might actually need to. And while I wouldn’t want to use w3m or any of the other CLI browsers as my daily driver, it and Google have bailed me out more than once when I was stuck at a console. I consider a CLI browser a must have for any Linux first-aid kit.
There are times when we just can’t seem to find what the problem is on our own, and we need more help than what we’re finding on the web. This is where IRC can make the difference. It’s one of the oldest forms of social networking. Almost every distro out there has an IRC channel, with many of the larger distros having several. Getting on a distro’s IRC channel and asking for help can sometimes make the difference in whether we get the system going again or not. Linux is blessed with many great CLI IRC clients. The two most popular are probably irssi and weechat. I prefer weechat myself, because it has the ability to split it’s own window into several panes, so you see multiple connected channels at the same time. I like weechat so much that it’s actually my daily driver on IRC. To get the full benefit of what it’s capable of takes some time in configuring and learning it, but it’s worth the effort in my opinion. However, as part of a first-aid kit, it doesn’t need any configuration, it’s perfectly usable with it’s default settings.
Whichever CLI IRC client you decide to use, learn how to connect to an IRC server and join a chat with it. If you’ve never used IRC before, now’s a great time to try it! You can learn most of the basics of IRC here and here. Then check with your distro’s website to see what IRC channels they offer, what network they’re on, and anything else they may think is important for you to know. When ready, take your CLI IRC client for a test drive and join the channel. You don’t have to say anything, you can just idle on the side and watch how things work if you wish, or jump right in and say “Hi”.
When using IRC for problem solving help: be patient. When you ask a question on IRC, it’s not uncommon for some time to pass before anyone answers. Even when you see several people chatting away on the channel, most will not answer a query when they honestly don’t know the answer themselves. Likely there’s someone there who can help, but they may not be looking at the IRC chat at the moment and don’t see the request. Give it some time, 5 or 10 minutes minimum to see if there’s a response. If not, then try again, politely. If you’ve been watching someone helping another with their problem, you could ask them directly if they have any experience with yours. Something like this:
“I’m having a problem with X. You seem to know quite a lot, think you could help? I’d greatly appreciate any advice.”
Most importantly, be patient and be polite. Over the years I’ve had great luck with help from IRC users, but admittedly it often took a while before someone came along who could help.
The last app for our first-aid kit is totally optional, but I find it quite handy. It’s what’s known as a console or terminal screen multiplexer. As I stated earlier, in Linux you can have several console sessions open on different terminals, switching with the CTRL+ALT+F# keys. You could have mc running in one, htop running in another, w3m in another, weechat in another, etc., etc.. Problem is, sometimes we’re following instructions from the web on something that’s a bit complex to do. Switching to another screen where we can’t see the original instructions can be an issue. Trying to remember a complicated sequence of commands or things that need to be done can lead to errors, possibly making things worse. Truth is, we’re quite spoiled with our GUIs, where we can have multiple windows open on the single screen, following the instructions directly. Fortunately, we can also do this in a CLI console with a multiplexer. The two most used and available in pretty much any distro are ‘screen‘ and ‘tmux‘. Personally I use tmux. With tmux you can have multiple screens, similar to tabs in a browser, as well as splits in the current screen, much like a tiling window manager. Like all of the other apps I’ve suggested tmux has many more things it can do, but for our purposes the most important is the ability to place several of the other tools into a single screen as shown in the screen-shot. It enables us to do what we need to while reading or getting instructions, all on the same screen. And like the other applications, tmux is extremely configurable but requires no configuration to use. The example in the screen-shot is plain out-of-the-box tmux. Learning some of it’s default key bindings so we can create splits in the window and move between them is really all that’s needed.
Admittedly, trying to diagnose and fix a problem with X—or anything else—from a command console is not for everyone. There are many of us who just want to use the computer and could care less about how it works. But if you’re like me, interested in how it functions, willing to try something new and learn new things, these tools can be invaluable when the GUI goes south. For many the command line is some mysterious place that only uber-geeks go. These tools put a more familiar, friendlier face on it. With them we can move around in our system, see what it’s doing, and connect to the outside world. Over the years I’ve tried many utilities and these are the ones I’ve settled with, because they’re simple to use and get the job done. Even though I’ve used Linux for nearly 20 years now, I’m not a programmer or command-line guru. But I do like to tinker and I’ve broken my systems more times than I can remember. Being able to find help from the web at the console has been priceless. And honestly, most fixes have been very simple once I found where the problem was. Things like the wrong permissions on an executable, a script calling something that doesn’t exist or had it’s name changed in an update, or a missing package. Only once in all those years have I ever screwed things up so bad that the only recourse was to completely reinstall the system. Besides, there’s a real feeling of accomplishment when we can fix something ourselves.