10. More powerful commands

The commands in this section are more command line applications with many diverse features, instead of just single-purpose tools. Still, they all have an option of reading from standard input and output, and can therefore be used in pipes.

ssh, scp

ssh allows secure remote login to other machines you have accounts on. To get an interactive login, use:

$ ssh your_username@hostname

Add the option -X to activate forwarding of graphical displays.

Alternatively, you can run commands remotely with:

$ ssh username@hostname command arguments

To copy files from remote locations, use:

$ scp filename filename

For a remote file, you can replace filename with username@hostname:filename.

A more advanced file copying program is rsync. This can save a lot of time if only a few files have changed, by only copying the ones which are different and by compressing them automatically. It uses ssh to do its network copying, but you may have to set RSYNC_RSH=ssh on older systems.

ssh keys

An alternative to password authentication are ssh keys. To generate a set, use:

ssh-keygen -t dsa   # and just hit "enter" at the first question

This creates a directory .ssh which contains a private key id_dsa and a public key id_dsa.pub. On the remote system that you want to log in to, the public key should now be appended to .ssh/authorized_keys. You can either use ssh-copy-id to do that, or use scp to copy the key to the other system by hand.

Warning

The private key should never be disclosed or made redable by anyone else!

At the next login to a machine that has your public key in the list of authorized keys, you’ll be asked for the key passphrase instead of the remote password. Try replicating this, then read on.

So far we haven’t gained any comfort: instead of the password you now have to type the key passphrase each time. But this part can be automated by using the ssh-agent service. On a graphical desktop it should already be running. Check with:

$ ps x | grep ssh-agent

If not, run it by hand for a child shell:

$ exec ssh-agent bash

You add a key to the agent with ssh-add. This will ask you for the passphrase only once. Any ssh or scp activity will now use the key, and will log you in to the remote machine without any additional requests. Using ssh keys will make remote version control operations especially convenient.

If you have trouble getting SSH keys to work, a common problem is file permissions: your home directory, ~/.ssh directory and the files within should not be group writeable. Also, the key files should be only readable by the owner. If these conditions aren’t met, chances are that SSH will just ask you for a password rather than explain what went wrong!

tar, gzip, bzip2

A convenient way of packaging directories is tar. Try:

$ tar cvf test.tar SomeDirectory

This will create a tarball containing a copy of all the files inside SomeDirectory. The option c stands for create, v is verbose and f is followed by the filename of the tarball. By convention, this should end in .tar.

To extract the files again somewhere else, run:

$ cd tmp
$ tar xvf /your/own/path/to/test.tar

tar tvf lists the files in a tarball.

gzip and bzip2 are two programs that can perform lossless compression on files to save space. Compare the size of 35.txt before and after running:

$ bzip2 35.txt

or:

$ gzip 35.txt

As you can see, by default the programs append .gz or .bz2 and erase the original file. To uncompress a file, run:

$ bunzip2 35.txt.bz2

or:

$ gunzip 35.txt.gz

tar can compress its tarball automatically, without needing to run gzip afterwards. To do this, add the option z for gzip or j for bzip2 to the tar command line. To avoid confusion, you should provide the right file endings .tar.bz2 or .tar.gz.

A piped use of tar is the following useful line. What does it do?:

$ ssh name@location "tar cj SomeDirectory" | tar xj

When dealing with thousands of files inside the directory, this is much faster than scp -r. If you need do such a transfer regularly, you’re better off with rsync, though.

What does the next line do?:

$ tar cj SomeDirectory | ssh name@host "cat > archive.tar.bz2"

Editing text

Text editors belong to the basic toolkit in any operating system. The choice in Linux is correspondingly large. On a typical system, you’ll find nano for quick edits, emacs or vi for larger projects. gedit or kate are more integrated into the graphical OS. Many more editors exist. Any of them do the basic job of editing a text file very well, and should have syntax highlighting.

To find an editor you like, try them out for a while. Here’s a large-ish file from project Gutenberg: [H.G. Wells, The Time Machine]. Right-click to copy the link’s URL and use wget to download it from the shell.

Open the file in the editors you want to try out with someeditorname 35.txt & (do you remember what the & does?) and try something like

  • Find the first occurrence of of course in the text.
  • Replace all occurrences of time with foo.
  • Mark a section of the text, cut it and move it somewhere else.