Notes about open source software, computers, other stuff.

Category: Linux (Page 3 of 8)

Using rsync to backup a ZFS file system to a remote Synology Diskstation

Some time ago I moved from using LVM to using ZFS on my home server. This meant I also had to change the backup script I used to make backups on a remote Synology Diskstation. Below is the updated script. I also updated it such that it now needs a single command line argument: the hostname of the Diskstation to backup to (because I now have two Diskstations at different locations). If you want to run this script from cron you should set up key-based SSH login (see also here and here).

#!/bin/bash
#
# This script makes a backup of my home dirs to a Synology DiskStation at
# another location. I use ZFS for my /home, so I make a snapshot first and
# backup from there.
#
# This script requires that the first command line argument is the
# host name of the remote backup server (the Synology NAS). It also
# assumes that the location of the backups is the same on each
# remote backup server.
#
# Time-stamp: <2014-10-27 11:35:39 (L.C. Karssen)>
# This script it licensed under the GNU GPLv3.
 
set -u
 
if [ ${#} -lt 1 ]; then
    echo -n "ERROR: Please specify a host name as first command" 1>&2
    echo " line option" 1>&2
    exit -1
fi
 
###############################
# Some settings
###############################
# Options for the remote (Synology) backup destination
DESTHOST=$1
DESTUSER=root
DESTPATH=/volume1/Backups/
DEST=${DESTUSER}@${DESTHOST}:${DESTPATH}
 
# Options for the client (the data to be backed up)
# ZFS options
ZFS_POOL=storage
ZFS_DATASET=home
ZFS_SNAPSHOT=rsync_snapshot
SNAPDIR="/home/.zfs/snapshot/$ZFS_SNAPSHOT"
 
# Backup source path. Don't forget to have trailing / otherwise
# rsync's --delete option won't work
SRC=${SNAPDIR}/
 
# rsync options
OPTIONS="--delete -azvhHSP --numeric-ids --stats"
OPTIONS="$OPTIONS --timeout=60 --delete-excluded"
OPTIONS="$OPTIONS --skip-compress=gz/jpg/mp[34]/7z/bz2/ace/avi/deb/gpg/iso/jpeg/lz/lzma/lzo/mov/ogg/png/rar/CR2/JPG/MOV"
EXCLUSIONS="--exclude lost+found --exclude .thumbnails --exclude .gvfs"
EXCLUSIONS="$EXCLUSIONS --exclude .cache --exclude Cache"
EXCLUSIONS="$EXCLUSIONS --exclude .local/share/Trash"
EXCLUSIONS="$EXCLUSIONS --exclude home/lennart/tmp/Downloads/*.iso"
EXCLUSIONS="$EXCLUSIONS --exclude home/lennart/.recycle"
EXCLUSIONS="$EXCLUSIONS --exclude _dev_dvb_adapter0_Philips_TDA10023_DVB*"
 
 
 
###############################
# The real work
###############################
 
# Create the ZFS snapshot
if [ -d $SNAPDIR ]; then
    # If the directory exists, another backup process may be running
    echo "Directory $SNAPDIR already exists! Is another backup still running?"
    exit -1
else
    # Let's make snapshots
    zfs snapshot $ZFS_POOL/$ZFS_DATASET@$ZFS_SNAPSHOT
fi
 
 
# Do the actual backup
rsync -e 'ssh' $OPTIONS $EXCLUSIONS $SRC $DEST
 
# Remove the ZFS snapshot
if [ -d $SNAPDIR ]; then
    zfs destroy $ZFS_POOL/$ZFS_DATASET@$ZFS_SNAPSHOT
else
    echo "$SNAPDIR does not exist!" 1>&2
    exit 2
fi
 
exit 0

Related Images:

Multiple accounts on an SSH server: managing key files

I’ve got several domains hosted at the same hosting company, and the company provides SSH access for each of them with a different user name, but with the same SSH server address. As I’m using key-based login to the server (see also my post here) I ran into the following problem: How do I set up my SSH config file such that it knows which key to use for which user name?

It turns out that the solution is easy (thanks Kelvin!): if you use the %r variable in the ~/.ssh/config file it contains the user name which you used when logging in. Similarly, the %h contains the host name you used on the command line. So all I needed to do was to create entries like this:

Host ssh.myhoster.com
     IdentityFile ~/.ssh/hosting-%r.key

and make sure that the corresponding key files are named hosting-domain1.key, hosting-domain2.key, etc. and then log in using a command like ssh domain1@ssh.myhoster.com.

Related Images:

SSH with several keys: fix “Too many authentication failures” error

Yesterday I created an SSH key for a new machine. Today I try to log in to a different machine, one that actually doesn’t use keys, and I got the error mentioned in the title. It turns out SSH offers all available keys by default, so I ran out of login attempts before I noticed it.

The solution is simple: Add

IdentitiesOnly yes

to your ~/.ssh/config file.

A more detailed explanation can be found here.

Related Images:

Enabling external commands in the Nagios web interface

After an upgrade of one of my Ubuntu server that runs Nagios, I ran into the following error message (again…) when I tried to issue a command from the web interface:

Error: Could not stat() command file ‘/var/lib/nagios3/rw/nagios.cmd’!

This post by Barry O’Donovan shows very nicely how this problem can/should be fixed in Ubuntu. Much cleaner than chmod/chown-ing the directories myself. Thanks Barry!

Related Images:

Fixing font errors when running qmon on a remote server

Sun Grid Engine (SGE) is a bath queue system that can be used to distribute computation intensive tasks across one or more servers/CPUs. SGE has a graphical configuration utility called qmon, but when you start it on a remote machine (using SSH), you may end up with errors like this:

Warning: Cannot convert string "-adobe-courier-medium-r-*--14-*-*-*-m-*-*-*" to type FontStruct
Warning: Cannot convert string "-adobe-courier-bold-r-*--14-*-*-*-m-*-*-*" to type FontStruct
Warning: Cannot convert string "-adobe-courier-medium-r-*--12-*-*-*-m-*-*-*" to type FontStruct
X Error of failed request:  BadName (named color or font does not exist)
  Major opcode of failed request:  45 (X_OpenFont)
  Serial number of failed request:  329
  Current serial number in output stream:  340

The warnings are not really a problem, but the error is. It can be solved by running the following on the client (i.e. your local) machine (assuming it runs Debian or Ubuntu):

sudo apt-get install xfonts-75dpi
xset +fp /usr/share/fonts/X11/75dpi
xset fp rehash

Related Images:

Using ‘expect’ to distribute files among users

I’m currently teaching at the Summmer School in Statistical Omics in Split, Croatia. A great experience!

Because of the computations involved in the project work, we have access to a server. However, since the machine is part of a university cluster, I haven’t been given full root permissions (in fact, I’m only allowed to use sudo to install packages).

Now, the problem I had to solve was that I needed to distribute a certain file (.Renviron) to each student’s home directory. Normally I’d use sudo to do that, but the admin hadn’t allowed me to use cp via sudo. Furtunately, I had a list of user names and passwords for the students (because I had to distribute those), so I thought I’d use su - to change to each student’s account and copy the file, something along the lines of

echo PASSWORD | su -

and then loop over each account. Unfortunately, while testing the script I found out it wouldn’t work since su complained:

su: must be run from a terminal

Then I remembered the expect tool, which executes commands based on what it ‘sees’ on the command line. In this case I wanted it to enter the password at su‘s prompt. This is the expect script I came up with, it accepts two command line arguments, the user name and the password:

#!/usr/bin/expect -f
 
set user [lindex $argv 0]
set pass [lindex $argv 1]
 
spawn su - $user
expect "Password: "
send "$pass\r"
expect "$ "
send "cp -i /common/WORK/school/lennart/.Renviron .\r"
expect "$ "
send "ls -l .Renviron\r"
expect "$ "
send "exit\r"

The script was wrapped in the Bash script that I had already written:

#!/bin/bash
#
# This script is used to copy files from this directory to the
# home directories of the users listed in $USERFILE.
 
USERFILE=accounts.txt
SRCFILE=/common/WORK/school/lennart/.Renviron
 
while read user passw; do
    ./copy_file_to_users.expect $user $passw
done < $USERFILE

Related Images:

Changing the default mode of the Emacs scratch buffer

After starting Emacs you end up in the *scratch* buffer (assuming you’ve disabled the startup message in your .emacs file). The *scratch* can be used for writing down notes and some Lisp experiments (since it uses the Emacs Lisp major mode by default).

Now, I’m not very much of a Lisp programmer, but I do use Org-mode a lot. Consequently, I found myself changing the buffer’s major mode to org-mode regularly. And Emacs wouldn’t be Emacs if you couldn’t change this to a default. So, thanks to Bozhidar Batsov over at Emacs Redux, I’ve added the following lines to my Emacs configuration file:

;; Set the default mode of the scratch buffer to Org
(setq initial-major-mode 'org-mode)
;; and change the message accordingly
(setq initial-scratch-message "\
# This buffer is for notes you don't want to save. You can use
# org-mode markup (and all Org's goodness) to organise the notes.
# If you want to create a file, visit that file with C-x C-f,
# then enter the text in that file's own buffer.
 
")

Related Images:

Fixing backlight control with Ubuntu on my ThinkPad T440s

Some time ago I bought a ThinkPad T440s for work. It’s an amazing machine! Before that I used a ThinkPad X121, which served me very well on my daily commute. This machine was getting a bit old, and given that my new job (more about which in a later post) also requires me to have a better machine with more screen real estate, it was high time to upgrade.

Ubuntu (13.10 and 14.04) runs well on the T440s, only two things didn’t work as expected:

  • The WWAN interface (mobile internet, from Ericsson) seems to connect when I select it in the network manager, but the adaptor seems to disappear almost immediately after that. A few seconds it appears again. [edit 20140514]I just found out that it’s working, probably this was fixed in Ubuntu 14.04[/edit]
  • The screen’s backlight brightness can be reduced/increase using the Fn-F5 and Fn-F6 keys, but only in a weird way: several key presses are needed for one unit of decrease/increase.

This last bug can be fixed by booting with the following kernel argument:

  acpi_backlight=vendor

Simply add this to the GRUB_CMDLINE_LINUX_DEFAULT line in /etc/default/grub:

  GRUB_CMDLINE_LINUX_DEFAULT="acpi_backlight=vendor quiet splash"

and run sudo update-grub. Reboot and you will be able to change the backlight brightness in finer steps.

I found this solution somewhere on the internet a few weeks ago, wrote it down, but can’t remember anymore what the original URL was. My apologies.

Related Images:

Fixing colours in git output after upgrading to Ubuntu 14.04

After upgrading my Ubuntu 13.10 installation to 14.04, I noticed that the output of several git commands (e.g. git diff and git log) didn’t show colours as they used to, but showed ESC[ ANSI codes instead.
A quick internet search lead to this post on unix.stackexchange.com where the LESS environment variable was ‘blamed’. Indeed, I have my LESS variable (re-) defined in my .bashrc and .zshrc files.

The solution was to add -R to the environment variable, which allows raw control characters to be displayed. I now have the environment variable defined as:

LESS='--quiet -X -F -R'

Related Images:

« Older posts Newer posts »

© 2024 Lennart's weblog

Theme by Anders NorénUp ↑