Lennart's weblog

Open source, computers, Africa and other more (or less) interesting stuff.

Tag: SSH

Getting the version of a remote SVN repository via SSH

A quick note to self: I wanted to find out what Subversion version was run on R-forge, which I access via SSH. This is how to do it:

$ ssh username@svn.r-forge.r-project.org svnserve --version
svnserve, version 1.6.17 (r1128011)
   compiled Nov 20 2011, 01:10:33

Copyright (C) 2000-2009 CollabNet.
Subversion is open source software, see http://subversion.apache.org/
This product includes software developed by CollabNet (http://www.Collab.Net/).

The following repository back-end (FS) modules are available:

* fs_base : Module for working with a Berkeley DB repository.
* fs_fs : Module for working with a plain file (FSFS) repository.

Cyrus SASL authentication is available.

Using rsync to backup a ZFS file system to a remote Synology Diskstation

Some time ago I moved from using LVM to using ZFS on my home server. This meant I also had to change the backup script I used to make backups on a remote Synology Diskstation. Below is the updated script. I also updated it such that it now needs a single command line argument: the hostname of the Diskstation to backup to (because I now have two Diskstations at different locations). If you want to run this script from cron you should set up key-based SSH login (see also here and here).

#!/bin/bash
#
# This script makes a backup of my home dirs to a Synology DiskStation at
# another location. I use ZFS for my /home, so I make a snapshot first and
# backup from there.
#
# This script requires that the first command line argument is the
# host name of the remote backup server (the Synology NAS). It also
# assumes that the location of the backups is the same on each
# remote backup server.
#
# Time-stamp: <2014-10-27 11:35:39 (L.C. Karssen)>
# This script it licensed under the GNU GPLv3.
 
set -u
 
if [ ${#} -lt 1 ]; then
    echo -n "ERROR: Please specify a host name as first command" 1>&2
    echo " line option" 1>&2
    exit -1
fi
 
###############################
# Some settings
###############################
# Options for the remote (Synology) backup destination
DESTHOST=$1
DESTUSER=root
DESTPATH=/volume1/Backups/
DEST=${DESTUSER}@${DESTHOST}:${DESTPATH}
 
# Options for the client (the data to be backed up)
# ZFS options
ZFS_POOL=storage
ZFS_DATASET=home
ZFS_SNAPSHOT=rsync_snapshot
SNAPDIR="/home/.zfs/snapshot/$ZFS_SNAPSHOT"
 
# Backup source path. Don't forget to have trailing / otherwise
# rsync's --delete option won't work
SRC=${SNAPDIR}/
 
# rsync options
OPTIONS="--delete -azvhHSP --numeric-ids --stats"
OPTIONS="$OPTIONS --timeout=60 --delete-excluded"
OPTIONS="$OPTIONS --skip-compress=gz/jpg/mp[34]/7z/bz2/ace/avi/deb/gpg/iso/jpeg/lz/lzma/lzo/mov/ogg/png/rar/CR2/JPG/MOV"
EXCLUSIONS="--exclude lost+found --exclude .thumbnails --exclude .gvfs"
EXCLUSIONS="$EXCLUSIONS --exclude .cache --exclude Cache"
EXCLUSIONS="$EXCLUSIONS --exclude .local/share/Trash"
EXCLUSIONS="$EXCLUSIONS --exclude home/lennart/tmp/Downloads/*.iso"
EXCLUSIONS="$EXCLUSIONS --exclude home/lennart/.recycle"
EXCLUSIONS="$EXCLUSIONS --exclude _dev_dvb_adapter0_Philips_TDA10023_DVB*"
 
 
 
###############################
# The real work
###############################
 
# Create the ZFS snapshot
if [ -d $SNAPDIR ]; then
    # If the directory exists, another backup process may be running
    echo "Directory $SNAPDIR already exists! Is another backup still running?"
    exit -1
else
    # Let's make snapshots
    zfs snapshot $ZFS_POOL/$ZFS_DATASET@$ZFS_SNAPSHOT
fi
 
 
# Do the actual backup
rsync -e 'ssh' $OPTIONS $EXCLUSIONS $SRC $DEST
 
# Remove the ZFS snapshot
if [ -d $SNAPDIR ]; then
    zfs destroy $ZFS_POOL/$ZFS_DATASET@$ZFS_SNAPSHOT
else
    echo "$SNAPDIR does not exist!" 1>&2
    exit 2
fi
 
exit 0

Multiple accounts on an SSH server: managing key files

I’ve got several domains hosted at the same hosting company, and the company provides SSH access for each of them with a different user name, but with the same SSH server address. As I’m using key-based login to the server (see also my post here) I ran into the following problem: How do I set up my SSH config file such that it knows which key to use for which user name?

It turns out that the solution is easy (thanks Kelvin!): if you use the %r variable in the ~/.ssh/config file it contains the user name which you used when logging in. Similarly, the %h contains the host name you used on the command line. So all I needed to do was to create entries like this:

Host ssh.myhoster.com
     IdentityFile ~/.ssh/hosting-%r.key

and make sure that the corresponding key files are named hosting-domain1.key, hosting-domain2.key, etc. and then log in using a command like ssh domain1@ssh.myhoster.com.

SSH with several keys: fix “Too many authentication failures” error

Yesterday I created an SSH key for a new machine. Today I try to log in to a different machine, one that actually doesn’t use keys, and I got the error mentioned in the title. It turns out SSH offers all available keys by default, so I ran out of login attempts before I noticed it.

The solution is simple: Add

IdentitiesOnly yes

to your ~/.ssh/config file.

A more detailed explanation can be found here.

Fixing font errors when running qmon on a remote server

Sun Grid Engine (SGE) is a bath queue system that can be used to distribute computation intensive tasks across one or more servers/CPUs. SGE has a graphical configuration utility called qmon, but when you start it on a remote machine (using SSH), you may end up with errors like this:

Warning: Cannot convert string "-adobe-courier-medium-r-*--14-*-*-*-m-*-*-*" to type FontStruct
Warning: Cannot convert string "-adobe-courier-bold-r-*--14-*-*-*-m-*-*-*" to type FontStruct
Warning: Cannot convert string "-adobe-courier-medium-r-*--12-*-*-*-m-*-*-*" to type FontStruct
X Error of failed request:  BadName (named color or font does not exist)
  Major opcode of failed request:  45 (X_OpenFont)
  Serial number of failed request:  329
  Current serial number in output stream:  340

The warnings are not really a problem, but the error is. It can be solved by running the following on the client (i.e. your local) machine (assuming it runs Debian or Ubuntu):

sudo apt-get install xfonts-75dpi
xset +fp /usr/share/fonts/X11/75dpi
xset fp rehash

Using SSH for WordPress updates via the web interface

Note to self: if you want to be able to upgrade WordPress via the web interface using SSH, you need to have the libssh2-php package installed on Debian/Ubuntu Linux.

Don’t forget to restart Apache after installing the package. Reload the WordPress admin pages and you’ll see the SSH option added.

Permantly ban an IP address with fail2ban

Over the last few days I noticed in my logwatch e-mails that one IP address kept trying to log in to my server, even though it was blocked regularly by fail2ban.

Here’s a post that explains how to simply add a list of IP addresses to block permanently. There’s only one catch: the listing provided there contains an error, the word <name> is missing in the iptables command, probably due to HTML conversion. This is the correct line to be insterted into the actionstart section of /etc/fail2ban/action.d/iptables-multiport.conf:

cat /etc/fail2ban/ip.blacklist | while read IP; do iptables -I fail2ban-<name> 1 -s $IP -j DROP; done

Use the following command to check if the IP address is indeed banned:

$ sudo iptables  -L fail2ban-ssh
Chain fail2ban-ssh (1 references)
target     prot opt source               destination         
DROP       all  --  192.168.20.25        anywhere            
RETURN     all  --  anywhere             anywhere 

Using rsync to backup to a remote Synology Diskstation

An updated version of the script can be found here.

I recently bought a NAS, a Synology DiskStation DS211j and stuffed two 1TB disks in it. I configured the disks to be in RAID 1 (mirrored) in case one of them decides to die. I then brought the NAS to a family member’s house and installed it there. Now she uses it to back up her important files (and as a storage tank for music and videos).

The good thing for me is that I can now make off-site backups of my home directories. I configured the DS211j to accept SSH connections so that I can log into it (as user admin or root). I used the web interface to create a directory for my backups (which appeared to be /volume1/BackupLennart after logging in with SSH).

After making a hole in her firewall that allowed me to connect to the DS211j, I created a backup script in /etc/cron.daily with the following contents:

#!/bin/bash
#
# This script makes a backup of my home dirs to a Synology DiskStation at
# another location. I use LVM for my /home, so I make a snapshot first and
# backup from there.
#
# Time-stamp: <2011-02-06 21:30:14 (lennart)>
 
###############################
# Some settings
###############################
 
# LVM options
VG=raidvg01
LV=home
MNTDIR=/mnt/home_rsync_snapshot/
 
# rsync options
DEST=root@remote-machine.example.com:/volume1/BackupLennart/
SRC=${MNTDIR}/*
OPTIONS="-e ssh --delete --progress -azvhHS --numeric-ids --delete-excluded "
EXCLUSIONS="--exclude lost+found --exclude .thumbnails --exclude .gvfs --exclude .cache --exclude Cache"
 
 
 
###############################
# The real work
###############################
 
# Create the LVM snapshot
if [ -d $MNTDIR ]; then
    # If the snapshot directory exists, another backup process may be
    # running
    echo "$MNTDIR already exists! Another backup still running?"
    exit -1
else
    # Let's make snapshots
    mkdir -p $MNTDIR
    lvcreate -L5G -s -n snap$LV /dev/$VG/$LV
    mount /dev/$VG/snap$LV $MNTDIR
fi
 
 
# Do the actual backup
rsync $OPTIONS $EXCLUSIONS $SRC $DEST
 
# Remove the LVM snapshot
if [ -d $MNTDIR ]; then
    umount /dev/$VG/snap$LV
    lvremove -f /dev/$VG/snap$LV
    rmdir $MNTDIR
else
    echo "$MNTDIR does not exist!"
    exit -1
fi

Let’s walk through it: in the first section I configure several variables. Since I use LVM on my server, I can use it to make a snapshot of my /home partition. The LVM volume group I use is called ‘raidvg01′. Withing that VG my /home partition resides in a logical volume called ‘home’. The variable MNTDIR is the place where I mount the LVM snapshot of ‘home’.

The rsync options are quite straight forward. Check the rsync man page to find out what they mean. Note that I used the --numeric-ids option because the DS211j doesn’t have the same users as my server and this way all ownerships will still be correct if I ever need to restore from this backup.

In the section called “The real work” I first create the MNTDIR directory. Subsequently I create the LVM snapshot and mount it. After this the rsync backup can be run and finally I unmount the snapshot and remove it, followed by the removal of the MNTDIR.

Since the script is placed in /etc/cron.daily it will be executed every day. Since we use SSH to connect to the remote DS211j I set up SSH key access without a password. This Debian howto will tell you how to set that up.

The only thing missing in this setup is that the backups are not stored in an encrypted form on the remote NAS, but for now this is good enough. I can’t wait until the network bandwidth on both sides of this backup connection get so fast (and affordable) that I can easily sync my music as well. Right now uploads are so slow that I hardly dare to include those. I know that I shouldn’t complain since the Netherlands has one of the highest broadband penetrations in the world, but, hey, don’t you just always want a little more, just like Oliver Twist?

Script to tunnel RDP connections through stepping stone server using SSH

In order to connect to the servers at work we need to connect through a stepping stone host (via SSH). Since some of the servers are MS Windows machines which can be managed via the Remote Desktop Protocol (RDP), this traffic needs to be tunneled over SSH as well.
I wrote the following bash script to automate setting up the tunnel. It sets some default variables and then looks for an available port between 1234 and 1254 (chosen completely arbitrarily) and uses it for the tunnel. Then it uses the rdesktop program to start the RDP connection.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
#!/bin/bash
#
# This script makes an ssh tunnel to a stepping stone server
# and uses it to start an rdesktop connection to the machine 
# given as the first argument of the script. 
#
# (C) L.C. Karssen
# $Id: winremote.sh,v 1.14 2010/02/10 13:03:08 lennart Exp $
#
 
# User-configurable variables
ssh_username=your_steppingstone_username_here
steppingstone=steppingstone.your_company.com
rdesktop_username=your_windows_username_here
rdesktop_domain=your_windows_domain_here
rdesktop_opts="-z -g 1024x768 -a 16"
rdesktop_port=3389 # This is the standard MS rdesktop port
 
 
# For ordinary users it should not be necessary to change anything below this line. 
# Some functions:
usage()
{
    cat <<EOF
Usage:
    $program [options] rdesktop_hostname 
 
Make a remote desktop connection through an SSH tunnel.
 
Options: 
    -h, --help                                   print this help message
    -s, --steppingstone steppingstone_hostname   set other stepping stone host
                                                   (default: $steppingstone)
    -t, --timeout sec                            set timeout (default: 1)
    -v, --verbose                                verbose output
     --version                                   print version
 
Note that some customisations need to be made in the first few lines of this 
script (e.g. user names and other defaults)
EOF
}
 
program=`basename $0`
 
# Command line option parsing. Shift all options 
verbose=
timeout=1
 
while [ $# -gt 0 ]
do 
    case $1 in
	-v | --verbose | -d | --debug ) 
	    verbose=true
	    ;;
	--version )
	    echo '$Revision: 1.14 $'
	    exit 0
	    ;;
	-t | --timeout ) 
	    shift
	    timeout="$1"
	   if [ $timeout -lt 1 ]; then
	       timeout=1
	   fi
	   if [ $verbose ]; then
	       echo "Timeout set to $timeout"
	   fi
	   ;;
	-s | --steppingstone ) 
	   shift
	   steppingstone="$1"
	   if [ $verbose ]; then
	       echo "Steppingstone server is $steppingstone"
	   fi
	   ;;
	-h | --help ) 
	   usage
	   exit 0
	   ;;
	-*) 
	   echo "$0: invalid option $1" >&2
 	   usage
	   exit 1
	   ;;
	*) 
	   break
	   ;;
    esac
    shift
done
 
# Server name (as seen on the steppingstone) that we want to connect to:
rdesktop_server=$1 
 
################### Config done, let's get to work ########################
 
# Simple usage description
if [ "$rdesktop_server" == "" ]; then
    echo "Error: No rdesktop host given" >&2
    usage
    exit 1
fi
 
# Find a free port on the local machine that we can use to connect through
min_port=1234
max_port=1254
used_ports=(`netstat -tan | awk '{print $4}' | grep 127.0.0.1 | awk -F: '{print $2}' | sort -g`)
if [ $verbose ]; then
    echo "Used ports are: ${used_ports[@]}"
fi
 
# In the next line we first print the $used_ports as an array, but with 
# each port on a single line. This is then piped to an awk script that 
# puts all the values in an array and subsequently walks through all ports 
# from $min_port to $max_port in order to find the first port that is not 
# in the array. This port is printed.
local_port=`printf "%i\n" ${used_ports[@]} | \
    awk -v minp=$min_port -v maxp=$max_port \
    '{ array[$1]=1 } END { for (i=minp; i<=maxp; i++) { if (i in array) continue; else { print i; break } } }'`
if [ "$local_port" == "" ]; then
    echo "Error: No ports free! Exiting..." >&2
    exit 2
fi
if [ $verbose ]; then
    echo "Selected port was: $local_port"
fi
 
# Create tunnel:
if [ $verbose ]; then
    echo "Creating SSH tunnel..."
fi
ssh_opts="-f -N -L"
ssh $ssh_opts $local_port:$rdesktop_server:$rdesktop_port \
    $ssh_username@$steppingstone 
 
# Allow the ssh tunnel to be established
sleep $timeout
 
# Abort if tunnel has not been established
pidof_ssh=`pgrep -f "ssh $ssh_opts $local_port"`
if [ "$pidof_ssh" == "" ]; then
    echo "Error: Timeout while establishing tunnel" >&2
    echo "Exiting..."
    exit 3
fi
 
# Make rdesktop connection
if [ $verbose ]; then
    echo "Opening Remote desktop connection to $rdesktop_server..."
fi
rdesktop $rdesktop_opts -u $rdesktop_username -p - \
    -d $rdesktop_domain localhost:$local_port
 
# Clean up tunnel
if [ $verbose ]; then
    echo "Cleaning up SSH tunnel with pid $pidof_ssh and local port $local_port"
fi
kill $pidof_ssh

© 2018 Lennart's weblog

Theme by Anders NorenUp ↑