Scripting an automated backup for an AVM FritzBox router

My home server is set up to backup pretty much every device in my network automatically every day. So I figured why not also backup my AVM FritzBox router. In case it breaks or an update fails and it has to be reset, I can easily restore a backup with all settings, phone book entries, logs etc.

Fortunately, there is a very basic interface for automation, so it is possible to shell script the process of exporting the entire configuration. The result is identical to the manual export in the administration interface to export everything to a backup file. The necessary steps are as follows:

  1. Request a challenge
  2. Log in by responding to the challenge
  3. Extract session ID
  4. Export the backup file
  5. Log out

For sake of simplicity, I will assume that the router is available at https://fritz.box (which it is by default) and that the admin password is supersecret.

A challenge can be requested with the URL https://fritz.box/login_sid.lua:

curl -s -k https://fritz.box/login_sid.lua

The response is a SessionInfo XML, which looks like this:

<?xml version="1.0" encoding="utf-8"?>
<SessionInfo>
  <SID>0000000000000000</SID>
  <Challenge>1ma6cy9c</Challenge>
  <BlockTime>0</BlockTime>
  <Rights/>
</SessionInfo>

The challenge is a random string that changes every time the URL is called. Without providing the challenge response, the session ID (SID) will remain empty. To extract the challenge and store it in a variable, simply add a sed statement:

CHALLENGE=`curl -s -k https://fritz.box/login_sid.lua | \
sed 's/.*<Challenge>\([a-z0-9]*\)<.*/\1/'`

So now we have to construct the response and call the URL again. The response is <challenge>-md5(<challenge>-<password>), but it is important to make sure the encoding is UTF-16LE before calculating the hash. Otherwise it will differ and won’t be accepted. This can be done by including iconv:

PASSWORD=supersecret
RESPONSE=$CHALLENGE-`echo -n "$CHALLENGE-$PASSWORD" | iconv -t UTF-16LE | md5`

Call the login URL again, this time with the response:

DATA='response='$RESPONSE'&username=&lp='
curl -s -k --data $DATA https://fritz.box/login_sid.lua

In case you want to use a different user than admin, just provide the username here. The response is again the SessionInfo XML, but SID should be set. It also lists the rights the now logged in user has:

<?xml version="1.0" encoding="utf-8"?>
<SessionInfo>
  <SID>962dd5a4b3d75fb2</SID>
  <Challenge>e56b0c6e</Challenge>
  <BlockTime>0</BlockTime>
  <Rights>
    <Name>Dial</Name>
    <Access>2</Access>
    <Name>App</Name>
    <Access>2</Access>
    <Name>HomeAuto</Name>
    <Access>2</Access>
    <Name>BoxAdmin</Name>
    <Access>2</Access>
    <Name>Phone</Name>
    <Access>2</Access>
    <Name>NAS</Name>
    <Access>2</Access>
  </Rights>
</SessionInfo>

In case the SID remains empty, the login failed. If the password is correct, but it still doesn’t work, it may be an encoding issue with the MD5 hash. Keep in mind that the encoding may also depend on the shell used. Use the following example to check your script:

$ echo -n "test" | iconv -t UTF-16LE | md5
c8059e2ec7419f590e79d7f1b774bfe6

To extract the SID and store it in a variable, add the following sed statement:

SID=`curl -s -k --data $DATA https://fritz.box/login_sid.lua | \
sed 's/.*<SID>\([a-z0-9]*\)<.*/\1/'`

This SID can now be used with the call to export all settings for backup. To do this, a form has to be sent to https://fritz.box/cgi-bin/firmwarecfg, including the SID, the desired operation, as well as the password required to restore the backup in this case. I’ll just use the password backup. This is necessary, because the export contains sensitive information, like ISP and SIP accounts, which will be encrypted.

OUT=/tmp/fritzbox.export
BAKPWD=backup
curl -s -k -o $OUT --form sid=$SID --form ImportExportPassword=$BAKPWD \
--form ConfigExport= https://fritz.box/cgi-bin/firmwarecfg

Now the only thing left to do is log out, so the session doesn’t stay open unnecessarily:

DATA='sid='$SID'&logout=1'
curl -s -k --data $DATA https://fritz.box/login_sid.lua

That’s it! The exported file includes all information stored in the FritzBox, so in case anything goes wrong, it allows a complete restore.

Using a Raspberry Pi as a serial console server for FreeBSD

My home server runs FreeBSD 11.0, and, of course, has full disk encryption. This requires a passphrase to be entered while booting, so I need a keyboard and monitor attached, and have to be in the same room for restarts and after power outages. This is not acceptable at all. There is no need for a display and keyboard otherwise, and I want to be able to enter the passphrase remotely, but without compromising security. Luckily, the server has a serial port and I have a spare Raspberry Pi. So I can connect the serial port to the Raspberry Pi, SSH into it and use the serial console of the server just like physical access. Hooray for stuff from the 1960s still being useful!

So first things first, I need to connect the Raspberry Pi to the serial console. Since it only has USB (aside from GPIO), a USB to serial port adapter is necessary. The Raspberry Pi runs FreeBSD 11.0, which has good out-of-the-box support for the Prolific PL2303 chipset, so I just ordered a cheap adapter based on it. It probably works with Linux as well, if you’re using Raspbian. You can use dmesg to check if it was detected correctly:

# dmesg | grep Prolific
ugen0.4: <Prolific Technology Inc.> at usbus0
uplcom0: <Prolific Technology Inc. USB-Serial Controller, class 0/0,
rev 2.00/3.00, addr 4> on usbus0

Don’t forget (like I did) that the USB to serial port adapter only provides you with a serial port. To connect it to the server’s serial port, you still need a null modem cable or adapter.

Now that we have the hardware, it’s time to set up the server to use the serial console. This is pretty straight-forward, just add the following to your /boot/loader.conf and reboot, as described in the FreeBSD handbook:

boot_multicons="YES"
boot_serial="YES"
console="comconsole,efi"

If you don’t use EFI yet, replace efi with vidconsole. This outputs all kernel messages also to the serial console and lets you type in the passphrase during the boot process. You can also only use comconsole, but then there won’t be much output on a real monitor in case you need one.

Additionally, you may like to see a login prompt after the boot process is done, in case the server’s network is down and SSH doesn’t work. A terminal on the serial port can be activated by adding the following line in /etc/ttys:

ttyu0   "/usr/libexec/getty std.9600" dialup  on secure

Now you should already be able to use the serial console via the Raspberry Pi, with a tool of your choice:

# tip ucom1
# screen /dev/cuaU0 9600
# cu -l /dev/cuaU0 -s 9600

If you don’t want to be root to access the console (you don’t), you can add your user to the group dialer:

# pw groupmod dialer -M solence

Now that all is up and running, be aware that the serial console works like a monitor. If you log in via serial console and only close serial connection, it’s just like turning off your monitor, you’re still logged in! So be aware to always log out before closing the serial connection.

 

Increasing the port speed

This setup is usable, but painfully slow, because the default port speed is still 9,600 baud. In case you don’t remember the ancient unit baud, it’s symbols per second. This equals bits/sec, so 9,600 baud convert to 1,200 bytes/sec. Considering that the standard terminal line has 80 characters, that’s only 15 lines/sec.

So increasing the serial port speed to the maximum of 115,200 baud would be a good idea. This equals 14,000 bytes/sec or 180 terminal lines/sec, which sounds much better. Some tweaking on the server’s side is required, however.

First, add the desired speed to /boot/loader.conf:

comconsole_speed="115200"

Then change the speed in the corresponding entry in /etc/ttys:

ttyu0   "/usr/libexec/getty std.115200" dialup  on secure

In theory, this is sufficient and only requires a reboot to work, but in my case I also had to configure the serial port device. Check with stty if the port has the correct speed on the server:

# stty -f /dev/ttyu0

If the speed is set to 115,200 baud, everything is fine. If it’s still 9,600 baud, it has to be adjusted like described in this great tutorial.

After that’s done, the serial console can be used with the increased speed on the Raspberry Pi:

$ screen /dev/cuaU0 115200
$ cu -l /dev/cuaU0 -s 115200

You can also add an alias in /etc/remote for easy use with tip on the Raspberry Pi:

server:dv=/dev/cuaU0:br#115200:pa=none:

Now you can just use:

$ tip server

If your Raspberry Pi is accessible remotely, directly or via VPN, power outages or remote maintenance are no problem anymore. But make sure you can still reach the Raspberry Pi when the server you connected it to is down, otherwise all this would be pointless. Also, now you don’t even need keyboard and monitor at all, because it’s way more convenient to use SSH and the serial console than getting up and moving over to the box 🙂

Installing and updating FreeBSD 11.0 release on a Raspberry Pi

FreeBSD runs great on a Raspberry Pi, but the official images are all stable or current branches and there is no support for binary updates, i.e. freebsd-update. So I tried to figure out how to install the latest release version and keep it updated as well, without starting from scratch with a new image every time.

While it is possible to build a kernel and world on the Raspberry Pi itself, it takes about a week to finish (trust me, I tried) and you need a much larger /tmp on a disk or share for it to work in the first place. Since FreeBSD can also be cross-compiled, I decided to use my Xeon box to do the heavy lifting and then just install it on the Raspberry Pi.

So let’s get started with getting the source and building it with armv6 as a target architecture on an amd64 system. I assume that /usr/src is empty, so first get the source. We want the Release Engineering (RelEng) branch, which is the latest release version, including security updates:

# cd /usr/src
# svn checkout https://svn.freebsd.org/base/releng/11.0

After getting the code, we can keep it updated with svn update. I recommend subscribing to the FreeBSD-Announce mailing list, to get a notification every time there is an update.

Now we can build the world and kernel, but with armv6 for the Raspberry Pi as the target architecture:

# make TARGET_ARCH=armv6 UBLDR_LOADADDR=0x2000000 buildworld
# make TARGET_ARCH=armv6 KERNCONF=RPI-B buildkernel

Please note that the UBLDR_LOADADDR variable has to be set, otherwise the kernel won’t boot. Also, the kernel configuration is for the first Raspberry Pi, for the newer Raspberry Pi 2 use KERNCONF=RPI2. This takes a while, but it can be sped up by using the -j<n> parameter with make, where n equals the number of CPU cores + 1, e.g. -j5 for a quad core. My Xeon with an SSD needs about 30 minutes to build the world and kernel.

There are basically several ways to install everything on the Raspberry Pi, but I could only figure out to get one working. The most convenient would be to mount /usr/src and /usr/obj via NFS on the Raspberry Pi and install everything locally. That didn’t work for me. The compiled files weren’t executable, probably because it’s a cross-compiled and not a native build. Another method is to mount / of the Raspberry Pi on the build box via NFS (yay security) and install from the build box. That did work for the kernel, but not for world. The problem is that most files are in use and therefore can’t be overwritten. That’s why the FreeBSD handbook advises to use single user mode, but then there is no networking. Also my USB keyboard wasn’t recognized early enough in the boot process to enter single user mode. So in the end, I just pulled the SD card and put it in a card reader plugged into the build box. This also turned out to be much faster than via network. Mounting the SD card locally is as simple as it can be:

# mount /dev/da0s2a /mnt

Then install kernel and install world can be done like in the handbook, but with a few added variables and parameters:

# make TARGET_ARCH=armv6 KERNCONF=RPI-B DESTDIR=/mnt installkernel
# mergemaster -p -A armv6 -D /mnt
# make TARGET_ARCH=armv6 DESTDIR=/mnt installworld
# mergemaster -iF -A armv6 -D /mnt
# make TARGET_ARCH=armv6 DESTDIR=/mnt delete-old
# make TARGET_ARCH=armv6 DESTDIR=/mnt delete-old-libs

That’s it! Now unmount the SD card, plug it into the Raspberry Pi again, and be greeted with the current release version with the lastest patch:

# uname -a
FreeBSD raspberrypi 11.0-RELEASE-p8 FreeBSD 11.0-RELEASE-p8 #3 r315180: Sun
Mar 12 23:15:58 CET 2017     solence@raspberrypi:/usr/obj/arm.armv6/usr/
src/sys/RPI-B  arm

# freebsd-version
11.0-RELEASE-p8