ADUFRAY

Do you run an nginx-based web server? Do you use the http_auth_basic_module module? This post might be for you!

I just set up a new FreeBSD server (blog post forthcoming) and decided to skip the install of httpd, preferring nginx. All the cool kids are using it, so I figured I should as well. One problem: no htpasswd command.

There are several webapps I’ve written over the years for my own personal uses, and they all use HTTP Basic authentication to regulate access. nginx happily supports this, if you have an existing htpasswd-formatted file to use. Unfortunately, you cannot create one since it does not supply a tool to do so.

On FreeBSD I found a simple enough workaround:

$ echo YOUR_PASSWORD | \
  pw useradd YOUR_USERNAME -h 0 -N | \
  awk -F: '{print $1"$"$2}' | \
  awk -F'$' '{print $1":$6$rounds=5000$"$4"$"$5;}'

Thankfully FreeBSD’s (and perhaps your distro’s) crypt(3) supports strong password hashes, the default being SHA512 with 5,000 rounds. When you supply the -N flag to the pw command, it will not actually perform the action, but instead print out the result (incluiing the hashed password, which is what we’re after).

SHA512 with 5,000 rounds (and a random, 32-character salt) is probably more security than anyone actually needs. If we want more than that, though, we have options — other than installing httpd and gaining htpasswd. I wrote a super-simple PHP script to generate a bcrypt hash with selectable cost (2^x rounds):

bcrypt_hash.php [user]

#!/usr/local/bin/php
<?php
  // 10 is reasonable, but time the output of this script to test what is reasonable for you
  $cost = 10; 

  // write prompt to stderr so we can redirect output with > and only include what we want
  $stderr = fopen('php://stderr', 'w+');
  fwrite($stderr, 'Enter password: ');

  // disable echoing the password to terminal
  system('stty -echo');
  $passwd = stream_get_line(STDIN, 1024, PHP_EOL);
  system('stty echo');

  fwrite($stderr, "\n");

  // at least on FreeBSD you need to replace PHP's bcrypt hash version 2y with OpenBSD's 2b
  // Fixed in secure/lib/libcrypt/crypt-blowfish.c revision 284483 
  // https://svnweb.freebsd.org/base?view=revision&revision=284483
  $hashed = str_replace('$2y$', '$2b$', password_hash($passwd, PASSWORD_BCRYPT, array('cost' => $cost)));
  echo  ( (isset($argv[1])) ? $argv[1] : "user" ) . ":$hashed\n";
?>

FreeBSD’s pw will generate a bcrypt-based hash if you edit /etc/login.conf and add

blf_users:\
    :passwd_format=blf:\
    :tc=default:

Then rebuild the mapping with cap_mkdb /etc/login.conf and add -L blf_users before the -h flag in the aforementioned command. Sadly pw will use the default cost value of 4, which seems pointless to me. Looking over /usr/src/usr.sbin/pw/pw_user.c the pw_pwcrypt() function does not support changing the random salt that is generated to a format specifying the cost value — better to use the above PHP script.

  1. Change size of virtual disk in VMware (increase only)
  2. Rescan the SCSI bus / devices

    # echo "- - -" > /sys/class/scsi_host/host0/scan
    # echo 1 > /sys/class/scsi_device/0\:0\:0\:0/device/rescan
    # echo 1 > /sys/class/scsi_device/2\:0\:0\:0/device/rescan
    
  3. Destroy & recreate the partition with the new boundary (hopefully you used the last partition on the disk!)

    # fdisk /dev/sda
       p            (print) 
       d            (delete partition)
       3            (third partition)
       n            (new partition)
       p            (a primary partition)
       3            (the third one)
       [return]     (start block default)
       [return]     (end block default)
       w            (write changes to disk)
    
  4. Pray

    # reboot
    
  5. Let BTRFS know

    # btrfs filesystem resize max /
    

Tired of the proprietary, outdated Ventrilo platform for in-game voice services? Me too. Thankfully, now there’s Mumble, an open source, encrypted, cross-platform, low-latency voice chat system. I’ve only been messing around with it for a short time, but so far it’s a huge improvement over Ventrilo and TeamSpeak.

Initially I became frustrated, because the server software, aka “Murmur”, is significantly less friendly. The installation instructions for CentOS were garbage. There’s no EPEL package, much less a base-system package, you have to install someone’s Dropbox-hosted RPMs, a middleware repo, and one of the dependencies is Qt. Obviously the Qt dependency is for the client aspect, but still: gross. No instructions for a non-X11 version.

Thankfully, as a I mentioned, it’s open source. I found a nifty minimalistic server implementation called uMurmur. It’s designed to run on small embedded systems, so it’s very light-weight. It seems to have been built around OpenWRT, the open source router OS. Assuming you already have gcc, autoconf, and make installed, I think the only other requirements are libconfig, protobuf-c, and OpenSSL (or PolarSSL). The whole setup was pretty easy:

# yum install -y libconfig{,-devel} protobuf-c{,-devel}
# curl -O 'https://umurmur.googlecode.com/files/umurmur-0.2.14.tar.gz'
# tar -zxvf umurmur-0.2.14.tar.gz
# cd umurmur-0.2.14
# ./configure --with-ssl=openssl
# make && make install

The Makefile installs a single binary to /usr/local/bin, but no init script or default configuration. Here’s a quick init script I created (it’s sloppy, but it works):

/etc/init.d/umurmurd:

# umurmurd  Minimal Mumble server
#
# chkconfig: 345 20 80
# description: umurmurd is a minimal Mumble server
# processname: umurmurd

# Source function library.
. /etc/init.d/functions

RETVAL=0
prog="umurmurd"
PIDFILE="/var/run/umurmurd.pid"
LOCKFILE="/var/lock/subsys/umurmurd"

start() {
        echo -n "Starting $prog: "
        /usr/local/bin/umurmurd -r -p $PIDFILE
        RETVAL=$?
        [ $RETVAL -eq 0 ] && touch $LOCKFILE && success || failure
        echo
        return $RETVAL
}

stop() {
        echo -n "Shutting down $prog: "
        kill $(cat "$PIDFILE") 2>/dev/null >/dev/null && success || failure
        RETVAL=$?
        [ $RETVAL -eq 0 ] && rm -f $LOCKFILE
        echo
        return $RETVAL
}

case "$1" in
    start)
        start
        ;;
    stop)
        stop
        ;;
    status)
    if [ -f "$PIDFILE" ]; then
          echo -n "PIDFILE exists:";
          ps -p ${PIDFILE}
        else
          echo "not running";
        fi
        ;;
    restart)
        stop
        start
        ;;
    *)
        echo "Usage: $prog {start|stop|status|restart"
        exit 1
        ;;
esac
exit $?

And a very basic config file:

/etc/umurmur.conf:

max_bandwidth = 128000;
welcometext = "Welcome to Murmur!";
certificate = "/etc/pki/tls/certs/umurmur.crt";
private_key = "/etc/pki/tls/private/umurmur.key";
#password = "password_for_users";
admin_password = "admin_password";   # Set to enable admin functionality.
ban_length = 3600;            # Length in seconds for a ban. Default is 0. 0 = forever.
enable_ban = true;        # Default is false
max_users = 20;

# bindport = 64738;
# bindaddr = "192.168.1.1";

channels = ( {
    name = "Root";
    parent = "";
    description = "Root channel. No entry.";
    noenter = true;
  },
  {
    name = "Lobby";
    parent = "Root";
    description = "Lobby channel";
  },
  {
    name = "Silent";
    parent = "Root";
    description = "Silent channel";
    silent = true; # Optional. Default is false
  }
);

default_channel = "Lobby";

Before you exit your terminal, make sure to perform these last few steps:

# openssl genrsa -out /etc/pki/tls/private/umurmur.key 4096
# openssl req -new -x509 -nodes -sha1 -days 3650 -key /etc/pki/tls/private/umumur.key -out /etc/pki/tls/certs/umurmur.crt
# chkconfig --add umurmurd
# chkconfig umurmurd on
# service umurmurd start

And, if you’re running iptables, don’t forget to add the appropriate rules.

# iptables -I INPUT 1 -m state --state NEW -m tcp -p tcp --dport 64738 -j ACCEPT
# iptables -I INPUT 1 -m state --state NEW -m udp -p udp --dport 64738 -j ACCEPT
# service iptables save

I recently wrote about how best to configure one’s SSL using Nginx. Unfortunately I recommended RC4 over many other ciphers because at the time it wasn’t completely broken. That time has come to pass.

Here is my updated configuration, unfortunately dropping SSLv3 entirely (and thus blocking the default configuration for Windows XP on IE6):

ssl_certificate      /path/to/combined.cert.and.ca.crt;
ssl_certificate_key  /path/to/cert.key; (Make sure it's 4096-bits!)
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers "!ADH:!MD5:!aNULL:ECDH+AES:DH+AES:@STRENGTH:RSA+AES:3DES";

ssl_dhparam /path/to/strong/dhparam-4096.pem;
add_header Strict-Transport-Security "max-age=31536000";

ssl_prefer_server_ciphers on;
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 10m;

ssl_stapling on;
ssl_stapling_verify on;
ssl_trusted_certificate /path/to/ca.crt;
resolver 8.8.8.8 valid=300s;
resolver_timeout 5s;

Make sure you create your strong DH parameters file (this took 45 minutes on a 2014 Mac Pro):

# openssl dhparam -out /path/to/strong/dhparam-4096.pem 4096

If you don’t care about IE on XP at all, go ahead and drop 3DES at the end of the ssl_ciphers list. That will give you a very strong rating on the Qualys SSL Labs Server Test tool. Also, if forward secrecy is important to you on Java 6u45, go ahead and comment out the ssl_dhparam part.

About the only thing you can do to get a better score at this point is to really cut back on your supported clients.

Here’s what that looks like:

ssl_certificate      /path/to/combined.cert.and.ca.crt;
ssl_certificate_key  /path/to/cert.key; (Make sure it's 4096-bits!)
ssl_protocols TLSv1.2;
ssl_ciphers "!ADH:!MD5:!aNULL:ECDH+AES256:DH+AES256:@STRENGTH:RSA+AES256";

ssl_dhparam /path/to/strong/dhparam-4096.pem;
add_header Strict-Transport-Security "max-age=31536000";

ssl_prefer_server_ciphers on;
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 10m;

ssl_stapling on;
ssl_stapling_verify on;
ssl_trusted_certificate /path/to/ca.crt;
resolver 8.8.8.8 valid=300s;
resolver_timeout 5s;

As of December 24, 2014, disabling TLSv1.0 and TLSv1.1 breaks these clients (according to Qualys), so I don’t recommend it:

Note that this is almost entirely due to dropping TLSv1.0 and not due to the lack of AES-128. By turning back on TLSv1.0 and TLSv1.1 (lowering your Protocol Support score from 100 to 95) you only lose these clients:

So, to summarize, here are my current recommended configs, one to maximize client compatibility and one to maximize (practical) security.

Compatibility:

ssl_certificate      /path/to/combined.cert.and.ca.crt;
ssl_certificate_key  /path/to/cert.key; (Make sure it's 4096-bits!)
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers "!ADH:!MD5:!aNULL:ECDH+AES:DH+AES:@STRENGTH:RSA+AES:3DES";

ssl_dhparam /path/to/strong/dhparam-4096.pem;
add_header Strict-Transport-Security "max-age=31536000";

ssl_prefer_server_ciphers on;
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 10m;

ssl_stapling on;
ssl_stapling_verify on;
ssl_trusted_certificate /path/to/ca.crt;
resolver 8.8.8.8 valid=300s;
resolver_timeout 5s;

Security:

ssl_certificate      /path/to/combined.cert.and.ca.crt;
ssl_certificate_key  /path/to/cert.key; (Make sure it's 4096-bits!)
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers "!ADH:!MD5:!aNULL:ECDH+AES256:DH+AES256:@STRENGTH:RSA+AES256";

ssl_dhparam /path/to/strong/dhparam-4096.pem;
add_header Strict-Transport-Security "max-age=31536000";

ssl_prefer_server_ciphers on;
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 10m;

ssl_stapling on;
ssl_stapling_verify on;
ssl_trusted_certificate /path/to/ca.crt;
resolver 8.8.8.8 valid=300s;
resolver_timeout 5s;

I expect I won’t have to update this post soon — or at least not until Nginx enables support for hybrid / dual SSL certificates for the same host. (NOT SNI, but rather having an ECC and RSA certificate for the same site. Apache already supports this and Nginx developers discussed it last year.) That or when OpenSSL supports ChaCha20-Poly1305 (this is possible today by using Google’s fork of OpenSSL, BoringSSL, but it requires manually compiling all the software).

Synology Woes

17 Dec 2014

Let’s say you have a Synology NAS for storing all your important data (i.e. Linux ISOs). Let’s say you’re a fairly paranoid person and use Synology Hybrid RAID (SHR) which can tolerate the loss of two drives simultaneously. Just in case. Let’s say you experience some kind of catostrophic failure that causes the system to think it has lost 5 of 12 drives at once.

This is what has happened to me.

To make matters worse, I had installed the bash ipkg to make using the command line more palatable. Did I follow best practices and exec bash from my ash .profile script? No. Do I regret this? Very much.

Since all of my volumes have failed, including /opt where bash is stored, I can no longer SSH into the Synology! And, more importantly: neither can support. If you haven’t created a separate user to SSH in from, this is really bad. The default configuration does not let you exec commands via ssh user@host <command> so you’re pretty much dead in the water.

If you still have an available volume, you can install a third party package that emulates a Terminal in the browser to repair /etc/passwd, but all my volumes have failed.

Just about thinking I was out of luck entirely, I stumbled on the Scheduled Tasks page in the Control Panel. A-ha! A little command-fu and I was able to send the output of commands to my webserver to fix the problems.

On server:
    $ tcpdump -s 0 -A -nni eth0 host <home IP> and port 80

In Scheduled Task:
    <command> | curl -d @- http://<server>/

Now I can execute commands and see the results in my tcpdump output. A little bit of sed and I was all patched up:

sed -i -e 's@/opt/bin/bash@/bin/ash@g' /etc/passwd 2>&1 | curl -d @- http://<server>/

Since the command executed successfully, there was no output and I was free to SSH back into my NAS. And, again, more importantly: so could support! Now, hopefully, they can repair my volumes.