Upgrade to PostgreSQL 9.6 on Ubuntu 14.04 or 16.04

First step before doing anything is create a backup of your database. Do NOT proceed past this point unless you have a complete backup.

Got a backup? Now you can proceed to the second step is to add the proper repository so we can install PostgreSQL 9.6:

sudo add-apt-repository "deb http://apt.postgresql.org/pub/repos/apt/ $(lsb_release -sc)-pgdg main"
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -
sudo apt-get update
sudo apt-get install postgresql-9.6 postgresql-server-dev-9.6 postgresql-contrib-9.6 -y

Next steps is to add two extensions:

sudo su - postgres -c "psql template1 -p 5433 -c 'CREATE EXTENSION IF NOT EXISTS hstore;'"
sudo su - postgres -c "psql template1 -p 5433 -c 'CREATE EXTENSION IF NOT EXISTS \"uuid-ossp\";'"

Now we can stop postgresql and run the upgrade. Be sure to have made a backup of your DB first, as any number of things can go wrong.

service postgresql stop
sudo su - postgres -c '/usr/lib/postgresql/9.6/bin/pg_upgrade -b /usr/lib/postgresql/9.3/bin -B /usr/lib/postgresql/9.6/bin \
-d /var/lib/postgresql/9.3/main/ -D /var/lib/postgresql/9.6/main/ \
-O "-c config_file=/etc/postgresql/9.6/main/postgresql.conf" -o "-c config_file=/etc/postgresql/9.3/main/postgresql.conf" --link'

We can now safely remove the old version, patch the configuration file:

sudo apt-get remove postgresql-9.3 -y
sudo sed -i "s:5433:5432:g" /etc/postgresql/9.6/main/postgresql.conf

If your version of Ubuntu has systemctl, you can setup PostgreSQL to automatically start on reboot:

sudo systemctl enable postgresql@9.6-main.service
sudo systemctl start postgresql@9.6-main.service

You should see the following message once you have completed the upgrade:

Upgrade Complete
Optimizer statistics are not transferred by pg_upgrade so,
once you start the new server, consider running:
sudo su - postgres -c  ./analyze_new_cluster.sh
Running this script will delete the old cluster's data files:
sudo su - postgres -c  ./delete_old_cluster.sh

Be sure to run analyze_new_cluster.sh to create the new statistics for your data.

This guide is modified from this gist which is a few modifications away from this original version.