Using Cloud storage safely – encrypt & sync with Ubuntu

The advent of “The Cloud” has meant that there are now services available on the internet for many of the tasks which traditionally would have been tackled by systems within your company or home.

One of the most basic is the storage of files on the internet which synchronise with a folder on your computer.  Although there are many examples, I am going to use a free 5Gb Ubuntu One (U1) account to keep my secret files backed up offsite.

When it comes to backups, there are a couple of issues that need to be addressed.

1. If you backup to an external hard-drive, what happens if someone steals it?  Do they get access to your personal information?
2. What if the house burns down? Do you lose both the original store of that information on the PC as well as the “safe” backup on the external hard drive in the same room?

I suspect that for most people the answer to both of those questions is yes.  It does not need to be and GNU/Linux allows you to fix these issues relatively easily.  I don’t know whether it can be done on Microsoft Windows or Mac OSX, you’re on your own there. 🙂

I have a PC with Ubuntu 11.10 “Oneiric Ocelot” installed.  As usual, I have a “home” folder for my own account but because there is no encryption on those files by default, anyone who gets their hands on my PC or external hard drive can read my files.

So this is what I did…

Install the package ecryptfs-utils with support for encfs to add an encrypted folder under your home folder called “Private”.

Open up a terminal window and execute the following command

sudo apt-get install ecryptfs-utils encfs

Then run the following command to create the Private folder

ecryptfs-setup-private

This will ask you for your login password/passphrase and will then create for you a mount passphrase which you should record somewhere as it is the key to decrypting your secure data in future.

(Hint, don’t just stick it in a text file in the same folder. Nature has a way to stop this sort of behaviour a.k.a. “Natural Selection”.  You could backup the folder ~/.ecryptfs as it contains the mount passphrase wrapped up by your login password.)

At this point there is a folder called “Private” in your home folder which looks much like all the other folders.  The difference however, is that every file or folder you place in there becomes encrypted and stored in a hidden folder called “.Private” (note the initial fullstop/period in that name).

In fact, the data is ONLY stored in that .Private folder and it is your mount passphrase that decrypts it and allows you to interact with the files through the Private folder. The .Private folder is what you can safely store out on the internet because nobody can decrypt it without your mount passphrase.  What we will do next is synchronise this folder to be stored out on a cloud file storage service.  I’ve chosen UbuntuOne but you could just as easily use Dropbox

Now this is where the fun starts…

Sign-up for an UbuntuOne account

or Dropbox here (using my referral)

Ubuntu One software is already included with Ubuntu, for other services you will need to install some software e.g. Dropbox client software.

Launch the Ubuntu One software from the Unity launchbar on the left of your screen in Oneiric. Once you have logged in, you will see that the “Ubuntu One” folder in your Home folder is automatically synchronised.  Now I am sure that the nice people at Canonical would not sift through your files or that they’d make a technical error that allowed your files to be seen by all and sundry on the internet.  But with this approach, we don’t have to worry.

In the Ubuntu One software, you need to add the .Private folder (the one starting with the period, remember?) which contains encrypted gobbledygook so that it gets synchronised from your PC to your 5Gb Ubuntu One (U1) account. To accomplish this, right click on the folder in Nautilus (the normal file explorer in Ubuntu) and choose the menu option for Ubuntu One to “Synchronise This Folder”.

If you like, you can login to the U1 website to check the progress of the upload.

Once that is completed synchronising, you will have your files safe and secure on the cloud in case you lose your primary source of those files.  Every time you edit one of those files or add another, they will immediately be encrypted and synchronised with that U1 account.

The mount passphrase is essential in getting access to that data so DO NOT SHARE OR LOSE IT.

If you want to retrieve that data on another Ubuntu machine, then merely setup U1 as before using the same account details. You will probably need to click the checkbox in the U1 software, “Cloud Folders” tab for the .Private folder to be downloaded to this new machine. (CAUTION: It is not necessary to perform the step earlier to create the Private folder on this second machine using ecryptfs-setup-private.  In fact, if the Private folder already exists on this second machine, don’t do this because encryption is already setup and you’re probably overwriting some other encrypted data.)

Give it some time to synchronise and run the following command in a terminal

sudo mount -t ecryptfs ~/.Private ~/Private

Enter your mount passphrase and you should be able to see your data in the Private folder.

Some additional things to note.

If you need more than the free 5Gb, you can pay for more storage.

Software clients exist for many operating systems (Windows, Android smartphones etc) but you will need a means to decrypt the data when you want to read or edit it. This guide assumes that we are only using Ubuntu 11.10.  Hint: For Android check out the boxcryptor application with a Dropbox account as it has encfs support and is well integrated with Dropbox.  I prefer to use free software and will keep an eye out to promote an Android solution in future.

I hope this helps explain how to protect yourself from data theft or loss by using readily available cloud services.  If you require further explanation, please let me know and I will try to clarify.

AMD video problems in Ubuntu 11.10 Oneiric Ocelot

I’ve had some issues with graphics in Oneiric Ocelot since the Alpha pre-releases.

There were issues with the AMD proprietary driver (“fglrx”) for both my 5770 and 5870 graphics cards and, since I would always prefer to use free software, I decided to revert to the free Radeon driver that does have some 3D support now.

The problem is that, even after removing the proprietary AMD driver, some residual issues still persist that stop the radeon driver from working properly.

The trick is to reinstall a few components after removing all the fglrx packages.  Then remove the xorg.conf file.

That was enough for me, I now have my dual monitor setup back in action with the compiz wobbly-windows effect that I like.

In a terminal, try these commands one-by-one and then reboot.

sudo apt-get remove --purge fglrx fglrx_* fglrx-amdcccle* fglrx-dev* xorg-driver-fglrx
sudo apt-get remove --purge xserver-xorg-video-ati xserver-xorg-video-radeon
sudo apt-get install xserver-xorg-video-ati
sudo apt-get install --reinstall libgl1-mesa-glx libgl1-mesa-dri xserver-xorg-core
sudo mv /etc/X11/xorg.conf /etc/X11/xorg.conf.backup

 

The free software radeon drivers wins over the proprietary AMD software.

For more help with getting AMD cards working try reading this excellent guide for Natty.

Upgrading from ext3 to ext4 in Ubuntu

If you have installed Ubuntu recently you will find that ext4 is the standard format used for creating filesystems. However, if you upgraded from an older version of Ubuntu you may be still using ext3.

The following instructions show how to upgrade the filesystem format with the data still in place. I am presuming you have a backup of your data in case this goes completely pear-shaped.

First confirm that you are using ext3 by typing the following command in a terminal window (Go to Applications menu, Accessories, Terminal)

sudo df -Th

One of the lines that showed up for me was

/dev/sdb1     ext3   241263968  93827044 137632456  41% /media/mirror

Now download, create and boot from a Ubuntu live CD so that none of your hard drives are in use.  Choose the 32-bit option of the latest version (ver 10.10 at time of writing) of Ubuntu.

Then restart the machine with this newly created CD and again confirm the name of the device you want to upgrade from ext3 to ext4.

sudo df -Th

Before making the change in format, let’s check the disk for any errors

sudo e2fsck -fDC0 /dev/sdb1

When that completed, we make the change from ext3 to ext4 with

sudo tune2fs -O extents,uninit_bg,dir_index /dev/sdb1

The only thing left to do is to edit the fstab file so that the filesystem loads with ext4

The fstab file will be on the drive that you use to boot the PC in the /etc directory.  Although not in my example here, this may be the drive you just modified.

We need to mount the drive that is used for booting (I assume it is sda1 here)

sudo mkdir /mnt/sda1
sudo mount -t ext4 /dev/sda1 /mnt/sda1

Now edit the fstab file

sudo nano /mnt/sda1/etc/fstab

Look for the line which contains your recently changed drive (sdb1 for me) and alter the format from ext3 to ext4 and hit control-x to exit.  Then hit the Y key to accept the changes and hit enter to replace the old fstab file.

Now it’s just a matter of restarting for the drive to be reloaded with ext4.

Downloading repo keys from behind a corporate firewall

Corporate firewalls commonly block port 11371 which launchpad PPA’s use for their keys.

It is possible though to get these through the normal port 80 for web traffic using the format below (replace the last reference to reflect the key you want to download)…
gpg --keyserver hkp://keyserver.ubuntu.com:80 %%recv-keys 0A5174AF

Using a USB modem for wireless 3G internet with Ubuntu 10.04 “Lucid Lynx”

Many of these devices contain their own software to work with Windows.
If you use them with a Windows machine, they act like a USB flash key containing the software and will install the necessary drivers.
Once this is done, they switch into a different mode of operation and act like a modem from then on.
This is not required in Ubuntu so we just need to make sure the modem skips that first stage. It’s a very simple fix actually but will involve getting an internet connection through another means first i.e. home internet, wifi at a cafe, a friend’s place etc

Open a Terminal window (under the Applications menu, Accessories).
Type the following to install what you need

sudo apt-get install usb-modeswitch

This will ask you for your password in order to install the software.
Once it is completed, you can just insert the USB modem again and connect using the Network Manager applet near the top right hand corner of the screen.
From here it depends on the internet provider that the USB modem relates to but it should get you a good deal closer to getting it working.

Installing Evolution 2.29.3 with mapi plugin under Ubuntu 9.10 Karmic

UPDATE: 2.29.5 is available.

Just change the version number in the wget lines below in Step 2 and follow the remaining steps as below taking care to update the new version number where appropriate.

You should not need to uninstall anything in advance.

Download, compile and install the following 4 files…

gtkhtml-3.29.5.tar.bz2

evolution-data-server-2.29.5.tar.bz2

evolution-2.29.5.tar.bz2

evolution-mapi-0.29.5.tar.bz2

ALERT: This posting relates to an “unstable release” of Evolution.  Although you may improve functionality against an Exchange 2007 server, you may also suffer from degraded performance. Install at your own risk.

These are the steps I followed to install the very latest unstable development version of Gnome Evolution.  It requires you to upgrade a few components over what is included in Ubuntu 9.10 Karmic Koala by default.

1. Run Applications menu-> Accessories-> terminal

2. Get the latest code tarballs by typing the following commands into the terminal window (note that we get two versions of the evolution code as the newest one appears to be missing a required file)

mkdir ~/evolution

cd ~/evolution

wget http://ftp.gnome.org/pub/GNOME/sources/gtkhtml/3.29/gtkhtml-3.29.3.tar.bz2

wget http://ftp.gnome.org/pub/GNOME/sources/evolution-data-server/2.29/evolution-data-server-2.29.3.tar.bz2

wget http://ftp.gnome.org/pub/GNOME/sources/evolution/2.29/evolution-2.29.3.tar.bz2

wget http://ftp.gnome.org/pub/GNOME/sources/evolution/2.29/evolution-2.29.3.2.tar.bz2

wget http://ftp.gnome.org/pub/GNOME/sources/evolution-mapi/0.29/evolution-mapi-0.29.3.tar.bz2

3. Get prereqs for building each of the packages by typing the following commands into the terminal window

sudo apt-get install libdb-dev libnspr4-dev libnss3-dev libical-dev libsqlite3-dev

sudo apt-get install bison intltool gnome-core-devel evolution-data-server-dev libcanberra-gtk-dev

sudo apt-get install libgtkhtml3.8-dev network-manager-dev libunique-dev libhal-dev

sudo apt-get install libgtkimageview-dev libpst-dev libnotify-dev

sudo apt-get install libmapi-dev samba4-dev libglib2.0-dev

4. Extract the source code from the tarballs with the following commands

tar xjvf gtkhtml-3.29.3.tar.bz2

tar xjvf evolution-data-server-2.29.3.tar.bz2

tar xjvf evolution-2.29.3.tar.bz2

tar xjvf evolution-2.29.3.2.tar.gz

tar xjvf evolution-mapi-0.29.3.tar.bz2

5. Now we should have a folder for each of the components under our ~/evolution folder, so we visit each folder in turn and build and install. Check for the screen for any errors, particularly after each install command, to see if the individual component built ok.  If you experience any errors, leave a comment here so that we can determine if a prerequisite is missing from your environment.

cd ~/evolution/gtkhtml-3.29.3

./configure

make

sudo make install

cd ~/evolution/gtkhtml-3.29.3

./configure

make

sudo make install

cd ~/evolution/evolution-data-server-2.29.3

./configure

make

sudo make install

cd ~/evolution/evolution-2.29.3

./configure

make

sudo make install

cd ~/evolution/evolution-2.29.3.2

./configure

make

sudo make install

cd ~/evolution/evolution-mapi-0.29.3

./configure

make

sudo make install

6. If everything built alright, you should now be able to launch Evolution and check in the Help menu -> About to confirm that you are running 2.29.3.2 now.  You should also have improved (but still buggy) calendar functionality if you have an Exchange 2007 email server  As stated at the top of this posting, this is an unstable release of code under very active development at the moment. Only try these steps if you can cope with Evolution not working or working intermittently.

If you are dependent on Evolution to work and it does not currently meet your requirements with the version you have already installed, then try the steps shown here.

Gnome Evolution Updates for Karmic

The latest unstable release of Gnome Evolution 2.29.3 was released yesterday with an updated 0.29.3 mapi plugin and provides a marked improvement over the current stable release in Ubuntu Karmic 9.10 (i.e. Evolution 2.28.1 and evolution-mapi 0.28.0).

The roadmap for releases of the latest Gnome components is shown here

I have tested the 2.29.3 release for Evolution against the Exchange 2007 server and can report that the Global Address List is fully functional and the email part of Evolution works well with the mapi plugin.  Unfortunately, the same cannot be said of the calendar functionality which had a caniption and froze on any attempt to access.

So the good news is, you can use Evolution cleanly to talk to an Exchange 2007 server now as long as you only want to use email.

I will document the steps required for anyone adventurous enough and prepared to deal with what is code under active development.

Creating Pareto Charts in OpenOffice – Ireland in a deep hole

If you would like to create a Pareto chart with free software that looks something like the one below, then read on. If not, and you are only interested in the data, it can be found here

Creating a Pareto chart is simple in OpenOffice

This is a short tutorial on how to produce the chart in an OpenOffice 3.1.1 spreadsheet (using Ubuntu 9.10 Karmic Koala).

Step 1. Select your data In the spreadsheet.

(I moved my columns beside each other first and only selected 40 rows to simplify this charting example – columns B, C and D of the .ods spreadsheet file)

External debt per capita data

 

Pareto_Underlying_Data_Sample

Step 2. Click on the chart icon on the toolbar (small red pie chart icon) and choose “Column and Line” chart type from the window that pops up.  Click “Finish”.

Step 3. Now we want to add another axis for the accumulative percentage so make sure you are in Chart edit mode by right-clicking the chart and choose “Edit”, then right click the graphed line in the chart and choose “Object Properties”.

Select the “Options” tab from the window that pops up and select the radio button to align the data series to the “Secondary Y axis”. Click OK, to close that window.

We want the line data to be scaled with the Secondary Y axis

Step 4. Format the Y-axis that has just appeared on the right hand side of the chart.

(You may need to right-click the chart again to go into “Edit” mode)

Right click the new Y axis and choose “Object Properties”.

Under the “Scale” tab deselect the Automatic radio buttons and set values as shown here

Scaling the new Secondary Y axis

Now place the line of the axis where you can see (from the data) that the 80% mark has been crossed

Positioning the Secondary Y Axis

Now a final formatting to display the secondary Y axis values as percentages

Format Secondary Y Axis

Then click OK for these changes to be reflected in the chart.

Step 5. Format the X-axis to rotate the labels by 90 degrees.

Right-click on the X-axis and choose “Object Properties”, then the “Label” tab and set the rotation to 90 degrees.

Rotate the label of the X Axis for clarity

Step 6. Move the Legend (to make room for the graph data to the right, you may not need this)

While in edit mode (double-click the chart if required), then go to the menu “Insert” and “Legend”.

Place the legend at the bottom.

Moving the Legend to bottom of chart

Step 7. Finish off with labelling the title and axes

Double click the chart again to enter edit mode and from the menus, choose Insert, Title.

Complete the values for Title and the axes in the window that pops up and then close it to see the results.

Name the chart and its axes as appropriate

Voila! A neat looking Pareto chart with completely free (as in speech) software

Given that this data is prior to the Global Financial Crisis, it doesn’t look like the good times are coming back to Ireland any time soon.

Note the complete absence of Iceland from the data also.

Using Evolution with Exchange 2007

Having just upgraded to Karmic Koala Ubuntu 9.10, I tried connecting to an MS Exchange 2007 Server in the office with the default email reader, Gnome Evolution (version 2.28.1), using its MAPI plugin (version 0.28.0).

Not sure if this worked with previous versions but it certainly doesn’t now.

There are two issues I noticed, one relates to a crash when trying to view certain emails (could not detect the pattern) and another crash after waiting for Evolution to retrieve the message-ids from my Inbox.

The good news is, there is a fix available.

You will need to download and manually install an updated MAPI plugin (version 0.28.1).

Here are the steps

1. Run Applications menu-> Accessories-> terminal

2. Uninstall the existing plugin if you have it installed by typing into the terminal window…

sudo apt-get remove evolution-mapi

3. Get some pre-requisite packages

sudo apt-get install evolution-data-server-dev libedataserverui1.2-dev libebackend1.2-dev libecal1.2-dev libedata-cal1.2-dev libedata-book1.2-dev libcamel1.2-dev

sudo apt-get install evolution-dev libmapi-dev samba4-dev intltool

4. Get the latest version of that MAPI plugin and build it

(UPDATED: 21st Dec 2009 There is now an updated version 0.28.2, so just substitute the version number in the instructions below)

cd ~

wget http://download.gnome.org/sources/evolution-mapi/0.28/evolution-mapi-0.28.1.tar.bz2

tar xvjf evolution-mapi-0.28.1.tar.bz2

cd evolution-mapi-0.28.1/

./configure

make

sudo make install

5. Bathe in the glory of Exchange 2007 connectivity 😀

NOTE: This circumvents the Ubuntu repositories for this particular plugin.  Once the updated version is available in the repositories, it would be prudent to revert to using the Ubuntu-provided version so you continue to get updates in future.

TODO: Our setup involves an Exchange Hub Transport which is what MS Outlook email client talks to, with separate server for Global Address List (GAL) and another server for the mailbox which I am using.  I assume this arrangement is what makes the GAL inaccessible as I have configured the settings for my mailbox server.  There does not appear to be anywhere I can set which server the GAL is on.