Author Archives: Matt

About Matt

Dev Ops Engineer at Lonely Planet; Cloud computing advocate, coder, blogger, idiot....

Getting started with Amazon Python AWS CLI tool

  June 12, 2013 - 12:05 pm |
Leave a comment

The new Amazon AWS CLI tool coded in Python is far superior to the old Java implementation. The official docs can be found at https://github.com/aws/aws-cli but here’s a quick run down of how I install and configure:

Installation

or if you’re not using a virtual environment

You can also use easy_install in the same way if you prefer

Configure Environment Variables

Add the following to your .bash_profile, .bashrc, or .profile:

The key ID and secret key are required, the default region means you do not need to specify the region for every command, and the default output, changes the output type; it is set to JSON by default, which is fine for scripting, but not the best for reading in a terminal.

You can also just specify an environment variable to point to a config file and add the credentials and options there (see the github wiki)

Enable auto-complete

Add the following line to your .bash_profile, .bashrc or .profile

Finishing and test

To apply the changes to your environment and test, run the following:

The first command will apply the environment changes, the second should display all instances in the default region, using your default output type ( I prefer text). The final line should prompt to display all options, and then list all available AWS sub-commands.

iTerm2 Keybindings for OS X

  June 7, 2013 - 7:40 pm |
Leave a comment

Word Jumping

If you use a mac and tried the iTerm2 application, you may have noticed that the keybindings for word jumping and deletion are not set up correctly. Here’s how to fix that:

From the iTerm2 menu select Preferences > Profiles and from the right select Keys. Now add the following key bindings by clicking (+) for each:

Alt + Left
Send Escape Sequence, Esc + b

Alt + Right
Send Escape Sequence, Esc + f

Alt + Backspace
Send Hex Code, 0×17

Split Terminals: Creating and Navigating

I also like to bind keys to split the terminal vertically and horizontally and move focus between them.

From the iTerm2 menu select Preferences > Profiles and from the right select Keys. Now add the following key bindings by clicking (+) for each:

Alt + Up
Split Horizontally with <profile>

Alt + Down
Split Vertically with <profile>

Cmd + Up
Select Split Pane Above

Cmd + Down
Select Split Pane Below

Cmd + Left
Select Split Pane on Left

Cmd + Right
Select Split Pane on Right

A simple cloud-init trick

  June 7, 2013 - 2:12 pm |
Leave a comment

If you use Amazon Web Services or other cloud providers that make use of cloud-init, you may occasionally have reason to re-trigger the initial cloud-init run; usually because something didn’t work quite right the first time round. Rather than having to kill the cloud instance and booting a fresh one, you can try this simple trick to trigger cloud-init on reboot:

 

The Cliff

  February 18, 2013 - 3:25 pm |
1 Comment

I found myself standing on the edge of a cliff,
The turbulent sea crashing at its rocky base,
I had been on an adventure through beautiful countryside,
The simplest things bring the greatest pleasures.

As I travelled I came across a beautiful butterfly,
She danced in front of me and settled for a moment,
Waving her beautiful wings, which shimmered in the sun,
Before she fluttered off again away from me.

I followed her as best I could, dancing among the trees,
But she fluttered ever onwards towards the cliff,
There I could follow her no longer; she was free but I was not,
I could only watch as she flew away from the cliff into the distance.

I looked out from my perch on the cliff,
Far away I could see even more beautiful scenes,
I could only imagine what wonders lay within them,
Places I longed to go, but the treacherous sea was in my path.

I looked back the way I had come and saw smoke rising,
Last night’s campfire still smouldering offering promises of warmth,
Should I brave the journey and try to cross the sea?
It would surely be much easier to just turn back…

 

In life you can never turn back the way you came,
So when you find yourself on a cliff you have but one option,
No matter what lies between you and your goals,
You must be brave; you must be fearless; you must JUMP!

 

You must JUMP

You must be brave; you must be fearless; you must JUMP

Sleep Paralysis: Of Hags and Horses

  February 16, 2013 - 3:19 pm |
Leave a comment

Last week I experienced sleep paralysis for the first time; I am not sure what brought this about, but I had been sleeping poorly all week which probably didn’t help.

Sleep paralysis is a phenomenon which causes the sufferer, usually when transitioning between sleep and wakefulness, to experience hallucinations and paralysis with a sensation of weight pushing down upon them. Many sufferers report seeing an intruder in the room to whom they cannot react due to paralysis.

I had heard of this before, several of my friends had described their experiences to me. I had no idea how real and terrifying it was. This is what happened to me:

I had been unable to sleep again that night; I remember looking at the clock, 3:20am. The next thing I knew I heard my front door opening and someone coming up the stairs. At first I thought “oh good my house mate is home, about time”, it took a few seconds for my thought to catch up with me as I heard the footsteps come closer and open my bedroom door when I sudden thought “wait, I live alone! I don’t have a housemate, so who is this who has just come into my room?!?!”.

I was lying face down on the bed, I could see a shadowy figure out of the corner of my eye, I tried to turn to see but before I could, the figure lunged on top of me and pressed me down hard onto the bed. For that instant I was in complete panic “HELP! Someone has broken into my home and they’re attacking me!” I thought. I struggled to move but I couldn’t; I was completely pinned, with the intruder’s weight pressing down on me and I could not breathe.

The sensation only lasted a few moments, after which I found I was able to move; at first just my arms, which I flailed about and then managed to flip myself violently on to my back, nearly leaping clear out of my bed! I looked around me; nothing there, the door still closed, everything dark and silent except for the sound of blood surging through my body; my heart was beating so hard I could see it thumping in my chest. I looked at the clock again 3:25am

I knew right away that this was sleep paralysis; based on accounts I have heard from friends I got off lightly; some have reported seeing demons, high pitched screaming, having to concentrate intensely in order to move a single finger before they could get out of that state. I certainly hope this isn’t going to become a regular occurrence for me!

The Nightmare

The Nightmare - John Henry Fuseli's iconic depiction of sleep paralysis

So long 2012 and thanks for all the memories

  January 1, 2013 - 5:16 am |
1 Comment

Well another year is overe, so here is yet another retrospective of 2012. I won’t dwell on the negative and just share some of my personal highlights. I had ended 2011 in a swing dance club surrounded by friends and more prosecco than I could drink. I began 2012 with a dreadful hangover still half dressed in my tuxedo, a sure sign of a good night the night before. Well, that’s not strictly true, I didn’t get home until well gone 5am but you get what I mean. Anyway, I digress, so these were my highlights of 2012.

Aerials Workshops

For the uninitiated, “aerials” is a part of swing dancing usually reserved for the elite or rehearsed performances. They involve high-adrenaline moves where the lead catapults his dance partner into the air in a variety of ways. My girlfriend and I decided to attend a workshop early in the year and enjoyed it so much that we attended another bringing some friends along. We learned to do the pankcake, the whitey flip, the lamp-post (aka the helicopter)  and the screamer. These moves and many more can be seen in this amazing clip

The Sunshine Boys

It had been many years since I’d been to any kind of theatre, but this year my girlfriend and I went to see “The Sunshine Boys”; a comedy about two ageing Vaudeville comedians who had vowed never to work together again; at the glorious Savoy Theatre (yes that’s the same place as the hotel). The play was excellent with top-notch performances from the stars Danny DeVito and Richard Griffiths.

The London Lindy Exchange

The London Lindy Exchange was a weekend packed with dancing events where Londoners host lindy hoppers from around the world. The result was three amazing evenings with live big-band performances from the The Shirt Tail Stompers, Gentlemen & Gangsters, Echoes of Ellington and Skin Deep. The bands were all amazing, I danced with over 30 new people from at least 10 different countries. I even picked up a few new moves, variations and styling techniques.

Big Lebowski Party

I’d never tried a white russian before; they’re good, even better when drunk whilst sat in a dressing gown watching The Big Lebowski. In case you were wondering, that is what a Lebowski party involves, it was great.

Row Row Row Your Boat

My girlfriend and I went on a day trip to Broxborne (I like to call it Frogspawn) which is in the Lea Valley region, north of London. It was a nice sunny day, so we had a nice pub lunch outside and hired a row boat for the afternoon. We stopped off at Waltham Abbey on the way back into London. This was the same day that the EDL (English Defence League) tried to march on Walthamstow for a hate speech rally. We were glad to be far away from their vile hatred, although the community spirit of opposition shown by Walthamstow residents made me wish we’d been around. This video demonstrates their victory

Republika Hrvatska

My girlfriend and I also went to Croatia for our holiday this year. We landed in Split, Dalmatia and had a wonderful time, our adventures too many to describe in detail here. We explored the ancient town of Split, the quaint town of Trogir, the island of Šolta and the fortress of Klis. My girlfriend had to return home a couple of days before I did, so I stuck around  and went on a day trip to the city of Šibenik. From there I went to see the beautiful waterfalls of Krka National Park. I stayed a little too long and missed the last bus back to Šibenik, oops! I had to barter with some locals to get a ride back to the city!

Career Prospects

I had been working for a tech start up called Green Man Gaming for several months before 2012. Early in the year my job role changed which saw me working alongside some exceptionally talented people, working with some very exciting technologies. This was a promotion, however, before the end of the year I had been headhunted for the new skills and experience I had gained. I am now working for The Lonely Planet at BBC Worldwide. Besides a 22% pay increase, my new position has opened up many opportunities which I hope to continue into 2013.

And Finally…

My final outing of 2012 was to The Vintage New Years Ball at London’s South Bank Centre. It took us ages to get there due to the crowds and the police hysterically closing bridges, but once we were there it was amazing; everyone was turned out in their best glamorous vintage outfits, and everyone was impressed with our dancing.

There have been many other things I have enjoyed this year; movies, music, parties (including 5 xmas parties), somethings I just don’t have time to detail, others I shouldn’t disclose here. Looking forward, all I have to say is: 2013, you have a hard act to follow!

VNC at graphical login for Ubuntu 12.10 or 12.04

  November 11, 2012 - 8:02 pm |
3 Comments

It sounds trivial; create an upstart script to start on boot, but this had me googling for a little while.

I used x11vnc as the vnc server. The main issue was that x11vnc needed to bind to an Xauthority file. In order to get it running for graphical login, the Xauthority needed to be for the desktop manager (in this case, lightdm) . It took me a while to find the right location ( ⁄var ⁄run ⁄lightdm ⁄root ⁄); once I had found the answer, the script was easy. I created a file in the upstart init script directory,  ⁄etc⁄init ⁄x11vnc.conf

Lets take a look at what this does; lines 1 and 3 are self explanatory, line 5, ensures that the vnc server is started at the right time (not before lightdm), otherwise the Xauthority file will not exist. Line 6 stops the vnc server on shutdown or reboot.

Lines 8 and 9 ensure that the vnc server is restarted if it fails, but if it respawns x times in y seconds it stops (10 times in 5 seconds in this case). This makes sure it is available but if there is a more serious problem, it stops trying. Line 10 ensures that the process has the correct permissions.

Finally, line 12 starts the x11vnc server. The -forever flag makes sure the server does not exit when a user logs out. The -auth flag is the Xauthority file, which lightdm stores in  ⁄var⁄run⁄lightdm⁄root⁄, usually :0, which indicates the display number lightdm binds to. The -display flag binds to display 0, though it should match the display number used for the Xauthority file. The -rfbauth flag is optional, used only if x11vnc has been configured to use a password file.

Reboot the system, and there it is. Simple but effective.

 

EDL vs Walthamstow: Why a 30 day protest ban was not a victory

  October 27, 2012 - 7:16 pm | ,
Leave a comment

On the 1st of September this year (2012), the English Defense League, an extreme right political group with a passion for hating muslims, attempted to hold a rally in Walthamstow. They were given permission by the police and the local council, however, their rally was derailed due to the overwhelming numbers of local opposition. For some extra detail, watch this video: EDL Defeat in Walthamstow

The clashes were mostly peaceful (mostly!) but the EDL were truly thwarted and humiliated; managing a pathetic 300 strong mob that was faced with 3-4k local protesters who blocked their route to the rally point. The EDL planned another march for today (27th October 2012), presumably with intent of enacting reprisals for their previous defeat. The local residents were more than ready to defeat them once again, however, this time the council and police decided to stop the rally. This was due to police intelligence suggesting imminent violent confrontation. The ban took the form of a 30 day ban on EDL and “related” protests. The counter rally still went ahead although it was quickly kettled and dispersed by a large police force after it stopped at the top of Walthamstow market.

  

The fact that the EDL never made it to Walthamstow today was obviously a relief for many residents and hailed as a victory by many involved in the counter protest. However, banning things in this way is not really a victory; it has some very negative connotations.

The EDL can (and do) claim this reaction as a victory, they claim to their supporters that Walthamstow and other areas are afraid of them, implying their strength which can result in greater confidence, an increase in recruiting and an impression of success and legitimacy. Without actually turning up, their members cannot see for themselves how strong and defiant the local people are. This is not a good thing.

This sort of action also results in reduced civil liberty for everyone in the local area. What happens if a local campaign group who opposed the EDL try to protest against cuts to council services? banned. What happens if they want to protest against a school or hospital closure? banned. If we slip into a culture that readily bans protests, pretty soon we will have no protests at all because there will be no right to protest. This is not a good thing.

Some will undoubtedly argue that letting protests and counter-protests happen is a bad thing because it costs the tax-payer, it disrupts traffic, puts people at risk and ties up police resources that could be deployed elsewhere. These are all valid points, it would obviously be better if there was no need in the first place. However, given that the EDL exist, and people like them will always exist we have a choice to make; stand up and fight for what we believe in or surrender to a court order that attempts to sweep it under the proverbial carpet. I know which I choose.

They who can give up essential liberty to obtain a little temporary safety, deserve neither liberty nor safety.

On 1st of September, the protest and counter-protests went ahead; this is how it should be every time the EDL think they can spread their message of hate. Let them come and express their abhorrent views; but let us come and express our views also, in much greater numbers and much louder voices.

Voyage: Coping with read-only file systems

  June 13, 2012 - 7:42 pm |
1 Comment

I am a user of Voyage Linux, a lightweight Debian flavour designed for integrated and low-powered computers. I use it with an ALIX 3D3 board which uses a Compact Flash card for it’s main storage. One of Voyage’s features that makes it ideal is that it operates with a read-only filesystem (a good idea if using a CF card). This introduces some interesting challenges; what about logs? what about home directories and command history?

Well Voyage attempts to resolve the first of these questions by mounting ⁄var⁄log as a tmpfs volume, which works ok but obviously useless if you restart or the box crashes. By default, home directories are stored on the main disk which makes them read-only; so no bash history and no meta files from applications such as vim or ssh. I also noticed sudo was outputting some warnings whenever I sudo’d and refused to remember that I had recently authenticated. Fixing these things are not very difficult, but I hope this helps someone out who has similar issues. Here is what I did.

First of all…

As Voyage mounts the root file system as read-only, it need to be remounted as writable in order to modify the system. Luckily there is a helpful script provided, so I just:

Now home directories

The obvious solution here is to mount an external drive (I used a USB pen drive, though I plan to use a NAS eventually). First I mounted the USB drive and made copies of my users’ home directories. Then I added the following line to ⁄etc⁄fstab

I then unmounted the pen drive and tested the config:

The first command will unmount, the second will trigger parsing the ⁄etc⁄fstab file and report any errors and the last lists all mounted volumes which included this line:

After a quick check that users could still read ⁄ write to their home directories I was good to go…. except for one thing. I use SSH keys, which shouldn’t be left on a USB pen drive. So I put them somewhere appropriate on the main disk and made symlinks back to them from ~⁄.ssh/id_rsa.

Next to fix the logs

Using the same USB volume I created a .log directory. I used the dot to prevent confusion with user home directories. I then copied all log files to it and created a symlink:

Now sudo

Whenever I sudo’d I was getting this mesage:

Sudo keeps information about a users authenticated state. I don’t really need this if I reboot, so I simply created a 1M tmpfs volume mounted on /var/lib/sudo by adding this line in the ⁄etc⁄fstab file:

Note the mode setting, this is important or sudo will keep complaining. Testing the mount again gave me this line:

After this sudo worked normally.

And Finally

Now that I had these things in place and everything running smoothly, there was just one bit of tidying required:

That remounted the root file system as read-only again and I was done.

RATM vs X-Factor

  December 16, 2009 - 9:26 pm | , ,
Leave a comment

(Reposted here by request)

To answer the question “what’s it all for?”, I think it is too simplistic to denote one reason – the creators of the group had theirs, I have mine, and the other 174997 people have theirs.

I can only tell you why I support it; I resent Pop[ular] music being presented and covered as if that is the entire scope of musical talent.

Sure some contestants may have real talent, sure they may deserve a record contract – but the extent to which the market is manufactured, manipulated and subverted, and quite frankly hi-jacked (X-Factor will try to claim the xmas no1 prize every year it runs) sickens me. The result is that other talented artists in other genres and tastes can never get a look in. Even if Joe does overtake RATM to become no1 it has made an impact, and it has also been fun – so we all win!

As for X-Factor [et al] itself, I believe the entire format to be based around exploitation, misinformation and lies:

- the contestants with no talent are exploited; they are just there for us all to laugh at and feel superior.

- the real contestants are exploited; the sheer amount of overemotional clips and sound bites derived from their interim activities gives the show producers hours of footage to pack out an hour on tv for virtually nothing.

- the winner is exploited; £1m record contract? T&Cs apply I bet you! Not one finalist from these shows has gone on to have a truly transcended music career, most only last a year, why? Because their fame only lasts as long as the X-Factor machinery holds them in the spot light. Says a lot about their talent if thats all it takes for the public to lose interest.

- and finally, the audience and viewers are exploited; they are sucked in by cheap laughs at the talentless entries, hooked by the fake sugary compliments thrown at those who do have talent, kept engrossed by the “emotional” rollercoaster of their X-Factor experience, and so after weeks and weeks of this, are so convinced that this is the epitaph of quality music, they go and buy the single – especially as it makes a marvellously convenient and easy Christmas gift!

Phew! That was a biggie *rant over*