About Matt

Cloud Systems Engineer at Reed Elsevier; cloud computing advocate, rock climber, swing dancer, amateur photographer, professional idiot....

So long 2012 and thanks for all the memories

Well another year is overe, so here is yet another retrospective of 2012. I won’t dwell on the negative and just share some of my personal highlights. I had ended 2011 in a swing dance club surrounded by friends and more prosecco than I could drink. I began 2012 with a dreadful hangover still half dressed in my tuxedo, a sure sign of a good night the night before. Well, that’s not strictly true, I didn’t get home until well gone 5am but you get what I mean. Anyway, I digress, so these were my highlights of 2012.

Aerials Workshops

For the uninitiated, “aerials” is a part of swing dancing usually reserved for the elite or rehearsed performances. They involve high-adrenaline moves where the lead catapults his dance partner into the air in a variety of ways. My girlfriend and I decided to attend a workshop early in the year and enjoyed it so much that we attended another bringing some friends along. We learned to do the pankcake, the whitey flip, the lamp-post (aka the helicopter)  and the screamer. These moves and many more can be seen in this amazing clip

The Sunshine Boys

It had been many years since I’d been to any kind of theatre, but this year my girlfriend and I went to see “The Sunshine Boys”; a comedy about two ageing Vaudeville comedians who had vowed never to work together again; at the glorious Savoy Theatre (yes that’s the same place as the hotel). The play was excellent with top-notch performances from the stars Danny DeVito and Richard Griffiths.

The London Lindy Exchange

The London Lindy Exchange was a weekend packed with dancing events where Londoners host lindy hoppers from around the world. The result was three amazing evenings with live big-band performances from the The Shirt Tail Stompers, Gentlemen & GangstersEchoes of Ellington and Skin Deep. The bands were all amazing, I danced with over 30 new people from at least 10 different countries. I even picked up a few new moves, variations and styling techniques.

Big Lebowski Party

I’d never tried a white russian before; they’re good, even better when drunk whilst sat in a dressing gown watching The Big Lebowski. In case you were wondering, that is what a Lebowski party involves, it was great.

Row Row Row Your Boat

My girlfriend and I went on a day trip to Broxborne (I like to call it Frogspawn) which is in the Lea Valley region, north of London. It was a nice sunny day, so we had a nice pub lunch outside and hired a row boat for the afternoon. We stopped off at Waltham Abbey on the way back into London. This was the same day that the EDL (English Defence League) tried to march on Walthamstow for a hate speech rally. We were glad to be far away from their vile hatred, although the community spirit of opposition shown by Walthamstow residents made me wish we’d been around. This video demonstrates their victory

Republika Hrvatska

My girlfriend and I also went to Croatia for our holiday this year. We landed in Split, Dalmatia and had a wonderful time, our adventures too many to describe in detail here. We explored the ancient town of Split, the quaint town of Trogir, the island of Šolta and the fortress of Klis. My girlfriend had to return home a couple of days before I did, so I stuck around  and went on a day trip to the city of Šibenik. From there I went to see the beautiful waterfalls of Krka National Park. I stayed a little too long and missed the last bus back to Šibenik, oops! I had to barter with some locals to get a ride back to the city!

Career Prospects

I had been working for a tech start up called Green Man Gaming for several months before 2012. Early in the year my job role changed which saw me working alongside some exceptionally talented people, working with some very exciting technologies. This was a promotion, however, before the end of the year I had been headhunted for the new skills and experience I had gained. I am now working for The Lonely Planet at BBC Worldwide. Besides a 22% pay increase, my new position has opened up many opportunities which I hope to continue into 2013.

And Finally…

My final outing of 2012 was to The Vintage New Years Ball at London’s South Bank Centre. It took us ages to get there due to the crowds and the police hysterically closing bridges, but once we were there it was amazing; everyone was turned out in their best glamorous vintage outfits, and everyone was impressed with our dancing.

There have been many other things I have enjoyed this year; movies, music, parties (including 5 xmas parties), somethings I just don’t have time to detail, others I shouldn’t disclose here. Looking forward, all I have to say is: 2013, you have a hard act to follow!

VNC at graphical login for Ubuntu 12.10 or 12.04

It sounds trivial; create an upstart script to start on boot, but this had me googling for a little while.

I used x11vnc as the vnc server. The main issue was that x11vnc needed to bind to an Xauthority file. In order to get it running for graphical login, the Xauthority needed to be for the desktop manager (in this case, lightdm) . It took me a while to find the right location ( ⁄var ⁄run ⁄lightdm ⁄root ⁄); once I had found the answer, the script was easy. I created a file in the upstart init script directory,  ⁄etc⁄init ⁄x11vnc.conf

Lets take a look at what this does; lines 1 and 3 are self explanatory, line 5, ensures that the vnc server is started at the right time (not before lightdm), otherwise the Xauthority file will not exist. Line 6 stops the vnc server on shutdown or reboot.

Lines 8 and 9 ensure that the vnc server is restarted if it fails, but if it respawns x times in y seconds it stops (10 times in 5 seconds in this case). This makes sure it is available but if there is a more serious problem, it stops trying. Line 10 ensures that the process has the correct permissions.

Finally, line 12 starts the x11vnc server. The -forever flag makes sure the server does not exit when a user logs out. The -auth flag is the Xauthority file, which lightdm stores in  ⁄var⁄run⁄lightdm⁄root⁄, usually :0, which indicates the display number lightdm binds to. The -display flag binds to display 0, though it should match the display number used for the Xauthority file. The -rfbauth flag is optional, used only if x11vnc has been configured to use a password file.

Reboot the system, and there it is. Simple but effective.

 

EDL vs Walthamstow: Why a 30 day protest ban was not a victory

On the 1st of September this year (2012), the English Defense League, an extreme right political group with a passion for hating muslims, attempted to hold a rally in Walthamstow. They were given permission by the police and the local council, however, their rally was derailed due to the overwhelming numbers of local opposition. For some extra detail, watch this video: EDL Defeat in Walthamstow

The clashes were mostly peaceful (mostly!) but the EDL were truly thwarted and humiliated; managing a pathetic 300 strong mob that was faced with 3-4k local protesters who blocked their route to the rally point. The EDL planned another march for today (27th October 2012), presumably with intent of enacting reprisals for their previous defeat. The local residents were more than ready to defeat them once again, however, this time the council and police decided to stop the rally. This was due to police intelligence suggesting imminent violent confrontation. The ban took the form of a 30 day ban on EDL and “related” protests. The counter rally still went ahead although it was quickly kettled and dispersed by a large police force after it stopped at the top of Walthamstow market.

The fact that the EDL never made it to Walthamstow today was obviously a relief for many residents and hailed as a victory by many involved in the counter protest. However, banning things in this way is not really a victory; it has some very negative connotations.

The EDL can (and do) claim this reaction as a victory, they claim to their supporters that Walthamstow and other areas are afraid of them, implying their strength which can result in greater confidence, an increase in recruiting and an impression of success and legitimacy. Without actually turning up, their members cannot see for themselves how strong and defiant the local people are. This is not a good thing.

This sort of action also results in reduced civil liberty for everyone in the local area. What happens if a local campaign group who opposed the EDL try to protest against cuts to council services? banned. What happens if they want to protest against a school or hospital closure? banned. If we slip into a culture that readily bans protests, pretty soon we will have no protests at all because there will be no right to protest. This is not a good thing.

Some will undoubtedly argue that letting protests and counter-protests happen is a bad thing because it costs the tax-payer, it disrupts traffic, puts people at risk and ties up police resources that could be deployed elsewhere. These are all valid points, it would obviously be better if there was no need in the first place. However, given that the EDL exist, and people like them will always exist we have a choice to make; stand up and fight for what we believe in or surrender to a court order that attempts to sweep it under the proverbial carpet. I know which I choose.

They who can give up essential liberty to obtain a little temporary safety, deserve neither liberty nor safety.

On 1st of September, the protest and counter-protests went ahead; this is how it should be every time the EDL think they can spread their message of hate. Let them come and express their abhorrent views; but let us come and express our views also, in much greater numbers and much louder voices.

Voyage: Coping with read-only file systems

I am a user of Voyage Linux, a lightweight Debian flavour designed for integrated and low-powered computers. I use it with an ALIX 3D3 board which uses a Compact Flash card for it’s main storage. One of Voyage’s features that makes it ideal is that it operates with a read-only filesystem (a good idea if using a CF card). This introduces some interesting challenges; what about logs? what about home directories and command history?

Well Voyage attempts to resolve the first of these questions by mounting ⁄var⁄log as a tmpfs volume, which works ok but obviously useless if you restart or the box crashes. By default, home directories are stored on the main disk which makes them read-only; so no bash history and no meta files from applications such as vim or ssh. I also noticed sudo was outputting some warnings whenever I sudo’d and refused to remember that I had recently authenticated. Fixing these things are not very difficult, but I hope this helps someone out who has similar issues. Here is what I did.

First of all…

As Voyage mounts the root file system as read-only, it need to be remounted as writable in order to modify the system. Luckily there is a helpful script provided, so I just:

Now home directories

The obvious solution here is to mount an external drive (I used a USB pen drive, though I plan to use a NAS eventually). First I mounted the USB drive and made copies of my users’ home directories. Then I added the following line to ⁄etc⁄fstab

I then unmounted the pen drive and tested the config:

The first command will unmount, the second will trigger parsing the ⁄etc⁄fstab file and report any errors and the last lists all mounted volumes which included this line:

After a quick check that users could still read ⁄ write to their home directories I was good to go…. except for one thing. I use SSH keys, which shouldn’t be left on a USB pen drive. So I put them somewhere appropriate on the main disk and made symlinks back to them from ~⁄.ssh/id_rsa.

Next to fix the logs

Using the same USB volume I created a .log directory. I used the dot to prevent confusion with user home directories. I then copied all log files to it and created a symlink:

Now sudo

Whenever I sudo’d I was getting this mesage:

Sudo keeps information about a users authenticated state. I don’t really need this if I reboot, so I simply created a 1M tmpfs volume mounted on /var/lib/sudo by adding this line in the ⁄etc⁄fstab file:

Note the mode setting, this is important or sudo will keep complaining. Testing the mount again gave me this line:

After this sudo worked normally.

And Finally

Now that I had these things in place and everything running smoothly, there was just one bit of tidying required:

That remounted the root file system as read-only again and I was done.

RATM vs X-Factor

(Reposted here by request)

To answer the question “what’s it all for?”, I think it is too simplistic to denote one reason – the creators of the group had theirs, I have mine, and the other 174997 people have theirs.

I can only tell you why I support it; I resent Pop[ular] music being presented and covered as if that is the entire scope of musical talent.

Sure some contestants may have real talent, sure they may deserve a record contract – but the extent to which the market is manufactured, manipulated and subverted, and quite frankly hi-jacked (X-Factor will try to claim the xmas no1 prize every year it runs) sickens me. The result is that other talented artists in other genres and tastes can never get a look in. Even if Joe does overtake RATM to become no1 it has made an impact, and it has also been fun – so we all win!

As for X-Factor [et al] itself, I believe the entire format to be based around exploitation, misinformation and lies:

– the contestants with no talent are exploited; they are just there for us all to laugh at and feel superior.

– the real contestants are exploited; the sheer amount of overemotional clips and sound bites derived from their interim activities gives the show producers hours of footage to pack out an hour on tv for virtually nothing.

– the winner is exploited; £1m record contract? T&Cs apply I bet you! Not one finalist from these shows has gone on to have a truly transcended music career, most only last a year, why? Because their fame only lasts as long as the X-Factor machinery holds them in the spot light. Says a lot about their talent if thats all it takes for the public to lose interest.

– and finally, the audience and viewers are exploited; they are sucked in by cheap laughs at the talentless entries, hooked by the fake sugary compliments thrown at those who do have talent, kept engrossed by the “emotional” rollercoaster of their X-Factor experience, and so after weeks and weeks of this, are so convinced that this is the epitaph of quality music, they go and buy the single – especially as it makes a marvellously convenient and easy Christmas gift!

Phew! That was a biggie *rant over*

Miind and beyond

After a few snags with a dependency that did not want to play nicely, I managed to get the MIIND simulator installed. This is required for the next step of the project.

I now need to demonstrate that as the mu increases (which we can now predict reasonably accurately), the output firing rate steadily increases. This produces a sigmoid shape on a histogram… after a fashion!

Today I ran the one population model [to test, we need to do the same with a two population model], with a variety of different rates, which equates to different mu values. The mu can also be varied by altering the synaptic weight as well, although this would be much less significant changes, and should in theory result in exactly the same curve.

Once I have done this in NEST I will need to replicate it in MIIND

Back on track

Since last week I managed to track down and resolve the problems I had been having with help from Marc. It turned out to be a quota problem; the data exceed my quota and thus overwrote parts of itself. I thought I had modified my script to do this, but evidently not. Once I had got my script using /tmp/ (and my portable hdd) instead of my home folder, everything works fine again.

I now have a working, one population model. The input and output is consistent as far as I can tell. This is something that is bothering me though, I don’t feel like I am understanding the theory as well as I should. It is certainly making my progress slow, which is frustrating. I am trying to make sense of the Amit & Brunel papers, but they’re big papers and it is all too easy to get lost n the text.

A firm grasp on the background is becoming most apparent now that I have started my final report. I will not be able to write up that section well at all unless I do.

Not so good

Yesterday was not a good day for my project; I found myself being propelled backwards from the doorstep of phase 2, ending up at the gates of phase 1, right where I started…. well sort of. It certainly felt that way.

The most significant acheivement I have made som far is confirming that I can correctly predict the properties of the output for a neuron population. The next stage is to use models with multiple populations. Before I could move on to this stage, I needed to remind myself of the tools that I had been using before.

The way to do this was to repeat the post-processing of the simulated output for the single population model. As the output files are mammoth in size, I can only store them in /tmp/. This means I had to run the simulation again as well.

Something is not right. While the size of the file is almost identical, the order of magnitude of spikes is correct, my post-processing tool was failing. Given that the output for any given model should be identical no matter how many times it is run, and that the tool worked before, this should not happen. I have modified the tool to cope with the error and produce the meaningful output.

The output I now get still fits the theoretical predictions, and so is correct. However, it is most disconcerting that it is not identical as it should be. I have yet to establish what exactly causes the NEST simulator to behave differently, or if the annomolies are due to the corrupt file. Even if that is the case, I do not know what is causing that to happen.

I will definately be mentioning this to Marc in our next meeting.

The Story So Far

So far, I have sucessfully managed to implement a one populatuion model and estimate that its behaviour is consistent. I have done this by measuring the spikes generated by the poisson generators (input of the population) and the spikes generated by the population itself (output of the population).

The results from the latter step need to be checked against theory. The formulea to do so is extremely complex and I am expecting some software from Marc to help with these calculations. I was able to calculate the input rate as these formulae are relatively simple.

I have implemented (I believe) a two population model, with both excitatory and inhibitory populations. The model has a poisson generator for each excitatory neuron. These provides the initial stimulus for the excitatory population. The excitatory and inhibitory populations each have 0.1 * the population size, random connections to other neurons (self or cross population). Each population also has a spike detector attached.

Using the same variable values as previously used for the one population model, this yielded very little activity (17 ex-spikes and 0 in-spikes). However, if the weights are tweaked upwards, I found that the excitatory population started off barely spiking, until about 40ms where its rate increased exponetially to 50000 before 50ms, after which it maintained this firing rate. The inhibitory population displayed similar behaviour but on a smaller scale (as we would expect with a smaller population).

The resulting histograms derived from these spiking data clearly show a sigmoid shape. This indicates stable regular firing, though I have yet to confirm this.