English: Extract from Raspberry Pi board at Tr...
The Raspberry Pi (photo credit: Wikipedia)

A few months ago, I suggested that Raspberry Pis could be used as a barebones experiment presentation machine. Since then I have got my hands on one and tinkered a little, only to be reminded yet again that my inability to do anything much in both Linux and python is a bit of a problem.

Fortunately, others with more technological nous have been busy exploring the capabilities of the Pi, with some exciting findings. On the Cognitive Science Stack Exchange, user appositive asked “Is the Raspberry Pi capable of operating as a stimulus presentation system for experiments?” and followed up at the end of January with a great answer to their own question, including this paragraph:

The RPi does not support OpenGL. I approached this system with the idea of using a python environment to create and present experiments. There are two good options for this that I know of, opensesame and psychopy. Psychopy requires an OpenGL python backend (pyglet), so it won’t run on the Rpi. Opensesame gives you the option of using the same backend as PsychoPy uses but has other options, one of which does not rely on openGL (based on pygames). This ‘legacy’ backend works just fine. But the absence of openGL means that graphics rely solely on the 700 mHz CPU, which quickly gets overloaded with any sort of rapidly changing visual stimuli (ie. flowing gabors, video, etc.).

Because of the lack of OpenGL support on the Pi, Psychopy is out (for now) leaving OpenSesame as the best cog psych-focused python environment for experiment presentation. The current situation seems to be that the Pi is suboptimal for graphics-intensive experiments, though this may improve as hardware acceleration is incorporated to take advantage of the Pi’s beefy graphics hardware. As things stand though, experiments with words and basic picture stimuli should be fine. It’s just a case of getting hold of one and brushing up on python.

UPDATE via Comments (1/4/2013) - Sebastiaan Mathôt has has published some nice Raspberry Pi graphics benchmarking data, which are well worth a look if you’re interested.
http://www.cogsci.nl/blog/miscellaneous/216-running-psychological-experiments-on-a-raspberry-pi-with-opensesame

A couple of months ago, the folks at codecademy were nice enough to respond to a complimentary e-mail I’d sent them by writing something nice back about me and publishing it on their codecademy.com/stories page.

cadecademic

It was great to get this sort of coverage on a site I think is fantastic (despite the picture of me with a rather supercilious looking one-year old on my back). The only problem was that I didn’t have a working example of code available for them to link to. The JavaScript experiments I had previously coded had run their course, garnering approximately 200 participants each, and had been taken offline, leaving the lab experiment page with nothing for people to try their hand at.

That changed today.  I have a very simple new experiment for people to try their hand at, which can be accessed via the experiments (online) link on the right, or directly, here: http://www.st-andrews.ac.uk/~oclab/memorywords/.

I’m afraid it doesn’t quite do justice to the “neuroscientist discovers javascript” headline on the codecademy stories page, but it’s something.

The lab’s first Javascript experiment has been online for about 3 weeks now, and has amassed close to 200 participants. It’s been a great experience discovering that the benefits of online testing (60+ participants a week, many of them run while I’m asleep!) easily outweight the costs (the time expended learning Javascript and coding all the fiddly bits, particularly the informed consent procedures and performance-appropriate feedback).

On top of the study completion data that’s obvious from the 7 KB csv file that each happily-debriefed participant leaves behind, the Google Analytics code embedded in each page of the experiment provides further opportunity to explore participation data.

Attrition

As the experiment structure is entirely linear, it’s possible to track the loss of participants from each page to the next.

Study Attrition

The major point of attrition is between the Participant Information Page and the Consent Form – not surprising given quite how text-heavy the first page was, and how ‘scary’ headings like “Are there any potential risks to taking part?” make the study sound. The content of that first page is entirely driven by the Informed Consent requirements of the University of St Andrews, but the huge attrition rate here has prompted a bit of a redesign in the next follow-up study.

Browser

New Visits by Browser

Other information useful for the design of future studies has been the browser data. As might be expected, Firefox and its relatives are the dominant browsers, with Chrome a distant second and Internet Explorer lagging far behind. Implementing fancy HTML5 code that won’t work in Firefox is therefore a bad idea. On top of that, despite how tablet- and phone-friendly the experiment was, very few people used this sort of device to complete the study – it’s probably a waste of time optimising the site specifically for devices like iPads.

Study Completions by Browser
Study Completions by Browser

Curiously enough, when the data for study completions are explored by browser, the three major platforms start to level up. Chrome, Firefox and IE all yield similar completion statistics, suggesting that IE browsers are far more likely to follow through and complete the study once they visit the site. I’m speculating here, but I suspect that this has something to do with a) this being a memory study and b) IE being used by an older demographic of internet user who may be interested in how they perform. Of the three major browsers, Firefox users have the worst completion rate.

Location

Another consideration with word-based experiments is the location of participants. This could impact on the choice of words used in future studies (American or UK spellings) and could be considered important by those who are keen to exclude those who don’t speak English as their first language. Finer grained information about participants’ first languages is something we got from participant self-reports in the demographic questionnaire, but the table of new visits and study completions is still rather interesting.

New Visits and Study Completions by Country

Once again, there are few surprises here, with the US dominating the new visits list, though one new visit from a UK- or India-based browser is more likely to lead to a study completion. A solid argument for using North American spellings and words could also be made from these data.

Source of Traffic

The most important thing to do to make potential participants aware of an online psychology study is to advertise it. But where?

Study Completions by Source

While getting the study listed on stumbleupon was a real coup, it didn’t lead to very many study completions (a measly 2.5%). That’s not surprising – the study doesn’t capture the attention from page 1 and doesn’t have much in the way of internet meme-factor. That is, of course, something that we should be rectifying in future studies if we want them to go viral, but it’s tough to do within the rigid constraints of the informed consent pages that must precede the study itself.

The most fruitful source of participants was the psych.hanover.edu Psychological Research on the Net page. It was much more successful at attracting visits and study completions than facebook, the best of the social networks, and the other online experiment listing sites on which we advertised the study (onlineresearch.co.uk and http://www.socialpsychology.org/expts.htm). What’s more, there has been a sustained stream of visitors from the psych.hanover.edu page that hasn’t tailed off as the study has been displaced from the top of the Recently Added Studies list.

These statistics, surprised me more than any other.  I assumed that social networking, not a dedicated experiment listing page, would be how people would find the study. But in retrospect, it all makes sense. There is clearly a large number of people out there who want to do online psychology studies, and what better way to find them than to use a directory that lists hundreds of them.  If there’s one place you should advertise your online studies, it’s psych.hanover.edu.

To present stimuli for my experiments in the lab, I use Psychophysics Toolbox (Psychtoolbox) in conjunction with Matlab.

One limitation of Psychtoolbox is that the included DrawFormattedText function does not allow text to be horizontally  centered on a point other than the horizontal center of the screen. That frustration doesn’t seem to make much sense, but what I mean by it is that you cannot offset the centering (as you could by choosing to centering within different columns of a table) – If you try and place the text anywhere other than the horizontal center of the screen, the text must be left-aligned.

This means that, when using the original DrawFormattedText,  instead of nice-looking screens like this:

Note that words 1 and 3 are well centered within their boxes

you get this:

Note that word 2 is centered, but words 1 and 3 are left-aligned within their boxes

which is a little messy.

To fix this, I have modified the  DrawFormattedText file to include an xoffset parameter. It’s a very basic modification, that allows text to be centered on points offset from the horizontal center of the screen.  For example, calling DrawFormattedText_mod with:
1) xoffset set to -100, centers text horizontally on a point 100 pixels to the left of the horizontal center of the screen.
2) xoffset set to rect(3)/4 (where rect = Screen dimensions e.g. [0 0 1024 768]), centers text horizontally 1/3 of the way from the left hand edge.
I haven’t replaced my DrawFormattedText.m with my DrawFormattedText_mod.m just yet, but it has been added to the path and seems to be doing the trick.

You can download my DrawFormattedText_mod.m here: https://dl.dropbox.com/u/4127083/Scripts/DrawFormattedText_mod.m

This year, I decided to learn how to present cognitive psychology experiments online. Five months in, I’m happy with my progress.

Since the second-year of my PhD, when I spent a couple of weeks getting nowhere with Java, I have been keen to use the web to present experiments. What enabled me to move from thinking about it to doing it was Codecademy. I’ve previously blogged about how useful I found the codecademy website in getting me familiar with the syntax of Javascript, but at the time of writing that post, I was unsure of how a knowledge of the coding architecture alone (and certainly not coding aesthetic) would translate into a webpage presenting a functional cognitive psychology experiment. Thankfully I did have a barebones knowledge of basic HTML, much of which is now obsolete and deprecated, from which I was able to salvage snippets to combine with CSS (thanks to w3schools) to get something functional and not hideously ugly.

Syllable-counting in the Study Phase.
(click to be taken to the experiment)

Before I present the experiment I have spent the past few months working on, here are a few things I have learned from the experience so far.

1) In choosing Javascript over a medium like Flash, I hoped to maximise the number of devices on which the experiments would run. I think I made the right choice. Pressing response buttons with your finger on an iPad or an Android phone feels like a Human Factors triumph!

2) Javascript-driven user-interaction operates quite differently to user-interaction in languages like Matlab. Javascript is user-driven, which means you can’t have the browser start an event that waits for a response – the browser will crash. Instead, you must start an event that changes the state of the elements within the browser, such that should those elements be responded to, it will be as if the browser had waited for a response.

3) It is very quick and very easy to learn how to code functionally… if it works – it is generally functional. It is much more difficult to learn how to code both elegantly and functionally. I do not know how to code elegantly and I don’t think I ever will. (I’m not flippant about this either.  This is something I would really like to learn how to do).

4) Getting everything to look OK in different browsers is a pain. It wasn’t so much the Javascript as the newer snippets of HTML5 that I have struggled to get to work in every browser.

5) Web security is a subject on which I have very little knowledge.

6) Sending information from a browser to a server is a pain in the arse.

 

And finally, here is the experiment:

http://www.st-andrews.ac.uk/~oclab/memoryframingexpt/

It is a fairly straightforward recognition experiment, takes about 15 minutes to complete and should provide data for use in a larger project, so do feel free to take it as seriously as you want. As I have already mentioned, it works on an iPad, and I thoroughly recommend you give it a go this way if you have access to one.

Points and badges

For the past week or so, I have been working my way through Codecademy’s JavaScript tutorials. I can’t recommend them highly enough.

As things stand, I have a full house of 480 points and 35 badges and, as the Codecademy creators would undoubtedly hope, I am rather satisfied with the JavaScript proficiency I have attained. ‘Attained’ is probably the wrong word to use though. Being a self-taught Matlab hacker, I have found most of my coding know-how has translated fairly well into Javascript. A few concepts (recursion in particular) have presented me with some difficulty, but the overall experience has been more like learning a new coding dialect  than a new language altogether. I haven’t attained a proficiency, so much as uncovered a hidden one.

Which brings me to why I sought out Codecademy in the first place (thanks to @m_wall for the twitter-solicited tip-off) – I am preparing to teach Psychology undergrads how to code. From 2012/2013 onwards, my academic life is going to be a little more ‘balanced’. As well as the research, admin and small-group teaching I currently enjoy, I’m also going to be doing some large-group teaching. Although I have plenty to say to undergraduates on cognitive neuroscience and cognitive psychology, I think giving them some coding skills will actually be much more useful to most. As my experience with Codecademy has recently reinforced to me, coding basics are the fundamental building-blocks of programming in any language. They will hold you in good stead whatever dialect you end up speaking to your computer in. What’s more, they will hold you in good stead whatever you end up doing, as long as it involves a computer: coding is the most versatile of transferable skills to be imparting to psychology graduates who (rightly) believe they are leaving university with the most versatile of degrees.

With all this in mind, one of Codecademy’s limitations is the difficulty with which its students can translate their new-found JavaScript skills into useful ‘stuff’ implemented outside the Codecademy editor. As Audrey Watters points out, there is barely any acknowledgement within the Codecademy tutorials that the goal of all of these points and badges is to encourage you to write interactive web contact in an IDE. Indeed, last night when I thought about how I would use JavaScript to administer online  memory experiments, I had to do a lot more reading. This could all be about to change though. If the latest Code Year class on HTML is anything to go by, the folks at Codecademy are mindful of this limitation, and are attempting to remedy it.

It’s just a shame that the html integration has come so late in the Code Year (yes, I say this with full awareness that we’re only on week 13).  If the HTML-Javascript confluence had come a little further upstream, I think there probably would have been a fledgling memory experiment linked to from this blogpost!