A few months ago, I suggested that Raspberry Pis could be used as a barebones experiment presentation machine. Since then I have got my hands on one and tinkered a little, only to be reminded yet again that my inability to do anything much in both Linux and python is a bit of a problem.
Fortunately, others with more technological nous have been busy exploring the capabilities of the Pi, with some exciting findings. On the Cognitive Science Stack Exchange, user appositive asked “Is the Raspberry Pi capable of operating as a stimulus presentation system for experiments?” and followed up at the end of January with a great answer to their own question, including this paragraph:
The RPi does not support OpenGL. I approached this system with the idea of using a python environment to create and present experiments. There are two good options for this that I know of, opensesame and psychopy. Psychopy requires an OpenGL python backend (pyglet), so it won’t run on the Rpi. Opensesame gives you the option of using the same backend as PsychoPy uses but has other options, one of which does not rely on openGL (based on pygames). This ‘legacy’ backend works just fine. But the absence of openGL means that graphics rely solely on the 700 mHz CPU, which quickly gets overloaded with any sort of rapidly changing visual stimuli (ie. flowing gabors, video, etc.).
Because of the lack of OpenGL support on the Pi, Psychopy is out (for now) leaving OpenSesame as the best cog psych-focused python environment for experiment presentation. The current situation seems to be that the Pi is suboptimal for graphics-intensive experiments, though this may improve as hardware acceleration is incorporated to take advantage of the Pi’s beefy graphics hardware. As things stand though, experiments with words and basic picture stimuli should be fine. It’s just a case of getting hold of one and brushing up on python.
UPDATE via Comments (1/4/2013) - Sebastiaan Mathôt has has published some nice Raspberry Pi graphics benchmarking data, which are well worth a look if you’re interested.
A couple of months ago, the folks at codecademy were nice enough to respond to a complimentary e-mail I’d sent them by writing something nice back about me and publishing it on their codecademy.com/stories page.
That changed today. I have a very simple new experiment for people to try their hand at, which can be accessed via the experiments (online) link on the right, or directly, here: http://www.st-andrews.ac.uk/~oclab/memorywords/.
On top of the study completion data that’s obvious from the 7 KB csv file that each happily-debriefed participant leaves behind, the Google Analytics code embedded in each page of the experiment provides further opportunity to explore participation data.
As the experiment structure is entirely linear, it’s possible to track the loss of participants from each page to the next.
The major point of attrition is between the Participant Information Page and the Consent Form – not surprising given quite how text-heavy the first page was, and how ‘scary’ headings like “Are there any potential risks to taking part?” make the study sound. The content of that first page is entirely driven by the Informed Consent requirements of the University of St Andrews, but the huge attrition rate here has prompted a bit of a redesign in the next follow-up study.
Other information useful for the design of future studies has been the browser data. As might be expected, Firefox and its relatives are the dominant browsers, with Chrome a distant second and Internet Explorer lagging far behind. Implementing fancy HTML5 code that won’t work in Firefox is therefore a bad idea. On top of that, despite how tablet- and phone-friendly the experiment was, very few people used this sort of device to complete the study – it’s probably a waste of time optimising the site specifically for devices like iPads.
Curiously enough, when the data for study completions are explored by browser, the three major platforms start to level up. Chrome, Firefox and IE all yield similar completion statistics, suggesting that IE browsers are far more likely to follow through and complete the study once they visit the site. I’m speculating here, but I suspect that this has something to do with a) this being a memory study and b) IE being used by an older demographic of internet user who may be interested in how they perform. Of the three major browsers, Firefox users have the worst completion rate.
Another consideration with word-based experiments is the location of participants. This could impact on the choice of words used in future studies (American or UK spellings) and could be considered important by those who are keen to exclude those who don’t speak English as their first language. Finer grained information about participants’ first languages is something we got from participant self-reports in the demographic questionnaire, but the table of new visits and study completions is still rather interesting.
Once again, there are few surprises here, with the US dominating the new visits list, though one new visit from a UK- or India-based browser is more likely to lead to a study completion. A solid argument for using North American spellings and words could also be made from these data.
Source of Traffic
The most important thing to do to make potential participants aware of an online psychology study is to advertise it. But where?
While getting the study listed on stumbleupon was a real coup, it didn’t lead to very many study completions (a measly 2.5%). That’s not surprising – the study doesn’t capture the attention from page 1 and doesn’t have much in the way of internet meme-factor. That is, of course, something that we should be rectifying in future studies if we want them to go viral, but it’s tough to do within the rigid constraints of the informed consent pages that must precede the study itself.
The most fruitful source of participants was the psych.hanover.edu Psychological Research on the Net page. It was much more successful at attracting visits and study completions than facebook, the best of the social networks, and the other online experiment listing sites on which we advertised the study (onlineresearch.co.uk and http://www.socialpsychology.org/expts.htm). What’s more, there has been a sustained stream of visitors from the psych.hanover.edu page that hasn’t tailed off as the study has been displaced from the top of the Recently Added Studies list.
These statistics, surprised me more than any other. I assumed that social networking, not a dedicated experiment listing page, would be how people would find the study. But in retrospect, it all makes sense. There is clearly a large number of people out there who want to do online psychology studies, and what better way to find them than to use a directory that lists hundreds of them. If there’s one place you should advertise your online studies, it’s psych.hanover.edu.
To present stimuli for my experiments in the lab, I use Psychophysics Toolbox (Psychtoolbox) in conjunction with Matlab.
One limitation of Psychtoolbox is that the included DrawFormattedText function does not allow text to be horizontally centered on a point other than the horizontal center of the screen. That frustration doesn’t seem to make much sense, but what I mean by it is that you cannot offset the centering (as you could by choosing to centering within different columns of a table) – If you try and place the text anywhere other than the horizontal center of the screen, the text must be left-aligned.
This means that, when using the original DrawFormattedText, instead of nice-looking screens like this:
you get this:
which is a little messy.
To fix this, I have modified the DrawFormattedText file to include an xoffset parameter. It’s a very basic modification, that allows text to be centered on points offset from the horizontal center of the screen. For example, calling DrawFormattedText_mod with:
1) xoffset set to -100, centers text horizontally on a point 100 pixels to the left of the horizontal center of the screen.
2) xoffset set to rect(3)/4 (where rect = Screen dimensions e.g. [0 0 1024 768]), centers text horizontally 1/3 of the way from the left hand edge.
I haven’t replaced my DrawFormattedText.m with my DrawFormattedText_mod.m just yet, but it has been added to the path and seems to be doing the trick.
You can download my DrawFormattedText_mod.m here: https://dl.dropbox.com/u/4127083/Scripts/DrawFormattedText_mod.m
This year, I decided to learn how to present cognitive psychology experiments online. Five months in, I’m happy with my progress.
Before I present the experiment I have spent the past few months working on, here are a few things I have learned from the experience so far.
3) It is very quick and very easy to learn how to code functionally… if it works – it is generally functional. It is much more difficult to learn how to code both elegantly and functionally. I do not know how to code elegantly and I don’t think I ever will. (I’m not flippant about this either. This is something I would really like to learn how to do).
5) Web security is a subject on which I have very little knowledge.
6) Sending information from a browser to a server is a pain in the arse.
And finally, here is the experiment:
It is a fairly straightforward recognition experiment, takes about 15 minutes to complete and should provide data for use in a larger project, so do feel free to take it as seriously as you want. As I have already mentioned, it works on an iPad, and I thoroughly recommend you give it a go this way if you have access to one.
Which brings me to why I sought out Codecademy in the first place (thanks to @m_wall for the twitter-solicited tip-off) – I am preparing to teach Psychology undergrads how to code. From 2012/2013 onwards, my academic life is going to be a little more ‘balanced’. As well as the research, admin and small-group teaching I currently enjoy, I’m also going to be doing some large-group teaching. Although I have plenty to say to undergraduates on cognitive neuroscience and cognitive psychology, I think giving them some coding skills will actually be much more useful to most. As my experience with Codecademy has recently reinforced to me, coding basics are the fundamental building-blocks of programming in any language. They will hold you in good stead whatever dialect you end up speaking to your computer in. What’s more, they will hold you in good stead whatever you end up doing, as long as it involves a computer: coding is the most versatile of transferable skills to be imparting to psychology graduates who (rightly) believe they are leaving university with the most versatile of degrees.