I often bang on about how useful twitter is for crowd-sourcing a research community. Today I was reminded of quite brilliant the people on twitter can be at helping to overcome an ‘I don’t know where to start’-type information problem.
I’m currently helping to design an fMRI study which could benefit considerably from the application of multivoxel pattern analysis (MVPA). Having no practical experience with MVPA means I’m trying to figure out what I need to do to make the MVPA bit of the study a success. After a few hours of searching, I have come across and read a number of broad theoretical methods papers, but nothing that gives me the confidence that anything I come up with will be viable. Of course, there’s no right way of designing a study, but there are a tonne of wrong ways, and I definitely want to avoid those.
So, I turned to twitter:
Twitter help please. Can anyone recommend a beginner’s guide to fMRI MVPA, from trial counts required to software, analysis steps etc? — Akira O’Connor (@akiraoc) January 23, 2013
Relays and Retweets from @hugospiers, @zarinahagnew and @neuroconscience led to the following tweets coming my way (stripped of @s for ease of reading… kind of).
Our lab works with min 40 trials per condition for MVPA. I think there is a poster out there maybe from the Haxby group on this. — M Barnett-Cowan (@multisensebrain) January 23, 2013
Sounds about right – depends a LOT on task/design though. Could perhaps get away with less. — Matt Wall (@m_wall) January 23, 2013
Sure, I could have come up with as many articles to read by typing “MVPA” into Google Scholar (as I have done in the past), but the best thing about my twitter-sourced reading list is that I’m confident it’s pitched at the right level.
I’m humbled by how generous people are with their time, and glad so many friendly academics are on twitter. I hope collegiality and friendliness like this encourages many more to join our ranks.
IFTTT, if this then that, is an online, multi-service task automation tool I first read about on Lifehacker last year. I finally started using it today, and am seriously impressed.
Once you’ve signed up for an account, you can create IFTTT ‘recipes’ to check for actions and events on one online service (e.g. Google Reader, Dropbox, WordPress, Facebook etc.) and use them as an automatic trigger of a predetermined action in another (e.g. Gmail, Google Calendar, Tumblr etc.)
Example: To keep track of journal articles I should read, I monitor journal table of contents RSS feeds and e-mail interesting posts to myself for later download and consumption. I use my iPad, my phone, and occasionally my PC browser to access Google Reader, but struggle with how fiddly it is to e-mail myself on my mobile devices (with my filter-trigger keywords in the message body) whenever I find an article I want to read. I’m sure I’ve missed articles I ought to have read through setting my action criterion a bit too high, as a direct result of how annoying it is to e-mail myself articles using the various Google Reader interfaces on my mobile devices. Today I set up IFTTT to check for starred Google Reader feed items, and automatically do everything else beyond this that I find annoying. Perfect!
IFTTT will check for custom recipe triggers every 15 minutes, so it isn’t something you’d want to use for actions you require to be instantaneous, but it’s perfect for situations like the above. The services with which it is integrated are many and varied, and the possibilities therefore nearly limitless.
UPDATE 16/04/2014: I just came across this page and found that I had referenced the now defunct Google Reader. When Reader died I moved all of my RSS feeds across to feedly, which IFTTT supports with identical functionality. I also apply the same rule to twitter posts I favourite, meaning that I have a Gmail folder in which IFTTT aggregates all of the stuff I want to read from both feedly and twitter.
A couple of months ago, the folks at codecademy were nice enough to respond to a complimentary e-mail I’d sent them by writing something nice back about me and publishing it on their codecademy.com/stories page.
On Monday I gave a talk on how internet tools can be used to make the job of being an academic a little easier. I had given a very short version of the talk to faculty in the department over a year ago, but this time I was given an hour in a forum for early career researchers, PhD students and postdocs. The subject of twitter, covered early on in the talk, aroused a lot of interest, probably because I got very animated about its benefits for those in the early stages of their careers.
To provide a little context for my enthusiasm, it probably helps to know a few things about me, about my situation, and about my recent experiences.
I am an introvert. Despite my best (and occasionally successful) efforts to project a different image, I do not find talking to people I don’t know very enjoyable.
I am an early career cognitive neuroscientist keen to build my own research programme and develop links with other researchers.
Last month I attended the Society for Neuroscience conference, at which I attended the best conference social I have ever attended.
Given the received wisdom that people in my position ought to be networking, I often drag myself kicking and screaming to conference socials. The result tends to be a lot of standing around on my own drinking beer, which gives me something to do, but which I could do much more comfortably with one or two people I know well. The major problem at these events is not my nature, or my status as an early career researcher, but the fact that the people I have imagined myself talking to usually don’t know who I am. Conversation is therefore awkward, one-sided and introductory. Once the niceties have dried up, and the level of accumulated conversational silence edges into awkward territory I invariably finish my drink and bugger off to get another one, ending the misery for all involved. This is probably a universal experience for those starting out in academia, though thankfully it is happening less and less to me as I build something of network of real friends who attend the same conferences as me. But as a PhD student and postdoc, the experience was excruciating.
I had a totally different experience when I attended the SfN Banter tweetup*. The event, organised by @doc_becca and @neuropolarbear, was a social for neuroscientists who use twitter and changed my view of conference socials. They do not have to be endured, even by those doing PhDs and postdocs. They can be enjoyed.
I was excited about going and that excitement didn’t leave me feeling shortchanged by the time I left. I spoke (actually spoke!) to everyone I wanted to speak to. Moreover, I had good conversations with people to whom I was speaking for the first time. The reason is fairly obvious – twitter allowed us to build on a body of shared (or at least assumed) knowledge. I follow people, they follow me, I reply to or retweet their tweets, they do the same – and this is all before we’ve introduced ourselves. When I finally meet someone with whom I have such a history of communication, introducing myself is the least awkward thing I can do. The barriers to conversation are removed**.
Sure, this pattern followed for most interactions at the tweetup because we were all there to do exactly that. Would the experience be the same at the ‘fMRI social’? No. But, I don’t think that matters. If I could have had one of those conference social experience during my time as a PhD student, it would have given me an idea of what I might have to look forward to from conferences if I stuck at it. Light at the end of the tunnel, a pot of gold at the end of the rainbow, a variable-ratio schedule-determined stimulation of the limbic system following an umpteenth lever press.
It will take a while (there’s no point joining in September 2013 and expecting great things at the SfN tweetup in San Diego), and it’s probably not the primary reason to join twitter (see Dorothy Bishop’s blog and Tom Hartley’s blog for far more comprehensive discussions of how and why you should join), but it’s another reason, and it’s one that could make you feel good about your role in academia. It’s worth a shot.
* tw(itter) (m)eetup, see?
** What you do afterwards is up to you. I still had some awkward interactions, but I think that’s probably down to me (see context point 1).
Tomorrow morning I fly to SfN 2012 in New Orleans. There has been some turbulence in the immediate lead-up to it – an unscheduled flight to Ireland putting back my departure by a couple of days and a trip to A&E with anaphylaxis being the major ones- but nonetheless, the trip is happening.
This is a pretty big deal for me and for the lab. It’s the first major presentation of data we have collected independently of more senior PIs (and they’re not bad-looking data). It’s also only the second time I will have presented at this particular conference, which can be a tad overwhelming. This could be the start of something good.
Of course, there’s also some more social fun planned. I’m looking forward to the #SfN12 #tweetup (Monday night), when I’ll be putting some real-life personalities to some online ones, and the Scottish Neuroscience Group drinks (Tuesday). I’m not anticipating any more storms, other than the odd hurricane, rum and all.
On top of the study completion data that’s obvious from the 7 KB csv file that each happily-debriefed participant leaves behind, the Google Analytics code embedded in each page of the experiment provides further opportunity to explore participation data.
As the experiment structure is entirely linear, it’s possible to track the loss of participants from each page to the next.
The major point of attrition is between the Participant Information Page and the Consent Form – not surprising given quite how text-heavy the first page was, and how ‘scary’ headings like “Are there any potential risks to taking part?” make the study sound. The content of that first page is entirely driven by the Informed Consent requirements of the University of St Andrews, but the huge attrition rate here has prompted a bit of a redesign in the next follow-up study.
Other information useful for the design of future studies has been the browser data. As might be expected, Firefox and its relatives are the dominant browsers, with Chrome a distant second and Internet Explorer lagging far behind. Implementing fancy HTML5 code that won’t work in Firefox is therefore a bad idea. On top of that, despite how tablet- and phone-friendly the experiment was, very few people used this sort of device to complete the study – it’s probably a waste of time optimising the site specifically for devices like iPads.
Curiously enough, when the data for study completions are explored by browser, the three major platforms start to level up. Chrome, Firefox and IE all yield similar completion statistics, suggesting that IE browsers are far more likely to follow through and complete the study once they visit the site. I’m speculating here, but I suspect that this has something to do with a) this being a memory study and b) IE being used by an older demographic of internet user who may be interested in how they perform. Of the three major browsers, Firefox users have the worst completion rate.
Another consideration with word-based experiments is the location of participants. This could impact on the choice of words used in future studies (American or UK spellings) and could be considered important by those who are keen to exclude those who don’t speak English as their first language. Finer grained information about participants’ first languages is something we got from participant self-reports in the demographic questionnaire, but the table of new visits and study completions is still rather interesting.
Once again, there are few surprises here, with the US dominating the new visits list, though one new visit from a UK- or India-based browser is more likely to lead to a study completion. A solid argument for using North American spellings and words could also be made from these data.
Source of Traffic
The most important thing to do to make potential participants aware of an online psychology study is to advertise it. But where?
While getting the study listed on stumbleupon was a real coup, it didn’t lead to very many study completions (a measly 2.5%). That’s not surprising – the study doesn’t capture the attention from page 1 and doesn’t have much in the way of internet meme-factor. That is, of course, something that we should be rectifying in future studies if we want them to go viral, but it’s tough to do within the rigid constraints of the informed consent pages that must precede the study itself.
These statistics, surprised me more than any other. I assumed that social networking, not a dedicated experiment listing page, would be how people would find the study. But in retrospect, it all makes sense. There is clearly a large number of people out there who want to do online psychology studies, and what better way to find them than to use a directory that lists hundreds of them. If there’s one place you should advertise your online studies, it’s psych.hanover.edu.
To present stimuli for my experiments in the lab, I use Psychophysics Toolbox (Psychtoolbox) in conjunction with Matlab.
One limitation of Psychtoolbox is that the included DrawFormattedText function does not allow text to be horizontally centered on a point other than the horizontal center of the screen. That frustration doesn’t seem to make much sense, but what I mean by it is that you cannot offset the centering (as you could by choosing to centering within different columns of a table) – If you try and place the text anywhere other than the horizontal center of the screen, the text must be left-aligned.
This means that, when using the original DrawFormattedText, instead of nice-looking screens like this:
you get this:
which is a little messy.
To fix this, I have modified the DrawFormattedText file to include an xoffset parameter. It’s a very basic modification, that allows text to be centered on points offset from the horizontal center of the screen. For example, calling DrawFormattedText_mod with:
1) xoffset set to -100, centers text horizontally on a point 100 pixels to the left of the horizontal center of the screen.
2) xoffset set to rect(3)/4 (where rect = Screen dimensions e.g. [0 0 1024 768]), centers text horizontally 1/3 of the way from the left hand edge.
I haven’t replaced my DrawFormattedText.m with my DrawFormattedText_mod.m just yet, but it has been added to the path and seems to be doing the trick.
This year, I decided to learn how to present cognitive psychology experiments online. Five months in, I’m happy with my progress.
Before I present the experiment I have spent the past few months working on, here are a few things I have learned from the experience so far.
3) It is very quick and very easy to learn how to code functionally… if it works – it is generally functional. It is much more difficult to learn how to code both elegantly and functionally. I do not know how to code elegantly and I don’t think I ever will. (I’m not flippant about this either. This is something I would really like to learn how to do).
5) Web security is a subject on which I have very little knowledge.
6) Sending information from a browser to a server is a pain in the arse.
It is a fairly straightforward recognition experiment, takes about 15 minutes to complete and should provide data for use in a larger project, so do feel free to take it as seriously as you want. As I have already mentioned, it works on an iPad, and I thoroughly recommend you give it a go this way if you have access to one.