Suspension of disbelief

I have been thinking about suspension of disbelief, and how it relates to role playing games, story telling (books, films, TV series) and therapy. Not sure exactly where this is going.

Genre buy in

If you do not like a genre you won't normally go out of your way to expose yourself to that genre. I do not particularly like the conventions of play D&D so while I will play one offs I won't commit to a D&D campaign. I don't like basic role play dice mechanics (although they seem to be getting better with the ability to spend luck points), so again I will avoid campaign games in that style. Likewise I don't read romantic novels, and I don't watch slasher horror films. All very understandable.

If/when I need support for mental health there are only two genres freely available (and one with long waiting lists) Cognitive Behavioural Therapy and medication. The only choice here is to choose to go private and try to find a therapist who I can work with, but most people don't have the finances to do that.

Of course you can try and force yourself to buy in, despite a disbelief in the genre, with CBT I can normally manage that for the 5-6 weeks necessary. I do take a pragmatic approach most of the time, the actual exercises and interventions are useful even if the claims about how human beings work are not.

One of the things I noticed with the CBT classes run locally (and probably nationally) is they spend a lot of the first session trying to get people to believe in the model. While this might be useful for people who know little about the approaches it becomes a struggle otherwise.

Breaking Immersion

In the Walking Dead they played a game with their audience. One of the major characters appeared to have been eaten by zombies. They removed his name from the credits for a couple of weeks, and then he reappears after making a lucky (off camera) escape. That broke my suspension of disbelief. Since then I have sought out other reasons for not believing in that fictional world, two other things now bug me, the petrol in car petrol tanks should have gone off, and ammunition should be running low. I might watch the show if I am in the house when it is on, but don't seek it out on catch up.

In Tales from the Loop we have been playing through the adventures in the corebook. The third one involved the recurring bad person dragging dinosaurs into the present day. Immersion shattered. I can still play along but it becomes more game than roleplaying at that point.

A friend of mine was in a private therapeutic relationship, seeing someone twice a week, at considerable expense. They discussed with the therapist stepping down to once a week, the therapist saw this as trying to resist the therapy, and suggested they came more often. My friend couldn't afford that and ended up stopping the therapeutic relationship.

Breaking immersion carries heavier penalties in therapy than in leisure activities.

Is there anyway back from breaking immersion

This is the kicker. I am not willing to invest time in Walking Dead (or the Amazon version of Top Gear, the 'Grand Tour'). Role playing groups are about more than just what happens at the table, and we will be switching from Tales from the Loop soon. The more tricky thing is therapeutic relationships.

Will the new reporting guidelines for quantitative studies end the replication crisis

The narrative around the new reporting guidelines is very technocratic but the subtext is all about the much reported replication crisis. Back when I used to teach critical reading of psychology journal articles my approach was if something that should be in the report is missing that is because the authors are covering up a mistake. Hopefully the new guidelines will mean that authors will have to report the dodgy things they are doing, or they will be outright fraudulent. This can only be a good thing.

So lets get into some detail.

Preregistration
 An encouragement to preregister all psychology studies, but an acknowledgement it is only really happening in studies with a clinical application. This really does have to become routine across quantitative psychology.

Sampling

"Describe procedures for selecting participants, including...
• Percentage of sample approached that actually participated
• Whether self-selection into the study occurred (either by individuals or by units, such as schools or clinics)
• Settings and locations where data were collected as well as dates of data collection."

The idea of recording the number of potential participants approached is good, will give some information about possible volunteer bias. Self selection is another way that the characteristics of the participants can be problematic.
The requirement for date and location seems to be an attempt to deal with fraudulent data collection, follow this link for a recent example. 

Measuring instruments

"Define all primary and secondary measures and covariates, including measures collected but not included in this report."

After several failures to replicate 'power posing' findings one of the original authors made the claim that there were a whole bunch of measures that were not reported. That was always poor practice but these reporting conventions make sure that things are reported accurately, or the author(s) deliberately mislead.

Masking

"Report whether participants, those administering the experimental manipulations, and those assessing the outcomes were aware of condition assignments.
If masking took place, provide statement regarding how it was accomplished and if and how the success of masking was evaluated."

Everyone knows it is best that the person talking to the participant doesn't know the hypothesis and conditions, and if there is any chance of human error in scoring it is better that the scorer doesn't as well. But that is difficult to achieve and makes doing psychology experiments more expensive.

  

Analytical strategy

This works best if these are registered in advance.

Describe the analytic strategy for inferential statistics and protection against experiment-wise error for
Primary hypotheses
Secondary hypotheses
Exploratory hypotheses 

One thing that I suspect happens too often is people pretending exploratory hypotheses were the hypothesis they were always looking for. 

Participant Flow
Careful accounting of why any data was dropped, and keeping all the original data so others can look at it, both are necessary.

There is a lot more detail, but these are the things that seem to be important additions. It will be 2-3 years before the impact of this is felt, hopefully it is a step in the right direction. 

 

Reacting to the new APA guidelines for reporting qualitative studies

Yesterday the APA published guidelines for both qualitative and quantitative studies. I have had a quick flick both and am generally very encouraged. I might write about the quantitative guidelines this weekend, but I am going to write about the second paper now.


If you want to read the paper

Journal Article Reporting Standards for Qualitative Primary, Qualitative Meta-Analytic, and Mixed Methods Research in Psychology: The APA Publications and Communications Board Task Force Report,” (PDF, 163KB) by Heidi M. Levitt, PhD, University of Massachusetts Boston,: Michael Bamberg, PhD, Clark University; John W. Creswell, PhD, University of Michigan Medical School; David M. Frost, PhD, University College London; Ruthellen Josselson, PhD, Fielding Graduate University; and Carola Suárez-Orozco, PhD, University of California, Los Angeles, American Psychologist, Jan. 18, 2018.

I am just going to concentrate on the qualitative primary reporting standards.

The first eight pages are a good primer for quantitative research. I am very encouraged by the amount of flexibility that Levitt et al suggest is important, a worry before I read this was that they would be too constrained, but that is not a worry.

The main meat of the article is the table that begins on page 34.
The table has three columns, the section/subsection heading, a description of information and notes to reviewers and authors.

Here are some guidance notes I am particularly happy to see.


"Reviewers: The introduction may include case examples, personal narratives, vignettes, or other illustrative material."

"Reviewers: Method sections can be written in a chronological or narrative format." 
 

"Reviewers: Findings section tends to be longer than in quantitative papers because of the demonstrative rhetoric needed to permit the evaluation of the analytic procedure."

"Reviewers: Depending on the approach to inquiry, findings and discussion may be combined or a personalized discursive style might be used to portray the researchers’ involvement in the analysis."

All of these suggestions reinforce the notion that qualitative research is a very different thing to quantitative research and to some extent this should be reflected in how qualitative research is written, rather than mirroring the way that quantitative research is written.

I am very encouraged, and hope we will adopt these guidelines into our teaching practice.