#OTalk Research Tuesday 3rd July: Face validity: what is it, why is it important and how do we evaluate it?

This week’s chat is on the topic of Face validity and will be hosted by Dr Alison Laver-Fawcett, Associate Professor at York St John University (@alisonlaverfaw).  Here is what Alison had to say…

Face validity is the extent an assessment subjectively appears to test what it is supposed to; good face validity ensures an assessment is client-centred, acceptable to the test-taker, and to the person administering it (Asher 2007).

I conducted my first face validity study around 1990-1 when I was undertaking a project to develop, standardise and evaluate the psychometric properties of an assessment, the Structured Observational Test of Function (SOTOF), as the focus for my PhD studies. At the time I struggled to find face validity reported and discussed in occupational therapy literature. A key psychometric text I was drawing on at the time by Anastasi (1988) also had noted there was a “paucity of available research on face validity, despite its probable contribution to prevalent attitudes towards tests” (p. 145). The COSMIN checklist manual (Mokkink et al., 2012: 31) stated that no standards were developed for assessing face validity because ‘face validity requires a subjective judgement’, so unlike other types of validity and reliability, there is a lack of agreed standards for face validity studies

Years later when writing about validity for a text book I found there was still a lack of face validity studies published, not just in occupational therapy but also wider allied health assessment literature. This seems counter-intuitive as occupational therapists are supposed to be client centred; so why aren’t we studying the face validity of occupational therapy assessments and outcome measures as a matter of routine? Do we really think that it doesn’t matter what our clients’ experiences of undertaking an assessment is? Or what they think about what is being assessed /measured and how the assessment is done?

In the last few years I have been undertaking work with occupational therapy students exploring the face validity of a couple of measures. For example, in the final year ‘Dissertation: Contributing to the Evidence base’ module on the Occupational Therapy Programme at York St John University, some small groups of students have been collaborating to undertake face validity studies on the Activity Card Sort – United Kingdom version (e.g. see Laver-Fawcett et al, 2016). This year students have explored the face validity of the SOTOF (2nd edition) with community living older people and a MSc by Research student is exploring the face validity of SOTOF (2nd ed) with people in an in-patient setting who have neurological conditions such as stroke.

In this chat we will explore what face validity is and how it is defined; we will debate whether it is important for occupational therapy researchers to consider face validity (both when developing and evaluating measures and when selecting outcome measures for research); and we will discuss methodology for evaluating and exploring face validity.

Whether you are an experienced researcher, a clinician or a student please join us on 2nd May for this #OTalk twitter chat and share your ideas and experience. It is never too early in your occupational therapy career to start engaging in the development and evaluation of occupational therapy assessments and outcome measures.

Suggested talking points and discussion questions to focus our chat:

  1. What do you understand by the term ‘face validity’? Do you have any definitions that you have found useful?
  2. Why is face validity important to study when developing or selecting occupational therapy assessments and outcome measures?
  3. When selecting an outcome measure for research how could you consider face validity?
  4. Have you ever undertaken research to evaluate the face validity of an assessment or outcome measure? How did you do this?
  5. What methodologies can be used to explore and evaluate face validity?

References

Asher I.E. (2007) Occupational Therapy Assessment Tools: An annotated index. 3rd ed. Bethesda, American Occupational Therapy Association.

Laver-Fawcett A J, Brain L, Brodie C, Cardy L, Manaton L (2016) The Face Validity and Clinical Utility of the Activity Card Sort – United Kingdom (ACS-UK). British Journal of Occupational Therapy, 79(8) 492–504. doi:10.1177/0308022616629167. Available from: http://journals.sagepub.com/doi/abs/10.1177/0308022616629167 (accessed 25th June 2018).

Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, Bouter LM and de Vet HCW (2012) COSMIN checklist manual. Available at: http://www.cosmin.nl/images/upload/files/COSMIN%20checklist%20manual%20v9.pdf (accessed 25 June 2018).

Post chat

Chat host : Dr Alison Laver-Fawcett @alisonlaverfaw

On OTalk account for support: Dr Jenny Preston @preston_jenny

Online transcript

#OTalk Healthcare Social Media Transcript July 3rd 2018

The Numbers

1.300MImpressions
316Tweets
22Participants
253Avg Tweets/Hour
14Avg Tweets/Participant

#OTalk Participants

Advertisements

#OTalk 26th June 2018- Mental Capacity Act and Deprivation of Liberty Safeguards.

This weeks chat is on the topic of the Mental Capacity Act (MCA) and The Deprivation of Liberty Safeguards (DoLs) and will be hosted by Sarah Sharland (@SarahSharland19).  Here is what Sarah had to say…

The Mental Capacity Act 2005 came into force in 2007. It was designed to protect those who were deemed vulnerable or who may have an impaired ability to make decisions for any reason that affects the functioning of their minds. It can also be used to support those who do have capacity and are planning for their future care needs. Everyone working within the MCA should feel confident in applying the principles and supporting people to make decisions for themselves. Should the person be assessed as lacking capacity, then any decision regarding their care should be made in their best interests.

The Deprivation of Liberty Safeguards (DOLS) were brought in as a result of a Supreme Court ruling, as an amendment to the MCA 2005, and apply in only England and Wales. They ensure that when restraints and restrictions are used under the Act, that the person is protected and represented. They also ensure that any implementations are the least restrictive possible and are regularly reviewed. DOLS can only be used if the person is being deprived of their liberty in a care home or hospital. As part of the legislation, the person must have an advocate or representative, and they also have the ability to challenge or appeal the authorisations within the legal system.

Questions for the OTalk chat:

  1. How does the Mental Capacity Act affect your practice?
  2. Are you involved in mental capacity assessments, and what is your role/setting?
  3. How confident do you feel in your knowledge of the Mental Capacity Act? Please give reasons for this (training, experience etc)
  4. How confident do you feel in your knowledge of the Deprivation of Liberty Safeguards? Please give reasons for this (training, experience etc)
  5. What do you think are the benefits of involving occupational therapists in MCA and DOLS assessments?

POST CHAT

Hosting the chat: Sarah Sharland @SarahSharland19

Support on @OTalk account Rachel @OT_rach

Online Tanscript

#OTalk Healthcare Social Media Transcript June 26th 2018

The Numbers

935.396K Impressions
250 Tweets
24 Participants
200 Avg Tweets/Hour
10 Avg Tweets/Participant

#OTalk Participants

#OTalk 19th June 2018 – Can you meet the RCOT strategic Intentions and keep occupation at the focus of your practise?

This week our own Rachel Booth @OT_rach will host the chat, following on from a pledge she made at this years Royal College of Occupational Therapists (RCOT) annual conference. #RCOT2018

During the conference we were asked to make a pledge about how we would do our part to ensure the new strategic intentions are met.

RCOT’s Strategic Intentions represent their leadership response to the challenges of the changing landscape of health and social care.

They position occupational therapy as a key contributor to the health and wellbeing of UK citizens and provide a framework for how the Royal College will support its staff, members and the wider profession.

Strategic intention 1

Position the profession, and our members, for the 21st century

Strategic intention 2

Enhance the profile of the profession to a range of audiences

Strategic intention 3

Ensure RCOT is a thriving membership organisation within which members flourish. 

You can watch the launch video here. https://www.youtube.com/watch?reload=9&v=bFF5PoVngKM

One of the things that struck me about lots of the pledges and the conference its self was the idea we need to ensure ‘occupation’ is central to our work.  I’m guilty of losing the occupation as I concentrate on all the daily tasks we are told we need to compete, and want to learn from others as to how to reintroduce this focus. 

We are the experts in ‘occupation’ and need to ensure that our practise holds this at its heart. My pledge was to host a #OTalk on what do we really mean by occupation? Hence tonights chat. 

The Questions 

Question 1 : Before tonight’s chat had you heard of RCOT’S strategic intentions? If so what are your thoughts? How do you think they could be met?

Question 2: What is you’re understanding of ‘occupation’ and it’s meaning with occupational therapy?

Question 3 How do you describe occupation to your colleagues and users of your service?

Question 4: How do you or can you ensure you are ‘occupation’ focused in your interventions?

Question 5: What if anything will you do differently to put ‘occupation’ at the heart of what you do?

Question 6: What is your pledge to meet the RCOT strategic intentions?

Post Chat

Hosting the chat Rachel Booth @OT_rach

On the @OTalk_ account for support Kirstie @kirstie_OT

Online Transcript

#OTalk Healthcare Social Media Transcript June 19th 2018

The Numbers

1.843M Impressions
516 Tweets
45 Participants
413 Avg Tweets/Hour
11 Avg Tweets/Participant

#OTalk Participants

 

RCOT 2018 Final Blog 20. Thank You

This is a short final blog with a very important message……

“Thank You” to all of the members of the blog squad who have given up some of their valuable conference time to provide personal insights into some of the sessions they attended and some of their conference experiences.

You can find out who they are and a little bit about them here. But for now lets thank

  • Marie Baistow
  • Elspeth Clark
  • Faye Dunford
  • Catherine Gray
  • Orla Hughes
  • Rachel Imms
  • Catherine McNulty
  • Amie Mowlam-Tette
  • Cathy Roberts

Everyone has included their personal Twitter handle at the end of the blogs they have written if you want to make contact.

It’s not easy to try to distil the essence of conference sessions, especially some of the main keynotes, but, having read them all, they have done a fabulous job.

Not only have they been writing away but also over coming a range of different technology challenges, batteries going flat, computers not connecting to wifi, photos not sending etc etc.

So on behalf of us all Thanks Guys – you’ve done a great job.

Written by Lynne Goodacre @lynnegoodacre

RCOT 2018 Blog 19  Sess. 57: Degree Level Apprenticeships for occupational therapy – a way forward for our profession?

This session was led by Dr Theresa  Baxter Sheffield Hallam University, Sue Waters Coventry University, Anita Cooper United Lincolnshire Health NHS Trust.

The aim of the seminar was to explore the development of degree apprenticeships in occupational therapy. A new development for the profession, and with the loss of commissioning for health courses (HEE 2016), one which may prove to be an opportunity to secure a workforce for the future.

The seminar began with Anita Cooper greeting the audience with ‘Hello my name is Anita’  she continued to introduce Theresa Baxter from Sheffield Hallam University and Sue Walters from Coventry University. They have all been working with the trailblazer group on developing the occupational therapy degree level apprenticeship.

Anita spoke of the history and the initial driving force behind this innovative opportunity. This included:

  • the loss of commissioning / education bursaries
  • reduction in students applying to universities to take up occupational therapy training programmes
  • concerns for Lincolnshire a large rural county where recruitment has historically been problematic
  • needing to make decisions for the future provision of occupational therapy services in Lincolnshire
  • the apprenticeship levy means all employers over a certain size will pay 1% Apprenticeship Levy. This includes all NHS Trusts. The only way they can benefit from the levy is to employ apprentices which subsequently enables the employer to draw down from those funds.

A link was drawn to Nick Pollard’s comments from his Elizabeth Casson Memorial Lecture of the need grow and promote diversity within profession. Developing the apprenticeship model would essentially enable people who would otherwise not be able to undertake their occupational therapy training to do so.

I can directly empathise with this situation. Employed as an occupational therapy support worker in Lincolnshire NHS mental health services I had the opportunity to undertake my occupational therapy degree as a part-time student with University of Ripon and York St John (known now as the University York St John). For me this was totally life changing.

Post graduation, I continued to work in Lincolnshire NHS.  Currently I am employed as Associate Lecturer with Sheffield Hallam University and now completing my MSc. Over the last 11years I have worked with the Sheffield Hallam University Lincolnshire based in Grantham, Practice Based Learning (PBL) occupational therapy students, (there is also a physiotherapy PBL programme). The programme is coming to an end, due to loss of commissioning / bursaries etc., as noted earlier. Students on the programme have come from a wide range of mostly support worker backgrounds and bring a rich diversity of knowledge skills and experience, which is of great benefit to both occupational therapy and physiotherapy services for Lincolnshire and beyond.

Anita mentioned working with Lincolnshire talent academy. She spoke of having much enthusiasm for such a proposal and the opportunities it could offer the profession. On a humourus note Anita  mentioned naivety, as initially she didn’t realise that this would lead to the national development for the profession and the amount of work this would involve. It is very clear to see her passion and commitment to developing the apprenticeship programme.

Theresa went on to talk about the work of the trail blazer group and how they set about making sense of a very new way of developing a teaching programme that would fit with the requirements the Institute For Apprenticeships (IFA), Health Care Professions Council (HCPC), Royal College Occupational Therapy (RCOT) and College Physiotherapy and also the employers of such apprentices, this included needing to gain the support of numerous stakeholders who would be committed to such a programme. This task is simply awe inspiring, I can only think that Theresa, Sue,  Anita and other members of the trail blazer group must have used every ounce of knowledge, skills, expertise and creative thinking in their combined tool bags to bring this amazing apprenticeship programme together.

A number of challenges have included needing to hold on to the word of occupation within the document. The IFA had insisted it should state activity. Standing firm, the word occupation (core to profession of occupational therapy) was finally accepted and is now an agreed term, part of the outline standards document. No mean feat as this is strictly limited to three A4 sides and 12 point Font. No room for squeezing in extra phrases or words.

Careful on-line consultation with subsequent meticulous analysis of all returning comments gave further amendments to the standards documents.

The Apprenticeship standards documents critically is required to map all the standards of proficiency as stated by HCPC.

The IFA finally accepted the standards document on 23 May 2018. This is very hot off the press news and an amazing achievement in a short time scale.

This degree level apprenticeship for occupational therapy essentially is the same as BSc Hons degree with HCPC accreditation and RCOT. It is 360 credits

The end point assessment EPI is approached differently to meet the specific requirements of IFA. It is the final assessment as it obviously states, with specific modes to ensure it is approved under IFA.  The trail blazer group have needed to discuss and carefully negotiate to make all of this come together.

Small group discussions around the potential benefits and challenges of such an apprenticeship for occupational therapy were encouraged and this was followed by opportunity for questions to be put to Theresa, Anita and Sue.

Some of the topics covered:

  • Apprentices will be earning a wage while they are studying. Therefore this enables people from variety of backgrounds to apply and importantly without the burden of the usual student debt. Getting paid is a big plus.
  • Student experience, it may well be different from other occupational therapy programmes but there are already a range of programmes such as full time BSc Hons, Pre-reg MSc, part-time degree courses, this can be seen as just another route which widens accessibility for occupational therapy training. Variety in training programs means different opportunities and will only strengthen diversity for the profession. This apprenticeship programme will have its own benefits and challenges for students. Those students who wish to choose a course which also enables what may be called the student experience may well choose a full time programme.
  • Loyalty of graduates to their initial employers. This was explored at length and reminded me of similar discussions about the PBL programme The reality agreed is that the PBL graduates, the majority have stayed in Lincolnshire and continue to work in occupational therapy services. It is expected that the apprenticeship graduates will follow a similar path.
  • Student Practice Placements, will there be enough out there for them? A resounding yes!! Plenty of capacity.

This is a fantastic opportunity for the occupational therapy profession, physiotherapy will be also taking this opportunity. It is one brilliant way forward to grow the occupational therapy profession, to promote diversity in our students, new graduates, future experienced occupational therapists and educators.

I reflect now again on my own occupational therapy 4yr part time degree, I was so excited and full of enthusiasm, hungry for my learning opportunity. I think of the people I have worked with over many years and now as Associate Lecturer I can continue to influence the next generation. I think that I have proved my worth along the way and will continue to do so, a good return on the initial investment

All the best for the occupational therapy Apprenticeship Degree Programme !! And finally we owe thanks to the Trail Blazer Team.

This blog has been written by Catherine McNulty find me on Twitter @cathymc9781

RCOT2018 Blog 18 #IAMchallengingbehaviour: We all have challenging behaviour, let’s challenge the labels in ‘serviceland’

Sam Sly (@SamSly2), the RCOT-people-with-learning-disabilities-specialist-section’s keynote speaker, shared about the social media campaign #IAmChallengingBehaviour.

Enough is enough, it’s time for change

The campaign starts the conversation that people with learning disabilities are being labelled as having ‘challenging behaviour’ unfairly, leading to extended time spent in hospitals and institutions. This can result in people being prevented from living life to their full potential.

Sam began by highlighting that we all display challenging behaviour in life’s tough times. The difference is that as ‘valuable’ members of society, instead it is said that we are angry, frustrated, or simply ‘having a bad day.’ Therefore, the campaign began as ‘I HAVE challenging behaviour’ but soon changed to ‘I AM challenging behaviour’ as this added action to the movement. It stated that it was time to stand against the unjust labels being put on people with learning disabilities when they express human emotions such as anxiety or fear.

Instead, Let’s talk in….

  1. …normal words:

Sam’s address called on us to consider our language in the realm she calls ‘serviceland.’ This is a term used for the health professionals and spaces that work with people with learning disabilities. Science jargon has replaced human words which can misrepresent and undervalue people with learning disabilities. For example, she discussed the use of the phrase ‘finding the client a placement’ which sounds like a temporary and uncertain place. Why don’t we help a person find a home instead?

  1. …human rights:

Today’s reality is that males and females living with a learning disability in the UK have a life expectancy 14 and 18 years less than the general population respectively. To put this in perspective, it is the life expectancy of developing countries for people living in one of the richest countries in the world.

The badges

5,000 badges with the campaign’s message have been distributed to date which has transported the conversation from social media to MDT meetings and local shops. Gold badges have also been awarded to people going above and beyond to challenge the language we use in practice and everyday life.

Back in the real world

I returned to my placement today and saw in my upcoming training that one module is titled ‘challenging behaviour.’ I’ll make sure to wear my badge loud and proud that day especially 😉

The next step

Make sure to join the campaign by tweeting/facebooking #IAMchallengingbehaviour and think about who you could nominate for a gold badge.

Written by Orla Hughes (@orlatheot)

RCOT 2018 Blog 17. Sess.87 Education

The two papers presented in this session truly complimented each other, stimulated interesting questions, and resulted in a positive energy amongst delegates for future improvements.

87.1 What students think is best practice when teaching them clinical reasoning in practice education, by Caroline Hills. A study of 3rdand 4thyear Australian students using a qualitative descriptive method.

Caroline, who moved from Australia to Ireland has completed this for her PhD study. She gave a definition of clinical reasoning, then expressed that there is no consensus about the best way to teach it to OT students. The overarching aim was to do a mixed methodology study to investigate learning preferences of OT students whilst on their clinical placements, what was the best thing about being on placement ?. Out of this came an important subtheme of teaching clinical reasoning. Caroline suggested that clinicians are often in a rush to get to the end-point when explaining to students, and that this can result in a lack of deep learning. The semi-structured face to face, Skype and telephone interviews were analysed thematically.

Three main themes resulted.

  1. Talk it through: students wanted the educator to take the time to explain the point, break it down to their level and explain possible alternatives.
  2. How to develop my reasoning; encourage me to think and apply theory, ask me a question to check I have understood.
  3. Preferred teaching and learning approaches for clinical reasoning; make it two way, don’t put me on the spot and give me time to reflect.

The conclusions drawn were that clinical reasoning must be a two way process that this should be graded to aid learning, and that there is a need for a clinical reasoning template. Instead of concentrating on being process led, we will do an initial assessment, then functional assessment, instead it is arguably better to discuss the reasons why each step happens. Another key theme that was discovered, was that concept of belongingness; students value feeling that they belong whilst on placement. There is a lot of evidence within medicine and nursing about this, more is needed within OT.

87.2. Using Q-methodology to identify the factors influencing occupational therapy practice educators offering placements to undergraduate OT students, by Jenny Devers from University of Northampton

Jenny introduces this by saying that practice placements are essential within OT training programs. However, there are rising concerns about placement shortages so there is needs to be a call to find strategies and solutions. Jenny looked at the number of qualified OT’s in practice (38000) and considered the number of OT students (5,500 per year) so felt that really there shouldn’t be a problem, but yet there is, so she was keen to find out why.

A comprehensive literature search gave a range of positive rationale, including that students are energising and a positive influence, but yet there are many negative influences also. Ethics was gained and a range of sites deemed suitable, 16 eligible sites were secured and sent the pack to complete. Participants answered all the questions according to the scale, and they were asked to give comments for the questions that were answered at each end of the scale (most agreed / most disagreed). The data was then transferred back into the matrix. Analysis revealed 3 main themes; strong professional value, student impact and placement support. The recommendations were

Screen Shot 2018-06-14 at 20.43.20

There was an interesting debate in the room with considerations for future improvements, with a key theme of universities and placement providers collaborating closely to build the future workforce.

Written by @imms_eh_OT