Critical appraisal tools are also available for appraising, and then rating, methodologically heterogeneous studies (studies that differ by design).

You may see these types of critical appraisal tools used in publications involving the synthesis of a range of study designs, for example, in an integrative review. Some of these tools use just one set of questions irrespective of study design.

The obvious downside of these 'one-size fits all' tools is that they are not as informative, or involve as comprehensive an assessment, as appraisal tools developed for specific study designs. Does that matter? We think it depends on the underlying purpose of your critical appraisal. If you are determining the quality of the evidence for an intervention it is important to undertake a robust risk of bias (quality) assessment to fully understand how trustworthy the evidence is before making a clinical practice decision. Poor quality of studies can overestimate the direction and size of any treatment effect. However, if you want a general indication of the quality of studies included in an integrative review (for example, there are many different types of reviews) when developing a conceptual framework or model, or to understand more about your topic of interest, the tools below might be OK. Scholarship is choosing the right tool for the job.

Examples of a 'one size fit all' tools are:

Hawker, S., Payne, S., Kerr, C., Hardey, M., & Powell, J. (2002). Appraising the evidence: reviewing disparate data systematically. Qualitative Health Research, 12(9), 1284-1299. doi:10.1177/1049732302238251 Full text
- Click here for the Hawker critical appraisal tool - how confident are you using this to appraising the validity of a range of study designs?

The Mixed Methods Appraisal Tool (MMAT) from MacGill University.
- MMAT is a good option when you have study designs in your review that other options (such as CASP, CEBM) do not have tools for. For example, the CASP option does not provide a tool for appraising mixed-method or quantitative descriptive studies. Descriptive studies are “concerned with and designed only to describe the existing distribution of variables without much regard to causal relationships or other hypotheses” (Porta et al., 2014, p. 72).
- Click Here for Version 2018

Caldwell, K., Henshaw, L., & Taylor, G. (2011). Developing a framework for critiquing health research: an early evaluation. Nurse education today, 31(8), e1-e7. Click here for link
- This tool was recently used in this article exploring Culturally sensitive communication at the end-of-life in the intensive care unit

The John Hopkins Tool:
- The John Hopkins tool for Research Evidence is a reasonable option for study designs that other appraisal tools (CASP, CEBM for example) do not apply to, such as meta-synthesis studies or surveys - click Here (2022 version) Work your way carefully through the list of questions when using this tool to ensure you select appraisal questions that match your study design.
- For completeness, I have included here the link to the John Hopkins tool for appraising Non-research Evidence - click Here Download Here Note that your evidence table should only include research evidence but the prompts may be a useful way for you to think through the adequacy of service clinical practice guidelines, position statements, integrative revies, program evaluations as you write Part A of assignment one.
- Citation for tools: Dang, D., Dearholt, S., Bissett, K., Ascenzi, J., & Whalen, M. (2022). Johns Hopkins evidence-based practice for nurses and healthcare professionals: Model and guidelines. 4th ed. Sigma Theta Tau International https://www.hopkinsmedicine.org/evidence-based-practice/ijhn_2017_ebp.htmlLinks to an external site.

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Reference: Porta, M. S., Greenland, S., Hernán, M., dos Santos Silva, I., Last, J. M. (2014). A dictionary of epidemiology. New York: Oxford University Press.