Critical appraisal tools are also available for appraising, and then rating, methodologically heterogeneous studies (studies that differ by design).

You may see these types of critical appraisal tools used in publications involving the synthesis of a range of study designs, for example, in an integrative review. Some of these tools use just one set of questions irrespective of study design.

The obvious downside of these 'one-size fits all' tools is that they are not as informative, or involve as comprehensive an assessment, as appraisal tools developed for specific study designs. Does that matter? We think it depends on the underlying purpose of your critical appraisal. If you are determining the quality of the evidence for an intervention it is important to undertake a robust risk of bias (quality) assessment to fully understand how trustworthy the evidence is before making a clinical practice decision. Poor quality of studies can overestimate the direction and size of any treatment effect. However, if you want a general indication of the quality of studies included in an integrative review (for example, there are many different types of reviews) when developing a conceptual framework or model, or to understand more about your topic of interest, the tools below might be OK. Scholarship is choosing the right tool for the job.

Examples of a 'one size fit all' tools are:

Hawker, S., Payne, S., Kerr, C., Hardey, M., & Powell, J. (2002). Appraising the evidence: reviewing disparate data systematically. Qualitative Health Research, 12(9), 1284-1299. doi:10.1177/1049732302238251 Full text
- Click here for the Hawker critical appraisal tool - how confident are you using this to appraising the validity of a range of study designs?

The Mixed Methods Appraisal Tool (MMAT) from MacGill University.
- MMAT is a good option when you have study designs in your review that other options (such as CASP, CEBM) do not have tools for. For example, the CASP option does not provide a tool for appraising mixed-method or quantitative descriptive studies. Descriptive studies are “concerned with and designed only to describe the existing distribution of variables without much regard to causal relationships or other hypotheses” (Porta et al., 2014, p. 72).
- Click Here for Version 2018

Caldwell, K., Henshaw, L., & Taylor, G. (2011). Developing a framework for critiquing health research: an early evaluation. Nurse education today, 31(8), e1-e7. Click here for link
- This tool was recently used in this article exploring Culturally sensitive communication at the end-of-life in the intensive care unit

The John Hopkins Tool:
- Available via this link https://www.hopkinsmedicine.org/evidence-based-practice/model-tools
- The tools are free, but you will need to request permission to download them.
- Critical appraisal tools for Research Evidence are in Appendix E
- Critical appraisal tools for Nonresearch Evidence are in Appendix F
- Work your way carefully through the list of questions when using these tools to ensure you select appraisal questions that match your study design.
- Citation for tools: Dang, D., Dearholt, S., Bissett, K., Ascenzi, J., & Whalen, M. (2022). Johns Hopkins evidence-based practice for nurses and healthcare professionals: Model and guidelines. 4th ed. Sigma Theta Tau International https://www.hopkinsmedicine.org/evidence-based-practice/model-tools

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Reference: Porta, M. S., Greenland, S., Hernán, M., dos Santos Silva, I., Last, J. M. (2014). A dictionary of epidemiology. New York: Oxford University Press.