{"id":{"repo_id":"soton","oai_identifier":"oai:eprints.soton.ac.uk:67475"},"canonical_url":"https://search.dev.ndltd.org/etd/soton/oai:eprints.soton.ac.uk:67475","repository":{"repo_id":"soton","name":"University of Southampton","base_url":"https://eprints.soton.ac.uk/cgi/oai2"},"display":{"title":"Assessing quality in systematic reviews of the effectiveness of health promotion: areas of consensus and dissension","abstract":"Systematic reviews have played an increasingly important role in health promotion in recent
years. Yet there are debates about how they should be conducted, particularly about how the
quality of evidence should be assessed. The aim of this research was to assess current
approaches to, and general views on, the use of quality assessment in systematic reviews of
effectiveness in health promotion, and to identify areas of consensus and dissension around the
choice of techniques, methods and criteria employed.
There were two stages of data collection. The first was a structured mapping of a random
sample of 30 systematic reviews of the effectiveness of health promotion to identify and explain
trends and themes in methods and approaches to quality assessment. During the second stage
semi-structured interviews were conducted with a purposive sample of 17 systematic reviewers
who had conducted at least one review of a health promotion topic, to investigate some of these
trends and approaches in greater detail.
The mapping found that the majority of systematic reviews had assessed the quality of the
included studies, to varying degrees. However, procedures were not always explicitly reported
or consistent. There was some degree of consensus over criteria, with experimental evaluation
methods commonly favoured. Most frequently used quality assessment criteria included
participant attrition, the validity and reliability of data collection and analysis methods, and
adequacy of sample sizes. External validity was commonly assessed, primarily in terms of
generalisability and replicability, but less so in terms of intervention quality.
The interviews revealed some of the barriers to effective systematic reviewing, including: lack
of time and resources, complexity of some health promotion interventions, inclusion of
observational evaluation designs, and poor reporting of primary studies. Systematic reviewing
was commonly done in small teams, mostly comprising academics, sometimes with
practitioners. Interviewees learned systematic review skills through a combination of training,
support from colleagues and mentors, literature and a strong emphasis on hands-on practical
learning. Subjective judgement was often required, contra to the popular belief that systematic
reviews are wholly objective.
The overall conclusions of this study are that systematic reviewing in health promotion is often
challenging due the complexity of interventions and evaluation designs. This places additional
demands on reviewers in terms of knowledge and skills required, often exacerbated by finite
time scales and limited funding. Initiatives are in place to foster shared ways of working,
although the extent to which complete consensus is achievable in a multi- disciplinary area such
as health promotion is questionable.","abstract_html":"Systematic reviews have played an increasingly important role in health promotion in recent<br/>years. Yet there are debates about how they should be conducted, particularly about how the<br/>quality of evidence should be assessed. The aim of this research was to assess current<br/>approaches to, and general views on, the use of quality assessment in systematic reviews of<br/>effectiveness in health promotion, and to identify areas of consensus and dissension around the<br/>choice of techniques, methods and criteria employed.<br/><br/>There were two stages of data collection. The first was a structured mapping of a random<br/>sample of 30 systematic reviews of the effectiveness of health promotion to identify and explain<br/>trends and themes in methods and approaches to quality assessment. During the second stage<br/>semi-structured interviews were conducted with a purposive sample of 17 systematic reviewers<br/>who had conducted at least one review of a health promotion topic, to investigate some of these<br/>trends and approaches in greater detail.<br/><br/>The mapping found that the majority of systematic reviews had assessed the quality of the<br/>included studies, to varying degrees. However, procedures were not always explicitly reported<br/>or consistent. There was some degree of consensus over criteria, with experimental evaluation<br/>methods commonly favoured. Most frequently used quality assessment criteria included<br/>participant attrition, the validity and reliability of data collection and analysis methods, and<br/>adequacy of sample sizes. External validity was commonly assessed, primarily in terms of<br/>generalisability and replicability, but less so in terms of intervention quality.<br/><br/>The interviews revealed some of the barriers to effective systematic reviewing, including: lack<br/>of time and resources, complexity of some health promotion interventions, inclusion of<br/>observational evaluation designs, and poor reporting of primary studies. Systematic reviewing<br/>was commonly done in small teams, mostly comprising academics, sometimes with<br/>practitioners. Interviewees learned systematic review skills through a combination of training,<br/>support from colleagues and mentors, literature and a strong emphasis on hands-on practical<br/>learning. Subjective judgement was often required, contra to the popular belief that systematic<br/>reviews are wholly objective.<br/><br/>The overall conclusions of this study are that systematic reviewing in health promotion is often<br/>challenging due the complexity of interventions and evaluation designs. This places additional<br/>demands on reviewers in terms of knowledge and skills required, often exacerbated by finite<br/>time scales and limited funding. Initiatives are in place to foster shared ways of working,<br/>although the extent to which complete consensus is achievable in a multi- disciplinary area such<br/>as health promotion is questionable.","abstract_has_math":false,"creators":["Shepherd, Jonathan Paul"],"institution":"University of Southampton","degree_name":"Ph.D.","degree_level":"doctoral","degree_discipline":null,"degree_department":null,"school":null,"contributors":[],"advisors":["Weare, Katherine"],"committee_chairs":[],"committee_members":[],"year":2009,"date_issued":"2009-06","date_published":"2009-06","updated_at":"2026-07-24T04:36:02Z","subjects":[],"languages":[],"rights":[],"rights_urls":[],"identifier_entries":[]},"links":{"outbound_url":null,"outbound_label":null,"outbound_source":null},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.advisor","label":"Advisor","values":["Weare, Katherine"]},{"key":"dc:creator","label":"Author","values":["Shepherd, Jonathan Paul"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2009-06"]},{"key":"dc:date.issued","label":"Date","values":["2009-06"]},{"key":"dc:publisher.department","label":"Dc Publisher Department","values":["Education (pre 2011 reorg)","School of Education"]},{"key":"dc:publisher.institution","label":"Dc Publisher Institution","values":["University of Southampton"]},{"key":"dc:relation.isreferencedby","label":"Dc Relation Isreferencedby","values":["https://eprints.soton.ac.uk/67475/"]},{"key":"dc:type","label":"Dc Type","values":["Thesis"]},{"key":"dc:type.qualificationlevel","label":"Dc Type Qualificationlevel","values":["doctoral"]},{"key":"dc:type.qualificationname","label":"Dc Type Qualificationname","values":["Ph.D."]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://eprints.soton.ac.uk/67475/1/Microsoft_Word_-_J_Shepherd_PhD_Thesis.pdf"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["Systematic reviews have played an increasingly important role in health promotion in recent
years. Yet there are debates about how they should be conducted, particularly about how the
quality of evidence should be assessed. The aim of this research was to assess current
approaches to, and general views on, the use of quality assessment in systematic reviews of
effectiveness in health promotion, and to identify areas of consensus and dissension around the
choice of techniques, methods and criteria employed.
There were two stages of data collection. The first was a structured mapping of a random
sample of 30 systematic reviews of the effectiveness of health promotion to identify and explain
trends and themes in methods and approaches to quality assessment. During the second stage
semi-structured interviews were conducted with a purposive sample of 17 systematic reviewers
who had conducted at least one review of a health promotion topic, to investigate some of these
trends and approaches in greater detail.
The mapping found that the majority of systematic reviews had assessed the quality of the
included studies, to varying degrees. However, procedures were not always explicitly reported
or consistent. There was some degree of consensus over criteria, with experimental evaluation
methods commonly favoured. Most frequently used quality assessment criteria included
participant attrition, the validity and reliability of data collection and analysis methods, and
adequacy of sample sizes. External validity was commonly assessed, primarily in terms of
generalisability and replicability, but less so in terms of intervention quality.
The interviews revealed some of the barriers to effective systematic reviewing, including: lack
of time and resources, complexity of some health promotion interventions, inclusion of
observational evaluation designs, and poor reporting of primary studies. Systematic reviewing
was commonly done in small teams, mostly comprising academics, sometimes with
practitioners. Interviewees learned systematic review skills through a combination of training,
support from colleagues and mentors, literature and a strong emphasis on hands-on practical
learning. Subjective judgement was often required, contra to the popular belief that systematic
reviews are wholly objective.
The overall conclusions of this study are that systematic reviewing in health promotion is often
challenging due the complexity of interventions and evaluation designs. This places additional
demands on reviewers in terms of knowledge and skills required, often exacerbated by finite
time scales and limited funding. Initiatives are in place to foster shared ways of working,
although the extent to which complete consensus is achievable in a multi- disciplinary area such
as health promotion is questionable."]},{"key":"dc:format","label":"Dc Format","values":["text"]},{"key":"dc:title","label":"Title","values":["Assessing quality in systematic reviews of the effectiveness of health promotion: areas of consensus and dissension"]}]}],"canonical_facts":{"dc:contributor.advisor":["Weare, Katherine"],"dc:creator":["Shepherd, Jonathan Paul"],"dc:date":["2009-06"],"dc:date.issued":["2009-06"],"dc:description.abstract":["Systematic reviews have played an increasingly important role in health promotion in recent
years. Yet there are debates about how they should be conducted, particularly about how the
quality of evidence should be assessed. The aim of this research was to assess current
approaches to, and general views on, the use of quality assessment in systematic reviews of
effectiveness in health promotion, and to identify areas of consensus and dissension around the
choice of techniques, methods and criteria employed.
There were two stages of data collection. The first was a structured mapping of a random
sample of 30 systematic reviews of the effectiveness of health promotion to identify and explain
trends and themes in methods and approaches to quality assessment. During the second stage
semi-structured interviews were conducted with a purposive sample of 17 systematic reviewers
who had conducted at least one review of a health promotion topic, to investigate some of these
trends and approaches in greater detail.
The mapping found that the majority of systematic reviews had assessed the quality of the
included studies, to varying degrees. However, procedures were not always explicitly reported
or consistent. There was some degree of consensus over criteria, with experimental evaluation
methods commonly favoured. Most frequently used quality assessment criteria included
participant attrition, the validity and reliability of data collection and analysis methods, and
adequacy of sample sizes. External validity was commonly assessed, primarily in terms of
generalisability and replicability, but less so in terms of intervention quality.
The interviews revealed some of the barriers to effective systematic reviewing, including: lack
of time and resources, complexity of some health promotion interventions, inclusion of
observational evaluation designs, and poor reporting of primary studies. Systematic reviewing
was commonly done in small teams, mostly comprising academics, sometimes with
practitioners. Interviewees learned systematic review skills through a combination of training,
support from colleagues and mentors, literature and a strong emphasis on hands-on practical
learning. Subjective judgement was often required, contra to the popular belief that systematic
reviews are wholly objective.
The overall conclusions of this study are that systematic reviewing in health promotion is often
challenging due the complexity of interventions and evaluation designs. This places additional
demands on reviewers in terms of knowledge and skills required, often exacerbated by finite
time scales and limited funding. Initiatives are in place to foster shared ways of working,
although the extent to which complete consensus is achievable in a multi- disciplinary area such
as health promotion is questionable."],"dc:format":["text"],"dc:identifier.uri":["https://eprints.soton.ac.uk/67475/1/Microsoft_Word_-_J_Shepherd_PhD_Thesis.pdf"],"dc:publisher.department":["Education (pre 2011 reorg)","School of Education"],"dc:publisher.institution":["University of Southampton"],"dc:relation.isreferencedby":["https://eprints.soton.ac.uk/67475/"],"dc:title":["Assessing quality in systematic reviews of the effectiveness of health promotion: areas of consensus and dissension"],"dc:type":["Thesis"],"dc:type.qualificationlevel":["doctoral"],"dc:type.qualificationname":["Ph.D."]},"updated_at":"2026-07-24T04:36:02Z"}