- Split View
-
Views
-
Cite
Cite
Aline C. Stolk-Vos, Joris J. van de Klundert, Niels Maijers, Bart L.M. Zijlmans, Jan J.V. Busschbach, Multi-stakeholder perspectives in defining health-services quality in cataract care, International Journal for Quality in Health Care, Volume 29, Issue 4, August 2017, Pages 470–476, https://doi.org/10.1093/intqhc/mzx048
- Share Icon Share
Abstract
To develop a method to define a multi-stakeholder perspective on health-service quality that enables the expression of differences in systematically identified stakeholders’ perspectives, and to pilot the approach for cataract care.
Mixed-method study between 2014 and 2015.
Cataract care in the Netherlands.
Stakeholder representatives.
We first identified and classified stakeholders using stakeholder theory. Participants established a multi-stakeholder perspective on quality of cataract care using concept mapping, this yielded a cluster map based on multivariate statistical analyses. Consensus-based quality dimensions were subsequently defined in a plenary stakeholder session.
Stakeholders and multi-stakeholder perspective on health-service quality.
Our analysis identified seven definitive stakeholders, as follows: the Dutch Ophthalmology Society, ophthalmologists, general practitioners, optometrists, health insurers, hospitals and private clinics. Patients, as dependent stakeholders, were considered to lack power by other stakeholders; hence, they were not classified as definitive stakeholders. Overall, 18 stakeholders representing ophthalmologists, general practitioners, optometrists, health insurers, hospitals, private clinics, patients, patient federations and the Dutch Healthcare Institute sorted 125 systematically collected indicators into the seven following clusters: patient centeredness and accessibility, interpersonal conduct and expectations, experienced outcome, clinical outcome, process and structure, medical technical acting and safety. Importance scores from stakeholders directly involved in the cataract service delivery process correlated strongly, as did scores from stakeholders not directly involved in this process.
Using a case study on cataract care, the proposed methods enable different views among stakeholders concerning quality dimensions to be systematically revealed, and the stakeholders jointly agreed on these dimensions. The methods helped to unify different quality definitions and facilitated operationalisation of quality measurement in a way that was accepted by relevant stakeholders.
Introduction
While many definitions of health-service quality have relied on objectively defined descriptions, others have argued that it should be considered subjectively [1]. Subjective definitions may be based, for instance, on expectations, perceptions, demands and values [2, 3]. Various health-service stakeholders may hold different views on health-service quality and perceive different defining dimensions and corresponding indicators [4]. In the Netherlands, this situation is illustrated by the distinct quality-measurement systems of professional societies, insurers and patient federations concerning cataract care [5–7]. The absence of an overall consensus in defining quality is not limited to ophthalmology; rather, it applies to many other fields, such as oncology [8, 9], urology [10] and geriatric care [11]. Dissent among stakeholders may complicate the measurement and improvement of health-service quality [12].
The importance of stakeholder involvement in indicator development has been increasingly recognized [13]. Delnoij et al. [14] concluded that stakeholder involvement is the only way to balance information needs, increase consensus and benefit from transparency. However, scientific literature offers few methods to systematically identify and involve relevant stakeholders [13]. Conversely, current contributions, often involve only stakeholder subsets, resulting in different perspectives and a multitude of variables. Hence, there is a need to compress the variables into a manageable number and measurable form [15, 16]. The variation among stakeholders is especially relevant in this process, as dynamic stakeholder interactions influence the priority attached to the various quality dimensions in practice [17, 18].
The current study aimed to develop and test a method for identifying health-service stakeholders and to develop a multi-stakeholder quality perspective that accounts for differences in viewpoints among stakeholders. We present a general method to establish a framework of quality dimensions that incorporates multi-stakeholder views, without involving a priori quality definitions or dimensions but discussing them afterwards. This reveals variations in the importance attached to these dimensions. We apply the method to cataract care. This is particularly suitable as a first case study, since cataract care represents a mature and well-outlined procedure for a highly prevalent condition that possess many quality indicators (see also the corresponding conclusions).
Methods
Stakeholder investigation
We conducted this mixed-method study between 2014 and 2015. To include multiple stakeholders that are highly relevant for cataract care, we started our study with a systematic identification and classification of stakeholders based on stakeholder theory [16]. As a first step in the stakeholder selection, the researchers compiled a list of putative stakeholders in cataract care in the Netherlands based on the literature and experience.
Second, stakeholders identified by the researchers as potential participants were invited to participate in our study. Seven out of sixteen accepted the invitation, namely a general practitioner (GP), an optometrist, an ophthalmologist specialized in cataract care in a peripheral hospital, a medical advisor at the largest insurance company in the Netherlands, a doctor/advisor at the central governmental body in healthcare called the Dutch National Healthcare Institute (Zorginstituut Nederland), a director of a private clinic in ophthalmology and a cataract patient. The other potential participants refused, namely the Ministry of Health, Welfare and Sport (Ministry of VWS), a professional association of ophthalmologists called the Dutch Ophthalmic Society (NOG), the consumer association Consumentenbond, the ANBO branch organization for seniors, the Dutch Patient Federation (NPCF) and health insurers. Two of the invitees declined because they were involved in an initiative that they considered comparable. The Ministry of VWS suggested that the more-specialized Zorginstituut Nederland should represent the government as a stakeholder in our study.
Next, the seven initially involved stakeholder representatives filled out an online questionnaire to identify and classify stakeholders. This consisted of the list of putative stakeholders, proposed by the researchers as well as additional stakeholders suggested by the initially involved stakeholder representatives. The questionnaire asked each of the representatives to score the three classical stakeholder attributes—power, legitimacy and urgency—using a 4-point Likert scale (1 = not at all, 4 = absolutely) [19]. When at least half of the participants granted a putative stakeholder a score of at least 3 on an attribute, the stakeholder was viewed to possess this attribute. Following stakeholder theory, stakeholders were subsequently classified based on their attributes as definitive stakeholders (possessing power, legitimacy and urgency), dormant stakeholders (only power), discretionary stakeholders (only legitimacy), demanding stakeholders (only urgency), dominant stakeholders (lacking urgency), dangerous stakeholders (lacking legitimacy), dependent stakeholders (lacking power) and non-stakeholders [19].
Based on the obtained initial stakeholder classification, we complemented the initial group of stakeholder representatives to include additional potentially relevant stakeholders. Thus, 24 more stakeholder representatives were invited to participate, of which 11 accepted the invitation, namely a GP, an optometrist, three ophthalmologists specializing in cataract surgery (one in a general hospital and two in a specialist hospital), a medical advisor at the second largest insurance company in the Netherlands, four cataract patients, and a board member from a categorical hospital. This recruitment resulted in a final group of 19 stakeholder representatives. The representatives declining the invitation to participate (three ophthalmologists, eight patients and two health insurer employees) mentioned lack of time and health problems as the most common reasons for their refusal. Figure 1 provides a flowchart detailing the participation of stakeholder representatives at each stage of the study.
Consensus building among stakeholders
The group of 19 stakeholder representatives defined health-service quality based on concept mapping, a type of structured conceptualization that can be used by groups to represent ideas in the form of a map [20, 21]. There is no strict limit to the number of participants that should be involved in concept mapping, although the inclusion of 10–20 participants is advised [20]. The number of stakeholder representatives was in this range, and all stakeholders but one were included in the definitive, dependent, dormant or weak category. Researchers often employ interviews or Delphi-like approaches. However, concept mapping, which is designed to measure complex constructs and has a participatory nature, has several notable advantages over these other approaches. Most notably, it consolidates variables using well-defined, reproducible, quantitative methods [20–23].
Concept mapping starts with the generation of items. In this study, items (quality indicators) were generated through a systematic search of the scientific and grey literature in Embase, PubMed, Scopus and Google that took place from September 2013 to January 2014. We were interested in published studies that included data about the indicators. The search terms were ‘questionnaire’, ‘benchmark’ and ‘health or healthcare quality’ in combination with ‘cataract’ or ‘phacoemulsification’ and additional Medical Subject Headings. Moreover, snowball techniques were employed by manually searching the reference lists in the primary selected studies to identify further research that was likely to fulfil the inclusion criteria. The initial search yielded many papers. After the duplicate papers were removed, the remaining titles and abstracts were manually reviewed. Papers that did not consider any quality indicators were removed. Moreover, the search results were complemented by quality indicators discussed in a digital brainstorm involving the researchers and the group of stakeholder representatives. Items in English were translated into Dutch. Items were removed if the researchers agreed that they were clearly irrelevant to the quality of cataract care. For the remaining items, duplicates were eliminated and closely related items were merged to create a final consolidated list.
As a next step, each stakeholder representative individually sorted the items into groups of items considered to be related and labelled each pile. They also individually rated each item according to importance for the quality of cataract care (5-point Likert scale: 1 = absolutely not important, 5 = extremely important compared to all other items). Since non-medical stakeholders had some difficulties in understanding the medical terms in the items, we provided additional information about those items.
After the sorting and ranking tasks, data were analysed by the researchers using Concept Systems Global MAX [24]. Items were represented in maps via multivariate statistical analyses of multidimensional scaling (MDS) and hierarchical clustering [21]. Based on a similarity index, all items were placed on a 2D point map in relation to each other, items sorted together frequently were close to one another on the map. A stress value statistic was calculated to reflect the goodness of fit of the MDS map with the dissimilarity matrix; for each pair of indicators, this displayed how often stakeholder representatives placed them in the same pile [25]. Cluster maps were generated from the point map to visualize how items could be organized into clusters using hierarchical clustering [20]. Rating maps were created to show the differences in importance per cluster, and bridge maps were generated to explore the relative agreement on rating variables between clusters and across stakeholder representatives [20]. Pattern matches yielded insight into the correlation among stakeholder representatives according to the importance of commonly agreed-upon dimensions. The resulting maps and information constituted a visual representation of the stakeholder representatives’ perspectives, as well as the organization and relative importance of the perceptions.
Finally, during a 3-h plenary session, stakeholder representatives interpreted the resulting clusters and maps. They discussed the content of the clusters and reached a consensus on the number of clusters in the map that best synthesized the quality of cataract care. Consensus was based on two conditions, as follows: (i) each cluster needed to have a meaningful interpretation and (ii) interpretation could not be improved by further dividing a cluster. To support this discussion, the researchers showed cluster maps with 5–12 clusters that included underlying quality items. Moreover, stakeholder representatives discussed the naming of the clusters and reached consensus on cluster labels. Stakeholder representatives were reimbursed for their time and travel costs.
Results
To achieve our goal of examining a multi-stakeholder perspective on health-service quality using the newly developed approach described above, we first classified the stakeholders (Table 1). Seventeen stakeholders were identified, of whom seven were considered definitive stakeholders because they were perceived to have all three attributes power, urgency and legitimacy. These definitive stakeholders were ophthalmologists, optometrists, GPs, healthcare insurers, private clinics, the NOG and hospital boards. Patients were categorized as dependent stakeholders because they were perceived to lack power. In contrast, the Ministry of VWS, the Dutch Healthcare Authority and Zorginstituut Nederland were classified as dormant stakeholders because they were considered to lack legitimacy and urgency. Seven stakeholders were classified as discretionary because they only scored high on legitimacy; these stakeholders included the Dutch organization for cataract patients, the NPCF, informal caregivers (relatives and volunteers) and anaesthetists. Finally, 13 representatives were identified as non-stakeholders. The inclusion of scores from stakeholder representatives who scored their own stakeholder groups did not influence the results.
. | Power . | Legitimacy . | Urgency . |
---|---|---|---|
Definitive stakeholders | + | + | + |
1. Dutch ophthalmic society NOG | 3.14 | 3.43 | 3.00 |
2. General practitioner | 3.33 | 3.50 | 3.17 |
3. Health insurer | 3.60 | 3.00 | 2.67 |
4. Hospital board | 2.71 | 3.57 | 2.86 |
5. Ophthalmologist | 3.67 | 4.00 | 4.00 |
6. Optometrist | 3.33 | 3.83 | 3.67 |
7. Private clinic board | 3.00 | 3.20 | 2.83 |
Dependent stakeholders | − | + | + |
8. Patient | 2.50 | 3.50 | 3.33 |
Dormant stakeholders | + | − | − |
9. Dutch healthcare authority | 3.17 | 2.67 | 2.00 |
10. Ministry of VWS | 3.33 | 2.83 | 2.17 |
11. Dutch National Healthcare Institute Zorginstiuut Nederland | 3.20 | 2.80 | 2.40 |
Weak stakeholders | − | + | − |
12. Anaesthetist | 2.71 | 3.00 | 2.57 |
13. Dutch organization for cataract patients | 2.29 | 3.43 | 2.86 |
14. Dutch patient federation NPCF | 2.57 | 3.00 | 2.57 |
15, Patient counsel of a hospital | 2.29 | 3.14 | 2.57 |
16, Relatives of patient | 2.29 | 3.00 | 2.57 |
17. Volunteers | 2.14 | 2.86 | 2.43 |
Non-stakeholders | − | − | − |
18. Bank | 2.00 | 1.67 | 1.83 |
19. Board of trustees of a hospital | 2.33 | 2.33 | 2.17 |
20. Foreign provider of cataract care | 2.43 | 2.33 | 2.17 |
21. Geriatrician | 2.50 | 2.83 | 2.67 |
22. Grant provider | 2.50 | 2.17 | 2.33 |
23. Guarantee fund for the healthcare sector WFZ | 2.33 | 2.00 | 2.00 |
24. Healthcare inspectorate | 3.17 | 3.17 | 2.83 |
25. Investor | 2.17 | 1.67 | 1.83 |
26. Medical liability insurance | 2.50 | 2.50 | 2.33 |
27. Municipality | 1.83 | 1.67 | 1.67 |
28. Politics: spokesman care | 2.50 | 2.17 | 2.00 |
29. Senior association ANBO | 1.80 | 2.33 | 2.00 |
30. Supplier (equipment, instruments, etc.) | 2.57 | 2.43 | 2.14 |
. | Power . | Legitimacy . | Urgency . |
---|---|---|---|
Definitive stakeholders | + | + | + |
1. Dutch ophthalmic society NOG | 3.14 | 3.43 | 3.00 |
2. General practitioner | 3.33 | 3.50 | 3.17 |
3. Health insurer | 3.60 | 3.00 | 2.67 |
4. Hospital board | 2.71 | 3.57 | 2.86 |
5. Ophthalmologist | 3.67 | 4.00 | 4.00 |
6. Optometrist | 3.33 | 3.83 | 3.67 |
7. Private clinic board | 3.00 | 3.20 | 2.83 |
Dependent stakeholders | − | + | + |
8. Patient | 2.50 | 3.50 | 3.33 |
Dormant stakeholders | + | − | − |
9. Dutch healthcare authority | 3.17 | 2.67 | 2.00 |
10. Ministry of VWS | 3.33 | 2.83 | 2.17 |
11. Dutch National Healthcare Institute Zorginstiuut Nederland | 3.20 | 2.80 | 2.40 |
Weak stakeholders | − | + | − |
12. Anaesthetist | 2.71 | 3.00 | 2.57 |
13. Dutch organization for cataract patients | 2.29 | 3.43 | 2.86 |
14. Dutch patient federation NPCF | 2.57 | 3.00 | 2.57 |
15, Patient counsel of a hospital | 2.29 | 3.14 | 2.57 |
16, Relatives of patient | 2.29 | 3.00 | 2.57 |
17. Volunteers | 2.14 | 2.86 | 2.43 |
Non-stakeholders | − | − | − |
18. Bank | 2.00 | 1.67 | 1.83 |
19. Board of trustees of a hospital | 2.33 | 2.33 | 2.17 |
20. Foreign provider of cataract care | 2.43 | 2.33 | 2.17 |
21. Geriatrician | 2.50 | 2.83 | 2.67 |
22. Grant provider | 2.50 | 2.17 | 2.33 |
23. Guarantee fund for the healthcare sector WFZ | 2.33 | 2.00 | 2.00 |
24. Healthcare inspectorate | 3.17 | 3.17 | 2.83 |
25. Investor | 2.17 | 1.67 | 1.83 |
26. Medical liability insurance | 2.50 | 2.50 | 2.33 |
27. Municipality | 1.83 | 1.67 | 1.67 |
28. Politics: spokesman care | 2.50 | 2.17 | 2.00 |
29. Senior association ANBO | 1.80 | 2.33 | 2.00 |
30. Supplier (equipment, instruments, etc.) | 2.57 | 2.43 | 2.14 |
Values are mean scores and were calculated by excluding scores in which a stakeholder rated his/her own stakeholder group. For example, when a patient (participant) rated the power of a patient (possible stakeholder), the score was excluded.
. | Power . | Legitimacy . | Urgency . |
---|---|---|---|
Definitive stakeholders | + | + | + |
1. Dutch ophthalmic society NOG | 3.14 | 3.43 | 3.00 |
2. General practitioner | 3.33 | 3.50 | 3.17 |
3. Health insurer | 3.60 | 3.00 | 2.67 |
4. Hospital board | 2.71 | 3.57 | 2.86 |
5. Ophthalmologist | 3.67 | 4.00 | 4.00 |
6. Optometrist | 3.33 | 3.83 | 3.67 |
7. Private clinic board | 3.00 | 3.20 | 2.83 |
Dependent stakeholders | − | + | + |
8. Patient | 2.50 | 3.50 | 3.33 |
Dormant stakeholders | + | − | − |
9. Dutch healthcare authority | 3.17 | 2.67 | 2.00 |
10. Ministry of VWS | 3.33 | 2.83 | 2.17 |
11. Dutch National Healthcare Institute Zorginstiuut Nederland | 3.20 | 2.80 | 2.40 |
Weak stakeholders | − | + | − |
12. Anaesthetist | 2.71 | 3.00 | 2.57 |
13. Dutch organization for cataract patients | 2.29 | 3.43 | 2.86 |
14. Dutch patient federation NPCF | 2.57 | 3.00 | 2.57 |
15, Patient counsel of a hospital | 2.29 | 3.14 | 2.57 |
16, Relatives of patient | 2.29 | 3.00 | 2.57 |
17. Volunteers | 2.14 | 2.86 | 2.43 |
Non-stakeholders | − | − | − |
18. Bank | 2.00 | 1.67 | 1.83 |
19. Board of trustees of a hospital | 2.33 | 2.33 | 2.17 |
20. Foreign provider of cataract care | 2.43 | 2.33 | 2.17 |
21. Geriatrician | 2.50 | 2.83 | 2.67 |
22. Grant provider | 2.50 | 2.17 | 2.33 |
23. Guarantee fund for the healthcare sector WFZ | 2.33 | 2.00 | 2.00 |
24. Healthcare inspectorate | 3.17 | 3.17 | 2.83 |
25. Investor | 2.17 | 1.67 | 1.83 |
26. Medical liability insurance | 2.50 | 2.50 | 2.33 |
27. Municipality | 1.83 | 1.67 | 1.67 |
28. Politics: spokesman care | 2.50 | 2.17 | 2.00 |
29. Senior association ANBO | 1.80 | 2.33 | 2.00 |
30. Supplier (equipment, instruments, etc.) | 2.57 | 2.43 | 2.14 |
. | Power . | Legitimacy . | Urgency . |
---|---|---|---|
Definitive stakeholders | + | + | + |
1. Dutch ophthalmic society NOG | 3.14 | 3.43 | 3.00 |
2. General practitioner | 3.33 | 3.50 | 3.17 |
3. Health insurer | 3.60 | 3.00 | 2.67 |
4. Hospital board | 2.71 | 3.57 | 2.86 |
5. Ophthalmologist | 3.67 | 4.00 | 4.00 |
6. Optometrist | 3.33 | 3.83 | 3.67 |
7. Private clinic board | 3.00 | 3.20 | 2.83 |
Dependent stakeholders | − | + | + |
8. Patient | 2.50 | 3.50 | 3.33 |
Dormant stakeholders | + | − | − |
9. Dutch healthcare authority | 3.17 | 2.67 | 2.00 |
10. Ministry of VWS | 3.33 | 2.83 | 2.17 |
11. Dutch National Healthcare Institute Zorginstiuut Nederland | 3.20 | 2.80 | 2.40 |
Weak stakeholders | − | + | − |
12. Anaesthetist | 2.71 | 3.00 | 2.57 |
13. Dutch organization for cataract patients | 2.29 | 3.43 | 2.86 |
14. Dutch patient federation NPCF | 2.57 | 3.00 | 2.57 |
15, Patient counsel of a hospital | 2.29 | 3.14 | 2.57 |
16, Relatives of patient | 2.29 | 3.00 | 2.57 |
17. Volunteers | 2.14 | 2.86 | 2.43 |
Non-stakeholders | − | − | − |
18. Bank | 2.00 | 1.67 | 1.83 |
19. Board of trustees of a hospital | 2.33 | 2.33 | 2.17 |
20. Foreign provider of cataract care | 2.43 | 2.33 | 2.17 |
21. Geriatrician | 2.50 | 2.83 | 2.67 |
22. Grant provider | 2.50 | 2.17 | 2.33 |
23. Guarantee fund for the healthcare sector WFZ | 2.33 | 2.00 | 2.00 |
24. Healthcare inspectorate | 3.17 | 3.17 | 2.83 |
25. Investor | 2.17 | 1.67 | 1.83 |
26. Medical liability insurance | 2.50 | 2.50 | 2.33 |
27. Municipality | 1.83 | 1.67 | 1.67 |
28. Politics: spokesman care | 2.50 | 2.17 | 2.00 |
29. Senior association ANBO | 1.80 | 2.33 | 2.00 |
30. Supplier (equipment, instruments, etc.) | 2.57 | 2.43 | 2.14 |
Values are mean scores and were calculated by excluding scores in which a stakeholder rated his/her own stakeholder group. For example, when a patient (participant) rated the power of a patient (possible stakeholder), the score was excluded.
As a next step, 125 items were determined to measure the quality of cataract care; these items appear in Appendix 1 with their identifying numbers and average importance ratings. Stakeholder representatives sorted the items into an average of 10 piles (mean [M] = 9.8, standard deviation [SD] = 2.7) with a mean importance of M = 3.79 (SD = 0.54), suggesting high overall importance.
During the plenary meeting, the MDS map with the seven-cluster solution was chosen as final. One stakeholder representative completed the sorting and rating assignments after the plenary session due to illness. These results had a slight effect on the concept map. Subsequent adjustments were communicated to and approved by all stakeholder representatives. The stress value for this MDS map was 0.27, indicating a satisfactory fit [25, 26]. The clusters are visualized in Fig. 2; considering the clusters in a clockwise fashion, their agreed-upon labels were as follows: patient centredness and accessibility, interpersonal conduct and expectations, experienced outcome, clinical outcome, process and structure, medical technical acting and safety. According to the stakeholder representatives, the horizontal axis in the MDS map represented a time axis, ranging from condition to outcome, while the vertical axis ranged from supply (technical acting, clinical aspects and structures) to demand (processes, experiences, service outcomes and patient values).
The average bridging values for each cluster in the final MDS map, indicating the relative agreement on rated items, are presented in Appendix 1. Average bridging values ranged from 0.15 for the most homogeneous cluster (cluster II, interpersonal conduct and expectations) to 0.73 for the least homogeneous cluster (cluster V, safety).
The importance ratings of the stakeholder groups were strongly correlated (Tables 2 and 3). On average, however, the correlation coefficients associated with the optometrists were considerably lower than those of other stakeholders (0.45 versus 0.70 or higher). Conversely, valuations by the NPCF and Zorginstituut Nederland yielded the largest average R values (0.98). When distinguishing the representatives of stakeholders directly involved in health-service delivery from the others, the correlation within the distinguished groups was larger than that between groups.
Clusters . | Ophthalmologist . | GP . | Optometrist . | Patients . | Zorginstituut Nederland . | Health insurer . | Care providersa . | NPCF . |
---|---|---|---|---|---|---|---|---|
I Patient centredness and accessibility | 3.21 | 3.62 | 2.80 | 3.61 | 2.80 | 3.12 | 3.64 | 4.00 |
II Personal conduct and expectations | 4.13 | 4.13 | 3.70 | 4.47 | 3.87 | 3.98 | 4.80 | 4.90 |
III Clinical outcome | 3.44 | 4.31 | 3.06 | 3.86 | 4.00 | 4.31 | 4.59 | 5.00 |
IV Experienced outcome | 3.79 | 4.13 | 3.00 | 4.41 | 3.63 | 3.82 | 4.47 | 4.75 |
V Safety | 3.66 | 4.09 | 3.88 | 4.18 | 3.88 | 4.00 | 4.34 | 4.78 |
VI Process en structure | 3.77 | 3.83 | 3.21 | 4.08 | 3.50 | 3.83 | 4.13 | 4.67 |
VII Medical technical acting | 2.75 | 3.46 | 3.29 | 3.45 | 3.50 | 3.18 | 3.79 | - |
Clusters . | Ophthalmologist . | GP . | Optometrist . | Patients . | Zorginstituut Nederland . | Health insurer . | Care providersa . | NPCF . |
---|---|---|---|---|---|---|---|---|
I Patient centredness and accessibility | 3.21 | 3.62 | 2.80 | 3.61 | 2.80 | 3.12 | 3.64 | 4.00 |
II Personal conduct and expectations | 4.13 | 4.13 | 3.70 | 4.47 | 3.87 | 3.98 | 4.80 | 4.90 |
III Clinical outcome | 3.44 | 4.31 | 3.06 | 3.86 | 4.00 | 4.31 | 4.59 | 5.00 |
IV Experienced outcome | 3.79 | 4.13 | 3.00 | 4.41 | 3.63 | 3.82 | 4.47 | 4.75 |
V Safety | 3.66 | 4.09 | 3.88 | 4.18 | 3.88 | 4.00 | 4.34 | 4.78 |
VI Process en structure | 3.77 | 3.83 | 3.21 | 4.08 | 3.50 | 3.83 | 4.13 | 4.67 |
VII Medical technical acting | 2.75 | 3.46 | 3.29 | 3.45 | 3.50 | 3.18 | 3.79 | - |
aHospital board and private clinic board.
Clusters . | Ophthalmologist . | GP . | Optometrist . | Patients . | Zorginstituut Nederland . | Health insurer . | Care providersa . | NPCF . |
---|---|---|---|---|---|---|---|---|
I Patient centredness and accessibility | 3.21 | 3.62 | 2.80 | 3.61 | 2.80 | 3.12 | 3.64 | 4.00 |
II Personal conduct and expectations | 4.13 | 4.13 | 3.70 | 4.47 | 3.87 | 3.98 | 4.80 | 4.90 |
III Clinical outcome | 3.44 | 4.31 | 3.06 | 3.86 | 4.00 | 4.31 | 4.59 | 5.00 |
IV Experienced outcome | 3.79 | 4.13 | 3.00 | 4.41 | 3.63 | 3.82 | 4.47 | 4.75 |
V Safety | 3.66 | 4.09 | 3.88 | 4.18 | 3.88 | 4.00 | 4.34 | 4.78 |
VI Process en structure | 3.77 | 3.83 | 3.21 | 4.08 | 3.50 | 3.83 | 4.13 | 4.67 |
VII Medical technical acting | 2.75 | 3.46 | 3.29 | 3.45 | 3.50 | 3.18 | 3.79 | - |
Clusters . | Ophthalmologist . | GP . | Optometrist . | Patients . | Zorginstituut Nederland . | Health insurer . | Care providersa . | NPCF . |
---|---|---|---|---|---|---|---|---|
I Patient centredness and accessibility | 3.21 | 3.62 | 2.80 | 3.61 | 2.80 | 3.12 | 3.64 | 4.00 |
II Personal conduct and expectations | 4.13 | 4.13 | 3.70 | 4.47 | 3.87 | 3.98 | 4.80 | 4.90 |
III Clinical outcome | 3.44 | 4.31 | 3.06 | 3.86 | 4.00 | 4.31 | 4.59 | 5.00 |
IV Experienced outcome | 3.79 | 4.13 | 3.00 | 4.41 | 3.63 | 3.82 | 4.47 | 4.75 |
V Safety | 3.66 | 4.09 | 3.88 | 4.18 | 3.88 | 4.00 | 4.34 | 4.78 |
VI Process en structure | 3.77 | 3.83 | 3.21 | 4.08 | 3.50 | 3.83 | 4.13 | 4.67 |
VII Medical technical acting | 2.75 | 3.46 | 3.29 | 3.45 | 3.50 | 3.18 | 3.79 | - |
aHospital board and private clinic board.
. | Ophthalmologists . | GPs . | Optometrists . | Patients . | Zorginstituut Nederland . | Health insurers . | Care providersa . | NPCF . | All . |
---|---|---|---|---|---|---|---|---|---|
Ophthalmologists | 0.71 | 0.39 | 0.95 | 0.44 | 0.67 | 0.78 | 0.62 | 0.85 | |
GPs | 1.62 | 0.27 | 0.72 | 0.75 | 0.94 | 0.91 | 0.92 | 0.84 | |
Optometrists | 2.55 | 2.43 | 0.44 | 0.61 | 0.39 | 0.46 | 0.52 | 0.45 | |
Patients | 0.84 | 1.36 | 2.01 | 0.53 | 0.65 | 0.81 | 0.63 | 0.89 | |
Zorginstituut Nederland | 2.69 | 1.27 | 1.86 | 2.25 | 0.86 | 0.84 | 0.98 | 0.70 | |
Health insurers | 2.00 | 1.04 | 2.55 | 1.60 | 1.09 | 0.89 | 0.97 | 0.80 | |
Care providersa | 1.52 | 0.80 | 2.35 | 1.34 | 1.29 | 1.18 | 0.93 | 0.91 | |
NPCF | 2.40 | 1.20 | 1.86 | 2.05 | 0.66 | 1.14 | 1.16 | 0.75 |
. | Ophthalmologists . | GPs . | Optometrists . | Patients . | Zorginstituut Nederland . | Health insurers . | Care providersa . | NPCF . | All . |
---|---|---|---|---|---|---|---|---|---|
Ophthalmologists | 0.71 | 0.39 | 0.95 | 0.44 | 0.67 | 0.78 | 0.62 | 0.85 | |
GPs | 1.62 | 0.27 | 0.72 | 0.75 | 0.94 | 0.91 | 0.92 | 0.84 | |
Optometrists | 2.55 | 2.43 | 0.44 | 0.61 | 0.39 | 0.46 | 0.52 | 0.45 | |
Patients | 0.84 | 1.36 | 2.01 | 0.53 | 0.65 | 0.81 | 0.63 | 0.89 | |
Zorginstituut Nederland | 2.69 | 1.27 | 1.86 | 2.25 | 0.86 | 0.84 | 0.98 | 0.70 | |
Health insurers | 2.00 | 1.04 | 2.55 | 1.60 | 1.09 | 0.89 | 0.97 | 0.80 | |
Care providersa | 1.52 | 0.80 | 2.35 | 1.34 | 1.29 | 1.18 | 0.93 | 0.91 | |
NPCF | 2.40 | 1.20 | 1.86 | 2.05 | 0.66 | 1.14 | 1.16 | 0.75 |
aHospital board and private clinic board.
. | Ophthalmologists . | GPs . | Optometrists . | Patients . | Zorginstituut Nederland . | Health insurers . | Care providersa . | NPCF . | All . |
---|---|---|---|---|---|---|---|---|---|
Ophthalmologists | 0.71 | 0.39 | 0.95 | 0.44 | 0.67 | 0.78 | 0.62 | 0.85 | |
GPs | 1.62 | 0.27 | 0.72 | 0.75 | 0.94 | 0.91 | 0.92 | 0.84 | |
Optometrists | 2.55 | 2.43 | 0.44 | 0.61 | 0.39 | 0.46 | 0.52 | 0.45 | |
Patients | 0.84 | 1.36 | 2.01 | 0.53 | 0.65 | 0.81 | 0.63 | 0.89 | |
Zorginstituut Nederland | 2.69 | 1.27 | 1.86 | 2.25 | 0.86 | 0.84 | 0.98 | 0.70 | |
Health insurers | 2.00 | 1.04 | 2.55 | 1.60 | 1.09 | 0.89 | 0.97 | 0.80 | |
Care providersa | 1.52 | 0.80 | 2.35 | 1.34 | 1.29 | 1.18 | 0.93 | 0.91 | |
NPCF | 2.40 | 1.20 | 1.86 | 2.05 | 0.66 | 1.14 | 1.16 | 0.75 |
. | Ophthalmologists . | GPs . | Optometrists . | Patients . | Zorginstituut Nederland . | Health insurers . | Care providersa . | NPCF . | All . |
---|---|---|---|---|---|---|---|---|---|
Ophthalmologists | 0.71 | 0.39 | 0.95 | 0.44 | 0.67 | 0.78 | 0.62 | 0.85 | |
GPs | 1.62 | 0.27 | 0.72 | 0.75 | 0.94 | 0.91 | 0.92 | 0.84 | |
Optometrists | 2.55 | 2.43 | 0.44 | 0.61 | 0.39 | 0.46 | 0.52 | 0.45 | |
Patients | 0.84 | 1.36 | 2.01 | 0.53 | 0.65 | 0.81 | 0.63 | 0.89 | |
Zorginstituut Nederland | 2.69 | 1.27 | 1.86 | 2.25 | 0.86 | 0.84 | 0.98 | 0.70 | |
Health insurers | 2.00 | 1.04 | 2.55 | 1.60 | 1.09 | 0.89 | 0.97 | 0.80 | |
Care providersa | 1.52 | 0.80 | 2.35 | 1.34 | 1.29 | 1.18 | 0.93 | 0.91 | |
NPCF | 2.40 | 1.20 | 1.86 | 2.05 | 0.66 | 1.14 | 1.16 | 0.75 |
aHospital board and private clinic board.
Discussion
In this study, we examined methods for identifying health-service quality concerning cataract care through a multi-stakeholder approach. While our study is far from the first in this domain, it appears to be the first that has explicitly and systematically developed inclusive multi-stakeholder health-service quality dimensions for cataract care in the Netherlands.
Another recent and well-known initiative is the International Consortium for Health Outcome Measurement (ICHOM) [16], which is comparable to our study, as both ICHOM and our approach aimed to create a minimum set of indicators using a multidisciplinary method. However, our study is also different from the ICHOM initiative in several ways. First, while ICHOM only included patient, provider and registry perspectives, we included stakeholder perspectives using a systematic approach without excluding stakeholders beforehand; this resulted in a broader group of stakeholder perspectives. Second, ICHOM only focused on outcome indicators, while we also were interested in process and structure indicators, which seemed to be important to our stakeholders. Third, ICHOM used a Delphi approach, whereas we used concept mapping to create a consensus among stakeholders. We preferred concept mapping because this approach can be employed not only to create consensus but also to quantify and visualize differences among stakeholders, which contribute to equate the contribution between different stakeholders.
For validation of our approach, we considered how the resulting dimensions related to existing classifications. The World Health Organization (WHO) distinguished six domains in their definition of health-service quality, namely effectiveness, efficiency, accessibility, patient centredness, equity and safety [27]. The effectiveness domain translates to the two outcome clusters in our study, while the WHO domains of accessibility and patient centredness combined into one cluster in our study. The WHO safety domain mapped directly to our safety cluster. Equity did not seem to play a role in the current investigation, as all Dutch citizens have mandatory insurance that fully covers the cost of cataract care. Our study identified several clusters that did not map to the WHO domains; these mostly involved health-service provisioning, including the following: interpersonal conduct and expectations, process and structure and medical–technical acting. Our analysis lacked a cluster that could be considered a natural counterpart to the WHO efficiency domain. This was partly the result of our item list, which was based on existing lists that apparently contained few items relating to efficiency. Moreover, stakeholder representatives did not add efficiency items to the list. Nonetheless, efficiency was discussed extensively during the plenary meeting of stakeholder representatives, as it was perceived to be missing. The stakeholder representatives debated whether efficiency is a quality dimension or whether it should be considered separately (cost versus quality). In this respect, the specific consensus of Dutch cataract care stakeholder representatives differed from the WHO view. Similar observations can be made when comparing the results with the six aims of improvement established by the Institute of Medicine in the United States [28]. In contrast to chronic care, cataract care is a relatively straightforward intervention with a short pre- and post-operative trajectory. This might have influenced the participation of stakeholders and their perspectives in our study. The differences in perspectives that emerged support the proposition that health-service quality is subjectively defined and varies among stakeholders, countries and conditions [2, 3].
Somewhat surprisingly and counterintuitively, our research revealed that most Dutch cataract care stakeholder representatives viewed patients as lacking power. Thus, patients are not classified as definite stakeholders. This appears to contradict the commonly held view that patients were the ultimate stakeholder. Our results signal that current Dutch (and international) efforts to improve patient centredness are required to empower patients and position their patient values as the basis for decision making [29, 30].
There was considerable consensus among the GP, Zorginstituut Nederland, insurer, provider organization and NPCF stakeholders concerning the importance of quality dimensions and items. At the same time, this consensus was not fully aligned with the viewpoints of representatives directly involved in the process of health-service delivery, namely optometrists, ophthalmologists, and most notably, patients. Since we found that ophthalmologists and optometrists were in line with patients in their perspectives towards quality, they can promote the patients during clinical decision making, and thereby support patient empowerment. We note too, however, that stakeholder classifications are not steady states; current policies and changing values may lead to increases in patients’ power over time.
A major strength of this study was the panel of stakeholder representatives, which was systematically selected based on of stakeholder theory. If identification of stakeholders is unsystematic or poorly structured, valuable stakeholder groups can be missed [13]. The methods used ensured that all stakeholders considered relevant by other stakeholders took part in the concept mapping; hence, their views were systematically included in our analyses. However, we note that the NOG, the professional association for ophthalmologists in the Netherlands, acknowledged as a definitive stakeholder, did not accept our invitation to participate. Considering that the NOG represented ophthalmologists, we obviated this setback by inviting more ophthalmologists.
Another strength was that the concept mapping was initiated through an exhaustive literature search to ensure that the most timely and reliable information about health-service quality indicators available was included. Moreover, the indicator list was complemented by a digital brainstorm involving the researchers and the group of stakeholder representatives. This approach provided a structured starting point for the stakeholder representatives, forming a systematic and sound basis for further involvement of the stakeholder representatives, and strengthening the internal validity and feasibility of the concept-mapping process.
The size of our panel was consistent with the recommended size for concept mapping. Further increases in the number of participants are expected to hamper discussion. However, it is possible that due to the small size of each subgroup, caution is needed when comparing results between subgroups. Based on the large within stakeholder-group agreement in sorting and rating the quality items, our findings indicate that the effect on internal validity is limited.
Conclusion
The globally felt urgency to improve health-service quality has led to extensive debate and considerable complexity because of differences in viewpoints on its definition and measurement. We have proposed a systematic approach to develop a condition/treatment-specific multi-stakeholder consensus-based definition of health-service quality. The consistent results obtained in the case study on cataract care confirmed the applicability of the methods. They enabled a commonly agreed definition to be developed, in which differences in viewpoints were expressed in relation to differences in importance valuations of the indicators defining the dimensions. Not only has this resulted in seven dimensions to define the quality of Dutch cataract care (which we further operationalize in future research), but it has also revealed that stakeholders presently perceive patient power to fall short of national policy aims. In practice, this may be compensated for through a general agreement on quality with other stakeholders involved in service delivery.
Supplementary material
Supplementary material is available at International Journal for Quality in Health Care online.
Funding
Aline Stolk-Vos has been funded by the Rotterdam Eye Hospital.
Acknowledgements
This project could not have been completed without the cooperation and assistance of the group of stakeholder representatives. We thank the group of stakeholder representatives for their participation in this study: Wim van Bodegom, Marthein Gaasbeek Janzen, Mr Geilman, Marion Heres, Jurgen Hermsen, Mr Hofman, Mr Hörchner, Sietske Huiskens, Niels Maijers, Heleen Post, Nic Reus, Savitri Ritoe, Marjolein Ronday, Albert Scholte, Nicola Smoljanic, Mr Thoen, Joan Veldkamp, Mr Voorrips and Bart Zijlmans.
Conflict of interest statement
Participants in the panel were asked to represent their affiliation.