Value to Whom? The Patient Voice in the Value Discussion

September 13, 2017  

Español I Português

Eleanor Perfetto, RPh, PhD, MS, Strategic Initiatives, National Health
Council, Washington, DC, USA

Objectives: Professional societies and other organizations recently have taken a visible role trying to define treatment value via value frameworks and assessments, providing payer or provider recommendations, and potentially impacting patient access. Patient perspectives routinely differ from those of other stakeholders. Yet, it is not always apparent patients were engaged in value framework development or assessments. This article describes the development and content of the National Health Council’s (NHC) Rubric, a tool that includes criteria for evaluation of value frameworks specifically with regard to patient centeredness and meaningful patient engagement.


Methods: The NHC held a multi-stakeholder, invitational roundtable in Washington D.C., United States. Participants reviewed existing patient-engagement rubrics; discussed experiences with value frameworks; debated and thematically grouped hallmark patient-centeredness characteristics; and developed illustrative examples of the characteristics. These materials were organized into the rubric, and subsequently vetted via multi-stakeholder peer review.


Results: The resulting rubric describes six domains of patient-centered value frameworks: Partnership, Transparency, Inclusiveness, Diversity, Outcomes, and Data Sources. Each domain includes specific examples illustrating how patient engagement and centeredness can be operationalized in value framework processes.


Conclusion: The NHC multi-stakeholder roundtable’s recommendations are captured in the NHC’s Rubric to assess value framework and model patient-centeredness and engagement. The Rubric is a tool that will be refined over time based upon feedback from patient, patient group, framework developer, and other stakeholder-use experiences.



The United States (US) health care system is undergoing a transformation in the way it delivers and pays for care. As traditional fee-for-service payment gives way to more value-based payment arrangements, understanding and defining the value of health care treatments and interventions has become a national priority. 1 Value frameworks have emerged to aid health care stakeholders in assessing the value of new treatments. While somewhat new to the US, value assessments have long been a routine approach used more formally in health care decision-making and priority setting in many other countries. 2,3 For example, in countries with public health insurance, such as Canada, Australia, or European countries, national and/or regional health technology assessment (HTA) agencies are well established. 4,5 In Asian and Latin American countries, HTA processes are becoming more widespread. 6,7  In 2015, four organizations in the US released value frameworks described as intended to support physicians and/or payers in assessing treatments. 8-11 These developers, including professional societies and think-tank organizations, are taking an increasingly visible role defining value and providing recommendations to payers and providers. 12 Unlike international efforts, the US-based frameworks are not mandated by law or endorsed by public payers.


While most stakeholders agree that spiraling health care costs must be addressed and discussions on value are necessary, framework developers have been criticized for “dangerously oversimplifying complex issues” and potentially limiting patient access to valuable treatments. 13 Developers have largely adopted a conventional, health-economic approach to value, in terms of treatment effectiveness and cost.14 Value frameworks often include quantitative, economic models, deriving cost effectiveness or utility, and/or budget impact. However, for patients, value is individualized and disease-dependent, and can evolve with the disease trajectory or a patient’s stage of life. 15-18 Importantly, patient perspectives on value can differ significantly from that of physicians and payers, often integrating considerations beyond clinical outcomes and cost, such as a treatment’s ability to help patients achieve personal goals. 19 Similarly, even among patients with the same condition, individual values and preferences may differ. 20 Despite lack of agreement over how value is defined and assessed, value frameworks have gained recent attention in the US, raising concerns regarding their appropriateness for determining access to care. 21 In countries or regions with established public health decision-making bodies, stakeholders have long echoed these concerns, resulting in formalized procedures to involve patients in health care priority setting. 22 Despite these efforts, even in countries with ongoing efforts to include patients in decision-making, concerns persist that patient involvement efforts are insufficient or “rhetoric.” 23,24


To the patient community, discussions regarding value are not a theoretical exercise and if value frameworks are to be used to inform decisions affecting treatment access, the patient community wants to have a central role in defining value and in value assessment. 25-27 Thus, robust processes are needed in order to incorporate the patient voice in value assessment. Yet, it has not always been apparent if individual patients or patient organizations were engaged in new value framework creation or use. 28


As developers work to improve the patient centricity of their frameworks and accompanying economic models, both in the US and abroad, bridging any gap between cost-effectiveness interpretations of “value” in health care and patient perceptions of value is increasingly important. 29-31 The patient advocacy group members of the NHC expressed that they were not sure how to help bridge these gaps as they were navigating new waters. 32 To assist in this effort, the National Health Council (NHC) convened a Value Roundtable to construct a rubric as a tool to support understanding of patient centeredness in value assessment. This paper provides an overview of the development and content of, and expected uses for the NHC Patient-Centered Value Model Rubric (Rubric), which conceptualizes patient centeredness and meaningful incorporation of the patient voice throughout value framework (and related economic model) development. 26


Development of the Rubric


A convenience sample of 28 organizations (12=patient organizations; 3=payers; 6=professional or policy organizations; 5=biopharmaceutical organizations; 2 research organizations), were invited to participate in a day-long roundtable discussion on February 1, 2016 in Washington D.C., US. The roundtable discussion was divided into three components. The first was reviewing the objective for the roundtable, to produce a rubric for value frameworks (and economic models they may include), capturing expectations and delineating consistent criteria for patient centeredness. For this project, the term “rubric” describes a tool with criteria and levels of achievement in patient centeredness and engagement. Examples of existing, publicly available patient-engagement rubrics were reviewed as models, including the Patient-Centered Outcomes Research Institute’s (PCORI) Patient and Family Engagement Rubric and the University of Maryland’s Center of Excellence in Regulatory Science and Innovation’s Rubric on Assessing Meaningful Patient Engagement in Drug Development. 33-35


Second, the group was asked to describe the hallmark characteristics of patient centeredness and frame those characteristics in the context of value assessments. Storyboarding was used throughout to ensure key points were accurately captured. The third part of the group discussion was dedicated to reviewing the storyboards, analyzing and grouping patient-centricity characteristics, and creating illustrative examples of high versus low engagement in the context of the grouped themes.


The storyboards, themes, and illustrative examples were subsequently used by NHC staff to craft a draft rubric, which was reviewed by the roundtable participants who provided comment, edits, and additions. The next iteration of the draft was vetted through multi-stakeholder peer review, including representatives from patient advocacy organizations, payers, professional societies, biopharmaceutical companies, and policy institutes. The draft was sent to 33 organizations (12=patient organizations; 3=payers; 7=professional or policy organizations; 9=biopharmaceutical organizations; 2 research organizations), comprising NHC members, and others recommended by roundtable participants. Input from three value model developers was specifically sought.



Roundtable participants included seven patient groups representing diseases/treatments with value assessment released in the previous year and those anticipating release in the coming year, and one representative from a consumer group. It also included representatives from three biopharmaceutical companies, two from a professional association, and one from a not-for-profit research organization.


The group discussed their views on the terms being used (e.g., patient, patient community, patient centeredness, value framework and model) and agreed on how they would be used in their discussion and in the Rubric (See Online Supplementary Material). Participants grouped the characteristics they listed into six key domains of patient centeredness: partnership, transparency, inclusiveness, diversity, outcomes, and data (Table). They discussed that direct patient engagement is the ideal goal, where the patient community has an active role as a co-investigator, partner, advisor or consultant. However, indirect means of engagement also contribute to patient centeredness and should not be overlooked. Indirect means, for example, include such things as using existing data sets that capture patient views and preferences.


The Patient-Centered Value Model Rubric

Based on the Roundtable discussion, a draft Rubric resulted describing the two complementary, direct and indirect, pathways for addressing the six domains of patient centeredness. 26 Specific examples from the discussion representing high or low activity were included. Of the 33 organizations invited to submit comments on the draft Rubric, 11 organizations (1=payers; 3=professional or policy organizations; 7=biopharmaceutical organizations), responded and their comments were incorporated.


In the Table, excerpts from the final Rubric document’s content are provided (the full Rubric is in Online Supplementary Materials). The characteristics capture aspects of direct engagement and indirect patient centeredness and illustrative examples for each of the six key domains of patient centricity. The Rubric was released publicly on March 28, 2016 as part of a public webinar held by the NHC. 36 (



The roundtable participants were generally supportive of the intent behind value assessments and stated that they do not wish to waste money or other resources on unnecessary procedures and/or prescriptions. But, patients want information to make informed decisions based on what is or is not valuable to them. Participants agreed that to be considered patient centered, value frameworks must have the key characteristics that resulted in six domains of patient centeredness. The Rubric formed around these domains promotes patient centeredness in the value discussion by engaging, informing, and actively listening to people with chronic conditions at every point of contact. 18,37


The participants were enthusiastic about the construction of the Rubric as a learning tool, but stopped short of defining a scoring system or conducting full evaluations of the frameworks that exist. Their rationale was that there is not enough US-based experience with value frameworks; therefore, a benchmark for scoring or rating did not exist. They advised caution in that the Rubric should not be viewed as punitive, but rather as informative and educational, much in line with the way the PCORI implemented its original patient and family engagement rubric. 38 Similar to PCORI’s Engagement Rubric, which illustrates “how input from patient and stakeholder partners can be incorporated throughout the entire research project,” the NHC Rubric can guide framework developers from the planning stages through the update and maintenance stage. 34 The recommendation from the group was to disseminate the Rubric and engage with patient groups and developers to get their feedback on its usability, validity and impact, and update it in the future as deemed necessary, instituting a scoring system if needed.


Since the public release of the Rubric, framework developers have acknowledged that elements of high value to patients may be missing from their work. For example, the American Society of Clinical Oncology’s revised framework, states, “the task force is well aware that there are many elements that might be important to individual patients in assessing the relative value of their treatment options that are not taken into account in our model.” 30 The authors justify leaving patient views out by stating that elements of high value to patients fit under the patient-reported outcomes (PROs) umbrella; however, reliable PRO assessments are not yet available. This statement demonstrates that developers have become more sensitive to the need for frameworks to be more patient centered. However, it also demonstrate there is more work to be done to promote patient centeredness. Patient centeredness extends beyond capturing PRO data and the outcomes that matter most to patients are not always PROs. In fact, depending on the particular patient population, individual patients may prioritize changes in biomarkers, improvements in clinician-reported outcomes, or other observable symptoms. 39


The Institute for Clinical and Economic Review recently solicited input from the health care community on how its methodology should be modified to include “important patient-relevant and patient-centered outcomes, especially those not adequately captured in the clinical trial data.” 40 These efforts to improve the patient centeredness of value frameworks are particularly important if physicians and payers look to frameworks to inform decisions that can affect the treatment options available to a patient.


In a more general sense, the domains and attributes included in the Rubric could apply to patient centeredness and engagement in contexts beyond value assessment. 41 Efforts to involve patients in value assessment align with other patient centricity efforts, such as involving patients in research, as is promoted by PCORI and increasingly by the Food and Drug Administration through their patient-focused drug development initiative. 41-43 While value frameworks have a different purpose and intended audience, each of them will likely have an impact on the care or access to care patients’ experience. Application of the Rubric in many of these other contexts can help to ensure processes reflects patient priorities, and opportunities and touch points for reaching out and involving patients, family caregivers, and patient advocacy organizations.


One outstanding question that still needs to be addressed is that of representativeness. 44 If developers are to be inclusive in the patient populations they engage, there must be clarity about what representativeness means. Future efforts must help to articulate clearly how to judge when a representative sample of patients have or have not been engaged in the processes of value framework development and assessment. Again, the relevance of representativeness is broad and can apply to value assessments as well a patient-focused drug development and patient-centered health care.


Importantly, for patient-centered value assessment to be successful, the onus is not only on value framework developers, but other stakeholders as well. For example, the research community must participate by pursuing and publishing research examining health outcomes that matter to patients. Methods for defining and enrolling heterogeneous populations in studies must be developed and patient-centered data sources should be utilized. While patient advocacy organizations are undoubtedly experts on their condition of interest, they must also be prepared to participate in value discussions, for example, by soliciting and providing information from their diverse memberships regarding the impact of disease on their lives, natural history of disease, preferred outcomes, and related epidemiologic information.


There are a number of limitations to the Rubric that should be noted. The Rubric content was based upon the roundtable discussion and vetting among a group of peer reviewers. The participants and reviewers were a convenience sample of NHC members, colleagues, and others who had been recommended by the roundtable. This approach may not have captured the views of all stakeholders or the breadth of representation within any one stakeholder group. The Rubric has not been formally tested and needs to be used by a variety of stakeholders to fully capture its utility, validity, and impact. Finally, while the domains of patient centricity and examples of high versus low patient engagement may be broadly applicable, this initiative was developed in the context of the US health care system and frameworks recently published in the US. The Rubric may require cultural adaptation prior to its use by international audiences.


Value frameworks can help advance the national dialogue on value in health care, but only if these frameworks incorporate the patient voice. The NHC Rubric is a first step toward creating patient-centered value assessments patients and their families can rely on. It is intended to assist all stakeholders, especially the patient community, in assessing the level of patient centeredness and engagement in a given framework or model. It can be a guide to support developers in conceptualizing plans for meaningfully engaging patients.


At present, the Rubric should be considered a guide for patient-centeredness good practices. The Rubric is intended to be a living document, to be refined over time based upon feedback from patients, patient groups, framework developers, and other stakeholder-use experiences. It will require maintenance, updating, and enhancement as experience and knowledge grow on its use.



  1. Burwell SM. Setting value-based payment goals — HHS efforts to improve U.S. health care. N Engl J Med. 2015; 372(10):897–9.
  2. Neumann PJ, Cohen JT. Measuring the value of prescription drugs. N Engl J Med. 2015; 373(27):2595–7.
  3. Garrison LP. Cost-effectiveness and clinical practice guidelines: Have we reached a tipping point?—An overview. Value in Health. 2016; 19(5):512–5.
  4. Banta D, Kristensen FB, Jonsson E. A history of health technology assessment at the european level. Int J Technol Assess Health Care. 2009; 25: 68–73.
  5. Oortwijn W, Broos P, Vondeling H, Banta D, Todorova L. Mapping of health technology assessment in selected countries. Int J Technol Assess Health Care. 2013; 29(4):424–34.
  6. Yang BM. Health technology assessment in asia: An emerging trend. J Comp Eff Res. 2012; 1(3):221–4.
  7. Augustovski F, Alcaraz A, Caporale J, Garcia Marti S, Pichon Riviere A. Institutionalizing health technology assessment for priority setting and health policy in latin america: From regional endeavors to national experiences. Expert Rev Pharmacoecon Outcomes Res. 2015; 15(1):9–12.
  8. Schnipper LE, Davidson NE, Wollins DS. American society of clinical oncology statement: A conceptual framework to assess the value of cancer treatment options. J Clin Oncol. 2015; 33(23):2563–77.
  9. ICER value assessment framework. Institute for Clinical and Economic Review; 2016.
  10. NCCN clinical practice guidelines in oncology (NCCN guidelines®) with NCCN evidence blocks. National Comprehensive Cancer Network; 2016.
  11. Drug Pricing Lab Memorial Sloan Kettering. Memorial Sloan Kettering Cancer Center; 2017.
  12. Chandra A, Shafrin J, Dhawan R. Utility of cancer value frameworks for patients, payers, and physicians. JAMA. 2016; 315(19):2069–70.
  13. International myeloma foundation (IMF) says controversial drug cost report could limit patient access to optimal treatment. International Myeloma Foundation. 2016.
  14. Porter ME. What is value in health care? N Engl J Med. 2010; 363(26):2477–81.
  15. Zafar SY, Alexander SC, Weinfurt KP, Schulman KA, Abernethy AP. Decision making and quality of life in the treatment of cancer: A review. Support Care Cancer. 2009; 17(2):117–27.
  16. Ubel PA. Beyond costs and benefits: Understanding how patients make health care decisions. The Oncologist. 2010;15.
  17. Hajjaj F, Salek M, Basra M, Finlay A. Non-clinical influences on clinical decision-making: A major challenge to evidence-based practice. Journal of the Royal Society of Medicine. 2010; 103(5):178–87.
  18. Epstein RM, Street RL. The values and value of patient-centered care. Ann Fam Med. 2011; 9(2).
  19. Havrilesky LG, Alvarez Secord A, Ehrisman JA, et al. Patient preferences in advanced or recurrent ovarian cancer. Cancer. 2014; 120(23):3651–9.
  20. Voice of the patient: Lung cancer. Food and Drug Administration; 2013.
  21. Centers for Medicare & Medicaid Services. Medicare program; part B drug payment model: A proposed rule by the centers for medicare & medicaid services on 03/11/2016. Centers for Medicare & Medicaid Services. 2016;42.
  22. Whitty JA. An international survey of the public engagement practices of health technology assessment organizations. Value Health. 2013;16(1):155-163.
  23. Wortley S, Wale J, Grainger D, Murphy P. Moving beyond the rhetoric of patient input in health technology assessment deliberations. Aust Health Rev. 2016.
  24. Abelson J, Wagner F, DeJean D, et al. PUBLIC AND PATIENT INVOLVEMENT IN HEALTH TECHNOLOGY ASSESSMENT: A FRAMEWORK FOR ACTION. Int J Technol. Assess Health Care. 2009:1-9.
  25. Boutin M. Cost of Care or Value of Care? Ask the Patient. Morning Consult. 2016.
  26. Perfetto EM, Balch A, Sennet C, Pearson S The Patient Voice in Value: The National Health Council Patient-Centered Value Model Rubric. Washington, DC: National Health Council; 2016.
  27. NCCN patient advocacy summit value in cancer care – patient perspectives. National Comprehensive Cancer Network. 2015.
  28. Thiboldeaux K. Value for patients? Huffington Post. 2014.
  29. A patient guide to open input. Institute for Clinical and Economic Review (ICER). 2016.
  30. Schnipper LE, Davidson NE, Wollins DS, et al. Updating the american society of clinical oncology value framework: Revisions and reflections in response to comments received. J Clin Oncol. 2016.
  31. Staley K, Doherty C. It’s not evidence, it’s insight: Bringing patients’ perspectives into health technology appraisal at NICE. Research Involvement and Engagement. 2016; 2(1):1-12.
  32. National Health Council. Session: The patient voice in assessing value. The National Health Council Washington Representatives Retreat. December 2, 2015. Annapolis, MD.
  33. Perfetto E, Oehrlein EM, M-CERSI Conference on Patient-Focused Drug Development Planning Committee. Assessing meaningful patient engagement in drug development: A definition, framework, and rubric. 2015.
  34. Engagement rubric. Patient-Centered Outcomes Research Institute (PCORI). 2015.
  35. FH foundation welcomes the opportunity to comment on ICER’s draft report on effectiveness, value, and pricing benchmarks for PCSK9 inhibitors for high cholesterol, specifically as it relates to the population affected by familial hypercholesterolemia (FH). FH Foundation. 2015.
  36. Patient-centered value model rubric released: National health council shares guidance for assessing the value of new treatments. The National Health Council. 2016.
  37. Crossing the Quality Chasm: A New Health System for the 21st Century. Institute of Medicine (US) Committee on Quality of Health Care in America. 2001.
  38. Beal A, Sheridan S. Understanding the patient and family engagement rubric. Patient Centered Outcomes Research Institute. 2014.
  39. Patrick DL, Burke LB, Gwaltney CJ, et al. Content validity–establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: Part 1–eliciting concepts for a new PRO instrument. Value Health. 2011; 14(8):967-977.
  40. ICER to review treatments for rheumatoid arthritis and psoriatic arthritis. The Institute for Clinical and Economic Review (ICER). 2016.
  41. Perfetto EM, Burke L, Oehrlein EM, Epstein RS. Patient-focused drug development: A new direction for collaboration. Med Care. 2015; 53(1):9-17.
  42. What we mean by engagement. Patient-Centered Outcome Research Institute (PCORI). 2015.
  43. Hoos A, Anderson J, Boutin M, et al. Partnering with patients in the development and lifecycle of medicines: A call for action. Ther Innov Regul Sci. 2015; 49(6):929-939.
  44. Perfetto EM, Balch A, Sennet C, Pearson S. The Patient Voice in Value: The National Health Council Patient-Centered Value Model Rubric. Washington, DC: National Health Council; 2016.