Skip to main content

Handbook of eHealth Evaluation: An Evidence-based Approach: Chapter 3: Clinical Adoption Framework

Handbook of eHealth Evaluation: An Evidence-based Approach
Chapter 3: Clinical Adoption Framework

  • Show the following:

    Annotations
    Resources

  • Adjust appearance:

    Font
    Font style
    Color Scheme
    Light
    Dark
    Annotation contrast
    Low
    High
    Margins
  • Search within:
    • Notifications
    • Privacy
  • Project HomeHandbook of eHealth Evaluation: An Evidence Based Approach
  • Projects
  • Learn more about Manifold

Notes

table of contents
  1. Cover
  2. Half Title Page
  3. Title and Copyright
  4. Contents
  5. List of Tables and Figures
  6. Preface
  7. Acknowledgements
  8. Introduction
    1. What is eHealth? by Francis Lau, Craig Kuziemsky
    2. What is eHealth Evaluation? by Francis Lau, Craig Kuziemsky
    3. What is in this Handbook? by Francis Lau, Craig Kuziemsky
  9. Part I: Conceptual Foundations
    1. 1. Need for Evidence, Frameworks and Guidance by Francis Lau
    2. 2. Benefits Evaluation Framework by Francis Lau, Simon Hagens, Jennifer Zelmer
    3. 3. Clinical Adoption Framework by Francis Lau, Morgan Price
    4. 4. Clinical Adoption Meta-Model by Morgan Price
    5. 5. eHealth Economic Evaluation Framework by Francis Lau
    6. 6. Pragmatic Health Information Technology Evaluation Framework by Jim Warren, Yulong Gu
    7. 7. Holistic eHealth Value Framework by Francis Lau, Morgan Price
  10. Part II: Methodological Details
    1. 8. Methodological Landscape for eHealth Evaluation by Craig Kuziemsky, Francis Lau
    2. 9. Methods for Literature Reviews by Guy Paré, Spyros Kitsiou
    3. 10. Methods for Comparative Studies by Francis Lau, Anne Holbrook
    4. 11. Methods for Descriptive Studies by Yulong Gu, Jim Warren
    5. 12. Methods for Correlational Studies by Francis Lau
    6. 13. Methods for Survey Studies by Francis Lau
    7. 14. Methods for eHealth Economic Evaluation Studies by Francis Lau
    8. 15. Methods for Modelling and Simulation Studies by James G. Anderson, Rong Fu
    9. 16. Methods for Data Quality Studies by Francis Lau
    10. 17. Engaging in eHealth Evaluation Studies by Craig Kuziemsky, Francis Lau
  11. Part III: Selected eHealth Evaluation Studies
    1. 18. Value for Money in eHealth: Meta-synthesis of Current Evidence by Francis Lau
    2. 19. Evaluation of eHealth System Usability and Safety by Morgan Price, Jens Weber, Paule Bellwood, Simon Diemert, Ryan Habibi
    3. 20. Evaluation of eHealth Adoption in Healthcare Organizations by Jim Warren, Yulong Gu
    4. 21. Evaluation of Picture Archiving and Communications Systems by Don MacDonald, Reza Alaghehbandan, Doreen Neville
    5. 22. Evaluation of Provincial Pharmacy Network by Don MacDonald, Khokan C. Sikdar, Jeffrey Dowden, Reza Alaghehbandan, Pizhong Peter Wang, Veeresh Gadag
    6. 23. Evaluation of Electronic Medical Records in Primary Care: A Case Study of Improving Primary Care through Information Technology by Lynne S. Nemeth, Francis Lau
    7. 24. Evaluation of Personal Health Services and Personal Health Records by Morgan Price, Paule Bellwood, Ryan Habibi, Simon Diemert, Jens Weber
    8. 25. Evaluating Telehealth Interventions by Anthony J. Maeder, Laurence S. Wilson
  12. Part IV: Future Directions
    1. 26. Building Capacity in eHealth Evaluation: The Pathway Ahead by Simon Hagens, Jennifer Zelmer, Francis Lau
    2. 27. Future of eHealth Evaluation: A Strategic View by Francis Lau
  13. Glossary
  14. About the Contributors

Chapter 3
Clinical Adoption Framework
Francis Lau, Morgan Price
3.1 Introduction
In 2006, Canada Health Infoway published the Benefits Evaluation (BE) Framework that was adapted from the Information System (IS) Success Model by DeLone and McLean (as cited in Lau, Hagens, & Muttitt, 2007). The BE Framework provides a conceptual model for understanding the quality, use and net benefits of eHealth adoption in healthcare organizations. The BE Framework has been well received by the healthcare community because it “made sense” as an organizing scheme when describing eHealth adoption and evaluation. However, the original IS Success Model was based on a stable business IS environment and did not take into account the organizational and social contexts. In 2009, we extended the BE Framework by incorporating a set of meso- and macro-level factors that could influence the success of eHealth systems (Lau, 2009). The extensions have led to the Clinical Adoption (CA) Framework described here.
This chapter describes the conceptual foundations of the CA Framework and the micro, meso and macro dimensions that made up this framework. We then describe the validation and use of this framework, and its implications on eHealth evaluation for healthcare organizations.
3.2 Conceptual Foundations
The CA Framework is built on theories and models from the disciplines of information systems, organization science, and health informatics. They include: the Information Technology Interaction Model by Silver, Markus, and Beath (1995); the Unified Theory of Acceptance and Use of Technology Model by Venkatesh, Morris, Davis, and Davis (2003); earlier work in implementation research by Cooper and Zmud (1990); task-technology fit by Goodhue and Thompson (1995) and Ammenwerth, Iller, and Mahler (2006); managing change and risks by Kotter and Schlesinger (1979) and Paré, Sicotte, Jaana, and Girouard (2008); and the people and socio-organizational aspects of eHealth by Berg, Aarts, and van der Lei (2003), Kaplan, Brennan, Dowling, Friedman, and Peel (2001), Kaplan and Shaw (2004), and Stead and Lorenzi (1999). These published sources are described below.
3.2.1 Information Technology Interaction Model
The Information Technology Interaction Model, or ITIM, was introduced by Silver, Markus, and Beath in 1995 as a teaching model for Master of Business Administration (MBA) students. The model describes the effects of an information system interacting on an organization over time. There are four interrelated dimensions in ITIM: the information system, implementation process, organizational context, and the system’s effects (Figure 3.1). Each of these dimensions is represented by a set of components and subcomponents, which are summarized below.
  • Information system – functionality, interface, restrictiveness, guidance, and decision-making
  • Implementation process – initiation, build/buy, introduction, and adaptation
  • Organizational context – firm’s structure, processes, strategies, culture, IT infrastructure, and external environment, more specifically:
  • Structure – de/centralization, functional/divisional/network, reporting relationships
  • Processes – order fulfillment, materials acquisition, product devel­opment
  • Strategies – differentiation, low-cost production, quality/service, right-sizing, just-in-time
  • Culture – artefacts, shared values, assumptions, individuality/teamwork, risk handling
  • IT infrastructure – hardware, software, databases, networks, training, personnel, skills
  • External environment – industry structure, competition, buyer/seller power, growth
  • System’s effects – use, consequences and adaptations, more specifically:
  • Use – whether the system is used or not, how it is used, by whom, and for what purpose
  • Consequences – performance effects such as profits, effects on people such as power and role, and future flexibility for the organization
  • Adaptations – feedback effects on the organization from performance, people, and flexibility
Since its publication in 1995, the ITIM has been applied and cited in many studies related to IS. One application is to use the ITIM’s organization, implementation and effect dimensions as a conceptual scheme to critique, refine and develop additional Information Technology (IT) or IS theories and models. For instance, in his re-specification of DeLone and McLean’s IS Success Model, Seddon (1997) argued the ITIM system’s effects on use and consequences are similar to the DeLone and McLean model’s net benefits, and that the greater IS use implied more consequences. Kohli and Limayen (2006) and Tams (2011) applied the ITIM as a foundational model to justify the legitimacy of IS as a reference discipline through its theoretical and methodological contributions in the areas of IS development, implementation, innovation, and business value. In healthcare, Ben-Zion, Pliskin, and Fink (2014) applied the ITIM dimensions in a literature review and prescriptive analysis to identify a set of critical success factors for the adoption of EHR systems.

eHealth_Figure_3-01.jpg

Figure 3.1. IT interaction model.
Note. From “The information technology interaction model: A foundation for the MBA core course,” by M. S. Silver, M. L. Markus, and C. M. Beath, 1995, Management Information Systems Quarterly, 19(3), p. 366. Copyright 1995 by Regents of the University of Minnesota. Reprinted with permission.
3.2.2 Technology Acceptance Models
The original Technology Acceptance Model (TAM) by Davis (1989) and its variants (e.g., TAM2) published over the years are considered the most widely applied theory on an individual’s acceptance of technology (Lee, Kozar, & Larsen, 2001; Yarbrough & Smith, 2007). In 2003, Venkatesh et al. published the Unified Theory of Acceptance and Use of Technology (UTAUT) Model based on a synthesis of eight TAM-related models. The UTAUT combined the best features from these models and has emerged as one of the most widely cited models on technology acceptance. The UTAUT has four attributes that are considered the direct determinants of technology use intention and/or behaviour: performance expectancy, effort expectancy, social influence, and facilitating conditions (i.e., the perceived technical and organizational infrastructure in place to support IS use). There are also four other attributes that have a moderating effect on the direct determinants with respect to their influence on technology use intention and/or behaviour: gender, age, voluntariness, and experience. The UTAUT Model is shown in Figure 3.2.

eHealth_Figure_3-02.jpg
Figure 3.2. Unified theory of acceptance and use of technology.  
Note. From “User acceptance of information technology: Toward a unified view,” by V. Venkatesh, M. G. Morris, G. B. Davis, and F. D. Davis, 2003, Management Information Systems Quarterly, 27(3), p. 447. Copyright 2003 by Regents of the University of Minnesota. Reprinted with permission.
Since its publication, the UTAUT Model has been applied in different healthcare settings to determine the acceptance of eHealth systems by care providers. For example, survey-based studies have examined the key organizational characteristics for successful telemedicine programs (Whitten, Holtz, & Nguyen, 2010), the factors that influence user acceptance of a hospital picture archiving and communication system (Duyck et al., 2008), acceptance of EMR systems by nurses, physician assistants, and nurse practitioners at the state level (Wills, El-Gayar, & Bennett, 2008), and perceptions of two outpatient electronic prescribing systems for primary care (Wang et al., 2009). Thus far, the UTAUT Model and its survey instrument have proved to be robust, valid and reliable when used in healthcare settings.
3.2.3 Implementation Research and Managing Change
There has been a significant amount of work done in IS implementation research regarding the theories, methods, processes and implications of IS implementation in organizations (e.g., Kukafka, Johnson, Linfante, & Allegrante, 2003). Examples are the technological diffusion approach by Cooper and Zmud (1990) and the improvisational model for change by Orlikowski and Hofman (1997). Of particular interest is the work on task-technology fit by Goodhue and Thompson (1995) and Ammenwerth et al. (2006) that focused on the relationships between an individual’s performance and his or her technology-enabled work. The importance of managing organizational change and its effects on IS implementation has also been recognized (e.g., Lorenzi, 2000; Iles & Sutherland, 2001).
The organizational change model by Kotter (2007) and the project risk assessment framework by Paré et al. (2008) are examples of practice-based change management approaches applied to ensure successful IS implementation. To transform an organization, Kotter emphasized the need for a sense of urgency, a powerful guiding coalition, a communicated vision empowering those to act on the vision, focusing on short-term wins, consolidating improvement to produce more change, and institutionalizing the new approach. Similarly, Paré and colleagues offered a systematic approach to ensuring successful IS implementation by reducing risks along the technological, human, usability, managerial, strategic, and political dimensions.
3.2.4 People and Socio-organizational Aspects
In health informatics there has been a shift from a technical focus on the deployment of local eHealth systems to a broader focus of sociotechnical systems with the emphasis on people, organizational and social issues. In 1999, Stead and Lorenzi (1999) suggested the health informatics agenda should “acknowledge the foundation provided by the health system … the role of financial issues, system impediments, policy and knowledge in effecting change” (p. 341). Similarly, Kaplan and colleagues (2001) outlined an informatics research agenda that involved the use of different social inquiry methods depending on settings at the individual, institutional, trans-organizational and transnational levels. Kaplan and Shaw (2004) further outlined the directions for informatics evaluation to include the reshaping of institutional boundaries, changing work practices and standards, the politicization of healthcare, and changing roles for providers and consumers. The sociotechnical approaches advocated by Berg et al. (2003) also emphasized the social nature of healthcare work that can influence the success of eHealth systems, including meso- and macro-level processes such as the financial status of the organization, jurisdictional healthcare policy, and politics at both the institutional and national levels.
3.3 CA Framework Dimensions
The CA Framework has three conceptual views of eHealth adoption by clinicians in different settings (Lau, Price, & Keshavjee, 2011). These are the micro-⁠, meso- and macro-level views of clinical adoption. They are described below.
  • The micro level addresses the quality of the information, system and service associated with an eHealth system, its use and user satisfaction, and net benefits in terms of care quality, productivity and access. These are the same dimensions and categories that are defined in the BE Framework.
  • The meso level addresses the people, organization and implementation dimensions that have a direct effect on the micro level eHealth adoption by clinicians. The people dimension is drawn from the constructs in the UTAUT, while the organization and implementation dimensions are from the ITIM, implementation research, and change management models described earlier.
  • The macro level addresses healthcare governance, standards, funding, and societal trends as the environmental factors that have direct influence on the extent to which the meso level can affect clinical adoption at the micro level. These macro-level factors are based on the sociotechnical approaches that transcend organizations to include overall societal trends.
  • At each level there is a feedback loop where the adoption efforts and results can reshape the higher levels. The CA Framework is shown in Figure 3.3 and the three views are elaborated next. The CA categories, subcategories and measures are summarized in the Appendix following the References section.
eHealth_Figure_3-03.jpg
Figure 3.3. Clinical adoption framework with its micro, meso and macro dimensions.
Note.From “From benefits evaluation to clinical adoption: Making sense of health information system success in Canada,” by F. Lau, M. Price, and K. Keshavjee, 2011, Healthcare Quarterly, 14(1), p. 41. Copyright 2011 by Longwoods™ Publishing Corp. Reprinted with permission.
3.3.1 Micro Level
At the micro level, our proposition is that successful clinical adoption of an eHealth system depends on its HIT quality, usage quality and net benefits. These are elaborated below.
  • HIT Quality refers to the accuracy, completeness and availability of the clinical information content of an eHealth system; the features, performance and security of the system; and responsiveness of the system’s support services.
  • Usage Quality refers to eHealth system usage intention/pattern; and user satisfaction in terms of usefulness, ease-of-use and competency.
  • Net Benefits refer to changes in care quality, access and productivity as a result of eHealth adoption by clinicians. Care quality covers patient safety, appropriateness/effectiveness and health outcomes. Access covers provider/patient participation and availability/access to services. Productivity covers care coordination, efficiency and net cost.

Our rationale is that the better the quality of the eHealth system adopted, the more it will be embraced by satisfied clinicians, leading to greater tangible net benefits over time.
3.3.2 Meso Level
At the meso level, our proposition is that successful clinical adoption depends on the people, organization and implementation process. These are elaborated below.
  • People refers to all types of individuals or groups in the healthcare system having to do with eHealth in some way, their personal characteristics and expectations, as well as their roles and responsibilities within the eHealth system.
  • Organization refers to how the system fits with the organization’s strategy, culture, structure/processes, information infrastructure and return on value.
  • Implementation refers to the eHealth adoption stages, project management approaches, and the extent of eHealth-practice fit planned in the future and operating at present.

Our rationale is that higher eHealth adoption can occur in the organization if clinicians have experience and clear expectations in using the system. Moreover, the system will be seen as adding value if it is designed to support organizational performance goals. To do so, the implementation process must be carefully planned, executed and managed throughout its life cycle. This ensures the eHealth system fits into the day-to-day work practices of clinicians. When these meso-level factors are aligned with those at the micro level, we can expect further magnified improvements in eHealth system quality, usage and net benefits.
3.3.3 Macro Level
At the macro perspective, our proposition is that successful clinical adoption depends on the environmental contexts with respect to governance, standards, funding and trends. These are elaborated below.
  • Governance refers to the influence of governing bodies, legislative acts, and the regulations or policies covering such bodies as professional associations/colleges, advocacy groups and their attitudes toward eHealth.
  • Standards refer to the types of eHealth, organizational performance and professional practice standards in place.
  • Funding refers to the payment, remuneration, and incentive programs in place.
  • Trends refer to public expectations, and the overall socio-political and economic climates toward technologies, eHealth and health care as a whole.  

Our rationale is that higher eHealth adoption by clinicians can be achieved if the organization aligns its effort with the macro environmental factors that influence clinical adoption. For instance, organizations should embrace eHealth systems that conform to industry-wide interoperable standards, help achieve external performance targets, and adapt to the changing scope of professional practice in care delivery. Where feasible, organizations should take advantage of incentives that encourage clinical adoption such as subsidized eHealth system deployment and automated patient safety surveillance. Adhering to established health information protection legislations, policies and practices with strong governance involving multiple stakeholders can further enhance clinical adoption through trust and relationship building. Lastly, staying abreast of the socio-political and economic trends — such as encouraging citizens to better manage their own health through the use of personal health records — allows the organization to be proactive in its eHealth planning and deployment efforts.
3.4 CA Framework Usage
3.4.1 Validation of the CA Framework
The CA Framework underwent three validation steps when it was introduced. First was a comparison of the framework elements (i.e., dimensions, categories and measures) against those identified in a meta-review of eHealth evaluation systematic reviews (Lau, Price, Kuziemsky, & Gardner, 2010). Second was a consultation session with Canadian eHealth practitioners to determine if they agreed with the framework elements (Lau & Charlebois, 2009). Third was a comparison against the questions/measures used in survey instruments of published eHealth adoption and evaluation studies (Oh, 2009). The three steps are summarized below.
  • In a meta-review of 50 eHealth evaluation systematic reviews published between 1995 and 2008, Lau et al. (2010) were able to map most of the evaluation measures from the reviews to the micro-level dimensions of the CA Framework. They also identified measures that did not fit the micro level and created new categories for them which were patient/provider, implementation, incentive, policy/legislation, change improvement, and interoperability. These factors mapped nicely under the meso- and macro-level dimensions of the CA Framework.
  • In 2009 Infoway held a consultation session with 23 eHealth practitioners from across Canada that provided their anonymized written feedback on the CA Framework. The practitioners responded to questions on whether the framework made sense, whether concepts were missing or required revisions, as well as their interest in, and the effort needed to apply the framework in their organizations. Based on their feedback, revisions were made to streamline the framework, for example by dropping the network dimension and making the people dimension more prominent (Lau & Charlebois, 2009).
  • Oh (2009) compared the CA Framework elements against 16 published survey instruments. They included 13 instruments from the Health IT Survey Compendium section of the Agency for Healthcare Research & Quality (AHRQ) Health IT website (AHRQ, 2010) and three from Canada Health Infoway. Of the 16 instruments examined, only the Infoway System and Use Assessment Survey items mapped to all 20 micro-level elements. At the meso level the 16 instruments mapped between 0 and 11/12 of the elements. At the macro level they mapped poorly from 0 to 5/12 elements. No question items were found missing from the framework, which suggested it was sufficiently comprehensive for all areas of eHealth.
3.4.2 Use of the CA Framework
The CA Framework provides an overarching conceptual model that makes sense of eHealth adoption by clinicians. Healthcare organizations involved with eHealth adoption should address as needed the micro-, meso- and macro-level factors described in this framework to achieve eHealth success. Given the large number of factors that affect clinical adoption, an organization should focus on a subset of these factors when evaluating its eHealth adoption effort and impacts. To apply the CA Framework, one needs different methods and tools to evaluate whether the factors are associated with the extent of adoption and impacts desired and/or achieved. Examples of evaluation methods that can be applied before, during and after adoption of an eHealth system are the Infoway System and Use Assessment (SUA) survey and the Rapid Response Evaluation Methods (RREM) from the eHealth Observatory (Lau, 2010). The RREM is made up of a suite of evaluation tools for conducting usability, workflow, system/data quality and impact studies, and practice reflections for different implementation stages. Depending on need, other evaluation methods can be applied to examine particular aspects of clinical adoption in specific settings.
To illustrate, an organization in the process of implementing a picture archiving and communication system (PACS) may wish to focus on specific micro-level factors in the CA Framework by examining the extent to which the quality of the PACS, its perceived usefulness, and actual system usage can affect the productivity of the clinicians and their workflow coordination. By conducting the SUA survey and RREM workflow analysis before and after PACS deployment, one can compare the extent of work practice change brought on by the system. On the other hand, an organization with a suite of existing eHealth systems such as order entry or lab and pharmacy systems may focus on specific meso-level people and organization factors to improve their clinical adoption. By conducting the RREM impact assessment surveys, one can identify areas that require attention such as the extent of eHealth alignment with the organization’s strategy, technical infrastructures and clinician expectations. Lastly, a jurisdiction wishing to evaluate the success of its primary healthcare EMR strategy may apply the RREM reactive analysis to see if the macro-level factors are adequately addressed. These may include EMR alignment with industry-wide eHealth standards, professional practice scope, medical service fee schedule, privacy legislations for patient record exchange, and societal expectations of value for money in EMR investments.
Since its debut in 2011, the CA Framework has been applied, adapted and cited in over 30 studies and publications. Examples where the CA Framework was applied are the ambulatory care clinic EMR evaluation study in a British Columbia health region by Lau, Partridge, Randhawa, and Bowen (2013) and a fuzzy modelling study to identify key meso-level factors for successful EMR adoption in eight Malaysian primary care clinics (Ahmadi et al., 2013). There are also two literature reviews where the CA Framework was applied as a conceptual scheme to organize the review findings (Lau, Price, Boyd, Partridge, Bell, & Raworth, 2012; Bassi, Lau, & Lesperance, 2012). In a coordinated Canadian EHR strategy white paper, Lau, Price, and Bassi (2014) adapted the CA Framework as a new eHealth Value Framework by expanding the investment, value and lag time aspects of eHealth adoption. In Finland, the National Institute for Health and Welfare incorporated the meso- and macro-level dimensions of the CA Framework into its eHealth Evaluation Framework to assess health information system implementation at the national level (Hypponen et al., 2011). See Table 3.1 for examples of studies where the CA Framework has been applied.
The CA Framework has been cited in different publications related to eHealth strategy, adoption and evaluation by health informaticians in several countries. For example, Axelsson and Melin (2014) acknowledged the importance of context when identifying critical success factors in Swedish eHealth systems. Yusof, Khodambashi, and Mokhtar (2012) cited the need to consider HIT-practice fit (part of the meso dimension in the CA Framework) as part of their lean method to study the implementation of a critical care information system in Malaysia. Similarly, Viitanen and colleagues (2011) emphasized the need to examine the contextual aspect of usability (i.e., eHealth-practice fit) when evaluating Finnish clinical IT systems. In their study of clinical governance and EMR adoption in the Australian primary care setting, Pearce, de Lusignan, Phillips, Hall, and Traveglia (2013) identified similar meso- and macro-level factors from the CA Framework that influenced EMR acceptance.  

eHealth_Table_3.1-1.jpgeHealth_Table_3.1-2.jpg

The CA Framework has also been cited in a number of graduate student theses related to eHealth. Examples include the study of EMR data quality and payment incentives in primary care (Bowen, 2013), the meaningful use in primary care EMRs (Watt, 2014), a review of health information exchanges’ success factors (Ng, 2012), an evaluation of multidisciplinary cancer care conference platforms (Ghaznavi, 2012), end user support for primary care EMRs (Dow, 2012), and critical success factors for Malaysian public hospital information systems (Abdullah, 2013).
3.5 Implications
The current CA Framework requires further work to improve its validity, relevance and utility. Some of the meso- and macro-level factors in the framework need to be refined as specific measures that can be applied and quantified in field settings. In particular, evaluation methods that measure specific factors in the CA Framework are needed in order for it to be applied more widely across different types of eHealth systems and organizational settings. Additional methods and tools are also required to evaluate factors that are not currently addressed, especially in the areas of health outcomes at the micro level, return on value at the meso level, and governance, funding and standards at the macro level.
Despite the limitations, it is important to keep in mind that to make major strides forward with clinical adoption of eHealth systems, healthcare organizations need to share a common vision of what constitutes eHealth success. The CA Framework provides a common ground by which eHealth adoption by clinicians can be described, measured, compared and aggregated as empirical evidence over time.
3.6 Summary
This chapter described the CA Framework for determining eHealth success. It is an extension of the BE Framework that takes into account the contextual factors involved. The CA Framework has three conceptual dimensions at the micro, meso and macro levels. Each dimension has its own set of factors that define eHealth success. The CA Framework has undergone an initial validation, and has been proposed as an overarching framework to plan, conduct and report eHealth evaluation studies. The advantage of having a common evaluation framework is the ability to measure, compare and aggregate eHealth evidence in a consistent manner across different eHealth systems and healthcare settings.
References
Abdullah, Z. S. (2013). Hospital information systems implementation framework: Critical success factors for Malaysian public hospitals (Doctoral thesis, Curtin University, Perth Australia). Retrieved from http://espace.library.curtin.edu.au/R?func=dbin-jump-full&local_base=gen01-era02&object_id=192723.
Agency for Healthcare Research and Quality (AHRQ). (2010). Health IT survey compendium. Rockville, MD: Author. Retrieved from https://healthit.ahrq.gov/health-it-tools-and-resources/health-it-survey-compendium
Ahmadi, H., Nilashi, M., Darvishi, M., Ibrahim, O., Zakaria, R., Zolghadri, H., & Alizadeh, M. (2013). Fuzzy multi-criteria approaches for evaluating the critical factors of electronic medical record adoption. Review of Contemporary Business Research, 3(2), 1–24.
Ammenwerth, E., Iller, C., & Mahler, C. (2006). IT-adoption and the interaction of task, technology and individuals: A fit framework and a case study. BMC Medical Informatics and Decision Making, 6(3), 1–13.
Axelsson, K., & Melin, U. (2014). Contextual factors influencing health information systems implementation in public sector: Investigating the explanatory power of critical success factors. In M. Janssen, H. J. Scholl, M. A. Wimmer, & F. Bannister (Eds.), Proceedings of the 13th Conference on Electronic Governance, Dublin, Ireland, September 1 to 3,lecture notes in computer science (pp. 59–71). Laxenburg, Austria: International Federation for Information Processing.
Bassi, J., Lau, F., & Lesperance, M. (2012). Perceived impact of electronic medical records in physician office practices: A review of survey-based research. Interactive Journal of Medical Research, 1(2), e3.1–e3.23.
Ben-Zion, R., Pliskin, N., & Fink, L. (2014). Critical success factors for adoption of electronic health record systems: Literature review and prescriptive analysis. Information Systems Management, 31(4), 296–312.
Berg, M., Aarts, J., & van der Lei, J. (2003). ICT in health care: sociotechnical approaches. Methods of Information in Medicine, 42(4), 287–301
Bowen, M. (2013). Understanding, evaluating and enhancing EMR adoption in a primary care setting (Unpublished master’s thesis). University of Victoria, Victoria, BC, Canada.
Cooper, R. B., & Zmud, R. W. (1990). Information technology implementation research: A technological diffusion approach. Management Science, 36(2), 123–139.
Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. Management Information Systems Quarterly, 13(3), 319–339.
Dow, R. (2012). The impact of end-user support on EMR success in Ontario primary care: A critical case study (Unpublished master’s thesis). University of Toronto, Toronto, ON, Canada.
Duyck, P., Pynoo, B., Devolder, P., Voet, T., Adang, L., & Vercruysse, J. (2008). User acceptance of a picture archiving and communication system. Methods of Information in Medicine, 47(2), 149–156.
Ghaznavi, F. (2012). Design and evaluation of a multidisciplinary cancer conferencing platform (Unpublished master’s thesis). Chalmers University of Technology, Göteborg, Sweden.
Goodhue, D. L., & Thompson, R. L. (1995). Task-technology fit and individual performance. Management Information Systems Quarterly,19(2), 213–⁠236.
Hypponen, H., Viitanen, J., Reponen, J., Doupi, P., Jormanainen, V., Lääveri, T., … Hämäläinen, P. (2011). Large-scale eHealth systems: Providing information to support evidence-based management. In L. Van Gemert-Pijnen, H. C. Ossebaard, & P. Hämäläinen (Eds.), eTELEMED: 3rd International Conference on eHealth, Telemedicine, and Social Medicine, February 23 to 28, Gosier, Guadeloupe, France (pp. 89–95).
Iles, V., & Sutherland, K. (2001). Organizational change: A review for healthcare managers, professionals and researchers. London: National Coordinating Centre for National Health Service Delivery and Organization R&D.
Kaplan, B., Brennan, P. F., Dowling, A. F., Friedman, C. P., & Peel, V. (2001). Toward an informatics research agenda: Key people and organizational issues. Journal of American Medical Informatics Association,8(3), 235–⁠241.
Kaplan, B., & Shaw, N. T. (2004). Future directions in evaluation research: People, organizational and social issues. Methods of Information in Medicine, 43(3), 215–231.
Kohli, R., & Limayen, M. (2006). Information systems research: Reference disciplines and theoretical contributions.  Proceedings of the Americas Conference on Information Systems (AMCIS), Acapulco, Mexico (Paper ⁠# ⁠3).
Kotter, J. P., & Schlesinger, L. A. (1979). Choosing strategies for change. Harvard Business Review, 57 (March-April), 106–114.
Kotter, J. P. (2007). Leading change: Why transformation efforts fail. Harvard Business Review, 85(1), 96–103.
Kukafka, R., Johnson, S. B., Linfante, A., & Allegrante, J. P. (2003). Grounding a new information technology implementation framework in behavioural science: A systematic analysis of the literature on IT use. Journal of Biomedical Informatics, 36(3), 218–227.
Lau, F. (2009). Extending the Infoway benefits evaluation framework for health information systems. In J. G. McDaniel (Ed.), Advances in information technology and communication in health (pp. 406–413). Amsterdam: IOS Press.
Lau, F. (2010). EMR Evaluation Toolkit v3.0. Retrieved from http://ehealth.uvic.ca/resources/tools/tools.php 
Lau, F., & Charlebois, M. M. (2009). Validating the clinical adoption framework. Clinical Adoption Forum, Nov 3, 2009 (unpublished).
Lau, F., Hagens, S., & Muttitt, S. (2007). A proposed benefits evaluation framework for health information systems in Canada. Healthcare Quarterly, 10(1), 112–118.
Lau, F., Partridge, C., Randhawa, G., & Bowen, M. (2013). Applying the clinical adoption framework to evaluate the impact of an ambulatory electronic medical record. Studies in Health Technology and Informatics, 83, 15–20.
Lau, F., Price, M., & Keshavjee, K. (2011). From benefits evaluation to clinical adoption: Making sense of health information system success in Canada. Healthcare Quarterly, 14(1), 39–45.
Lau, F., Price, M., Kuziemsky, C., & Gardner, J. (2010). A review on reviews of health information system studies. Journal of American Medical Informatics Association, 17(6), 637–645.
Lau, F., Price, M., Boyd, J., Partridge, C., Bell, H., & Raworth, R. (2012). Impact of electronic medical record on physician practice in office settings: A systematic review. BMC Medical Informatics and Decision Making, 12(10), 1–10.
Lau, F., Price, M., & Bassi, J. (2014). Toward a coordinated electronic health record strategy for Canada. Creating strategic change in Canadian healthcare. Conference Series 2, Queen’s University. Toronto, May 14-15.
Lee, Y., Kozar, K. A., & Larsen, K. R. T. (2001). The technology acceptance model: Past, present and future. Communications of the Association for Information Systems, 12, 752–780.
Lorenzi, N. (2000). Managing change: An overview. Journal of American Medical Informatics Association 7(2), 116–124.
Ng, Y. (2012). Key characteristics of health information exchange: A scoping review (Unpublished master’s thesis). University of Victoria, Victoria, BC, Canada.
Oh, J. (2009). Clinical adoption framework — Survey mapping, December 4, 2009 (Unpublished).
Orlikowski, W. J., & Hofman, J. D. (1997). An improvisational model for change management: The case of groupware technologies. Sloan Management Review, 38(2), 11–21.
Paré, G., Sicotte, C., Jaana, M., & Girouard, D. (2008). Prioritizing clinical information system project risk factors: a Delphi study. Paper presented at the Proceedings of the 21st Hawaii International Conference on System Sciences, Hawaii, January 5 to 7.
Pearce, C. M., de Lusignan, S., Phillips, C., Hall, S., & Travaglia, J. (2013). The computerized medical record as a tool for clinical governance in Australian primary care. Interactive Journal of Medical Research, 2(2), e26.
Seddon, P. B. (1997). A respecification and extension of the DeLone and McLean model of IS success. Information Systems Research, 8(3), 240–⁠253.
Silver, M. S., Markus, M. L., & Beath, C. M. (1995). The information technology interaction model: A foundation for the MBA core course. Management Information Systems Quarterly, 19(3), 361–390.
Stead, W., & Lorenzi, N. M. (1999). Health informatics: Linking investments to value. Journal of American Medical Informatics Association, 6(5), 341–⁠348.
Tams, S. (2011). On the legitimacy of IS as an independent discipline: Research that sets the field apart. Proceedings of the 14th Southern Association for Information Systems Conference (pp. 170–174). Atlanta, GA, March 25-26.
Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User acceptance of information technology: Toward a unified view. Management Information Systems Quarterly, 27(3), 425–478.
Viitanen, J., Hypponen, H., Laaveri, T., Vanska, J., Reponen, J., & Winblad, I. (2011). National questionnaire study on clinical ICT systems proofs: Physicians suffer from poor usability. International Journal of Medical Informatics, 80(10), 708–725.
Watt, R. (2014). Does customized in-practice support improve EMR meaningful use in primary care? Evidence from a retrospective mixed methods evaluation. MSc Project in Health Informatics. University of Victoria, Victoria, BC.
Wang, C. J., Patel, M. H., Schueth, A. J., Bradley, M., Wu, S., Crosson, J. C., Glassman, P. A., & Bell, D. S. (2009). Perceptions of standards-based electronic prescribing systems as implemented in outpatient primary care: A physician survey. Journal of American Medical Informatics Association, 16(4), 493–502.
Whitten, P., Holtz, B., & Nguyen, L. (2010). Keys to a successful and sustainable telemedicine program. International Journal of Technology Assessment in Health Care, 26(2), 211–216.
Wills, M. J., El-Gayar, O. F., & Bennett, D. (2008). Examining healthcare professionals’ acceptance of electronic medical records using UTAUT. Issues in Information Systems, 9(2), 396–401.
Yarbrough, A. K., & Smith, T. B. (2007). Technology acceptance among physicians: A new take on TAM. Medical Care Research Review, 64(6), 650–672.
Yusof, M. M., Khodambashi, S., & Mokhtar, A. M. (2012). Evaluation of the clinical process in a critical care information system using the Lean method: a case study. BMC Medical Informatics and Decision Making, 12(150), 1–14.
Appendix


eHealth_Table_3.2-1.jpgeHealth_Table_3.2-1.jpgeHealth_Table_3.2-1.jpg

Annotate

Next Chapter
Chapter 4: Clinical Adoption Meta-Model
PreviousNext
EPUB
Powered by Manifold Scholarship. Learn more at
Opens in new tab or windowmanifoldapp.org