Skip to main content
“Chapter 7: Holistic eHealth Value Framework” in “Handbook of eHealth Evaluation: An Evidence-based Approach”
Chapter 7
Holistic eHealth Value Framework
Francis Lau, Morgan Price
7.1 Introduction
Canadian jurisdictions have been investing in health information technology (HIT) as one strategy to address healthcare sustainability. Investments have
included the migration to electronic patient records, and the automation of
service delivery to improve the efficiency, access and quality of care
provided. In this context, eHealth emerged over 10 years ago as a shared
priority for the federal, provincial and territorial jurisdictions in their
health care renewal effort. To date, the federal government has invested over
$2 billion in Canada Health Infoway (Infoway) through incremental and targeted
funding. Provinces and territories have also invested in the cost sharing of
eHealth projects. Progress has been made towards achieving the eHealth vision.
Examples are: (a) the adoption of pan-Canadian approaches among provinces and
territories in the planning and development of common EHR architectures and standards; (b) the creation of jurisdictional registries, such
as patient and provider registries; and (c) the creation of jurisdictional
repositories of patient data, such as imaging, lab and drug information
systems.
Yet there is conflicting evidence on eHealth benefit. Some reports suggest
strong benefit while others showed few to no benefits in spite of the eHealth
investments made. For example, in their 2009-2010 performance audit reports,
the Auditor General of Canada and six provincial auditors’ offices raised questions on whether there was sufficient “value for money” from the EHR investments (e.g., Office of the Auditor General of Canada [OAG], 2010). In light of the investments made, an effort is needed to make sense of
the evidence on eHealth benefits. To do so, we created a high-level conceptual
eHealth Value Framework as an organizing scheme to examine the current evidence
on Canadian eHealth value, and the underlying reasons for the conflicting
evidence so that future eHealth investment and work is better informed.
This chapter describes a proposed holistic eHealth Value Framework to make sense
of the value of eHealth systems in the Canadian setting. The chapter contains
an overview of this framework, its use in a Canadian literature review on
eHealth value, and implications on policy and practice.
7.2 A Sense-making Scheme for eHealth Value
The proposed holistic eHealth Value Framework is described in this section in
terms of its conceptual foundations and the respective framework dimensions.
7.2.1 Conceptual Foundations
The eHealth Value Frameworkincorporates several foundational frameworks and models from the literature. The
underpinnings of this framework are the following: the Infoway Benefits
Evaluation (BE) Framework (Lau, Hagens, & Muttitt, 2007); the Clinical Adoption Framework (Lau, Price, & Keshavjee, 2011); the Clinical Adoption and Maturity Model (eHealth
Observatory, 2013); Canada’s Health Informatics Association [COACH] Canadian EMR Adoption and Maturity Model (COACH, 2013); the HIMSS EMR Adoption Model (HIMSS Analytics, 2014); Meaningful Use Criteria (Blumenthal & Tavenner, 2010); and the Information Systems Business Value Model (Schryen,
2013). By combining features of these models, this framework provides a
comprehensive view of eHealth, incorporating, for example, the EHR and its value.
7.2.2 Value Framework Dimensions
The eHealth Value Framework for Clinical Adoption and Meaningful Use (hereafter referred to as the eHealth Value Framework) describes how the value of eHealth components, such as an EHR, is influenced by the dynamic interactions of a complex set of contextual
factors at the micro, meso, and macro adoption levels. The outcomes of these
interactions are complex. The realized benefits (i.e., the value of an EHR) depend on the type of investment made, the system being adopted, the
contextual factors involved, the way these factors interact with each other,
and the time for the system to reach a balanced state. Depending on the
adjustments made to the system and the adoption factors along the way, the
behaviour of this system and its value may change over time.
Specifically, there are four interrelated dimensions that can be used to explain
the benefits of EHRs. They are: Investment, Adoption, Value, and Time. Each is made up of a set of contextual factors that interact dynamically over
time to produce specific EHR impacts and benefits (see Figure 7.1). These dimensions are described next.
Figure 7.1. A proposed holistic eHealth value framework for clinical adoption and
meaningful use.
7.2.3 Investment
Investments can be made directly towards achieving EHR adoption or indirectly to influence larger contextual factors that impact
adoption.
7.2.4 Adoption
Adoption can be considered at a micro level, consistent with the Infoway BE Framework. It also has contextual factors at the meso and macro levels, ranging
from people and organizational structures to larger standards, funding
structures, and pieces of legislation.
- Micro – The quality of the system and its use can influence the intended benefits. The technology, information, and support services provided can influence how the system performs. This can impact the actual or intended use of the system and user satisfaction. If a system does not support certain functionality (e.g., system quality), or is not used appropriately or as intended, value is not likely to be seen.
- Meso – People, organization, and implementation processes can influence the intended benefits of the system. People refer to those individuals/groups that are the intended users, their personal characteristics and expectations, and their roles and responsibilities. Organizations have strategies, cultures, structures, processes, and info/infrastructures. Implementation covers the system’s life cycle stages, its deployment planning/execution process, and the system’s fit for purpose.
- Macro – Governance, funding, standards and trends can influence the benefits. Governance refers to legislation, policies and accountability. Funding includes remunerations, incentives and added values for the system. Standards include HIT, performance, and practice standards. Trends cover the general public, political and economic investment climates toward EHR systems.
7.2.5 Value
Value of EHR is defined as the intended benefits from the clinical adoption and meaningful
use of the EHR system. Value can be in the form of improved care quality, better access, and
increased productivity affecting care processes, health outcomes, and economic
return. It can be measured by different methods and at various times in
relation to adoption.
7.2.6 Lag Time
There is an acknowledged lag time to implement and realize benefits from EHR adoption. Lag effects occur as EHR systems become incorporated into practice, where adoption factors at the micro,
meso and macro levels can all impact lag time until benefits from the adoption
are evident.
7.3 Framework Use and Implications
This section describes the use of the eHealth Value Framework to make sense of
eHealth benefit with respect to a literature review undertaken in 2014 on a set
of Canadian eHealth evaluation studies published between 2009 and 2013. Three
Canadian literature sources were included: 12 Infoway co-funded benefits
evaluation studies; 25 primary studies in peer-reviewed journals; and one
federal government auditor’s report. The systems evaluated were EHRs, drug information systems (DIS), lab information systems, diagnostic imaging (DI/PACS), ePrescribing, computerized provider order entries (CPOEs), provincial drug viewers, and physician office EMRs (Lau, Price, & Bassi, 2014).
7.3.1 Use
The eHealth Value Framework was applied to organize the review findings; eHealth
benefit was examined through the value dimensions of care process, health
outcomes, and economic return. Factors that influence adoption were examined at
the micro, meso and macro level of the adoption dimension. Of the 38 Canadian
studies reviewed, 21 had reported benefit findings, 29 had reported adoption
factors, and 21 had evaluated and reported on the adoption factors. Of the 21
studies on benefit, there was a combination of positive, mixed, neutral and
negative benefits reported (see Figure 7.2). Overall, there appears to be a
small but growing body of evidence on the adoption, impact and value of eHealth
systems in Canada. These benefits are summarized below according to the value
dimension of the framework.
Figure 7.2. Summary of eHealth value findings from Canadian studies.
- Care Process – Most of the studies reported benefits in care process (actual or perceived improvements). These care processes involved activities that could improve patient safety (Tamblyn et al., 2010; Geffen, 2013), guideline compliance (Holbrook et al., 2009; PricewaterhouseCoopers [PwC], 2013; Geffen, 2013), patient/provider access to services (Geffen, 2013; Prairie Research Associates [PRA], 2012), patient-provider interaction (Holbrook et al., 2009; Centre for Research in Healthcare Engineering [CRHE], 2011), productivity/efficiency (Prince Edward Island [P.E.I.] Department of Health and Wellness, 2010; Paré et al., 2013; CRHE, 2011; Lapointe et al., 2012; Syed et al., 2013), and care coordination (Paré et al., 2013; PwC, 2013; Lau, Partridge, Randhawa, & Bowen, 2013). There were also some negative impacts which included poor EMR data quality that affected drug-allergy detection (Lau et al., 2013), perceived inability of the EMR to facilitate decision support (Paré et al., 2013), increased pharmacist callback in ePrescribing (Dainty, Adhikari, Kiss, Quan, & Zwarenstein, 2011), and reduced ability of a DIS to coordinate care and share information (P.E.I. Department of Health and Wellness, 2010).
- Health Outcomes – The overall evidence on health outcome benefits is smaller and is more mixed. Two controlled DIS studies reported improved patient safety with reduced inappropriate medications (Dormuth, Miller, Huang, Mamdani, & Juurlink, 2012) and errors (Fernandes et al., 2011), while a third study reported low accuracy of selected medications in a provincial medication dispensing repository (Price, Bowen, Lau, Kitson, & Bardal, 2012). On the other hand, two descriptive studies reported user expectations of improved compliance and reduced adverse events with full DIS adoption and use. For EMR, Holbrook et al. (2009) reported improved A1c and blood pressure control levels, while Paré et al. (2013), PwC (2013) and Physician Information Technology Office [PITO] (2013) all reported expectations of improved safety from the EMR. At the same time, PITO (2013) reported that less than 25% of physicians believed EMR could enhance patient-physician relationships and Paré et al. (2013) reported few physicians believed EMR could improve screening. For ePrescribing and CPOE there were no improved outcomes in patient safety reported (Tamblyn et al., 2010; Dainty et al., 2011; Lee et al., 2010).
- Economic Return – The overall evidence on economic return is also mixed. For EMR, O’Reilly, Holbrook, Blackhouse, Troyan, and Goeree, (2012) reported a positive return on diabetes care from Holbrook et al.’s original 2009 RCT study that showed an improved health outcome of 0.0117 quality-adjusted life years with an incremental cost-effectiveness ratio of $160,845 per quality-adjusted life year. PRA (2012) reported mixed returns where the screening of breast and colorectal cancers was cost-effective but not in cervical cancer. In Paré et al.’s (2013) survey less than 25% of Quebec physicians reported direct linkage between the EMR and financial health of their clinics. The PITO (2013) survey also reported that less than 25% of British Columbia physicians believed EMR could reduce overall office expenses. The PwC study (2013) estimated the combined economic return from productivity and care quality improvements to be $300 million per year with full EMR adoption and use. For DI/PACS, MacDonald and Neville (2010) reported a negative return of the P.E.I. PACS system from their cost-benefit analysis with an increased cost per exam, which was estimated to take six years to amortize with the higher cost. On the other hand, Geffen (2013) estimated a positive return of $89.8 million per year in DI/PACS based on its full adoption and optimal use in B.C. For DIS, both the Deloitte (2010) and Geffen (2013) studies estimated positive returns in excess of $435 million and $200 million per year nationally and in B.C., respectively. Their predictions are based on full adoption and use of the systems.
7.3.2 Clinical Adoption of eHealth Systems
To better understand why the value of eHealth is not consistently being
realized, it is prudent to consider the contextual factors surrounding adoption
that influence these findings. Put differently, the value derived from eHealth
is dependent on these contextual factors, which affect the extent of system
adoption that takes place in an organization. Not all studies addressed issues
of adoption to explain their findings; 29 of the Canadian studies did report
contextual factors for adoption. The identified factors were mapped to the
adoption dimension of the eHealth Value Framework, with specific examples in
each category. They are explained below and summarized in Table 7.1.
- Micro level – The design of the system in terms of its functionality, usability and technical performance had a major influence on how it was perceived and used, which in turn influenced the actual benefits. For instance, the P.E.I. DIS (P.E.I. Department of Health and Wellness, 2010) users had mixed perceptions on the system’s ease of use, functionality, speed, downtime and security that influenced their use and satisfaction. The quality of the clinical data in terms of accuracy, completeness and relevance influenced its clinical utility. The actual system use and its ability to assist in decision-making, data exchange and secondary analysis also influenced its perceived usefulness. For instance, seven of the EMR studies involved the development and validation of algorithms to identify patients with specific conditions (Tu et al., 2010a; Tu et al., 2010b; Tu et al., 2011; Harris et al., 2010; Poissant, Taylor, Huang, & Tamblyn, 2010; Roshanov, Gerstein, Hunt, Sebaldt, & Haynes, 2013), generate quality indicators (Burge, Lawson, Van Aarsen, & Putnam, 2013), and conduct secondary analyses (Tolar & Balka, 2011). The type and extent of user training and support also influenced adoption. Shachak, Montgomery, Tu, Jadad, and Lemieux-Charles (2013) identified different types of end user support sources, knowledge and activities needed to improve EMR use over time.
- Meso level – For people, the level of user competence, experience and motivation, the capability of the support staff, and the availability of mentors all influenced adoption. For instance, Lapointe et al. (2012) found providers had varying abilities in performing EMR queries to engage in reflective practice on their patient populations. The end user support scheme identified by Shachak et al. (2012) directly influenced the confidence and capabilities of the users and support staff. Even after implementation, time was still needed for staff to learn the system, as was reported by Terry, Brown, Denomme, Thind, and Stewart (2012) with respect to users of EMRs that had been implemented for two years. For organizations, having management commitment and support, realistic workload, expectations and budgets, and an interoperable infrastructure influenced adoption. These factors were reported by McGinn et al. (2012) in their Dephi study on successful implementation strategies with representative EHR user groups. For implementation, the ability to manage the project timeline, resources and activities, and to engage providers all had major influences on successful adoption. An example was the health information exchange (HIE) study reported by Sicotte and Paré (2010), where the implementation efforts had major influences on the success or failure of two HIE systems. The Auditor General’s report (OAG, 2010) raised concerns with EHR implementation initiatives in terms of insufficient planning, governance, monitoring and public reporting that led to unclear value for money.
- Macro level – One study addressed the standards, funding, and policy aspects of the Canadian eHealth plan to adopt an interoperable EHR (Rozenblum et al., 2011). Rozenblum and colleagues acknowledged Canada’s national eHealth standards, EHR funding, registries and DI/PACS as tangible achievements over the past 10 years. Yet these authors felt the Canadian plan fell short of having a coordinated eHealth policy, active clinician engagement, a focus on regional interoperability, a flexible EHR blueprint, and a business case to justify the value of an EHR. As recommendations, their study called for an eHealth policy that is tightly aligned with major health reform efforts, a bottom-up approach by placing clinical needs first with active clinician and patient engagements, coordinated investments in EMRs to fill the missing gap, and financial incentives on health outcomes that can be realized with EHRs. Similarly, McGinn et al. (2012) and PITO (2013) suggested physician reimbursement and incentives as ways to encourage EMR adoption. Burge et al. (2013), Holbrook et al. (2009) and Eguale, Winslade, Hanley, Buckeridge, and Tamblyn (2010) all emphasized the need for data standards to improve interoperability. Note that Infoway received additional funding in 2010 to expand their scope to include support for physician EMRs, which include clinician engagement through such efforts as the Clinician Peer Support Network (Infoway, 2013).
7.3.3 Evaluations of Clinical Adoption Factors
In addition to mentioning the contextual factors that may have been facilitators
or barriers to achieving value in care, process and return, 21 studies actually
evaluated some influencing adoption factors themselves. For example, in the
three papers by Tu and colleagues (2010a, 2010b, 2011), the primary focus was
on the content of the EMRs and its ability to help identify patient populations. While this may not be an
example of a health value outcome, it is an example of an information content
measure that contributes to care provision. The rationale is that the value of
the EMR is dependent on the quality of the data. If data quality is lacking, then value
at the health outcome level will be impacted. Therefore, looking at individual
factors from an evaluation perspective may also help to make sense of the
evidence. Simply having the factor present — for example, training for end users — does not ensure successful outcomes. The findings for factors examined in the
Canadian studies are summarized in Figure 7.3.
Figure 7.3. Summary of adoption factors assessed in micro, meso, and macro categories. There
is a considerable focus on micro factors and it was challenging to find
assessment of macro level factors.
7.4 Implications
The current evidence on Canadian eHealth benefits is confusing and difficult to
interpret even for the experienced eHealth researcher and practitioner. There
are four types of issues that should be considered when navigating the eHealth
benefits landscape. These are the definition of eHealth, one’s views or perception of the eHealth system, the methods used to study benefits,
and system adoption that can influence eHealth benefits. These issues and their
implications for healthcare organizations are discussed below.
- Definitions – The field of eHealth is replete with jargon, acronyms and conflicting descriptions. For instance, eHealth refers to the application of health information and communication technology or ICT in health. It is a term often seen in the Canadian and European literature. On the other hand, health information technology or HIT describes the use of ICT in health especially in the United States. The terms EHR and EMR can have different meanings depending on the countries in which they are used. In the U.S., EHR and EMR are used interchangeably to mean electronic records that store patient data in healthcare organizations. However, in Canada EMR refers specifically to electronic patient records in a physician’s office. The term EHR can also be ambiguous. According to the Institute of Medicine, an EHR has four core functions of health information and data, order entry (i.e., CPOE), results management, and decision support (Blumenthal et al., 2006). Sometimes it may also include patient support, electronic communication and reporting, and population health management. Even CPOE can be ambiguous as it may or may not include decision support functions. The challenge with eHealth definitions, then, is that there are often implicit, multiple and conflicting meanings. The Canadian eHealth literature is no exception. Thus, when reviewing the Canadian evidence on eHealth benefits one needs to understand what system and/or function is involved, how it is defined and where it is used.
- Views or perceptions – The type of eHealth system and function being evaluated, the care setting involved, and the focus of the evaluation are important considerations that influence how the system is viewed or perceived by different stakeholders as to its intentions, roles and values. Most evaluation studies would identify the eHealth system and/or function being investigated, such as an EHR with CDS and/or CPOE. The care setting can influence how a system is adopted since it embodies the type of care and organizational practices being provided. The focus is the clinical area being evaluated and the benefit expected, such as medication management with CPOE to reduce errors. The challenge with eHealth views as articulated in these studies, then, is that the descriptions of the system, setting and focus are often incomplete in the evaluation write-up, which makes it difficult to determine the relevance of the findings to the local setting. For example, in studies of CPOE with alerts, it is often unclear how they are generated and to whom, and whether a response is required. For a setting such as a primary care clinic it is often unclear whether the actual site is a hospital outpatient department or a stand-alone community-based practice. For focus, some studies include such a multitude of benefit measures that it can be difficult to decide if the system has led to overall benefit. The Canadian eHealth studies face the same challenge of having to tease out such detail to determine the relevance and applicability of the findings.
- Methods of study – There is a plethora of scientific, psychosocial and business methods used to evaluate eHealth benefits. At one end of the spectrum are such experimental methods as the randomized control trial (RCT) used to compare two or more groups for notable changes from the implementation of an eHealth system as the intervention. At the other end is the descriptive method used to explore and understand the interactions between an eHealth system and its users. The choice of benefit measures selected, the type of data collected and the analytical method used can all affect the study results. In contrast to controlled studies that strive for statistical and clinical significance in the outcome measures, descriptive studies offer explanations of the observed changes as they unfold. There are also economic evaluation methods that examine the relationships between the costs and return of an investment, and simulation methods that model changes based on a set of input parameters and analytical algorithms. The challenge, then, is that one needs to know the principles and rigour of different methods in order to plan, execute, and appraise eHealth benefits evaluation studies. The Canadian eHealth evidence identified in this chapter has been derived from different approaches such as RCTs, descriptive studies and simulation methods. The quality of these studies varies depending on the rigour of the design/method used. The different outcome measures used has made it difficult to aggregate the findings. Finally, timing of studies in relation to adoption and use will influence benefits, which may or may not be seen.
- System adoption – There are mixed and even conflicting results from evaluation studies on eHealth benefits. To understand these differences one has to appreciate the context surrounding the implementation, use and impacts of eHealth systems in organizations. The success of an eHealth system in producing the expected benefits is dependent on many contextual factors. Examples are the usability of the system involved, prior experience of its users, the training and support available, the organizational culture and commitment toward eHealth and the system, how well the implementation process is managed, the funding and incentives in place and the overall expectations. The contextual factors are described in detail under the investment, micro, meso, macro and value dimensions of the proposed eHealth Value Framework presented in this chapter. These contextual factors apply equally well to the Canadian eHealth systems being evaluated. The challenge, then, is whether the level of detail provided in the evaluation write-up is sufficient, and whether it can explain why the system had worked or not, and if not, what could be done to achieve the benefits.
7.5 Summary
This chapter introduced the holistic eHealth Value Framework to make sense of
eHealth value in the Canadian setting. This framework is made up of four
dimensions of investment, adoption, value and time lag. It was applied in a
review of Canadian literature on eHealth evaluation studies to examine eHealth
value within the Canadian context. The framework helped to make sense of the
conflicting evidence found in the literature on eHealth benefits in Canada.
References
Blumenthal, D., DesRoches, C., Donelan, K., Ferris, T., Jha, A., Kaushal, R., … Shield, A. (2006). Health information technology in the United States: The information base for
progress. Princeton, NJ: Robert Wood Johnson Foundation.
Blumenthal, D., & Tavenner, M. (2010). The “meaningful use” regulation for electronic health records. The New England Journal of Medicine, 363(6), 501–504.
Burge, F., Lawson, B., Van Aarsen, K., & Putnam, W. (2013). Assessing the feasibility of extracting clinical information
to create quality indicators from primary healthcare practice EMRs. Healthcare Quarterly, 16(3), 34–41.
Canada’s Health Informatics Association [COACH]. (2013). Canadian EMR adoption and maturity model. Toronto: Author. Retrieved from
http://www.coachorg.com/en/resourcecentre/Green_White-Papers.asp
Centre for Research in Healthcare Engineering [CRHE]. (2011). EMR integrated labs workflow evaluation. Toronto: University of Toronto.Retrieved from https://www.infoway-inforoute.ca/index.php/resources/reports
Dainty, K. N., Adhikari, N. K. J., Kiss, A., Quan, S., & Zwarenstein, M. (2011). Electronic prescribing in an ambulatory care setting: a
cluster randomized trial. Journal of Evaluation in Clinical Practice, 18(4), 761–767.
Deloitte. (2010). National impacts of generation 2 drug information systems technical report. Toronto: Canada Health Infoway. Retrieved from
https://www.infoway-inforoute.ca/index.php/resources/reports
Dormuth, C., Miller, T. A., Huang, A., Mamdani, M. M., & Juurlink, D. N. (2012). Effect of a centralized prescription network on
inappropriate prescriptions for opioid analgesics and benzodiazepines. Canadian Medical Informatics Journal, 184(16), E852–E856.
Eguale, T., Winslade, N., Hanley, J. A., Buckeridge, D. L., & Tamblyn, R. (2010). Enhancing pharmacosurveillance with systematic collection
of treatment indication in electronic prescribing: a validation study in
Canada. Drug Safety, 33(7), 559–567.
eHealth Observatory. (2013). Clinical adoption maturity model. Victoria, BC: University of Victoria. Retrieved from http://ehealth.uvic.ca/methodology/models/CMM.php
Fernandes, O. A., Etchells, E. E., Lee, A. W., Siu, V., Bell, C., Wong, G., … Department of Medicine, McMaster University. (2011). What is the impact of a
centralized provincial drug profile viewer on the quality and efficiency of
patient admission medication reconciliation? A randomized controlled trial.Canadian Journal of Hospital Pharmacy, 64(1), 85.
G. Braha & Associates Ltd. (2012). eChart Manitoba release 1: a benefits evaluation. Winnipeg, MB: Author. Retrieved from
https://www.infoway-inforoute.ca/index.php/resources/reports
Geffen, M. (2013). British Columbia eHealth benefits estimates. Stamford, CT: Gartner Consulting. Retrieved from https://www.infoway-inforoute.ca/index.php/resources/reports
Harris, S. B., Glazier, R. H., Tompkins, J. W., Wilton, A. S., Chevendra, V.,
Stewart, M. A., & Thind, A. (2010). Investigating concordance in diabetes diagnosis between
primary care charts (electronic medical records) and health administrative
data: a retrospective cohort study. BMC Health Services Research, 10, 347.
HIMSS Analytics. (2014). EMR adoption model. Chicago: Author. Retrieved from http://www.himssanalytics.org/home/index.aspx
Holbrook, A., Thabane, L., Keshavjee, K., Dolovich, L., Bernstein, B., Chan, D.,
… Gerstein, H. (2009). Individualized electronic decision support and reminders
to improve diabetes care in the community: COMPETE II randomized trial. Canadian Medical Association Journal, 181(1/2), 37–44.
Infoway. (2013). Jurisdictional peer-to-peer networks. Toronto: Author. Retrieved from
https://www.infoway-inforoute.ca/en/communities/pan-canadian-clinician-peer-network/jurisdictional-peer-to-peer-networks
Lapointe, L., Hughes, J., Simkus, R., Lortie, M., Sanche, S., & Law, S. (2012). The population health management challenge final report. Montreal: St. Mary’s Research Centre.Retrieved from https://www.infoway-inforoute.ca/index.php/resources/reports
Lau, F., Hagens, S., & Muttitt, S. (2007). A proposed benefits evaluation framework for health
information systems in Canada. Healthcare Quarterly, 10(1), 112–118.
Lau, F., Partridge, C., Randhawa, G., & Bowen, M. (2013). Applying the clinical adoption framework to evaluate the
impact of an ambulatory electronic medical record. Studies in Health Technology & Informatics, 183, 15–20.
Lau, F., Price, M., & Bassi, J. (2014). Making sense of eHealth benefits and their policy implications in Canada. Unpublished discussion paper.
Lau, F., Price, M., & Keshavjee, K. (2011). From benefits evaluation to clinical adoption: Making
sense of health information system success. Healthcare Quarterly, 14(1), 39–45.
Lee, J. Y., Leblanc, K., Fernandes, O. A., Huh, J., Wong, G. G., Hamandi, B., … Harrison, J. (2010). Medication reconciliation during internal hospital
transfer and impact of computerized prescriber order entry. Annals of Pharmacotherapy, 44(12), 1887–1895.
MacDonald, D., & Neville, D. (2010). Evaluating the implementation of picture archiving and
communication systems in Newfoundland and Labrador: A cost benefit analysis. Journal of Digital Imaging, 23(6), 721–731.
McGinn, C. A., Gagnon, M., Shaw, N., Sicotte, C., Mathieu, L., Leduc, Y., … Legare, F. (2012). Users’ perspectives of key factors to implementing electronic health records in
Canada: a delphi study. BMC Medical Informatics and Decision Making, 12, 105.
Mensink, N., & Paterson, G. (2010). The evolution and uptake of drug information system: The
case of a small Canadian province. Studies in Health Technology & Informatics, 160(Pt. 1), 352–355.
Office of the Auditor General of Canada [OAG]. (2010, April). Electronic health records in Canada: An overview of federal and provincial audit
reports. Ottawa: Author. Retrieved from
http://www.oag-bvg.gc.ca/internet/docs/parl_oag_201004_07_e.pdf
O’Reilly, D., Holbrook, A., Blackhouse, G., Troyan, S., & Goeree, R. (2012). Cost-effectiveness of a shared computerized decision support
system for diabetes linked to electronic medical records. Journal of the American Medical Informatics Association, 19(3), 341–345.
Paré, G., de Guinea, A. O., Raymond, L., Poba-Nzaou, P., Trudel, M., Marsan, J., & Micheneau, T. (2013). Computerization of primary care medical clinics in Quebec: Results from a survey
on EMR adoption, use and impacts. Toronto: Canada Health Infoway. Retrieved from
https://www.infoway-inforoute.ca/index.php/resources/reports
Paterson, G., Shaw, N., Grant, A., Leonard, K., Delisle, E., Mitchell, S., … Kraetschmer, N. (2010). A conceptual framework for analyzing how Canadian
physicians are using electronic medical records in clinical care. Studies in Health Technology & Informatics, 160(Pt. 1), 141–145.
Physician Information Technology Office [PITO]. (2013). EMR adoption study. Vancouver: Insights West and Cientis Technologies.Retrieved from
https://www.infoway-inforoute.ca/index.php/resources/reports
Poissant, L., Taylor, L., Huang, A., & Tamblyn, R. (2010). Assessing the accuracy of an inter-institutional automated
patient-specific health problem list. BMC Medical Informatics and Decision Making, 10, 10.
Prairie Research Associates [PRA]. (2012). Manitoba’s physician integrated network (PIN) initiative: A benefits evaluation report. Winnipeg, MB: Author. Retrieved from https://www.infoway-inforoute.ca/index.php/resources/reports
PricewaterhouseCoopers [PwC]. (2013). The emerging benefits of electronic medical record use in community-based care. Toronto: Canada Health Infoway. Retrieved from
https://www.infoway-inforoute.ca/index.php/resources/reports
Price, M., Bowen, M., Lau, F., Kitson, N., & Bardal, S. (2012). Assessing accuracy of an electronic provincial medication
repository. BMC Medical Informatics and Decision Making, 12, 42.
Prince Edward Island (P.E.I.) Department of Health and Wellness. (2010). Prince Edward Island drug information system evaluation report. Charlottetown: Author. Retrieved from
https://www.infoway-inforoute.ca/index.php/resources/reports
Roshanov, P., Gerstein, H. C., Hunt, D. L., Sebaldt, R. J., & Haynes, R. B. (2012). Impact of a computerized system for evidence-based
diabetes care on completeness of records: A before-after study. BMC Medical Informatics and Decision Making, 12, 63. doi: 10.1186/1472-6947-12-63
Rozenblum, R., Jang, Y., Zimlichman, E., Salzberg, C., Tamblyn, M., Buckeridge,
D., … Tamblyn, R. (2011). A qualitative study of Canada’s experience with the implementation of electronic health information. Canadian Medical Association Journal, 183(5), E281–E288.
Schryen, G. (2013). Revisiting IS business value research: what we already know, what we still need to know, and
how we can get there. European Journal of Information Systems, 22(2), 139–169.
Shachak, A., Montgomery, C., Tu, K., Jadad, A. R., & Lemieux-Charles, L. (2013). End-user support for a primary care electronic
medical record: a qualitative case study of a vendor’s perspective. Informatics in Primary Care, 20(3), 185–196.
Sicotte, C., & Paré, G. (2010). Success in health information exchange projects: solving the
implementation puzzle. Social Science & Medicine, 70(8), 1159–1165.
Syed, S., Wang, D., Goulard, D., Rich, T., Innes, G., & Lang, E. (2013). Computer order entry systems in the emergency department
significantly reduce the time to medication delivery for high acuity patients. International Journal of Emergency Medicine, 6, 20. doi: 10.1186/1865-1380-6-20
Tamblyn, R., Reidel, K., Huang, A., Taylor, L., Winslade, N., Bartlett, G., … Pinsonneault, A. (2010). Increasing the detection and response to adherence
problems with cardiovascular medication in primary care through computerized
drug management systems: A randomized control trial. Medical Decision Making, 30(2), 176–188.
Terry, A. L., Brown, J. B., Denomme, L. B., Thind, A., & Stewart, M. (2012). Perspectives on electronic medical record implementation
after two years of use in primary health care practice. The Journal of the American Board of Family Medicine, 25(4), 522–527.
Tolar, M., & Balka, E. (2011). Beyond individual patient care: enhanced use of EMR data in a primary care setting. Studies in Health Technology & Informatics, 164, 143–147.
Tu, K., Mitiku, T., Lee, D. S., Guo, H., & Tu, J. V. (2010a). Validation of physician billing and hospitalization data to
identify patients with ischemic heart disease using data from the electronic
medical record administrative data linked database (EMRALD). Canadian Journal of Cardiology, 26(7), e225–e228.
Tu, K., Mitiku, T., Guo, H., Lee, D. S., & Tu, J. V. (2010b). Myocardial infarction and the validation of physician
billing and hospitalization data using electronic medical records. Chronic Diseases in Canada, 30(4), 141–146.
Tu, K., Manuel, D., Lam, K., Kavanaugh, D., Mitiku, T. F., & Guo, H. (2011). Diabetics can be identified in an electronic medical record
using laboratory tests and prescriptions. Journal of Clinical Epidemiology, 64(4), 431–435.
EPUB
Manifold uses cookies
We use cookies to analyze our traffic. Please decide if you are willing to accept cookies from our website. You can change this setting anytime in Privacy Settings.