Advertisement

Satisfaction with modes of telemedicine delivery during COVID-19: A randomized, single-blind, parallel group, noninferiority trial

      Abstract

      Background

      Little is known about satisfaction with different modes of telemedicine delivery. The objective of this study was to determine whether patient satisfaction with phone-only was noninferior to video visits.

      Methods

      We conducted a parallel group, randomized (1:1), single-blind, noninferiority trial in multispecialty clinics at a tertiary academic medical center. Adults age ≥ 60 years or with Medicare/Medicaid insurance were eligible. Primary outcome was visit satisfaction rate (9 or 10 on a 0-10 satisfaction scale). Noninferiority was determined if satisfaction with phone-only (intervention) versus video visits (comparator) was no worse by a -15% prespecified noninferiority margin. We performed modified intent-to-treat (mITT) and per protocol analyses, after adjusting for age and insurance.

      Results

      200 participants, 43% Black, 68% women completed surveys. Visit satisfaction rates were high. In the mITT analysis, phone-only visits were noninferior by an adjusted difference of 3.2% (95% CI, -7.6% to 14%). In the per protocol analysis, phone-only were noninferior by an adjusted difference of -4.1% (95% CI, -14.8% to 6.6%). The proportion of participants who indicated they preferred the same type of telemedicine visit as their next clinic visit were similar (30.2% vs 27.9% video vs phone-only, p = 0.78) and a majority said their medical concerns were addressed and would recommend a telemedicine visit.

      Conclusions

      Among a group of diverse, established older or underserved patients, the satisfaction rate for phone-only was noninferior to video visits. These findings could impact practice and policies governing telemedicine.

      Keywords

      Introduction

      The availability of new technologies and interest in making medical care more accessible and convenient has spurred the development of remotely-delivered care (e.g., telemedicine).
      • Weinstein RS
      • Lopez AM
      • Joseph BA
      • et al.
      Telemedicine, telehealth, and mobile health applications that work: opportunities and barriers.
      The COVID-19 pandemic
      • Hollander JE
      • Carr BG.
      Virtually perfect? Telemedicine for covid-19.
      • Bachireddy C
      • Chen C
      • Dar M.
      Securing the safety net and protecting public health during a pandemic: medicaid's response to COVID-19.
      • Kuy S
      • Gupta R
      • Correa R
      • et al.
      Best practices for a covid-19 preparedness plan for health systems.
      has led to a significant shift to home-based telemedicine in many medical specialties.
      • Kulcsar Z
      • Albert D
      • Ercolano E
      • et al.
      Telerheumatology: A technology appropriate for virtually all.
      • McDougall JA
      • Ferucci ED
      • Glover J
      • et al.
      Telerheumatology: a systematic review.
      • Piga M
      • Cangemi I
      • Mathieu A
      • et al.
      Telemedicine for patients with rheumatic diseases: systematic review and proposal for research agenda.
      Home-based telemedicine uses phone and/or videoconferencing to deliver care to patients in their own homes,
      • Hollander JE
      • Carr BG.
      Virtually perfect? Telemedicine for covid-19.
      • Bachireddy C
      • Chen C
      • Dar M.
      Securing the safety net and protecting public health during a pandemic: medicaid's response to COVID-19.
      • Kuy S
      • Gupta R
      • Correa R
      • et al.
      Best practices for a covid-19 preparedness plan for health systems.
      ,

      American Telehealth Association. Home telehealth clinical guidelines. 2003, Accessed May 20, 2020.

      and these visits are variably reimbursed.
      • Bachireddy C
      • Chen C
      • Dar M.
      Securing the safety net and protecting public health during a pandemic: medicaid's response to COVID-19.
      Because avoiding travel decreases COVID-19 transmission risks,
      • Uscher-Pines L
      • Fischer S
      • Tong I
      • et al.
      Virtual first responders: The role of direct-to-consumer telemedicine in caring for people impacted by natural disasters.
      telemedicine has been recommended for socially and medically vulnerable groups,
      • Hayes BL
      • Curtis JR
      • Laster A
      • et al.
      Osteoporosis care in the United States after declines in reimbursements for dxa.
      • Heath B
      • Salerno R
      • Hopkins A
      • et al.
      Pediatric critical care telemedicine in rural underserved emergency departments.
      • Feldman CH
      • Ramsey-Goldman R.
      Widening disparities among patients with rheumatic diseases in the COVID-19 era: An urgent call to action.
      such as those residing in rural areas
      • Nesbitt TS
      • Marcin JP
      • Daschbach MM
      • et al.
      Perceptions of local health care quality in 7 rural communities with telemedicine.
      , the elderly, those with comorbidities, and other groups at higher risk of COVID-19 complications.
      • Richardson S
      • Hirsch JS
      • Narasimhan M
      • et al.
      Presenting characteristics, comorbidities, and outcomes among 5700 patients hospitalized with COVID-19 in the new york city area.
      • Yang J
      • Zheng Y
      • Gou X
      • et al.
      Prevalence of comorbidities in the novel wuhan coronavirus (COVID-19) infection: A systematic review and meta-analysis.
      • Zhou F
      • Yu T
      • Du R
      • et al.
      Clinical course and risk factors for mortality of adult inpatients with COVID-19 in wuhan, china: A retrospective cohort study.
      • Wu C
      • Chen X
      • Cai Y
      • et al.
      Risk factors associated with acute respiratory distress syndrome and death in patients with coronavirus disease 2019 pneumonia in wuhan, china.
      However, many patients lack access to video capacity,
      • Nouri S
      • Khoong EC
      • Lyles CR
      • et al.
      Addressing equity in telemedicine for chronic disease management during the covid-19 pandemic.
      ,
      • Eberly LA
      • Kallan MJ
      • Julien HM
      • et al.
      Patient characteristics associated with telemedicine access for primary and specialty ambulatory care during the COVID-19 pandemic.
      which may create significant inequities in receiving care, particularly if the video component of telemedicine is associated with better care quality and patient satisfaction. Phone-only visits, which may be preferred by patients who do not have the required technology for video visits or who may have lower digital literacy, may mitigate health disparities. Despite their potentially higher reimbursement than phone-only visits, video visits may offer few if any tangible medical care advantages over phone-only visits for certain medical conditions
      • Saliba-Gustafsson EA
      • Miller-Kuhlmann R
      • Kling SMR
      • et al.
      Rapid implementation of video visits in neurology during COVID-19: Mixed methods evaluation.
      , and patients may be agnostic about the relative value and satisfaction with one form of telemedicine compared to another.
      Higher patient satisfaction is positively associated with adherence to treatment protocols,
      • Jha AK
      • Orav EJ
      • Zheng J
      • et al.
      Patients' perception of hospital care in the United States.
      which is notable given the high individual and societal costs associated with treatment non-adherence.
      • Viswanathan M
      • Golin CE
      • Jones CD
      • et al.
      Interventions to improve adherence to self-administered medications for chronic diseases in the United States: a systematic review.
      Thus, the goal of our study was to determine if patient satisfaction with phone-only visits was noninferior to video visits at a large tertiary referral center in the South, a region which is home to many socioeconomically disadvantaged older individuals.
      • Scott Kruse C
      • Karem P
      • Shifflett K
      • et al.
      Evaluating barriers to adopting telemedicine worldwide: a systematic review.

      Methods with statistical considerations

      Study design and participants

      We conducted a parallel, single-blinded, noninferiority randomized patient-level clinical trial comparing satisfaction rates for two telemedicine delivery methods: phone-only (intervention group) and video visits (standard group). The two modes of telemedicine were compared to ensure that the less advanced telephone communication was not inferior to videoconferencing by a pre-specified margin of 15%.
      Patients who had established care at participating clinics (at least one face-to-face visit with their physician in the past 12 months) were recruited and randomized to have their next scheduled routine clinical follow-up either via a phone-only or a video visit. The study was performed at the University of Alabama at Birmingham (UAB). Participants were enrolled from three clinical practices, including three physicians in rheumatology, five in cardiology, and three in family medicine, all of whom agreed to collaborate in this project and were blinded to which of their patients participated in the study.
      Participants were eligible if they: 1) were scheduled for a routine in-office follow-up visit; 2) were age 60 years or older or had public insurance (i.e., Medicare/Medicaid); and 3) had videoconferencing capability. We restricted our study population to individuals 60 years of age or older or those with public insurance because of perceived concerns about their comfort and facility in using videoconferencing technology in these socially vulnerable groups.
      Human subject protocols and consent procedures were approved by the UAB's Institutional Review Board (IRB). We obtained a waiver of informed consent given the study's minimal risk, the uncertain superiority of one form of telemedicine visit over another, the high probability of participants modifying their behavior due to their awareness of being observed (i.e., Hawthorne effect),
      • McCambridge J
      • Witton J
      • Elbourne DR.
      Systematic review of the hawthorne effect: new concepts are needed to study research participation effects.
      and to mitigate participants’ burden. After the completion of the study procedures, previously blinded participants were debriefed about their study participation.

      Blinding and randomization

      Participants were randomly assigned to phone-only or video groups in a 1:1 ratio using computer-generated lists of random numbers implemented in REDCap.
      • Harris PA
      • Taylor R
      • Thielke R
      • et al.
      Research electronic data capture (redcap)–a metadata-driven methodology and workflow process for providing translational research informatics support.
      Patients, physicians, and investigators were blinded to the randomization assignment.

      Interventions

      The primary study goal was to compare visit satisfaction rates between phone-only and video groups. Our study protocol required that the research team contacted eligible patients by phone 2-3 weeks prior to a routinely scheduled in-office follow-up to invite participation in telemedicine visits. For each pre-appointment call, to ensure that all potential participants are contacted in a reproducible and standardized manner, research assistants followed an IRB-approved phone script that informed patients of the date and time of the scheduled appointment, that their appointment would be conducted by phone or a video call, confirmed the patient's best contact number, and verified their ability to participate in a video call by conducting a test video call using the videoconferencing platform in use for telemedicine services at our medical center. Only those patients who were able to successfully complete the test video call were subsequently randomized. Our study protocol was similar to the usual local telemedicine clinical protocols during the COVID-19 pandemic. We assessed participants’ videoconferencing capacity by conducting a test video call. Then the randomization procedure occurred. Participants were scheduled to have a phone-only or a video visit based on their randomly assigned group. Similar to the usual clinical procedures on the day of their appointment, physicians contacted patients by a phone or a video call. However, because physicians being blinded to patients’ study participation and because of prevailing institutional practices, a patient assigned to the video group could have a phone-only visit (e.g., due to videoconferencing difficulties) and a patient assigned to the phone-only group could have a video visit (e.g., if the physician decided that a video visit was needed).

      Covariates

      We collected participant data via phone surveys 36-72 hours post-visit to minimize recall bias and via electronic medical record (EMR) review. The survey collected information on self-rated health (excellent to poor response scale),
      • Stewart AL
      • Ware JE.
      Measuring functioning and well-being: The medical outcomes study approach. Accessed.
      hearing/vision impairment, education, health literacy,
      • Chew LD
      • Bradley KA
      • Boyko EJ.
      Brief questions to identify patients with inadequate health literacy.
      transportation difficulties, income, and employment status. Demographic characteristics (e.g., age, sex, race) were captured from the EMR. Area deprivation index (ADI), which measures neighborhood disadvantage,
      • Singh GK.
      Area deprivation and widening inequalities in us mortality, 1969-1998.
      was derived using the patients’ residence zip code.
      • Kind AJ
      • Jencks S
      • Brock J
      • et al.
      Neighborhood socioeconomic disadvantage and 30-day rehospitalization: A retrospective cohort study.

      Outcomes

      The primary outcome of interest was the patient's satisfaction, assessed using the validated visit satisfaction scale from the Consumer Assessment of Healthcare Providers and Systems (CAHPS)®30 survey. The patients were asked: "Using any number from 0 to 10, where 0 is the worst visit possible and 10 is the best visit possible, what number would you use to rate your telemedicine visit?”.

      Agency for Healthcare Research and Quality. Consumer assessment of healthcare providers and systems. Accessed May 30, 2020.

      Scores of 9 or 10 were prespecified to be grouped as satisfied and the satisfaction rate (proportion of satisfied participants) was calculated. Secondary outcomes included whether medical concerns were addressed at the visit, willingness to recommend a telemedicine visit, and an assessment of preference for the next visit type
      • Walter SD
      • Turner RM
      • Macaskill P
      • et al.
      Estimation of treatment preference effects in clinical trials when some participants are indifferent to treatment choice.
      using the following question: “If you had a choice, what type of visit would you prefer as your next visit with [insert provider's name]
      • Genevay S
      • Finckh A
      • Ciurea A
      • et al.
      Tolerance and effectiveness of anti-tumor necrosis factor alpha therapies in elderly patients with rheumatoid arthritis: A population-based cohort study.
      ?” with ‘telemedicine visit by phone’, ‘telemedicine visit by video’, or ‘in-person visit’ as possible choices. Exploratory outcomes included telemedicine acceptability (Telemedicine Perception Questionnaire [TMPQ] score, range 17 - 85, higher values are better),
      • Demiris G
      • Speedie S
      • Finkelstein S.
      A questionnaire for the assessment of patients' impressions of the risks and benefits of home telecare.
      perceived autonomy support (healthcare climate questionnaire [HCCQ], range 0-15, higher values are better),
      • Williams GC
      • Gagné M
      • Ryan RM
      • et al.
      Facilitating autonomous motivation for smoking cessation.
      perceptions about telemedicine relative to an in-person visit, and aspects of patient-physician communication (CAHPS survey).
      • Mukherjee S
      • Rodriguez HP
      • Elliott MN
      • et al.
      Modern psychometric methods for estimating physician performance on the clinician and group cahps® survey.
      ,

      Quality AfHRa. Consumer assessment of healthcare providers and systems.

      Sample size and statistical analysis

      We chose a noninferiority design based on the expectation that video visits might be preferred, that demonstrating noninferiority of satisfaction with phone-only visits would provide evidence for a patient's perception about phone-only visits and support continued patient access to and potential reimbursement for phone-only visits. We tested the noninferiority of phone-only versus video visits for the primary outcome of satisfaction rate. We assumed that the satisfaction rate for the home-based video visit (the superior group) would be 65%, slightly lower than the 68.5% satisfaction rate with facility-based telemedicine observed in a previous report.
      • Donelan K
      • Barreto EA
      • Sossong S
      • et al.
      Patient and clinician experiences with telehealth for patient follow-up care.
      We established a ‘noninferiority margin’ so that the satisfaction rate with phone-only visits would be 15% lower than with video visits, the null hypothesis. The 15% noninferiority margin was chosen based on the assumption that less than 15% satisfaction rate difference in favor of video visits would be clinically irrelevant in declaring one mode of telemedicine superior to the other. We determined that a sample size of N = 100 per group would provide 72% power to reject the null hypothesis that phone-only visits were inferior to video visits by 15% or more when both were equivalent to each other with a 5% alpha significance level.
      Variables were summarized using mean and standard deviation (SD) or median and interquartile range (IQR), as appropriate. Two sample t-test, chi-square test (Fisher Exact, if appropriate) examined differences between variables. Logistic regression evaluated the primary outcome, accounting for residual differences between groups and the potential differential effects of age versus insurance status. Because the fidelity of the intervention is key in noninferiority designs, where nonadherence to intervention can bias the assessments away from the null hypothesis by making the groups more similar than they may be in practice, we analyzed outcomes of individuals randomized who completed the post-visit surveys using both a modified intent-to-treat (mITT, as assigned by randomization and receiving a telemedicine visit) and per protocol (i.e., receiving the assigned visit type) analyses.

      United States Food & Drug Administration. Guidance document: Non-inferiority clinical trials.2016.

      We performed a sensitivity analysis that also included participants who partially completed post-visit surveys but who answered the visit satisfaction question. We conducted another sensitivity analysis that included the entire randomized population. Using a tipping point analysis,
      • Yan X
      • Lee S
      • Li N.
      Missing data handling methods in medical device clinical trials.
      we imputed missing data using the satisfaction rate consistent with the null hypothesis (65% and 50% for video and phone-only groups, respectively) and assuming worst-best case scenario (assigning missing data as ‘not satisfied’ for phone-only and ‘satisfied’ for video group).
      We explored heterogeneity of treatment effects using pre-specified subgroup analyses defined by race, sex, lack of transportation, income, and employment status. All analyses were performed using SAS 9.4 or R 4.0.4.

      Results

      Study participants and characteristics

      The patients were screened for eligibility between May 28 and November 5, 2020 and the telemedicine visits were conducted between June 4 and December 2, 2020. Study enrollment was stopped when the pre-specified number of participants who fully completed surveys was reached (N=200). As seen in the CONSORT (Consolidated Standards of Reporting Trials) diagram (Fig. 1), a total of 2,800 people receiving routine care in the three clinical areas were assessed for eligibility. Of the 1,267 potentially eligible patients identified, 269 were randomized to receive either a video or a phone-only visit; 229 participants (85.1%) attended these visits. There were no significant differences in reasons for not participating in a telemedicine visit between groups (p = 0.12). A total of 200 participants (87.3%), including 96 assigned to video group and 104 assigned to phone-only group, completed surveys on average 2.7 (3.4) days post-visit. This population defines the modified intent-to-treat (mITT) analysis.
      Fig 1
      Fig. 1CONSORT diagram showing patient enrollment, randomization, and follow up.
      Overall, participants who completed the surveys were predominately women (N=136, 68%), 86 (43%) were Black with a mean age in the early sixties consistent with the inclusion criteria, and the majority had at least some college education (N=148, 74%) (Table 1). Compared to participants in the video group, those in the phone-only group were slightly younger and fewer had a Medicare plan as their medical insurance. The median (Q25, Q75) ADI national ranking was 68 (40, 85) indicating that a majority of participants were living in socio-economically disadvantaged neighborhoods.
      Table 1Demographic characteristics of participants who completed phone surveys and were included in the modified intent to treat analysis; p  <  0.05 in bold font.
      Telemedicine Video (N=96)Telemedicine Phone (N=104)p
      Age, years, median (Q 25-Q 75)66.75 (62.10-72.45)62.25 (53.60-68.45)0.001
      Age group No. (%)0.074
       <65 years39 (40.6)63 (60.6)
       ≥65 years57 (59.4)41 (39.4)
       Sex, female, No. (%)67 (69.8)69 (66.3)0.71
      Race, No. (%)0.97
       White51 (53.1)56 (53.8)
       Black42 (43.8)44 (42.3)
       Other3 (3.1)4 (3.9)
      Insurance plans, No. (%)0.03
       Medicare25 (26.0)16 (15.4)
       Medicaid65 (67.7)71 (68.3)
       Other
      Viva, Blue Cross Blue Shield, United Health Care, Tricare; †Inadequate health literacy grouped the following answers: “Somewhat”, “A little bit”, and “Not at all”; ‡Employed is full-time, part-time, or temporary work; §State decile from 1 (least disadvantaged) to 10 (most disadvantaged); ‖National percentile from 1 (least disadvantaged) to 100 (most disadvantaged), missing for 9 participants.
      6 (6.2)17 (16.3)
      Specialty, No. (%)0.13
       Cardiology28 (29.2)18 (17.3)
       Family medicine39 (40.6)51 (49.0)
       Rheumatology29 (30.2)35 (33.7)
      Outcome assessment timing, days, mean (SD)2.88 (3.59)2.58 (3.31)0.54
      Past telemedicine experience, yes, No. (%)65 (67.7)70 (68.0)1
      Device used for the telemedicine visit
       Smartphone No. (%)90 (93.8)96 (92.3)0.90
       Computer/laptop, No. (%)3 (3.1)3 (2.9)
       Tablet, No. (%)1 (1.0)1 (1.0)
      Health status, excellent or very good, No. (%)70 (72.9)69 (66.3)0.48
      Transportation difficulties, No. (%)7 (7.3)14 (13.5)0.23
      Education, some college or more, No. (%)73 (76.0)75 (72.1)0.64
      Health literacy, inadequate†, No. (%)23 (24.0)20 (19.2)0.52
      Employment status, unemployed‡, No. (%)73 (76.0)83 (79.8)0.64
      Annual income, No. (%)0.40
       Low, < $29,99914 (14.6)21 (20.2)
       Medium, $30,000-79,99928 (29.2)29 (27.9)
       High, > $80,00016 (16.7)10 (9.6)
       Prefer not to answer38 (39.6)44 (42.3)
      Area deprivation index (ADI) ranking, state decile, median (Q 25-Q 75)§5.00 [2.00, 8.00]5.00 [2.00, 7.75]0.95
      Area deprivation index (ADI) ranking, national percentile, median (Q 25-Q 75) ‖67.00 [41.00, 85.00]69.00 [40.00, 84.75]0.95
      low asterisk Viva, Blue Cross Blue Shield, United Health Care, Tricare; †Inadequate health literacy grouped the following answers: “Somewhat”, “A little bit”, and “Not at all”; ‡Employed is full-time, part-time, or temporary work; §State decile from 1 (least disadvantaged) to 10 (most disadvantaged); ‖National percentile from 1 (least disadvantaged) to 100 (most disadvantaged), missing for 9 participants.

      Modified intent-to-treat analysis

      Because there were statistically significant differences for age and insurance between groups, analyses were adjusted for age and insurance. The age- and insurance- adjusted difference in the overall visit satisfaction rate for the phone versus video group was 3.2% with 95% CI -7.6% to 14%, which did not contain -15% establishing noninferiority. The unadjusted satisfaction rate difference between phone-only and video groups was 6.5% (95% CI, -4.3% to 17.2%) (p  <  0.0001) (Fig. 2). Thus, phone-only visits were not inferior to video visits for visit satisfaction rate (p  < 0.0001). The satisfaction rates were higher than anticipated in both groups (78.1% for video vs 84.6% for phone-only) and not significantly different, (p = 0.32) (Table 2).
      Fig 2
      Fig. 2Satisfaction (scores of 9 or 10) rates (95% confidence intervals) from modified intent-to-treat (mITT), per protocol, and sensitivity analyses; NI, noninferiority margin.
      Table 2Patient experience with telemedicine phone or video, modified intent-to-treat (mITT) and per protocol analyses; N (%) represented unless otherwise stated.
      Modified Intent-to-treatPer Protocol
      Telemedicine Video (N=96)Telemedicine Phone (N=104)pTelemedicine Video (N=66)Telemedicine Phone (N=79)p
      Primary Outcome
      Satisfaction rate, score ≥ 975 (78.1)88 (84.6)0.3256 (84.8)66 (83.5)1
      Satisfaction, median (IQR)10 (9, 10)10 (9, 10)0.2610 (9, 10)10 (9, 10)0.52
      Secondary Outcomes
      Preference for next visit0.650.35
       Telemedicine, same type29 (30.2)29 (27.9)27 (40.9)27 (34.2)
       Telemedicine, different type13 (13.5)19 (18.3)4 (6.1)10 (12.7)
       In-office54 (56.2)56 (53.8)35(53.0)42 (53.2)
      Would recommend telemedicine0.440.62
       Yes, definitely69 (71.9)74 (71.2)51 (77.3)58 (73.4)
       Yes, somewhat21 (21.9)25 (24.0)12 (18.2)17 (21.5)
       No6 (6.2)3 (2.9)3 (4.5)2 (2.5)
      No answer0 (0.0)2 (1.9)0 (0.0)2 (2.5)
      Medical concerns addressed0.220.09
       All80 (83.3)94 (90.4)53 (80.3)71 (89.9)
       Most12 (12.5)9 (8.7)10 (15.2)8 (10.1)
       Some4 (4.2)1 (1.0)3 (4.5)0 (0.0)
      Exploratory
      Telemedicine compared to office visit0.670.73
       Telemedicine better20 (20.8)23 (22.1)13 (19.7)21 (26.6)
       No difference22 (22.9)30 (28.8)19 (28.8)24 (30.4)
       Office visit better49 (51.0)48 (46.2)32 (48.5)32 (40.5)
       No answer5 (5.2)3 (2.9)2 (3.0)2 (2.5)
      Satisfaction with physician (0-10 scale), score ≥ 992 (95.8)102 (98.1)0.6163 (95.5)77 (97.5)0.84
      Visit is convenient, yes84 (87.5)94 (90.4)0.6758 (87.9)74 (93.7)0.36
      Perceived autonomy support, yes, HCCQ* ≥ 777 (80.2)80 (76.9)0.6957 (86.4)61 (77.2)0.23
      Telemedicine acceptability, TMPQ†, median [Q 25, Q 75]65.0

      [5.25, 68.00]
      65.0

      [59.00, 69.00]
      0.2065.5

      [55.75, 68.00]
      65.0

      [59.00, 69.00]
      0.36
      *HCCQ, healthcare climate questionnaire; †TMPQ, telemedicine perception questionnaire, higher values are better.
      When we examined the subgroups of sex, race, employment status, ADI, transportation availability, we found no heterogeneity of treatment effects by these characteristics (Fig. 3).
      Fig 3
      Fig. 3ORs and 95% CIs for satisfaction (scores of 9 or 10) rate in the phone-only versus video group for age-adjusted modified intent to treat analysis (mITT) and unadjusted subgroup analysis; OR, odds ratio, CI, confidence interval.

      Per protocol analysis

      Out of the 200 participants who completed post-visit surveys, 145 people had their randomized type of telemedicine visit (79 in the phone group [74.5%]; 66 in the video group [68.8%]). This population was included as the per protocol analysis. There were no significant differences in the sociodemographic characteristics of participants in the per protocol analysis. In this per protocol analysis, the unadjusted satisfaction rate difference between phone-only and video groups was -1.3% (95% CI, -12.6% to 10%), which did not contain the inferiority boundary of -15% (p = 0.01) (Fig. 2). After adjusting for age and insurance, the satisfaction rate in the phone-only group was lower than in the video group by -4.1% (95% CI -14.8% to 6.6%). The 95% lower bound confidence limit was -14.8%, which did not include the -15% noninferiority limit, establishing noninferiority.

      Sensitivity analyses

      In a sensitivity analysis that also included 10 participants who partially completed post-visit surveys, the phone-only visits remained noninferior to video visits with respect to the satisfaction rate in each group. The tipping point analysis imputation
      • Yan X
      • Lee S
      • Li N.
      Missing data handling methods in medical device clinical trials.
      using the anticipated satisfaction rates confirmed noninferiority (77.0% phone-only vs 73.1% video) as did the imputation using worst-best case scenario (66.7% phone-only vs 81.3%).

      Secondary outcomes

      In the mITT population, the proportion of participants who indicated they preferred a telemedicine visit of the same type for their next visit was similar (30.2% in the video vs 27.9% in the phone group, p = 0.78) (Table 2). In addition, the proportion of individuals who would definitely recommend telemedicine (71.9% video vs 71.2% phone-only group, p = 0.44) and the proportion of participants who believed that their concerns were addressed during the visit (83.3% video vs 90.4% phone-only group, p = 0.22) were similar. Interestingly, 20 (20.8%) participants in the video group and 23 (22.1%) in the phone-only group indicated that their telemedicine visit was superior to an office visit (p = 0.67). Telemedicine acceptability measured by the TMPQ score was high in both groups (65 [55.25, 68] phone vs 65 [59, 69] video, p = 0.2).

      Discussion

      In a randomized clinical trial, we found that the visit satisfaction rates (grouping satisfaction scores of 9 and 10, 0-10 scale) of established patients in cardiology, family medicine, and rheumatology clinics at a large multiple specialty clinic affiliated with an academic institution were high for both types of telemedicine visits and that the satisfaction rate with phone-only visits was not inferior to video visits. Indeed, the visit satisfaction rates we observed with phone-only telemedicine trended slightly higher than those for video visits. This finding may support some tendency in the older Medicare and younger Medicaid population to be more comfortable with phone-only rather than video visits, even among those who have video conferencing capability. The overall satisfaction with either modality was high and argues for reimbursement of both types of service at least and until the ease of use of videoconferencing by these populations becomes greater. To our knowledge, this is the first randomized trial to evaluate whether telemedicine phone-only visits are noninferior to video visits for visit satisfaction rate.
      Satisfaction with a healthcare service, as assessed using the CAHPS® measures, influences whether a patient is likely to use that service in the future and thus, it is important to evaluate whether new healthcare services, including the recent reliance on telemedicine for chronic disease care, meet patients’ needs. Because some patients may lack broadband internet access, newer telecommunication equipment, or digital literacy required for videoconferencing, our finding that phone-only visits are not inferior to video visits in terms of visit satisfaction rate is encouraging. Moreover, our results suggest that patients without access to these technologies do not forgo desired services. While a phone-only visit significantly limits the extent of the physical examination and the subtle non-verbal cues afforded to clinicians by a telemedicine video are missing, our results show that from patients’ point of view both types of telemedicine visits were associated with high satisfaction rates and were favorably experienced by patients who undergo periodic evaluations in rheumatology, cardiology, and family medicine clinics.
      The major strength of our study is the use of a study design that used randomization and blinding to minimize the potential for biases from either the providers or patients. Additional key strengths to our study are that we also targeted older and socioeconomically disadvantaged populations, recruited a large proportion of minority population (over half of the participants did not identify as white race) in whom healthcare disparities are well recognized, and, because the use of a waiver of consent, minimized selection bias of nonparticipation due to the consent process. Despite its strengths, our study has some limitations. We did not collect information about specific reasons or diseases evaluated during the telemedicine visit and it is possible that not all clinical conditions render themselves suitable for delivering/receiving care via different formats of telemedicine.
      • Wood PR
      • Caplan L.
      Outcomes, satisfaction, and costs of a rheumatology telemedicine program: a longitudinal evaluation.
      We collected outcomes via telephone interviews, and our estimates of patient satisfaction may be subject to a social desirability bias.
      • Fisher RJ.
      Social desirability bias and the validity of indirect questioning.
      However, the participants in both groups were blinded to the intervention and data collected by the same research assistants; therefore, it is unlikely that this social desirability bias would be differential between the two groups. As observed in past studies that used similar measures of visit satisfaction,
      • Voutilainen A
      • Pitkäaho T
      • Kvist T
      • et al.
      How to ask about patient satisfaction? The visual analogue scale is less vulnerable to confounding factors and ceiling effect than a symmetric likert scale.
      ,
      • Dell-Kuster S
      • Sanjuan E
      • Todorov A
      • et al.
      Designing questionnaires: Healthcare survey to compare two different response scales.
      the patient satisfaction scale we deployed demonstrated a ceiling effect. Since we compared the proportion of patients with high satisfaction rather than a continuous satisfaction rating, this issue is unlikely to affect either the reliability or validity of our findings. Nevertheless, this observation underscores the need for development of novel instruments to assess patient satisfaction that may be less prone to ceiling effects. Our results are not generalizable to the adult population who do not have access to video capacity, because we excluded these individuals in an attempt to ensure that each participant had equal opportunity to see their respective physicians using either technology. Our results are also limited to a small but important spectrum of medical specialties and a limited number of clinicians. Our per protocol analysis relied on a patient's report of the type of visit (phone-only vs video) they had because neither clinician notes, nor billing data, captured the type of telemedicine visit conducted, which could not be independently adjudicated by the research team. As such recall bias or information bias may have affected these results, but our sensitivity analyses suggest this would not have changed the findings.
      Spurred by the pandemic, home-based telemedicine has garnered substantial increased attention from patients, healthcare professionals and administrators, insurers, and policy makers. Our findings provide added data on patients’ acceptance and satisfaction with different types of telemedicine in populations of concern, which can inform clinical, regulatory, and administrative context of telemedicine and related reimbursement policies for medical care of patients with chronic diseases during and beyond the COVID-19 era.

      Declaration of Competing Interests

      The authors do not have any conflict of interest to disclose related to this work.

      Acknowledgements

      We thank the physicians and clinical staff who agreed to participate in this study.

      Funding

      This work was supported from University of Alabama at Birmingham institutional funds.

      References

        • Weinstein RS
        • Lopez AM
        • Joseph BA
        • et al.
        Telemedicine, telehealth, and mobile health applications that work: opportunities and barriers.
        Am J Med. 2014; 127: 183-187
        • Hollander JE
        • Carr BG.
        Virtually perfect? Telemedicine for covid-19.
        N Engl J Med. 2020; 382: 1679-1681
        • Bachireddy C
        • Chen C
        • Dar M.
        Securing the safety net and protecting public health during a pandemic: medicaid's response to COVID-19.
        JAMA. 2020; (Epublication)
        • Kuy S
        • Gupta R
        • Correa R
        • et al.
        Best practices for a covid-19 preparedness plan for health systems.
        NEJM Catal Innov Care Deliv. 2020; 1
        • Kulcsar Z
        • Albert D
        • Ercolano E
        • et al.
        Telerheumatology: A technology appropriate for virtually all.
        Semin Arthritis Rheum. 2016; 46: 380-385
        • McDougall JA
        • Ferucci ED
        • Glover J
        • et al.
        Telerheumatology: a systematic review.
        Arthritis Care Res. 2017; 69: 1546-1557
        • Piga M
        • Cangemi I
        • Mathieu A
        • et al.
        Telemedicine for patients with rheumatic diseases: systematic review and proposal for research agenda.
        Semin Arthritis Rheum. 2017; 47: 121-128
      1. American Telehealth Association. Home telehealth clinical guidelines. 2003, Accessed May 20, 2020.

        • Uscher-Pines L
        • Fischer S
        • Tong I
        • et al.
        Virtual first responders: The role of direct-to-consumer telemedicine in caring for people impacted by natural disasters.
        J Gen Intern Med. 2018; 33: 1242-1244
        • Hayes BL
        • Curtis JR
        • Laster A
        • et al.
        Osteoporosis care in the United States after declines in reimbursements for dxa.
        J Clin Densitom. 2010; 13: 352-360
        • Heath B
        • Salerno R
        • Hopkins A
        • et al.
        Pediatric critical care telemedicine in rural underserved emergency departments.
        Pediatr Crit Care Med. 2009; 10: 588-591
        • Feldman CH
        • Ramsey-Goldman R.
        Widening disparities among patients with rheumatic diseases in the COVID-19 era: An urgent call to action.
        Arthritis Rheumatol. 2020; (Epublication)
        • Nesbitt TS
        • Marcin JP
        • Daschbach MM
        • et al.
        Perceptions of local health care quality in 7 rural communities with telemedicine.
        J Rural Health. 2005; 21: 79-85
        • Richardson S
        • Hirsch JS
        • Narasimhan M
        • et al.
        Presenting characteristics, comorbidities, and outcomes among 5700 patients hospitalized with COVID-19 in the new york city area.
        JAMA. 2020; 323: 2052-2059
        • Yang J
        • Zheng Y
        • Gou X
        • et al.
        Prevalence of comorbidities in the novel wuhan coronavirus (COVID-19) infection: A systematic review and meta-analysis.
        Int J Infect Dis. 2020; (Accessed)
        • Zhou F
        • Yu T
        • Du R
        • et al.
        Clinical course and risk factors for mortality of adult inpatients with COVID-19 in wuhan, china: A retrospective cohort study.
        Lancet. 2020; 395: 1054-1062
        • Wu C
        • Chen X
        • Cai Y
        • et al.
        Risk factors associated with acute respiratory distress syndrome and death in patients with coronavirus disease 2019 pneumonia in wuhan, china.
        JAMA Intern Med. 2020; (Accessed)
        • Nouri S
        • Khoong EC
        • Lyles CR
        • et al.
        Addressing equity in telemedicine for chronic disease management during the covid-19 pandemic.
        NEJM Catal Innov Care Deliv. 2020; 1
        • Eberly LA
        • Kallan MJ
        • Julien HM
        • et al.
        Patient characteristics associated with telemedicine access for primary and specialty ambulatory care during the COVID-19 pandemic.
        JAMA Network Open. 2020; 3 (e2031640-e2031640)
        • Saliba-Gustafsson EA
        • Miller-Kuhlmann R
        • Kling SMR
        • et al.
        Rapid implementation of video visits in neurology during COVID-19: Mixed methods evaluation.
        J Med Internet Res. 2020; 22: e24328
        • Jha AK
        • Orav EJ
        • Zheng J
        • et al.
        Patients' perception of hospital care in the United States.
        N Engl J Med. 2008; 359: 1921-1931
        • Viswanathan M
        • Golin CE
        • Jones CD
        • et al.
        Interventions to improve adherence to self-administered medications for chronic diseases in the United States: a systematic review.
        Ann Intern Med. 2012; 157: 785-795
        • Scott Kruse C
        • Karem P
        • Shifflett K
        • et al.
        Evaluating barriers to adopting telemedicine worldwide: a systematic review.
        J Telemed Telecare. 2018; 24: 4-12
        • McCambridge J
        • Witton J
        • Elbourne DR.
        Systematic review of the hawthorne effect: new concepts are needed to study research participation effects.
        J Clin Epidemiol. 2014; 67: 267-277
        • Harris PA
        • Taylor R
        • Thielke R
        • et al.
        Research electronic data capture (redcap)–a metadata-driven methodology and workflow process for providing translational research informatics support.
        J Biomed Inform. 2009; 42: 377-381
        • Stewart AL
        • Ware JE.
        Measuring functioning and well-being: The medical outcomes study approach. Accessed.
        duke university Press, 1992
        • Chew LD
        • Bradley KA
        • Boyko EJ.
        Brief questions to identify patients with inadequate health literacy.
        Fam Med. 2004; 36: 588-594
        • Singh GK.
        Area deprivation and widening inequalities in us mortality, 1969-1998.
        Am J Public Health. 2003; 93: 1137-1143
        • Kind AJ
        • Jencks S
        • Brock J
        • et al.
        Neighborhood socioeconomic disadvantage and 30-day rehospitalization: A retrospective cohort study.
        Ann Intern Med. 2014; 161: 765-774
      2. Agency for Healthcare Research and Quality. Consumer assessment of healthcare providers and systems. Accessed May 30, 2020.

        • Walter SD
        • Turner RM
        • Macaskill P
        • et al.
        Estimation of treatment preference effects in clinical trials when some participants are indifferent to treatment choice.
        BMC Med Res Methodol. 2017; 17: 29
        • Genevay S
        • Finckh A
        • Ciurea A
        • et al.
        Tolerance and effectiveness of anti-tumor necrosis factor alpha therapies in elderly patients with rheumatoid arthritis: A population-based cohort study.
        Arthritis Rheum. 2007; 57: 679-685
        • Demiris G
        • Speedie S
        • Finkelstein S.
        A questionnaire for the assessment of patients' impressions of the risks and benefits of home telecare.
        J Telemed Telecare. 2000; 6: 278-284
        • Williams GC
        • Gagné M
        • Ryan RM
        • et al.
        Facilitating autonomous motivation for smoking cessation.
        Health Psychol. 2002; 21: 40-50
        • Mukherjee S
        • Rodriguez HP
        • Elliott MN
        • et al.
        Modern psychometric methods for estimating physician performance on the clinician and group cahps® survey.
        Health Services Outcomes Res Methodol. 2013; 13: 109-123
      3. Quality AfHRa. Consumer assessment of healthcare providers and systems.

        • Donelan K
        • Barreto EA
        • Sossong S
        • et al.
        Patient and clinician experiences with telehealth for patient follow-up care.
        Am J Manag Care. 2019; 25: 40-44
      4. United States Food & Drug Administration. Guidance document: Non-inferiority clinical trials.2016.

        • Yan X
        • Lee S
        • Li N.
        Missing data handling methods in medical device clinical trials.
        J Biopharm Stat. 2009; 19: 1085-1098
        • Wood PR
        • Caplan L.
        Outcomes, satisfaction, and costs of a rheumatology telemedicine program: a longitudinal evaluation.
        J Clin Rheum: Practical Rep Rheumatic Musculoskelet. Dis. 2019; 25: 41-44
        • Fisher RJ.
        Social desirability bias and the validity of indirect questioning.
        J Consum Res. 1993; 20: 303-315
        • Voutilainen A
        • Pitkäaho T
        • Kvist T
        • et al.
        How to ask about patient satisfaction? The visual analogue scale is less vulnerable to confounding factors and ceiling effect than a symmetric likert scale.
        J Adv Nurs. 2016; 72: 946-957
        • Dell-Kuster S
        • Sanjuan E
        • Todorov A
        • et al.
        Designing questionnaires: Healthcare survey to compare two different response scales.
        BMC Med Res Method. 2014; 14: 1-13