|Ahead of print
|Randomized controlled trials of remdesivir in hospitalized coronavirus disease 2019 patients: A meta-analysis
Azza Sarfraz1, Zouina Sarfraz2, Marcos Sanchez-Gonzalez3, Jack Michel3, George Michel3, Odalys Frontela3, Jorge Posada3, Jose Cardona3, Eugenio Angueira3
1 Department of Pediatrics and Child Health, Aga Khan University, Karachi, Pakistan; Division of Research and Academic Affairs, Larkin Health System, South Miami, Florida, USA
2 Division of Research and Academic Affairs, Larkin Health System, South Miami, Florida, USA; Department of Research and Publication, Fatima Jinnah Medical University, Lahore, Pakistan
3 Division of Research and Academic Affairs, Larkin Health System, South Miami, Florida, USA
Click here for correspondence address and email
|Date of Submission||10-Sep-2020|
|Date of Decision||04-Oct-2020|
|Date of Acceptance||06-Oct-2020|
|Date of Web Publication||12-Feb-2021|
BACKGROUND: The first cases of the coronavirus disease 2019 (COVID-19) were reported in Wuhan, China. No antiviral treatment options are currently available with proven clinical efficacy. However, preliminary findings from phase III trials suggest that remdesivir is an effective and safe treatment option for COVID-19 patients with both moderate and severe disease.
OBJECTIVE: The aim of the present meta-analysis was to investigate whether remdesivir was effective for treating COVID-19 including reduced in-hospital adverse events, oxygen support, and mortality rates.
METHODS: According to the PRISMA reporting guidelines, a review was conducted from January 1, 2020, until August 25, 2020, with MeSH terms including COVID-19, COVID, coronavirus, SARS-CoV-2, remdesivir, adenosine nucleoside triphosphate analog, and Veklury using MEDLINE, Scopus, and CINAHL Plus. A modified Delphi process was utilized to include the studies and ensure that the objectives were addressed. Using dichotomous data for select values, the unadjusted odds ratios (ORs) were calculated applying Mantel–Haenszel random-effects method in Review Manager 5.4.
RESULTS: Randomized controlled trials pooled in 3013 participants with 46.3% (n = 1395) in the remdesivir group and 53.7% (n = 1618) in the placebo group. The placebo group had a higher risk of mortality as compared to the intervention group with significant OR (0.61) (95% confidence interval of 0.45–0.82; P = 0.001). There was minimal heterogeneity among the studies (I2 = 0%).
CONCLUSIONS: Our findings suggest that remdesivir extends clinical benefits by reducing mortality, adverse events, and oxygen support in moderate to severely ill COVID-19 patients. Concerted efforts and further randomized placebo-controlled trials are warranted to examine the potency of antiviral drugs and immunopathological host responses contributing to the severity of COVID-19.
Keywords: Coronavirus, emergency use authorization, hospitalized, remdesivir, Veklury
|How to cite this URL:|
Sarfraz A, Sarfraz Z, Sanchez-Gonzalez M, Michel J, Michel G, Frontela O, Posada J, Cardona J, Angueira E. Randomized controlled trials of remdesivir in hospitalized coronavirus disease 2019 patients: A meta-analysis. Turk J Emerg Med [Epub ahead of print] [cited 2022 Jun 27]. Available from: https://www.turkjemergmed.org/preprintarticle.asp?id=309139
| Introduction|| |
Since the first cases of the coronavirus disease 2019 (COVID-19) were reported in Wuhan, Hubei Province, China, in December 2019, a large-scale spread internationally led the World Health Organization (WHO) to declare COVID-19 as a public health emergency of international concern on January 30, 2020. Antiviral treatment options of proven clinical efficacy in COVID-19 infections are under investigation. Remdesivir is an investigational nucleotide prodrug which intracellularly metabolizes to the active nucleoside triphosphate and interferes with viral RNA-dependent RNA polymerase activity, thereby disrupting viral exoribonuclease activity. However, the pharmacokinetics of remdesivir within the respiratory tract of critically ill COVID-19 patients are not well known. Hospitalized COVID-19 patients with oxygen saturation ≤94% on room air or requiring oxygen support are eligible to receive remdesivir under the US Food and Drug Administration (FDA) emergency use authorization (EUA). While previous studies have reported a reduction in median time to clinical improvement, insufficient power of sample sizes limited the deductibility of clinical outcomes of remdesivir. In addition, initiating remdesivir earlier in the COVID-19 treatment protocols must be considered before immune-mediated epithelial damage due to the fact that elevated viral replication occurs and may reduce mortality and disease severity as observed previously in severe acute respiratory syndrome (SARS) and Middle East respiratory syndrome (MERS).
Based on the preliminary reports and findings from in vitro and in vivo activity in animal models of SARS-CoV-1 and MERS-CoV, remdesivir treatment for 5 or 10 days is being administered to COVID-19 patients with comparable efficacy and safety.,, While most COVID-19 infections are self-limiting, some individuals may contract severe disease warranting increased length of hospital stay and ventilator support, which places burden on health infrastructures. Use of remdesivir has resulted in reduced oxygen support in a cohort with 53 hospitalized COVID-19 patients. Consequently, with revised recommendations suggesting uncertain efficacy of remdesivir and benefits among patients already on high-flow oxygen, mechanical ventilation, or extracorporeal membrane oxygenation (ECMO), the initiation and duration of remdesivir treatment among COVID-19 hospitalized patients receiving oxygen support remain unclear. Given the uncertainty on the beneficial outcomes of remdesivir-treated COVID-19 patients, we aimed to examine the following differences between remdesivir and placebo groups: (1) oxygen support status at day 1 and day 14, (2) any adverse events at day 14, and (3) death from any cause at day 14.
| Methods|| |
According to the PRISMA reporting guidelines, a review was conducted from January 1, 2020, until August 6, 2020, with MeSH terms including “COVID-19,” “coronavirus,” “SARS-CoV-2,” “COVID,” “remdesivir,” “adenosine nucleoside triphosphate analog,” “Veklury” using Medline, Scopus, and CINAHL Plus. Quantitative primary research articles were added to the systematic review and meta-analysis. The inclusion criteria of the included studies were having COVID-19–infected patients aged 18 or older being treated with remdesivir and/or placebo. Duplicates were removed using EndNote X9 a reference management software package developed by Clarivate Analytics. We manually cross-checked the searches for authors, title, and abstract to remove duplicates.
Two investigators (AS and ZS) independently screened the titles and abstracts before reaching to a consensus to determine the included studies. The third investigator (MSG) was present for any disagreements. Exclusion criteria were applied to full texts during the final selection. A modified Delphi process was used to include studies and to ensure that our objective was identified in selected studies. The a priori methods for conducting the Delphi process for meta-analyzing the clinical effectiveness are described in [Figure 1]. We included studies if they were randomized control trials, they had an intervention arm as compared to placebo, and the end point of interest was clinical outcomes and mortality. Two investigators (AS and ZS) re-confirmed all data entries and checked imported data from all studies at least thrice for accuracy.
We evaluated the risk of bias for all included studies using the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) criteria. We aimed to evaluate the risk of bias associated with the selection of participants, confounding, and health outcome assessment. In doing so, we found the risk of bias of all four individual studies included for quantitative analysis using the GRADE criteria. Since less than 10 studies were included, we did not check for publication bias using funnel plots.
The primary outcomes included death from any cause at day 14. The secondary outcomes were to identify any adverse events at day 14 of the treatment and the requirement for supplemental oxygen, high-flow nasal cannula, noninvasive ventilation, invasive ventilation, or ECMO at day 1 and 14. The time to recovery in days, total patients recovered, and findings of serious adverse effects among remdesivir and placebo groups were identified.
Two independent reviewers (AS and ZS) assessed the eligibility of all full-text articles; the third (MSG) arbitrated for cases to reach a consensus. The first reviewer (AS) extracted the data, and the second reviewer (ZS) validated the data extraction for all studies. The quantitative data were entered into a spreadsheet. If more than one study reported data on posttreatment outcomes, data were extracted separately for each study. We independently extracted data from the published randomized placebo-controlled trials.
Using dichotomous data for select values, summary measures namely the unadjusted odds ratios (ORs) were calculated using the Mantel–Haenszel random-effects methods. We calculated the ORs and 95% confidence intervals (CIs) for each measure evaluated in two or more studies. A meta-analysis was conducted using Review Manager V.5.4 the desktop version of the software used for review formats (diagnostic, methodology, overviews), for non-Cochrane reviews, and for offline working, developed by Cochrane Training. Findings were presented using 95% CIs along with the I2 test for heterogeneity between studies.
Source of funding
No funding was obtained for the purpose of this study.
| Results|| |
The search process is shown in [Figure 2]. The initial screening yielded 1242 results. After the exclusion of duplicates, 946 results were withheld for the screening of title and abstract. Consequently, 704 records were excluded due to ineligibility (reviews, editorials, non-random controlled trials, ongoing trials, and abstracts). Finally, after screening 242 full-text articles, only four studies reporting 3013 patients (remdesivir n = 1395; placebo n = 1618) were included in the qualitative and quantitative syntheses. While three out of four studies were of high quality based on the GRADE scale, Olender et al.'s study was graded moderate.
Mortality at day 14 of treatment
All four studies reported the data on mortality at day 14 and thus were eligible to be included in the meta-analysis. Compared with the remdesivir-treated group, the placebo group had higher risks of mortality (OR: 0.61; 95% CI: 0.45–0.82; P = 0.001) [Figure 3]. For the sensitivity analysis, we tested if the removal of study by Beigel et al. would lead to changes in the OR and significance. After excluding this study, the results suggested that the risk of mortality was still higher in the placebo group (OR: 0.66; 95% CI: 0.44–0.98; P = 0.04), with homogenous findings (I2 = 0%).
Supplemental oxygen at day 1 and 14 of treatment
All four studies presented the data of supplemental oxygen requirement at day 1 of treatment among the remdesivir and placebo groups. Using a random-effects model, we determined that the remdesivir group had similar odds as compared to the placebo group in requiring supplemental oxygen at the 1st day of treatment (OR: 1.03; CI: 0.87–1.23; P = 0.70), with limited heterogeneity among all studies (I2 = 8%) [Figure 3].
Three out of four studies evaluated the supplemental oxygen use at day 14 of the treatment among the remdesivir group and the placebo group. However, there was a higher likelihood of the placebo group to require supplemental oxygen at the end of the 2nd week of treatment (OR: 0.88; CI: 0.62–1.24; P = 0.46), with no heterogeneity among the studies (I2 = 1%) [Figure 3].
High-flow nasal cannula or noninvasive mechanical ventilation at day 1 and 14 of treatment
All four studies presented the data of high-flow nasal cannula or noninvasive mechanical ventilation required at day 1 of treatment. Patients in the placebo group as compared to the remdesivir group had high odds of requiring high-flow nasal cannula or noninvasive mechanical ventilation (OR: 0.81; CI: 0.64–1.04; P = 0.10; I2 = 9%) [Figure 3].
Three of the four studies presented the requirements of high-flow nasal cannula or noninvasive mechanical ventilation at day 14 of treatment. The likelihood of the placebo group was higher as compared to the remdesivir group of requiring intervention (OR: 0.90; CI: 0.53–1.53; P = 0.69), with no heterogeneity among the studies (I2 = 0%) [Figure 3].
Invasive ventilation or extracorporeal membrane oxygenation at day 1 and 14 of the treatment
Three of the four studies presented the data of invasive ventilation or ECMO at the 1st day of treatment. While the difference was negligible, there was a very slight preponderance of the remdesivir group to require invasive ventilation or ECMO at day 1 of the treatment (OR: 1.06; CI: 0.73–1.54; P = 0.77; I2 = 28%) [Figure 3].
Three of the four studies reported the data on invasive ventilation or ECMO at day 14 of the treatment. Patients in the placebo group had a higher likelihood of requiring invasive ventilation or ECMO at the 2nd week of the treatment as compared to the patients in the remdesivir group (OR: 0.39; CI: 0.13–1.14; P = 0.09) [Figure 3]. There was moderately high heterogeneity among the studies included for the analysis (I2 = 62%).
Overall serious adverse events after initiation of treatment
Three of the four studies reported the data on the overall serious adverse effects initiation of treatment, and thus, they were included in the meta-analyses. The placebo group had a higher risk or likelihood of presenting with adverse outcomes as compared to the remdesivir group but with less statistical significance (OR: 0.75; CI: 0.55–1.02; P = 0.07) [Figure 3]. There was mild heterogeneity between the studies (I2 = 26%).
| Discussion|| |
The purpose of the study was to comprehensively review the efficacy of remdesivir compared to placebo among hospitalized patients with moderate and severe COVID-19. Our inclusion criteria, determined by the input of all panel members, were specific for adult hospitalized COVID-19 patients treated with either remdesivir or placebo, which distinguishes the findings from other meta-analyses. Based on the analysis of four randomized placebo-controlled trials, the overall findings support the use of remdesivir to reduce oxygen support, adverse events, and all-cause mortality after 5 or 10 days of remdesivir treatment.,, Overall, the mortality rate for remdesivir-treated patients with COVID-19 of the three included studies ranged 1.3%–10% as compared to the 2%–12.5% mortality rates of the placebo-treated patients. This findings were consistent with recent clinical data reporting positive outcomes for the compassionate use of remdesivir in moderate and severe COVID-19 patients.,,
The time to clinical recovery was significantly lower among patients who received remdesivir compared to placebo across two studies (21 vs. 23 days and 11 vs. 15 days). A randomized, open-label, phase 3 three-arm trial including 584 patients with moderate COVID-19 disease compared the efficacy of 5- and 10-day courses of remdesivir treatment, compared with standard care. The median time to clinical recovery across the 5- and 10-day treatment course was 6 and 8 days, respectively, with recovery in the standard care group being 7 days. There were observed differences in the requirements of supplemental oxygen with the remdesivir group requiring less supplemental oxygen at day 14 than the placebo group with day-1 data, demonstrating significant use of supplemental oxygen in the remdesivir group. While there was a very slight preponderance of the remdesivir group to require the use of high-flow nasal cannula or noninvasive mechanical ventilation at day 1, the remdesivir group had reduced likelihood of being on invasive ventilation or ECMO at day 14. Along with reduced overall oxygen support required in the remdesivir group, the all-cause mortality and any adverse events were significantly reduced in the remdesivir group in comparison to the placebo group. An analysis of 138 healthy volunteers was treated with remdesivir, and it appears to have a safe clinical profile and is well-tolerated with transaminase elevation identified as the only adverse event. Special attention should be given to renal events, pregnancy, hypersensitivity reactions, and concomitant vasopressor use before remdesivir initiation.
To the best of our knowledge, this is the first meta-analysis and systematic review of remdesivir and control groups, which determines oxygen support status at day 1 and 14, any adverse events at day 14, and all-cause mortality at day 14. We synthesize various clinical outcomes of interest using statistical analysis methods that are widely applicable and relevant to key stakeholders in healthcare. Our results demonstrate that remdesivir use in adults may improve in-hospital mortality, oxygen support status and adverse events within 2 weeks of treatment initiation. Our findings synthesize the results of primary and secondary outcomes of ongoing or completed clinical trials.,, FDA's press release on August 28, 2020, broadened the EUA for remdesivir (e.g., Veklury) to include all hospitalized patients for the treatment of COVID-19. The scope of existing authorization is based on the conclusions that remdesivir is an effective treatment option for suspected or laboratory-confirmed COVID-19 patients in hospitalized settings, with further trials required to explore the efficacy according to the clinical stratification.
We found over 35 trials registered on the ClinicalTrials.gov classified as remdesivir group versus placebo group using 200 mg loading dose on the 1st day, followed by 100 mg intravenous once-daily doses for 5–10 days. The outcomes of the ongoing trials are to determine the time to clinical improvement, clinical status, time to hospital discharge, all-cause mortality, duration of mechanical ventilation, ECMO, supplemental oxygen, length of hospital stay, change in viral load assessed by area under viral load curve, and the frequency of adverse events.
The baseline health and disease severity were not matched in the remdesivir and placebo groups in our included studies. In addition, the use of remdesivir in high-risk populations, e.g., elderly age, multiple comorbidity, Blacks, and sociodemographic disparity, may be considered before moderate or severe COVID-19 manifestations occur. The most adequate time of administering antiviral treatment is soon after the onset of disease to promote benefits, with previous reports recommending initiation within 5 or 10 days after the onset of symptoms.,, Early results based on the interim data may lack generalizability, but the use of remdesivir has already obtained an approval for EUA by the US FDA. The benefits of administrating remdesivir may outweigh the risks in hospitalized COVID-19 patients with oxygen saturation below 94%. Patients who have been intubated for a short period can also benefit from remdesivir dosage every 24–28 h. However, limited clinical effectiveness is expected among patients being mechanically ventilated.
In addition, the next steps in finding a consensus toward remdesivir use follow the evaluation of potential short-term and long-term side effects of remdesivir, taking into consideration the concomitant use of other medication. For instance, the off-label use of medications such as lopinavir–ritonavir, hydroxychloroquine, and immunomodulatory drugs including glucocorticoids and tocilizumab may confound reports of currently promising and beneficial outcomes of remdesivir use.
Reporting biases of currently published trial results may be taken into consideration. The clinical benefits ought to be predicted within all severity subgroups to confer rigorous support for clinical guidance toward remdesivir. As the world strives to overcome structural and social healthcare disparities, we must accentuate the underrepresentation or lack of available data interpreting the incidence and clinical outcomes of minority groups in the remdesivir COVID-19 trials. In a preliminary cohort study published by Grein et al., data of ethnicity were omitted for 53 patients. While the vetting for preliminary results was obtained from limited datasets, the proportion of Black, Latinx, and Native Americans were around 20%, 23%, and 0.7% respectively, in trials published by Beigel et al. and Goldman et al., In addition, while Asia's population is roughly equivalent to 60% of the world population, Spinner et al.'s trial only consisted of 17.5% Asian participants. The modest benefits in time to clinical improvement post treatment may not be generalizable in minority groups due to probable differences in severity of disease, outcomes, and treatment efficacy. The lack of ethnic diversity in clinical studies during the COVID-19 pandemic ought to be addressed at the administrative level by mandating the inclusion of minority groups and reporting data at the governmental level. A prioritization of populations reflecting the demographics of high-risk groups impacted by the ongoing pandemic is crucial, by expanding clinical trial sites and employing random sampling.
Our findings were limited due to a paucity of available data between a 5-day and a 10-day course of intravenous remdesivir treatment among severe and moderate COVID-19 patients, with only one randomized placebo trial reporting these findings. All studies had open-label designs, which potentially led to biases in both patient care and reporting of data. Another limitation was the lack of corroboration of clinical efficacy with the viral loads of the patients in both groups. While the biological mechanisms of remdesivir are required to interpret the clinical efficacy, not all studies reported the viral loads in our meta-analysis.
| Conclusion|| |
Our findings provide strong evidence of clinical improvement in randomized, placebo-controlled trials of remdesivir therapy. Implications of our meta-analyses results are strong with a moderately large sample size and randomized placebo group. Ongoing placebo-controlled trails employing larger sample sizes will remain our informative source of the outcomes and adverse events of remdesivir administered to hospitalized COVID-19 patients. Strategies must be used to enhance the potency of remdesivir while reducing the immunopathological host responses that contribute to the infection severity. In addition, the efficacy of 5 versus 10 days dosing of remdesivir warrants further exploration. Our findings suggest that remdesivir merits extended clinical use and may also be efficacious among nonsevere hospitalized COVID-19 patients.
Financial support and sponsorship
Conflicts of interest
| References|| |
WHO. Events As They Happen. Rolling Updates on Coronavirus Disease (COVID-19). Geneva, Switzerland: WHO; 2020.
Eastman RT, Roth JS, Brimacombe KR, Simeonov A, Shen M, Patnaik S, et al
. Remdesivir: A review of its discovery and development leading to emergency use authorization for treatment of COVID-19. ACS Cent Sci 2020;6:672-83.
Wang Y, Zhang D, Du G, Du R, Zhao J, Jin Y, et al
. Remdesivir in adults with severe COVID-19: A randomised, double-blind, placebo-controlled, multicentre trial. Lancet 2020;395:1569-78.
Sheahan TP, Sims AC, Graham RL, Menachery VD, Gralinski LE, Case JB, et al
. Broad-spectrum antiviral GS-5734 inhibits both epidemic and zoonotic coronaviruses. Sci Transl Med 2017;9:1-10. [doi: 10.1126/scitranslmed.aal3653].
Goldman JD, Lye DC, Hui DS, Marks KM, Bruno R, Montejano R, et al
. Remdesivir for 5 or 10 days in patients with severe COVID-19. N Engl J Med 2020:1-10. [doi: 10.1056/nejmoa2015301].
Sheahan TP, Sims AC, Leist SR, Schäfer A, Won J, Brown AJ, et al
. Comparative therapeutic efficacy of remdesivir and combination lopinavir, ritonavir, and interferon beta against MERS-CoV. Nat Commun 2020;11:222.
Yethindra V. Role of GS-5734 (remdesivir) in inhibiting SARS-CoV and MERS-CoV: The expected role of GS-5734 (remdesivir) in COVID-19 (2019-nCoV)-VYTR hypothesis. Int J Res Pharm Sci 2020;1:1-6.
Hua W, Xiaofeng L, Zhenqiang B, Jun R, Ban W, Liming L. Consideration on the strategies during epidemic stage changing from emergency response to continuous prevention and control. Chinese J Endem 2020;41:297-300.
Grein J, Ohmagari N, Shin D, Diaz G, Asperges E, Castagna A, et al
. Compassionate use of remdesivir for patients with severe Covid-19. N Engl J Med 2020;382:2327-36.
Holshue ML, DeBolt C, Lindquist S, Lofy KH, Wiesman J, Bruce H, et al.
First case of 2019 novel coronavirus in the United States. N Engl J Med 2020;382:929-36.
Spinner CD, Gottlieb RL, Criner GJ, López JRA, Cattelan AM, Viladomiu AS et al
. Effect of Remdesivir vs Standard Care on Clinical Status at 11 Days in Patients With Moderate COVID-19: A Randomized Clinical Trial. JAMA. 2020;324:1048–57. doi:10.1001/jama.2020.16349.
Olender SA, Perez KK, Go AS, Balani B, Price-Haywood EG, Shah NS, et al
. Remdesivir for severe COVID-19 versus a cohort receiving standard of Care. Clin Infect Dis. 2020 Jul 24:ciaa1041. doi: 10.1093/cid/ciaa1041. Epub ahead of print.
Beigel JH, Tomashek KM, Dodd LE, Mehta AK, Zingman BS, Kalil AC, et al
. Remdesivir for the treatment of COVID-19-Preliminary report. N Engl J Med 2020:Oct 8:NEJMoa2007764. Epub ahead of print. [doi: 10.1056/nejmoa2007764].
Wang Y, Zhou F, Zhang D, Zhao J, Du R, Hu Y, et al
. Evaluation of the efficacy and safety of intravenous remdesivir in adult patients with severe COVID-19: study protocol for a phase 3 randomized, double-blind, placebo-controlled, multicentre trial. Trials 2020;21:422.
Ferner RE, Aronson JK. Remdesivir in COVID-19. BMJ 2020;369:m1610
Davies M, Osborne V, Lane S, Roy D, Dhanda S, Evans A, et al
. Remdesivir in treatment of COVID-19: A systematic benefit-risk assessment. Drug Saf 2020;43:645-56.
Golestaneh L, Neugarten J, Fisher M, Billett HH, Gil MR, Johns T, et al
. The association of race and COVID-19 mortality. EClinicalMedicine 2020;25:100455.
Treanor JJ, Hayden FG, Vrooman PS, Barbarash R, Bettis R, Riff D, et al.
Efficacy and safety of the oral neuraminidase inhibitor oseltamivir in treating acute influenza: A randomized controlled trial. US Oral Neuraminidase Study Group. JAMA 2000;283:1016-24.
Mertz GJ, Critchlow CW, Benedetti J, Reichman RC, Dolin R, Connor J, et al.
Double-blind placebo-controlled trial of oral acyclovir in first-episode genital herpes simplex virus infection. JAMA 1984;252:1147-51.
Chastain DB, Osae SP, Henao-Martínez AF, Franco-Paredes C, Chastain JS, Young HN. Racial disproportionality in COVID clinical trials. N Engl J Med 2020;383:e59.
Rochwerg B, Agarwal A, Zeng L, Leo YS, Appiah JA, Agoritsas T, et al
. Remdesivir for severe COVID-19: A clinical practice guideline. BMJ 2020;370:m2924.
Department of Pediatrics and Child Health, Faculty Office Building, Aga Khan University, P.O Box 3500, Stadium Road, Karachi 74800
Source of Support: None, Conflict of Interest: None
[Figure 1], [Figure 2], [Figure 3]
| Article Access Statistics|
| Viewed||1014 |
| PDF Downloaded||39 |
| Cited by others ||4 |