Systematic review of near patient test evaluations in primary care
BMJ 1999; 319 doi: https://doi.org/10.1136/bmj.319.7213.824 (Published 25 September 1999) Cite this as: BMJ 1999;319:824
- Brendan C Delaney, senior lecturera,
- Chris J Hyde, director, aggressive research intelligence facilityb,
- Richard J McManus, clinical research fellowa,
- Sue Wilson, senior research fellowa,
- David A Fitzmaurice, senior lecturera,
- Sue Jowett, research associatea,
- Ros Tobias, research fellowa,
- Gary H Thorpe, directorc,
- F D Richard Hobbs, professor (f.d.r.hobbs{at}bham.ac.uk)a
- a Department of Primary Care and General Practice, University of Birmingham, Birmingham B15 2TT
- b Department of Public Health and Epidemiology, University of Birmingham
- c Wolfson Applied Technology Laboratory, University of Birmingham
- Correspondence to: F D R Hobbs
- Accepted 9 May 1999
Abstract
Objective: To identify and qualitatively synthesise the findings from all studies that have examined the performance and effect of near patient tests in the primary care setting.
Design: Systematic review of published and unpublished research 1986-99.
Main outcome measures: Test performance characteristics, measures of effect on clinical practice or patient outcome.
Results: 101 relevant publications were identified. The general quality of these papers was low, and consequently only 32 papers were assessed in detail. Although these papers gave some indication of the value of near patient testing in areas such as anticoagulation monitoring and group A β haemolytic streptococcus testing, the research raised many more questions than it answered. Almost no reports were found of unbiased assessment of the effect of near patient tests in primary care on patient outcomes, organisational outcomes, or cost.
Conclusions: Available research provides little evidence to guide the expansion of use of near patient testing in primary care. Further research is needed in areas of clinical practice where near patient tests might be most beneficial.
Key messages
Near patient testing is a rapidly evolving technology with potential to improve the quality of diagnosis and management in primary care
The performance of most tests has not been adequately evaluated in primary care
No robust studies of the effectiveness of near patient tests in improving patient outcomes have been conducted
High quality evaluations of the performance and effectiveness of near patient tests in defined clinical situations are needed before further expansion
Introduction
Near patient testing is defined as any investigation carried out in a clinical setting or the patient's home for which the result is available without reference to a laboratory and perhaps rapidly enough to affect immediate patient management.1 Technological advances in solid phase chemistry and miniaturisation of analysers have spawned a huge expansion of potential near patient tests.2 In the United States, near patient or point of care testing comprises a fifth of all diagnostic testing.3 European countries are following suit.4–6 Near patient tests for blood sugar and urine analysis are routine in most British primary care centres, and some practices also use tests for Helicobacter pylori,7 haemoglobin, and international normalised ratio.8
Near patient testing has potential benefits for primary care, in particular having the test result immediately available.3 However, these benefits may be offset by reduced accuracy compared with a laboratory result, and the strategy assumes that an immediate result will actually make a difference to the patient's management and outcome.
The effectiveness of near patient testing is likely to vary according to the circumstances of its use (population, setting, operator, and clinical value of the result). Rigorous evaluation is therefore required of the growing number of tests being marketed for use in primary care 9 10 This systematic review examines the evidence on the performance and effectiveness of near patient tests in primary care to determine whether the continuing expansion of these tests is supportable and, if not, what further evidence is needed before implementation of near patient tests.
Methods
We sought evaluations of the performance and effect of near patient tests in primary care published between January 1986 and December 1996 by computerised searches of electronic databases (Medline, Embase, Science Citation Index, GP-Lit, CINAHL) using the keywords near patient test, point of care, home test, rapid test, desktop/office test/laboratory in primary care (or accepted synonyms). We also did a hand search of trade journals and primary care conference proceedings, conducted a postal survey of researchers and companies active in the field, and examined the reference lists of articles found by these sources.11 12
All publications identified were examined independently by two assessors (BD and DF) for relevance either to near patient testing or primary care.11 Remaining articles were then considered for validity by one of the authors and an external reviewer against standard appraisal criteria for performance of diagnostic tests.13 14 This provided a broad assessment of openness to bias on a five point scale (0 greatest potential for bias; 5 least potential for bias).
Papers that received a methodological score of 4 or 5 were considered in detail by a reviewer (CH) who had not been directly involved in selection of papers. Characteristics of the papers were tabulated, and the quality of the study assessed in detail. A seven point checklist was used to assess the validity of any measures of accuracy of diagnostic tests,15 paying particular attention to the possibility of spectrum, work up, and review or verification biases. For assessment of the small numbers of papers which had evaluated effectiveness as well as performance we used the framework suggested by the Cochrane Collaboration.16 No formal score was applied owing to the extreme heterogeneity of study designs encountered.
The search of electronic databases was subsequently updated to cover January 1997 to February 1999. Relevant papers were selected and appraised by two reviewers (BD and RM).
Results
We identified 904 unique papers. Of these, 90 were relevant to near patient testing in primary care. Only 26 scored 4 or 5 on the initial assessment of validity. The main reasons for failure to reach the cut off were absence or inadequacy of the reference standard and inappropriate statistical analyses. The additional electronic searches for 1997-9 found 11 more relevant papers, of which six passed the quality filter.
The 32 papers described 209 comparisons. Of these, 49 related to repeatability (intraobserver variability) of tests, which is not considered further here. The most interesting data emerged in the comparisons of test performance (n=150) and impact (n=10). Tables giving the key points from these papers are available on the BMJ's website. Test performance and impact were considered separately. Details of the other relevant papers have been published.11
We extended the traditional view of test performance to include comparisons of the same test when operated and read by different people. This is often considered as an aspect of repeatability—interobserver variability However, results obtained with near patient tests by trained technicians in hospital settings are likely to overestimate the performance obtained by non-technicians in primary care.17 18 An example of this problem is the evaluation of urine test strips for detecting urine infection. A study in one British practice found a satisfactory negative predictive value of 92% for nitrite and leucotest pads together.19 However, a multicentre study in the Netherlands found a negative predictive value of only 57%.20 The differences between the studies include the definition of the standard for midstream urine, the prevalence of infection, the subjects tested, and the number of operators. Not only are we left uncertain about the performance of the test (although the larger study was objectively of better quality) but we also lack information about the value of using urine analysis in diagnosing suspected urine infection or haematuria21 in primary care.
Assessments of other microbiology near patient tests (streptococcal throat tests,22–25 Helicobacter pylori,26–28 and Plasmodium falciparum antigens29) were severely hampered by a lack of agreement on the standard for evaluation. For example, the three H pylori studies used the same test (Helisal Rapid Blood, Cortecs) but different reference standards.
A randomised controlled trial in Birmingham showed that measurement of international normalised ratio as part of a practice based anticoagulation clinic or patient self management is feasible and safe.8 Four studies evaluated measurement of international normalised ratio in settings varying from office laboratories 30 31 to the patient's home32 with either trained technicians, nurses, or the patient operating the systems. Correlation with laboratory results was generally good (r>0.75), but there was little discussion about whether this result was clinically acceptable, and numbers of patients tested were both small and highly selected. One study evaluated the performance of the Coagucheck test in one British practice. Most (95%) of the results lay between −1.2 and 0.85 units of the laboratory result.33 C reactive protein,34 35 erythrocyte sedimentation rate,36 and haemoglobin tests 37 were all less accurate in primary care than when used in laboratories by trained technicians.
Assessment of tests for microalbuminuria was affected by the fact that the standard is based on absolute protein excretion values and the tests determine spot concentration.38 Repeated sampling would be needed for adequate screening. A study of the glycosylated haemoglobin test showed promising indications, but the “general practice” clinic may have been a hospital setting.39 Studies of cholesterol measurement used desktop analysers that have now been superseded by updated technology,40–42 although the PCA i-STAT is still available.43 44 One study compared seven blood glucose meters in a practice setting.45 The results indicated that the percentage bias is inadequate for accurate initial diagnosis based on the current British Diabetic Association criteria of a fasting blood sugar of 7.8 mmol/l or greater but that the meters would be suitable for monitoring Pregnancy test kits were also less reliable when used by lay operators.46
Eight studies evaluated impact of the test alongside performance.23 28 32 34 36 39 44 47 In general, these evaluations were extremely poor, seeming to be inadequately planned add ons to studies evaluating performance Studies were mostly uncontrolled, and, at best, non-randomised before and after designs. The quality of the outcome measures was also poor, with almost no objective measurement. Even when a change in practice was noted, the effect of this on patient outcome was not determined.
Discussion
Little rigorous research exists on many important near patient tests with potential uses in primary care. These include urine analysis dip tests and tests for glycosylated haemoglobin, C reactive protein, H pylori, and microalbuminuria. Even in the best researched areas—anticoagulation monitoring and identification of group A β haemolytic streptococcal throat infection—further research is required to confirm initial findings in single small studies that are open to spectrum bias or the lack of appropriate reference standards makes generalisation impossible.
One major difficulty in conducting this review was the lack of a universally accepted terminology to describe the technologies referred to as “near patient tests.” Many papers used the terms “rapid test”24 or “dip stick”38 or referred to specific technologies such as desktop analysers or slide agglutination tests. Searching the literature for all diagnostic tests would not have been a feasible or economic method to identify near patient tests. In addition, many rapid tests are designed as laboratory based methods rather than near patient tests, particularly in microbiology.
The differences between healthcare systems means that caution is needed in generalising from, for example, American primary care physicians to United Kingdom general practice and vice versa. Many studies of measurement of cholesterol and use of desktop analysers were done in American office laboratories. These are often staffed by trained technologists and have to be accredited.5 In addition, techniques such as primary care (office) based throat swab or urine culture are common in the United States and Scandinavia but are unusual in Britain and the rest of Europe. Furthermore, many papers were unclear about whether patients were recruited from outpatient departments or from primary care. Some primary care clinics seemed to be secondary care based open access services or outreach clinics.
Do near patient tests affect outcome?
The wide variety of technologies available and lack of, or poor quality of, evaluations illustrate the considerable gap between marketplace and evaluation for near patient testing. Most tests are evaluated for accuracy and safety by the manufacturer before marketing and perhaps by the Medical Devices Agency. The performance characteristics of some tests have been evaluated in primary care, but most of these evaluations are of limited scope and quality.
A recent randomised trial48 examined the effect of the i-STAT portable analyser in an accident and emergency department. Results for chemistry, haemoglobin, and blood gases were available up to 1.5 hours earlier than with conventional testing, but no objective differences in patient care could be shown The effect of other desktop analysers on practice and cost effectiveness have been evaluated.49 50 Unfortunately, however, the analysers used are now largely obsolete. Rigorous research on the effect of near patient tests in primary care on patient outcomes, organisational outcomes, and cost is virtually absent, yet its importance is paramount.
Evaluation methods
These factors raise the question of whether prospective trials in this area are worth while. Small but high quality evaluations of performance in primary care, coupled with careful assessment of healthcare need such as those carried out by development evaluation committees, may be more valuable.51 Without data on the potential for new technologies to contribute to improved care in a defined clinical environment, manufacturers will have limited stimulus to develop appropriate systems.
Primary care doctors in many healthcare systems are being expected to take on a role in earlier recognition of disease and improving monitoring of established disease. This can occur safely only if primary care doctors have access to accurate and functional diagnostic technologies. A better understanding of how general practitioners manage particular clinical problems is required before the effects of near patient testing can be accurately predicted. Such research would be more generic in that the results could be applied to a range of technologies by modelling.52
Existing research indicates the potential of near patient testing in various areas of clinical practice, including anticoagulation monitoring, measurement of glycosylated haemoglobin concentrations, and identification of microalbuminuria by urine test strips in diabetes; identification of group Aβ haemolytic streptococcal throat infection; and microscopy and dipsticks to identify urinary tract infection. However, without good evidence on cost effectiveness, healthcare purchasers are unwilling to fund these tests. The absence of possible funding has in turn meant that assessment of cost effectiveness of near patient tests has not been seen as a priority. A system is needed in which purchasers undertake to fund these tests if their cost effectiveness if proved. If primary care is to respond to the challenge of more accurate diagnosis and less varied disease management, improved access to investigations is essential. Near patient tests may, in certain situations, provide that support. Further, high quality evaluation of near patient tests is therefore urgently required.
Acknowledgments
We thank the members of the external review panel (listed on BMJ's website).
Contributors: FDRH, BD, and DF wrote the protocol and were awarded funding SW, RT, and SJ ran searches, obtained papers, organised reviewing and constructed the tables of data. GT contacted manufacturers. BD and DF selected papers for eligibility. All authors and the panel assessed validity. CH, BD, and RM conducted the detailed assessments. BD updated the search. BD, RM, CH, SW, and FDRH wrote the paper with comments from all the authors. FDRH is the guarantor.
Footnotes
-
Funding NHS Research and Development Health Technology Assessment Programme (project No 93/15/01).
-
Competing interests BD has received research funding and sponsorship to attend meetings from Cortecs diagnostics. DF has received research funding from Roche Diagnostics (the manufacturer of Coaguchek) and has been sponsored to attend conferences by Nycomed UK (the manufacturer of Thrombotrak) and Roche Diagnostics. FDRH has received research grants and has been part sponsored to meetings by Nycomed UK, Roche Diagnostics, and Cortecs (H pylori antibody test).