Skip Over Navigation Links jump over navigation bar
NIH Consensus Development Program Homepage
jump over navigation barHome Page for the NIH Consensus Development ProgramPrevious Conference Statements from the NIH Consensus Development Program

The programs listed are provided for reference purposes only. They were current when produced, but are no longer maintained and may now be outdated. Persons with disabilities having difficulty accessing information on this page may contact us for assistance. Please select the ODP's home page to access current information.

Health Care Delivery Research Using Hospital Firms

National Institutes of Health
Workshop Summary
April 30 - May 1, 1990

This statement is more than five years old and is provided solely for historical purposes. Due to the cumulative nature of medical research, new knowledge has inevitably accumulated in this subject area in the time since the statement was initially prepared. Thus some of the material is likely to be out of date, and at worst simply wrong. For reliable, current information on this and other health topics, we recommend consulting the National Institutes of Health's MedlinePlus

This statement was originally published as: Health care delivery research using hospital firms. Workshop summary; 1990 Apr 30-May 1. Bethesda (MD): National Institutes of Health, Office of Medical Applications of Research; [1990]:17.

For making bibliographic reference to the statement in the electronic form displayed here, it is recommended that the following format be used: Health care delivery research using hospital firms. NIH Technol Assess Statement Online 1990 Apr 30-May 1[cited year month day];(5).

Executive Summary

Firms are hospital-based and consist of more than one group of physicians, nurses, and allied staff, who provide ongoing medical care for a defined patient population. These teams of parallel caregivers are similar and provide inpatient or outpatient care. They are more likely to be found in general internal medicine services in large hospitals. They were often started to promote continuity of care, better residency training and to share equally the patient care work load.

Firms research is based on the concept of ongoing random assignment of new patients or new physicians to one of these parallel provider groups. An experimental change is made in one firm, leaving another as an unchanged control. The effect of the change is evaluated by comparison and, if beneficial, applied to both groups. Then the next change is introduced in an ongoing process. This can increase ethical, low-cost, high-quality medical care and improve the organization of care.

This summary report will describe the organization of several hospitals' firms, along with the clinical trials carried out so far, and will present the methodological, organizational, ethical issues, and future directions discussed at the workshop. There is now enough experience with this innovative approach to efficient clinical research to show that it can be widely adopted. Firms research is particularly appropriate for the evaluation and use of clinical guidelines and policies.

The Basic Concept

Research using firms applies three core concepts: parallel providers of care; ongoing patient random assignment to providers; and continuous experimental changes, evaluation, and, if successful, implementation. Each of these concepts has been observed in the past by themselves, but their combination is new.(37)

When the volume of patients is sufficiently large, more than one provider or group of providers is needed to provide care. When there is more than one provider group, there has to be some way of allocating patients to these providers. In large organizations such as hospitals, large group practices, or government health services, various methods are often used. The new patient who contacts a large elite, private clinic for an evaluation may be assigned to the next available internist. Depending on the day of the week, there may be different teams covering the hospital emergency room. Several American teaching hospitals have affiliations with two medical schools and new patients are assigned alternately to these two separate services. (1, 32) In some hospitals, there are separate inpatient or outpatient care teams, often called firms, who are similar in staffing and provide care to similar patient populations. Haphazard or rotational assignment of patients occurs in many settings.(10, 30) The next step is to change haphazard assignment to random assignment of patients to parallel providers. If the parallel providers (teams, firms) are already in existence, and assignment of new patients is now haphazard, then the change to random assignment is a small step.

In health care organizations like hospitals, changes in administration, education, organization, information flow, and staffing are many and occur constantly. These changes occur out of necessity or out of the belief that they will result in better care. Rarely are such changes ever examined in any way, much less carefully evaluated. If there are parallel teams, random assignment of patients should be made to provide the best assurance of similar patient populations. Changes made for one provider group can be compared to the provider group that was not changed. If the change is demonstrably beneficial, it can be made for all providers and the next change begun in a continuous process.

What Are Firms?

Firms have mostly started for reasons other than research. They are seen where there are large volumes of patients coming for care to one institution. In large urban teaching hospitals, where care has been provided by residents and subspecialty clinics have multiplied, the patient never saw the same doctor twice, the resident physicians never saw the outcome of the care they provided, and there was no coordination or continuity of care across many subspecialty units.

This situation argued for creating teams of generalist providers of care out of a large amorphous clinical service. These teams would include residents, attending physicians, nurses, and other ancillary staff. Because this group would work together, they would know each other's work habits. Patients were assigned to one of these groups and therefore have a greater change of seeing the same physicians and staff over time. These teams have been created in inpatient units, outpatient units, or both.

Attending physicians assigned to a firm would get to know the residents better and understand their educational needs. General internal medicine firms have become the coordinating center base for their patients' care. Patients remain with the firms unless referred to subspecialty clinics.

Similar teams needed a fair distribution of the patient work load. Therefore, patients were assigned to a firm based on the next available bed, by rotation, or by the whim of a clerk.

On the inpatient side, hospital architecture that creates separate floors or nursing units leads to the creation of separate teams associated with each unit. In some hospitals these teams have been called firms, a term taken from British hospital usage.

Given the existence of these similar parallel teams and haphazard assignment of new patients, the step to formal randomization is a relatively easy one.(29)

A table of random numbers can be generated and the admissions office, emergency room, and other points of entry use them to assign new patients. Stratified random assignment of resident physicians also can be used. Once patients are assigned to a firm they remain with that firm to promote continuity.

Randomization has to be monitored and maintained. There is a form of housekeeping required to keep such systems going.(39)

Examples of Firm Systems

Cleveland MetroHealth Center

This 742-bed, acute care public hospital and rehabilitation facility is affiliated with Case Western Reserve University School of Medicine. All department of medicine staff are full-time faculty members. There are 4,300 discharges per year and 35,000 outpatient visits from the general internal medicine service. General internal medicine is divided into three firms that are three geographically distinct 28-bed inpatient and outpatient areas. Each firm has 20 residents, 3 full-time directors, and 20 other attending subspecialty physicians who spend only part of their time with the firms. The firm outpatient clinic is open for 10 half-day sessions per week. Nurses and other staff are permanently associated with each firm. Residents in internal medicine spend their first and third years in the firm and in their second year rotate off to subspecialty areas. Specific details of the organization of these firms are found in several publications.(2, 10, 25, 27, 29, 38, 39)

The firms were first organized in 1976. Patient randomization was started in 1980, incoming house staff have been randomly assigned to a firm since 1981, and subspecialist faculty were randomly assigned in 1986. Once assigned to a firm, a patient remains with that firm to promote continuity of care. This linkage occurs at first contact with the general internal medicine service and is recorded in the hospitals' computerized information system. This patient may later be referred to a subspecialty clinic or admitted for surgery. However, if they need general internal medicine care, they return to their firm. This structure provides equality of access, equality of work load, continuity of care, and a closer working relationship between firm staff members.

This structure, coupled with a computerized information system that links patients, physicians, and firms, allows for retrieval of data at these three levels. If a patient wishes to select a particular firm rather than be randomly assigned, the patient's number can be recorded and deleted if the patient appears during a study. Such requests are rare.

The average age of patients in the firms is about 63 years. Thirty percent are male, and 50 percent are Caucasian. Forty percent are Medicare patients, 17 percent are Medicaid patients, 10 percent are county welfare recipients, and 33 percent account for other sources of payment. The average inpatient length-of-stay is 10.5 days, with an average diagnostic-related group (DRG) weight of 1.00.

Trials using the Metro firms have now been conducted for 10 years. They have examined test usage and prescribing (3, 4, 11, 21, 28), intravenous therapy (5, 6), and decentralization and reorganization of the firms. (13, 14, 15, 20, 22, 23, 24) These will be described later. Over time a research team has evolved at Metro that is familiar with firms research, and much thought has been given to the methodologic issues associated with such studies.(38, 39)

The Regenstrief Health Center, Indianapolis

The Division of General Internal Medicine at Indiana University has maintained a laboratory for outpatient clinical investigation for more than 15 years. Currently there are four parallel ambulatory general internal medicine practices staffed by 109 residents, 29 faculty internists, and 8 fellows who care for 12,000 patients a year and handle 40,000 patient visits.(44)

Starting in 1977 all patients have been registered in the computerized Regenstrief medical record system, which now includes 350,000 patients. Pharmacy and laboratory data are included. Beginning in 1976 new residents and attending physicians have been randomly assigned to practice sessions. The four parallel practices each have faculty, residents, nurses, nurses' aides, and two clerks. New patients are not randomly assigned but are scheduled for the next available appointment slot.

This structure has been used for a series of trials where the experimental interventions have been randomly assigned to different practices, sessions, or physicians.(7, 8, 17, 18, 19, 26, 31, 33, 34)

Henry Ford Hospital, Detroit

This 937-bed tertiary care teaching hospital is staffed by an 800- physician multispecialty group practice with more than 450 residents in 21 specialty areas. It is the hub for more than 30 ambulatory care centers and serves the largest health maintenance organization (HMO) in the state. In October 1987, one of the internal medicine inpatient units was divided into two teams: a traditional resident team and a team of attending physicians and a physician assistant. Patients were randomly assigned to one of these teams. Costs and mortality were tracked in this only known trial of the teaching effect on medical practice. The residents had higher costs and no significant difference in mortality. (40, 41)

The experience with this trial has started a process of thinking about a continuous series of trials to improve the delivery of inpatient care.

Harborview Medical Center, Seattle

This public hospital affiliated with the University of Washington School of Medicine has three firms providing care in the adult ambulatory medical clinic. Each firm is staffed by seven residents, five to six faculty members, and allied staff. Providers and ancillary personnel are randomized to one of the firms. New patients are randomly assigned to one of the firms unless they express a preference (about 5 percent of new patients).

The Medical Center of Central Massachusetts, Worcester

Internal medicine inpatient units have been created at this teaching hospital.(45) However, unlike the Cleveland MetroHealth Center, Henry Ford Hospital, or Harborview Medical Center, it is organized as a voluntary attending private practice medical staff, like most American community hospitals. Patients are admitted by their private physicians. Therefore, physicians have been randomly assigned to firms, and their patients are admitted to their firms. This is an example that shows what can be done in a hospital with a private practice medical staff.

Other Hospitals

A number of other hospitals have firm systems or are in the process of organizing them. A few of these have carried out research based on these firms. No two are alike. Most have haphazard patient assignment rather than random assignment.

University Hospitals of Cleveland general ambulatory internal medicine clinic assigned new patients to one of two teams based on the last digit of their social security number.(9, 16, 35) This center's trial promotes occult blood testing and will be described briefly later. (35)

At the St. Louis Veteran's Administration Medical Center, new patients are assigned by rotation to one of two medical school affiliated teaching services. Almost no variation in resource use between the two services was observed.(32)

Brook Army Medical Center in Houston has teams of physicians providing general internal medicine care, and efforts have been made to make systematic changes there.(16)

At the University of Michigan hospital, there are two inpatient general internal medicine units. Attention has focused on development of a computer-based system for analyzing the costs of care. This system is in place for one firm but not the other; the effect of such a system is being evaluated.

Examples of Research

The types of research carried out to date include trials of test usage, promoting preventive care, changing provider roles, organizational change, and provider education. Many of the trials could fit into more than one of these categories. It is not our purpose here to describe these trials in detail. The interested reader is referred to the original publications.

The first Metro trial consisted of feeding back to house staff information on the costs of the laboratory tests they ordered in two randomly chosen firms and on the costs of x-ray tests in the other two firms. Total costs per resident per week were posted on the inpatient office wall. Costs for lab and x-ray tests were tracked before and during the trial. This information, which the residents could use or ignore, resulted in a significant reduction in testing. A special feature of this study was an effort to control and monitor the Hawthorne effect by having two concurrent experimental interventions.(3)

The second trial used chart reminders and education to promote preventive procedures in the ambulatory clinic.(4) This resulted in a modest, yet statistically significant, improvement in preventive measures. Observing that resident use of preventive measures is woefully limited in spite of a belief by the residents that these procedures (mammography, colon cancer screening) should be done led to two trials, one at Harborview Hospital in Seattle and one at University Hospitals of Cleveland. In each of these trials, clinical responsibilities were shifted from physicians to nurses. In Seattle, nurses were given the responsibility for screening for alcoholism and, if appropriate, making referral to an alcoholism counselor. The result was a significant increase in appropriate referrals.(43)

At University Hospitals of Cleveland, nurse practitioners took charge of colon cancer screening using occult blood tests. This caused a major increase in the percent of patients who returned their tests.(35)

At Cleveland MetroHealth Center, intravenous therapy had been managed by the residents and nursing staff.(5, 6) A firm trial introduced a specialized IV therapy team. Infections (phlebitis) were counted for the team and compared with firms that remained unchanged. The IV teams had a significantly lower infection rate and as a result were introduced in all the firms. Later, in a budget cutting effort, the elimination of these IV teams was proposed. By bringing forth the persuasive evidence from the trial, the IV teams have survived to this day.

The costs of medical education in teaching hospitals have been a major issue for many years. A number of nonexperimental and often expensive studies have been carried out to measure the costs of residency training in hospitals.

In an inpatient general internal medicine unit at Henry Ford Hospital, patients were randomly assigned either to a traditional team of residents with an attending advisor or to a team of full-time attending physicians. Henry Ford Hospital has a full-time salaried medical staff, some of whom were assigned to the team in this unit. Length-of-stay, test usage, salary costs, and mortality were compared in this trial between the resident team and the attending team. Despite their higher salaries, the attending group had costs that were up to 25 percent lower with no difference in mortality. The savings resulted from a shorter length-of-stay and less testing.(41)

As far as is known this is the only randomized clinical trial of the costs of residency education. Because Henry Ford Hospital is at the center of a 420,000-member HMO, these cost savings directly affect the economics of the organization. To the degree that teaching costs can be passed on to third-party payers, it lessens the pressure on the hospital to worry about these added costs.

The most elaborate firm trial carried out to date at Cleveland Metro Health Center was a major reorganization of the firms' ambulatory services.(14, 15, 20, 22, 23, 24) Ambulatory clinic registration was decentralized, clinic hours were increased to five days a week, and off-hour access to physicians by ambulatory patients was made easier. These changes resulted in a significant reduction in inpatient length- of-stay and improved patient satisfaction. It showed that organization of ambulatory care can influence inpatient care utilization.

Methodological Issues

While carrying out these firm trials, researchers have been working on and thinking about the methodological issues involved.(10, 16, 30, 38, 39) Some of these include:

The Hawthorne Effect

The experimental changes studied so far cannot be blinded in the way that placebo-controlled drug trials can be. This is not a problem of the firm trials, but of the experimental changes themselves. Carrying out two trials at once is one possible answer.

Cross-firm Contamination

Will the control team hear about the experiment and be affected by it? Pre-study data collection and significant differences between experimental and control firms during the experiment can be used to detect this effect if it occurs.


Are the patient populations really similar? This is a general problem of all randomized trials. Study and control groups can be compared regarding characteristics that might influence the results to see if they are in fact similar.

The Decay of Randomization

The patient population randomly assigned may be similar at the start, but become increasingly different over time. One answer is to compare experimental effects on newly randomized patients to those of all patients. If the results are the same, this adds to our confidence about the results.

The Unit of Analysis

Is the test, the patient, the physician, or the firm itself the appropriate unit of analysis? This makes a difference in the statistical power of the study. One answer is to analyze the differences using more than one unit of analysis.


Each hospital with a firm system has special features. Can results in one place be generalized? Repeated studies and multicenter trials can help resolve this issue.

Ethics of Firms Research

Although all firm trials have been reviewed and approved by the hospitals' human subjects committees, they do raise new questions about the ethics of research. A substantial amount of thought has been given to these issues.(34, 42)

Under what circumstances can general organizational changes be made that do not require individual patient consent? The changes that have been studied can be viewed as the type of administrative and educational changes that occur in all hospitals without careful evaluation or without written informed consent from the patients.

What makes it possible to introduce such changes without formal individual informed consent is that these changes do not restrict decisionmaking by the patient and the physician. The firm trials that include changing organization, increased information provided to physicians, or more education do not compel change in treatment. These changes still allow physician, nurse, and patient to make their own agreed-upon decisions.

Dr. Goldberg has provided the following notice to all patients in his clinics. This is a general statement addressed to all patients. "The staff of the Adult Medicine Clinic is always interested in improving the care we provide. We sometimes try what we think is a better way of doing things on one of our group practices but not on another. We might, for example, try a different method of scheduling appointments. By comparing groups after a short period, we can decide whether any changes work well enough to be worth adopting for all our patients. For this kind of evaluation we only select administrative changes that do not constrain the clinical process; that is, you and your physician are always able to decide what medical tests and treatments you do or do not receive. If information contained in your medical record is used in such an evaluation, your anonymity will be carefully protected."(42)

A firm trial that includes a change in required testing or treatment is different. In one proposed trial, a new way to organize a diabetic consult service was to be put in place in one firm to see if this resulted in better patient outcome. The outcome would be based on a blood test to measure control of this problem. The human subjects committee at the Cleveland MetroHealth Center decided that each patient being tested in this way should give specific permission for this test. This ruling met with general approval. In this case, the diabetic patient could directly benefit from the knowledge gained from the test results. Firm trials that constrain patient and physician choice have otherwise not been conducted.

Efficiency in Research

One of the most important features of firm trials is that they can be done inexpensively at a fraction of the cost of traditional randomized clinical trials. If we are to have more good evidence of what works and what does not in clinical medicine, we will need to have many more randomized trials. If each one will cost millions of dollars, that number will be limited.

If we can drive the costs of trials toward zero, we can afford more and more of them. By having a research team in place, having randomization as part of the ongoing care system, and having computer-based management information systems that can easily aggregate the needed data, then the costs can be vanishingly small for some trials.

Estimating the true costs of a firm trial can quickly entangle one into accounting and economic controversy (joint product problems, microcosting analysis, costs to whom, etc.).

We have used a more simple approach to cost measurement that is directly relevant to researchers. How much new money did the researcher have to go out and find to do this study? Using this rough but relevant rule of thumb, two of the firm trials have been carried out for add-on-costs of under $1,000.(11, 12, 43)

That is not to say that all envisioned trials will cost so little. Some will cost much more. Two of the largest trials have drawn on external funding of over $100,000. However, the general commitment to efficiency in clinical research is a guiding principle of firms research.

Management Information Systems

One key to efficient firms research is the development of computer-based information systems that routinely record and aggregate information about care provided on the firms. How many male patients from 50-60 years of age had high blood pressure in February? What is the total price of x-rays ordered by this resident physician? What is the mean DRG score for this firm? How many people cared for by the firms in 1989 are alive in 1991?

The routine, easy-to-get, low-cost availability of data to answer such questions is essential for efficient trials. The development of such systems does not happen immediately. The Regenstrief Group in Indianapolis has been working to develop such a system for ambulatory care over the last 17 years. The University of Michigan Hospital in Ann Arbor has developed a detailed inpatient costing information system for their general internal medicine inpatient firms.

For the future, if there are to be multicenter firm trials, each hospital information system should be compatible with the others so that the same questions can easily be answered for all hospitals participating in the trial.

Clinical Guidelines Trials

The new Federal Government initiative to develop clinical practice guidelines will call for a large number of clinical trials to evaluate physician compliance, problems with compliance, costs, patient satisfaction, and outcome.

To evaluate several hundred such guidelines by traditional multicenter trials at millions of dollars per trial will be impossible. Firm systems may be an answer to this need for evaluation. Firm trials can be done at a fraction of the cost and are ideal for evaluating compliance with guidelines for frequently occurring medical conditions. In one firm, the use of the guideline is encouraged, while the other firm practices "business as usual" medicine. The second control firm could be the experimental firm for a different guideline about a different illness, thus controlling for the Hawthorne effect and doubling the research productivity.

The Future: Multicenter Trials

Already on the drawing board are multicenter firm trials. Such trials could increase the power of trials by increasing the number of subjects. Such trials can be a step toward answering the question of generalizability of results. If it works in Seattle, will it work in Cleveland? Each hospital will be slightly different and this can add strength to the argument of generalizability.

Applying basic ideas behind firm trials can result in different approaches to good management. There are 100 primary care clinics in Stockholm, Sweden, 173 Veterans Administration Medical Centers, and 82 regional health clinics in Costa Rica. These multiple sites can be used as the basis for experimental changes. In Stockholm, 40 clinics may ask to participate in an educational program to improve diabetic care. Only 20 can be accommodated this year. Fairness suggests randomly choosing 20 clinics this year. The effects of the program can be measured by the health status of diabetics cared for in these clinics. The Karolinska Hospital has carried out such projects. Management based on ongoing randomization is a rational strategy, and this idea may someday spread beyond medical care and be a contribution to the general theory of management.

So far no trial has forced a change in patient-physician decisions. The mammography, alcohol counseling, x-ray, or IV can be offered (or rejected) by every doctor (and every patient). Under what circumstances would it be acceptable to conduct a firm trial for alternative drugs, for a new drug, for surgery or not? At this writing it is not known how these questions will be answered.

Firm trials are not useful for the study of rare diseases but are better for the common problems that occupy so much of medical care.


Parallel providers of care (inpatient or outpatient firms), the random assignment of new patients, and a commitment to efficient, continuous, careful evaluation and improvement are the basic ideas behind firm trials. There is now enough experience with firm trials to say that they are a viable addition to our tools for improving the delivery of medical care.


1. Halperin, W, Neuhauser, D.
MEU: A Way of Measuring Efficient Utilization of Hospital Services. Health Care Manage Rev. 1976; 1(1):63-70.
2. Waggoner, DM, Frengley, JD, Griggs, RC, Rammelkamp, CH.
A "Firm" System for Graduate Training in General Internal Medicine. J Med Educ. 1979; 54:556-561.
3. Cohen, DI, Jones, P, Littenberg, B, Neuhauser, D.
Does Cost Information Availability Reduce Physician Test Usage? Med Care. 1982a; 20:286-292.
4. Cohen, DI, Littenberg, B, Wetzel, C, Neuhauser, D.
Improving Physician Compliance with Preventive Medicine Guidelines. Med Care. 1982; 20:1040-1045.
5. Tomford, JW, Hershey, CO, McLaren, CE, Porter, DK, Cohen, DI.
Intravenous Therapy Team and Peripheral Venous Catheter-Associated Complications: A Prospective Controlled Study. Arch Intern Med. 1984; 144:1191-1194.
6. Hershey, CO, Tomford, JW, McLaren, CE, Porter, DK, Cohen, DI.
The Natural History of Intravenous Catheter-Associated Phlebitis. Arch Intern Med. 1984; 144:1373-1375.
7. McDonald, CJ, Hui, SL, Smith, DM, et al.
Reminders to Physicians from an Introspective Computer Medical Record: A Two-Year Randomized Trial. Ann Intern Med. 1984; 100:130-139.
8. Cohen, SJ, Weinberger, M, Hui, SL, Teirney, WM, McDonald, CJ.
The Impact of Reading on Physicians' Nonadherence to Recommended Standards of Medical Care. Soc Sci Med. 1985; 21:909-914.
9. Neuhauser, D, Cohen, D, Austin, S, et al.
Ongoing Patient Randomization for Better Health Care. Bull Clev Med Libr. 1985; 31:5-7.
10. Cohen, DI, Neuhauser, D.
The Metro Firm Trials: An Innovative Approach to Ongoing Randomized Clinical Trials. In: Assessing Medical Technology , Institute of Medicine, Washington, DC, National Academy Press, 1985; 529-534.
11. Hershey, CO, Porter, DK, Breslau, D, Cohen, DI.
Influence of Simple Computerized Feedback on Prescription Charges in an Ambulatory Clinic: A Randomized Clinical Trial. Med Care. 1986; 24:472-481.
12. Neuhauser, D,
Editorial. Med Care. 1986; 24:471.
13. Cohen, DI, Breslau, D, Porter, DK, Hershey, CO, et al.
Academic Group Practice: The Patient's Perspective. Med Care. 1986; 24:990-998.
14. Cohen, DI, Breslau, D, Porter, DK, Goldberg, HI, et al.
The Cost Implications of Academic Group Practice: A Randomized Controlled Trial. N Engl J Med . 1986; 314:1553-1557.
15. Cohen, DI, Dawson, NV, Goldberg, HI, Hershey, CO.
Correspondence: Academic Group Practice. N Engl J Med. 1986; 315:1420.
16. Cargill, V, Cohen, D, Kroenke, K, Neuhauser, D.
Ongoing Patient Randomization: An Innovation in Medical Care Research. Health Serv Res. 1986; 21:663-678.
17. Tierney, WM, Hui, SL, McDonald, CJ.
Delayed Feedback of Physician Performance versus Immediate Reminds to Perform Preventive Care: Effects on Physician Compliance. Med Care. 1986; 24:659-666.
18. Tierney, WM, McDonald, CJ, Martin, DK, Hui, SL, Rogers, MP.
Computerized Display of Past Test Results: Effect on Outpatient Testing. Ann Intern Med. 1987; 107:569-574.
19. Vinicor, F, Cohen, SJ, Mazzuca, SA, et al.
DIABEDS: A Randomized Trial of the Effects of Physician and/or Patient Education on Diabetes Patient Outcomes. J Chron Dis .1987; 40:345-356.
20. Hershey, CO, Cohen, DI, Goldberg, HI, McLaren, CE, et al.
Effect of an Academic Group Practice on Patient Show Rates: A Randomized Controlled Trial. Med Care. 1987; 25:72-77.
21. Hershey, CO, Dawson, NV, McLaren, CE, et al.
Resident Knowledge of Charges: Are We Asking the Right Questions? Am J Med Sci. 1987; 293:182-186.
22. Cohen, DI, Breslau, D, Porter, DK, Goldberg, HI, et al.
Academic Group Practice: The Physician's View. Med Care. 1987; 25:686-694.
23. Goldberg, HI, Cohen, DI, Hershey, CO, Hsiue I-L, et al.
A Randomized Controlled Group of Academic Group Practice: Improving the Operation of the Medicine Clinic. JAMA. 1987; 257:2051-2055.
24. Goldberg, HI, Cohen, DI, Hershey, CO.
Correspondence: A Randomized Controlled Trial of Academic Group Practice. JAMA. 1987; 258:613-614.
25. Cebul, RD, Dawson, NV.
Translating Theory into Practice: The Metro Firm System. Soc Gen Intern Med Newsletter. 1987; 10(5):2-3.
26. Tierney, WM, McDonald, CJ, Hi, SL, Martin, DK.
Computer Predictions of Abnormal Test Results: Effects on Outpatient Testing. JAMA. 1988; 259:1194-1198.
27. Goldberg, HI.
Innovations and Institutions: The Metro Firm System: A Primer. Soc Gen Intern Med Newsletter. 1988; 11(2):4-5.
28. Hershey, CO, Goldberg, HI, Cohen, DI.
The Effect of Computerized Feedback Coupled with a Newsletter upon Outpatient Prescribing Charges. Med Care. 1988; 26:88-93.
29. Dawson, NV.The Metro Firm System: Meeting the Challenges of a
Changing Health Care Environment. Health Matrix. 1988; 5:27-33.
30. Neuhauser, D.
The Metro Firm Trials and Ongoing Randomization. In: JM Tanur, F Mosteller, WH Kruskal, EL Lehman, RF Link, RS Pieters, GR Rising (Eds). Statistics: A Guide to the Unknown. Pacific Grove, CA, Wadsworth and Brooks, 1989; 25-30.
31. Cohen, SJ, Stookey, GK, Katz, BP, Drook, CA, Smith, DM.
Encouraging Primary Care Physicians to Help Smokers Quit. A Randomized, Controlled Trial. Ann Intern Med . 1989; 110:648-652.
32. Gillespie, KN, Romeis, JC, Virgo, KS, et al. Practice Pattern
Variation Between Two Medical Schools. Med Care. 1989; 27:537-542.
33. Tierney, WM, Miller, ME, McDonald, CJ. The Effect on Test
Ordering of Informing Physicians of the Charges for Outpatient Diagnostic Tests. N Engl J Med. 1990; 322:1499-1504.
34. Goldberg, HI. Ethical Issues in Administrative Continuous
Improvement: Applying the Concept of Prior Notification to the Conduct of Firm Trials. Med Care (in press) 1991.
35. Cargill, V, Conti, M, Neuhauser, D, McClish, D.
Improving the Effectiveness of Screening Colorectal Cancer by Involving Nurse Clinicians. Med Care. 1991; 29:1.

The following papers (36-45), presented at this workshop, were published in a supplement of Medical Care (1991, Vol. 29, No. 7).

36. Mosteller, F.
The Contributions of Firms: A Fresh Movement in Medicine.
37. Neuhauser, D.
Parallel Providers, Ongoing Randomization, and Continuous Improvement.
38. Cebul, R.
Randomized, Controlled Trials Using the Metro Firm System.
39. Dawson, N.
Organizing the Metro Firm System for Research.
40. Nerenz, DR, Zajac, BM, Miron, LL, Romans, J, Simmer, TL.
Inpatient Firms in a Teaching Hospital: The Henry Ford Hospital Experience.
41. Simmer, TL, Nerenz, DR, Rutt, WM, Newcomb, CS, Benfer, DW.
A Randomized, Controlled Trial of An Attending Staff Service in General Internal Medicine.
42. Goldberg, HI, McGough, H.
The Ethics of Ongoing Randomization Trials: Investigation Amongst Intimates.
43. Goldberg, HI, Mullen, M, Ries, RK, Psaty, BM, Ruch, BP.
Alcohol Counseling in a General Medicine Clinic: A Randomized, Controlled Trial of Strategies to Improve Referral and Show Rates.
44. Tierney, WM, Miller, ME, Hui, SL, McDonald, CJ.
Practice Randomization and Clinical Research: The Indiana Experience.
45. Ebert, TH.
Initiating a Firm System at a Community Teaching Hospital.


Workshop Day 1

Frederick Mosteller, Ph.D.
Roger I. Lee Professor of Mathematical Statistics Emeritus
Director of Technology Assessment Program
Harvard University
Boston, Massachusetts
Randall D. Cebul, M.D.
"Research Using the Metro Firm System"
Associate Professor of Medicine
Case Western Reserve University
Chief of General Internal Medicine
Division of General Medicine
Department of Medicine
MetroHealth System
Cleveland, Ohio
Neal V. Dawson, M.D.
"Organizing the Metro Firms for Research"
Associate Professor of Medicine
Case Western Reserve University at MetroHealth Medical Center
Cleveland, Ohio
Thomas Ebert, M.D.
"Ongoing Randomization in a Community Hospital"
Associate Chief of Medicine
Program Director
The Medical Center of Central Massachusetts--Memorial
Worcester, Massachusetts
Harold Goldberg, M.D.
"The Ethics of Ongoing Randomization Trials: The Harborview Experience"
Assistant Professor of Medicine
University of Washington
Harborview Medical Center
Seattle, Washington
Laurence McMahon, M.D., M.P.H.
"Computerized Clinical and Cost Information Systems for Research"
Chief Director of General Medicine
University of Michigan Medical Center
Department of General Medicine
Ann Arbor, Michigan
David Nerenz, Ph.D.
"Resident or Special Staff Attendings: Patient Management on an Internal Medicine Inpatient Unit"
Center for Health System Studies
Henry Ford Health System
Duncan Neuhauser, Ph.D.
"Ongoing Randomization, Parallel Providers of Care, and Continuous Improvement of Care"
Department of Epidemiology and Biostatistics
Case Western Reserve University Medical School
Cleveland, Ohio
William Tierney, M.D.
"Ongoing Randomization and the Research Agenda in Indianapolis"
Assistant Director
Computer Science Research Group
Regenstrief Institute
Indiana University
Indianapolis, Indiana

Workshop Day 2

Frederick Mosteller, Ph.D.
Roger I. Lee Professor of Mathematical Statistics Emeritus
Director of Technology Assessment Program
Harvard University
Boston, Massachusetts
Michael Barry, M.D.
"Status of New and Evolving Programs: Goals, Plans/Progress, and Problems"
Assistant Chief of Medicine for Residency Training
Massachusetts General Hospital
General Internal Medicine Unit
Boston, Massachusetts
Neal V. Dawson, M.D.
"Results of a Survey"
Associate Professor of Medicine
Case Western Reserve University at MetroHealth Medical Center
Cleveland, Ohio
Thomas Ebert, M.D.
"Status of New and Evolving Programs: Goals, Plans/Progress, and Problems"
Associate Chief of Medicine
Program Director
The Medical Center of Central Massachusetts--Memorial
Worcester, Massachusetts
Daniel W. Rahn, M.D.
"Status of New and Evolving Programs: Goals, Plans/Progress, Successes and Problems"
Assistant Professor of Medicine
Yale University Medical School
New Haven, Connecticut
Peter Rudd, M.D.
"Status of New and Evolving Programs: Goals, Plans/Progress, and Problems"
Associate Professor of Medicine
Division of Internal Medicine
Stanford University Medical School
Stanford, California
James P. Whelan, M.D.
"Status of New and Evolving Programs: Goals, Plans/Progress, and Problems"
Program Director for Internal Medicine
Associate Professor of Clinical Medicine
University of Chicago Medical School
Chicago, Illinois

About the NIH Technology Assessment Program

NIH Technology Assessment Conferences and Workshops are convened to evaluate available scientific information related to a biomedical technology when topic selection criteria for a Consensus Development Conference are not met. The resultant NIH Technology Assessment Statements are intended to advance understanding of the technology or issue in question and to be useful to health professionals and the public.

Some Technology Assessment Conferences and Workshops adhere to the Consensus Development Conference format because the process is altogether appropriate for evaluating highly controversial, publicized, or politicized issues. Other Conferences and Workshops are organized around unique formats. In this format, NIH Technology Assessment Statements are prepared by a nonadvocate, nonfederal panel of experts, based on: (1) presentations by investigators working in areas relevant to the consensus questions typically during a 1-1/2-day public session; (2) questions and statements from conference attendees during open discussion periods that are part of the public session; and (3) closed deliberations by the panel during the remainder of the second day and morning of the third. This statement is an independent report of the panel and is not a policy statement of the NIH or the Federal Government.

Preparation and distribution of these reports are the responsibility of the Office of Medical Applications of Research, National Institutes of Health, Bldg 31, Room 1B03, Bethesda, MD 20892.

Privacy Notice | Disclaimer | Accessibility | Contact NIH
Department of Health and Human Services HomepageNational Institutes of Health Home PageFirstGov Home Page - The U.S. Government's Official Web Portal  National Guideline ClearingHouse - a public resource for evidence-based clinical practice guidelines