Implementing community-based provider participation in research: an empirical study

Conceptual framework

We regarded CBPPR as an innovation and employed an organizational model of innovation implementation to guide our study (Figure 1). Briefly, the model posits that consistent, high-quality innovation use (implementation effectiveness) is a function of the organization’s readiness for change, the level of management support and resources available, the implementation policies and practices (IPPs) that the organization puts into place, the climate for implementation that results from these policies and practices, and the extent to which intended users of the innovation perceive that innovation use fosters the fulfillment of their values.

Organizational readiness for change (ORC) is a pre-implementation phase construct that refers to organizational members’ collective confidence to implement change and collective commitment to pursue courses of action that will lead to successful change[13]. Organizational readiness sets the stage for implementation and affects innovation use through other constructs in the model.

Management support and resource availability are two implementation-phase constructs that shape the organizational context of implementation[14–16]. Management support is critical because managers set organizational priorities and control resources needed in implementation. If management supports exists and resources are available, organizational members can put into place more, high-quality policies and practices to promote innovation use[15].

IPPs refer to the plans, practices, structures, and strategies that organizations employ to promote innovation use[15, 17]. When IPPs allow organizational members to incorporate the innovation with a significant level of operational, cultural, and strategic fit, there is an increased chance for effective implementation of the innovation. Effective implementation can be achieved through different combinations of policies and practices[16, 17]. The collective effect of these policies and practices shape the optimum climate to implement consistent, high-quality innovation use.

Implementation climate refers to organizational members’ shared perception that innovation use is expected, supported, and rewarded[17, 18]. Implementation climate emerges from organizational members’ shared experiences with, observations of, and discussions about the organization’s IPPs. Organizations create a positive climate for implementation by employing a variety of mutually reinforcing policies and practices to enhance organizational members’ means, motives, and opportunity for innovation use.

Innovation-values fit (IVF) refers to the extent to which organizational members perceive that innovation use will foster the fulfillment of their values[17]. Although individuals may vary in their values, emphasis here is given to values shared by groups (e.g., oncologists in group-practice). IVF is held to moderate the relationship of implementation climate and implementation effectiveness. Even in a strong implementation climate, innovation use can range from non-use to compliant use to committed use depending on IVF.

Study setting

Established in 1983, the CCOP is a three-way partnership involving the NCI’s Division of Cancer Prevention (NCI/DCP), selected cancer centers and clinical cooperative groups (CCOP research bases), and community-based networks of hospitals and physician practices (CCOP organizations) (Figure 2)[12]. NCI/DCP provides overall direction and funding for community hospitals and physician practices to participate in clinical trials; CCOP research bases design clinical trials; and CCOP organizations assist with patient accruals, data collection, and dissemination of study findings. As of December 2010, 47 CCOP organizations located in 28 states, the District of Columbia, and Puerto Rico participated in NCI-sponsored clinical trials. The CCOP includes 400 hospitals and more than 3,520 community physicians.

In FY 2010, the CCOP budget totaled $93.6 million. The median CCOP organization award was $850,000. NCI funds CCOP organizations through a cooperative agreement whereby participating organizations are expected to share the costs with NCI. Continued funding depends on the performance of the CCOP organization in meeting clinical trial enrollment goals (i.e., accrual) set by NCI. CCOP organizations are required to accrue patients in both cancer treatment and cancer prevention and control (CP/C) clinical trials. Cancer treatment clinical trials refer to the evaluation of cancer therapies that generally fall into one of three categories: medical, surgical, or radiation oncology. CP/C clinical trials refer to evaluation of new methods of detecting cancer risk and preventing primary and secondary cancers. It also refers to the evaluation of symptom management, rehabilitation, and continuing care interventions designed to minimize the burden of cancer and improve quality of life. The NCI establishes accrual goals for cancer treatment and CP/C clinical trials based on the CCOP’s past accrual performance and expected accrual performance given the available menu of NCI-sponsored clinical trials.

CCOP organizations are led by a physician Principal Investigator (PI) who provides local program leadership[12]. CCOP staff members typically include an associate PI, a program administrator, research nurses or clinical research associates, data managers, and regulatory specialists. These staff members coordinate the review and selection of new clinical trial protocols for CCOP participation, disseminate protocol updates to the participating physicians, and collect and submit study data. CCOP-affiliated physicians accrue or refer participants to clinical trials, and typically include medical, surgical and radiation oncologists, general surgeons, urologists, gastroenterologists, and primary care physicians. CCOP-affiliated physicians, through their membership in CCOP research bases, also participate in the development of clinical trials by proposing study ideas, providing input on study design, and occasionally, serving in the role of PI or co-PI for a clinical trial.

Study design

Data collection procedures

In addition, the research team obtained data from CCOP annual progress reports and grant applications. The NCI requires that CCOP organizations file annual progress reports and periodic re-applications for funding. These documents included detailed data on the CCOP organization’s structure, operations, and accrual data for each treatment and CP/C clinical trial, sorted by CCOP research base, accruing hospital, and accruing physician.

Analysis

Data analysis involved three phases: data coding, within-case analysis, and between-case analysis. In the first phase, we used qualitative data analysis software, ATLAS.ti 5.0 (and later 6.0), to code the study data. The conceptual framework provided a starting list of codes, which we supplemented with emergent codes as analysis proceeded. Using a common codebook, two investigators (BW, DB) conducted a pilot test by independently coding four transcripts. They then fine-tuned the coding manual’s definitions, decision rules, and examples. Two other research team members (MJ, RT) coded the remaining documents and an investigator (DB) reviewed the coding for accuracy and consistency. The research team coded over 1,000 pages of interview transcripts compiled from the three CCOP organizations.

Due to wide variation in the formatting of progress reports and grant applications across the three CCOP organizations, we could not assign these documents to ATLAS.ti. Instead, we extracted numerical data from these documents (e.g., patient accrual figures for individual physicians) and used reported information to triangulate the results of this study. Over 500 pages of progress reports and grant applications were reviewed.

In the third phase, we applied the same criteria across the cases to determine if cross-case variation in implementation was consistent with the hypothesized relationships in the model. We generated 12 meta-reports that summarized each code across all three sites from Year 1 through Year 3 of the study. Over 120 summary reports were generated from coded text segments.

Implementation effectiveness

As an organization-level construct, implementation effectiveness refers to the consistency and quality of aggregate innovation use among intended innovation users. We operationally defined innovation use as patient accrual, or the enrollment of new patients in NCI-sponsored clinical trials. Aggregate innovation use is measured in terms of CCOP-level patient accrual.

Figure 3 shows that CCOP A exhibited the greatest consistency over time in total patient accrual to NCI-sponsored clinical trials (i.e., treatment and CP/C accrual combined). CCOP C exhibited a substantial jump in total patient accrual in the first three years of CCOP funding, followed by an even larger decline. CCOP B exhibited a similar, but less pronounced pattern. All three CCOP organizations struggled with patient accrual in 2010 and 2011 due, in part, to external factors (described below). Viewed somewhat differently, CCOP A met the NCI’s accrual goals in eight of the nine observed years for CP/C trials and six of the nine observed years for treatment trials. CCOP C met the NCI’s accrual goals in eight of the nine observed years for CP/C trials, but only two of the nine years for treatment trials. CCOP B met the NCI’s accrual goals in two of the six observed years for CP/C trials and none of the six years for treatment trials. In sum, the consistency of aggregate innovation use was highest in CCOP A, then CCOP C, then CCOP B.

The quality of aggregate innovation use did not vary meaningfully across the three cases. There were minor audit issues reported around eligibility criteria and protocol selection in all three cases, but the data quality and research procedures were generally robust.

The three CCOP organizations also varied in the consistency with which physicians engaged in CBPPR. To examine how and why this is so, we turn to our organizational model of innovation implementation, beginning with an analysis of organizational readiness for change.

Organizational readiness for change

Success in the start-up phase of implementation depends in large part on ORC. As noted earlier, ORC concerns organizational members’ collective confidence to implement change and collective commitment to pursue courses of action that will lead to successful change. In all three CCOP organizations, collective confidence was high. Prior to becoming a CCOP, all three CCOP organizations had participated in NCI-sponsored clinical trials as research affiliates of cooperative groups. As research affiliates, they gained experience enrolling and managing patients on clinical trials and used the per-case reimbursement that they received for enrolling patients to hire research staff to support participating physicians.

Interview participants said that, when they applied for CCOP funding, they knew they would have to significantly ramp up their clinical trials research operations to meet the NCI’s accrual goals. Moreover, they acknowledged some trepidation about having to enroll patients in CP/C trials, a type of clinical research with which they had less experience. Nonetheless, they felt confident that they could quickly increase the scope and scale of their research operations:

‘We were pretty confident we’d be able to meet the treatment trials [accrual goal], if the right trials were open…. The prevention [trials accrual goal] has always been kind of an iffy in the very beginning, just because there weren’t [large prevention] trials available…. There wasn’t a SELECT [the Selenium and Vitamin E Cancer Prevention Trial], there wasn’t a STAR [Study of Tamoxifen and Raloxifene]. So we thought we’d be able to meet the treatment. We never knew about the prevention.’—CCOP B, CRA group interview.

Although collective confidence was high, collective commitment was more variable in all three CCOP organizations. Some physicians were committed and enthusiastic about becoming a CCOP organization; other physicians were ‘supportive’ or merely ‘interested.’ In retrospect, it is not clear whether physicians understood that, as a CCOP, they collectively would be ‘on the hook’ for a much higher level of accrual than they were used to generating as a research affiliate. To meet the NCI’s expectations, physicians would have to step up their personal participation in the clinical research, enrolling patients routinely rather than occasionally in clinical trials. The CCOP PIs of all three CCOP organizations succeeded in building physician support to become a CCOP organization, but the broad support they garnered did not translate into broad (or deep) commitment to enrolling more patients in clinical trials after the CCOP grant was awarded:

‘I think that they [physicians] were tolerant of it but I don’t know that they would have independently thought of it and I don’t know that they perceived that to become a CCOP was going to move us to a different level or establish a different regard for our practice in the community. But, you know, I do think that they heard me out and I think they were supportive of it when they heard the reasons I felt it was a good idea.’—CCOP B, PI

In addition, it is not clear that the CCOP PIs made the strategic or competitive case to hospital managers for becoming a CCOP organization (or, if they did make the case, that hospital managers understood it). In none of the cases did the decision to apply for CCOP funding come from hospital managers. Instead, the PIs came up with the idea to apply and wrote the CCOP proposal. In one case, the PI waited until a ‘friendly’ but interim hospital manager was in place before asking the hospital to sign-off on the CCOP proposal. While the PI received the sign-off, the stage was set for low levels of management support once the program was funded.

In all three cases, the minimal involvement of hospital management in the decision to apply for CCOP funding set the stage for minimal management support once the grant was awarded. In all three cases, managers viewed the CCOP as a ‘doctor’s program,’ with potential benefits or advantages for the hospital either misunderstood or unrealized:

‘For many years here, they weren’t supportive. Then they became supportive when they thought that it was [good] for the hospital’s existence. So, hospital administrators will not support programs that are for research, and maybe that was my wrong approach because I told them this was a research project. In retrospect, what I should have done is said this is not a research program, this is a quality of care program.’—CCOP C, PI

In sum, the three CCOP PIs overestimated collective confidence and underestimated collective commitment. Although all three CCOP organizations met NCI’s accrual goals right away, the mixed organizational readiness for change exhibited in the pre-implementation phase foreshadowed the problems that all three cases encountered later in implementation.

Management support and resource availability

Management support matters because implementation is resource intensive. Managers can support CBPPR by allocating needed financial, human, and material resources. In addition, they can shape the implementation context by communicating that CBPPR is an organizational priority, encouraging organizational members to implement policies and practices to support CBPPR. Finally, managers can support implementation by overcoming resistance to change and resolving disputes over resource allocations, organizational routines, and lines of authority.

Given the minimal involvement of hospital managers in the decision to become a CCOP organization, management support during implementation was largely symbolic. The managers we interviewed expressed verbal support for the CCOP organization, but did not see CBPPR as important in advancing the hospital’s strategic goals or improving its competitive position:

‘When I attract physicians to the organization, there’s eight or nine things that come to the top and this [the CCOP] would probably be one of I’d say five or six, as opposed to the first five. So, I would tell you it’s not in my top three. I would say that when I’m talking about attracting doctors to the organization, I’m generally talking more at the global level.’—CCOP B, hospital manager.

In two cases, hospital managers made no financial commitment to CBPPR over the course of our study, despite the fact that NCI funds CCOP organizations with expectations that institutional cost-sharing will occur. Instead, managers expected the CCOP organization to ‘live on its CCOP grant’ and any supplemental funding that physicians generated from participating in pharmaceutical industry trials:

‘Basically, it amounts to the CCOP grant, we get a little bit of money from the cooperative groups and we get the money from industry trials and the hospital helps in the sense that they own the foundation that gives us some philanthropic support that helps us fill the space between what we can generate and what our true costs are…they’re [the hospital] kicking in zero dollars for every dollar that we generate.’—CCOP A, PI

In the third case, hospital managers realized that CBPPR had strategic value in light of the hospital’s plans to become a major teaching affiliate of a newly opened medical school. Three years into implementation, hospital managers began making substantial financial commitments to the CCOP organization, starting at one million dollars and, later, as the Great Recession continued, reducing the amount to $800,000:

‘I think it’s evolved through the years, but I think research has taken a greater importance…as that has happened people have seen, well, you have the CCOP, it’s been the gem, it’s been sitting there all these years and it’s just kind of like rediscovered every year. But now, like I said, with the medical school, the importance of research is clearer.’—CCOP C, physician

Although management support increased over time in one case, the modest management support that existed in all three CCOP organizations during start-up and early implementation was, in interview participants’ minds, both a blessing and a curse. On the one hand, the CCOP organization could ‘fly under the radar’ with little management interference. Moreover, it was largely shielded from the staffing reductions and budget cuts that the hospital imposed elsewhere as the ‘Great Recession’ persisted. On the other hand, the CCOP organization could not expand its operations beyond what the CCOP grant could sustain. Further, the CCOP organization could not use hospital managers’ authority and resources to implement policies and practices that would make physicians accountable for helping the CCOP organization meet its accrual goals.

Implementation policies and practices

Organizations can make use of a wide variety of IPPs to enhance organizational members’ means, motives, and opportunities for innovation use. IPPs are cumulative, compensatory, and equifinal[15, 17]. That is to say, all other things being equal, the more IPPs an organization employs to support innovation use the better. However, high-quality IPPs can sometimes compensate for missing or poor-quality IPPSs. For example, excellent in-person training could substitute for mediocre program manuals. Finally, different combinations of IPPs can produce the same results.

Our analysis identified three IPP themes. First, all three CCOP organizations provided a high level of staffing and operational support for some, but not all, of the physicians who wanted to enroll patients in NCI-sponsored clinical trials. Although the specific IPPs used by the three CCOP organizations differed somewhat (e.g., some used a mix of registered nurses and clinical research associates while others used only registered nurses to staff the clinical trials program), all three provided support by screening charts to identify potentially eligible patients, flagging charts to prompt physicians to introduce trials to eligible patients, educating and consenting interested patients, following patients once enrolled in a study, and managing IRB and other regulatory requirements. However, they could only provide such high levels of support to those physician practices located ‘on-site’ or in close proximity to the hospitals where research staff worked. Support for ‘off-site’ physician practices was harder to provide or sustain due to travel time and logistical difficulties. The limited support CCOP organizations could offer these physicians contributed to uneven distribution of patient accrual.

Second, all three CCOP organizations avoided using IPPs to create formal expectations or accountability for physicians to help the CCOP organization meet its accrual goals. The CCOP PIs and CCOP administrators routinely cajoled, sometimes exhorted, and occasionally begged physicians to enroll more patients in clinical trials. They regularly provided physicians with feedback on the CCOP organization’s accrual performance relative to NCI’s accrual goals and usually provided physician-level patient accrual figures. Yet, none were willing to put into place a policy specifying the minimum number of patients per year that a physician had to enroll in order to maintain membership in the CCOP organization and, thereby, enjoy the reputational benefit of having an ‘NCI affiliation’:

‘This institution will not tell the referring doctors that they can’t be members of the CCOP for some criteria. You’ve got to put, let’s say, ten patients on a year. In fact, the institution is going to the opposite direction.’—CCOP C, PI

Moreover, none were willing to confront physicians who maintained their membership in the CCOP organization, yet consistently contributed no patient accrual. Interview participants mentioned several reasons why they were unwilling or unable to implement such policies and practices, including not wanting to shut anyone out who might at some point contribute accrual; not wanting to exclude off-site physicians who received little or no support from the CCOP organization; and not wanting to confront colleagues who, in some cases, were business partners:

‘No, the group has to provide support for the clinical research program but there’s no individual expectation that every physician will accrue ten patients per year to clinical trials. We keep trying to bang that drum and we provide feedback to our doctors on a regular basis about how they are doing, but I can’t tell that they have the requisite shame that I would hope that they would have with some of the numbers.’—CCOP B, PI

CCOP organizations sought to compensate for these missing policies and practices by providing more research support, but with little success.

Finally, all three CCOP organizations used relatively weak IPPs to recognize and reward physicians for enrolling patients in clinical trials. Financial incentives or rewards were out of the question due to ethical, legal, and regulatory concerns. So, CCOP organizations fell back on social recognition and non-monetary rewards, neither of which remained effective over time. Social recognition included verbal praise in meetings, certificates of appreciation, complimentary write-ups in newsletters, and token awards given to the highest-accruing physician. Non-monetary rewards included paid travel to conferences, batches of cookies, and other small gifts. The motivating effects of these rewards diminished over time:

‘I think that we’ve tried all the rewards that probably many other research organizations have tried. You know, we’ve given, boxes of candy and we’ve given gift certificates and cookies…we found our doctors are not going to enter patients on clinical trials for the benefit of winning the chocolate chip cookies or winning a gift certificate.’—CCOP B, PI

In sum, all CCOP organizations employed IPPs unevenly. Although some IPPs can substitute for other IPPs, the challenges that the CCOP organizations experienced providing support to off-site physicians and the minimal or low-power IPPs that they used create expectations, accountability, recognition, and rewards produced a weak implementation climate.

Implementation climate

IPPs influence innovation use through implementation climate. Organizational members ascribe meaning to the IPPs that they experience directly or vicariously. Through experience, observation, and discussion, they develop a shared sense of whether innovation use is expected, supported, and rewarded. This shared sense is important when the organizational benefits of innovation use depend on consistent, high-quality use by many organizational members. Such is the case with CBPPR in CCOP organizations. The NCI evaluates and funds CCOP organizations based on aggregate (i.e., CCOP-level) accrual.

Given the uneven IPPs that CCOP organizations put into place, implementation climate was weak in all three cases. None of the interview participants reported that they were expected either by the CCOP organization or by hospital management to enroll any particular number of patients in clinical trials each year to maintain membership in the CCOP organization. Likewise, none reported that they were recognized or rewarded in a meaningful way for helping the CCOP organization meet its accrual goals. Physicians practicing in or near the hospital felt supported by the CCOP to enroll patients in clinical trials; off-site physicians did not.

Although implementation climate was weak in all three CCOP organizations, all three nonetheless met or exceeded NCI’s accrual goals. To understand how the CCOP organizations achieved this feat, we must examine the pattern of IVF among physician groups.

Innovation-values fit

As noted above, IVF refers to the extent to which organizational members perceive that innovation use will foster the fulfillment of their values. Values are concepts or beliefs that pertain to desirable end states or behaviors. Although individuals can hold different values, so too can groups. Different physician groups, for example, may hold different beliefs about clinical trials participation. Individuals and groups can also differ in the amount of feeling they attach to a concept or a belief. Clinical trials participation might be valued strongly and considered a priority, or it might be valued weakly and considered desirable, but not necessary.

When implementation climate is weak, innovation use depends primarily on IVF. Individuals and groups that strongly value CBPPR will enroll patients in clinical trials even if they perceive little expectation or extrinsic reward for doing so, provided adequate support exists. If inadequate support exists, ‘true believers’ will find it difficult to enroll patients on a consistent basis, leading perhaps to frustration and disappointment. By contrast, individuals and groups that do not strongly value CBPPR will enroll few patients, if any, because they see little reason to do so when enrolling patients is neither expected, nor supported, nor rewarded.

We observed individual differences in IVF in all three cases. Not surprisingly, the CCOP PIs highly valued CBPPR and enrolled many patients. In one CCOP organization, a physician enrolled many patients in pharmaceutical industry trials but relatively few in NCI-sponsored clinical trials. Industry trials exhibited a greater fit with his high-intensity values because, in his view, they gave him access to newer, more exciting drugs. In another CCOP organization, a physician stated that he did not like the NCI’s treatment trials because they were not, in his view, always addressing important scientific questions; however, he was willing to help the CCOP organization achieve its accrual goals by enrolling patients in cancer control trials.

We also observed group-level differences in IVF in all three cases. Medical oncologists as a group more strongly valued CBPPR, and therefore enrolled more patients in NCI-sponsored clinical trials, than did radiation oncologists or other physician groups. Interview participants in two CCOP organizations noted that radiation oncologists as a group were ‘supportive,’ but not ‘committed’ to enrolling patients in NCI-sponsored clinical trials. The radiation oncologists would follow the clinical trial protocol for those patients that the medical oncologists enrolled, but they would not enroll patients in radiation-therapy clinical trials in sufficient numbers to help the CCOP organization meet its accrual goals:

‘The total again is 8% of the patients. So that means they put eight, nine, ten patients on this whole year…I’ve been here for 20 years and this is pretty typical for the way the radiation therapists have performed over the last 20 years. This is nothing different. They just don’t do a lot of randomized, you know, Phase III trials, which is what we do as a CCOP. They do a lot of Phase II trials.’—CCOP C, PI

Even when provided with additional research staff support, radiation oncologists as a group would not contribute much on a consistent basis. CBPPR was simply not a high-intensity value for them, interview participants said.

In a third case, radiation oncologists as a group strongly valued CBPPR, as evidenced by their active in-house research program. However, they preferred to enroll patients in in-house studies rather than NCI-sponsored clinical trials because such studies were more instrumental in fulfilling their values (i.e., they felt such studies were more interesting and easier to do).

Given weak implementation climate and variable IVF, all three CCOP organizations exhibited inconsistent, uneven distribution of patient accrual among individuals and groups of physicians. Of the three cases, CCOP A exhibited the strongest IVF among medical oncologists as a group. This CCOP organization recruited and hired physicians who valued CBPPR, and it reinforced CBPPR as a group value in a variety of formal and informal ways. Reflecting a more distributed (or even) pattern of patient accrual, the PI of CCOP A enrolled on average only 28% of the patients per year that CCOP A accrued to NCI-sponsored clinical trials from 2002 to 2010. By comparison, accrual was more concentrated at CCOP C and CCOP B. In the former case, the average annual accrual contribution of the PI was 34%, while in the latter case it was 44%. At start-up, CCOP C exhibited more evenly distributed accrual among physicians, with the CCOP PI accounting for only 18% of patient accrual in the first year. With off-site physicians receiving little support, and radiation oncologists focused on in-house studies, initial enthusiasm faded and accrual become increasingly concentrated among the few ‘true believers.’ By 2008, six years after start-up, the CCOP PI’s contribution accounted for over 42% of total patient accrual. In CCOP B, the concentration of patient accrual was high at start-up and increased steadily over time. In the first year, the PI’s contribution accounted for 39% of total patient accrual. By 2009, it amounted to 59%, despite personal appeals from the highly regarded PI and increased research support for low- or non-accruing physicians. An interview participant described the situation:

‘And our accruals here are actually up but once again, we’re really concerned that [the CCOP PI] has [put] 40 some odd patients on treatment trials. Everyone else in the CCOP is at single digits. And we don’t know, we’re getting ready to have a discussion with the other physicians and [the PI] and I are mulling things over on how to present this but this can’t go on. It just can’t go on. If you want us to maintain ourselves as a viable CCOP, everybody’s got to pull their weight.’ –CCOP B, CCOP Administrator

External factors

Although the organizational model of innovation implementation used in this study focused our attention on intra-organizational facilitators and barriers, we observed that CCOP organizations are heavily dependent on two external resources needed for CBPPR—trials and patients. In varying degrees, all three CCOP organizations experienced difficulties obtaining these resources, which, in turn, exacerbated the challenges posed by a weak implementation climate and variable IVF.

Interview participants noted that the menu of NCI-sponsored clinical trials available to CCOP organizations has shrunk in the past two years and contains significant gaps in common disease areas, such as colon cancer. Interview participants attributed the trial availability problem to the consolidation of the cooperative group system that is currently taking place in response to an influential, yet critical report by the Institute of Medicine[25], a not-for-profit, non-governmental organization that advises the federal government on issues related to biomedical science, medicine, and health. With fewer treatment trials available, CCOP organizations are struggling to meet their treatment trial accrual goals. In addition, with no large cancer prevention trials available, CCOP organizations are trying to meet their CP/C accrual goals through symptom management trials. These smaller trials often close quickly to new patient accrual, disadvantaging those CCOP organizations (like CCOP C) that work with slow IRBs, which must approve studies locally before CCOP organizations can enroll patients in them.

Interview participants further noted that NCI-sponsored clinical trials are increasingly testing targeted therapies applicable only to small subgroups of cancer patients. Not long ago, breast cancer treatment trials would open to almost any patient with, say, advanced disease and no co-morbidities. Today, trials are testing different therapies for node positive and negative breast cancer, which can be further divided into pre- and post- menopausal categories, which can be further sub-divided into other categories, creating approximately 16 different groups of breast cancer patients. Similar trends are occurring in treatment trials for other cancers. Interview participants referred to this as a ‘needle in a haystack’ problem:

‘What’s happening is [that] what used to be gigantic baskets of patients are [now] being divided up by biology. It’s better medicine to divide them…. Each one of these breast cancers is a bit different, [and] each one of them has a different [treatment] protocol. So now…I just cut breast cancer into 16 different groups…. And I think that accounts for a large degree why we don’t really have any rainmaker studies.’—CCOP A, PI

To meet their accrual goals, some interview participants observed, they cannot afford to miss any opportunity to enroll an eligible patient. Hence, they feel frustrated when low- or non-accruing physicians let an eligible patient go un-approached.

Finally, interview participants commented that meeting the NCI’s expectations for patient accrual became increasingly difficult in the face of persistent economic recession and increased market competition. Midwestern states, where the three COCP organizations operate, have been hit especially hard. Patient volume has declined across the board as patients put off medical care to manage other living expenses. Interview participants noted that patients have voiced more concern about enrolling in a clinical trial because they are worried about their health insurance coverage and out-of-pocket costs. Given job insecurity, patients are more reluctant to take time off work to receive multiple rounds of cancer treatment and return for follow-up clinical visits. At the same time, local market competition has grown. The PI of CCOP A noted, for example, that he and his four partners were the only medical oncologists in town in the first year of the CCOP grant. Now, he has eight partners and there are two competing oncology practices not affiliated with the CCOP organization that have seven more medical oncologists. The number of cancer patients has not increased threefold over time.

Study limitations

All studies have limitations. Ours is no exception. Case study research emphasizes depth over breadth, insight over generality[20–22]. Three cases, no matter how closely studied, do not provide a sufficient basis for statistical generalization. To help others assess the reasonableness of applying our findings to other practice-based research networks (PBRN), we offer a ‘proximal similarity model’[33] that suggests two important gradients along which PBRNs vary: the type of clinical research conducted by the PBRN, and the strength of the science upon which it is based; and the level of federal support provided to the PBRN. The more proximal a PBRN is to the CCOP on these gradients, the greater the reasonableness of applying study findings beyond the specific people, places, settings, and times from which they were generated. With this proximal similarity model in mind, two factors increase our confidence in the transferability of our study findings. First, the CCOP has already served as a model for other federally supported PBRNs[34], and the NIH continues to look to the CCOP for guidance as it implements the Roadmap. Second, a recent survey found that many clinical research networks funded by the NIH share with the CCOP key organizational features with respect to funding, design, and operation[35]. This means that many NIH-funded clinical research networks are proximally similar to the CCOP.

As is true of all research, case study research involves an irreducible element of expert judgment. We employed several case study research tactics to increase the dependability and credibility of our study results, including the use of theory to conceptually and operationally define constructs; use of multiple sources of data; use of experienced interviewers; use of a common interview guide and codebook; audio-recording and transcription of interviews; use of software to code transcripts and document coding decisions; and duplicate coding and systematic review of coded transcripts, summary reports, and meta-reports by multiple research team members. We feel confident in our use of these time-honored case study research tactics, but we cannot discount fully the possibility that investigator bias in interpretation influenced our results.