A multi-country, one-wave, observational, cross-sectional, self-reported physician survey was conducted to assess the receipt, understanding, use of the materials, and knowledge and behavior in a sample of physicians with experience in the use of atezolizumab. The study, classified as a Post-Authorization Safety Study [2], was conducted according to Good Vigilance Practice Modules XVI [1] and VIII [3] and best practices based on guidelines [4,5,6] and publications [7,8,9,10,11,12,13,14,15]. The PRAC approved the study protocol. As no patients or patient-level information was involved, ethics approval from national or local ethics committees was not required. However, countries such as Germany, Denmark, and Italy required national notification to their health agencies.

The survey was conducted between April and August 2019 in a selection of European countries: Denmark, Germany, Italy, Spain, Sweden, and the UK. These countries were selected based on the predicted market uptake by Roche to ensure that the study was conducted where atezolizumab was available and accessible to patients and physicians. Such selection also attempted to support the external validity of the study findings, by including a wide and diverse range of healthcare systems (single-payer, public and private payers, mixed) and countries, including northern and southern Europe.

The target population included physicians who may initiate or manage patients on atezolizumab, specifically, medical oncologists, pulmonologists, and urologists. Only physician groups for whom the materials were targeted in the study period were eligible. The start of data collection in each country occurred within the 9–18 months period from the launch of atezolizumab in each country to allow physicians time for the uptake of atezolizumab, familiarity and use of the materials, as well as to ensure that the start of data collection did not vary notably amongst countries.

The study aimed to recruit 300 physicians to allow precision of ± 5.8% to ± 3.6% around plausible estimates (range 50–90%, respectively) of correct responses for process indicators related to the educational materials. All completed responses received by the cut-off date were included in the analysis. The ideal approach for sampling would have been a proportional split of the 300 physicians according to usage of atezolizumab in each country. However, because of the large variance in the number of physicians using atezolizumab in the targeted countries, such an approach would have resulted in very small numbers of participants in the smaller countries. Thus, the approach to sampling consisted of an approximate proportional split of the 300 physicians according to the population in each country, with a minimum of ten physicians for the smaller countries of Denmark and Sweden. Data collection stopped for each country once the country-specific target was met.

The study was conducted by OXON Epidemiology, a scientific services company on behalf of the MAH. Physicians were selected from physician panels of oncologists, pulmonologists, and urologists in each of the participating countries. Panels were formed by voluntary participation of physicians in various ways in the different countries for market research purposes. A stratified sampling method was applied with pre-defined quotas based on the population size of participating countries and the number of materials in each country by specialty distributed by the MAH. Consecutive random samples of physicians in the panel strata were invited by e-mail to participate in the survey and asked to complete the on-line questionnaire until the quota for each stratum was reached.

Eligible physicians who agreed to participate in the survey received a link to access the e-questionnaire and were provided information on how to access the survey online. Financial compensation to physicians was based on a fair market value assessment (i.e., time and effort) in each participating country.

The 20-item questionnaire was developed using best practices identified in Good Vigilance Practice Module XVI. The questionnaire was developed in English and conceptually reviewed using cognitive interviews by two UK clinicians, followed by translation and back‐translation into the languages of the participating countries by certified translators. The questionnaire was subsequently culturally adapted and reviewed by 18 oncologists, three in each of the participating countries, to ensure the clinical terminology was appropriate for each country. Qualitative techniques were used to ensure that the healthcare professional questionnaires were readable, understandable, and easy to use.

The self-reported survey questionnaire consisted of multiple-choice questions with no free text fields. The questionnaire structure included screening questions about eligibility, consent to participate, sociodemographic characteristics, specialty of the physician (oncologist, pulmonologist, urologist, other), years of experience with oncology patients, as well as experience with atezolizumab. The main questionnaire domains assessed the receipt, understanding and use of the materials, knowledge, and behavior. Questions related to knowledge included awareness, identification, monitoring, and management of irADRs, and awareness of the need to report them to their national authorities. Questions related to self-reported behavior included identification, monitoring, and management of irADRs. Questions about knowledge and behavior were capable of being answered by all respondents, whereas receipt, understanding, and use of the materials were limited to those who received the educational materials.

Responses to the survey questionnaires were collected through an on-line Electronic Data Capture system (proprietary, validated, ‘21 Case Report Form 11’ complaint, OXON software). The on-line questionnaire was self-administered and completed at the participants’ convenience. Although participants were advised to complete the questionnaire in a timely manner, once they started the questionnaire, they were able to stop at any point and, later, pick up where they left off, if necessary. Participants were not able to go back and change answers to previous questions. Participants were not allowed to access the questionnaire once it had been submitted. Multiple-choice questions included conditional branching based on responses to previous questions to improve user friendliness and reduce missing data. There was internal and external testing of the online questionnaire before release to study participants. All data collected during the survey were held confidentially. The identities of healthcare professionals taking part in the survey were pseudonymized using unique identification codes. These identification codes were held securely, and these data were used solely for the purpose of monitoring the progress of the completion of questionnaires.

Data analyses were descriptive using frequencies and percentages with 95% confidence intervals (CIs). Continuous scores were summarized using number, mean, standard deviation, median, quartiles, minimum, and maximum. No formal hypothesis testing was performed. Summary scores of usage, knowledge, and behavior were created to summarize all questions pertaining to usage, knowledge, and behavior. Summary scores were calculated as 100 times the sum of correct responses in the domain variables, divided by the number of variables in that domain.

A pre-specified threshold of 70 was used as the criterion to guide interpretation of results: ≥ 70 was deemed adequate performance and < 70 as inadequate performance. However, this criterion was not used for formal statistical testing. Results are presented overall, by country and HCP specialty. Knowledge and behavior were summarized overall and receipt of the educational materials was also calculated.

Missing, illegible, and “I don’t know”, “I’m not sure”, or “I don’t remember” responses were treated as missing values. For questions without a correct or best response, missing data were ignored in the analysis. As a sensitivity analysis, a conservative assumption considered missing values as incorrect or worst responses.

Responders and non-responders to the invitation to participate were compared by type of physician (oncologists, pulmonologists, and urologists) and country. All questionnaires received before database closure were considered responders. Participation rates were defined in two ways: number of responders divided by the number of invited, and the number of responders divided by the number of eligible (responders who fulfilled all the inclusion and none of the exclusion criteria) [4, 5]. All statistical analyses were performed using SAS® software, version 9.4 (SAS Institute Inc., Cary, NC, USA).