In the first phase (interviews), 125 people were identified as eligible and 34 (27%) participated. Of non-participants, only 20% actively declined participation; 80% did not respond to attempted contact. There was no statistically significant difference in the age, gender, HVD type, HVD severity, or ethnicity of participants and non-participants (t-test p < 0.05). Of the HVD participants, 7 had AS, 7 had RHD, 9 had MR, and 11 had VRR. Five clinical experts and three family members of HVD participants were interviewed. The clinical experts, recruited from New Zealand (NZ) and Australia, consisted of a clinical cardiologist, an imaging cardiologist, two structural interventional cardiologists, and a general practitioner specialising in the care of rural patients with RHD (i.e. a total of 42 interviews).

Of the 34 HVD participants, the median age was 67 (min 19, max 93) years and 19 (56%) were female. The ethnicity of 28 (82%) of the population studied were NZ European (Table 1: Phase one participant demographics).

In the second phase (pilot testing), 111 potential participants were identified and were posted questionnaires. Fifty questionnaires were returned though four people did not return consent forms, leaving 46 for analysis and follow-up (see Table 2: Phase two participant demographics). Most participants either had MR (39%) or VRR (37%). Median age was 77 years, 54% were male, and 89% were NZ European. Most HVD participant had severe rather than mild forms of heart valve disease, save for RHD participants, who had more cases of mild native valve disease. 61% of all participants had ischaemic heart disease/coronary artery disease as comorbidities and 41% had atrial fibrillation/flutter.

Sixteen potential participants did not return completed questionnaires but did provide feedback when contacted by phone. The median age was 75, 50% were male, those with HVD mostly had severe valve disease, and all were of NZ European ethnicity.

Participants (both those who returned questionnaires and those who only provided feedback over the phone) were older than non-participants (76 versus 68, t-test p = 0.002), but did not significantly differ in HVD group, sex, or ethnicity.

Interviews were conducted between the 15th of March 2020 and the 2nd of October 2020 and took between 10 and 35 min. Interviews with clinicians were approximately 30 min long. Interview transcripts were used to generate a list of 374 features related to QOL. When transformed into questionnaire items in a semantic differential format, some features could be paired as opposing statements and a list of 255 items was generated. The following themes were identified: physical limitation, especially limitations relative to a prior 'normal'; symptom burden, especially fatigue; sleep, social stressors/support, medication burden, treatment concerns, individuals' perception of their disease and identity, emotional status, and interactions with the healthcare system. The research team met to discuss and remove items considered to be either not indicative of QOL or likely insensitive to changes in HVD status, leaving a 64-item questionnaire which was then pilot tested over October to November 2020. Items and the interview findings they were developed from are available in the Appendix (Appendix Table 4).

The use of a semantic differential allowed for a wider range of methods in which features could be transformed into items, rather than a Likert where respondents’ experiences and perspectives can only be summated into a single statement. These methods are shown in Table 3: Semantic differential item generation and described using participant quotes directly where able, or a summation of participant’s quotes when required for clarity or brevity.

It became apparent there were four ways in which features of QOL could be used to generate items. Firstly, a semantic differential could be directly and organically generated by using points given by two participants where one participant reported a negative extreme and the other reported the corresponding positive extreme.

Secondly, a participant could report a single experience of one extreme. This was used to generate one statement, and the opposing statement was generated by the research team to oppose it.

Thirdly, a participant could report an experience that was in the middle of the spectrum between extremes, and so two statements were generated by the research team to form opposing extremes around the participant’s moderate experience.

Finally, participants could report a wide range of experiences that could not be immediately transformed into a single item. In these cases, the experiences were used to generate multiple subtly different items covering similar topics, allowing participant information to be used with the expectation that later phases of testing would identify which item(s) should be retained.

The reader may notice that not all items in 3: Semantic Differential are ‘balanced’. That is, item statement pairs are not truly conceptually opposite; for example, the statement “I feel embarrassed because I need help even in basic activities” states embarrassment as deriving from needing help in tasks, whereas its paired statement, “I have no need to feel embarrassed about my health” states embarrassment as deriving from health status. These concepts are not the same, making the item flawed. A Likert scale only requires a single statement to be generated, and so this discrepancy could not occur—meaning that the researcher generating the statements would not have to consider the difference between the two concepts posed. A semantic differential has the potential to highlight subtle differences between concepts.

Table 4 shows examples where two statements described different concepts, highlighting an issue, which was then corrected by statement wording changes. The final items were assembled into a 64-item questionnaire for further testing.

In Phase Two, participants completed a form asking what they liked about the full 64-item questionnaire, what they disliked or found confusing, and what they thought was important but was not covered in the questionnaire. Participants found the questionnaire generally good, understandable, and in no need of further changes.

In the section of what participants found disliked or confusing, no participant expressed confusion over the semantic differential format. Only one participant gave specific feedback in this section, and it related to clinical communication rather than the questionnaire content: “A little confusing with some questions as…my problem hasn't been fully discussed or explained by my [general practitioner]” (participant 015).

In the section on what the questionnaire had missed, multiple participants gave specific feedback, including that the questionnaire should have more questions on sleep and life stressors (participant 045), medications and foods they have to avoid (participant 046), the effect of other operations (participant 061), and the effect of COVID-19 restrictions (participant 064). Since these topics did not directly reflect QOL in HVD, they were not added into the questionnaire.

Three items had at least three respondents provide no response, leaving the question blank, across the entire questionnaire: items 20 and 50 had three respondents provide no response, and item 27 had four responders provide no response. This may indicate these items are possibly flawed. If we investigate these items further:

Item 20 consisted of the statements “I’m able to do less than doctors predicted”/“I’m able to do more than doctors predicted”. The participants who did not give a response to the item left the following feedback on the item: “No predictions made”, “Dr hasn’t predicted anything”, “I don’t know what the doctors ‘predicted’”, and “? he didn’t predict”. Participants who completed the item gave similar feedback. Therefore, the item was deemed flawed and was removed from the questionnaire.

Item 50 consisted of the statements, “I'm worried about the consequences I could have if I need an operation for my heart valve disease in the future (such as needing time off work, or pain)”/“I don't have any worries about the consequences of potential future operations for heart valve disease (such as needing time off work, or pain)”. Participants who left the item blank gave the feedback, “this is the doctor’s call”, “N/A”, and “not having an operation”. Participants who completed the item gave different feedback, noting specific features relating to their response, such as, “I worry about recovery time” and “OK – luck may be on my side”. The item was deemed inapplicable to a potentially large subset of the study population and was therefore removed from the questionnaire.

Item 27 consisted of the statements, “I can't keep up with my friends when we do physical activity together”, “I can easily keep up with my friends when we do physical activity together”. Participants who did not complete the item gave the feedback, “N/A”, “not applicable”, “never tried”, and “rest home”. Participants who completed the item gave similar feedback, and the item was removed.

Feedback on other items mostly pertained to clinical features and other details of the participants’ experiences, not the format. Some items, however, had feedback relating to their format and the phrasing of their statements.

Item five, which consisted of the statements, “I get short of breath even when I'm sitting quietly”/“I only get short of breath when I exercise very hard” had two respondents leave the item blank, saying “never short of breath” and “don’t exercise”. Feedback from respondents who completed the item included “I do get short of breath when I’m walking, but not when I’m sitting quietly. I found this question ambiguous—is it ONLY referring to when you are sitting quietly?” Another participant crossed out the “only” and “very hard” on the right-hand statement. The item, therefore, did not balance; a statement had to be adjusted. Interestingly, while statements containing multiple concepts are generally flawed, this question required an addition: the right-hand statement was changed to “I only get short of breath when I exercise very hard. OR I never get short of breath.”

Item six, which consisted of the statements, “My symptoms are unpredictable. I don't know when they will occur” /My symptoms are predictable. I know when they will occur” had similar feedback and also required an addition; the right-hand statement was changed to “My symptoms are predictable. I know when they will occur. OR I don't get symptoms”.

Item 42 consisted of the statements, “I don't trust doctors/the healthcare system”/“I trust doctors/the healthcare system”. A respondent did not leave a response to the item, but left the comment, “I trust my doctor but not too much the healthcare system”. Due to the two concepts within this single item, respondents can have different levels of agreement for each concept, and so one concept was removed from the item; the item was changed to “I don’t trust the healthcare system”/“I trust the healthcare system”.