Table 2 Example usability testing and interview guide for prototype report testing

Introduction questions/rapport building

● Could you tell me about your role at the [participant’s facility name]?

● [Phone interviews] Are you in a place where you can view the first report?

● Do you have any questions before we look at the first prototype report?

Viewing 1st report prototype

● What are your initial reactions to this report?

○ [If no verbalizations are occurring] What are you thinking?

○ Listen and repeat back key points with rationales to check for understanding

● Comprehension tasks:

○ In what quarter was the percentage of Veterans with GoCCs highest?

○ What level of performance does the display show?

● Interpretation of the primary comparison in the report (e.g. a benchmark):

○ This chart contains a green line that represents average performance of other facilities in your region. Is that a meaningful comparison for you? If not, what would be meaningful?

● Time ranges:

○ What is the optimal reporting interval/timing for this report? Monthly? Quarterly?

○ What range of dates would you prefer to see? 1 year? more? less?

● Language and terminology:

○ The report uses the phrase "conversations documented" - does that make sense? If not, how would you say that?

● Organizational structure:

○ Are there organizational or team divisions that we could differentiate to better show this information?

Viewing subsequent report prototypes

● [Repeat questions from 1st report prototype]

● Report comparison:

○ Do you have a preference for seeing the data in one of these reports over another?

○ What are the characteristics that you prefer, and why?

Wrap-up

● Are there any other thoughts you would like to share, or any suggestions at all that you have for us today?