Experiment

The experiment consists of three phases: (1) a short interview about the participant's background, (2) diagnosis tasks, and (3) a semi-structured interview.

Phase 1's Questions

(Role) What is your current role?
(Years of experience in the current role) How many years that you have been in this role?
(Years of experience in skin cancer) What is your experience in diagnosing skin cancer? (No experience, Beginner, Intermediate, Expert)
(AI expertise) How would you describe your experience in using AI tools, either in your work or outside of work? (No experience, Beginner, Intermediate, Expert)
(DA tools used) Have you ever used a decision-support tool to support skin cancer diagnosis? If yes, can you please describe your experience? What tools have you used?

Phase 2's Tasks

Recommendation-driven

View Full Size

Hypothesis-driven

View Full Size

Recommendation-driven

View Full Size

Hypothesis-driven

View Full Size

It consists of 4 questions, each with a Likert scale where -5 means recommendation-driven is the best, 0 means equally likely and 5 means hypothesis-driven is the best.

Phase 3's Questions

In this phase, we conduct a semi-structured interview by asking them to reflect on how they made the diagnoses in Phase 2 using open questions about the design of my decision aids.

How accurate and reliable do you think this decision support is, by comparing between the recommendation-driven and hypothesis-driven?
What did not work well when you used this decision support? Is there anything that you are concerned about?
What do you think about the quality of the provided evidence? Did you look at the segmentations or the weigh of evidence when making the decision?
Are there any other evidence that you used to make the decision? Specifically, evidence that you found yourself based on the original image.
What are the advantages and disadvantages of the recommendation-driven and the hypothesis-driven interface?
What changes would you propose for the DA to help you make better decisions?