Improving surgical decision making through interactive video: what the UCD studies found

Why surgical judgment is hard to measure

Many operative decisions are visual. In colorectal and esophageal surgery, one of them is where to divide the bowel so that the tissue left behind has enough blood supply to heal. Indocyanine green fluorescence angiography (ICGFA) helps with that choice. The surgeon injects ICG dye into a vein, and a near-infrared camera shows the dye as it reaches the tissue. The surgeon still has to read the signal and choose where to cut.

The evidence for ICGFA has grown. A 2026 meta-analysis in Lancet Gastroenterology & Hepatology pooled 9 randomized trials with 4,754 patients and found that ICGFA reduced overall anastomotic leak (risk ratio 0.66). In the IntAct trial (published 2025; 766 patients at 28 centers in eight European countries), clinical leaks occurred in 10% of patients in the ICG group and 15% of patients in the standard-care group. That difference was not statistically significant, and the IntAct authors called for research to standardize how surgeons assess the fluorescence signal.

So the benefit depends on how each surgeon reads the angiogram. If two surgeons would choose different lines on the same case, nobody sees it in the operating room, where each case happens once and one surgeon decides. Recorded video lets many surgeons make the same decision on the same case.

How the UCD Centre for Precision Surgery used interactive video

Professor Ronan Cahill's team at the UCD Centre for Precision Surgery wanted to know how surgeons with different experience read ICGFA video. The team invited participants through clinical and academic networks, so the study had to run online and fit around each surgeon's schedule. A position on the bowel is hard to record with a multiple choice question, so the team used Drawing Response questions in Mindstamp, which let each participant draw the answer on the video frame. The Mater surgery department announced the collaboration on X (then Twitter) in August 2020.

The team's papers describe the same basic path for every participant:

1. Open the study link and complete it at a convenient time.
2. Answer a questionnaire about training grade, experience with ICGFA, and how many cases a surgeon needs to become competent.
3. Watch a short tutorial on the drawing tool.
4. Watch each case. A short card describes the patient and the operation, then the angiogram plays as the dye reaches the bowel.
5. At a fixed frame, the video pauses and asks the participant to draw where they would place the stapler.
6. Save the drawing and go on to the next case.

The screenshots below show the colorectal study in the 2020 version of the Mindstamp player. They show laparoscopic surgery.

What the published studies found

Four studies from the group collected surgeon judgments in Mindstamp. The papers report agreement as an intraclass correlation coefficient (ICC), where 1.0 means perfect agreement.

Peer-reviewed studies from the UCD Centre for Precision Surgery that collected judgments in Mindstamp (abstracts and open full texts checked on PubMed, October 2026)

Study	Participants	Cases	Decision recorded	Main finding
Hardy et al., Surgical Endoscopy, 2021	40, including 6 ICGFA experts	14 colorectal ICGFA videos	Where to divide the bowel, and whether perfusion was sufficient	Experts agreed well on the transection site and non-experts agreed moderately. Experts' marks clustered more distally. Neither group agreed on the yes or no perfusion call.
Hardy et al., Diseases of the Esophagus, 2022	30, including 13 consultants and 6 ICGFA experts	8 esophagectomy videos	A line at the highest point of the gastric conduit judged well perfused	Agreement was moderate among experts (ICC 0.717) and among non-experts (ICC 0.641). Experts kept more conduit length in 7 of the 8 videos.
Dalli et al., Surgical Endoscopy, 2024	58, including 12 ICGFA experts	9 colorectal ICGFA videos, 6 recorded with a standard protocol	Where to divide the bowel	Overall agreement was good (ICC 0.869). Agreement was poor on the standardized videos shown in overlay mode (ICC 0.208 to 0.345).
Hardy et al., Langenbeck's Archives of Surgery, 2024	32 surgeons who perform colonoscopy	14 rectal polyp videos, 7 benign and 7 malignant	Benign or malignant, confidence, polyp size, and a short written reason	Average visual accuracy was 49%, with fair agreement. Higher confidence did not go with higher accuracy. A machine learning classifier was 86% accurate on the same lesions.

A fifth paper, Dalli et al. in Surgical Endoscopy (2022), went back to the experts' stapler marks from the 2021 study. It measured the fluorescence curves at the sites the experts chose and interviewed six experts. Three curve measures agreed very closely between experts (ICC 0.925 to 0.95), yet each expert described the signal in different words. The authors conclude that experts read the signal tacitly.

Across the five papers, a few lessons apply to any decision you want to teach or study:

• A location mark exposes disagreement that a yes or no question hides. In the 2021 study, the location marks produced measurable agreement scores, while the yes or no perfusion question showed no agreement in either group.
• Experience changed where surgeons chose to cut. Experts' marks sat more distally in the 2021 study, and experts saved more gastric conduit in the 2022 study.
• In the polyp study, surgeons who felt more certain were not more often right, so a confidence rating cannot replace a checked answer.
• In the 2024 study, a protocol meant to standardize recordings lowered agreement for most groups. Pilot your display format with a few surgeons before you run the full study.
• Participants expected a long learning curve. In the 2021 study, 70% of participants said a surgeon needs more than 10 cases to become competent, and most experts said more than 50 cases.

The group then moved toward computer methods, which the 2021 paper had proposed as the next phase. A 2024 paper describes software that recommends a transection zone, and a 2024 report describes artificial intelligence that shows an expert-style transection level during live operations.

Other research that measures surgical judgment with video

The UCD studies belong to a wider body of work that uses recorded surgery to measure what surgeons see and decide:

• Madani et al., Surgical Endoscopy, 2017, built a web-based assessment of decision making during laparoscopic cholecystectomy. Five of its 12 items asked participants to draw the answer on the surgical field, and scores came from comparison with expert drawings. Among 39 participants, test-retest reliability was high (ICC 0.95), and scores separated novices, intermediates, and experts.
• Birkmeyer et al., New England Journal of Medicine, 2013, had peer surgeons rate videos from 20 bariatric surgeons. Patients of surgeons in the bottom quartile of rated skill had a 14.5% complication rate, compared with 5.2% for the top quartile. The ratings covered technical skill rather than decisions, and they show that recorded video can reveal differences that matter to patients.
• Chen et al., Journal of Surgical Research, 2014, showed that 501 crowd workers rated a dry-lab robotic suturing video within 24 hours, with scores close to those of 10 faculty surgeons, who took 24 days.

The UCD studies ask each clinician to commit to their own decision on real patient footage, so every answer records a judgment about tissue that you can compare with the experts' answers.

How to build a surgical decision study in Mindstamp

The same setup works for a research study and for a teaching module. These steps follow the Mindstamp editor:

1. Get approvals and consent first. The UCD studies ran with institutional ethics approval, and the esophageal paper reports written patient consent to use anonymized video for training and research. Remove names, dates, and record numbers from the frame and the audio before you upload anything.
2. Edit one short clip per case. The UCD clips ranged from up to 30 seconds for the polyp videos to 90 to 120 seconds for the esophageal angiograms. Upload the files to Mindstamp. Our drawing response guidance treats YouTube links as suitable for testing only.
3. Identify participants with as little personal data as possible. Turn on Lead Capture at the start of the first video, collect a Custom ID that holds a participant code, and add a dropdown custom field for training grade.
4. Add a Drawing Response question at the decision frame. Name the decision in the prompt, for example "Draw a line where you would divide the bowel." The video pauses, the participant draws on the frame with a mouse, finger, or stylus, and Mindstamp saves the drawing as that person's answer.
5. Add other question types for the other parts of the decision. The table below matches common needs to question types.
6. Decide how much replay to allow. The esophageal study allowed one viewing of each video, and the polyp study allowed several viewings and used the final answer. To stop scrubbing back and forth, follow our guide to preventing skipping: turn off Allow Seeking in Design, keep Allow skipping off on each question, and turn on Prevent passing required interactions (Core and higher plans).
7. Put the cases in order. A playlist gives participants one link and can require the order. You can also end each video with a button that switches to the next case.
8. Limit access with a password or an access list. An access list records each view with the person's email and name.
9. Test the whole path in a private browser window. Playback in the editor preview does not record a viewer session.

Mindstamp question types for parts of a surgical decision

What you want to record	Question type	Notes
A location: where to cut, staple, dissect, or look	Drawing Response	Not scored automatically. You review or measure each drawing.
A category: benign or malignant, proceed or convert	Multiple Choice	Up to 5 answers. Can be scored, and each answer can run its own action.
Confidence in the decision	Rating (1 to 5 stars) or Multiple Choice with labeled options	Likert questions use fixed agree-to-disagree labels, so they do not fit a confidence scale.
An estimate, such as polyp size in millimeters	Number Response	Can be scored against a range.
The reason for a decision	Free Response	Text answers appear in reports and exports.

For a teaching module, add feedback after the drawing. Place an image overlay that shows the expert line on the same frame, with one sentence on what the experts saw in the signal. For a full guide to choosing question types, see interactive video question types.

How to analyze drawing responses

Open Reporting, select Interactions, and filter to the study videos. Each drawing appears as a row with the participant and the prompt; select Drawing to open the image. A single session's report shows each drawing next to the participant's other answers. To work with the full set, export the interactions as a CSV file. Each drawing is listed as a link to its PNG image, next to the participant's email or custom ID. For questionnaire answers and scores by session, export Views filtered to one video, group, or playlist with Include Interactions, Variables, and Location turned on.

Then measure. The UCD teams used ImageJ (National Institutes of Health) to measure the distance from a fixed point on each frame to each participant's line, and they calculated agreement with intraclass correlation coefficients. Two points save time:

• Each saved image shows the paused frame with the participant's marks, and its size in pixels follows the size of that participant's player. A phone and a large monitor give images of different sizes, so convert each position to a fraction of the image width and height, or measure from a landmark in the frame, before you compare them.
• Choose the reference answer before you look at the results. Options include the line the operating surgeon used or the median expert line.

Our guide to assessing learning with interactive video covers how to turn answers like these into a score you can track over time.

Privacy, consent, and limits

Operative video is sensitive even when the participants are clinicians. Check these points with your ethics committee and data protection officer:

• Mindstamp is not HIPAA compliant. Do not put protected health information in the video, the questions, or the answers. Use anonymized clips that patients consented to.
• The Mindstamp Privacy Policy describes the rights people have under the GDPR. Mindstamp does not publish a fixed storage region or an EU data-residency option, so confirm where data is stored during a security review if your approval depends on it.
• Collect a participant code and training grade, not names, unless the study needs them.
• Show a warning about surgical images at the start of the first video.

The method also has limits. The polyp study's authors note that an online video platform removes tactile feedback and other tools a surgeon would use in a real procedure. A fixed frame is also simpler than a live, moving field, and the 2024 study shows that how footage is recorded and displayed can change answers. Treat the results as a measure of visual judgment on the cases you chose.

Common questions

Does interactive video improve surgical decision making?

It shows where decisions differ, which tells you what to teach. The UCD studies measured judgment. They did not test whether training with interactive video changes later decisions in the operating room. To test that, give learners a set of cases, teach the expert reasoning, and then give them a second set of cases.

What is a drawing response question?

It is a question type that pauses the video and opens a drawing tool over the current frame. The viewer marks the answer, and Mindstamp saves the drawing as that viewer's response, which you can review in reports and download.

Can I use operative video of patients in Mindstamp?

Use only anonymized footage that patients consented to, and keep protected health information out of the video and the answers, because Mindstamp is not HIPAA compliant.

Do participants need a Mindstamp account?

No. Participants open a link or an embedded video. You identify them with Lead Capture, a personalized link, or an access list.

Can I use this for surgical training instead of research?

Yes. Add expert feedback after each drawing and score the categorical questions. If your program uses an LMS, a SCORM package sends completion and score to a compatible LMS. Our page on healthcare training covers clinical case studies and learner records.

To try the method, pick one decision your trainees find hard, record or select three cases with consent, and place a Drawing Response question at the decision frame of each. Ask two experts to complete the set first, and discuss any case where their lines differ before trainees see it.