Qualitative Evaluation of AI Contours


Abstract

Purpose:

The differences between prostate CT contours generated by artificial intelligence (Limbus Contour, Limbus AI, Regina, SK, Canada) and those expertly drawn are characterized qualitatively.

Materials and Methods:

Patient CT scans imaged at three Alberta sites were contoured with Limbus Contour and independently by dosimetrists. For a minority of patients, the Limbus Contours were available to dosimetrists during the contouring step, allowing editing or re-contouring as deemed appropriate. The clinical contours were compared to the Limbus contour using Dice-coefficient (DC) and distance-to-agreement (DT) metrics. Further the contours were qualitatively reviewed to identify volumes of disagreement, and the potential for downstream clinical impact.

Results:

Quantitative measurements are in line with past studies of U-Net algorithm based auto-segmentation. Qualitatively, auto-segmentation contours differ from the clinical contours for different reasons.

The agreement for bladder (DC=97%, DT=4.3 mm) speaks to its relatively consistent shape and position in the pelvic region, as well as soft tissue contrast. Auto-segmentation generates overlapping bladder and prostate, and this may contribute to disagreement in prostate contours (90%, 7.7 mm) when clinically mutual exclusivity is enforced.

Disagreement for the high-contrast femurs (83%, 12.1 mm) results from variation in the number of slices contoured. While auto-segmentation tends to contour the full extent of the femoral heads, expert-generated contours may truncate to only the relevant slices, i.e. those likely to receive doses relevant for plan optimization or assessment. This was also seen in the rectum (80%, 14 mm) and bowel bag (56%, 33 mm).

Penile bulb (59%, 6 mm) highlights a common evaluation challenge with small structures. Even single slice differences in contours lead to significant DC differences, the 6 mm DT better captures the variation seen.

Conclusions:

This study suggests auto segmentation can be used effectively to begin the contouring process. It allows trained experts to focus more time on difficult contours and defining problem areas.

Poster
non-peer-reviewed

Qualitative Evaluation of AI Contours


Author Information

Radim Barta Corresponding Author

Medical Physics, Central Alberta Cancer Centre, Red Deer, CAN

Ian Nygren

Medical Physics, Central Alberta Cancer Centre, Red Deer, CAN

Charles Kirkby

Jack Ady Cancer Centre (lethbridge), University of Calgary, Lethbridge, CAN

Conor Shaw

Jack Ady Cancer Centre, University of Calgary, Lethbridge, CAN

Samuel See

Radiation Therapy Dosimetrist, Grande Prairie Cancer Centre, Grande Prairie, CAN


PDF Share