Interactive Segments Editing

The interactive segments editing user interface is used via XF.Interactive.DrawSegments.

Basic Drawing

The basic drawing operations in the image view are:

  • Left mouse button: Draw.

  • Right mouse button: Erase.

  • CTRL + mouse wheel: Change brush size.

  • CTRL + Z: Undo.

  • CTRL + Y: Redo.

In the sidebar various tools offer additional operations:

Draw

  • Radius changes the brush size.

  • Shape changes the brush type.

  • Segment value changes the label value for drawing.

  • Transparency affects visualization of the segmented overlay.

Brush Types

Shapes:

  • 2D disk

  • 3D cylinder

  • 3D sphere

  • 2D thresholded disk

  • 3D thresholded cylinder

  • 3D thresholded sphere

Thresholded brushes observe the range configured in the thresholding tool (see below).

Init Segments

  • Clear or fill the image (or VOI)

  • Initialize the entire image (or VOI) with a disk, sphere, cylinder or checkerboard pattern.

  • By default only the current 2D view is affected. If the 3D checkbox is activated, the entire 3D image (or VOI) is initialized.

VOI

  • Enable an interactive volume of interest selection box.

  • When this is enabled it limits other tools to the indicated area.

Thresholding

  • Segment the image (or VOI) histogram using the indicated thresholds.

Advanced Operations

Active Balloon

  • Grow or shrink a balloon to fit the image, using iterative level-set evolution.

  • Use for example Init Segments first to create an starting mask.

  • Click Balloon Step to grow or shrink the current mask.

  • Use the VOI tool to limit the processing to a certain area. (This can be important for large images where the processing is time consuming.)

  • Use the Thresholding tool to indicate the image values that should cause growing or shrinking.

Interpolate

  • Switch to the Ortho views to enable this tool.

  • Draw segments on selected key Z slices, for example the first and last slice in Z.

  • Click Interpolate All to fill in the space between those slices.

By default the foreground segment value 1 (red) should be used on the key breakpoint slices. Only the pixels with value 1 (red) are used for the interpolation. Value 2 (green) is used for the interpolated slices.

  • Click Set breakpoint to change the values on the current slice from green to red.

napari

Alternatively external programs can also be integrated. For example with Example.napari.DrawSegments the napari program can be used seamlessly as a step in a workflow.

Machine Learning Models: Segment Anything

Many new machine learning models promise to offer powerful image segmentation, and can be integrated in XamFlow. For example Meta’s Segment Anything Model (SAM) is one such model. The fully automatic task type Example.SAM.SegmentImage creates a multi-valued mask of all detected objects in an image.

Integration with the interactive UI for a semi-interactive guided segmentation is also possible. Lucid.Core.DrawSegmentsWithSAM integrates the SAM model in addition to all the normal interactive functionality.

  • Click Start SAM Server to start the model.

  • Click Enable VOI in the VOI tool to position the volume of interest.

  • Click SAM VOI 2D Z to segment the object in the indicated region with the current segment value.

  • The keyboard shortcut V can also be used instead.