Contents

Digital Slide Archive (DSA) Visualization Tutorial

Digital Slide Archive (DSA) Visualization Tutorial

Welcome to the Digial Slide Archive (DSA) visualization notebook!

Digital Slide Archive (DSA) is a platform where you can manage your pathology images and annotations. A set of CLIs are available to help you convert your pathologist or model-generated annotations and push them to DSA. For more details on the DSA platform, please refer to their documentation.

In this notebook, we will use dsa_viz and dsa_upload CLIs to convert your annotations to a DSA compatible format and to upload them to DSA. We support results from Qupath/Stardist dectection models, tile scores in a tabular format, and also expert annotations in geojson format. Here are the steps we will review:

  • Setup DSA

  • DSA Visuzaliation CLIs

  • Upload Slidveiewer regional annotations

  • Upload Qupath regional annotation results

  • Upload a heatmap generated from tile scores

  • Upload bitmasks PNGs

  • Upload Stardist object detection results

  • Upload Stardist cell detection results

Setup DSA

Before running this notebook, make sure you have your pathology slides organized in DSA under a collection/folder.

For one of our examples, we use an SVS image from TCGA. This image file in tcga collection, in slides folder on the DSA platform.

DSA Organization Screenshot

The collection name (e.g. tcga) and image file name (e.g. TCGA-GM-A2DB-01Z-00-DX1.9EE36AA6-2594-44C7-B05C-91A0AEC7E511.svs) on DSA will be used while uploading the annotations.

Be sure to modify the configuration files based on your DSA setup.

DSA Visualization CLIs

Luna-pathology offers 2 CLIs to help convert your annotation results to a DSA compatible json format and to upload them to DSA. The conversion and upload is divided in to 2 separate steps, so each step can be parallelized based on your computing capabilities and DSA platform setup.

dsa_viz CLI takes two inputs, and generates a DSA compatible annotation json. - source_type - describes the data source - data_config - a file detailing the input, output and visualization details

dsa_upload CLI takes two inputs, and uploads the DSA compatible annotation json. - config - a file with DSA instance information We use a DSA token to push annotations to the platform. Make sure this token is generated via the HistomicsUI API (/api/v1/token/current) for the user with edit permission on the slides. HistomicsUI is the high-magnification viewer part DSA platform. - data_config - a file specifying the path to the DSA compatible annotation json and the collection/image file name from DSA.

Once upload is done, we print the link to HistomicsUI viewer, so you can easily navigate to the uploaded annotation result.

Note: Pushing and rendering a large number of annotation elements can take a long time. Please refer to DSA `documentation <https://digitalslidearchive.github.io/HistomicsTK/examples/tips_for_scalable_annotation_rendering>`__, for user expectations and some tricks for managing annotations.

[4]:

!dsa_viz --help

Usage: dsa_viz [OPTIONS]

  DSA annotation builder

Options:
  -d, --data_config PATH  path to data config file  [required]
  -s, --source_type TEXT  string describing data source that is to be
                          transformed into a dsa json.                supports
                          stardist-polygon, stardist-cell, regional-polygon,
                          qupath-polygon, bitmask-polygon, heatmap  [required]

  --help                  Show this message and exit.

[8]:

!dsa_upload --help

Usage: dsa_upload [OPTIONS]

  DSA Annotation Upload CLI

Options:
  -c, --config PATH       json including DSA host, port and token info
                          [required]

  -d, --data_config PATH  path to data config file  [required]
  --help                  Show this message and exit.

Upload Slideviewer regional annotations

Regional annotations made by expert pathologists are stored in our databases in a GeoJSON format that is compatible with many downstream analysis tools.

With regional-polygon option, we can convert the GeoJSON annotations to DSA json annotation format, and upload the annotation to the DSA platform.

[1]:

# example data_config
!cat configs/dsa_configs/regional_config.yaml

# Path to regional annotation in GeoJSON format
input: ./input/regional_annotation.json

# image file name from DSA
image_filename: 2551571.svs

# path to write converted json annotations
output_folder: ./dsa_annotations/

# annotation name to be shown in DSA
annotation_name: Regional Annotation

# line and fill colors in label:rgb/a format
line_colors:
  lympho_rich_tumor: rgb(0,191,255)
  lympho_poor_tumor: rgb(0,255,0)
  necrosis: rgb(255,255,0)
  lympho_rich_stroma: rgb(255,0,0)
  veins: rgb(0,255,255)
  adipocytes: rgb(0,0,255)
  qc_tissue_fold: rgb(0,0,0)

fill_colors:
  lympho_rich_tumor: rgb(0,191,255,100)
  lympho_poor_tumor: rgb(0,255,0,100)
  necrosis: rgb(255,255,0,100)
  lympho_rich_stroma: rgb(255,0,0,100)
  veins: rgb(0,255,255,100)
  adipocytes: rgb(0,0,255,100)
  qc_tissue_fold: rgb(0,0,0,100)


[57]:

# generate DSA annotation
!dsa_viz -s regional-polygon -d configs/dsa_configs/regional_config.yaml

Building annotation for image: 2551571.svs
Time to build annotation 0.0010259151458740234
Annotation written to ./dsa_annotations/Regional_Annotation_2551571.json

[58]:

# push annotation to DSA
!dsa_upload -c dsa_configs/dsa_config.yaml -d configs/dsa_configs/regional_upload.yaml

Successfully connected to DSA.
collection_id_dict {'_accessLevel': 2, '_id': '60807410150bd39c9df579cf', '_modelType': 'collection', '_textScore': 15.75, 'created': '2021-04-21T18:50:56.730000+00:00', 'description': 'test images', 'meta': {}, 'name': 'test-path', 'public': True, 'publicFlags': [], 'size': 2350018806, 'updated': '2021-04-21T18:50:56.730000+00:00'}
Collection test-path found with id: 60807410150bd39c9df579cf
Annotation successfully pushed to DSA.
Time to push annotation 0.020156383514404297
http://localhost:8080/histomics#?image=610aedc9150bd39c9d7adadd

Upload a heatmap generated from tile scores

heatmap option provides means to visualize your tile scores from your classification model.

The input here is a CSV file with a column of (0, 1) range and the tile coordinates. In this example, we visualize the “otsu_score” column.

We use the color palette “viridis” where the output color ranges from purple to yellow, for scores from 0 to 1.

[48]:

# example CSV
!head -5 input/tile_scores_and_labels.csv

address,coordinates,otsu_score,purple_score
x1_y1_z10,"(1, 1)",0.03125,0.03125
x1_y2_z10,"(1, 2)",0.0,0.0
x1_y3_z10,"(1, 3)",0.0,0.0546875
x1_y4_z10,"(1, 4)",0.0,0.0

[2]:

# example data_config
!cat configs/dsa_configs/heatmap_config.yaml

# path to CSV containing tile coordinates and tile scores
input: input/tile_scores_and_labels.csv

# image file name from DSA
image_filename: TCGA-GM-A2DB-01Z-00-DX1.9EE36AA6-2594-44C7-B05C-91A0AEC7E511.svs

# path to write converted json annotations
output_folder: ./dsa_annotations/

# column in CSV with value range (0, 1)
column: otsu_score

# annotation name to be shown in DSA
annotation_name: tile-based heatmap

# size of the tiles
tile_size: 128

# scale to match the image on DSA
scale_factor: 4


[32]:

# generate DSA annotation
!dsa_viz -s heatmap -d configs/dsa_configs/heatmap_config.yaml

Building annotation for image: TCGA-GM-A2DB-01Z-00-DX1.9EE36AA6-2594-44C7-B05C-91A0AEC7E511.svs
Annotation written to ./dsa_annotations/otsu_score_tile-based_heatmap_TCGA-GM-A2DB-01Z-00-DX1.9EE36AA6-2594-44C7-B05C-91A0AEC7E511.json

[7]:

# push annotation to DSA
!dsa_upload -c dsa_configs/dsa_config.yaml -d configs/dsa_configs/heatmap_upload.yaml

Usage: dsa_upload [OPTIONS]
Try 'dsa_upload --help' for help.

Error: Invalid value for '-c' / '--config': Path 'dsa_configs/dsa_config.yaml' does not exist.

Upload bitmasks PNGs

Simple PNG bitmasks can also be visualized in DSA. Use bitmask-polygon option and provide pngs with the corresponding labels.

[3]:

# example data_config
!cat configs/dsa_configs/bitmask_polygon_config.yaml

# paths to bitmask png images in label:path format
input:
  Tumor: /gpfs/mskmindhdp_emc/user/shared_data_folder/ov-mind-project/input/Tumor.png
  Fat: /gpfs/mskmindhdp_emc/user/shared_data_folder/ov-mind-project/input/Fat.png
  Necrosis: /gpfs/mskmindhdp_emc/user/shared_data_folder/ov-mind-project/input/Necrosis.png
  Stroma: /gpfs/mskmindhdp_emc/user/shared_data_folder/ov-mind-project/input/Stroma.png

# image file name from DSA
image_filename: HobI16-670289739721.svs

# path to write converted json annotations
output_folder: ./dsa_annotations/

# annotation name to be shown in DSA
annotation_name: Full Res Tile-Based Pixel Classifier Inference,

# line and fill colors in label:rgb/a format
line_colors:
  Stroma: rgb(0,191,255)
  Tumor: rgb(0,255,0)
  Fat: rgb(255,255,0)
  Necrosis: rgb(255,0,0)

fill_colors:
  Stroma: rgba(0,191,255, 100)
  Tumor: rgba(0,255,0, 100)
  Fat: rgba(255,255,0, 100)
  Necrosis: rgba(255,0,0, 100)


[16]:

# generate DSA annotation
!dsa_viz -s bitmask-polygon -d configs/dsa_configs/bitmask_polygon_config.yaml

Building annotation for image: HobI16-670289739721.svs
Time to build annotation 138.78823733329773
Annotation written to ./dsa_annotations/Full_Res_Tile-Based_Pixel_Classifier_Inference_HobI16-670289739721.json

[19]:

# push annotation to DSA
!dsa_upload -c dsa_configs/dsa_config.yaml -d configs/dsa_configs/bitmask_upload.yaml

Successfully connected to DSA.
collection_id_dict {'_accessLevel': 2, '_id': '60807410150bd39c9df579cf', '_modelType': 'collection', '_textScore': 15.75, 'created': '2021-04-21T18:50:56.730000+00:00', 'description': 'test images', 'meta': {}, 'name': 'test-path', 'public': True, 'publicFlags': [], 'size': 936444465, 'updated': '2021-04-21T18:50:56.730000+00:00'}
Collection test-path found with id: 60807410150bd39c9df579cf
Annotation successfully pushed to DSA.
Time to push annotation 0.11921572685241699
http://localhost:8080/histomics#?image=60807ad8150bd39c9df579d2

Upload Qupath regional annotation results

Regional annotations generated by Qupath includes regional polygons from object detection along with nuclear properties.

For object and cell detection models in QuPath, please checkout our Qupath/Stardist docker.

[4]:

# example data_config
!cat configs/dsa_configs/qupath_config.yaml

# path to geojson file generated by Qupath
input: input/qupath_annotation_results.json

# image file name from DSA
image_filename: HobI20-681330526186.svs

# path to write converted json annotations
output_folder: ./dsa_annotations/

# annotation name to be shown in DSA
annotation_name: Qupath Pixel Classifier Polygons

# labels to visualize
classes_to_include:
- Adipocytes
- Stroma
- Necrosis
- Tumor
- Other

# line and fill colors in label:rgb/a format
line_colors:
  Stroma: rgb(0, 191, 255)
  Tumor: rgb(0, 255, 0)
  Adipocytes: rgb(255, 255, 0)
  Necrosis: rgb(255, 0, 0)

fill_colors:
  Stroma: rgba(0, 191, 255, 100)
  Tumor: rgba(0, 255, 0, 100)
  Adipocytes: rgba(255, 255, 0, 100)
  Necrosis: rgba(255, 0, 0, 100)


[44]:

# generate DSA annotation
!dsa_viz -s qupath-polygon -d configs/dsa_configs/qupath_config.yaml

Building annotation for image: HobI20-681330526186.svs
Time to build annotation 0.0030074119567871094
Annotation written to ./dsa_annotations/Qupath_Pixel_Classifier_Polygons_HobI20-681330526186.json

[45]:

# push annotation to DSA
!dsa_upload -c dsa_configs/dsa_config.yaml -d configs/dsa_configs/qupath_upload.yaml

Successfully connected to DSA.
collection_id_dict {'_accessLevel': 2, '_id': '60807410150bd39c9df579cf', '_modelType': 'collection', '_textScore': 15.75, 'created': '2021-04-21T18:50:56.730000+00:00', 'description': 'test images', 'meta': {}, 'name': 'test-path', 'public': True, 'publicFlags': [], 'size': 2350018806, 'updated': '2021-04-21T18:50:56.730000+00:00'}
Collection test-path found with id: 60807410150bd39c9df579cf
Annotation successfully pushed to DSA.
Time to push annotation 0.0221097469329834
http://localhost:8080/histomics#?image=60807ad8150bd39c9df579d2

Upload Stardist object detection results

Stardist is a nuclear segmentation algorithm that is quite capable in detecting and segmenting cells/nuclei in pathology images. stardist-polygon option converts Stardist object detection results as polygons capturing different types of cells.

Note: this command can take a few minutes if object detection is run on the whole slide.

[5]:

# example data_config
!cat configs/dsa_configs/stardist_polygon_config.yaml

# regional annotation in GeoJSON format generated by Stardist
input: input/object_detection_results.geojson

# image file name from DSA
image_filename: HobI20-681330526186.svs

# path to write converted json annotations
output_folder: ./dsa_annotations/

# annotation name to be shown in DSA
annotation_name: StarDist Segmentations with Lymphocyte Classifications

# line and fill colors in label:rgb/a format
line_colors:
  Other: rgb(0, 255, 0)
  Lymphocyte: rgb(255, 0, 0)

fill_colors:
  Other: rgba(0, 255, 0, 100)
  Lymphocyte: rgba(255, 0, 0, 100)


[24]:

# generate DSA annotation
!dsa_viz -s stardist-polygon -d configs/dsa_configs/stardist_polygon_config.yaml

Building annotation for image: HobI20-681330526186.svs
Time to build annotation 251.02770948410034
Annotation written to ./dsa_annotations/StarDist_Segmentations_with_Lymphocyte_Classifications_HobI20-681330526186.json

[26]:

# push annotation to DSA
!dsa_upload -c dsa_configs/dsa_config.yaml -d configs/dsa_configs/stardist_polygon_upload.yaml

Successfully connected to DSA.
collection_id_dict {'_accessLevel': 2, '_id': '60807410150bd39c9df579cf', '_modelType': 'collection', '_textScore': 15.75, 'created': '2021-04-21T18:50:56.730000+00:00', 'description': 'test images', 'meta': {}, 'name': 'test-path', 'public': True, 'publicFlags': [], 'size': 936444465, 'updated': '2021-04-21T18:50:56.730000+00:00'}
Collection test-path found with id: 60807410150bd39c9df579cf
Annotation successfully pushed to DSA.
Time to push annotation 584.9241287708282
http://localhost:8080/histomics#?image=60807ad8150bd39c9df579d2

This is a screenshot form HistomicsUI, the high-magnification viewer. You can zoom in and view your annotation results with the desired opacity. As specified in dsa_configs/stardist_polygon_config.yaml, the red objects are classified as lymphocytes and the green cells are “other” cells.

Stardist Polygon Screenshot

Upload Stardist cell detection results

Here we use cellular detection results generated from Stardist. The x,y coordinates of the cells in the input TSV file will be visualized as a point, as opposed to a more complex polygon that we saw in the previous step with stardist-polygon. You’ll notice that the point annotation is faster to upload compared to the polygon represenation of the cells.

We also set fill color alpha value to 0 makes annotation upload faster.

[6]:

# example data_config
!cat configs/dsa_configs/stardist_cell_config.yaml

# TSV with cell coordinates generated by Stardist
input: input/object_detection_results.tsv

# image file name from DSA
image_filename: HobI20-681330526186.svs

# path to write converted json annotations
annotation_name: Points of Classsified StarDist Cells

# annotation name to be shown in DSA
output_folder: ./dsa_annotations/

# line and fill colors in label:rgb/a format
line_colors:
  Other: rgb(0, 255, 0)
  Lymphocyte: rgb(255, 0, 0)

fill_colors:
  Other: rgba(0, 255, 0, 0)
  Lymphocyte: rgba(255, 0, 0, 0)


[28]:

# generate DSA annotation
!dsa_viz -s stardist-cell -d configs/dsa_configs/stardist_cell_config.yaml

Building annotation for image: HobI20-681330526186.svs
Time to build annotation 90.03980946540833
Annotation written to ./dsa_annotations/Points_of_Classsified_StarDist_Cells_HobI20-681330526186.json

[29]:

# push annotation to DSA
!dsa_upload -c dsa_configs/dsa_config.yaml -d configs/dsa_configs/stardist_cell_upload.yaml

Successfully connected to DSA.
collection_id_dict {'_accessLevel': 2, '_id': '60807410150bd39c9df579cf', '_modelType': 'collection', '_textScore': 15.75, 'created': '2021-04-21T18:50:56.730000+00:00', 'description': 'test images', 'meta': {}, 'name': 'test-path', 'public': True, 'publicFlags': [], 'size': 936444465, 'updated': '2021-04-21T18:50:56.730000+00:00'}
Collection test-path found with id: 60807410150bd39c9df579cf
Annotation successfully pushed to DSA.
Time to push annotation 202.15096521377563
http://localhost:8080/histomics#?image=60807ad8150bd39c9df579d2

Below is another screenshot from HistomicsUI, from the link printed above. The results are the same as stardist-polygon visualization. Notice the cells are captured more minimally as circles, and not polygons. For rapid prototyping, stardist-cell offers faster annotation upload speed compared to stardist-polygon.

Stardist Cell Screenshot

Congratulations! Now you can visualize your annotations and results on DSA platform.

[ ]:

previous

Snakemake End-To-End Tutorial

next

DSA Annotation