{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Digital Slide Archive (DSA) Visualization Tutorial" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Welcome to the Digial Slide Archive (DSA) visualization notebook! \n", "\n", "Digital Slide Archive (DSA) is a platform where you can manage your pathology images and annotations. A set of CLIs are available to help you convert your pathologist or model-generated annotations and push them to DSA. For more details on the DSA platform, please refer to their [documentation](https://digitalslidearchive.github.io/digital_slide_archive/documentation/).\n", "\n", "In this notebook, we will use **dsa** and **dsa_upload** CLIs to convert your annotations to a DSA compatible format and to upload them to DSA. We support results from Qupath/Stardist dectection models ([link to docker image](https://hub.docker.com/r/mskmind/qupath-stardist)), tile scores in a tabular format, and also expert annotations in geojson format. Here are the steps we will review:\n", "\n", "- Setup DSA\n", "- DSA Visuzaliation CLIs\n", "- Upload Qupath regional annotation results\n", "- Upload a heatmap generated from tile scores\n", "- Upload bitmasks PNGs\n", "- Upload bmp results\n", "- Upload Stardist object detection results\n", "- Upload Stardist cell detection results\n" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "env: LUNA_HOME=/Users/rosed2/Documents/msk-mind/luna\n", "env: PYTHONPATH=/Users/rosed2/Documents/msk-mind/luna/pyluna-pathology:/Users/rosed2/Documents/msk-mind/luna/pyluna-common:.\n" ] } ], "source": [ "# TEMP\n", "%env LUNA_HOME=/Users/rosed2/Documents/msk-mind/luna\n", "%env PYTHONPATH=/Users/rosed2/Documents/msk-mind/luna/pyluna-pathology:/Users/rosed2/Documents/msk-mind/luna/pyluna-common:.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Setup DSA" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Before running this notebook, make sure you have your pathology slides organized in DSA under a collection/folder.\n", "\n", "For one of our examples, we use an SVS image from TCGA. This image file in *tcga* collection, in *slides* folder on the DSA platform.\n", "\n", "![DSA Organization Screenshot](../img/dsa-organization-screenshot.png)\n", "\n", "The collection name (e.g. tcga) and image file name (e.g. TCGA-GM-A2DB-01Z-00-DX1.9EE36AA6-2594-44C7-B05C-91A0AEC7E511.svs) on DSA will be used while uploading the annotations.\n", "\n", "Be sure to modify the configuration files based on your DSA setup." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## DSA Visualization CLIs" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Luna Pathology offers 2 CLIs to help convert your annotation results to a DSA compatible json format and to upload them to DSA. The conversion and upload is divided in to 2 separate steps, so each step can be parallelized based on your computing capabilities and DSA platform setup. \n", "\n", "`dsa [COMMAND]` generates a DSA compatible annotation json.\n", "\n", "`dsa_upload` uploads the DSA compatible annotation json.\n", "\n", "Once upload is done, we print the link to HistomicsUI viewer, so you can easily navigate to the uploaded annotation result.\n", "\n", "*Note: Pushing and rendering a large number of annotation elements can take a long time. Please refer to DSA [documentation](https://digitalslidearchive.github.io/HistomicsTK/examples/tips_for_scalable_annotation_rendering), for user expectations and some tricks for managing annotations.*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For `dsa_upload` you can set credentials in environment variables, or pass in the username and passwords as parameters to the cli. For example, update your user credentials in the cell below:" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "env: DSA_USERNAME=username\n", "env: DSA_PASSWORD=password\n" ] } ], "source": [ "%env DSA_USERNAME=username\n", "%env DSA_PASSWORD=password" ] }, { "cell_type": "code", "execution_count": 21, "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "2022-04-05 17:32:52,102 - INFO - root - Initalized logger, log file at: data-processing.log\n", "Usage: dsa [OPTIONS] COMMAND [ARGS]...\n", "\n", " Convert segmentations, bitmasks, heatmaps to DSA annotation Json format.\n", "\n", "Options:\n", " --help Show this message and exit.\n", "\n", "Commands:\n", " bitmask-polygon Example: dsa bitmask-polygon '{\"Tumor\":...\n", " bmp-polygon Example: dsa bmp-polygon results.bmp --output_dir...\n", " heatmap Example: dsa heatmap score.csv --output_dir...\n", " qupath-polygon Example: dsa qupath-polygon...\n", " regional-polygon Example: dsa regional-polygon...\n", " stardist-cell Example: dsa stardist-cell...\n", " stardist-polygon Example: dsa stardist-polygon...\n" ] } ], "source": [ "# check available dsa cli commands\n", "!dsa --help" ] }, { "cell_type": "code", "execution_count": 35, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "2022-04-05 10:36:47,657 - INFO - root - FYI: Initalized logger, log file at: data-processing.log with handlers: [ (INFO)>, ]\r\n", "Usage: dsa stardist-polygon [OPTIONS] INPUT\r\n", "\r\n", " Example:\r\n", "\r\n", " dsa stardist-polygon\r\n", " ../dsa_input/test_object_classification.geojson\r\n", " --output_dir ../dsa_annotations/stardist_polygon\r\n", " --annotation_name stardist_polygon_segmentations\r\n", " --image_filename 123.svs\r\n", " --line_colors '{\"Other\": \"rgb(0,255,0)\", \"Lymphocyte\": \"rgb(255,0,0)\"}'\r\n", " --fill_colors '{\"Other\": \"rgba(0,255,0,100)\", \"Lymphocyte\": \"rgba(255,0,0,100)\"}'\r\n", "\r\n", "Options:\r\n", " -m, --method_param_path TEXT path to a metadata json/yaml file with method\r\n", " parameters to reproduce results\r\n", "\r\n", " -fc, --fill_colors TEXT user-provided line color map with {feature\r\n", " name:rgba values}\r\n", "\r\n", " -lc, --line_colors TEXT user-provided line color map with {feature\r\n", " name:rgb values}\r\n", "\r\n", " -a, --annotation_name TEXT name of the annotation to be displayed in DSA\r\n", " -f, --image_filename TEXT name of the image file in DSA e.g. 123.svs\r\n", " -o, --output_dir TEXT directory to save the DSA compatible\r\n", " annotation json\r\n", "\r\n", " --help Show this message and exit.\r\n" ] } ], "source": [ "# check dsa cli commands help messages\n", "!dsa stardist-polygon --help" ] }, { "cell_type": "code", "execution_count": 30, "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Traceback (most recent call last):\r\n", " File \"/Users/rosed2/Documents/msk-mind/env/bin/dsa_upload\", line 33, in \r\n", " sys.exit(load_entry_point('pyluna-pathology==0.1.1', 'console_scripts', 'dsa_upload')())\r\n", " File \"/Users/rosed2/Documents/msk-mind/env/bin/dsa_upload\", line 25, in importlib_load_entry_point\r\n", " return next(matches).load()\r\n", " File \"/usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/metadata.py\", line 77, in load\r\n", " module = import_module(match.group('module'))\r\n", " File \"/usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/__init__.py\", line 127, in import_module\r\n", " return _bootstrap._gcd_import(name[level:], package, level)\r\n", " File \"\", line 1030, in _gcd_import\r\n", " File \"\", line 1007, in _find_and_load\r\n", " File \"\", line 986, in _find_and_load_unlocked\r\n", " File \"\", line 680, in _load_unlocked\r\n", " File \"\", line 850, in exec_module\r\n", " File \"\", line 228, in _call_with_frames_removed\r\n", " File \"/Users/rosed2/Documents/msk-mind/luna/pyluna-pathology/luna/pathology/cli/dsa_upload.py\", line 8, in \r\n", " from luna.pathology.dsa.dsa_api_handler import (\r\n", " File \"/Users/rosed2/Documents/msk-mind/luna/pyluna-pathology/luna/pathology/dsa/dsa_api_handler.py\", line 13, in \r\n", " import histomicstk\r\n", " File \"/Users/rosed2/Documents/msk-mind/env/lib/python3.9/site-packages/histomicstk/__init__.py\", line 7, in \r\n", " from . import segmentation # must be imported before features\r\n", " File \"/Users/rosed2/Documents/msk-mind/env/lib/python3.9/site-packages/histomicstk/segmentation/__init__.py\", line 15, in \r\n", " from . import label\r\n", " File \"/Users/rosed2/Documents/msk-mind/env/lib/python3.9/site-packages/histomicstk/segmentation/label/__init__.py\", line 15, in \r\n", " from .trace_object_boundaries import trace_object_boundaries\r\n", " File \"/Users/rosed2/Documents/msk-mind/env/lib/python3.9/site-packages/histomicstk/segmentation/label/trace_object_boundaries.py\", line 4, in \r\n", " from ._trace_object_boundaries_cython import _trace_object_boundaries_cython\r\n", " File \"_trace_object_boundaries_cython.pyx\", line 1, in init histomicstk.segmentation.label._trace_object_boundaries_cython\r\n", "ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject\r\n" ] } ], "source": [ "# check dsa_upload help messages\n", "!dsa_upload --help" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Upload a heatmap generated from tile scores" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**heatmap** option provides means to visualize your tile scores from your classification model.\n", "\n", "The input here is a CSV file with a column of (0, 1) range and the tile coordinates. In this example, we visualize the \"purple_score\" column.\n", "\n", "We use the color palette \"viridis\" where the output color ranges from purple to yellow, for scores from 0 to 1." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "address,coordinates,otsu_score,purple_score\r\n", "x1_y1_z10,\"(1, 1)\",0.03125,0.03125\r\n", "x1_y2_z10,\"(1, 2)\",0.0,0.0\r\n", "x1_y3_z10,\"(1, 3)\",0.0,0.0546875\r\n", "x1_y4_z10,\"(1, 4)\",0.0,0.0\r\n" ] } ], "source": [ "# example CSV\n", "!head -5 ../dsa_input/tile_scores.csv" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "2022-04-05 17:20:01,456 - INFO - root - Initalized logger, log file at: data-processing.log\r\n", "Usage: dsa heatmap [OPTIONS] INPUT\r\n", "\r\n", " Example:\r\n", "\r\n", " dsa heatmap\r\n", " score.csv\r\n", " --output_dir ../dsa_annotations/heatmap\r\n", " --annotation_name heatmap\r\n", " --image_filename 123.svs\r\n", " --tile_size 256\r\n", " --column tumor\r\n", " --scale_factor 1\r\n", "\r\n", "Options:\r\n", " -m, --method_param_path TEXT path to a metadata json/yaml file with method\r\n", " parameters to reproduce results\r\n", "\r\n", " -sc, --scale_factor INTEGER scale to match image DSA. (default 1)\r\n", " -ts, --tile_size TEXT tile size\r\n", " -a, --annotation_name TEXT name of the annotation to be displayed in DSA\r\n", " -f, --image_filename TEXT name of the image file in DSA e.g. 123.svs\r\n", " -o, --output_dir TEXT directory to save the DSA compatible\r\n", " annotation json\r\n", "\r\n", " --help Show this message and exit.\r\n" ] } ], "source": [ "# example data_config\n", "!dsa heatmap --help" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "2022-04-05 17:23:28,917 - INFO - root - Initalized logger, log file at: data-processing.log\r\n", "2022-04-05 17:23:28,918 - INFO - luna.common.utils - Running with {'output_dir': '../dsa_annotations/heatmap', 'annotation_name': 'heatmap', 'image_filename': '123.svs', 'tile_size': '256', 'column': 'purple_score', 'scale_factor': 1, 'input': '../dsa_input/tile_scores.csv', 'method_param_path': None}\r\n", "2022-04-05 17:23:28,918 - INFO - luna.common.utils - Param column set = purple_score\r\n", "2022-04-05 17:23:28,918 - INFO - luna.common.utils - Param tile_size set = 256\r\n", "2022-04-05 17:23:28,919 - INFO - luna.common.utils - Param annotation_name set = heatmap\r\n", "2022-04-05 17:23:28,919 - INFO - luna.common.utils - Param image_filename set = 123.svs\r\n", "2022-04-05 17:23:28,919 - INFO - luna.common.utils - Param output_dir set = ../dsa_annotations/heatmap\r\n", "2022-04-05 17:23:28,919 - INFO - luna.common.utils - Param input set = ../dsa_input/tile_scores.csv\r\n", "2022-04-05 17:23:28,919 - INFO - luna.common.utils - Param scale_factor set = 1\r\n", "2022-04-05 17:23:28,919 - INFO - luna.common.utils - Full segment key set: {}\r\n", "\r\n", "----------------------------------- Running transform::heatmap_main -----------------------------------\r\n", "\r\n", "2022-04-05 17:23:28,952 - INFO - luna.common.utils - Code block 'transform::heatmap_main' took: 0.032535148000000014s\r\n", "2022-04-05 17:23:28,953 - INFO - luna.common.utils - Done.\r\n" ] } ], "source": [ "# generate DSA annotation\n", "!dsa heatmap ../dsa_input/tile_scores.csv \\\n", "--output_dir ../dsa_annotations/heatmap \\\n", "--annotation_name heatmap \\\n", "--image_filename 123.svs \\\n", "--tile_size 256 \\\n", "--column purple_score \\\n", "--scale_factor 1" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Usage: dsa_upload [OPTIONS]\n", "Try 'dsa_upload --help' for help.\n", "\n", "Error: Invalid value for '-c' / '--config': Path 'dsa_configs/dsa_config.yaml' does not exist.\n" ] } ], "source": [ "# push annotation to DSA\n", "!dsa_upload http://localhost:8080/dsa/api/v1 \\\n", "--collection_name tcga-data \\\n", "--image_filename 123.svs \\\n", "--annotation_filepath ../dsa_annotations/heatmap/purple_score_heatmap_123.json" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Upload bitmasks PNGs" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Simple PNG bitmasks can also be visualized in DSA. Use **bitmask-polygon** option and provide pngs with the corresponding labels." ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "2022-04-05 17:30:05,873 - INFO - root - Initalized logger, log file at: data-processing.log\n", "2022-04-05 17:30:05,874 - INFO - luna.common.utils - Running with {'output_dir': '../dsa_annotations/bitmask', 'annotation_name': 'bitmask', 'image_filename': '123.svs', 'line_colors': '{\"Tumor\": \"rgb(255,0,0)\"}', 'fill_colors': '{\"Tumor\": \"rgba(255,0,0,100)\"}', 'input': '{\"Tumor\": \"../dsa_input/Tumor.png\"}', 'method_param_path': None}\n", "2022-04-05 17:30:05,875 - INFO - luna.common.utils - Param fill_colors set = {'Tumor': 'rgba(255,0,0,100)'}\n", "2022-04-05 17:30:05,875 - INFO - luna.common.utils - Param line_colors set = {'Tumor': 'rgb(255,0,0)'}\n", "2022-04-05 17:30:05,875 - INFO - luna.common.utils - Param annotation_name set = bitmask\n", "2022-04-05 17:30:05,875 - INFO - luna.common.utils - Param image_filename set = 123.svs\n", "2022-04-05 17:30:05,875 - INFO - luna.common.utils - Param output_dir set = ../dsa_annotations/bitmask\n", "2022-04-05 17:30:05,875 - INFO - luna.common.utils - Param input set = {'Tumor': '../dsa_input/Tumor.png'}\n", "2022-04-05 17:30:05,875 - INFO - luna.common.utils - Full segment key set: {}\n", "\n", "----------------------------------- Running transform::bitmask_polygon_main -----------------------------------\n", "\n", "2022-04-05 17:30:41,244 - INFO - luna.common.utils - Code block 'transform::bitmask_polygon_main' took: 35.368211009s\n", "2022-04-05 17:30:41,246 - INFO - luna.common.utils - Done.\n" ] } ], "source": [ "# generate DSA annotation\n", "!dsa bitmask-polygon \\\n", "'{\"Tumor\": \"../dsa_input/Tumor.png\"}' \\\n", "--output_dir ../dsa_annotations/bitmask \\\n", "--annotation_name bitmask \\\n", "--image_filename 123.svs \\\n", "--line_colors '{\"Tumor\": \"rgb(255,0,0)\"}' \\\n", "--fill_colors '{\"Tumor\": \"rgba(255,0,0,100)\"}'" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Successfully connected to DSA.\n", "collection_id_dict {'_accessLevel': 2, '_id': '60807410150bd39c9df579cf', '_modelType': 'collection', '_textScore': 15.75, 'created': '2021-04-21T18:50:56.730000+00:00', 'description': 'test images', 'meta': {}, 'name': 'test-path', 'public': True, 'publicFlags': [], 'size': 936444465, 'updated': '2021-04-21T18:50:56.730000+00:00'}\n", "Collection test-path found with id: 60807410150bd39c9df579cf\n", "Annotation successfully pushed to DSA.\n", "Time to push annotation 0.11921572685241699\n", "http://localhost:8080/histomics#?image=60807ad8150bd39c9df579d2\n" ] } ], "source": [ "# push annotation to DSA\n", "!dsa_upload http://localhost:8080/dsa/api/v1 \\\n", "--collection_name tcga-data \\\n", "--image_filename 123.svs \\\n", "--annotation_filepath ../dsa_annotations/bitmask/bitmask_123.json" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Upload bmp results" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "BMP is another file format used to store segmentation or classification results. **bmp-polygon** converts the bmps to DSA annotation json." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "2022-04-05 19:03:29,452 - INFO - root - Initalized logger, log file at: data-processing.log\n", "2022-04-05 19:03:29,454 - INFO - luna.common.utils - Running with {'output_dir': '../dsa_annotations/bmp', 'annotation_name': 'bmp', 'image_filename': '123.svs', 'label': '{0: \"Tumor\", 3: \"Other\"}', 'scale_factor': '1', 'line_colors': '{\"Other\": \"rgb(0,255,0)\", \"Tumor\": \"rgb(255,0,0)\"}', 'fill_colors': '{\"Other\": \"rgba(0,255,0,100)\", \"Tumor\": \"rgba(255,0,0,100)\"}', 'input': '../dsa_input/123_label.bmp', 'method_param_path': None}\n", "2022-04-05 19:03:29,454 - INFO - luna.common.utils - Param fill_colors set = {'Other': 'rgba(0,255,0,100)', 'Tumor': 'rgba(255,0,0,100)'}\n", "2022-04-05 19:03:29,454 - INFO - luna.common.utils - Param line_colors set = {'Other': 'rgb(0,255,0)', 'Tumor': 'rgb(255,0,0)'}\n", "2022-04-05 19:03:29,454 - INFO - luna.common.utils - Param annotation_name set = bmp\n", "2022-04-05 19:03:29,454 - INFO - luna.common.utils - Param image_filename set = 123.svs\n", "2022-04-05 19:03:29,455 - INFO - luna.common.utils - Param output_dir set = ../dsa_annotations/bmp\n", "2022-04-05 19:03:29,455 - INFO - luna.common.utils - Param input set = ../dsa_input/123_label.bmp\n", "2022-04-05 19:03:29,455 - INFO - luna.common.utils - Param label set = {0: 'Tumor', 3: 'Other'}\n", "2022-04-05 19:03:29,455 - INFO - luna.common.utils - Param scale_factor set = 1\n", "2022-04-05 19:03:29,455 - INFO - luna.common.utils - Full segment key set: {}\n", "\n", "----------------------------------- Running transform::bmp_polygon_main -----------------------------------\n", "\n", "2022-04-05 19:03:29,628 - INFO - luna.common.utils - Code block 'transform::bmp_polygon_main' took: 0.17287053799999974s\n", "2022-04-05 19:03:29,630 - INFO - luna.common.utils - Done.\n" ] } ], "source": [ "!dsa bmp-polygon \\\n", "../dsa_input/123_label.bmp \\\n", "--output_dir ../dsa_annotations/bmp \\\n", "--annotation_name bmp \\\n", "--image_filename 123.svs \\\n", "--label '{0: \"Tumor\", 3: \"Other\"}' \\\n", "--scale_factor 1 \\\n", "--line_colors '{\"Other\": \"rgb(0,255,0)\", \"Tumor\": \"rgb(255,0,0)\"}' \\\n", "--fill_colors '{\"Other\": \"rgba(0,255,0,100)\", \"Tumor\": \"rgba(255,0,0,100)\"}'" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# push annotation to DSA\n", "!dsa_upload http://localhost:8080/dsa/api/v1 \\\n", "--collection_name tcga-data \\\n", "--image_filename 123.svs \\\n", "--annotation_filepath ../dsa_annotations/bmp/bmp_123.json" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Upload Qupath regional annotation results" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Regional annotations generated by [Qupath](https://qupath.github.io/) includes regional polygons from object detection along with nuclear properties.\n", "\n", "For object and cell detection models in QuPath, please checkout our [Qupath/Stardist docker](https://hub.docker.com/r/mskmind/qupath-stardist)." ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "2022-04-05 17:46:41,381 - INFO - root - Initalized logger, log file at: data-processing.log\r\n", "2022-04-05 17:46:41,382 - INFO - luna.common.utils - Running with {'output_dir': '../dsa_annotations/quppath', 'annotation_name': 'quppath', 'image_filename': '123.svs', 'classes_to_include': 'Other', 'line_colors': '{\"Other\": \"rgb(0,255,0)\"}', 'fill_colors': '{\"Other\": \"rgba(0,255,0,100)\"}', 'input': '../dsa_input/test_object_classification.geojson', 'method_param_path': None}\r\n", "2022-04-05 17:46:41,382 - INFO - luna.common.utils - Param fill_colors set = {'Other': 'rgba(0,255,0,100)'}\r\n", "2022-04-05 17:46:41,382 - INFO - luna.common.utils - Param line_colors set = {'Other': 'rgb(0,255,0)'}\r\n", "2022-04-05 17:46:41,382 - INFO - luna.common.utils - Param annotation_name set = quppath\r\n", "2022-04-05 17:46:41,383 - INFO - luna.common.utils - Param image_filename set = 123.svs\r\n", "2022-04-05 17:46:41,383 - INFO - luna.common.utils - Param output_dir set = ../dsa_annotations/quppath\r\n", "2022-04-05 17:46:41,383 - INFO - luna.common.utils - Param input set = ../dsa_input/test_object_classification.geojson\r\n", "2022-04-05 17:46:41,383 - INFO - luna.common.utils - Param classes_to_include set = ['O', 't', 'h', 'e', 'r']\r\n", "2022-04-05 17:46:41,384 - INFO - luna.common.utils - Full segment key set: {}\r\n", "\r\n", "----------------------------------- Running transform::qupath_polygon_main -----------------------------------\r\n", "\r\n", "2022-04-05 17:46:41,385 - INFO - luna.common.utils - Code block 'transform::qupath_polygon_main' took: 0.0009054450000001157s\r\n", "2022-04-05 17:46:41,386 - INFO - luna.common.utils - Done.\r\n" ] } ], "source": [ "# generate DSA annotation\n", "!dsa qupath-polygon \\\n", "../dsa_input/test_object_classification.geojson \\\n", "--output_dir ../dsa_annotations/quppath \\\n", "--annotation_name quppath \\\n", "--image_filename 123.svs \\\n", "--classes_to_include Other \\\n", "--line_colors '{\"Other\": \"rgb(0,255,0)\"}' \\\n", "--fill_colors '{\"Other\": \"rgba(0,255,0,100)\"}'" ] }, { "cell_type": "code", "execution_count": 45, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Successfully connected to DSA.\n", "collection_id_dict {'_accessLevel': 2, '_id': '60807410150bd39c9df579cf', '_modelType': 'collection', '_textScore': 15.75, 'created': '2021-04-21T18:50:56.730000+00:00', 'description': 'test images', 'meta': {}, 'name': 'test-path', 'public': True, 'publicFlags': [], 'size': 2350018806, 'updated': '2021-04-21T18:50:56.730000+00:00'}\n", "Collection test-path found with id: 60807410150bd39c9df579cf\n", "Annotation successfully pushed to DSA.\n", "Time to push annotation 0.0221097469329834\n", "http://localhost:8080/histomics#?image=60807ad8150bd39c9df579d2\n" ] } ], "source": [ "# push annotation to DSA\n", "!dsa_upload http://localhost:8080/dsa/api/v1 \\\n", "--collection_name tcga-data \\\n", "--image_filename 123.svs \\\n", "--annotation_filepath ../dsa_annotations/qupath/quppath_123.json" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Upload Stardist object detection results" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Stardist is a nuclear segmentation algorithm that is quite capable in detecting and segmenting cells/nuclei in pathology images. **stardist-polygon** option converts Stardist object detection results as polygons capturing different types of cells.\n", "\n", "*Note: this command can take a few minutes if object detection is run on the whole slide.*" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "2022-04-05 09:06:50,899 - INFO - root - FYI: Initalized logger, log file at: data-processing.log with handlers: [ (INFO)>, ]\n", "2022-04-05 09:06:50,900 - INFO - luna.common.utils - Running with {'output_dir': '../dsa_annotations/stardist_polygon', 'annotation_name': 'stardist_polygon_segmentations', 'image_filename': '123.svs', 'line_colors': '{\"Other\": \"rgb(0,255,0)\", \"Lymphocyte\": \"rgb(255,0,0)\"}', 'fill_colors': '{\"Other\": \"rgba(0,255,0,100)\", \"Lymphocyte\": \"rgba(255,0,0,100)\"}', 'input': '../dsa_input/test_object_classification.geojson', 'method_param_path': None}\n", "2022-04-05 09:06:50,900 - INFO - luna.common.utils - Param fill_colors set = {'Other': 'rgba(0,255,0,100)', 'Lymphocyte': 'rgba(255,0,0,100)'}\n", "2022-04-05 09:06:50,900 - INFO - luna.common.utils - Param line_colors set = {'Other': 'rgb(0,255,0)', 'Lymphocyte': 'rgb(255,0,0)'}\n", "2022-04-05 09:06:50,900 - INFO - luna.common.utils - Param annotation_name set = stardist_polygon_segmentations\n", "2022-04-05 09:06:50,900 - INFO - luna.common.utils - Param image_filename set = 123.svs\n", "2022-04-05 09:06:50,900 - INFO - luna.common.utils - Param output_dir set = ../dsa_annotations/stardist_polygon\n", "2022-04-05 09:06:50,900 - INFO - luna.common.utils - Param input set = ../dsa_input/test_object_classification.geojson\n", "2022-04-05 09:06:50,901 - INFO - luna.common.utils - Full segment key set: {}\n", "\n", "----------------------------------- Running transform::stardist_polygon_main -----------------------------------\n", "\n", "2022-04-05 09:06:50,902 - INFO - luna.common.utils - Code block 'transform::stardist_polygon_main' took: 0.001199355000000013s\n", "2022-04-05 09:06:50,903 - INFO - luna.common.utils - Done.\n" ] } ], "source": [ "# generate DSA annotation\n", "!dsa stardist-polygon \\\n", "../dsa_input/test_object_classification.geojson \\\n", "--output_dir ../dsa_annotations/stardist_polygon \\\n", "--annotation_name stardist_polygon_segmentations \\\n", "--image_filename 123.svs \\\n", "--line_colors '{\"Other\": \"rgb(0,255,0)\", \"Lymphocyte\": \"rgb(255,0,0)\"}' \\\n", "--fill_colors '{\"Other\": \"rgba(0,255,0,100)\", \"Lymphocyte\": \"rgba(255,0,0,100)\"}'" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\"description\": \"\", \"elements\": [{\"fillColor\": \"rgba(0,255,0,100)\", \"lineColor\": \"rgb(0,255,0)\", \"lineWidth\": 2, \"type\": \"polyline\", \"closed\": true, \"points\": [[61781.95, 929.31, 0], [61771.77, 932.48, 0], [61763.49, 938.38, 0], [61757.63, 946.76, 0], [61756.68, 949.76, 0], [61756.34, 957.75, 0], [61756.81, 960.34, 0], [61759.75, 967.25, 0], [61761.4, 969.48, 0], [61766.69, 973.96, 0], [61774.61, 976.99, 0], [61776.99, 977.25, 0], [61787.6, 976.28, 0], [61793.96, 974.36, 0], [61803.92, 972.54, 0], [61812.41, 969.1, 0], [61819.84, 962.2, 0], [61821.52, 958.67, 0], [61822.29, 950.5, 0], [61821.42, 947.99, 0], [61816.19, 940.43, 0], [61810.58, 936.37, 0], [61803.19, 933.06, 0], [61793.31, 929.64, 0], [61781.95, 929.31, 0]], \"label\": {\"value\": \"Other\"}}], \"name\": \"stardist_polygon_segmentations\"}" ] } ], "source": [ "# check json annotation\n", "!head ../dsa_annotations/stardist_polygon/stardist_polygon_segmentations_123.json" ] }, { "cell_type": "code", "execution_count": 22, "metadata": { "collapsed": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "env: DSA_USERNAME=username\n", "env: DSA_PASSWORD=password\n", "Traceback (most recent call last):\n", " File \"/Users/rosed2/Documents/msk-mind/env/bin/dsa_upload\", line 33, in \n", " sys.exit(load_entry_point('pyluna-pathology==0.1.1', 'console_scripts', 'dsa_upload')())\n", " File \"/Users/rosed2/Documents/msk-mind/env/bin/dsa_upload\", line 25, in importlib_load_entry_point\n", " return next(matches).load()\n", " File \"/usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/metadata.py\", line 77, in load\n", " module = import_module(match.group('module'))\n", " File \"/usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/__init__.py\", line 127, in import_module\n", " return _bootstrap._gcd_import(name[level:], package, level)\n", " File \"\", line 1030, in _gcd_import\n", " File \"\", line 1007, in _find_and_load\n", " File \"\", line 986, in _find_and_load_unlocked\n", " File \"\", line 680, in _load_unlocked\n", " File \"\", line 850, in exec_module\n", " File \"\", line 228, in _call_with_frames_removed\n", " File \"/Users/rosed2/Documents/msk-mind/luna/pyluna-pathology/luna/pathology/cli/dsa_upload.py\", line 8, in \n", " from luna.pathology.dsa.dsa_api_handler import (\n", " File \"/Users/rosed2/Documents/msk-mind/luna/pyluna-pathology/luna/pathology/dsa/dsa_api_handler.py\", line 13, in \n", " import histomicstk\n", " File \"/Users/rosed2/Documents/msk-mind/env/lib/python3.9/site-packages/histomicstk/__init__.py\", line 7, in \n", " from . import segmentation # must be imported before features\n", " File \"/Users/rosed2/Documents/msk-mind/env/lib/python3.9/site-packages/histomicstk/segmentation/__init__.py\", line 15, in \n", " from . import label\n", " File \"/Users/rosed2/Documents/msk-mind/env/lib/python3.9/site-packages/histomicstk/segmentation/label/__init__.py\", line 15, in \n", " from .trace_object_boundaries import trace_object_boundaries\n", " File \"/Users/rosed2/Documents/msk-mind/env/lib/python3.9/site-packages/histomicstk/segmentation/label/trace_object_boundaries.py\", line 4, in \n", " from ._trace_object_boundaries_cython import _trace_object_boundaries_cython\n", " File \"_trace_object_boundaries_cython.pyx\", line 1, in init histomicstk.segmentation.label._trace_object_boundaries_cython\n", "ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject\n" ] } ], "source": [ "# push annotation to DSA\n", "!dsa_upload http://localhost:8080/dsa/api/v1 \\\n", "--collection_name tcga-data \\\n", "--image_filename 123.svs \\\n", "--annotation_filepath ../dsa_annotations/stardist_polygon/stardist_polygon_segmentations_123.json" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This is a screenshot form HistomicsUI, the high-magnification viewer. You can zoom in and view your annotation results with the desired opacity. As specified in `dsa_configs/stardist_polygon_config.yaml`, the red objects are classified as lymphocytes and the green cells are \"other\" cells.\n", "\n", "![Stardist Polygon Screenshot](../img/stardist-polygon-screenshot.png)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Upload Stardist cell detection results\n", "\n", "Here we use cellular detection results generated from Stardist. The x,y coordinates of the cells in the input TSV file will be visualized as a point, as opposed to a more complex polygon that we saw in the previous step with **stardist-polygon**. You'll notice that the point annotation is faster to upload compared to the polygon represenation of the cells.\n", "\n", "We also set fill color alpha value to 0 makes annotation upload faster." ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "2022-04-05 09:19:48,224 - INFO - root - FYI: Initalized logger, log file at: data-processing.log with handlers: [ (INFO)>, ]\n", "2022-04-05 09:19:48,225 - INFO - luna.common.utils - Running with {'output_dir': '../dsa_annotations/stardist_cell', 'annotation_name': 'stardist_cell_segmentations', 'image_filename': '123.svs', 'line_colors': '{\"Other\": \"rgb(0,255,0)\", \"Lymphocyte\": \"rgb(255,0,0)\"}', 'fill_colors': '{\"Other\": \"rgba(0,255,0,100)\", \"Lymphocyte\": \"rgba(255,0,0,100)\"}', 'input': '../dsa_input/test_object_detection.tsv', 'method_param_path': None}\n", "2022-04-05 09:19:48,225 - INFO - luna.common.utils - Param fill_colors set = {'Other': 'rgba(0,255,0,100)', 'Lymphocyte': 'rgba(255,0,0,100)'}\n", "2022-04-05 09:19:48,225 - INFO - luna.common.utils - Param line_colors set = {'Other': 'rgb(0,255,0)', 'Lymphocyte': 'rgb(255,0,0)'}\n", "2022-04-05 09:19:48,226 - INFO - luna.common.utils - Param annotation_name set = stardist_cell_segmentations\n", "2022-04-05 09:19:48,226 - INFO - luna.common.utils - Param image_filename set = 123.svs\n", "2022-04-05 09:19:48,226 - INFO - luna.common.utils - Param output_dir set = ../dsa_annotations/stardist_cell\n", "2022-04-05 09:19:48,226 - INFO - luna.common.utils - Param input set = ../dsa_input/test_object_detection.tsv\n", "2022-04-05 09:19:48,226 - INFO - luna.common.utils - Full segment key set: {}\n", "\n", "----------------------------------- Running transform::stardist_cell_main -----------------------------------\n", "\n", "2022-04-05 09:19:48,231 - INFO - luna.common.utils - Code block 'transform::stardist_cell_main' took: 0.004638240000000016s\n", "2022-04-05 09:19:48,232 - INFO - luna.common.utils - Done.\n" ] } ], "source": [ "# generate DSA annotation\n", "!dsa stardist-cell \\\n", "../dsa_input/test_object_detection.tsv \\\n", "--output_dir ../dsa_annotations/stardist_cell \\\n", "--annotation_name stardist_cell_segmentations \\\n", "--image_filename 123.svs \\\n", "--line_colors '{\"Other\": \"rgb(0,255,0)\", \"Lymphocyte\": \"rgb(255,0,0)\"}' \\\n", "--fill_colors '{\"Other\": \"rgba(0,255,0,100)\", \"Lymphocyte\": \"rgba(255,0,0,100)\"}'" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\"description\": \"\", \"elements\": [{\"fillColor\": \"rgba(0,255,0,100)\", \"lineColor\": \"rgb(0,255,0)\", \"lineWidth\": 2, \"type\": \"point\", \"center\": [61788.06625424067, 953.4224705647575, 0], \"label\": {\"value\": \"Other\"}}, {\"fillColor\": \"rgba(0,255,0,100)\", \"lineColor\": \"rgb(0,255,0)\", \"lineWidth\": 2, \"type\": \"point\", \"center\": [63033.92536419876, 957.214128916384, 0], \"label\": {\"value\": \"Other\"}}, {\"fillColor\": \"rgba(0,255,0,100)\", \"lineColor\": \"rgb(0,255,0)\", \"lineWidth\": 2, \"type\": \"point\", \"center\": [64150.069846338054, 957.3538215924966, 0], \"label\": {\"value\": \"Other\"}}], \"name\": \"stardist_cell_segmentations\"}" ] } ], "source": [ "# check json annotation\n", "!head ../dsa_annotations/stardist_cell/stardist_cell_segmentations_123.json" ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Successfully connected to DSA.\n", "collection_id_dict {'_accessLevel': 2, '_id': '60807410150bd39c9df579cf', '_modelType': 'collection', '_textScore': 15.75, 'created': '2021-04-21T18:50:56.730000+00:00', 'description': 'test images', 'meta': {}, 'name': 'test-path', 'public': True, 'publicFlags': [], 'size': 936444465, 'updated': '2021-04-21T18:50:56.730000+00:00'}\n", "Collection test-path found with id: 60807410150bd39c9df579cf\n", "Annotation successfully pushed to DSA.\n", "Time to push annotation 202.15096521377563\n", "http://localhost:8080/histomics#?image=60807ad8150bd39c9df579d2\n" ] } ], "source": [ "# push annotation to DSA\n", "!dsa_upload http://localhost:8080/dsa/api/v1 \\\n", "--collection_name tcga-data \\\n", "--image_filename 123.svs \\\n", "--annotation_filepath ../dsa_annotations/stardist_cell/stardist_cell_segmentations_123.json" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Below is another screenshot from HistomicsUI, from the link printed above. The results are the same as **stardist-polygon** visualization. Notice the cells are captured more minimally as circles, and not polygons. For rapid prototyping, **stardist-cell** offers faster annotation upload speed compared to **stardist-polygon**.\n", "\n", "![Stardist Cell Screenshot](../img/stardist-cell-screenshot.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Congratulations! Now you can visualize your annotations and results on DSA platform." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.3" } }, "nbformat": 4, "nbformat_minor": 4 }