{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Inference and Visualization Tutorial" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Welcome to the inference and visualization notebook! At this point, you should have a trained model and tiles to run inference. In this notebook we will run inference on a slide and visualize the results. Here are the steps we will review:\n", "\n", "- Run inference with a trained model.\n", "- Visualize the inference results\n", " - (optional) Add the test slide to Digital Slide Archive (DSA)\n", " - Generate heatmaps\n", " - View the heatmap thumbnails" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Run inference with a trained model" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Often tissue-based analysis on whole slide images benefit from annotations provided by expert pathologists. However, having pathologists annotate 1000s of slides is very time consuming and expensive. To overcome this bottleneck, it is common to have pathologist annotate a subset of the slides, and use that dataset to train a model. This model is then used to label the rest of the dataset.\n", "\n", "In the model training notebook, we trained a ResNet-18 model on a subset of our slides with the annotated regions and labels. We will now use this trained model and the prepared tiles from the test slide to run the inference step." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Usage: infer_tiles [OPTIONS]\n", "\n", "Options:\n", " -a, --app_config TEXT application configuration yaml file. See\n", " config.yaml.template for details. [required]\n", "\n", " -s, --datastore_id TEXT datastore name. usually a slide id.\n", " [required]\n", "\n", " -m, --method_param_path TEXT json file with method parameters for loading a\n", " saved model. [required]\n", "\n", " --help Show this message and exit.\n" ] } ], "source": [ "%%bash\n", "\n", "infer_tiles --help" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**infer_tiles** CLI takes in details on your trained model, and loads the tiles data for inference." ] }, { "cell_type": "code", "execution_count": 39, "metadata": {}, "outputs": [], "source": [ "%%bash\n", "\n", "infer_tiles -a configs/app_config.yaml -s 2551129 -m configs/infer_tiles.yaml" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The output of the inference is saved in a CSV. Let's take a look at the results." ] }, { "cell_type": "code", "execution_count": 40, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "total 512\n", "-rwxrwxrwx 1 pashaa pashaa 3.3M Jul 14 08:59 tile_scores_and_labels_pytorch_inference.csv\n", "-rwxrwxrwx 1 pashaa pashaa 439 Jul 14 08:59 metadata.json\n" ] } ], "source": [ "%%bash\n", "\n", "ls -lhtr PRO_12-123/tiles/2551129/infer_ov_clf/TileScores/data/" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For each tile, we populate:\n", "\n", "- *model_score* - the most probable label\n", "- *tumor_score* - the probability of the tile being classified as tumor\n", "- *label_score* - the probability of the model score label\n", "\n", "Note that for the purpose of the tutorials, we are working with a small toy dataset, and therefore the inference result is not optimal. This result is for demonstration purposes only.\n" ] }, { "cell_type": "code", "execution_count": 41, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
| \n", " | Unnamed: 0 | \n", "address | \n", "coordinates | \n", "otsu_score | \n", "purple_score | \n", "tile_image_offset | \n", "tile_image_length | \n", "tile_image_size_xy | \n", "tile_image_mode | \n", "model_score | \n", "tumor_score | \n", "label_score | \n", "
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", "0 | \n", "x1_y1_z20 | \n", "(1, 1) | \n", "1.0000 | \n", "0.0 | \n", "0.000000e+00 | \n", "49152.0 | \n", "128.0 | \n", "RGB | \n", "Label-0 | \n", "0.673368 | \n", "0.673368 | \n", "
| 1 | \n", "1 | \n", "x1_y2_z20 | \n", "(1, 2) | \n", "1.0000 | \n", "0.0 | \n", "4.915200e+04 | \n", "49152.0 | \n", "128.0 | \n", "RGB | \n", "Label-0 | \n", "0.939261 | \n", "0.939261 | \n", "
| 2 | \n", "2 | \n", "x2_y1_z20 | \n", "(2, 1) | \n", "1.0000 | \n", "0.0 | \n", "9.830400e+04 | \n", "49152.0 | \n", "128.0 | \n", "RGB | \n", "Label-0 | \n", "0.675424 | \n", "0.675424 | \n", "
| 3 | \n", "3 | \n", "x2_y2_z20 | \n", "(2, 2) | \n", "1.0000 | \n", "0.0 | \n", "1.474560e+05 | \n", "49152.0 | \n", "128.0 | \n", "RGB | \n", "Label-0 | \n", "0.954000 | \n", "0.954000 | \n", "
| 4 | \n", "4 | \n", "x3_y1_z20 | \n", "(3, 1) | \n", "1.0000 | \n", "0.0 | \n", "1.966080e+05 | \n", "49152.0 | \n", "128.0 | \n", "RGB | \n", "Label-0 | \n", "0.688574 | \n", "0.688574 | \n", "
| ... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "
| 28750 | \n", "28750 | \n", "x636_y2_z20 | \n", "(636, 2) | \n", "0.8750 | \n", "0.0 | \n", "1.413120e+09 | \n", "49152.0 | \n", "128.0 | \n", "RGB | \n", "Label-0 | \n", "0.849509 | \n", "0.849509 | \n", "
| 28751 | \n", "28751 | \n", "x636_y3_z20 | \n", "(636, 3) | \n", "0.8125 | \n", "0.0 | \n", "1.413169e+09 | \n", "49152.0 | \n", "128.0 | \n", "RGB | \n", "Label-0 | \n", "0.944446 | \n", "0.944446 | \n", "
| 28752 | \n", "28752 | \n", "x637_y1_z20 | \n", "(637, 1) | \n", "1.0000 | \n", "0.0 | \n", "1.413218e+09 | \n", "49152.0 | \n", "128.0 | \n", "RGB | \n", "Label-0 | \n", "0.672313 | \n", "0.672313 | \n", "
| 28753 | \n", "28753 | \n", "x637_y2_z20 | \n", "(637, 2) | \n", "0.9375 | \n", "0.0 | \n", "1.413267e+09 | \n", "49152.0 | \n", "128.0 | \n", "RGB | \n", "Label-0 | \n", "0.788589 | \n", "0.788589 | \n", "
| 28754 | \n", "28754 | \n", "x637_y3_z20 | \n", "(637, 3) | \n", "0.8750 | \n", "0.0 | \n", "1.413317e+09 | \n", "49152.0 | \n", "128.0 | \n", "RGB | \n", "Label-0 | \n", "0.933061 | \n", "0.933061 | \n", "
28755 rows × 12 columns
\n", "