Developer Guide#

A comprehensive guide for those interested in making contributions or supporting the development of Luna.

Warning

This documentation is still a work in progress

Development Setup Instructions#

Follow the instructions below to install the latest version of Luna and setup a development environment.

Development Installation#

Developers interested in using the newest features of Luna, or are actively contributing to Luna should install the latest version available via:

git clone git@github.com:msk-mind/luna.git

Download the dependencies with:

pip install --upgrade pip
pip install numpy
cd luna
pip install -r requirements_dev.txt

For development and testing purposes, add the subpackages to your PYTHONPATH:

export PYTHONPATH=.:pyluna-core:pyluna-common:pyluna-radiology:pyluna-pathology

OR use``setup_local.sh`` to setup your python paths and LUNA_HOME config:

source setup_local.sh

Development with Docker#

luna-dev docker image is available on DockerHub.

This docker image includes the pre-requisites and python dependencies. This is primarily used for circleci testing at the moment, but can be extended based on your development needs.

Once we have a stable release, docker images that includes pyluna packages can be made available.

Documentation Generation#

API documentation in Luna is generated via sphinx-autodoc. Docstrings are written according to the Google Python Style Guide.

In order to generate the documentation:

cd docs
# generate module docs
sphinx-apidoc --implicit-namespaces -o ./common ../pyluna-common ../pyluna-common/tests ../pyluna-common/setup*
sphinx-apidoc --implicit-namespaces -o ./pathology ../pyluna-pathology ../pyluna-pathology/tests ../pyluna-pathology/setup*

# clean up duplicate modules
# custom common/common-modules.rst pathology/pathology-modules.rst are used.
rm common/modules.rst pathology/modules.rst
rm common/pyluna-common.rst pathology/pyluna-pathology.rst

# generate html
make html

# clean _build directory
make clean

Contributing#

Thank you for your interest in contributing to Luna!

This document is a quick guide to contributing.

Creating a new issue#

  • Enhancement For new feature requests, create an issue with enhancement tag and describe the feature and its behavior in detail.

  • Bug Describe the error, provide logs and code version. Apply bug tag.

  • Question To ask a question or provide a suggestion, open an issues with question tag.

Setting up local environment#

Refer to the development setup guide to:

  • Get the latest code from git

  • Add Luna packages to $PYTHONPATH

  • Setup $LUNA_HOME environment variable

Code Development#

We follow the git branching workflow.

Create your branch with the naming convention IssueNumber-Description (e.g. 123-annotation) from the default branch.

Development guide#

  1. Use Google docstring format to clearly document all classes, methods, variables. In PyCharm, you can change the settting in Settings > Tools > Python Integrated Tools > Docstring format

  2. Setup pre-commit Python linter for uniform code styles. In the cloned repo, run pre-commit install. When you attempt to make a commit, black will reformat your code and flake8 will check for PEP8 compliance. Once these tests pass, you can re-add and commit properly formatted files.

  3. Follow clean code principles

  4. Add new dependencies to requirements_dev.txt, and setup.cfg in the appropriate packages. The circleci tests depend on the docker image defined in docker/Dockerfile. To update the docker, you need to trigger the docker workflow in circleci. To trigger this workflow using circleci API, first create a circleci personal token. In your terminal, send the POST request to the circlcie API with your branch, circle-token.:

    curl --request POST \
--url https://circleci.com/api/v2/project/gh/msk-mind/luna/pipeline \
--header 'Circle-Token: <REPLACE_WITH_YOUR_CIRCLECI_TOKEN>' \
--header 'content-type: application/json' \
--data '{"branch": "<REPLACE_WITH_YOUR_BRANCH>", "parameters" : {"run_workflow_test": false, "run_workflow_docker":  true }}'

  5. Aim for 85% test coverage for new modules and ensure these tests pass.

  6. Apply meaningful log statements.

  7. Check that no sensitive information (credentials, hostnames, author info) is accidentally committed.

  8. Add or update the tutorials in the documentation site. Refer to Sphinx documenation generation guide

Unit tests#

Luna uses pytest for testing. Each code contribution should have appropriate test coverage, as per the development guide. Tests should follow the code package structure and any data required for testing should be added under the same package under testdata directory.

To run all tests locally, run

make test

To run test coverage, run

make coverage

Individual tests can be run by specifying the test file and method.

pytest -s path/to/test_file.py::test_method

Pull Requests (PR)#

Creating PR#

Once you are done (see development guide) with the changes in your branch, create a PR to merge your branch into the default dev branch.

  1. Luna uses semantic release for version management. The PR title must start with one of these types specified in the commit guidelines. Additionally, provide Luna subpackage and a brief description of the PR in the title. For example, perf: pathology – improve inference speed

  2. Add a description of the changes - list of changes, screenshots of functional tests, and any notes. This summary will help the reviewer engage in a more meaningful conversation with the author.

  3. Link the PR to an issue with Connect issue.

  4. Ensure all checks pass.

  5. Add at least one person from the core development team as a reviewer.

Reviewing PR#

If you are assigned as a reviewer, add your comments based on these guidelines.

  1. Check that the PR follows the development guide.

  2. Is there any code/documentation update that is missed in the PR?

We encourage the author and reviewer to engage in active conversations to ensure all suggestions are clearly understood and to improve the code quality of the overall library.

Merging PR#

PRs should be merged by the author who created the PR. In order to merge a PR to dev branch, the PR should have

  • All checks (circleci tests, readthedocs) passed

  • At least 1 approval from a reviewer

  • No merge conflicts

Once all the requirements are met, double check that the PR title adheres to the commit guidelines.

Squash and merge your PR. This will add your branch as 1 commit in the dev branch.

Once the PR is merged, delete your branch.

Additional Resources#

Release#

Branches#

  • dev: default branch. All PRs are merged to dev branch

  • master: release branch. Always stable

  • other branches - created from dev branch. Contributors add and test their features, bug fixes etc.

NOTE: when merging dev to master branch for a release, use “Create a Merge commit” to show the PRs and commits. Do not use squash and merge, as it will only make the release commit available for the semantic release parsing.

Release workflow#

Release is automated via Github actions from the master branch. During the release:

  • Semantic release determines the new version and updates the version

  • Github tag/release is created with specification fo the new changes.

  • Pyluna and its subpackages are pushed to pypi.

Technical Architecture#

Here we provide an overview of the Luna technical architecture and design methodologies.

Resources#

Here are some links to useful developer resources: