Tutorial

Introduction

This is a short python tutorial for using the panhumanpy package for hierarchical cell type annotation. To get started, we recommend that you install conda or an equivalent, and create a fresh conda environment wherein to install panhumanpy as follows:

# From the command line, to create and activate conda env:
# conda create -n panhumanpy_env python=3.9
# conda activate panhumanpy_env

# To install panhumanpy,
# For CPU based use:
# pip install panhumanpy
# or from GitHub:
# pip install git+https://github.com/satijalab/panhumanpy.git
# For GPU based use:
# pip install panhumanpy[gpu]
# or from GitHub:
# pip install git+https://github.com/satijalab/panhumanpy.git#egg=panhumanpy[gpu]

panhumanpy offers two interfaces to build a pipeline for cell type annotation and to obtain a low dimensional representation of the dataset for visualisation. The high-level interface provides quick and easy acess to inference using the Azimuth Neural Network, has defaults set to a recommended pipeline, and is more memory efficient. The low-level interface is more modular and allows for finer control and customisation of the workflow. To begin, import the panhumanpy package (and anndata, optional).

import panhumanpy as ph
import anndata as ad       #optional

By default, the panhumanpy API points to a model version consistent with the package major version. For example for panhumanpy 0.2.1 (Andromeda), the default model version is ‘v0’. However, the user can choose to specify a different version.

model_version = different_model_version      #optional

Note that the model weights are saved on zenodo (refer to README for details) and downloaded to cache at first usage.

High-level interface

The high-level interface of the panhumanpy package is through the AzimuthNN class. We can pass either an h5ad object, or an anndata object read from the h5ad object to the AzimuthNN object for annotation. The annotation pipeline through the high-level interface employs a minibatched protocol under the hood which enables greater memory efficiency, and follows a recommended set of options and parameters by default for ease of use.

# file path to query h5ad
query_path = "path/to/h5ad"

# optional
query = ad.read_h5ad(query_path)

# For documentation
help(ph.AzimuthNN)

Simplified workflow

#azimuth = ph.AzimuthNN(query_path)     # if passing the filepath
azimuth = ph.AzimuthNN(query, model_version = model_version)              # model_version arg is optional
embeddings = azimuth.azimuth_embed()
umap = azimuth.azimuth_umap()
cell_metadata = azimuth.cells_meta

In the above, cell_metadata is a pandas dataframe with the annotations from the Azimuth Neural Network. Some of the important columns are as follows: -‘full_hierarchical_labels’: The complete cell type label with all hierarchical levels. -‘level_zero_labels’: Cell type labels at the lowest resolution. -‘final_level_labels’: Cell type labels at the highest resolution, note that this label comes from different levels for different cells as the maximum hierarchical depth is not uniform across all cells. -‘final_level_softmax_prob’: The confidence values for the predicted final level labels. Note: ‘final_level_softmax_prob’ is being renamed to ‘final_level_confidence’ for the ongoing updates. Note: ‘level_zero_labels’ is replaced by ‘azimuth_broad’ by default, more on that in the section on ‘Label refinement’.

Digging a little deeper

Note that in the above we’ve assumed that the index column of the genes metadata in the anndata object ie query.var consists of gene names. However that is not always the case. In these cases, you need to inspect query.var and select the correct column name where the gene names are stored, and specify this column name when you instantiate the AzimuthNN object. Quite often, gene names are stored in a column called “gene_symbol” or in a column called “feature_name”.

With these options, you would instantiate the AzimuthNN object as follows:

azimuth = ph.AzimuthNN(query, feature_names_col = "gene_symbol")

Label refinement

We offer you a built-in post-processing step on the hierarchical annotations provided by the Azimuth Neural Network which you can use to obtain annotations with consistent granularity across tissues at three levels for each cell. These refined annotations ‘azimuth_broad’ - equivalent to ‘level_zero_labels’, ‘azimuth_medium’, ‘azimuth_fine’, are added to the metadata dataframe, and lend themselves to easy interpretation. Refinement at all three levels is included in the defaults for version 0.3.0. If you wish to turn off label refinement, you can set the ‘refine’ parameter to None or False. In this case, the broad cell type labels are found at ‘level_zero_labels’ as specified above. If you want refinement at, say, only the medium level, you can specify that during initialization as follows:

azimuth = ph.AzimuthNN(query, feature_names_col = "gene_symbol", refine = ['medium'])

Post-processing for ‘azimuth_medium’ requires ‘azimuth_broad’ as a prerequisite, and the corresponding column is applied to the metadata as well. Note that in versions pre-dating 0.3.0, refinement was defined as a separate method as follows (which is now simply a trivial pass) and not included as a default step in the initialization of the class:

azimuth.azimuth_refine()
cell_metadata = azimuth.cells_meta

Saving the annotated object

Finally, you can pack the annotations, and any embeddings and umaps created into an annotated object. You can optionally also save the annotated object at a specified filepath. In case a file with the filename exists already, a datetime stamp is added to the filename.

annotated_query = azimuth.pack_adata(save_path="path/to/save/h5ad")

Cell Ontology mapping

You can map any annotation column in the cell metadata to Cell Ontology terms using the map_to_cell_ontology method. This adds a new column immediately after the source column with the suffix _CL. Optionally, you can also add the CL identifier (e.g. ‘CL:0000236’) by setting include_cl_id=True, which adds a further column with the suffix _CL_ID. Labels that cannot be mapped are set to ‘unmapped’, and a single warning is emitted listing all unmapped labels.

# Map a refined annotation column to CL terms
azimuth.map_to_cell_ontology('azimuth_fine')

# Map with CL IDs included
azimuth.map_to_cell_ontology('azimuth_broad', include_cl_id=True)

# Then save as before
annotated_query = azimuth.pack_adata(save_path="path/to/save/h5ad")

Low-level flexible interface

This is intended for low-level interactive usage of the Azimuth Neural Network annotation pipeline. This class provides a comprehensive framework for single-cell RNA-seq annotation using neural network models. It handles the complete workflow from data loading and preprocessing to inference, post-processing, and result visualization. It can be used to create memory-efficient and scalable pipelines for atlas-scale annotation, and also for more exploratory analysis of the annotation process. We shall not be detailing all possible options available in this case, and shall only provide a minimal set of steps that can reproduce the results of the previous section. The user is encouraged to read the documentation and the source-code, and explore all the attributes and the methods in the class.

# For documentation
help(ph.AzimuthNN_base)

azimuth = ph.AzimuthNN_base(model_version=model_version)      # model_version arg is optional
# Note that this class is not instantiated with a query, the query is only passed to it later.

# Pass a query in the form of an anndata object, an h5ad file, or the components thereof separately
# You can optionally specify the column in query.var with the gene names just like in the previous section
azimuth.query_adata(query)

# Inference
azimuth.process_query()
azimuth.run_inference_model()
# If using model_version = 'v1', we recommend adding the confidence calibration step here
# _ = azimuth.calibrate_predictions()
_ = azimuth.process_outputs()

# (Optional) refinement of labels for consistent granularity and ease of interpretation
_ = azimuth.refine_labels(refine_level = 'broad')
_ = azimuth.refine_labels(refine_level = 'medium')
_ = azimuth.refine_labels(refine_level = 'fine')

# Update metadata with annotations and read the updated cell metadata
azimuth.update_cells_meta()
cell_metadata = azimuth.cells_meta

# (Optional) Map annotation column to Cell Ontology terms
azimuth.map_to_cell_ontology('azimuth_fine')
azimuth.map_to_cell_ontology('azimuth_broad', include_cl_id=True)

# For Azimuth NN embeddings and the corresponding umap
embeddings = azimuth.inference_model_embeddings(embedding_layer_name = 'dense_3')
umap = azimuth.inference_model_umaps(embedding_layer_name='dense_3')

# To pack metadata updated with annotations, and any low dimensional representations computed into an anndata obj
annotated_query = azimuth.pack_adata()

Direct use of map_to_cell_ontology

The map_to_cell_ontology function in panhumanpy.ANNotate_tools can also be used independently of the annotation pipeline — for example to map an existing annotated h5ad or metadata file, or to work directly with a list of labels.

from panhumanpy.ANNotate_tools import map_to_cell_ontology

# Apply to a list of labels directly — returns a list
cl_labels = map_to_cell_ontology(['B cell', 'T cell', 'NK cell'])

# With CL IDs — returns a tuple (cl_labels, cl_ids)
cl_labels, cl_ids = map_to_cell_ontology(
    ['B cell', 'T cell'],
    include_cl_id=True
)

# Apply to a pandas DataFrame (e.g. the obs from an existing AnnData)
# Returns the DataFrame with a new column added immediately after src_col
df = map_to_cell_ontology(df, src_col='cell_type')

# Apply to an AnnData object directly
# Returns the AnnData with the new column added to .obs
adata = map_to_cell_ontology(adata, src_col='cell_type')

# Apply to an h5ad file on disk
# Reads, annotates, writes a new file with '_CL' appended to the filename
# e.g. my_data.h5ad -> my_data_CL.h5ad
adata = map_to_cell_ontology('my_data.h5ad', src_col='cell_type')

# Apply to a CSV metadata file on disk
# e.g. my_metadata.csv -> my_metadata_CL.csv
df = map_to_cell_ontology('my_metadata.csv', src_col='cell_type')

# Optionally specify a model version explicitly (defaults to the package default)
cl_labels = map_to_cell_ontology(['B cell'], model_version='v1')