{ "cells": [ { "cell_type": "markdown", "id": "heard-license", "metadata": {}, "source": [ "# Metabolic Networks with Escher" ] }, { "cell_type": "markdown", "id": "bizarre-spanish", "metadata": {}, "source": [ "[](https://colab.research.google.com/github/earmingol/scCellFie/blob/main/docs/source/notebooks/escher_viz.ipynb)" ] }, { "cell_type": "markdown", "id": "material-pontiac", "metadata": {}, "source": [ "In this tutorial, we will walk you through how to use [Escher](https://escher.github.io/) to contextualize metabolic activities with visualizations of metabolic networks.\n", "\n", "\"*Escher is a web-based tool for building, viewing, and sharing visualizations of metabolic pathways*\".\n", "\n", "You can learn more in its [paper](https://doi.org/10.1371/journal.pcbi.1004321) or [readthedocs](https://escher.readthedocs.io/). We strongly recommend this video of **[Escher in 3 minutes](https://youtu.be/qUipX-xzZjQ?si=UWOSsDV7Hc5t73Ea)** to learn how to use it." ] }, { "cell_type": "raw", "id": "flush-shuttle", "metadata": { "vscode": { "languageId": "raw" } }, "source": [ "" ] }, { "cell_type": "markdown", "id": "everyday-colors", "metadata": {}, "source": [ "## This tutorial includes following steps:\n", "* [Loading libraries](#loading-libraries)\n", "* [Loading endometrium results](#loading-endometrium-results)\n", "* [Loading Escher maps](#loading-escher-maps)\n", "* [Preparing Escher inputs](#preparing-escher-inputs)\n", "* [Online visualizations](#online-visualizations)" ] }, { "cell_type": "markdown", "id": "peripheral-penetration", "metadata": {}, "source": [ "## Loading libraries " ] }, { "cell_type": "code", "execution_count": 1, "id": "handmade-angle", "metadata": {}, "outputs": [], "source": [ "import json\n", "import sccellfie\n", "\n", "import pandas as pd\n", "import numpy as np" ] }, { "cell_type": "markdown", "id": "occupied-township", "metadata": {}, "source": [ "## Loading endometrium results " ] }, { "cell_type": "markdown", "id": "surrounded-ferry", "metadata": {}, "source": [ "We start opening the results previously generated and exported as shown in [this tutorial](https://sccellfie.readthedocs.io/en/latest/notebooks/quick_start_human.html#Save-single-cell-results).\n", "\n", "In this case, we will load the objects that were present in ``results['adata']`` in that tutorial. This object contains:\n", "- ``results['adata']``: contains gene expression in ``.X``.\n", "- ``results['adata'].layers['gene_scores']``: contains gene scores as in the original CellFie paper.\n", "- ``results['adata'].uns['Rxn-Max-Genes']``: contains determinant genes for each reaction per cell.\n", "- ``results['adata'].reactions``: contains reaction scores in ``.X`` so every scanpy function can be used on this object to visualize or compare values.\n", "- ``results['adata'].metabolic_tasks``: contains metabolic task scores in ``.X`` so every scanpy function can be used on this object to visualize or compare values.\n", "\n", "Here, we will name this object directly as ``adata``. Each of the previous elements should be under ``adata.``, as for example ``adata.reactions``." ] }, { "cell_type": "code", "execution_count": 2, "id": "junior-guatemala", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "./results//Human_HECA_scCellFie.h5ad was correctly loaded\n", "./results//Human_HECA_scCellFie_reactions.h5ad was correctly loaded\n", "./results//Human_HECA_scCellFie_metabolic_tasks.h5ad was correctly loaded\n" ] } ], "source": [ "adata = sccellfie.io.load_adata(folder='./results/',\n", " filename='Human_HECA_scCellFie'\n", " )" ] }, { "cell_type": "markdown", "id": "passing-mitchell", "metadata": {}, "source": [ "Escher can visualize metabolic networks covering different metabolic tasks. However, to represent the activity of these tasks, we need the activity of the distinct reactions that compose them. In this case, they can be found in the ``adata.reactions`` AnnData object." ] }, { "cell_type": "code", "execution_count": 3, "id": "round-yacht", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "AnnData object with n_obs × n_vars = 90001 × 748\n", " obs: 'n_genes', 'sample', 'percent_mito', 'n_counts', 'Endometriosis_stage', 'Endometriosis', 'Hormonal treatment', 'Binary Stage', 'Stage', 'phase', 'dataset', 'Age', 'lineage', 'celltype', 'label_long'\n", " uns: 'Binary Stage_colors', 'Biopsy_type_colors', 'Endometrial_pathology_colors', 'Endometriosis_stage_colors', 'GarciaAlonso_celltype_colors', 'Group_colors', 'Hormonal treatment_colors', 'Library_genotype_colors', 'Mareckova_celltype_colors', 'Mareckova_epi_celltype_colors', 'Mareckova_lineage_colors', 'Processing_colors', 'Rxn-Max-Genes', 'Symbol_colors', 'Tan_cellsubtypes_colors', 'Tan_celltype_colors', 'Treatment_colors', 'celltype_colors', 'dataset_colors', 'genotype_colors', 'hvg', 'label_long_colors', 'leiden', 'leiden_R_colors', 'leiden_colors', 'lineage_colors', 'neighbors', 'normalization', 'phase_colors', 'umap'\n", " obsm: 'X_scVI', 'X_umap'\n", " obsp: 'connectivities', 'distances'" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "adata.reactions" ] }, { "cell_type": "markdown", "id": "spoken-quilt", "metadata": {}, "source": [ "## Loading Escher maps " ] }, { "cell_type": "markdown", "id": "confident-statement", "metadata": {}, "source": [ "To visualize the distinct metabolic tasks, we need Escher maps that were previously created for this purpose. We provide maps for multiple tasks in [this link](https://github.com/earmingol/scCellFie/tree/main/escher_maps) where you can fin a series of ``json`` files containing the maps. These files can be downloaded into your local computer, which you will later use on the Escher website to visualize the metabolic activities. ***At the moment, these maps are available only for human datasets.***\n", "\n", "We start loading the Escher map that contains metabolic tasks associated with the metabolism of amino acids." ] }, { "cell_type": "code", "execution_count": 4, "id": "headed-dynamics", "metadata": {}, "outputs": [], "source": [ "with open('amino_acid_metabolism.json') as f:\n", " escher_map = json.load(f)" ] }, { "cell_type": "markdown", "id": "personalized-wound", "metadata": {}, "source": [ "We gather the reaction names contained in this map, which we will use later to filter our predictions with scCellFie." ] }, { "cell_type": "code", "execution_count": 5, "id": "french-steps", "metadata": {}, "outputs": [], "source": [ "rxns = [d['bigg_id'] for d in escher_map[1]['reactions'].values()]" ] }, { "cell_type": "code", "execution_count": 6, "id": "objective-formula", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "305" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "len(rxns)" ] }, { "cell_type": "markdown", "id": "developmental-discovery", "metadata": {}, "source": [ "## Preparing Escher inputs " ] }, { "cell_type": "markdown", "id": "asian-persian", "metadata": {}, "source": [ "Here, we show three different approaches to prepare the inputs for Escher. Depending on our interest, we can use any of them:\n", "\n", "1. We start with the reaction activities, directly calculated with scCellFie. \n", "2. Alternatively, we show how to use these activities in a scaled manner by using the whole human cell atlas as reference for these values to put in context what are low or high activities.\n", "3. Finally, we show how we can export values comparing conditions, where negative and positive values are associated with each of the compared conditions." ] }, { "cell_type": "markdown", "id": "greenhouse-lancaster", "metadata": {}, "source": [ "### Direct activity of reactions\n", "\n", "We can directly use the activity of the distinct reactions in the scCellFie's database by first aggregating the single cells into a cell type level. Here we use the [Tuckey's trimean](https://en.wikipedia.org/wiki/Trimean) to summarize each reaction activity per cell type." ] }, { "cell_type": "code", "execution_count": 7, "id": "organizational-mortality", "metadata": {}, "outputs": [], "source": [ "agg_rxns = sccellfie.expression.agg_expression_cells(adata=adata.reactions, groupby='celltype', agg_func='trimean')" ] }, { "cell_type": "markdown", "id": "useful-psychiatry", "metadata": {}, "source": [ "Then, we identify which reactions in our dataset are present in the Escher map that we previously loaded." ] }, { "cell_type": "code", "execution_count": 8, "id": "growing-taste", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "172" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "included_rxns = [rxn for rxn in rxns if rxn in agg_rxns.columns]\n", "len(included_rxns)" ] }, { "cell_type": "markdown", "id": "consecutive-steering", "metadata": {}, "source": [ "We can specify one cell type at the time to inspect its activities. Here we use the Luminal cells, which are a subtype of epithelial in the endometrium." ] }, { "cell_type": "code", "execution_count": 9, "id": "residential-memphis", "metadata": {}, "outputs": [], "source": [ "escher_data = agg_rxns.loc['Luminal', included_rxns].fillna(0).to_dict()" ] }, { "cell_type": "markdown", "id": "aboriginal-gentleman", "metadata": {}, "source": [ "We finally export this activity to later load it in the Escher website." ] }, { "cell_type": "code", "execution_count": 10, "id": "alike-rates", "metadata": {}, "outputs": [], "source": [ "with open(\"./results/escher_data_Luminal.json\", \"w\") as outfile:\n", " json.dump(escher_data, outfile, indent=4, sort_keys=False)" ] }, { "cell_type": "markdown", "id": "dental-chrome", "metadata": {}, "source": [ "### Scaled activity of reactions" ] }, { "cell_type": "markdown", "id": "informed-harris", "metadata": {}, "source": [ "Alternatively, we can scale the activity of each reactions by using the pre-computed min and max values across the whole CELLxGENE human cell atlas. We load these reference values from the GitHub hosting the scCellFie website." ] }, { "cell_type": "code", "execution_count": 11, "id": "opened-living", "metadata": {}, "outputs": [], "source": [ "minmax = pd.read_csv('https://raw.githubusercontent.com/ventolab/sccellfie-website/refs/heads/main/data/CELLxGENEReactionsMinMax.csv', index_col=0)\n", "\n", "# Subset reactions to only those that are present in our dataset\n", "minmax = minmax[agg_rxns.columns]" ] }, { "cell_type": "markdown", "id": "civil-medicare", "metadata": {}, "source": [ "Then, we take these min and max values to perform a minmax normalization to scale the values into a range between 0 and 1." ] }, { "cell_type": "code", "execution_count": 12, "id": "developed-bryan", "metadata": {}, "outputs": [], "source": [ "scaled_rxns = agg_rxns.subtract(minmax.loc['cell_type_min', :], axis='columns') / (minmax.loc['cell_type_max', :] - minmax.loc['cell_type_min', :])\n", "scaled_rxns = scaled_rxns.clip(lower=0., upper=1.)" ] }, { "cell_type": "markdown", "id": "durable-greensboro", "metadata": {}, "source": [ "Again, we then select one cell type, in this case Luminal cells." ] }, { "cell_type": "code", "execution_count": 13, "id": "vocational-alliance", "metadata": {}, "outputs": [], "source": [ "scaled_escher_data = scaled_rxns.loc['Luminal', included_rxns].fillna(0.).to_dict()" ] }, { "cell_type": "markdown", "id": "sapphire-vinyl", "metadata": {}, "source": [ "We finally export this activity to later load it in the Escher website." ] }, { "cell_type": "code", "execution_count": 14, "id": "italian-rider", "metadata": {}, "outputs": [], "source": [ "with open(\"./results/scaled_escher_data_Luminal.json\", \"w\") as outfile:\n", " json.dump(scaled_escher_data, outfile, indent=4, sort_keys=False)" ] }, { "cell_type": "markdown", "id": "historic-appointment", "metadata": {}, "source": [ "### Differential activity of reactions " ] }, { "cell_type": "markdown", "id": "molecular-weekend", "metadata": {}, "source": [ "If we are interested in identifying activities changing between conditions, we can perform a differential analysis at the reaction level (instead of the metabolic task level as shown in our [general overview tutorial](https://sccellfie.readthedocs.io/en/latest/notebooks/extended_quick_start.html)) and export the fold changes (or Cohen's D score) to easily distinguish activities associated with one condition or the other." ] }, { "cell_type": "markdown", "id": "thorough-superior", "metadata": {}, "source": [ "This dataset contains samples associated with endometriosis and control." ] }, { "cell_type": "code", "execution_count": 15, "id": "capital-packaging", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['Control', 'Endometriosis']\n", "Categories (2, object): ['Control', 'Endometriosis']" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "adata.obs.Endometriosis.unique()" ] }, { "cell_type": "markdown", "id": "distinct-encyclopedia", "metadata": {}, "source": [ "We define our conditions to compare, where the first one is used as reference. An indicate the column where this information is contained." ] }, { "cell_type": "code", "execution_count": 16, "id": "proprietary-channels", "metadata": {}, "outputs": [], "source": [ "contrasts = [('Control', 'Endometriosis')]\n", "condition_key = 'Endometriosis'" ] }, { "cell_type": "markdown", "id": "civil-diploma", "metadata": {}, "source": [ "For this tutorial, we will focus only on Luminal cells. So we filter our dataset to include only these cells, and contain only the reactions that are present in the Escher map we loaded." ] }, { "cell_type": "code", "execution_count": 17, "id": "apart-interest", "metadata": {}, "outputs": [], "source": [ "test_adata = adata.reactions.copy()\n", "test_adata = test_adata[test_adata.obs.celltype == 'Luminal', included_rxns]" ] }, { "cell_type": "markdown", "id": "charged-monroe", "metadata": {}, "source": [ "This results in a subset of ~ 3k Luminal cells, and 172 reactions." ] }, { "cell_type": "code", "execution_count": 18, "id": "chinese-governor", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(2942, 172)" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "test_adata.shape" ] }, { "cell_type": "markdown", "id": "balanced-revolution", "metadata": {}, "source": [ "Then we perform the differential analysis using the Wilcoxon test in scCellFie." ] }, { "cell_type": "code", "execution_count": 19, "id": "increased-aggregate", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Processing DE analysis: 0%| | 0/1 [00:00, ?it/s]/home/jovyan/my-conda-envs/single_cell/lib/python3.10/site-packages/sccellfie/stats/differential_analysis.py:39: RuntimeWarning: invalid value encountered in scalar divide\n", " d = (mean2 - mean1) / pooled_std\n", "Processing DE analysis: 100%|██████████| 1/1 [00:03<00:00, 3.17s/it]\n" ] } ], "source": [ "dma = sccellfie.stats.scanpy_differential_analysis(test_adata,\n", " cell_type=None, \n", " cell_type_key='celltype', \n", " condition_key=condition_key,\n", " min_cells=20,\n", " condition_pairs=contrasts)" ] }, { "cell_type": "markdown", "id": "regulation-terrorist", "metadata": {}, "source": [ "Which results in the following dataframe:" ] }, { "cell_type": "code", "execution_count": 20, "id": "rural-limit", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
| \n", " | cell_type | \n", "feature | \n", "group1 | \n", "group2 | \n", "log2FC | \n", "test_statistic | \n", "p_value | \n", "cohens_d | \n", "n_group1 | \n", "n_group2 | \n", "median_group1 | \n", "median_group2 | \n", "median_diff | \n", "adj_p_value | \n", "
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", "Luminal | \n", "TKT2 | \n", "Control | \n", "Endometriosis | \n", "1.488377 | \n", "11.502789 | \n", "1.277202e-30 | \n", "2.075280 | \n", "2871 | \n", "71 | \n", "0.4106477963907158 | \n", "1.4828520991382361 | \n", "1.072204 | \n", "1.098393e-28 | \n", "
| 1 | \n", "Luminal | \n", "TKT1 | \n", "Control | \n", "Endometriosis | \n", "1.488377 | \n", "11.502789 | \n", "1.277202e-30 | \n", "2.075280 | \n", "2871 | \n", "71 | \n", "0.4106477963907158 | \n", "1.4828520991382361 | \n", "1.072204 | \n", "1.098393e-28 | \n", "
| 2 | \n", "Luminal | \n", "GAPD | \n", "Control | \n", "Endometriosis | \n", "0.508739 | \n", "10.883880 | \n", "1.375795e-27 | \n", "1.714388 | \n", "2871 | \n", "71 | \n", "6.377502975157647 | \n", "9.10553057837871 | \n", "2.728028 | \n", "7.887890e-26 | \n", "
| 3 | \n", "Luminal | \n", "ENO | \n", "Control | \n", "Endometriosis | \n", "1.087755 | \n", "10.388724 | \n", "2.790609e-25 | \n", "1.678669 | \n", "2871 | \n", "71 | \n", "1.6954213931183895 | \n", "3.698023241259177 | \n", "2.002602 | \n", "1.199962e-23 | \n", "
| 4 | \n", "Luminal | \n", "LDH_Lm | \n", "Control | \n", "Endometriosis | \n", "0.678056 | \n", "10.307401 | \n", "6.524223e-25 | \n", "1.768239 | \n", "2871 | \n", "71 | \n", "1.2406285884712251 | \n", "2.103329791976264 | \n", "0.862701 | \n", "2.244333e-23 | \n", "