dbgap2x.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Using dbgap2x, R package to explore, download and decrypt phenotypic and genomic data from dbGaP\n",
    "## Introduction\n",
    "### Load the package"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "#devtools::install_github(\"gversmee/dbgap2x\", force = TRUE)\n",
    "library(dbgap2x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get the list of the function for this new package"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "browse.dbgap : function (phs, no.browser = FALSE)  \n",
       "browse.study : function (phs, no.browser = FALSE)  \n",
       "consent.groups : function (phs)  \n",
       "datatables.dict : function (phs)  \n",
       "dbgap.data_dict : function (xml, dest)  \n",
       "dbgap.decrypt : function (files, key = FALSE)  \n",
       "dbgap.download : function (krt, key = FALSE)  \n",
       "is.parent : function (phs)  \n",
       "n.pop : function (phs, consentgroups = TRUE, gender = TRUE)  \n",
       "n.tables : function (phs)  \n",
       "n.variables : function (...)  \n",
       "parent.study : function (phs)  \n",
       "phs.version : function (phs)  \n",
       "search.dbgap : function (term, no.browser = FALSE)  \n",
       "study.name : function (phs)  \n",
       "sub.study : function (phs)  \n",
       "variables.dict : function (phs)  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "lsf.str(\"package:dbgap2x\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Search for dbGaP studies\n",
    "### Let's try to explore the \"Jackson Heart Study\" cohort that exists on dbGaP.\n",
    "##### We created the function \"browse.dbgap\", which helps you to find the studies related to the term that you search for."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "https://www.ncbi.nlm.nih.gov/gap/?term=Jackson%5BStudy+Name%5D \n",
      "                          \n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<thead><tr><th scope=col>Study ID</th><th scope=col>Study Name</th><th scope=col>Release Date</th><th scope=col>Nb Participants</th><th scope=col>Study Design</th><th scope=col>Project</th><th scope=col>Diseases</th><th scope=col>Ancestor ID</th><th scope=col>Ancestor Name</th><th scope=col>Molecular Data Type</th><th scope=col>Tumor Type</th><th scope=col>UID</th></tr></thead>\n",
       "<tbody>\n",
       "\t<tr><td>phs001356.v1.p2                                         </td><td>Exome Chip Genotyping: The Jackson Heart Study          </td><td> 2019-05-10                                             </td><td>2788                                                    </td><td>Prospective Longitudinal Cohort                         </td><td>National Heart, Lung, Blood Institute                   </td><td>Cardiovascular Diseases, Hypertension, Diabetes Mellitus</td><td>phs000286.v6.p2                                         </td><td>The Jackson Heart Study (JHS)                           </td><td>SNP Genotypes (Array)                                   </td><td>germline                                                </td><td>1692088                                                 </td></tr>\n",
       "\t<tr><td>phs001098.v2.p2                                                   </td><td>T2D-GENES Multi-Ethnic Exome Sequencing Study: Jackson Heart Study</td><td> 2019-05-10                                                       </td><td>1029                                                              </td><td>Case-Control                                                      </td><td>NHLBI GO-ESP                                                      </td><td>Diabetes Mellitus, Type 2                                         </td><td>phs000286.v6.p2                                                   </td><td>The Jackson Heart Study (JHS)                                     </td><td>SNP/CNV Genotypes (NGS), WXS                                      </td><td>germline                                                          </td><td>1597258                                                           </td></tr>\n",
       "\t<tr><td>phs000499.v4.p2                                                     </td><td>NHLBI Jackson Heart Study Candidate Gene Association Resource (CARe)</td><td> 2019-05-10                                                         </td><td>3352                                                                </td><td>Prospective Longitudinal Cohort                                     </td><td>NHLBI CARe                                                          </td><td>Longitudinal Studies                                                </td><td>phs000286.v6.p2                                                     </td><td>The Jackson Heart Study (JHS)                                       </td><td>SNP Genotypes (Array)                                               </td><td>germline, unspecified                                               </td><td>1597257                                                             </td></tr>\n",
       "\t<tr><td>phs000498.v4.p2                             </td><td>Jackson Heart Study Allelic Spectrum Project</td><td> 2019-05-10                                 </td><td>1983                                        </td><td>Prospective Longitudinal Cohort             </td><td>National Heart, Lung, Blood Institute       </td><td>Cardiovascular Diseases                     </td><td>phs000286.v6.p2                             </td><td>The Jackson Heart Study (JHS)               </td><td>SNP Genotypes (NGS), WXS                    </td><td>germline                                    </td><td>1597256                                     </td></tr>\n",
       "\t<tr><td>phs000286.v6.p2                                                                                                                                                                                       </td><td>The Jackson Heart Study (JHS)                                                                                                                                                                         </td><td> 2019-05-10                                                                                                                                                                                           </td><td>3889                                                                                                                                                                                                  </td><td>Prospective Longitudinal Cohort                                                                                                                                                                       </td><td>National Heart, Lung, Blood Institute, NHLBI GO-ESP, NHLBI CARe                                                                                                                                       </td><td>Cardiovascular Diseases, Coronary Artery Disease, Diabetes Mellitus, Type 2, Obesity, Hypertension, Kidney Failure, Chronic, Stroke, Heart Failure, Peripheral Vascular Diseases, Arrhythmias, Cardiac</td><td>                                                                                                                                                                                                      </td><td>                                                                                                                                                                                                      </td><td>                                                                                                                                                                                                      </td><td>germline, unspecified                                                                                                                                                                                 </td><td>1597254                                                                                                                                                                                               </td></tr>\n",
       "\t<tr><td>phs000964.v3.p1                                         </td><td>NHLBI TOPMed: The Jackson Heart Study                   </td><td> 2018-05-18                                             </td><td>3596                                                    </td><td>Prospective Longitudinal Cohort                         </td><td>National Human Genome Research Institute                </td><td>Cardiovascular Diseases, Hypertension, Diabetes Mellitus</td><td>                                                        </td><td>                                                        </td><td>SNP/CNV Genotypes (NGS), WGS                            </td><td>germline                                                </td><td>1768620                                                 </td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "\\begin{tabular}{r|llllllllllll}\n",
       " Study ID & Study Name & Release Date & Nb Participants & Study Design & Project & Diseases & Ancestor ID & Ancestor Name & Molecular Data Type & Tumor Type & UID\\\\\n",
       "\\hline\n",
       "\t phs001356.v1.p2                                          & Exome Chip Genotyping: The Jackson Heart Study           &  2019-05-10                                              & 2788                                                     & Prospective Longitudinal Cohort                          & National Heart, Lung, Blood Institute                    & Cardiovascular Diseases, Hypertension, Diabetes Mellitus & phs000286.v6.p2                                          & The Jackson Heart Study (JHS)                            & SNP Genotypes (Array)                                    & germline                                                 & 1692088                                                 \\\\\n",
       "\t phs001098.v2.p2                                                    & T2D-GENES Multi-Ethnic Exome Sequencing Study: Jackson Heart Study &  2019-05-10                                                        & 1029                                                               & Case-Control                                                       & NHLBI GO-ESP                                                       & Diabetes Mellitus, Type 2                                          & phs000286.v6.p2                                                    & The Jackson Heart Study (JHS)                                      & SNP/CNV Genotypes (NGS), WXS                                       & germline                                                           & 1597258                                                           \\\\\n",
       "\t phs000499.v4.p2                                                      & NHLBI Jackson Heart Study Candidate Gene Association Resource (CARe) &  2019-05-10                                                          & 3352                                                                 & Prospective Longitudinal Cohort                                      & NHLBI CARe                                                           & Longitudinal Studies                                                 & phs000286.v6.p2                                                      & The Jackson Heart Study (JHS)                                        & SNP Genotypes (Array)                                                & germline, unspecified                                                & 1597257                                                             \\\\\n",
       "\t phs000498.v4.p2                              & Jackson Heart Study Allelic Spectrum Project &  2019-05-10                                  & 1983                                         & Prospective Longitudinal Cohort              & National Heart, Lung, Blood Institute        & Cardiovascular Diseases                      & phs000286.v6.p2                              & The Jackson Heart Study (JHS)                & SNP Genotypes (NGS), WXS                     & germline                                     & 1597256                                     \\\\\n",
       "\t phs000286.v6.p2                                                                                                                                                                                        & The Jackson Heart Study (JHS)                                                                                                                                                                          &  2019-05-10                                                                                                                                                                                            & 3889                                                                                                                                                                                                   & Prospective Longitudinal Cohort                                                                                                                                                                        & National Heart, Lung, Blood Institute, NHLBI GO-ESP, NHLBI CARe                                                                                                                                        & Cardiovascular Diseases, Coronary Artery Disease, Diabetes Mellitus, Type 2, Obesity, Hypertension, Kidney Failure, Chronic, Stroke, Heart Failure, Peripheral Vascular Diseases, Arrhythmias, Cardiac &                                                                                                                                                                                                        &                                                                                                                                                                                                        &                                                                                                                                                                                                        & germline, unspecified                                                                                                                                                                                  & 1597254                                                                                                                                                                                               \\\\\n",
       "\t phs000964.v3.p1                                          & NHLBI TOPMed: The Jackson Heart Study                    &  2018-05-18                                              & 3596                                                     & Prospective Longitudinal Cohort                          & National Human Genome Research Institute                 & Cardiovascular Diseases, Hypertension, Diabetes Mellitus &                                                          &                                                          & SNP/CNV Genotypes (NGS), WGS                             & germline                                                 & 1768620                                                 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "Study ID | Study Name | Release Date | Nb Participants | Study Design | Project | Diseases | Ancestor ID | Ancestor Name | Molecular Data Type | Tumor Type | UID | \n",
       "|---|---|---|---|---|---|\n",
       "| phs001356.v1.p2                                          | Exome Chip Genotyping: The Jackson Heart Study           |  2019-05-10                                              | 2788                                                     | Prospective Longitudinal Cohort                          | National Heart, Lung, Blood Institute                    | Cardiovascular Diseases, Hypertension, Diabetes Mellitus | phs000286.v6.p2                                          | The Jackson Heart Study (JHS)                            | SNP Genotypes (Array)                                    | germline                                                 | 1692088                                                  | \n",
       "| phs001098.v2.p2                                                    | T2D-GENES Multi-Ethnic Exome Sequencing Study: Jackson Heart Study |  2019-05-10                                                        | 1029                                                               | Case-Control                                                       | NHLBI GO-ESP                                                       | Diabetes Mellitus, Type 2                                          | phs000286.v6.p2                                                    | The Jackson Heart Study (JHS)                                      | SNP/CNV Genotypes (NGS), WXS                                       | germline                                                           | 1597258                                                            | \n",
       "| phs000499.v4.p2                                                      | NHLBI Jackson Heart Study Candidate Gene Association Resource (CARe) |  2019-05-10                                                          | 3352                                                                 | Prospective Longitudinal Cohort                                      | NHLBI CARe                                                           | Longitudinal Studies                                                 | phs000286.v6.p2                                                      | The Jackson Heart Study (JHS)                                        | SNP Genotypes (Array)                                                | germline, unspecified                                                | 1597257                                                              | \n",
       "| phs000498.v4.p2                              | Jackson Heart Study Allelic Spectrum Project |  2019-05-10                                  | 1983                                         | Prospective Longitudinal Cohort              | National Heart, Lung, Blood Institute        | Cardiovascular Diseases                      | phs000286.v6.p2                              | The Jackson Heart Study (JHS)                | SNP Genotypes (NGS), WXS                     | germline                                     | 1597256                                      | \n",
       "| phs000286.v6.p2                                                                                                                                                                                        | The Jackson Heart Study (JHS)                                                                                                                                                                          |  2019-05-10                                                                                                                                                                                            | 3889                                                                                                                                                                                                   | Prospective Longitudinal Cohort                                                                                                                                                                        | National Heart, Lung, Blood Institute, NHLBI GO-ESP, NHLBI CARe                                                                                                                                        | Cardiovascular Diseases, Coronary Artery Disease, Diabetes Mellitus, Type 2, Obesity, Hypertension, Kidney Failure, Chronic, Stroke, Heart Failure, Peripheral Vascular Diseases, Arrhythmias, Cardiac |                                                                                                                                                                                                        |                                                                                                                                                                                                        |                                                                                                                                                                                                        | germline, unspecified                                                                                                                                                                                  | 1597254                                                                                                                                                                                                | \n",
       "| phs000964.v3.p1                                          | NHLBI TOPMed: The Jackson Heart Study                    |  2018-05-18                                              | 3596                                                     | Prospective Longitudinal Cohort                          | National Human Genome Research Institute                 | Cardiovascular Diseases, Hypertension, Diabetes Mellitus |                                                          |                                                          | SNP/CNV Genotypes (NGS), WGS                             | germline                                                 | 1768620                                                  | \n",
       "\n",
       "\n"
      ],
      "text/plain": [
       "  Study ID       \n",
       "1 phs001356.v1.p2\n",
       "2 phs001098.v2.p2\n",
       "3 phs000499.v4.p2\n",
       "4 phs000498.v4.p2\n",
       "5 phs000286.v6.p2\n",
       "6 phs000964.v3.p1\n",
       "  Study Name                                                          \n",
       "1 Exome Chip Genotyping: The Jackson Heart Study                      \n",
       "2 T2D-GENES Multi-Ethnic Exome Sequencing Study: Jackson Heart Study  \n",
       "3 NHLBI Jackson Heart Study Candidate Gene Association Resource (CARe)\n",
       "4 Jackson Heart Study Allelic Spectrum Project                        \n",
       "5 The Jackson Heart Study (JHS)                                       \n",
       "6 NHLBI TOPMed: The Jackson Heart Study                               \n",
       "  Release Date Nb Participants Study Design                   \n",
       "1  2019-05-10  2788            Prospective Longitudinal Cohort\n",
       "2  2019-05-10  1029            Case-Control                   \n",
       "3  2019-05-10  3352            Prospective Longitudinal Cohort\n",
       "4  2019-05-10  1983            Prospective Longitudinal Cohort\n",
       "5  2019-05-10  3889            Prospective Longitudinal Cohort\n",
       "6  2018-05-18  3596            Prospective Longitudinal Cohort\n",
       "  Project                                                        \n",
       "1 National Heart, Lung, Blood Institute                          \n",
       "2 NHLBI GO-ESP                                                   \n",
       "3 NHLBI CARe                                                     \n",
       "4 National Heart, Lung, Blood Institute                          \n",
       "5 National Heart, Lung, Blood Institute, NHLBI GO-ESP, NHLBI CARe\n",
       "6 National Human Genome Research Institute                       \n",
       "  Diseases                                                                                                                                                                                              \n",
       "1 Cardiovascular Diseases, Hypertension, Diabetes Mellitus                                                                                                                                              \n",
       "2 Diabetes Mellitus, Type 2                                                                                                                                                                             \n",
       "3 Longitudinal Studies                                                                                                                                                                                  \n",
       "4 Cardiovascular Diseases                                                                                                                                                                               \n",
       "5 Cardiovascular Diseases, Coronary Artery Disease, Diabetes Mellitus, Type 2, Obesity, Hypertension, Kidney Failure, Chronic, Stroke, Heart Failure, Peripheral Vascular Diseases, Arrhythmias, Cardiac\n",
       "6 Cardiovascular Diseases, Hypertension, Diabetes Mellitus                                                                                                                                              \n",
       "  Ancestor ID     Ancestor Name                 Molecular Data Type         \n",
       "1 phs000286.v6.p2 The Jackson Heart Study (JHS) SNP Genotypes (Array)       \n",
       "2 phs000286.v6.p2 The Jackson Heart Study (JHS) SNP/CNV Genotypes (NGS), WXS\n",
       "3 phs000286.v6.p2 The Jackson Heart Study (JHS) SNP Genotypes (Array)       \n",
       "4 phs000286.v6.p2 The Jackson Heart Study (JHS) SNP Genotypes (NGS), WXS    \n",
       "5                                                                           \n",
       "6                                               SNP/CNV Genotypes (NGS), WGS\n",
       "  Tumor Type            UID    \n",
       "1 germline              1692088\n",
       "2 germline              1597258\n",
       "3 germline, unspecified 1597257\n",
       "4 germline              1597256\n",
       "5 germline, unspecified 1597254\n",
       "6 germline              1768620"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "search.dbgap(\"Jackson\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### dbGaP returns the list of the studies related to your term. As you see, there are 6 studies associated with the \"Jackson Heart Study\" (JHS). One of these study is the main one a.k.a the \"parent study\", whereas the other ones are substudies. In this case, phs000286.v5.p1 is the parent study. Firslty, we can use the phs.version() function in order to be sure that this is the latest version of the study. We can abbreviate the phs name by giving just the digit, or we can use the full dbGaP id."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "'phs000286.v6.p2'"
      ],
      "text/latex": [
       "'phs000286.v6.p2'"
      ],
      "text/markdown": [
       "'phs000286.v6.p2'"
      ],
      "text/plain": [
       "[1] \"phs000286.v6.p2\""
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "phs.version(\"286\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### The is.parent() function is usefull to test if a study is a parent study or a substudy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "TRUE"
      ],
      "text/latex": [
       "TRUE"
      ],
      "text/markdown": [
       "TRUE"
      ],
      "text/plain": [
       "[1] TRUE"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "FALSE"
      ],
      "text/latex": [
       "FALSE"
      ],
      "text/markdown": [
       "FALSE"
      ],
      "text/plain": [
       "[1] FALSE"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "is.parent(\"000286\") # JHS main cohort\n",
    "is.parent(\"phs499\") # substudy \"CARe\" for JHS"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### If you don't know the parent study of a substudy, try parent.study()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<ol class=list-inline>\n",
       "\t<li>'phs000286.v6.p2'</li>\n",
       "\t<li>'Jackson Heart Study (JHS) Cohort'</li>\n",
       "</ol>\n"
      ],
      "text/latex": [
       "\\begin{enumerate*}\n",
       "\\item 'phs000286.v6.p2'\n",
       "\\item 'Jackson Heart Study (JHS) Cohort'\n",
       "\\end{enumerate*}\n"
      ],
      "text/markdown": [
       "1. 'phs000286.v6.p2'\n",
       "2. 'Jackson Heart Study (JHS) Cohort'\n",
       "\n",
       "\n"
      ],
      "text/plain": [
       "[1] \"phs000286.v6.p2\"                  \"Jackson Heart Study (JHS) Cohort\""
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "parent.study(\"phs000499\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### On the other side, use sub.study() to get the name and IDs of the substudies from a parent one"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<thead><tr><th scope=col>phs</th><th scope=col>name</th></tr></thead>\n",
       "<tbody>\n",
       "\t<tr><td>phs001356.v1.p2                                                     </td><td>Exome Chip Genotyping: The Jackson Heart Study                      </td></tr>\n",
       "\t<tr><td>phs000498.v4.p2                                                     </td><td>Jackson Heart Study Allelic Spectrum Project                        </td></tr>\n",
       "\t<tr><td>phs001069.v1.p2                                                     </td><td>MIGen_ExS: JHS                                                      </td></tr>\n",
       "\t<tr><td>phs000402.v4.p2                                                     </td><td>NHLBI GO-ESP: Heart Cohorts Exome Sequencing Project (JHS)          </td></tr>\n",
       "\t<tr><td>phs000499.v4.p2                                                     </td><td>NHLBI Jackson Heart Study Candidate Gene Association Resource (CARe)</td></tr>\n",
       "\t<tr><td>phs001098.v2.p2                                                     </td><td>T2D-GENES Multi-Ethnic Exome Sequencing Study: Jackson Heart Study  </td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "\\begin{tabular}{ll}\n",
       " phs & name\\\\\n",
       "\\hline\n",
       "\t phs001356.v1.p2                                                      & Exome Chip Genotyping: The Jackson Heart Study                      \\\\\n",
       "\t phs000498.v4.p2                                                      & Jackson Heart Study Allelic Spectrum Project                        \\\\\n",
       "\t phs001069.v1.p2                                                        & MIGen\\_ExS: JHS                                                      \\\\\n",
       "\t phs000402.v4.p2                                                      & NHLBI GO-ESP: Heart Cohorts Exome Sequencing Project (JHS)          \\\\\n",
       "\t phs000499.v4.p2                                                      & NHLBI Jackson Heart Study Candidate Gene Association Resource (CARe)\\\\\n",
       "\t phs001098.v2.p2                                                      & T2D-GENES Multi-Ethnic Exome Sequencing Study: Jackson Heart Study  \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "phs | name | \n",
       "|---|---|---|---|---|---|\n",
       "| phs001356.v1.p2                                                      | Exome Chip Genotyping: The Jackson Heart Study                       | \n",
       "| phs000498.v4.p2                                                      | Jackson Heart Study Allelic Spectrum Project                         | \n",
       "| phs001069.v1.p2                                                      | MIGen_ExS: JHS                                                       | \n",
       "| phs000402.v4.p2                                                      | NHLBI GO-ESP: Heart Cohorts Exome Sequencing Project (JHS)           | \n",
       "| phs000499.v4.p2                                                      | NHLBI Jackson Heart Study Candidate Gene Association Resource (CARe) | \n",
       "| phs001098.v2.p2                                                      | T2D-GENES Multi-Ethnic Exome Sequencing Study: Jackson Heart Study   | \n",
       "\n",
       "\n"
      ],
      "text/plain": [
       "     phs            \n",
       "[1,] phs001356.v1.p2\n",
       "[2,] phs000498.v4.p2\n",
       "[3,] phs001069.v1.p2\n",
       "[4,] phs000402.v4.p2\n",
       "[5,] phs000499.v4.p2\n",
       "[6,] phs001098.v2.p2\n",
       "     name                                                                \n",
       "[1,] Exome Chip Genotyping: The Jackson Heart Study                      \n",
       "[2,] Jackson Heart Study Allelic Spectrum Project                        \n",
       "[3,] MIGen_ExS: JHS                                                      \n",
       "[4,] NHLBI GO-ESP: Heart Cohorts Exome Sequencing Project (JHS)          \n",
       "[5,] NHLBI Jackson Heart Study Candidate Gene Association Resource (CARe)\n",
       "[6,] T2D-GENES Multi-Ethnic Exome Sequencing Study: Jackson Heart Study  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sub.study(\"286\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### If you want to get the name of a study from its dbGaP id, use study.name()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "'Jackson Heart Study (JHS) Cohort'"
      ],
      "text/latex": [
       "'Jackson Heart Study (JHS) Cohort'"
      ],
      "text/markdown": [
       "'Jackson Heart Study (JHS) Cohort'"
      ],
      "text/plain": [
       "[1] \"Jackson Heart Study (JHS) Cohort\""
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "study.name(\"286\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Finally, you can watch your study on dbGaP with browse.dbgap().\n",
    "##### If a website exists for this study, you can browse it using browse.study()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "'https://www.ncbi.nlm.nih.gov/projects/gap/cgi-bin/study.cgi?study_id=phs000286.v6.p2'"
      ],
      "text/latex": [
       "'https://www.ncbi.nlm.nih.gov/projects/gap/cgi-bin/study.cgi?study\\_id=phs000286.v6.p2'"
      ],
      "text/markdown": [
       "'https://www.ncbi.nlm.nih.gov/projects/gap/cgi-bin/study.cgi?study_id=phs000286.v6.p2'"
      ],
      "text/plain": [
       "[1] \"https://www.ncbi.nlm.nih.gov/projects/gap/cgi-bin/study.cgi?study_id=phs000286.v6.p2\""
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "'https://www.jacksonheartstudy.org'"
      ],
      "text/latex": [
       "'https://www.jacksonheartstudy.org'"
      ],
      "text/markdown": [
       "'https://www.jacksonheartstudy.org'"
      ],
      "text/plain": [
       "[1] \"https://www.jacksonheartstudy.org\""
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "browse.dbgap(\"286\", no.browser = TRUE)\n",
    "browse.study(\"286\", no.browser = TRUE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Explore the characteristics of your study\n",
    "##### For each dbGaP study, there can be multiple consent groups that will have there specificities. Use consent.groups to know the number and the name of the consent groups in the study that you are exploring. Let's keep focusing on JHS."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<thead><tr><th></th><th scope=col>shortName</th><th scope=col>longName</th></tr></thead>\n",
       "<tbody>\n",
       "\t<tr><th scope=row>0</th><td>NRUP                                                                                                                                                                             </td><td>Subjects did not participate in the study, did not complete a consent document and are included only for the pedigree structure and/or genotype controls, such as HapMap subjects</td></tr>\n",
       "\t<tr><th scope=row>1</th><td>HMB-IRB-NPU                                                                                                                                                                      </td><td>Health/Medical/Biomedical (IRB, NPU)                                                                                                                                             </td></tr>\n",
       "\t<tr><th scope=row>2</th><td>DS-FDO-IRB-NPU                                                                                                                                                                   </td><td>Disease-Specific (Focused Disease Only, IRB, NPU)                                                                                                                                </td></tr>\n",
       "\t<tr><th scope=row>3</th><td>HMB-IRB                                                                                                                                                                          </td><td>Health/Medical/Biomedical (IRB)                                                                                                                                                  </td></tr>\n",
       "\t<tr><th scope=row>4</th><td>DS-FDO-IRB                                                                                                                                                                       </td><td>Disease-Specific (Focused Disease Only, IRB)                                                                                                                                     </td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ll}\n",
       "  & shortName & longName\\\\\n",
       "\\hline\n",
       "\t0 & NRUP                                                                                                                                                                              & Subjects did not participate in the study, did not complete a consent document and are included only for the pedigree structure and/or genotype controls, such as HapMap subjects\\\\\n",
       "\t1 & HMB-IRB-NPU                                                                                                                                                                       & Health/Medical/Biomedical (IRB, NPU)                                                                                                                                             \\\\\n",
       "\t2 & DS-FDO-IRB-NPU                                                                                                                                                                    & Disease-Specific (Focused Disease Only, IRB, NPU)                                                                                                                                \\\\\n",
       "\t3 & HMB-IRB                                                                                                                                                                           & Health/Medical/Biomedical (IRB)                                                                                                                                                  \\\\\n",
       "\t4 & DS-FDO-IRB                                                                                                                                                                        & Disease-Specific (Focused Disease Only, IRB)                                                                                                                                     \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "| <!--/--> | shortName | longName | \n",
       "|---|---|---|---|---|\n",
       "| 0 | NRUP                                                                                                                                                                              | Subjects did not participate in the study, did not complete a consent document and are included only for the pedigree structure and/or genotype controls, such as HapMap subjects | \n",
       "| 1 | HMB-IRB-NPU                                                                                                                                                                       | Health/Medical/Biomedical (IRB, NPU)                                                                                                                                              | \n",
       "| 2 | DS-FDO-IRB-NPU                                                                                                                                                                    | Disease-Specific (Focused Disease Only, IRB, NPU)                                                                                                                                 | \n",
       "| 3 | HMB-IRB                                                                                                                                                                           | Health/Medical/Biomedical (IRB)                                                                                                                                                   | \n",
       "| 4 | DS-FDO-IRB                                                                                                                                                                        | Disease-Specific (Focused Disease Only, IRB)                                                                                                                                      | \n",
       "\n",
       "\n"
      ],
      "text/plain": [
       "  shortName     \n",
       "0 NRUP          \n",
       "1 HMB-IRB-NPU   \n",
       "2 DS-FDO-IRB-NPU\n",
       "3 HMB-IRB       \n",
       "4 DS-FDO-IRB    \n",
       "  longName                                                                                                                                                                         \n",
       "0 Subjects did not participate in the study, did not complete a consent document and are included only for the pedigree structure and/or genotype controls, such as HapMap subjects\n",
       "1 Health/Medical/Biomedical (IRB, NPU)                                                                                                                                             \n",
       "2 Disease-Specific (Focused Disease Only, IRB, NPU)                                                                                                                                \n",
       "3 Health/Medical/Biomedical (IRB)                                                                                                                                                  \n",
       "4 Disease-Specific (Focused Disease Only, IRB)                                                                                                                                     "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "JHS <- \"phs000286\"\n",
    "consent.groups(JHS)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Use n.pop() to know the number of patient included in each groups"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<thead><tr><th scope=col>consent_group</th><th scope=col>male</th><th scope=col>female</th><th scope=col>total</th></tr></thead>\n",
       "<tbody>\n",
       "\t<tr><td>HMB-IRB       </td><td>2409          </td><td>3046          </td><td>5885          </td></tr>\n",
       "\t<tr><td>HMB-IRB-NPU   </td><td> 265          </td><td> 511          </td><td> 883          </td></tr>\n",
       "\t<tr><td>DS-FDO-IRB-NPU</td><td>  63          </td><td> 109          </td><td> 201          </td></tr>\n",
       "\t<tr><td>HMB-IRB       </td><td> 793          </td><td>1249          </td><td>2289          </td></tr>\n",
       "\t<tr><td>DS-FDO-IRB    </td><td> 174          </td><td> 295          </td><td> 516          </td></tr>\n",
       "\t<tr><td>TOTAL         </td><td>3704          </td><td>5210          </td><td>9774          </td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "\\begin{tabular}{r|llll}\n",
       " consent\\_group & male & female & total\\\\\n",
       "\\hline\n",
       "\t HMB-IRB        & 2409           & 3046           & 5885          \\\\\n",
       "\t HMB-IRB-NPU    &  265           &  511           &  883          \\\\\n",
       "\t DS-FDO-IRB-NPU &   63           &  109           &  201          \\\\\n",
       "\t HMB-IRB        &  793           & 1249           & 2289          \\\\\n",
       "\t DS-FDO-IRB     &  174           &  295           &  516          \\\\\n",
       "\t TOTAL          & 3704           & 5210           & 9774          \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "consent_group | male | female | total | \n",
       "|---|---|---|---|---|---|\n",
       "| HMB-IRB        | 2409           | 3046           | 5885           | \n",
       "| HMB-IRB-NPU    |  265           |  511           |  883           | \n",
       "| DS-FDO-IRB-NPU |   63           |  109           |  201           | \n",
       "| HMB-IRB        |  793           | 1249           | 2289           | \n",
       "| DS-FDO-IRB     |  174           |  295           |  516           | \n",
       "| TOTAL          | 3704           | 5210           | 9774           | \n",
       "\n",
       "\n"
      ],
      "text/plain": [
       "  consent_group  male female total\n",
       "1 HMB-IRB        2409 3046   5885 \n",
       "2 HMB-IRB-NPU     265  511    883 \n",
       "3 DS-FDO-IRB-NPU   63  109    201 \n",
       "4 HMB-IRB         793 1249   2289 \n",
       "5 DS-FDO-IRB      174  295    516 \n",
       "6 TOTAL          3704 5210   9774 "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "9774"
      ],
      "text/latex": [
       "9774"
      ],
      "text/markdown": [
       "9774"
      ],
      "text/plain": [
       "[1] 9774"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "n.pop(JHS)\n",
    "n.pop(JHS, consentgroups = FALSE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Use n.tables() and n.variables() to get the number of datatables in your study and the total number of variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "112"
      ],
      "text/latex": [
       "112"
      ],
      "text/markdown": [
       "112"
      ],
      "text/plain": [
       "[1] 112"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "4856"
      ],
      "text/latex": [
       "4856"
      ],
      "text/markdown": [
       "4856"
      ],
      "text/plain": [
       "[1] 4856"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "n.tables(JHS)\n",
    "n.variables(JHS)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### datatables.dict() will return a data frame with the datatables IDs (phtxxxxxx) and description of your study"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<thead><tr><th scope=col>pht</th><th scope=col>dt_study_name</th><th scope=col>dt_label</th></tr></thead>\n",
       "<tbody>\n",
       "\t<tr><td>pht008811.v1                                                                                                                                                                                                                                                                                                                                                                                          </td><td>MIGen_JHS_AA_Subject_Phenotypes                                                                                                                                                                                                                                                                                                                                                                       </td><td>Subject ID, age, sex, cohort, consortium, T2D affection status, weight, BMI, waist circumference, height, LDL, HDL, total cholesterol, blood pressure, adiponectin, debates age, creatinine, fasting glucose, fasting insulin, HbA1C, leptin, triglycerides, and medication of participants involved in the \"Myocardial Infarction Genetics Exome Sequencing Consortium: Jackson Heart Study\" project.</td></tr>\n",
       "\t<tr><td>pht008783.v1                                                                                                                                                                                                                                                                                                                                                                                          </td><td>sbpc                                                                                                                                                                                                                                                                                                                                                                                                  </td><td>sbpc                                                                                                                                                                                                                                                                                                                                                                                                  </td></tr>\n",
       "\t<tr><td>pht008727.v1                                                                                                                                                                                                                                                                                                                                                                                          </td><td>allevthf                                                                                                                                                                                                                                                                                                                                                                                              </td><td>allevthf                                                                                                                                                                                                                                                                                                                                                                                              </td></tr>\n",
       "\t<tr><td>pht001959.v2                                                                                                                                                                                                                                                                                                                                                                                          </td><td>loca                                                                                                                                                                                                                                                                                                                                                                                                  </td><td>loca                                                                                                                                                                                                                                                                                                                                                                                                  </td></tr>\n",
       "\t<tr><td>pht001945.v2                                                                                                                                                                                                                                                                                                                                                                                          </td><td>cena                                                                                                                                                                                                                                                                                                                                                                                                  </td><td>cena                                                                                                                                                                                                                                                                                                                                                                                                  </td></tr>\n",
       "\t<tr><td>pht001957.v2                                                                                                                                                                                                                                                                                                                                                                                          </td><td>hcaa                                                                                                                                                                                                                                                                                                                                                                                                  </td><td>hcaa                                                                                                                                                                                                                                                                                                                                                                                                  </td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "\\begin{tabular}{r|lll}\n",
       " pht & dt\\_study\\_name & dt\\_label\\\\\n",
       "\\hline\n",
       "\t pht008811.v1                                                                                                                                                                                                                                                                                                                                                                                           & MIGen\\_JHS\\_AA\\_Subject\\_Phenotypes                                                                                                                                                                                                                                                                                                                                                                & Subject ID, age, sex, cohort, consortium, T2D affection status, weight, BMI, waist circumference, height, LDL, HDL, total cholesterol, blood pressure, adiponectin, debates age, creatinine, fasting glucose, fasting insulin, HbA1C, leptin, triglycerides, and medication of participants involved in the \"Myocardial Infarction Genetics Exome Sequencing Consortium: Jackson Heart Study\" project.\\\\\n",
       "\t pht008783.v1                                                                                                                                                                                                                                                                                                                                                                                           & sbpc                                                                                                                                                                                                                                                                                                                                                                                                   & sbpc                                                                                                                                                                                                                                                                                                                                                                                                  \\\\\n",
       "\t pht008727.v1                                                                                                                                                                                                                                                                                                                                                                                           & allevthf                                                                                                                                                                                                                                                                                                                                                                                               & allevthf                                                                                                                                                                                                                                                                                                                                                                                              \\\\\n",
       "\t pht001959.v2                                                                                                                                                                                                                                                                                                                                                                                           & loca                                                                                                                                                                                                                                                                                                                                                                                                   & loca                                                                                                                                                                                                                                                                                                                                                                                                  \\\\\n",
       "\t pht001945.v2                                                                                                                                                                                                                                                                                                                                                                                           & cena                                                                                                                                                                                                                                                                                                                                                                                                   & cena                                                                                                                                                                                                                                                                                                                                                                                                  \\\\\n",
       "\t pht001957.v2                                                                                                                                                                                                                                                                                                                                                                                           & hcaa                                                                                                                                                                                                                                                                                                                                                                                                   & hcaa                                                                                                                                                                                                                                                                                                                                                                                                  \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "pht | dt_study_name | dt_label | \n",
       "|---|---|---|---|---|---|\n",
       "| pht008811.v1                                                                                                                                                                                                                                                                                                                                                                                           | MIGen_JHS_AA_Subject_Phenotypes                                                                                                                                                                                                                                                                                                                                                                        | Subject ID, age, sex, cohort, consortium, T2D affection status, weight, BMI, waist circumference, height, LDL, HDL, total cholesterol, blood pressure, adiponectin, debates age, creatinine, fasting glucose, fasting insulin, HbA1C, leptin, triglycerides, and medication of participants involved in the \"Myocardial Infarction Genetics Exome Sequencing Consortium: Jackson Heart Study\" project. | \n",
       "| pht008783.v1                                                                                                                                                                                                                                                                                                                                                                                           | sbpc                                                                                                                                                                                                                                                                                                                                                                                                   | sbpc                                                                                                                                                                                                                                                                                                                                                                                                   | \n",
       "| pht008727.v1                                                                                                                                                                                                                                                                                                                                                                                           | allevthf                                                                                                                                                                                                                                                                                                                                                                                               | allevthf                                                                                                                                                                                                                                                                                                                                                                                               | \n",
       "| pht001959.v2                                                                                                                                                                                                                                                                                                                                                                                           | loca                                                                                                                                                                                                                                                                                                                                                                                                   | loca                                                                                                                                                                                                                                                                                                                                                                                                   | \n",
       "| pht001945.v2                                                                                                                                                                                                                                                                                                                                                                                           | cena                                                                                                                                                                                                                                                                                                                                                                                                   | cena                                                                                                                                                                                                                                                                                                                                                                                                   | \n",
       "| pht001957.v2                                                                                                                                                                                                                                                                                                                                                                                           | hcaa                                                                                                                                                                                                                                                                                                                                                                                                   | hcaa                                                                                                                                                                                                                                                                                                                                                                                                   | \n",
       "\n",
       "\n"
      ],
      "text/plain": [
       "  pht          dt_study_name                  \n",
       "1 pht008811.v1 MIGen_JHS_AA_Subject_Phenotypes\n",
       "2 pht008783.v1 sbpc                           \n",
       "3 pht008727.v1 allevthf                       \n",
       "4 pht001959.v2 loca                           \n",
       "5 pht001945.v2 cena                           \n",
       "6 pht001957.v2 hcaa                           \n",
       "  dt_label                                                                                                                                                                                                                                                                                                                                                                                              \n",
       "1 Subject ID, age, sex, cohort, consortium, T2D affection status, weight, BMI, waist circumference, height, LDL, HDL, total cholesterol, blood pressure, adiponectin, debates age, creatinine, fasting glucose, fasting insulin, HbA1C, leptin, triglycerides, and medication of participants involved in the \"Myocardial Infarction Genetics Exome Sequencing Consortium: Jackson Heart Study\" project.\n",
       "2 sbpc                                                                                                                                                                                                                                                                                                                                                                                                  \n",
       "3 allevthf                                                                                                                                                                                                                                                                                                                                                                                              \n",
       "4 loca                                                                                                                                                                                                                                                                                                                                                                                                  \n",
       "5 cena                                                                                                                                                                                                                                                                                                                                                                                                  \n",
       "6 hcaa                                                                                                                                                                                                                                                                                                                                                                                                  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "tablesdict <- datatables.dict(JHS)\n",
    "head(tablesdict)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### variables.dict() will return a data frame with the variables IDs (phvxxxxxx), their name in the study, the datatable where they come from and their description"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<thead><tr><th scope=col>dt_study_name</th><th scope=col>phv</th><th scope=col>var_name</th><th scope=col>var_desc</th></tr></thead>\n",
       "<tbody>\n",
       "\t<tr><td>MIGen_JHS_AA_Subject_Phenotypes                                                                                                                                                                                       </td><td>phv00404354.v1                                                                                                                                                                                                        </td><td>SUBJECT_ID                                                                                                                                                                                                            </td><td>De-identified Subject ID                                                                                                                                                                                              </td></tr>\n",
       "\t<tr><td>MIGen_JHS_AA_Subject_Phenotypes                                                                                                                                                                                       </td><td>phv00404355.v1                                                                                                                                                                                                        </td><td>sex                                                                                                                                                                                                                   </td><td>Gender of participant                                                                                                                                                                                                 </td></tr>\n",
       "\t<tr><td>sbpc                                                                                                                                                                                                                  </td><td>phv00403830.v1                                                                                                                                                                                                        </td><td>SUBJECT_ID                                                                                                                                                                                                            </td><td>PARTICIPANT ID [Visit 1] [Sitting Blood Pressure Form, SBP]                                                                                                                                                           </td></tr>\n",
       "\t<tr><td>sbpc                                                                                                                                                                                                                  </td><td>phv00403831.v1                                                                                                                                                                                                        </td><td>VISIT                                                                                                                                                                                                                 </td><td>CONTACT OCCASION [Visit 1] [Sitting Blood Pressure Form, SBP]                                                                                                                                                         </td></tr>\n",
       "\t<tr><td>sbpc                                                                                                                                                                                                                  </td><td>phv00403832.v1                                                                                                                                                                                                        </td><td>SBPC1                                                                                                                                                                                                                 </td><td>Q1. A. Temperature. Room temperature (degrees centigrade) [Visit 1] [Sitting Blood Pressure Form, SBP]                                                                                                                </td></tr>\n",
       "\t<tr><td>sbpc                                                                                                                                                                                                                  </td><td>phv00403833.v1                                                                                                                                                                                                        </td><td>SBPC2                                                                                                                                                                                                                 </td><td>Q2. B. Tobacco and caffeine use, physical activity, and medication. Have you smoked or used chewing tobacco, nicotine gum or snuff today or do you wear a nicotine patch? [Visit 1] [Sitting Blood Pressure Form, SBP]</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "\\begin{tabular}{r|llll}\n",
       " dt\\_study\\_name & phv & var\\_name & var\\_desc\\\\\n",
       "\\hline\n",
       "\t MIGen\\_JHS\\_AA\\_Subject\\_Phenotypes                                                                                                                                                                                & phv00404354.v1                                                                                                                                                                                                         & SUBJECT\\_ID                                                                                                                                                                                                           & De-identified Subject ID                                                                                                                                                                                              \\\\\n",
       "\t MIGen\\_JHS\\_AA\\_Subject\\_Phenotypes                                                                                                                                                                                & phv00404355.v1                                                                                                                                                                                                         & sex                                                                                                                                                                                                                    & Gender of participant                                                                                                                                                                                                 \\\\\n",
       "\t sbpc                                                                                                                                                                                                                           & phv00403830.v1                                                                                                                                                                                                                 & SUBJECT\\_ID                                                                                                                                                                                                                   & PARTICIPANT ID {[}Visit 1{]} {[}Sitting Blood Pressure Form, SBP{]}                                                                                                                                                           \\\\\n",
       "\t sbpc                                                                                                                                                                                                                           & phv00403831.v1                                                                                                                                                                                                                 & VISIT                                                                                                                                                                                                                          & CONTACT OCCASION {[}Visit 1{]} {[}Sitting Blood Pressure Form, SBP{]}                                                                                                                                                         \\\\\n",
       "\t sbpc                                                                                                                                                                                                                           & phv00403832.v1                                                                                                                                                                                                                 & SBPC1                                                                                                                                                                                                                          & Q1. A. Temperature. Room temperature (degrees centigrade) {[}Visit 1{]} {[}Sitting Blood Pressure Form, SBP{]}                                                                                                                \\\\\n",
       "\t sbpc                                                                                                                                                                                                                           & phv00403833.v1                                                                                                                                                                                                                 & SBPC2                                                                                                                                                                                                                          & Q2. B. Tobacco and caffeine use, physical activity, and medication. Have you smoked or used chewing tobacco, nicotine gum or snuff today or do you wear a nicotine patch? {[}Visit 1{]} {[}Sitting Blood Pressure Form, SBP{]}\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "dt_study_name | phv | var_name | var_desc | \n",
       "|---|---|---|---|---|---|\n",
       "| MIGen_JHS_AA_Subject_Phenotypes                                                                                                                                                                                        | phv00404354.v1                                                                                                                                                                                                         | SUBJECT_ID                                                                                                                                                                                                             | De-identified Subject ID                                                                                                                                                                                               | \n",
       "| MIGen_JHS_AA_Subject_Phenotypes                                                                                                                                                                                        | phv00404355.v1                                                                                                                                                                                                         | sex                                                                                                                                                                                                                    | Gender of participant                                                                                                                                                                                                  | \n",
       "| sbpc                                                                                                                                                                                                                   | phv00403830.v1                                                                                                                                                                                                         | SUBJECT_ID                                                                                                                                                                                                             | PARTICIPANT ID [Visit 1] [Sitting Blood Pressure Form, SBP]                                                                                                                                                            | \n",
       "| sbpc                                                                                                                                                                                                                   | phv00403831.v1                                                                                                                                                                                                         | VISIT                                                                                                                                                                                                                  | CONTACT OCCASION [Visit 1] [Sitting Blood Pressure Form, SBP]                                                                                                                                                          | \n",
       "| sbpc                                                                                                                                                                                                                   | phv00403832.v1                                                                                                                                                                                                         | SBPC1                                                                                                                                                                                                                  | Q1. A. Temperature. Room temperature (degrees centigrade) [Visit 1] [Sitting Blood Pressure Form, SBP]                                                                                                                 | \n",
       "| sbpc                                                                                                                                                                                                                   | phv00403833.v1                                                                                                                                                                                                         | SBPC2                                                                                                                                                                                                                  | Q2. B. Tobacco and caffeine use, physical activity, and medication. Have you smoked or used chewing tobacco, nicotine gum or snuff today or do you wear a nicotine patch? [Visit 1] [Sitting Blood Pressure Form, SBP] | \n",
       "\n",
       "\n"
      ],
      "text/plain": [
       "  dt_study_name                   phv            var_name  \n",
       "1 MIGen_JHS_AA_Subject_Phenotypes phv00404354.v1 SUBJECT_ID\n",
       "2 MIGen_JHS_AA_Subject_Phenotypes phv00404355.v1 sex       \n",
       "3 sbpc                            phv00403830.v1 SUBJECT_ID\n",
       "4 sbpc                            phv00403831.v1 VISIT     \n",
       "5 sbpc                            phv00403832.v1 SBPC1     \n",
       "6 sbpc                            phv00403833.v1 SBPC2     \n",
       "  var_desc                                                                                                                                                                                                              \n",
       "1 De-identified Subject ID                                                                                                                                                                                              \n",
       "2 Gender of participant                                                                                                                                                                                                 \n",
       "3 PARTICIPANT ID [Visit 1] [Sitting Blood Pressure Form, SBP]                                                                                                                                                           \n",
       "4 CONTACT OCCASION [Visit 1] [Sitting Blood Pressure Form, SBP]                                                                                                                                                         \n",
       "5 Q1. A. Temperature. Room temperature (degrees centigrade) [Visit 1] [Sitting Blood Pressure Form, SBP]                                                                                                                \n",
       "6 Q2. B. Tobacco and caffeine use, physical activity, and medication. Have you smoked or used chewing tobacco, nicotine gum or snuff today or do you wear a nicotine patch? [Visit 1] [Sitting Blood Pressure Form, SBP]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "vardict <- variables.dict(JHS)\n",
    "head(vardict)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Extract your study\n",
    "### Get your dbGaP repository key\n",
    "In order to download or decrypt your data from dbGaP, you will need to request an access and to get a decryption key. Follow those steps to access your dbGaP repository key:\n",
    "##### - Go to https://www.ncbi.nlm.nih.gov/gap and click on `controlled access data`\n",
    "\n",
    "##### - Click on Log in to dbGaP\n",
    "\n",
    "##### - Identify yourself with your era common ID and password\n",
    "\n",
    "##### - Get a PI dbGaP repository key:\n",
    "   In order to download the files and to decrypt them, you will need a decryption key. This key can be found on a PI dbGaP account. Go to the `Authorized Access` and then `My Projects` tabs. Then, in the column `Actions` on the right of your screen, find `Get no password dbGaP repository key`.\n",
    "\n",
    "### Decrypt the .ncbi_enc files\n",
    "On dbGaP, the phenotypic files are encrypted. We created a decryption function that uses a dockerized version on sratoolkit. To use that function, you need to have docker installed on your device (www.docker.com). If you are using the dockerized version of this software (available at hub.docker.com/r/gversmee/dbgap2x), docker is already pre-installed, but you'll need to upload your key on the jupyter working directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "key <- \"path/to/your/key.ngc\"\n",
    "files <- \"path/to/directory/ofencrypted_files\"\n",
    "dbgap.decrypt(files, key)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You should see a \"decrypted_files\" directory in the directory where your encrypted files are located"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Download dbGaP files\n",
    "##### - Click on \"file selector\"\n",
    "This gives you access to the dbGaP file selector where you can find all the files available for the selected project. To find it, go to the `Authorized Access` and then `My Projects` tabs. Then, in the column `Actions` on the right of your screen, find `file selector`.\n",
    "\n",
    "##### - Filter by study accession\n",
    "Here, we want to get the phenotypic data for the study \"Early onset COPD\", so after checking `Study accession`, we select \"phs000946\".\n",
    "\n",
    "##### - Filter again\n",
    "Since we are only interested in getting the phenotypic data, let's filter by `Content type` and select `phenotype individual-auxiliary` and `phenotype individual-traits`.\n",
    "\n",
    "##### - Select the files\n",
    "Click on \"+\" to select all the files.\n",
    "\n",
    "##### - Click on \"Cart file\"\n",
    "This will downlaod a .krt file in your download folder.\n",
    "\n",
    "### Download and decrypt the files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "key <- \"path/to/your/key.ngc\"\n",
    "cart <- \"path/to/your/cartfile.krt\"\n",
    "dbgap.download(cart, key)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You should see in your working directory a new folder named dbGaP-*** that contains your files"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "R",
   "language": "R",
   "name": "ir"
  },
  "language_info": {
   "codemirror_mode": "r",
   "file_extension": ".r",
   "mimetype": "text/x-r-source",
   "name": "R",
   "pygments_lexer": "r",
   "version": "3.4.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}