Skip to content

Commit 4a7bf0f

Browse files
timmaniktimmanik
committed
Updated prompts
1 parent 91e970c commit 4a7bf0f

File tree

3 files changed

+233
-17
lines changed

3 files changed

+233
-17
lines changed

‎01-cancer-data-analysis/fm-ad-notebook-exploration.ipynb

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -158,6 +158,7 @@
158158
"source": [
159159
"import boto3\n",
160160
"import pandas as pd\n",
161+
"import numpy as np\n",
161162
"from botocore import UNSIGNED\n",
162163
"from botocore.config import Config\n",
163164
"from io import StringIO\n",

‎01-cancer-data-analysis/fm-ad-notebook-processing.ipynb

Lines changed: 76 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -34,9 +34,17 @@
3434
"metadata": {},
3535
"outputs": [],
3636
"source": [
37-
"# convert the combined_data.csv to dataframe called combined_df\n",
3837
"import pandas as pd\n",
39-
"\n",
38+
"import numpy as np"
39+
]
40+
},
41+
{
42+
"cell_type": "code",
43+
"execution_count": null,
44+
"metadata": {},
45+
"outputs": [],
46+
"source": [
47+
"# convert the combined_data.csv to dataframe called combined_df\n",
4048
"combined_df = pd.read_csv('combined_data.csv')"
4149
]
4250
},
@@ -95,7 +103,14 @@
95103
"metadata": {},
96104
"outputs": [],
97105
"source": [
98-
"# list the name of columns that have more than or equal to 18004 unique values"
106+
"# create a dictionary to store names of columns that have great than or equal to 18004 unique values"
107+
]
108+
},
109+
{
110+
"cell_type": "markdown",
111+
"metadata": {},
112+
"source": [
113+
"As we can see from the list above, the `case_id` column is included in the list. However, since we want to use this list to specifiy which columns to remove from the dataframe, we should remove `case_id` from this list."
99114
]
100115
},
101116
{
@@ -104,7 +119,31 @@
104119
"metadata": {},
105120
"outputs": [],
106121
"source": [
107-
"# create a new dataframe and drop columns that have more than or equal to 18004 unique values. however, do not drop the 'case_id' column"
122+
"# remove case_id from the dictionary above"
123+
]
124+
},
125+
{
126+
"cell_type": "markdown",
127+
"metadata": {},
128+
"source": [
129+
"Let's verify that the `case_id` column has been removed."
130+
]
131+
},
132+
{
133+
"cell_type": "code",
134+
"execution_count": null,
135+
"metadata": {},
136+
"outputs": [],
137+
"source": []
138+
},
139+
{
140+
"cell_type": "code",
141+
"execution_count": null,
142+
"metadata": {},
143+
"outputs": [],
144+
"source": [
145+
"# create a copy of the current dataframe\n",
146+
"# drop columns from the dictionary above"
108147
]
109148
},
110149
{
@@ -250,7 +289,7 @@
250289
"metadata": {},
251290
"outputs": [],
252291
"source": [
253-
"# show the records with the case_id aff95088-8760-46d2-a404-b545807e0735"
292+
"# show the records with the case_id fcd9637f-00f2-49e9-bb87-94e556d5d7eb"
254293
]
255294
},
256295
{
@@ -268,9 +307,10 @@
268307
"metadata": {},
269308
"outputs": [],
270309
"source": [
271-
"# Write a Python code that selects the two records with the same 'case_id' value of 'aff95088-8760-46d2-a404-b545807e0735'.\n",
272-
"# Display these records for visual inspection. Then, verify that these records complement each other in terms of null and non-null values for all columns after the first four columns.\n",
310+
"# Select the two records with the same 'case_id' value of 'fcd9637f-00f2-49e9-bb87-94e556d5d7eb.\n",
311+
"# Display these records for visual inspection. Then, verify that these records complement each other in terms of null and non-null values for all columns after the first five columns.\n",
273312
"# In other words, if one record has NaN values in a column, the other record should have non-NaN values in that same column, and vice versa.\n",
313+
"# Also, if both records have NaN values in the same column and ignore it from the comparison\n",
274314
"# If the two records complement each other, print \"The two records complement each other.\" Otherwise, print \"The two records do not complement each other.\""
275315
]
276316
},
@@ -288,9 +328,19 @@
288328
"outputs": [],
289329
"source": [
290330
"# write a Python code snippet that iterates over all unique 'case_id' values. For each 'case_id', select all records associated with that 'case_id'.\n",
291-
"# Verify that these records complement each other in terms of null and non-null values for all columns after the first four columns.\n",
331+
"# Verify that these records complement each other in terms of null and non-null values for all columns AFTER the first five columns.\n",
292332
"# In other words, if one record has NaN values in a column, the other records should have non-NaN values in that same column, and vice versa.\n",
293-
"# Print a dictionary where each 'case_id' is a key and the corresponding value is a boolean indicating whether all records with that 'case_id' perfectly complement each other in terms of null and non-null values."
333+
"# Print a dictionary where each 'case_id' is a key and the corresponding value is a boolean indicating whether all records with that 'case_id' perfectly complement each other in terms of null and non-null values.\n",
334+
"# For example, if all records with 'case_id' = 'fcd9637f-00f2-49e9-bb87-94e556d5d7eb' perfectly complement each other, the dictionary should have the key 'fcd9637f-00f2-49e9-bb87-94e556d5d7eb' with a value of True."
335+
]
336+
},
337+
{
338+
"cell_type": "code",
339+
"execution_count": null,
340+
"metadata": {},
341+
"outputs": [],
342+
"source": [
343+
"# Check if all the values in the dictionary are True if so print \"All records complement each other.\" otherwise print \"Not all records complement each other.\""
294344
]
295345
},
296346
{
@@ -489,7 +539,7 @@
489539
"cell_type": "markdown",
490540
"metadata": {},
491541
"source": [
492-
"Verify the range of the new age column is as expected."
542+
"Verify the range of the new age column is as expected. The proper range should be between 19-88."
493543
]
494544
},
495545
{
@@ -501,6 +551,22 @@
501551
"# show statistical summary of the diagnoses.age_at_diagnosis_years column"
502552
]
503553
},
554+
{
555+
"cell_type": "markdown",
556+
"metadata": {},
557+
"source": [
558+
"Once the range is expected, we can drop the diagnosis.age_at_diagnosis column so that in the next notebook, GitHub Copilot is able to automatically choose our single \"age column\" for vidsualizations."
559+
]
560+
},
561+
{
562+
"cell_type": "code",
563+
"execution_count": null,
564+
"metadata": {},
565+
"outputs": [],
566+
"source": [
567+
"# drop diagnosis.age_at_diagnosis column"
568+
]
569+
},
504570
{
505571
"cell_type": "markdown",
506572
"metadata": {},

‎01-cancer-data-analysis/fm-ad-notebook-visualization.ipynb

Lines changed: 156 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -42,8 +42,34 @@
4242
"outputs": [],
4343
"source": [
4444
"import pandas as pd\n",
45-
"\n",
46-
"combined_data_cleansed_df = pd.read_csv('combined_data_cleansed.csv')"
45+
"import numpy as np\n",
46+
"import seaborn as sns\n",
47+
"import matplotlib.pyplot as plt"
48+
]
49+
},
50+
{
51+
"cell_type": "code",
52+
"execution_count": null,
53+
"metadata": {},
54+
"outputs": [],
55+
"source": [
56+
"combined_data_cleansed_df = pd.read_csv('combined_data_cleaned.csv')"
57+
]
58+
},
59+
{
60+
"cell_type": "markdown",
61+
"metadata": {},
62+
"source": [
63+
"Let's rename our dataframe to df so that it will be easier to use the code suggestions from GitHub Copilot chat."
64+
]
65+
},
66+
{
67+
"cell_type": "code",
68+
"execution_count": null,
69+
"metadata": {},
70+
"outputs": [],
71+
"source": [
72+
"df = combined_data_cleansed_df"
4773
]
4874
},
4975
{
@@ -82,21 +108,36 @@
82108
"execution_count": null,
83109
"metadata": {},
84110
"outputs": [],
85-
"source": []
111+
"source": [
112+
"# show first few records"
113+
]
86114
},
87115
{
88116
"cell_type": "code",
89117
"execution_count": null,
90118
"metadata": {},
91119
"outputs": [],
92-
"source": []
120+
"source": [
121+
"# show df shape"
122+
]
93123
},
94124
{
95125
"cell_type": "code",
96126
"execution_count": null,
97127
"metadata": {},
98128
"outputs": [],
99-
"source": []
129+
"source": [
130+
"# show df columns"
131+
]
132+
},
133+
{
134+
"cell_type": "code",
135+
"execution_count": null,
136+
"metadata": {},
137+
"outputs": [],
138+
"source": [
139+
"# show df column and data types"
140+
]
100141
},
101142
{
102143
"cell_type": "markdown",
@@ -112,20 +153,121 @@
112153
"#### 4.1 Data visualization"
113154
]
114155
},
156+
{
157+
"cell_type": "markdown",
158+
"metadata": {},
159+
"source": [
160+
"##### Distribution of disease types\n",
161+
"\n",
162+
"Understanding the distribution of disease types helps identify the most common and rare cancers in the dataset, which is crucial for allocating resources and prioritizing research."
163+
]
164+
},
115165
{
116166
"cell_type": "code",
117167
"execution_count": null,
118168
"metadata": {},
119169
"outputs": [],
120170
"source": [
121-
"# create a bar graph of the diagnoses.age_at_diagnosis_years column to see the distribution of ages"
171+
"# create a pie chart of top 10 cases.disease_type from df"
122172
]
123173
},
124174
{
125175
"cell_type": "markdown",
126176
"metadata": {},
127177
"source": [
128-
"Now let's share with GitHub copilot chat the columns in our dataset and what visualizations and correlations it thinks that we can create from these columns."
178+
"##### Gender demographic\n",
179+
"\n",
180+
"Let's take a look at how the data is distributed with respect to gender."
181+
]
182+
},
183+
{
184+
"cell_type": "code",
185+
"execution_count": null,
186+
"metadata": {},
187+
"outputs": [],
188+
"source": [
189+
"# show the distribution of the column demographic.gender in bar chart"
190+
]
191+
},
192+
{
193+
"cell_type": "markdown",
194+
"metadata": {},
195+
"source": [
196+
"As we can see from above, the gender information was available from all but 9 samples and showed a slight bias toward females versus males.\n",
197+
"\n",
198+
"According to the [study](https://aacrjournals.org/cancerres/article/77/9/2464/625134/High-Throughput-Genomic-Profiling-of-Adult-Solid), this bias can be explained in part by the large number of breast and GYN cancer samples within the dataset since both breast and gynecological cancers are specific to females. Let's try to visually see if that is the case."
199+
]
200+
},
201+
{
202+
"cell_type": "code",
203+
"execution_count": null,
204+
"metadata": {},
205+
"outputs": [],
206+
"source": [
207+
"# show the relationship between cases.primary_site and demographic.gender"
208+
]
209+
},
210+
{
211+
"cell_type": "markdown",
212+
"metadata": {},
213+
"source": [
214+
"A similar analysis we can look at is the relationship between the disease type and the gender of the patient\n",
215+
"\n",
216+
"Identifying gender differences in disease prevalence can highlight gender-specific vulnerabilities or protective factors, influencing personalized treatment approaches."
217+
]
218+
},
219+
{
220+
"cell_type": "code",
221+
"execution_count": null,
222+
"metadata": {},
223+
"outputs": [],
224+
"source": [
225+
"# visualize the relationship between cases.disease_type and demographic.gender"
226+
]
227+
},
228+
{
229+
"cell_type": "markdown",
230+
"metadata": {},
231+
"source": [
232+
"##### Age distribution\n",
233+
"\n",
234+
"The study \"High-Throughput Genomic Profiling of Adult Solid Tumors\" utilized patient samples that were part of routine clinical care, which were submitted for genomic profiling by Foundation Medicine. So the study did not do a random sampling as part of their data collection.\n",
235+
"\n",
236+
"That being said, let's see how close to a normal distribution the dataset is with respect to age."
237+
]
238+
},
239+
{
240+
"cell_type": "code",
241+
"execution_count": null,
242+
"metadata": {},
243+
"outputs": [],
244+
"source": [
245+
"# show distribution of diagnoses.age_at_diagnosis_years"
246+
]
247+
},
248+
{
249+
"cell_type": "markdown",
250+
"metadata": {},
251+
"source": [
252+
"What is the relationship between age at diagnosis and disease type?\n",
253+
"\n",
254+
"This question helps determine if certain cancers are more likely to occur at specific ages, which can inform targeted awareness and early detection efforts in particular demographics."
255+
]
256+
},
257+
{
258+
"cell_type": "code",
259+
"execution_count": null,
260+
"metadata": {},
261+
"outputs": [],
262+
"source": []
263+
},
264+
{
265+
"cell_type": "markdown",
266+
"metadata": {},
267+
"source": [
268+
"Is there a relationship between the primary diagnosis and the sample type?\n",
269+
"\n",
270+
"This question is important to understand if certain diagnoses are more likely to be made from specific types of samples, affecting diagnostic strategies and the feasibility of certain tests."
129271
]
130272
},
131273
{
@@ -142,6 +284,13 @@
142284
"#### 4.X Additional analysis"
143285
]
144286
},
287+
{
288+
"cell_type": "markdown",
289+
"metadata": {},
290+
"source": [
291+
"Now let's share with GitHub copilot chat the columns in our dataset and what visualizations and correlations it thinks that we can create from these columns."
292+
]
293+
},
145294
{
146295
"cell_type": "code",
147296
"execution_count": null,

0 commit comments

Comments
 (0)