Skip to content

Commit 21d7628

Browse files
mikemaas-ciscomikemaas-cisco
committed
Fixes for changes to MSE URL and API
1 parent 0b7699d commit 21d7628

File tree

7 files changed

+172
-189
lines changed

7 files changed

+172
-189
lines changed

‎labs/coding-202-parsing-json/1.md

Lines changed: 52 additions & 50 deletions
Original file line numberDiff line numberDiff line change
@@ -1,80 +1,82 @@
1+
# Coding 202: Parsing JSON using Python
12

2-
# Coding 202: Parsing JSON using Python #
3+
In this Learning Lab you will learn the basics of parsing JSON content using Python. We will use the [Connected Mobile Experiences (CMX) Mobility Services](https://developer.cisco.com/site/cmx-mobility-services/ "CMX Mobility Services") as a data source and query for the names of the available access points and put them into a simple list for display. If you want to dig deeper into CMX, review the [Mobility Services Engine (MSE) Learning Lab](#/labs/cmx/step/1).
34

4-
In this Learning Lab you will learn the basics of parsing JSON content using Python. We will use the [Connected Mobile Experiences (CMX) Mobility Services](https://developer.cisco.com/site/cmx-mobility-services/ "CMX Mobility Services") as a data source and query for the names of the available access points and put them into a simple list for display. If you want to dig deeper into CMX, review the [Mobility Services Engine (MSE) Learning Lab](#/labs/cmx/step/1).
5-
6-
7-
## Objectives ##
5+
## Objectives
86

97
Completion Time: 15 minutes
108

11-
* Understand the basics of reading and parsing HTTP content using Python
12-
* Learn how to use Python to extract only the JSON data you want using the json library
13-
9+
- Understand the basics of reading and parsing HTTP content using Python
10+
- Learn how to use Python to extract only the JSON data you want using the json library
1411

1512
## Prerequisites
1613

1714
You will want to make sure you have gone through the [Coding 101 lab](#/labs/coding-101-rest-basics/step/1 "Coding 101 Lab") if you are unfamiliar with Python and retrieving results from a RESTful service and the [Coding 201 lab](#/labs/coding-201-parsing-xml/step/1 "Coding 201 Parsing XML using Python Lab") for a similar approach to retrieving data using XML.
1815

16+
You should also have a basic familiarity with JSON. If you do not, consider visiting the [W3Schools JSON Tutorial](http://www.w3schools.com/json/ "W3Schools JSON Tutorial") to get a firm base to build upon.
1917

20-
You should also have a basic familiarity with JSON. If you do not, consider visiting the [W3Schools JSON Tutorial](http://www.w3schools.com/json/ "W3Schools JSON Tutorial") to get a firm base to build upon.
18+
For this lab we will be using Python 3.4+. To verify your version run the following command in a terminal window:
2119

22-
For this lab we will be using Python 3.4+. To verify your version run the following command in a terminal window:
2320
```
2421
python --version
2522
```
26-
If you are on a DevNet Zone station, the correct version of Python should already be installed.
2723

24+
If you are on a DevNet Zone station, the correct version of Python should already be installed.
2825

29-
## Step 1. Make a HTTP REST call with Python
26+
## Step 1\. Make a HTTP REST call with Python
3027

3128
To get started, let's create a simple Python script that can make a HTTP request to the CMX server.
3229

3330
1. Open the text editor and create a file named **get-ap-json.py**.
3431
2. Save **get-ap-json.py** to the desktop.
3532
3. Within the text editor add the following lines to **get-ap-json.py**:
36-
```
37-
from urllib.request import Request, urlopen
38-
import ssl
39-
req = Request('https://64.103.26.61/api/contextaware/v1/maps/info/DevNetCampus/DevNetBuilding/DevNetZone')
40-
req.add_header('Authorization', 'Basic bGVhcm5pbmc6bGVhcm5pbmc==')
41-
ctx = ssl.create_default_context()
42-
ctx.check_hostname = False
43-
ctx.verify_mode = ssl.CERT_NONE
44-
response = urlopen(req, context=ctx)
45-
responseString = response.read().decode("utf-8")
46-
print(responseString)
47-
response.close()
48-
```
49-
In this snippet, we are:
50-
- Importing the required *Request* and *urlopen* libraries
51-
- Constructing a request to the CMX URI
52-
- Adding Basic authentication to our Request - the string of characters is the login/password encoded to Base64
53-
- Opening the request and getting a Response back from the CMX URI
54-
- Parsing the Response as a String and printing it out to the console
55-
<br/>
56-
<br/>
57-
4. Save the **get-ap-json.py** file. If you want to just download or review the current code, you can get it from GitHub <a href="https://github.com/CiscoDevNet/coding-skills-sample-code/blob/master/coding202-parsing-json/get-ap-json-1.py" target="_blank">here</a>.
58-
5. Open a command prompt.<br/><br/>
59-
&nbsp;&nbsp;If your are using the Windows OS, type the following command into the command prompt window to move to your desktop:
60-
```
61-
cd %USERPROFILE%\Desktop
62-
```
63-
&nbsp;&nbsp;If you are using OS X, type the following command into the terminal window to move to your desktop:
64-
```
65-
cd ~/Desktop
66-
```
33+
34+
```
35+
from urllib.request import Request, urlopen
36+
req = Request('https://devnetapi.cisco.com/sandbox/mse/api/config/v1/maps/info/DevNetCampus/DevNetBuilding/DevNetZone')
37+
req.add_header('Authorization', 'Basic bGVhcm5pbmc6bGVhcm5pbmc=')
38+
response = urlopen(req)
39+
response_string = response.read().decode('utf-8')
40+
print(response_string)
41+
response.close()
42+
```
43+
44+
In this snippet, we are:
45+
46+
- Importing the required _Request_ and _urlopen_ libraries
47+
- Constructing a request to the CMX URI
48+
- Adding Basic authentication to our Request - the string of characters is the login/password encoded to Base64
49+
- Opening the request and getting a Response back from the CMX URI
50+
- Parsing the Response as a String and printing it out to the console<br><br>
51+
52+
4. Save the **get-ap-json.py** file. If you want to just download or review the current code, you can get it from GitHub [here](https://github.com/CiscoDevNet/coding-skills-sample-code/blob/master/coding202-parsing-json/get-ap-json-1.py).
53+
54+
5. Open a command prompt.<br>
55+
<br>
56+
If your are using the Windows OS, type the following command into the command prompt window to move to your desktop:
57+
58+
```
59+
cd %USERPROFILE%\Desktop
60+
```
61+
62+
If you are using OS X, type the following command into the terminal window to move to your desktop:
63+
64+
```
65+
cd ~/Desktop
66+
```
67+
6768
6. In the command prompt window, type the following command to execute your newly created file using Python:
68-
```
69-
python get-ap-json.py
70-
```
71-
7. When you run the Python script, you should get an screen full of XML data returned to the terminal.
7269

70+
```
71+
python get-ap-json.py
72+
```
73+
74+
7. When you run the Python script, you should get an screen full of JSON data returned to the terminal.
7375

7476
<div style="text-align:center" markdown="1">
75-
![XML Command Prompt](/posts/files/coding-201-parsing-xml/xml-output.png)
77+
<img src="/posts/files/coding-202-parsing-json/json-output.png" alt="JSON Command Prompt">
7678
</div>
7779

78-
----------
80+
--------------------------------------------------------------------------------
7981

80-
Unfortunately, XML isn't the data format we wanted - but it is the data format CMX returns by default. Let's ask CMX to give us back JSON data in the next step.
82+
This dump of JSON data isn't that useful. Let's look at cleaning it up to understand it better in the next step.

‎labs/coding-202-parsing-json/2.md

Lines changed: 69 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -1,28 +1,78 @@
1-
## Step 2. Getting JSON from CMX
1+
# Step 2\. Using the JSON Python Library
22

3-
### Asking for JSON instead of XML ###
3+
## Using Python's JSON object
44

5-
We don't want XML returned from CMX, so let's ask it to return JSON instead. We can accomplish this by telling CMX that we only want to accept JSON through the use of the "Accept" header in our request. To make this happen, all we need to do is add an "application/json" media type to the "Accept" header.
5+
The large JSON result that we get back and print out is just a string. It has no special properties beyond that and if we want to parse it we would parse it as a string. However, Python comes with a standard library, _json_, that allows you to encode/decode and access JSON content in patterns familiar to Python developers. When you load a string using the _json_ decoder, it converts JSON objects into Python [dictionaries](https://docs.python.org/3.4/reference/expressions.html#dictionary-displays "Python Dictionaries") and JSON arrays into Python [lists](https://docs.python.org/3.4/reference/expressions.html#list-displays "Python Lists"). This is useful because it then allows us to access and iterate the JSON content like Python content.
66

7-
1. Modify the **get-ap-json.py** file to include the following line immediately after the existing 'req.add_header' call adding the Authorization header:
8-
```
9-
req.add_header('Accept', 'application/json')
10-
```
11-
This line modifies our request to add a header stating that we want to accept a JSON result back from the server. If we wanted to get XML back we could have left it our (it is the default return format) or we could have explicitly added an Accept header that stated 'application/xml' was what we wanted returned.
12-
<br/>
13-
<br/>
14-
2. Save the **get-ap-json.py** file. If you want to just download or review the current code, you can get it from GitHub <a href="https://github.com/CiscoDevNet/coding-skills-sample-code/blob/master/coding202-parsing-json/get-ap-json-2.py" target="_blank">here</a>.
15-
2. Run the following command to execute your **get-ap-json.py** file:
16-
```
17-
python get-ap-json.py
18-
```
19-
3. When you run the Python script, you should get an screen full of JSON data returned to the terminal.
7+
Another useful tool of the json library is it's ability to "pretty print" the JSON output into a hierarchical structure to make it easier to see the overall structure of the data. Let's use that to print the data into a more human-readable output.
208

9+
1. Modify the **get-ap-json.py** file and and insert this import statement on the second line:
10+
11+
```
12+
import json
13+
```
14+
15+
This line loads the required _json_ library.<br>
16+
<br>
17+
18+
2. Comment out the print(response_string) line by putting a pound or hash sign at the front of the line like the following:
19+
20+
```
21+
#print(response_string)
22+
```
23+
24+
We are commenting out this line because we are going to use the _json_ library to pretty print our output.<br>
25+
<br>
26+
27+
3. Update the **get-ap-json.py** file and add the following line right after the commented out #print(response_string) line:
28+
29+
```
30+
json_object = json.loads(response_string)
31+
```
32+
33+
The json.loads() method loads the response_string as JSON. If it is successful, you can now get a particular element from the json_object with the square brackets. In short, this line loads the response string from the URI into a JSON object.<br><br>
34+
35+
4. Next, add the following after the line you just added:
36+
37+
```
38+
print(json.dumps(json_object, sort_keys=True, indent=4))
39+
```
40+
41+
This line prints the dumps() method from the json library. The arguments passed to the dumps() method control how the output is displayed.<br><br>
42+
43+
5. Save the **get-ap-json.py** file. If you want to just download or review the current code, you can get it from GitHub [here](https://github.com/CiscoDevNet/coding-skills-sample-code/blob/master/coding202-parsing-json/get-ap-json-3.py).
44+
45+
6. Run the following command to execute your newly created file:
46+
47+
```
48+
python get-ap-json.py
49+
```
50+
51+
7. When you run the Python script, you should get JSON data returned to the terminal.
2152

2253
<div style="text-align:center" markdown="1">
23-
![XML Command Prompt](/posts/files/coding-201-parsing-xml/xml-output.png)
54+
<img src="/posts/files/coding-202-parsing-json/json-output-pretty.png" alt="JSON Pretty Command Prompt">
2455
</div>
2556

26-
----------
57+
--------------------------------------------------------------------------------
58+
59+
The returned data is now a bit more readable an allows us to see its structure.
60+
61+
## Understanding the returned JSON data
62+
63+
Since we are asking for JSON data, the content we get back has a particular order or schema. Elements of the returned 'Floor' JSON data have sub-object and even sub-arrays. For this exercise, we are interested in the collection of Access Points for this floor and, in particular, the name and Radio MAC Address of each. There is other information returned, but this brief hierarchy to the name and radioMacAddress elements shows what we are interested in.
64+
65+
{DevNetZone}<br>
66+
[accessPoints]<br>
67+
**name**<br>
68+
**radioMacAddress**<br>
69+
ethMacAddress<br>
70+
ipAddress<br>
71+
numOfSlots<br>
72+
apMode<br>
73+
mapCoordinates<br>
74+
...
75+
76+
In the above description, accessPoints is an array of multiple AccessPoint instances - each with their own **name**, **radioMacAddress**, and other attributes.
2777

28-
If we really look closely, the returned text is no longer XML but is now JSON. So, by changing a header element on our call into CMX we can get the right format of data back. Unfortunately, it is still a compact representation of the JSON and hard to read. In the next step, let's take a look at the returned data and format it for readability.
78+
Now that we have a little understanding of what the structure looks like, let's go to the next step and see how to get just the data we care about.

‎labs/coding-202-parsing-json/3.md

Lines changed: 25 additions & 54 deletions
Original file line numberDiff line numberDiff line change
@@ -1,68 +1,39 @@
1-
## Step 3. Using the JSON Python Library
1+
# Step 3\. Iterating through a JSON array in Python
22

3-
### Using Python's JSON object ###
3+
## Looping through a Python array
44

5-
The large JSON result that we get back and print out is just a string. It has no special properties beyond that and if we want to parse it we would parse it as a string. However, Python comes with a standard library, *json*, that allows you to encode/decode and access JSON content in patterns familiar to Python developers. When you load a string using the *json* decoder, it converts JSON objects into Python [dictionaries](https://docs.python.org/3.4/reference/expressions.html#dictionary-displays "Python Dictionaries") and JSON arrays into Python [lists](https://docs.python.org/3.4/reference/expressions.html#list-displays "Python Lists"). This is useful because it then allows us to access and iterate the JSON content like Python content.
5+
In the previous step, we decoded the JSON content into a _json_object_ variable. To loop through the Access Points within as an array, let's first get the Access Points using the _json_ library and then use a Python _for_ loop to go through each.
66

7-
Another useful tool of the json library is it's ability to "pretty print" the JSON output into a hierarchical structure to make it easier to see the overall structure of the data. Let's use that to print the data into a more human-readable output.
7+
1. Insert these three lines at the end of **get-ap-json.py** file but before the response.close() line. Be sure to use 4 spaces on the second line as python is sensitive to indentation.
88

9-
1. Modify the **get-ap-json.py** file and and insert this import statement on the second line:
10-
```
11-
import json
12-
```
13-
This line loads the required *json* library.
14-
<br/>
15-
<br/>
16-
2. Comment out the print(responseString) line by putting a pound or hash sign at the front of the line like the following:
17-
```
18-
#print(responseString)
19-
```
20-
We are commenting out this line because we are going to use the *json* library to pretty print our output.
21-
<br/>
22-
<br/>
23-
3. Update the **get-ap-json.py** file and add the following line right after the commented out #print(responseString) line:
24-
```
25-
jsonObject = json.loads(responseString)
26-
```
27-
The json.loads() method loads the responseString as JSON. If it is successful, you can now get a particular element from the jsonObject with the square brackets. In short, this line loads the response string from the URI into a JSON object.
28-
<br/>
29-
4. Next, add the following after the line you just added:
30-
```
31-
print(json.dumps(jsonObject, sort_keys=True, indent=4))
32-
```
33-
This line prints the dumps() method from the json library. The arguments passed to the dumps() method control how the output is displayed.
34-
<br/>
35-
5. Save the **get-ap-json.py** file. If you want to just download or review the current code, you can get it from GitHub <a href="https://github.com/CiscoDevNet/coding-skills-sample-code/blob/master/coding202-parsing-json/get-ap-json-3.py" target="_blank">here</a>.
36-
6. Run the following command to execute your newly created file:
37-
```
38-
python get-ap-json.py
39-
```
40-
7. When you run the Python script, you should get JSON data returned to the terminal.
9+
```
10+
access_points = json_object['accessPoints']
11+
for ap in access_points:
12+
print('Access Point: ' + ap['name'] + '\t mac: ' + ap['radioMacAddress'])
13+
```
4114

15+
In this snippet, we are:
4216

43-
<div style="text-align:center" markdown="1">
44-
![XML Command Prompt](/posts/files/coding-202-parsing-json/json-output-pretty.png)
45-
</div>
17+
- Referencing the AccessPoints inside the JSON Object as a Dictionary using square brackets
18+
- Loop through the accessPoints array printing out the information from each ap<br>
19+
<br>
4620

47-
----------
21+
2. Save the **get-ap-json.py** file. If you want to just download or review the current code, you can get it from GitHub [here](https://github.com/CiscoDevNet/coding-skills-sample-code/blob/master/coding202-parsing-json/get-ap-json-4.py).
4822

49-
The returned data is now a bit more readable an allows us to see its structure.
23+
3. Run the following command to execute your file.
5024

51-
### Understanding the returned JSON data ###
25+
```
26+
python get-ap-json.py
27+
```
5228

53-
Since we are asking for JSON data, the content we get back has a particular order or schema. Elements of the returned 'Floor' JSON data have sub-object and even sub-arrays. For this exercise, we are interested in the collection of Access Points for this floor and, in particular, the name, Ethernet MAC Address, and IP Address Address of each. There is other information returned, but this brief hierarchy to the name and ipAddress elements shows what we are interested in.
29+
4. When you run the Python script, at the end of the output you should get the JSON data values returned for each Access Point in a format similar to the following:
5430

55-
Floor
56-
&nbsp;&nbsp;&nbsp;&nbsp;[AccessPoint]
57-
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;**name**
58-
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;radioMacAddress
59-
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;**ethMacAddress**
60-
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;**ipAddress**
61-
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;numOfSlots
62-
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;apMode
63-
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;MapCoordinate
31+
<div style="text-align:center" markdown="1">
32+
<img src="/posts/files/coding-202-parsing-json/json-output-parse-001.png" alt="JSON Parse Command Prompt">
33+
</div>
6434

65-
In the above description, AccessPoint is an array of multiple AccessPoint instances - each with their own **name**, **ipAddress**, and other attributes.
35+
--------------------------------------------------------------------------------
6636

67-
Now that we have a little understanding of what the structure looks like, let's go to the next step and see how to get just the data we care about.
37+
This is a very direct way to use Python to get data from a REST service using JSON and parse out only the information we care about. Depending upon the capabilities of the REST API, you can also sometimes pass in query parameters or other information to reduce the scope of returned data or search for a single or group of specific items.
6838

39+
For more on what is available from CMX, look to the [Mobility Services Engine (MSE) API Documentation](https://developer.cisco.com/site/cmx-mobility-services/documents/api-reference-manual/). For a summary of resources let's move to the last step.

0 commit comments

Comments
 (0)