TeX is a typesetting system. It's quite helpful for scientific document as it makes the process of writing equations and compiling your bibliography.
To download the TeX software:
- Unix/Linux: TeX Live
- Mac: MacTex plus TeXShop frontend
- Windows: proTeXt (comes with TeXStudiohttp://texstudio.sourceforge.net) frontend)
In addition, I like to use Excel to format tables (and for easy viewing) and thus often use the Excel plug-in Excel2Latex.
To get started using TeX, you might look at the LaTeX tutorial PDF chapter in the /Tutorials/LaTeX/
directory of the Open Source Macro Bootcamp. This provides a great reference for installing and running LaTeX. Also included in that directory is a template LaTeX_probset_template.tex
as well as the PDf file (LaTeX_probset_template.pdf
) generated by compiling that .tex
file. You won't structure a document exactly like this template, but it will help you get started.
In our recommended Python development workflow, you will write Python scripts and modules (*.py
files) in a text editor. Then you will run those scripts from your terminal. You will want a capable text editor for developing your code. Many capable text editors exist, but I recommend two:
Both Visual Studio (VS) Code and Atom are completely free. In the following subsections, we provide some of the details about these text editors.
VS Code will be included with your installation of Anaconda. This is a very capable text editor and will include syntax highlighting for Python and and built in Git controls. In addition to the basics, you may want to use a more advanced linter for Python. This will help you correct syntax errors on the fly and provide helpful information as you declare objects and call functions. This link provides step-by-step instructions on using more advanced linting in VS Code.
Atom is an open source text editor developed by people at GitHub.com. This editor has all the features of Sublime Text 3, but it also allows users full customizability. Further, it has been a while now that the users of Atom have surpassed the critical mass necessary to keep the editor progressing with the most cutting edge additions.
There are several packages you'll want to install with Atom. Once Atom is installed, you can add packages by navigating Atom->Preferences->Install and then typing in the name of the package you would like to install.
For work with Python, I recommend the following packages be installed:
- MagicPython
- python-indent
- tabs-to-spaces
- minimap
- open-recent
- linter-python-pep8
For development with GitHub I recommend:
- merge-conflict
If using LaTex in this editor, the following packages are helpful:
- atom-latex
- latextools
- autocomplete-bitex
- dictionary
- latexer
- pdf-view
In addition, you will also want to download the Skim PDF viewer to aid in displaying PDF files compiled from TeX with Atom.
It's helpful to know your way around the Unix/DOS command line. This is the way I recommend interacting with Git (discussed below) and is also how I typically run Python scripts. It's also the only way you can typically interact with remote servers on which you might store data or run software (such as the Research Cyber Infrastructure clusters on campus).
Some of the most common commands you'll use are summarized in the table below.
Command | Unix | DOS |
---|---|---|
Change directory | cd <directory path> (could be relative path) |
cd |
List files in directory | ls |
dir |
Move up one level in directory structure | cd .. |
cd .. |
List current processes | ps |
tasklist |
Kill a running process | kill <process id> |
Taskkill /PID <process id> /F |
Connect to remote machine via secure shell | ssh -p <port number> <user@hostname> |
<path to PuTTY.exe> -ssh <username@host> <port number> |
Transfer files to a remote machine (via Secure Copy) | scp [options] <username1@source_host:directory1/filename1> <username2@destination_host:directory2/filename2> |
pscp -scp [options] <username1@source_host:directory1/filename1> <username2@destination_host:directory2/filename2> |
Submit a batch script | qsub <filename.sh> |
unlikely to do this. If need to, see here |
I have included a tutorial on using Git and GitHub.com. Git is a powerful version control software that comes natively installed on many machines and is widely used. GitHub.com is the most widely used online platform for hosting open source projects and integrating with Git software. Git has a significant learning curve, but it is essential for large collaborations that involve software development.
Functionality | Git Command |
---|---|
See active branch and uncommitted changes for tracked files | git status -uno |
Change branch | git checkout <branch name> |
Create new branch and change to it | git checkout -b <new branch name> |
Track file or latest changes to file | git add <filename> |
Commit changes to branch | git commit -m "message describing changes" |
Push committed changes to remote branch | git push origin <branch name> |
Merge changes from master into development branch | (change working branch to master, then…) git merge <branch name> |
Merge changes from development branch into master | (change to development branch, then…) git merge master |
List current tags | git tag |
Create a new tag | git tag -a v<version number> -m "message with new tag" |
Pull changes from remote repo onto local machine | git fetch upstream |
Merge changes from remote into active local branch | git merge upstream/<branch name> |
Clone a remote repository | git clone <url to remote repo> |
We will be using the Python programming language and many of its powerful libraries for writing the code to solve and estimate economic models. Using an open source language, such as Python, has the advantage of being free and accessible for anyone who wishes to contribute to this project. Being open source also allows Python users to go into the source code of any function to modify it to suit one's needs.
I recommend that you download the Anaconda distribution of Python provided by Anaconda. I recommend the most recent stable version of Python, which is currently Python 3.6. This can be done from the Anaconda download page for Windows, Mac OSX, and Linux machines. The code we will be writing uses common Python libraries such as NumPy
, SciPy
, pickle
, os
, matplotlib
, and time
.
Computer code executes some set of commands in an organized way. In every case, there are often many ways to execute a set of instructions--some ways more efficient than others. However, code has at least three functions.
- Efficiently execute the task at hand.
- Be accessible and usable to other programmers.
- Be scalable and integrable with other projects and procedures.
Bill Gates is credited with the following plea for efficiency and parsimony in code writing.
"Measuring programming progress by lines of code is like measuring aircraft building progress by weight."
Strong support for points (2) and (3) is Eagleson's Law.
"Any code of your own that you haven't looked at for six or more months might as well have been written by someone else."
Because of the latter two characteristics, Python code has developed some conventions and best practices, some of which have been institutionalized in the PEP 8--Style Guide for Python Code ("PEP" stands for Python Enhancement Proposals). Key examples PEP 8 Python coding conventions are the following.
- Indents should be 4 spaces (not tab)
- Limit all lines to a maximum of 79 characters long blocks of text being limited to 72 characters
- Use a space after a comma
- Use a space before and after arithmetic operators
In the text editors Atom and Sublime Text 3 you can install Linter packages that highlight areas of your code that break PEP 8 rules and tell you what the violation is.
Jupyter notebooks are files that end with the *.ipynb
suffix. These notebooks are opened in a browser environment and are an open source web application that combines instructional text with live executable and modifyable code for many different programming platforms (e.g., Python, R, Julia). Jupyter notebooks are an ideal tool for teaching programming as they provide the code for a user to execute and they also provide the context and explanation for the code. We have provided a number of Jupyter notebooks in the Tutorials folder of this repository.
These notebooks used to be Python-specific, and were therefore called iPython notebooks (hence the *.ipynb
suffix). But Jupyter notebooks now support many programming languages, although the name still pays homage to Python with the vestigal "py" in "Jupyter". The notebooks execute code from the kernel of the specific programming language on your local machine.
Jupyter notebooks capability will be automatically installed with your download of the Anaconda distribution of Python. If you did not download the Anaconda distribution of Python, you can download Jupyter notebooks separately by following the instructions on the Jupyter install page.
Once Jupyter is installed--whether through Anaconda or through the Jupyter website--you can open a Jupyter notebook by the following steps.
- Navigate in your terminal to the folder in which the Jupyter notebook files reside. In the case of the Jupyter notebook tutorials in this repository, you would navigate to the
~/WB-India-2019/Tutorials/
directory. - Type
jupyter notebook
at the terminal prompt. - A Jupyter notebook session will open in your browser, showing the available
*.ipynb
files in that directory.
- In some cases, you might receive a prompt in the terminal telling you to paste a url into your browser.
- Double click on the Jupyter notebook you would like to open.
It is worth noting that you can also simply navigate to the URL of the Jupyter notebook file in the GitHub repository on the web (e.g., https://github.com/OpenRG/WB-India-2019/blob/master/Tutorials/PythonReadIn.ipynb). You can read the Jupyter notebook on GitHub.com, but you cannot execute any of the cells. You can only execute the cells in the Jupyter notebook when you follow the steps above and open the file from a Jupyter notebook session in your browser.
Once you have opened a Jupyter notebook, you will find the notebook has two main types of cells: Markdown cells and Code cells. Markdown cells have formatted Jupyter notebook markdown text, and serve primarily to present context for the coding cells. A reference for the markdown options in Jupyter notebooks is found in the Jupyter markdown documentation page.
You can edit a Markdown cell in a Jupyter notebook by double clicking on the cell and then making your changes. Make sure the cell-type box in the middle of the top menu bar is set to Markdown
. To implement your changes in the Markdown cell, type Shift-Enter
.
A Code cell will have a In [ ]:
immediately to the left of the cell for input. The code in that cell can be executed by typing Shift-Enter
. For a Code cell, the cell-type box in the middle of the top menu bar says Code
.
When you are done with a Jupyter notebook, you first save any changes that you want to remain with the notebook. Then you close the browser windows associated with that Jupyter notebook session. You should then close the local server instance that was opened to run the Jupyter notebook in your terminal window. On a Mac or Windows, this is done by going to your terminal window and typing Cmd-C
or Ctrl-C
and then selecting y
for yes and hitting Enter
.
- Code and Data for the Social Sciences
- LaTeX math symbols
- Typesetting equations in TeX
- Pimp my Editor (Sublime Text focused, but may similar plug-ins/features available in Atom)
- Unix commands
- DOS commands
- Git Basics
- Git Workflows
- Research Cyberinfrastructure at USC