Updates to CSV import/export

.ruby-gemset

@@ -0,0 +1 @@
+csv_github_import_export

.ruby-version

@@ -0,0 +1 @@
+ruby-2.3.1

Gemfile

@@ -0,0 +1,5 @@
+source 'https://rubygems.org'
+ruby '2.3.1'
+
+gem 'octokit'
+gem 'highline'

Gemfile.lock

@@ -0,0 +1,26 @@
+GEM
+  remote: https://rubygems.org/
+  specs:
+    addressable (2.4.0)
+    faraday (0.9.2)
+      multipart-post (>= 1.2, < 3)
+    highline (1.7.8)
+    multipart-post (2.0.0)
+    octokit (4.3.0)
+      sawyer (~> 0.7.0, >= 0.5.3)
+    sawyer (0.7.0)
+      addressable (>= 2.3.5, < 2.5)
+      faraday (~> 0.8, < 0.10)
+
+PLATFORMS
+  ruby
+
+DEPENDENCIES
+  highline
+  octokit
+
+RUBY VERSION
+   ruby 2.3.1p112
+
+BUNDLED WITH
+   1.13.1

README.md

@@ -1,55 +1,54 @@
-Import-issues-to-GitHub-from-CSV
-================================
+# Import-issues-to-GitHub-from-CSV
 
-###Caveat
+# Caveat
 **You should use two-factor authentication for git**. 2FA breaks this script (it's beyond my coding skills to overcome that, at the moment). You have been warned.
 
-###Intro (WTF is this?)
+# Intro (WTF is this?)
 Product Owners and Project Mangers _love_ spreadsheets! They want everything in spreadsheet form.
 
 If you're using GitHub Issues (and we do) this means that one often needs to move things from GitHub into a spreadsheet ("I need a list of issues to show the client"), or from a spreadsheet into GitHub ("Here is the list of features we are committing to this sprint.") And, personally, I don't like doing things by hand if a computer can do them. And do them better.
 
 Thus this repo.
 
-There are two scripts here. One is for importing into GitHub Issues from a CSV file. The other is for getting issues out of GitHub and into a CSV file. 
+There are two scripts here. One is for importing into GitHub Issues from a CSV file. The other is for getting issues out of GitHub and into a CSV file.
 
 NOTE: currently, if you have two-factor authentication on, you'll have to turn it off for this to work. Which is a pain.
 
-###Documentation (The stuff you actually care about)
-
-#####Getting Started
+# Getting Started
 
+## Github Access
 Make sure you have a GitHub account, that you know your username and password, and that you have access to the repository (repo) that you want to import to, or from which you wish to export.
 
 You can use the "Clone in Desktop" or "Download Zip" functions on the GitHub page to download the files. If you want to do it using the command line, via ssh, use `git clone [email protected]:controlgroup/CSV-GitHub-import-export.git`
 
-You will also need to install [octokit](https://github.com/octokit/octokit.rb).  
+## Installation
+This utility is setup to specify a ruby version and create a gemset. It works with RVM. If you use something else make sure it uses the right Ruby and gets the gems installed. (If that requires changes, please send a pull request).
 
+When you `cd` into this project, RVM will set ruby to 2.3.1 and create a gemset named `csv_github_import_export`.
 
+To get the gems you need installed, do:
+```
+$ gem install bundler --no-ri --no-rdoc
+$ bundle install
+```
 
-#####Importing issues from a CSV file into GitHub using csv_issues_to_github.rb
+# Importing issues from a CSV file into GitHub using csv_issues_to_github.rb
 
 `./csv_issues_to_github.rb` to import issues *into* GitHub from a CSV
 
-There are two ways to use this. You use it interactively (the default) or you can hard-code your information into the script. If you want to switch, you need to comment out the former and un-comment the latter. 
+There are two ways to use this. You use it interactively (the default) or you can hard-code your information into the script. If you want to switch, you need to comment out the former and un-comment the latter.
 
 You will need to provide a CSV file to import.
 You will also need to provide it your GitHub username, your GitHub password, the name of your organization, and the name of your repository. If you're working on your own account on not as part of an org, the org is just your GitHub username.
 
 The CSV file MUST be in the following format:
 `title,description,assignee_username,label1,label2,label3`
 
-You can add more labels or remove labels to it if you wish. However...
-
-_**Important note**_: all label fields need to be filled. So, if you know you have an issue with only one label, then you need to have only one label field. You do this by removing (or you can add) labels fields at the end of the script. The line for this is marked with a comment (currently it's line 66, but that could change.)
-This is a known deficiency that will, with luck, eventually be fixed. 
+It's fine to leave label fields blank. The code will skip over blank labels.
 
 Fields that have commas in them need to be in double quotes. (Some punctuation doesn't require this, other punctuation may.)
 
-Either way you use this, you will have to provide your GH password. If you use it interactively, it will show your password on the command line. P-(
-
-
-#####Exporting issues from GitHub to a CSV file using github_issues_to_csv.rb
+# Exporting issues from GitHub to a CSV file using github_issues_to_csv.rb
 
 `./github_issues_to_csv.rb` to export issues *from* GitHub into a CSV
 
@@ -60,7 +59,7 @@ You can also export only a specific milestone if you give it a milestone. If you
 Note: It will put all labels in one column, without any delimiter. P-(
 
 
-###Credits (The smart people who did the real work)
+# Credits (The smart people who did the real work)
 It should be noted that I'm not much of a programmer, and I got a tremendous amount of help from [Vik](https://github.com/datvikash) and [Evan](https://github.com/evan108108)
 
 In addition `github_issues_to_csv` was adapted from the work of others, specifically: https://gist.github.com/henare/1106008 and https://gist.github.com/tkarpinski/2369729

csv_issues_to_github.rb

@@ -1,41 +1,44 @@
-#!/usr/local/bin/ruby
+#!/usr/bin/env ruby
 
 # See the README for how to use this.
 
 require 'rubygems'
 require 'octokit'
 require 'faraday'
 require 'csv'
+require 'highline/import'
+require 'pathname'
 
 # BEGIN INTERACTIVE SECTION
 # Comment out this section (from here down to where the end is marked) if you want to use this interactively
 
 puts "Username:"
-username = gets.chomp  
+username = gets.chomp
 if username == ""
 	abort("You need to supply a username. Thank you, come again.")
 end
 
-puts "Password:"
-password = gets.chomp  
+password = ask("Password:  ") { |q| q.echo = false }.chomp
 if password == ""
 	abort("You need to supply a password. Thank you, come again.")
 end
 
 puts "Path for the CSV file you want to use?"
-input_file = gets.chomp  
+input_file = gets.chomp
 if input_file == ""
 	abort("You need to supply a CSV file. Thank you, come again.")
 end
+full_path = Pathname.new(input_file).expand_path
+abort("Can't find file at that location: #{full_path}") unless full_path.exist?
 
 puts "Organization?"
-org = gets.chomp  
+org = gets.chomp
 if org == ""
 	abort("You need to supply an organization. Thank you, come again.")
 end
 
 puts "Repository?"
-repo = gets.chomp  
+repo = gets.chomp
 if repo == ""
 	abort("You need to supply a repository. Thank you, come again.")
 end
@@ -47,24 +50,41 @@
 # Un-comment out this section (from here down to where the end is marked) if you want to use this without any interaction
 # All of these need to be filled out in order for it to work
 =begin
-input_file = ""
 username = ""
 password = ""
-org = "" 
+full_path = Pathname.new('/path/to/your.csv').expand_path
+org = ""
 repo = ""
 =end  # END HARD-CODED SECTION
 
 org_repo = org + "/" + repo
 
 client = Octokit::Client.new(:login => username, :password => password)
 
-csv_text = File.read(input_file)
-csv = CSV.parse(csv_text, :headers => true)
+count = 0
+skipped = 0
 
-csv.each do |row|
-	client.create_issue(org_repo, row['title'], row['description'], options = {
-		:assignee => row['assignee_username'], 
-		:labels => [row['label1'],row['label2'],row['label3']]})  #Add or remove label columns here.
-	puts "Imported issue:  #{row['title']}"
-end
+CSV.foreach(full_path, headers: true) do |row|
+  count += 1
+
+  if row['title'] == ""
+    skipped += 1
+    next
+  end
+
+  puts "Importing issue: #{row['title']}: #{row['description']}"
+  options = {labels: []}
 
+  unless row['assignee_username'] == ""
+    options[:assignee] = row['assignee_username']
+  end
+
+  1.upto(3) do |i|
+    if row["label#{i}"] != "" && !row["label#{i}"].nil?
+      options[:labels] << row["label#{i}"]
+    end
+  end
+
+	client.create_issue(org_repo, row['title'], row['description'], options)
+end
+puts "Import completed. Processed #{count} issues. Skipped #{skipped}."

github_issues_to_csv.rb

@@ -1,45 +1,45 @@
-#!/usr/bin/ruby
+#!/usr/bin/env ruby
 
 require 'octokit'
 require 'csv'
 require 'date'
+require 'highline/import'
 
 
 # BEGIN INTERACTIVE SECTION
 # Comment out this section (from here down to where the end is marked) if you want to use this interactively
 
 puts "Username:"
-USERNAME = gets.chomp  
+USERNAME = gets.chomp
 if USERNAME == ""
 	abort("You need to supply a username. Thank you, come again.")
 end
 
-puts "Password:"
-PASSWORD = gets.chomp  
+PASSWORD = ask("Password:  ") { |q| q.echo = false }.chomp
 if PASSWORD == ""
 	abort("You need to supply a password. Thank you, come again.")
 end
 
 puts "Name of the file you want to create? (.csv will be appended automatically)"
-OUTPUT_FILE = gets.chomp  
+OUTPUT_FILE = gets.chomp
 if OUTPUT_FILE == ""
 	abort("You need to supply a CSV file. Thank you, come again.")
 end
 
 puts "Organization?"
-ORG = gets.chomp  
+ORG = gets.chomp
 if ORG == ""
 	abort("You need to supply an organization. Thank you, come again.")
 end
 
 puts "Repository?"
-REPO = gets.chomp  
+REPO = gets.chomp
 if REPO == ""
 	abort("You need to supply a repository. Thank you, come again.")
 end
 
 puts "Do you want just one Milestone? If so, enter it now. Leave this blank to get the entire repo."
-TARGET_MILESTONE = gets.chomp  
+TARGET_MILESTONE = gets.chomp
 
 # END INTERACTIVE SECTION
 
@@ -50,7 +50,7 @@
 =begin
 OUTPUT_FILE = ""
 USERNAME = ""   # Put your GitHub username inside the quotes
-PASSWORD = ""   # Put your GitHub password inside the quotes 
+PASSWORD = ""   # Put your GitHub password inside the quotes
 ORG = ""        # Put your organization (or username if you have no org) name here
 REPO = ""       # Put the repository name here
 # Want to only get a single milestone? Put the milestone name in here:
@@ -60,11 +60,11 @@
 
 # Your local timezone offset to convert times
 TIMEZONE_OFFSET="-4"
- 
+
 client = Octokit::Client.new(:login => USERNAME, :password => PASSWORD)
- 
+
 csv = CSV.new(File.open(File.dirname(__FILE__) + "/" + OUTPUT_FILE + ".csv", 'w'))
- 
+
 puts "Initialising CSV file..."
 #CSV Headers
 header = [
@@ -81,7 +81,7 @@
 ]
 
 csv << header
- 
+
 puts "Getting issues from Github..."
 temp_issues = []
 issues = []
@@ -98,29 +98,29 @@
 	temp_issues = client.list_issues("#{ORG}/#{REPO}", :state => "open", :page => page)
 	issues = issues + temp_issues;
 end while not temp_issues.empty?
- 
+
 puts "Processing #{issues.size} issues..."
 issues.each do |issue|
 
 	labels = ""
 	label = issue['labels'] || "None"
 	if (label != "None")
 		label.each do |item|
-    		labels += item['name'] + " " 
-    	end	
+    		labels += item['name'] + " "
+    	end
 	end
 
 	assignee = ""
 	assignee = issue['assignee'] || "None"
 	if (assignee != "None")
 		assignee = assignee['login']
-	end	
+	end
 
 	milestone = issue['milestone'] || "None"
 	if (milestone != "None")
 		milestone = milestone['title']
 	end
- 
+
 	if ((TARGET_MILESTONE == "") || (milestone == TARGET_MILESTONE))
     # Needs to match the header order above, date format are based on Jira default
     row = [