book

Github API

The objective is to learn how to use Github's API to pull in real data. Your need to figure out how to compose an URL for accessing a particular API to grab the data you want.

Github provides a wide range of APIs. We will focus on these three for now.

Carefully go through the examples below.

What is the data of the first issue?

Data is loaded from https://api.github.com/repos/bigdatahci2015/forum/issues/1

The data is saved in data. It looks like below.

GET /repos/:owner/:repo

{
 "url": "https://api.github.com/repos/bigdatahci2015/forum/issues/1",
 "labels_url": "https://api.github.com/repos/bigdatahci2015/forum/issues/1/labels{/name}",
 "comments_url": "https://api.github.com/repos/bigdatahci2015/forum/issues/1/comments",
 "events_url": "https://api.github.com/repos/bigdatahci2015/forum/issues/1/events",
 "html_url": "https://github.com/bigdatahci2015/forum/issues/1",
 "id": 102129972,
 "number": 1,
 "title": "Introduction",
 "user": {
  "login": "doubleshow",
  "id": 129562,
  "avatar_url": "https://avatars.githubusercontent.com/u/129562?v=3",
  "gravatar_id": "",
  "url": "https://api.github.com/users/doubleshow",
  "html_url": "https://github.com/doubleshow",
  "followers_url": "https://api.github.com/users/doubleshow/followers",
  "following_url": "https://api.github.com/users/doubleshow/following{/other_user}",
  "gists_url": "https://api.github.com/users/doubleshow/gists{/gist_id}",
  "starred_url": "https://api.github.com/users/doubleshow/starred{/owner}{/repo}",
  "subscriptions_url": "https://api.github.com/users/doubleshow/subscriptions",
  "organizations_url": "https://api.github.com/users/doubleshow/orgs",
  "repos_url": "https://api.github.com/users/doubleshow/repos",
  "events_url": "https://api.github.com/users/doubleshow/events{/privacy}",
  "received_events_url": "https://api.github.com/users/doubleshow/received_events",
  "type": "User",
  "site_admin": false
 },
 "labels": [],
 "state": "open",
 "locked": false,
 "assignee": null,
 "milestone": null,
 "comments": 28,
 "created_at": "2015-08-20T12:43:48Z",
 "updated_at": "2015-09-11T13:57:02Z",
 "closed_at": null,
 "body": "Name: Tom Yeh\r\nDepartment: Computer Science\r\nFavorite Programming Language: Javascript\r\nFavorite Food: BBQ",
 "closed_by": null
}

What is the title of the first issue?

https://api.github.com/repos/bigdatahci2015/forum/issues/1

Introduction

Who created this issue?

doubleshow

What are the comments of this issue?

Data is loaded from https://api.github.com/repos/bigdatahci2015/forum/issues/1/comments

The comments data look like

[
 {
  "url": "https://api.github.com/repos/bigdatahci2015/forum/issues/comments/133203904",
  "html_url": "https://github.com/bigdatahci2015/forum/issues/1#issuecomment-133203904",
  "issue_url": "https://api.github.com/repos/bigdatahci2015/forum/issues/1",
  "id": 133203904,
  "user": {
   "login": "willzfarmer",
   "id": 546524,
   "avatar_url": "https://avatars.githubusercontent.com/u/546524?v=3",
   "gravatar_id": "",
   "url": "https://api.github.com/users/willzfarmer",
   "html_url": "https://github.com/willzfarmer",
   "followers_url": "https://api.github.com/users/willzfarmer/followers",
   "following_url": "https://api.github.com/users/willzfarmer/following{/other_user}",
   "gists_url": "https://api.github.com/users/willzfarmer/gists{/gist_id}",
   "starred_url": "https://api.github.com/users/willzfarmer/starred{/owner}{/repo}",
   "subscriptions_url": "https://api.github.com/users/willzfarmer/subscriptions",
   "organizations_url": "https://api.github.com/users/willzfarmer/orgs",
   "repos_url": "https://api.github.com/users/willzfarmer/repos",
   "events_url": "https://api.github.com/users/willzfarmer/events{/privacy}",
   "received_events_url": "https://api.github.com/users/willzfarmer/received_events",
   "type": "User",
   "site_admin": false
  },
  "created_at": "2015-08-20T22:39:50Z",
  "updated_at": "2015-08-20T22:39:50Z",
  "body": "Name: William Farmer\r\nMajor: Applied Math w/ Minor in Computer Science\r\nFavorite Programming Language: Python\r\nFavorite Food: Sushi"
 },
 {
  "url": "https://api.github.com/repos/bigdatahci2015/forum/issues/comments/133227037",
  "html_url": "https://github.com/bigdatahci2015/forum/issues/1#issuecomment-133227037",
  "issue_url": "https://api.github.com/repos/bigdatahci2015/forum/issues/1",
  "id": 133227037,
  "user": {
   "login": "twagar95",
   "id": 10174237,
   "avatar_url": "https://avatars.githubusercontent.com/u/10174237?v=3",
   "gravatar_id": "",
   "url": "https://api.github.com/users/twagar95",
   "html_url": "https://github.com/twagar95",
   "followers_url": "https://api.github.com/users/twagar95/followers",
   "following_url": "https://api.github.com/users/twagar95/following{/other_user}",
   "gists_url": "https://api.github.com/users/twagar95/gists{/gist_id}",
   "starred_url": "https://api.github.com/users/twagar95/starred{/owner}{/repo}",
   "subscriptions_url": "https://api.github.com/users/twagar95/subscriptions",
   "organizations_url": "https://api.github.com/users/twagar95/orgs",
   "repos_url": "https://api.github.com/users/twagar95/repos",
   "events_url": "https://api.github.com/users/twagar95/events{/privacy}",
   "received_events_url": "https://api.github.com/users/twagar95/received_events",
   "type": "User",
   "site_admin": false
  },
  "created_at": "2015-08-21T00:32:27Z",
  "updated_at": "2015-08-21T00:32:27Z",
  "body": "Name: Tristan Wagar\r\nMajor: Computer Science\r\nFavorite Programming Language: Javascript\r\nFavorite Food: Steak"
 },
 {
  "url": "https://api.github.com/repos/bigdatahci2015/forum/issues/comments/133571232",
  "html_url": "https://github.com/bigdatahci2015/forum/issues/1#issuecomment-133571232",
  "issue_url": "https://api.github.com/repos/bigdatahci2015/forum/issues/1",
  "id": 133571232,
  "user": {
   "login": "Malaokia",
   "id": 10319733,
   "avatar_url": "https://avatars.githubusercontent.com/u/10319733?v=3",
   "gravatar_id": "",
   "url": "https://api.github.com/users/Malaokia",
   "html_url": "https://github.com/Malaokia",
   "followers_url": "https://api.github.com/users/Malaokia/followers",
   "following_url": "https://api.github.com/users/Malaokia/following{/other_user}",
   "gists_url": "https://api.github.com/users/Malaokia/gists{/gist_id}",
   "starred_url": "https://api.github.com/users/Malaokia/starred{/owner}{/repo}",
   "subscriptions_url": "https://api.github.com/users/Malaokia/subscriptions",
   "organizations_url": "https://api.github.com/users/Malaokia/orgs",
   "repos_url": "https://api.github.com/users/Malaokia/repos",
   "events_url": "https://api.github.com/users/Malaokia/events{/privacy}",
   "received_events_url": "https://api.github.com/users/Malaokia/received_events",
   "type": "User",
   "site_admin": false
  },
  "created_at": "2015-08-21T21:44:26Z",
  "updated_at": "2015-08-21T21:44:26Z",
  "body": "Name: Mingqi Liew\r\nMajor: Computer Science\r\nFavorite Programming Language: C\r\nFavorite Food: Thai food"
 },
 {
  "url": "https://api.github.com/repos/bigdatahci2015/forum/issues/comments/133776171",
  "html_url": "https://github.com/bigdatahci2015/forum/issues/1#issuecomment-133776171",
  "issue_url": "https://api.github.com/repos/bigdatahci2015/forum/issues/1",
  "id": 133776171,
  "user": {
   "login": "fadhilfath",
   "id": 5599246,
   "avatar_url": "https://avatars.githubusercontent.com/u/5599246?v=3",
   "gravatar_id": "",
   "url": "https://api.github.com/users/fadhilfath",
   "html_url": "https://github.com/fadhilfath",
   "followers_url": "https://api.githu

There are 28 comments.

return _.pluck(data, 'user.login')

The github account names are willzfarmer,twagar95,Malaokia,fadhilfath,hswitte,ZachLamb,zhya215,jocr1627,drewdinger,94kazakov,nicolele,satchelspencer,tiro6090,Linenfelser,KevinKGifford,calebhsu,sumi6109,pail4944,DomoYeti,boanding,kjblakemore,SankethSukumarShetty,AndreyShprengel,co-bri,karisantos,anbe6083,jraesly,doubleshow.

Exercises

Now it's your turn to answer the following questions using the real data.

How many issues have been created to date in our class's forum repository?

Data is loaded from https://api.github.com/repos/bigdatahci2015/forum/issues

[
 {
  "url": "https://api.github.com/repos/bigdatahci2015/forum/issues/30",
  "labels_url": "https://api.github.com/repos/bigdatahci2015/forum/issues/30/labels{/name}",
  "comments_url": "https://api.github.com/repos/bigdatahci2015/forum/issues/30/comments",
  "events_url": "https://api.github.com/repos/bigdatahci2015/forum/issues/30/events",
  "html_url": "https://github.com/bigdatahci2015/forum/issues/30",
  "id": 109531136,
  "number": 30,
  "title": "Question about using Mongo DB for the android app project",
  "user": {
   "login": "co-bri",
   "id": 5174490,
   "avatar_url": "https://avatars.githubusercontent.com/u/5174490?v=3",
   "gravatar_id": "",
   "url": "https://api.github.com/users/co-bri",
   "html_url": "https://github.com/co-bri",
   "followers_url": "https://api.github.com/users/co-bri/followers",
   "following_url": "https://api.github.com/users/co-bri/following{/other_user}",
   "gists_url": "https://api.github.com/users/co-bri/gists{/gist_id}",
   "starred_url": "https://api.github.com/users/co-bri/starred{/owner}{/repo}",
   "subscriptions_url": "https://api.github.com/users/co-bri/subscriptions",
   "organizations_url": "https://api.github.com/users/co-bri/orgs",
   "repos_url": "https://api.github.com/users/co-bri/repos",
   "events_url": "https://api.github.com/users/co-bri/events{/privacy}",
   "received_events_url": "https://api.github.com/users/co-bri/received_events",
   "type": "User",
   "site_admin": false
  },
  "labels": [],
  "state": "open",
  "locked": false,
  "assignee": null,
  "milestone": null,
  "comments": 0,
  "created_at": "2015-10-02T17:00:31Z",
  "updated_at": "2015-10-02T17:00:31Z",
  "closed_at": null,
  "body": "Khalid, it looks like the data is relatively structured and could have been put into an SQL data base (MySQL?) for queries.  Why did you choose to use a NoSQL database and why MongoDB versus others?"
 },
 {
  "url": "https://api.github.com/repos/bigdatahci2015/forum/issues/29",
  "labels_url": "https://api.github.com/repos/bigdatahci2015/forum/issues/29/labels{/name}",
  "comments_url": "https://api.github.com/repos/bigdatahci2015/forum/issues/29/comments",
  "events_url": "https://api.github.com/repos/bigdatahci2015/forum/issues/29/events",
  "html_url": "https://github.com/bigdatahci2015/forum/issues/29",
  "id": 109196520,
  "number": 29,
  "title": "Trouble getting mongodb  data",
  "user": {
   "login": "co-bri",
   "id": 5174490,
   "avatar_url": "https://avatars.githubusercontent.com/u/5174490?v=3",
   "gravatar_id": "",
   "url": "https://api.github.com/users/co-bri",
   "html_url": "https://github.com/co-bri",
   "followers_url": "https://api.github.com/users/co-bri/followers",
   "following_url": "https://api.github.com/users/co-bri/following{/other_user}",
   "gists_url": "https://api.github.com/users/co-bri/gists{/gist_id}",
   "starred_url": "https://api.github.com/users/co-bri/starred{/owner}{/repo}",
   "subscriptions_url": "https://api.github.com/users/co-bri/subscriptions",
   "organizations_url": "https://api.github.com/users/co-bri/orgs",
   "repos_url": "https://api.github.com/users/co-bri/repos",
   "events_url": "https://api.github.com/users/co-bri/events{/privacy}",
   "received_events_url": "https://api.github.com/users/co-bri/received_events",
   "type": "User",
   "site_admin": false
  },
  "labels": [],
  "state": "open",
  "locked": false,
  "assignee": null,
  "milestone": null,
  "comments": 5,
  "created_at": "2015-09-30T22:50:11Z",
  "updated_at": "2015-10-01T14:23:02Z",
  "closed_at": null,
  "body": "I installed mongodb without any problems but I can't download the data.  \r\nNone of the suggestions I found on-line for this worked for me. (There were not many)\r\nHere is my command and the error:\r\n$mongodump --host ds041663.mongolab.com --port 41663 --username bigdatahci --password bigdatahci2015 --out /Users/brian/bmckean/week6\r\n2015-09-30T12:38:10.109-0600\tFailed: error connecting to db server: server returned error on SASL authentication step: Authentication failed."
 },
 {
  "url": "https://api.github.com/repos/bigdatahci2015/forum/issues/27",
  "labels_url": "https://api.github.com/repos/bigdatahci2015/forum/issues/27/labels{/name}",
  "comments_url": "https://api.github.com/repos/bigdatahci2015/forum/issues/27/comments",
  "events_url": "https://api.github.com/repos/bigdatahci2015/forum/issues/27/events",
  "html_url": "https://github.com/bigdatahci2015/forum/issues/27",
  "id": 109140700,
  "number": 27,
  "title": "adding axes to SVG charts",
  "user": {
   "login": "karisantos",
   "id": 6976121,
   "avatar_url": "https://avatars.githubusercontent.com/u/6976121?v=3",
   "gravatar_id": "",
   "url": "https://api.github.com/users/karisantos",
   "html_url": "https://github.com/karisantos",
   "followers_url": "https://api.github.com/users/karisantos/followers",
   "following_url": "https://api.github.com/users/karisantos/following{/other_user}",
   "gists_url": "https://api.github.com/users/karisantos/g

return data.length

24 has issues has been created.

What are the titles of these issues?

return _.pluck(data, "title")

Question about using Mongo DB for the android app project,Trouble getting mongodb data,adding axes to SVG charts,Week 6 team homework is released,serving gitbook's specific pages,Viz drills are posted,Questions for the Birdstrike Hackathon,Corrupt file in .git,Converting JSON to Tableau ready (on OSX),Week 4 Learning Challenge ,Your FCQ question,Everyone's book,This week's challenge and next week's hackathon,Week 3 Learning Challenge Updated / Posted,Tableau,lodash variables,Week 2 Learning Challenge Updated / Posted,Week 2 Team Learning Clarification,Knowing Your Classmates Hackathon,Problems uploading book to github.io,Hackathons,First Learning Challenge (Update),Learning challenges,Introduction is the title of the issue

How many repository have been created to date for our class?

Our class's Github organization is bigdatahci2015.

Data is loaded from https://api.github.com/users/bigdatahci2015/repos

[
 {
  "id": 41165825,
  "name": "book",
  "full_name": "bigdatahci2015/book",
  "owner": {
   "login": "bigdatahci2015",
   "id": 13886111,
   "avatar_url": "https://avatars.githubusercontent.com/u/13886111?v=3",
   "gravatar_id": "",
   "url": "https://api.github.com/users/bigdatahci2015",
   "html_url": "https://github.com/bigdatahci2015",
   "followers_url": "https://api.github.com/users/bigdatahci2015/followers",
   "following_url": "https://api.github.com/users/bigdatahci2015/following{/other_user}",
   "gists_url": "https://api.github.com/users/bigdatahci2015/gists{/gist_id}",
   "starred_url": "https://api.github.com/users/bigdatahci2015/starred{/owner}{/repo}",
   "subscriptions_url": "https://api.github.com/users/bigdatahci2015/subscriptions",
   "organizations_url": "https://api.github.com/users/bigdatahci2015/orgs",
   "repos_url": "https://api.github.com/users/bigdatahci2015/repos",
   "events_url": "https://api.github.com/users/bigdatahci2015/events{/privacy}",
   "received_events_url": "https://api.github.com/users/bigdatahci2015/received_events",
   "type": "Organization",
   "site_admin": false
  },
  "private": false,
  "html_url": "https://github.com/bigdatahci2015/book",
  "description": "Everything I've done and learned from the Big Data HCI course",
  "fork": false,
  "url": "https://api.github.com/repos/bigdatahci2015/book",
  "forks_url": "https://api.github.com/repos/bigdatahci2015/book/forks",
  "keys_url": "https://api.github.com/repos/bigdatahci2015/book/keys{/key_id}",
  "collaborators_url": "https://api.github.com/repos/bigdatahci2015/book/collaborators{/collaborator}",
  "teams_url": "https://api.github.com/repos/bigdatahci2015/book/teams",
  "hooks_url": "https://api.github.com/repos/bigdatahci2015/book/hooks",
  "issue_events_url": "https://api.github.com/repos/bigdatahci2015/book/issues/events{/number}",
  "events_url": "https://api.github.com/repos/bigdatahci2015/book/events",
  "assignees_url": "https://api.github.com/repos/bigdatahci2015/book/assignees{/user}",
  "branches_url": "https://api.github.com/repos/bigdatahci2015/book/branches{/branch}",
  "tags_url": "https://api.github.com/repos/bigdatahci2015/book/tags",
  "blobs_url": "https://api.github.com/repos/bigdatahci2015/book/git/blobs{/sha}",
  "git_tags_url": "https://api.github.com/repos/bigdatahci2015/book/git/tags{/sha}",
  "git_refs_url": "https://api.github.com/repos/bigdatahci2015/book/git/refs{/sha}",
  "trees_url": "https://api.github.com/repos/bigdatahci2015/book/git/trees{/sha}",
  "statuses_url": "https://api.github.com/repos/bigdatahci2015/book/statuses/{sha}",
  "languages_url": "https://api.github.com/repos/bigdatahci2015/book/languages",
  "stargazers_url": "https://api.github.com/repos/bigdatahci2015/book/stargazers",
  "contributors_url": "https://api.github.com/repos/bigdatahci2015/book/contributors",
  "subscribers_url": "https://api.github.com/repos/bigdatahci2015/book/subscribers",
  "subscription_url": "https://api.github.com/repos/bigdatahci2015/book/subscription",
  "commits_url": "https://api.github.com/repos/bigdatahci2015/book/commits{/sha}",
  "git_commits_url": "https://api.github.com/repos/bigdatahci2015/book/git/commits{/sha}",
  "comments_url": "https://api.github.com/repos/bigdatahci2015/book/comments{/number}",
  "issue_comment_url": "https://api.github.com/repos/bigdatahci2015/book/issues/comments{/number}",
  "contents_url": "https://api.github.com/repos/bigdatahci2015/book/contents/{+path}",
  "compare_url": "https://api.github.com/repos/bigdatahci2015/book/compare/{base}...{head}",
  "merges_url": "https://api.github.com/repos/bigdatahci2015/book/merges",
  "archive_url": "https://api.github.com/repos/bigdatahci2015/book/{archive_format}{/ref}",
  "downloads_url": "https://api.github.com/repos/bigdatahci2015/book/downloads",
  "issues_url": "https://api.github.com/repos/bigdatahci2015/book/issues{/number}",
  "pulls_url": "https://api.github.com/repos/bigdatahci2015/book/pulls{/number}",
  "milestones_url": "https://api.github.com/repos/bigdatahci2015/book/milestones{/number}",
  "notifications_url": "https://api.github.com/repos/bigdatahci2015/book/notifications{?since,all,participating}",
  "labels_url": "https://api.github.com/repos/bigdatahci2015/book/labels{/name}",
  "releases_url": "https://api.github.com/repos/bigdatahci2015/book/releases{/id}",
  "created_at": "2015-08-21T16:24:10Z",
  "updated_at": "2015-08-24T02:46:53Z",
  "pushed_at": "2015-10-02T21:43:45Z",
  "git_url": "git://github.com/bigdatahci2015/book.git",
  "ssh_url": "git@github.com:bigdatahci2015/book.git",
  "clone_url": "https://github.com/bigdatahci2015/book.git",
  "svn_url": "https://github.com/bigdatahci2015/book",
  "homepage": null,
  "size": 87840,
  "stargazers_count": 0,
  "watchers_count": 0,
  "language": "HTML",
  "has_issues": true,
  "has_downloads": true,
  "has_wiki": true,
  "has_pages": false,
  "forks_count": 32,
  "mirror_url": null,
  "open_issues_count": 9,
  "forks": 32,
  "open_

return data.length

5 number of repository.

What are the fork counts of our class's repositories?

data = _.pluck(data, "forks_count")

return _.size(data)

5 fork counts in our class's repositories.

How many public repositories does the user `doubleshow` have?

Data is loaded from https://api.github.com/users/doubleshow

{
 "login": "doubleshow",
 "id": 129562,
 "avatar_url": "https://avatars.githubusercontent.com/u/129562?v=3",
 "gravatar_id": "",
 "url": "https://api.github.com/users/doubleshow",
 "html_url": "https://github.com/doubleshow",
 "followers_url": "https://api.github.com/users/doubleshow/followers",
 "following_url": "https://api.github.com/users/doubleshow/following{/other_user}",
 "gists_url": "https://api.github.com/users/doubleshow/gists{/gist_id}",
 "starred_url": "https://api.github.com/users/doubleshow/starred{/owner}{/repo}",
 "subscriptions_url": "https://api.github.com/users/doubleshow/subscriptions",
 "organizations_url": "https://api.github.com/users/doubleshow/orgs",
 "repos_url": "https://api.github.com/users/doubleshow/repos",
 "events_url": "https://api.github.com/users/doubleshow/events{/privacy}",
 "received_events_url": "https://api.github.com/users/doubleshow/received_events",
 "type": "User",
 "site_admin": false,
 "name": "Tom Yeh",
 "company": null,
 "blog": "http://tomyeh.info",
 "location": null,
 "email": "tom.yeh@colorado.edu",
 "hireable": null,
 "bio": null,
 "public_repos": 34,
 "public_gists": 531,
 "followers": 27,
 "following": 0,
 "created_at": "2009-09-21T15:06:30Z",
 "updated_at": "2015-09-28T19:14:24Z"
}

return data.public_repos

34 number of public repos.

How many public gists does the user `doubleshow` have?

return data.public_gists

531 number of doubleshow