API for Dashboard #329

ashmaroli · 2017-03-19T18:25:48Z

Proposing an API for the dashboard.
The API outputs a modified version the current site's site-wide payload:

remove file contents from data files, posts, pages, collections.
sort the pages by recent edits

{
  "content": null,
  "jekyll": {
    "version": "3.4.2",
    "environment": "development"
  },
  "highlighter_prefix": null,
  "highlighter_suffix": null,
  "site": {
    "time": "2017-03-19 14:18:53 -0400",
    "tags": {},
    "pages": [
      "page.md",
      "page-dir/test/page2.md",
      "page-dir/page1.md",
      "assets/style.scss",
      "assets/app.coffee"
    ],
    "posts": [
      "2016-01-01-test-post.md",
      "test/2016-01-02-test2.md",
      "2016-02-01-test-post-2.md",
      "2016-03-01-test-post-3.md",
      "more posts/2016-04-01-post-within-subdirectory.md",
      "more posts/some more posts/2016-05-01-a-test-post-within-subdirectory.md",
      "more posts/some more posts/2016-05-02-another-test-post-within-subdirectory.md"
    ],
    "static_files": [
      "#<Jekyll::StaticFile:0x3cf4860>",
      "#<Jekyll::StaticFile:0x3cf47d0>",
      "#<Jekyll::StaticFile:0x3cf46e0>",
      "#<Jekyll::StaticFile:0x3cf4638>"
    ],
    "categories": {},
    "collections": [
      "posts",
      "puppies"
    ],
    "html_pages": [
      "page.md",
      "page1.md",
      "page2.md"
    ],
    "source": "[--redacted--]/spec/fixtures/site",
    "destination": "[--redacted--]/spec/fixtures/site/_site",
    "plugins_dir": "_plugins",
    "layouts_dir": "_layouts",
    "data_dir": "_data",
    "includes_dir": "_includes",
    "safe": false,
    "include": [
      ".htaccess"
    ],
    "exclude": [
      "Gemfile",
      "Gemfile.lock",
      "README.md"
    ],
    "keep_files": [
      ".git",
      ".svn"
    ],
    "encoding": "utf-8",
    "markdown_ext": "markdown,mkdown,mkdn,mkd,md",
    "show_drafts": null,
    "limit_posts": 0,
    "future": false,
    "unpublished": false,
    "whitelist": [],
    "gems": [
      "jekyll-admin"
    ],
    "markdown": "kramdown",
    "highlighter": "rouge",
    "lsi": false,
    "excerpt_separator": "\n\n",
    "incremental": false,
    "detach": false,
    "port": "4000",
    "host": "127.0.0.1",
    "baseurl": "",
    "show_dir_listing": false,
    "permalink": "date",
    "paginate_path": "/page:num",
    "timezone": null,
    "quiet": false,
    "verbose": true,
    "defaults": [
      {
        "scope": {
          "path": ""
        },
        "values": {
          "some_front_matter": "default"
        }
      }
    ],
    "liquid": {
      "error_mode": "warn"
    },
    "rdiscount": {
      "extensions": []
    },
    "redcarpet": {
      "extensions": []
    },
    "kramdown": {
      "auto_ids": true,
      "toc_levels": "1..6",
      "entity_output": "as_char",
      "smart_quotes": "lsquo,rsquo,ldquo,rdquo",
      "input": "GFM",
      "hard_wrap": false,
      "footnote_nr": 1,
      "syntax_highlighter": "rouge",
      "syntax_highlighter_opts": {},
      "coderay": {}
    },
    "title": "Your awesome title",
    "foo": "bar",
    "watch": true,
    "serving": true,
    "url": "http://localhost:4000",
    "related_posts": null,
    "data_files": [
      "data_file.yml",
      "template-config.yml"
    ]
  },
  "layout": null,
  "page": null,
  "paginator": null
}

benbalter · 2017-03-20T13:26:12Z

@ashmaroli can you explain a bit more about what's going on here? It looks like you're aggregating many existing endpoints into a single call?

ashmaroli · 2017-03-20T13:44:46Z

It looks like you're aggregating many existing endpoints into a single call

With this call, the current site's entire payload gets streamed in to the API after being stripped off any content data. Except for appearing to be similar to existing endpoints, this cumulative call is a different object.

We can either use this goldmine of data or further strip to make things leaner.

benbalter · 2017-03-20T13:58:00Z

Except for appearing to be similar to existing endpoints, this cumulative call is a different object.

Why do we need a different object here? Is there data in this endpoint not exposed in those other endpoints? Can we expose that data via the existing endpoints? Can we accomplish the same user experience with multiple calls? My concern here is that we're creating a purpose-build endpoint for a single view when the data is already exposed in subject-specific (and generalized endpoints).

ashmaroli · 2017-03-20T15:05:45Z

relevant data (IMO) not exposed via existing endpoints:

{
  "content": null,
+  "jekyll": {
+    "version": "3.4.2",
+    "environment": "development"
+  },
  "highlighter_prefix": null,
  "highlighter_suffix": null,
  "site": {
    "time": "2017-03-19 14:18:53 -0400",
    "tags": {},
    "pages": [
      "page.md",
      "page-dir/test/page2.md",
      "page-dir/page1.md",
      "assets/style.scss",
      "assets/app.coffee"
    ],
    "posts": [
      "2016-01-01-test-post.md",
      "test/2016-01-02-test2.md",
      "2016-02-01-test-post-2.md",
      "2016-03-01-test-post-3.md",
      "more posts/2016-04-01-post-within-subdirectory.md",
      "more posts/some more posts/2016-05-01-a-test-post-within-subdirectory.md",
      "more posts/some more posts/2016-05-02-another-test-post-within-subdirectory.md"
    ],
    "static_files": [
      "#<Jekyll::StaticFile:0x3cf4860>",
      "#<Jekyll::StaticFile:0x3cf47d0>",
      "#<Jekyll::StaticFile:0x3cf46e0>",
      "#<Jekyll::StaticFile:0x3cf4638>"
    ],
    "categories": {},
    "collections": [
      "posts",
      "puppies"
    ],
    "html_pages": [
      "page.md",
      "page1.md",
      "page2.md"
    ],
    "source": "[--redacted--]/spec/fixtures/site",
    "destination": "[--redacted--]/spec/fixtures/site/_site",
    "plugins_dir": "_plugins",
    "layouts_dir": "_layouts",
    "data_dir": "_data",
    "includes_dir": "_includes",
    "safe": false,
    "include": [
      ".htaccess"
    ],
    "exclude": [
      "Gemfile",
      "Gemfile.lock",
      "README.md"
    ],
    "keep_files": [
      ".git",
      ".svn"
    ],
    "encoding": "utf-8",
    "markdown_ext": "markdown,mkdown,mkdn,mkd,md",
    "show_drafts": null,
    "limit_posts": 0,
    "future": false,
    "unpublished": false,
    "whitelist": [],
    "gems": [
      "jekyll-admin"
    ],
    "markdown": "kramdown",
    "highlighter": "rouge",
    "lsi": false,
    "excerpt_separator": "\n\n",
    "incremental": false,
    "detach": false,
    "port": "4000",
    "host": "127.0.0.1",
    "baseurl": "",
    "show_dir_listing": false,
    "permalink": "date",
    "paginate_path": "/page:num",
    "timezone": null,
    "quiet": false,
    "verbose": true,
    "defaults": [
      {
        "scope": {
          "path": ""
        },
        "values": {
          "some_front_matter": "default"
        }
      }
    ],
    "liquid": {
      "error_mode": "warn"
    },
    "rdiscount": {
      "extensions": []
    },
    "redcarpet": {
      "extensions": []
    },
    "kramdown": {
      "auto_ids": true,
      "toc_levels": "1..6",
      "entity_output": "as_char",
      "smart_quotes": "lsquo,rsquo,ldquo,rdquo",
      "input": "GFM",
      "hard_wrap": false,
      "footnote_nr": 1,
      "syntax_highlighter": "rouge",
      "syntax_highlighter_opts": {},
      "coderay": {}
    },
    "title": "Your awesome title",
    "foo": "bar",
+    "watch": true,
    "serving": true,
    "url": "http://localhost:4000",
    "related_posts": null,
    "data_files": [
      "data_file.yml",
      "template-config.yml"
    ]
  },
  "layout": null,
  "page": null,
  "paginator": null
}

the above data also exposes the entire site.config hash that includes data defined by Jekyll::Configuration::DEFAULTS which is not exposed via Configuration API

Can we accomplish the same user experience with multiple calls?

I'm sorry, I don't have the stats to comment on this. The data files, html_pages, pages and posts objects in the data above are trimmed versions of existing API calls

benbalter · 2017-03-20T15:08:26Z

Nice. Thanks @ashmaroli. Since #320 watch should be disabled in all cases, but would a version or meta endpoint for the version and environment make sense?

ashmaroli · 2017-03-20T15:16:11Z

Since #320 watch should be disabled in all cases, but would a version or meta endpoint for the version and environment make sense?

yes, watch should be disabled, yet the data returned shows its set to true. Need to investigate further.
I think Jekyll version on the dashboard makes sense: The user need not refer their Gemfile.lock to see which version of Jekyll is currently in use. ~~environment: is not relevant as of now.~~

ashmaroli · 2017-03-22T13:49:40Z

a point in favor of outputting jekyll environment value via this API:

JekyllAdmin works locally both with default JEKYLL_ENV and with the variable set to any arbitrary value via JEKYLL_ENV=. So having the dashboard display this to the user is a good thing, especially when Jekyll.logger doesn't output this value to the user, as of this writing.

ashmaroli · 2017-03-22T13:53:27Z

watch returned true above owing to the --watch switch passed via script/test-server. I've added a commit to output this as false at all times.

ashmaroli · 2017-04-08T08:53:21Z

Any feedback here @mertkahyaoglu ?

mertkahyaoglu · 2017-04-08T10:13:38Z

If @benbalter also approves, we can merge this.

benbalter · 2017-04-10T15:31:21Z

@ashmaroli see my questions above

Why do we need a different object here? Is there data in this endpoint not exposed in those other endpoints? Can we expose that data via the existing endpoints? Can we accomplish the same user experience with multiple calls? My concern here is that we're creating a purpose-build endpoint for a single view when the data is already exposed in subject-specific (and generalized endpoints).

In my mind, the API should be modeled after the data, not the presentation. If there's data in this endpoint not exposed elsewhere, let's expose it where it makes sense, but I'm less inclined to add (and support) and new endpoint to support a specific view, especially when making additional local network calls are so cheap.

ashmaroli · 2017-04-10T17:21:11Z

@benbalter Thank you for the feedback.

the API should be modeled after the data, not the presentation

Excellent diagnosis. You caught me! 😃

If there's data in this endpoint not exposed elsewhere, let's expose it where it makes sense,

sounds reasonable 👍
Hmm... that leaves me with just.. data related to Jekyll Drop.. 😒

ashmaroli added 3 commits March 19, 2017 23:36

extract mtimes of Pages and Documents to API

88afdba

initialize API for dashboard

c2f7dab

test dashboard API

a04644e

ashmaroli force-pushed the dashboard-api branch from 1859e6f to a04644e Compare March 20, 2017 01:39

ashmaroli mentioned this pull request Mar 20, 2017

Initialize dashboard front-end #330

Open

set payload[watch] to false

99f5cbb

mertkahyaoglu approved these changes Apr 8, 2017

View reviewed changes

ashmaroli closed this Apr 10, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

API for Dashboard #329

API for Dashboard #329

ashmaroli commented Mar 19, 2017

benbalter commented Mar 20, 2017

ashmaroli commented Mar 20, 2017

benbalter commented Mar 20, 2017

ashmaroli commented Mar 20, 2017

benbalter commented Mar 20, 2017

ashmaroli commented Mar 20, 2017 •

edited

Loading

ashmaroli commented Mar 22, 2017

ashmaroli commented Mar 22, 2017

ashmaroli commented Apr 8, 2017

mertkahyaoglu commented Apr 8, 2017

benbalter commented Apr 10, 2017

ashmaroli commented Apr 10, 2017

API for Dashboard #329

API for Dashboard #329

Conversation

ashmaroli commented Mar 19, 2017

benbalter commented Mar 20, 2017

ashmaroli commented Mar 20, 2017

benbalter commented Mar 20, 2017

ashmaroli commented Mar 20, 2017

benbalter commented Mar 20, 2017

ashmaroli commented Mar 20, 2017 • edited Loading

ashmaroli commented Mar 22, 2017

ashmaroli commented Mar 22, 2017

ashmaroli commented Apr 8, 2017

mertkahyaoglu commented Apr 8, 2017

benbalter commented Apr 10, 2017

ashmaroli commented Apr 10, 2017

ashmaroli commented Mar 20, 2017 •

edited

Loading