• scripts (model.js, client.js, universal services, megabundler, kiln plugins)
• templates (precompilation)
• styles (woah boy this is a large discussion)
• media/assets (fonts? definitely svgs and images, at a minimum)

https://github.com/yeoman/update-notifier

• check against standard template rules
• head components need comment with data-uri
• body components need one root element with data-uri={{ default _ref _self }}
• layouts need doctype

https://github.com/clay/clay-cli/blob/master/lib/cmd/touch.js#L21

"this would run GETs against x instances... OF <COMPONENT NAME>"

https://github.com/sindresorhus/log-symbols

• echo "domain.com/components/foo" | clay touch
• only need to support full component uri

This line will break on windows. use a library that determines if a filepath is absolute or not (make sure it handles ~)

Readme says import takes page URI or URL --page <uri or url> a specific page url, but it is throwing error when using the page URL.

Error: Cannot parse url for site prefix!
    at getUrlPrefix (/Users/mcroney/_dev/claycli/lib/utils/urls.js:37:11)
    at Object.uriToUrl (/Users/mcroney/_dev/claycli/lib/utils/urls.js:49:17)
    at importSingleUrl (/Users/mcroney/_dev/claycli/lib/cmd/import.js:132:22)
    at Object.handler (/Users/mcroney/_dev/claycli/lib/cmd/import.js:235:12)
    at Object.self.runCommand (/Users/mcroney/_dev/claycli/node_modules/yargs/lib/command.js:233:22)
    at Object.Yargs.self._parseArgs (/Users/mcroney/_dev/claycli/node_modules/yargs/yargs.js:990:30)
    at Object.get [as argv] (/Users/mcroney/_dev/claycli/node_modules/yargs/yargs.js:927:19)
    at Object.<anonymous> (/Users/mcroney/_dev/claycli/index.js:40:3)
    at Module._compile (module.js:570:32)
    at Object.Module._extensions..js (module.js:579:10)
    at Module.load (module.js:487:32)
    at tryModuleLoad (module.js:446:12)
    at Function.Module._load (module.js:438:3)
    at Module.runMain (module.js:604:10)
    at run (bootstrap_node.js:393:7)
    at startup (bootstrap_node.js:150:9)```

when we have backpressure and stuff, we should be able to debug the amount of http calls per second. one nice way to do that would be to log.debug() a sparkline with some info https://github.com/sindresorhus/sparkly

• move megabundler to claycli
• allow megabundler to work with client.js
• determine script compilation api (watch, minify, etc)

Since this is the first compilation aspect that we're bringing into claycli:

• determine how compilation should be scaffolded (streams? gulp? etc)
• write helper functions to deal with incremental compilation, file watching, etc

• remove file handling
• keep keys / site aliases
• expand site aliases to allow any urls (rename to urls?)
• change env variables to use CLAYCLI_ prefix

• export components, pages, users, lists, etc
• export to dispatches / yaml bootstraps
• layouts flag
• limit / offset
• export newline-delineated uris from stdin

Consider the following clay-paragraph instance:

{
  "_ref": "nymag.com/scienceofus/components/clay-paragraph/instances/ciwxnzevi000oj2y9h3ptv73o@published",
  "text": "The Big Five model is a very big deal to psychologists, and has been for decades. At the moment, it is the most respected, widely studied method for accomplishing the tricky task of summing up someone’s personality. It consists of five dimensions, and as the University of Oregon social psychologist Sanjay Srivastava <a href=\"http://pages.uoregon.edu/sanjay/bigfive.html#whatisit\" target=\"_blank\">explains on his website</a>, each is perhaps best understood as containing a bundle of traits:"
}

Notice the text that contains the string /pages, which is part of an <a> element and has nothing to do with a Clay resource. When importing the page containing this paragraph, its text field gets mangled into:

{
  "_ref": "localhost.thecut.com/components/clay-paragraph/instances/ciwxnzevi000oj2y9h3ptv73o",
  "text": "localhost.thecut.com/pages.uoregon.edu/sanjay/bigfive.html#whatisit” target=”_blank”>explains on his website</a>, each is perhaps best understood as containing a bundle of traits:"
}

Relates to this line

• _description
• _version (needed?)

When pages are imported using Clay CLI, they overwrite the layout.

I would like to be able to bootstrap feature branches before DNS is ready.

For example instead of
scratch bootstrap --amphora http://branch-setup.nymag.sites.aws.nymetro.com/daily/intelligencer ./first-run/di/

I'd like to do something like

scratch bootstrap --amphora http:///daily/intelligencer ./first-run/di/

when importing the following:

pages:
  style:
    layout: /components/cut-layout/instances/section
    url: /style/
    head:
      - /components/clay-meta-title/instances/style
      - /components/clay-meta-description/instances/style
      - /components/meta-image/instances/wwwthecut-index
      - /components/meta-icons/instances/wwwthecut
      - /components/clay-meta-site/instances/section
      - /components/clay-meta-url/instances/style
      - /components/clay-meta-keywords/instances/style
    top:
      - /components/ad-takeover-homepage/instances/style
      - /components/global-nav/instances/social-off
    main:
      - /components/cut-section-feed/instances/style

clay-cli does not append the host to the url path. When you try to publish the page via Kiln, Amphora throws an error saying the url is not valid.

• import from stdin (newline-delineated dispatch, yaml bootstraps)
• publish flag

• support --limit and --offset in every command that has queueable updates
• write update uris (and data?) to a temp file so it can be resumed

• investigate and document current workflows using ad-hoc / scratch-cli scripts
• establish use cases that are currently not being met by claycli

I know the bulk import functionality is coming soon, but a big problem I've encountered using the scratch cli with the --publish flag is that it still tries to publish pages that are unpublished on the source clay instance. It would be nice if on clay-cli, one could:

1. publish a page on import (using a --publish flag or something)
2. only import articles that are published on the source instance when using the --publish flag (on a bulk page import)

People have URL prop in page data, will convert to custom URL.

make sure we check for default site before throwing an error about not specifying the site

• rest → type
• urls → urlToUri
• chunk → validate
• clay → recursiveGet

I would like the ability to pass a url (for instance: http://nymag.com/selectall/2017/04/the-secrets-of-the-times-square-olive-garden-revealed.html) to clay-cli and discover possible reasons why the url might be returning a 404 or 500. Ideally this would:

1. Check that there is an existing uri for the published page
2. If so, check that the page has a published version
3. lint the layout of the page
4. lint all non-layout components on the page

and if any of these failed, it would display an error message explaining the problem. This would greatly assist in debugging imported pages, as QA and Product usually pass a broken page to dev by its url and checking all of these things can be a manual process.

rewrite api calls to use dispatches, use cascading PUTs as much as possible

write a consistent service to normalize and denormalize data (between dispatch and bootstrap formats)

write a more consistent service to add/remove prefixes from dispatch-formatted data

it's running config.normalizeSite, but it really should pass in the prefix (plus protocol and port) from the original url

Exporters:

• Clay Site
• Clay Page
• Clay Component

Importing With Scratch

Importing with Clay CLI

Test YAML

pages:
  index:
    layout: /components/two-column-layout/instances/index
    url: /
    main:
      - /components/newsfeed/instances/index

https://github.com/samverschueren/listr

• retries https://github.com/demmer/bluebird-retry
• throttling? petkaantonov/bluebird#570

Importing of this component is failing but the component is functioning properly in production: http://nymag.com/strategist/components/article/instances/cj44cbckd007l7pyeld0w06cg

• postcss
• sass-like?
• component style conventions
• npm-component style conventions
• compilation rules
• linting rules

use Clay-log instead of the logger that is being used now.

we might as well be consistent with the other clay things.

see here for usage

Optimize recursive linting by checking the .json endpoint for a "quick pass" check

• always be recursive
• allow linting against stdin
• allow limit/offset queries against pages index (linting against multiple pages)
• allow linting against components, pages, public urls, uris, users (?), files (?)

add comment to https://github.com/clay/clay-cli/blob/master/lib/cmd/touch.js#L65 explaining why we're debugging here --> it's because it'll attempt to use the site you specify as a url if you accidentally didn't add it to your config

default https://github.com/clay/clay-cli/blob/master/lib/cmd/touch.js#L30 to true, since that's literally the whole reason we're running the command

• watch
• immersive watch
• coverage
• mocks
• snapshots

Importers:

• stdin → Clay
• Clay Site → Clay
• Clay Page → Clay
• Clay Component → Clay
• Files(s) → Clay

All links in the README point to github.com/nymag/claycli.

clay config sites should show all sites in the config, plz.

e.g. WARNING: target article[main] has 4 items, your YAML has 3. Continue y/N?

figure out the intermediary format for importers and exporters

• how does it work with streams?
• how does it work with stdin?
• how do we parse yaml/json into it?
• how do we iterate over it without running out of memory?
• how do we lint it?

• we should allow all of the claycli methods to be used programmatically
• individual commands should be importable, lodash-style (e.g. require('claycli/import'))
• full api should be importable, lodash-style (e.g. require('claycli').import)
• programmatic claycli should still use streams (and use all of the logic in io/) under the hood, but maybe should expose a promise-based api?