Giter VIP home page Giter VIP logo

plenti's Introduction

Plenti

Build-Time Render Engine (aka Static Site Generator) with Go backend and Svelte frontend
Ships with a fully integrated Git-CMS that you can host for cheap/free right with your static site

Requirements:exclamation:

You must have NodeJS version 13 or newer As of v0.2.0 you no longer need NodeJS, Go, or any dependency other than Plenti itself.

Installation 💾

Homebrew:

  1. Add the tap: brew tap plentico/homebrew-plenti
  2. Install: brew install plenti

Snap:

  1. Install: snap install plenti

Scoop (Windows is not supported yet, see details):

  1. Add the bucket: scoop bucket add plenti https://github.com/plentico/scoop-plenti
  2. Install: scoop install plenti

Manual:

  1. Download the latest release
  2. Move it somewhere in your PATH (most likely /usr/local/bin)

Getting Started 🚀

  1. Create a new site: plenti new site my-new-site
  2. Move into the folder you created: cd my-new-site
  3. Start up the development server: plenti serve
  4. Navigate to the site in your browser: localhost:3000

Learning the Basics 🎓

  1. Documentation: https://plenti.co/docs
  2. Videos: YouTube playlist

Contributing 💜

Plenti is still new, but constantly improving. The API might change from time to time before we hit a stable v1.0.x release. If you find bugs or have any questions, please open a new issue to let us know! If you want to help keep Plenti free from commercial interests, please consider making a donation so we can spend more time making improvements 🌱

plenti's People

Contributors

danedavid avatar dependabot[bot] avatar gustavocd avatar hlanderdev avatar jastuccio avatar jimafisk avatar joas8211 avatar khang859 avatar padraicbc avatar stephanieluz avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

plenti's Issues

Extend "new" cmd: Type

A dev should be able to create a new content type via the cli by running plenti new type <name>. This should:

  • Create a folder inside content/ with the type name
  • Create a _blueprint.json file inside the type folder created above (prompt user input for defaults?)
  • Create a corresponding svelte file in layout/content/ with the type name

The router.svelte should pick up on this automatically since it will get build into nodes.js.

Cannot import UI frameworks like sveltestrap or svelte-material-UI

I tried to add any kind of prebuilt components to test plenti (svelte material ui, sveltestrap, anything really).
I tried their examples by putting their sample code from their docs in /layout/content/index.svelte, and since I was having errors, I tried a bunch of syntaxes for import statements like:

import {Foo} from 'bar';
import Foo from 'bar';
import Foo from 'bar/src';

I also tried to install as dependency and devDependency.

I keep getting errors, the most frequent being:
SyntaxError: Cannot use import statement outside a module
I also get:
<SOME_COMPONENT_I_IMPORT> is not a valid SSR component. You may need to review your build config to ensure that dependencies are compiled, rather than imported as pre-compiled modules

any idea what I'm doing wrong?

Add flag to "new type" command for single file types

The plenti new type MY_TYPE command will create a folder at content/MY_TYPE. Since plenti allows for single file Types (like content/index.json and content/404.json) we should add a flag to this command to create a single type file instead of a content type folder. Something like --single should work: plenti new type whatever --single

Content driven components

Users should have the flexibility to load components based on the keys in their content structure (JSON).

Example JSON structure
{
  "title": "Anything",
  "components": [
    {
      "hero": {
        "slogan": "Something",
        "image": "/path/to/image.png"
      } 
    },
    {
      "grid": {
        "title": "Grid title",
        "items": [
          {"img": "/path/to/image1.png", "link": "/path/to/page1"},
          {"img": "/path/to/image2.png", "link": "/path/to/page2"},
          {"img": "/path/to/image3.png", "link": "/path/to/page3"},
        ],
      }
    },
    {
      "quote": "To be or not to be..."
    },
    {
      "slider": [
        {"scene": {"title": "whatever", "body": "more stuff"}},
        {"scene": {"title": "whatever", "body": "more stuff"}},
        {"scene": {"title": "whatever", "body": "more stuff"}},
      ]
    },
  ]
} 

In the example ^ there are 4 components: hero, grid, quote, slider. The challenge is dynamically loading the svelte component that corresponds to these using a simple text match since it will require having a component class instance.

In example, we're using the components key in the content source JSON. Ideally there will be a way to do this without having a required key.

Installation via snapcraft can't write to filesystem

Installing plenti manually by placing the binary at /usr/local/bin/plenti works on Ubuntu 20.04, however if instead you try using snap install plenti it appears that it can't write to the filesystem correctly. For example, running plenti new site test results in:

Unable to write file: open test/content/blog/_blueprint.json: no such file or directoryUnable to write file: open test/content/pages/about.json: no such file or directoryUnable to write file: open test/layout/components/template.svelte: no such file or directoryUnable to write file: open test/layout/content/404.svelte: no such file or directoryUnable to write file: open test/layout/global/head.svelte: no such file or directoryUnable to write file: open test/layout/scripts/load_component.svelte: no such file or directoryUnable to write file: open test/assets/favicon.svg: no such file or directoryUnable to write file: open test/assets/planarian.svg: no such file or directoryUnable to write file: open test/layout/scripts/sort_by_date.svelte: no such file or directoryUnable to write file: open test/content/blog/post2.json: no such file or directoryUnable to write file: open test/content/index.json: no such file or directoryUnable to write file: open test/layout/content/blog.svelte: no such file or directoryUnable to write file: open test/layout/content/index.svelte: no such file or directoryUnable to write file: open test/layout/global/footer.svelte: no such file or directoryUnable to write file: open test/layout/global/html.svelte: no such file or directoryUnable to write file: open test/content/blog/adding_pletiform.json: no such file or directoryUnable to write file: open test/content/blog/post1.json: no such file or directoryUnable to write file: open test/layout/global/nav.svelte: no such file or directoryUnable to write file: open test/content/pages/contact.json: no such file or directoryUnable to write file: open test/layout/content/pages.svelte: no such file or directoryUnable to write file: open test/layout/scripts/make_title.svelte: no such file or directoryUnable to write file: open test/plenti.json: no such file or directoryUnable to write file: open test/.gitignore: no such file or directoryUnable to write file: open test/content/pages/_blueprint.json: no such file or directoryUnable to write file: open test/layout/components/grid.svelte: no such file or directoryUnable to write file: open test/package.json: no such file or directoryCreated plenti site scaffolding in "test" folder
Installing NPM dependencies...
NPM install complete!

Note: If switching between installation methods, for example if you were to try to snap remove plenti then manually install you'll likely get an error like:

bash: /snap/bin/plenti: No such file or directory

To fix this you simply need to close out of your current terminal and reopen a new one.

External APIs

I've been chatting through some ideas with @Holben888:

I do think there's a place for dynamic content tho, like supporting Go files instead of JSON files in /content. These could export a function that gets executed at build time and returns content as JSON. Still think plugins at the CMS level are the way to go for popular APIs (Twitter, CodePen, etc.), but it would be nice to be able to manually poll APIs at build time as well!

Couldn't agree more. I don't know exactly how to approach accomplishing this, but I think it's good practice to discuss the experience we want achieve and work backwards to a technical solution.

I tend to agree with Ben and I think the data from external APIs should live in the content/ folder inside a Type just like any other content (vs breaking it out into a separate folder like data or api). The developer would use the field data like any other content and would not have to adjust their workflow. The potential downside is certain folders would be controlled by and API and if a user wasn't aware of this they might make edits to it that could potentially get overwritten on the next build.

It would be nice to be able to simply add "plugins" from the Plenti CLI to integrate with popular third party services. Something like plenti mesh salesforce for a supported plugin or plenti mesh salesforce --from="https://github.com/jimafisk/my-sf-plugin" for community contributed plugins. Not sure if "mesh" is the best keyword, could be: pull, sync, add, integrate, or even extending an existing command like new (e.g. plenti new plugin salesforce).

Integrated CMS

This will need to be converted into an epic with individual tasks, but for now just jotting down some ideas:

  • Should be git-backed (like NetlifyCMS or TinaCMS)
  • Built with svelte so any previews use site's native templates and only one framework is needed
  • Editing should be done on the page via .contentEditable
  • Widgets will be needed for things like links, buttons, images, and other non flat text data sources (this will be challenging to make work with flexible keys in data source)
  • The _blueprint.json file should define the type of widget that is used by the CMS
  • Local editing should write to project filesystem unless optional flag is passed to serve command, something like: --ignore-edits
  • Remote editing should create a commit in remote repo and kick off CI rebuild (would be nice to support Github, Gitlab, and Gogs)
  • Should have to optional flag to build a site without the CMS --cms=false

Down the road:

  • Support open authoring (automatic forks for non-members)
  • Support drafts (using PRs + review + merge)
  • Flexible theming of CMS components

Action items:

  1. Think through / mockup admin menu. This would likely be an edit/preview toggle and widget for adding new instances of a type.
  2. Brainstorm how to inject CMS during build... do we identify field values in templates and match them to their keys in the data source, then lookup the widget type from _blueprint.json, then wrap the field value in a custom component?

Client SPA can't hydrate

During development on the client build process I sometimes break the hydration process which makes the client spa routing stop working (the url changes, but the page content stays the same). I've hit this a couple of times and always forget what the fix is so I figured I'd document here for future reference.

Error messages in browser

FireFox:

Uncaught (in promise) DOMException: Element.replaceWith: Cannot insert a Text as a child of a Document

Chrome:

Uncaught (in promise) DOMException: Failed to execute 'replaceWith' on 'Element': Nodes of type '#document-fragment' may not be inserted inside nodes of type '#document'.
    at replaceContainer (http://localhost:3000/spa/ejected/main.js:20:20)
    at http://localhost:3000/spa/ejected/main.js:25:13

This often happens because in the built components (/public/spa/*) are simply referencing the svelte component filename, not the actual component. So the c() {t = text(whatever_component)}, will be different, but it doesn't actually have the context of the component. The build needs to pass the actual component to svelte.compile(), not just the filename or else the actual filename will be used as the component text.

Ejectable core files

There are several core javascript files that should be abstracted from the typical dev, namely:

  • layout/ejected/build.js (creates client SPA + static html files)
  • layout/ejected/main.js (entry point + SPA hydration)
  • layout/ejected/router.svelte (client routes + passes component class instance)

The advantage of hiding these files:

  1. Allows plenti team to update core behind the scenes and updates will apply to already created sites without having to replace those core files per site (assuming the dev hasn't ejected any core files).
  2. Avoids confusing devs with additional files that if updated without understanding the implications can break how plenti functions on a fundamental level.

Dev should be able to run plenti eject and be presented with a select list in the cli for the (currently 3) ejectable core files. They should also be able to eject a specific file by name, for example plenti eject router.svelte. This should give them a warning about the action voiding their warranty, and if they choose to continue it should write the file to the file system (kind of like these files are currently). Then when running commands like plenti build, the binary should check if any core files have been ejected, if so try to use them, if not use the hidden core files like normal.

Custom path replacement patterns

In the site config file (plenti.json) you can set a custom path using a replacement pattern for the content source JSON filename: :filename.

It would be nice to be able to use any key (field) from the Types content source, for example:

"types": {
  "pages": "/:field(title)"
}

Windows 64bit error on build and serve

Downloaded the Windows 64 bit version
plenti_0.1.26_Windows_64-bit.tar.gz

[Created a site using plenti on windows:]

PS C:\Code\www\examples> plenti new site plenti-test-site
Created plenti site scaffolding in "plenti-test-site" folder
Installing NPM dependencies...
npm notice created a lockfile as package-lock.json. You should commit this file.
added 4 packages from 3 contributors and audited 5 packages in 2.639s
found 0 vulnerabilities

NPM install complete!
PS C:\Code\www\examples> cd .\plenti-test-site\

When i run either serve or build I get the same error...

PS C:\Code\www\examples\plenti-test-site> plenti serve
Total build took 20.0303ms
panic: runtime error: index out of range [1] with length 1

goroutine 1 [running]:
plenti/cmd/build.DataSource.func1(0xc00001c940, 0x1c, 0xa54c00, 0xc000182700, 0x0, 0x0, 0x0, 0x0)
/home/runner/work/plenti/plenti/cmd/build/data_source.go:44 +0x151a
path/filepath.walk(0xc00001c940, 0x1c, 0xa54c00, 0xc000182700, 0xc00010fa00, 0x0, 0x0)
/opt/hostedtoolcache/go/1.14.4/x64/src/path/filepath/path.go:360 +0x42c
path/filepath.walk(0xc000020b20, 0xc, 0xa54c00, 0xc0001824d0, 0xc00010fa00, 0x0, 0x0)
/opt/hostedtoolcache/go/1.14.4/x64/src/path/filepath/path.go:384 +0x306
path/filepath.walk(0x985c03, 0x7, 0xa54c00, 0xc000182310, 0xc00010fa00, 0x0, 0x0)
/opt/hostedtoolcache/go/1.14.4/x64/src/path/filepath/path.go:384 +0x306
path/filepath.Walk(0x985c03, 0x7, 0xc00010fa00, 0xf, 0xf)
/opt/hostedtoolcache/go/1.14.4/x64/src/path/filepath/path.go:406 +0x106
plenti/cmd/build.DataSource(0xc000086480, 0x6, 0xc000086460, 0x6, 0xbb8, 0xc00009c4e0, 0x0, 0x0, 0x0, 0x0)
/home/runner/work/plenti/plenti/cmd/build/data_source.go:40 +0x3dd
plenti/cmd.Build()
/home/runner/work/plenti/plenti/cmd/build.go:92 +0x3ad
plenti/cmd.glob..func1(0xda7e60, 0xdddf08, 0x0, 0x0)
/home/runner/work/plenti/plenti/cmd/build.go:43 +0x27
plenti/cmd.glob..func4(0xda7e60, 0xdddf08, 0x0, 0x0)
/home/runner/work/plenti/plenti/cmd/serve.go:55 +0x54f
github.com/spf13/cobra.(*Command).execute(0xda7e60, 0xdddf08, 0x0, 0x0, 0xda7e60, 0xdddf08)
/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:842 +0x2a4
github.com/spf13/cobra.(*Command).ExecuteC(0xda7bc0, 0x444e01, 0xd6a100, 0xc000043f78)
/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:943 +0x31e
github.com/spf13/cobra.(*Command).Execute(...)
/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:883
plenti/cmd.Execute()
/home/runner/work/plenti/plenti/cmd/root.go:33 +0x38
main.main()
/home/runner/work/plenti/plenti/main.go:8 +0x27

Add a bare-bones starter flag

It's hard finding a balance for what the main starter should provide. Ideally it:

  1. has some useful features that most sites need
  2. demonstrates some basic functionality for new users
  3. isn't overwhelming

For power users, this boilerplate might be annoying to have to delete every time. Adding a --bare flag to the site cmd that clears out all the defaults besides the required structure would be nice. For example plenti new site my-new-site --bare

scoop install fails

I tried to use scoop to install plenti, but it fails as per below:

PS C:\code\www\examples> scoop bucket add org https://github.com/plentico/scoop-plenti.git
WARN The 'org' bucket already exists. Use 'scoop bucket rm org' to remove it.
PS C:\code\www\examples> scoop install plentico/scoop-plenti
WARN Bucket 'plentico' not installed. Add it with 'scoop bucket add plentico' or 'scoop bucket add plentico '.
Couldn't find manifest for 'scoop-plenti' from 'plentico' bucket.

`plenti serve` on fresh site creation doesnt build

Wanted to test this out so did created a new site (both when installed via brew, or installed manually didn't make a difference).

plenti new site my-site
plenti serve

Generates this error:

internal/modules/cjs/loader.js:1172
      throw new ERR_REQUIRE_ESM(filename, parentPath, packageJsonPath);
      ^

Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /Users/username/Code/my-site/ejected/build.js
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1172:13)
    at Module.load (internal/modules/cjs/loader.js:1000:32)
    at Function.Module._load (internal/modules/cjs/loader.js:899:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:74:12)
    at internal/main/run_main_module.js:18:47 {
  code: 'ERR_REQUIRE_ESM'
}
Total build took 152.0681ms

Serving site from your "public" directory.
Visit your site at http://localhost:3000/

Remove nodejs dependency

https://github.com/zeit/pkg

Should:

  • Not require devs to have nodejs installed on their computer
  • Not require internet access to start new project (currently runs npm install)
  • Still be extensible via NPM
  • Still allow custom Gopack package to create web_modules for ESM

Static Assets

Need to allow for static assets, like images. This can probably live at layout/static/ and get copied over directly to the public output on build.

Adding new file does not trigger rebuild

Currently if you add a new content source or layout, either manually or via the CLI tool (e.g. plenti new type whatever), it does not try to build the site again to incorporate the new file. It ignores changes even if you try to edit the new file and save it. You have to stop the webserver and restart it to get it to recognize the file.

GitHub Actions build fails when trying to use v8go

Failed build: https://github.com/plentico/plenti/runs/930279376#step:8:128

Error:

   ⨯ release failed after 87.44s error=failed to build for linux_amd64: # plenti/cmd/build
##[error]cmd/build/client.go:18:13: undefined: v8go.Context
##[error]cmd/build/client.go:39:12: undefined: v8go.NewContext
##[error]cmd/build/client.go:52:14: undefined: v8go.NewContext
##[error]cmd/build/client.go:92:25: undefined: v8go.Context

##[error]The process '/opt/hostedtoolcache/goreleaser-action/0.140.1/x64/goreleaser' failed with exit code 1

Benchmark flag / shorter build logs

Currently the build process outputs a lot of verbose info to the terminal that looks something like this:

Build output
Removing old "public" build directory

Creating "public" build directory

Ejecting core files to be used in build:
Temp writing '/router.svelte' file.
Temp writing '/build.js' file.
Temp writing '/main.js' file.
Creating non-ejected core files for build took 341.817µs

Copying ejectable core files to their destination:
Number of ejectable core files copied: 1
Copying ejectable core files for build took 308.993µs

Gathering data source from "content/" folder
Number of content files used: 6
Creating data_source took 1.453528ms

Prepping client SPA for svelte compiler
Number of components to be compiled: 11
Prepping client SPA data took 480.037µs
(node:7798) ExperimentalWarning: The ESM module loader is experimental.

Compiling components and creating static HTML took 366.408723ms

Running gopack to build esm support for npm dependencies:
- navaid, version ^1.0.5
- regexparam, version ^1.3.0
- svelte, version ^3.21.0
Gopack took 9.9061ms

Removing core files that aren't ejected:
Removing temp file 'ejected/router.svelte'
Removing temp file 'ejected/build.js'
Removing temp file 'ejected/main.js'
Removing the ejected directory.
Cleaning up non-ejected core files took 107.844µs

Total build took 382.136953ms

It would be nice to add a --benchmark flag to display this info, and if omitted simply show the total build time.

We should consolidate all the timers that are scattered throughout the build process into a single unified space that checks for the presence of the flag.

Add --benchmark flag to serve command

We currently have a benchmark flag that we can pass to the "build" command to get timing information about how long different aspects of the build take to compile: plenti build --benchmark (or plenti build -b).

It would be nice to allow access to this information when using the local webserver as well: plenti serve --benchmark

One challenge is that we already have a "build" flag for the "serve" command that lets us start the webserver without recompiling the site: plenti serve --build=false. The problem is the shorthand for this is also -b which is a conflict with --benchmark's shorthand. We'll need to change one of these.

Creating placeholder content

It would be nice to be able to create n number of placeholder pages in order to quickly mock a test site and run build time benchmarks.

Proposed command format: plenti gen <#> <type>

Example: plenti gen 35 pages

The scaffolding for the placeholder nodes would be based on the type's _blueprint.json file.

Optional endpoint for content type

It would be useful to be able to define a Type that does not have an endpoint.

Scenario 1: You want a type called content/main_menu/ to hold link information for a navbar.

Scenario 2: You have a component with information that you want to be able to share across different pages but it needs to be flexible enough that it shouldn't be hard-coded into a template.

In either scenario you don't want a node / endpoint that visitors can go to directly, you just want to pull the data in from the allNodes prop and use it however you'd like. Users should be able to extend the site config file (plenti.json) to account for this, something like:

"types": {
  "main_menu": ":none"
},

Currently the only replacement pattern we account for is :filename, but should extend this to look for :none.

It would also be nice to be able to specify this when creating a new type by using a flag: plenti new type <type_name> --endpoint=false

New starter style

The original vision for the base starter was to look "unfinished" so it's a blank slate for the user to start creating from. Minimalist style can look nice if done right (nice fonts and whitespace). Here was an early attempt at this:

Grayscale starter

gray

In the recent pushes, we've moved away from this vision a little bit by introducing a color palette based on the Plenti logo:

Color starter

color

Would love input on whether this is headed in the right direction in creating a welcoming introduction to the project, or if we're overdoing it.

Naming conventions

I'm unsure about how intuitive some of the default folder names are. My intent was that this would be mostly decoupled from the generator so users could name them whatever they want, and although that's somewhat true, some of this namespace is required for plenti to function.

The goal is not necessarily to keep with tradition of any particular methodology (e.g. in JS it's common to see src/ and dist/ folders), but instead aim for what is most intuitive and least intimidating to new users, especially those with limited technical experience. Ideally the dev experience will be easy for everyone, but if tradeoffs have to be made, I'd emphasize adaptation happening with experienced devs over inexperienced devs who are learning many concepts simultaneously.

Proposed change 1:
Rename assets/ => files/
I originally used "assets" because it seemed more all encompassing of static material that might go into this folder and fit with convention of some other projects. This term might be obscure for folks who have never heard of terms like "asset pipeline" and might not intuitively know what this folder should be used for. The term "files" on the other hand is a laymen term for things like photos, videos, etc. I originally didn't like this term because technically everything in plenti is a file, but I think the way it is commonly used would be easily understood by more people without having to look to documentation.

Proposed change 2:
Rename layout/content/ => layout/routes/
I originally used "content" because I wanted it to match the folder name of the data source that feeds it (there should be a 1-to-1 relationship between types and route layouts). Although it probably does help make that connection, it also obfuscates the fact that this folder is directly related to endpoints that visitors can navigate to. A downside of changing this to "route" is that each file in this folder can serve many routes so they are really just layouts for individual content files that determine the route.

Incrementing version on broken commits

I have a history of doing multiple deployments to fix GitHub Actions issues. In an attempt to stay true to semantic versioning I increment the minor version each time, even if the failed CI prevents a release from actually containing any binaries. This appears to be the correct thing to do: https://stackoverflow.com/questions/53660223/general-question-on-semantic-versioning-releases-using-ci-cd-pipeline

Just found this project that lets you test GitHub Actions locally: https://github.com/nektos/act

Thought it was worth sharing here in case other folks want to contribute to this project and suffer the same deployment woes I do. To run on an Ubuntu machine similar to our CI you can use the following (warning this image is ~10Gb):

act -P ubuntu-latest=nektos/act-environments-ubuntu:18.04

Components are undefined in HTML

The default starter now includes a content driven component example inspired by #9.

This works well once the page hydrates, but the fallback HTML shows undefined instead of the component.

Bundler

Currently Plenti is not using a bundler, instead we're trying to build ESM support using an internal tool called gopack. This is similar to Snowpack, which we originally used, but it was taking almost a full second to run (Snowpack itself took about .5s, but there was another .5s delay because of Go's exec.Command). Gopack takes less than 10ms to run, which is fast enough for now (although I think it could be faster). It's light on features and doesn't handling dynamic imports very well. It should work fine with Plenti core, but might hit walls as folks extend their projects.

Even earlier we used Rollup but it was significantly slower than that. Would consider revisiting bundlers if they were fast enough, something like esbuild which is written in Go. Watching evanw/esbuild#8 for svelte support.

kazzkiq and akaibukai made a good point about svelte compiling in JS being the bottleneck.

it would be different story if we managed to write a Svelte compiler in Go

Ultimately that ^ would be turning point for this project to get us to the build speed desired for a realtime Git CMS.

Can't serve site on Mac

plenti serve

Creating "public" build directory
Temp writing '/build.js' file.
Temp writing '/main.js' file.
Temp writing '/router.svelte' file.
Creating non-ejected core files for build took 603.266µs

Gathering data source from "content/" folder
Number of content files used: 6
Creating data_source took 1.614468ms

Prepping client SPA for svelte compiler
Number of source files copied: 1
Number of components to be compiled: 12
Prepping client SPA data took 1.003576ms

Compiling components and creating static HTML took 424.775935ms

Running gopack to build esm support for:
- navaid, version ^1.0.5
- regexparam, version ^1.3.0
- svelte, version ^3.21.0
Gopack took 28.402194ms
Removing temp file 'layout/ejected/build.js'
Removing temp file 'layout/ejected/main.js'
Removing temp file 'layout/ejected/router.svelte'
Removing the ejected directory.
Cleaning up non-ejected core files took 325.16µs

Total build took 457.058828ms

Serving site from your "public" directory.
Visit your site at http://localhost:3000/
ERROR open node_modules/svelte/types/compiler/parse/utils: too many open files
Unable to read site config file: unexpected end of JSON input

Removing old "public" build directory
open .: too many open files
Unable to read site config file: unexpected end of JSON input

Removing old "public" build directory
open .: too many open files
Unable to read site config file: unexpected end of JSON input

Removing old "public" build directory
open .: too many open files
Unable to read site config file: unexpected end of JSON input

Removing old "public" build directory
open .: too many open files
2020/05/05 14:34:37 http: Accept error: accept tcp [::]:3000: accept: too many open files; retrying in 5ms
2020/05/05 14:34:37 http: Accept error: accept tcp [::]:3000: accept: too many open files; retrying in 10ms
2020/05/05 14:34:37 http: Accept error: accept tcp [::]:3000: accept: too many open files; retrying in 20ms
2020/05/05 14:34:37 http: Accept error: accept tcp [::]:3000: accept: too many open files; retrying in 40ms
Unable to read site config file: unexpected end of JSON input

Removing old "public" build directory
open .: too many open files
Unable to read site config file: unexpected end of JSON input

Removing old "public" build directory
open .: too many open files
Unable to read site config file: unexpected end of JSON input

Removing old "public" build directory
open .: too many open files
Unable to read site config file: unexpected end of JSON input

Removing old "public" build directory
open .: too many open files
Unable to read site config file: unexpected end of JSON input

Removing old "public" build directory
open .: too many open files
2020/05/05 14:34:42 http: Accept error: accept tcp [::]:3000: accept: too many open files; retrying in 80ms
2020/05/05 14:34:42 http: Accept error: accept tcp [::]:3000: accept: too many open files; retrying in 160ms
Unable to read site config file: unexpected end of JSON input

Removing old "public" build directory
open .: too many open files
2020/05/05 14:34:42 http: Accept error: accept tcp [::]:3000: accept: too many open files; retrying in 320ms
Unable to read site config file: unexpected end of JSON input

Removing old "public" build directory
open .: too many open files
Unable to read site config file: unexpected end of JSON input

Removing old "public" build directory
open .: too many open files
^C

Maximum call stack size exceeded

In Firefox the error is: "too much recursion"
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Errors/Too_much_recursion

Problem: This is happening because Navaid is encountering a path in /public/spa/ejected/nodes.js that has a trailing forward slash. Since we try to automatically redirect these client side to the path without the trailing slash, it enters an infinite loop (the HTML fallback loads fine).

Scenario: If you add a new type (e.g. "docs") and you try to create an index file for that type (e.g. /content/docs/index.json), Plenti will handle the index filename but not remove the trailing slash from the client path (e.g. example.com/docs/)

Solution: Update cmd/data_source.go to properly remove the trailing forward slash from the data source (nodes.js).

Define _blueprint.json allowed values

The keys for _blueprint.json are flexible since they correspond to field values in the content source, which have no restrictions. The values however should be optional (can be left as "" if desired), but should be pull from a predefined list. Ultimately this will inform the editing behavior of the CMS that will get injected at build time.

Optional nodejs build?

We're finally at a working solution for removing NodeJS as a dependency of the project (#40) 🎸 🤘

That brings up the question of what to do with the old NodeJS dependent build process. My initial thought was to completely remove this, which is what I've already done for the client build. Now I'm thinking there is some advantages to adding it back in 🤦 (and leaving the static html nodejs build as an option as well).

So leaving a NodeJS option would have the following pros / cons:

Pros Cons
Can test broken things against native NodeJS builds More code, harder to maintain, changes to things like props have to be made in two places
Allows devs to continue ejecting the core build process for modification Slightly slower builds since prep for both builds will be run every time because I'm planning on only breaking out the exec.Command()
Could be faster in some scenarios (with defaults it's slower though) More work to set up the flags for build and serve commands, as well as make these optional when nodejs is run

References:

If keeping this in as an option for now, we should add a flag to that would look like this when applied in the CLI:

  • plenti build --nodejs=true
  • plenti serve --nodejs=true

Alternatively you should be able to set this in the site-wide configuration file (plenti.json). We'd have to modify the API to allow something like:

{
  "build": {
    "output": "public",
    "nodejs": true
  }
}

Custom starters

Currently a plenti site is really made up of:

  • data (the content/ folder)
  • structure + logic (the layout/ folder)
  • site-wide config (the plenti.json file)
  • third party dependencies (package.json + node_modules/)

The CLI tool lives independent of the project structure, so you could just download a starting point that someone else has put together and continue building on it. It might be nice if this was built into the new site command so you could run something like:

plenti new site your-site --starter=github.com/jimafisk/custom-plenti-starter

With this approach, once you pull a "starter" you immediately start to diverge from the parent as you begin development. If you wanted to pull upstream changes from the original designer, we would need to build in a "themes" system where templates have an inheritance hierarchy. With that approach (should be a separate ticket) the user could override the base theme templates but still pull changes that the original author creates.

Svelte stores.js

As plenti stands as of v0.1.26, it only looks for .svelte file extensions in the layout/ directory. So using a stores.js file as the documentation suggests won't work as expected. Replicating their example to work in plenti would look something like this:

layout/scripts/stores.svelte
<script context="module">
  import { writable } from 'svelte/store';
  export const count = writable(0);
</script>
layout/components/incrementer.svelte
<script>
  import { count } from '../scripts/stores.svelte';

  function increment() {
    count.update(n => n + 1);
  }
</script>

<button on:click={increment}>
  +
</button>
layout/components/decrementer.svelte
<script>
  import { count } from '../scripts/stores.svelte';

  function decrement() {
    count.update(n => n - 1);
  }
</script>

<button on:click={decrement}>
  -
</button>
layout/content/index.svelte (or wherever you want to use this)
<script>
  import { count } from '../scripts/stores.svelte';
  import Incrementer from '../components/incrementer.svelte';
  import Decrementer from '../components/decrementer.svelte';
  let count_value;
  const unsubscribe = count.subscribe(value => {
    count_value = value;
  });
</script>

<h1>The count is {count_value}</h1>

<Incrementer/>
<Decrementer/>

@ksallee actually pointed out a bug that will prevent this from working at the moment. The current behavior is it loads a 404 page, this is because Gopack is not updating the static export in public/spa/web_modules/svelte/store/index.js to make is ESM friendly. This is a bug that needs to be fixed, but in the meantime if you'd like to get it working for testing, just change the following line:

export { get_store_value as get } from '../internal';

to this:

export { get_store_value as get } from '/spa/web_modules/svelte/internal/index.js';
Here's a screenshot of the updated file

make_esm

Important: Once that line is changed you need to run your webserver with the build turned off or it will overwrite your change and put it back to the way it was. You can do that with plenti serve --build=false

Official docker image for CI

https://hub.docker.com/orgs/plentico

Should use lightweight container based on Alpine for quick CI builds:

Alpine Linux is... designed for security, simplicity, and resource efficiency... Because of its small size, it is commonly used in containers providing quick boot-up times.

Ideally it will only have the Plenti binary on it (once node dependency is removed per #3)

Error: LD_PRELOAD cannot be preloaded

On a fresh Ubuntu 20.04 (Unity) laptop, after installing with snap install plenti, running commands logs the following:

ERROR: ld.so: object 'libgtk3-nocsd.so.0' from LD_PRELOAD cannot be preloaded (failed to map segment from shared object): ignored.

Themes

Picking up the discussion on themes started here: #19

It would be nice to have a command for adding themes, for example: plenti new theme [email protected]:jimafisk/my-theme.git. This should download the git repo into a themes/ folder and could even make an entry in plenti.json in order to keep track of what theme is currently being used + could allow checking for updates on the remote repo.

Maybe also add a flag to the new site command, e.g. plenti new site MY_SITE --theme=REPO.git. This should run the --bare flag in the background.

fsnotify does not recursively watch nested folders that are added while "serve" is running

Plenti was updated to rebuild when new files are added to watched folders: #33

So something like plenti new type events which adds the following files:

  • content/events/
  • content/events/_blueprint.json
  • layout/content/events.svelte

Will trigger the following events:

File create detected: "content/events": CREATE
File create detected: "layout/content/events.svelte": CREATE

Note that the _blueprint.json file nested in the newly created content/events folder was not recognized as an event. This is because we've already walked the filepath recursively in the content folder to listen for files and we aren't rewalking it now for a new folder that was added.

This can be shown another way. Assuming we've already run the command above, creating a new event touch content/events/event1.json does not trigger a rebuild. However, if we add a new piece of content to a directory that's already been walked with touch content/pages/page1.json, it recognized the event with File create detected: "content/pages/page1.json": CREATE and rebuilds the site (and has some errors because the blank file isn't valid json, but that's besides the point).

Proposed solution: When new create events are triggered we should try to walk those now folders to add them to the watch list so subsequent changes trigger a rebuild.

Add --verbose flag to serve command

You can currently do a "verbose" build to see additional logging information: plenti build --verbose (or plenti build -v).

It would be nice to be able to see this additional information when running the webserver as well: plenti serve --verbose

404 handling broken

It appears commit d6d3bca "Add component helper + dynamic import to gopack (Fix #9)." broke 404 handling behavior. It continues to show the previous template with undefined var but the browser tab title is working. Then if you click the bad link again it moves over to the 404 page properly.

Concurrent build steps

Build steps:

  • Eject temporary files
  • Direct copy .js files to build dir
  • Collect data sources from /content
  • Collect svelte templates from /layout
  • Execute the build.js script with NodeJS
  • Run Gopack to convert files to support ESM
  • Delete core files that weren't ejected by the user

These should be broken into goroutines so they run concurrently. If one step relies on a return value from another, we can use channels to block that thread until that data is available. This should speed up build times.

404 error when trying to use Lifecycle functions

It's inevitable that someone will run into this, so figured I'd document it here. If you try to import something from the base svelte package, like the documentation tells you to do for a lifecycle function like onMount:

<script>
  import { onMount } from 'svelte';
</script>

You will get a 404 error. This is due to Gopack not being smart enough to read this reference. You can get around this by simply referencing the exact package, which is "internal" in this case:

<script>
  import { onMount } from 'svelte/internal';
</script>
Here are the errors you will see in the browser

Error in Firefox:

Loading module from “http://localhost:3000/spa/web_modules/svelte/internal/” was blocked because of a disallowed MIME type (“text/html”).

Error in Chrome:

Failed to load module script: The server responded with a non-JavaScript MIME type of "text/html". Strict MIME type checking is enforced for module scripts per HTML spec.

fsnotify events should logged as --verbose output

Currently when you add, edit/save, delete a file while running the webserver, it rebuilds the site and you see logging information about which fsnotify events were triggered:

File create detected: fsnotify.Event{Name:"content/events", Op:0x1}

File create detected: fsnotify.Event{Name:"layout/content/events.svelte", Op:0x1}

Or

File delete detected: fsnotify.Event{Name:"layout/content/4913", Op:0x4}

File rename detected: fsnotify.Event{Name:"layout/content/index.svelte", Op:0x8}

File create detected: fsnotify.Event{Name:"layout/content/index.svelte", Op:0x1}

File write detected: fsnotify.Event{Name:"layout/content/index.svelte", Op:0x2}

File delete detected: fsnotify.Event{Name:"layout/content/index.svelte~", Op:0x4}

We should use the build.Log utility to only show these messages if the --verbose flag is passed, e.g. plenti serve -v

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.