dillonkearns / elm-pages Goto Github PK

View Code? Open in Web Editor NEW

643.0 14.0 94.0 25.69 MB

Hybrid Elm framework with full-stack and static routes.

Home Page: https://elm-pages.com

License: BSD 3-Clause "New" or "Revised" License

Elm 83.33% JavaScript 16.30% CSS 0.03% TypeScript 0.31% Shell 0.03%

elm elm-pages seo blog static-site-generator jamstack

elm-pages's Introduction

`elm-pages`

elm-pages is a framework for building an Elm single-page app that is able to seamlessly interface with data from an Elm Backend. elm-pages is a hybrid framework, allowing you to define Routes that are either server-rendered (for more dynamic content with user-specific or request-specific data) or pre-rendered at build-time (for generating static HTML files that are hosted through a CDN). You can mix and match server-rendered and pre-rendered routes in your app.

elm-pages also has a command for running pure Elm scripts with a single command. See the elm-pages Scripts docs page.

Getting Started Resources

elm-pages Docs Site
elm-pages site showcase
elm-pages Elm API Docs
Quick start repo (live site hosted here)
Introducing elm-pages blog post
examples folder (includes https://elm-pages.com site source) Use git clone --recurse-submodules https://github.com/dillonkearns/elm-pages.git so that there aren't missing files when you try to build the examples.

Compatibility Key

You will see an error if the NPM and Elm package do not have a matching Compatibility Key. Usually it's best to upgrade to the latest version of both the Elm and NPM packages when you upgrade. However, in case you want to install versions that are behind the latest, the Compatibility Key is included here for reference.

Current Compatibility Key: 21.

Contributors ✨

Thanks goes to these wonderful people (emoji key):

_{Daniel Marín}
💻

_{Steven Vandevelde}
💻

_{Johannes Maas}
📓 💻

_{Wiktor Toporek}
💻

_{Luke Westby}
💻

This project follows the all-contributors specification. Contributions of any kind welcome!

elm-pages's People

Contributors

Stargazers

Watchers

Forkers

razzeee forkkit itopplus twistingtwists leojpod kmbn jciel ianmackenzie bgins tennety danmarcab matheus23 nidico synalysis lukewestby digitalsatori cyberglot icidasset kyasu1 canrau kannapoix sailfish009 simplystuart vviktorpl j-maas christophp kingderp tgelu sevfurneaux eye942 aaronwhite carlfredrikhero e-stonia masa-310 galanggg marcodaniels garybozek georgesboris rtunreal martinsstewart zinggi bigardone skyblueballykid rsulli55 svitebskiy ljnicol acyuta108 aotarola adoringonion supermario tsfoster zeshhaan johncrosby nathanbraun thesacredlipton canceraiddev newmana elm-review-bot sdaves asteroidb612 swissbite angusjf software-garden blaix bramvdnheuvel bcardiff noredink miyamotonitobe orinokai caseywebb erlandsona grrinchas kyepskee anagrius frontmesh jreusch4 evanpiro hadrienmp dreyeck qbit absynce shahnhogan kraklin tanyabouman daviddta andrewmacmurray minibill abradley2 doubledup adamdicarlo0 declension joshuanianji ricardo-rp

elm-pages's Issues

Sitemap generation

In order to be SEO friendly sitemap generation would be a great feature.

Do code-splitting on metadata

Provide an API for fetching exactly the metadata you need for each individual page. A pattern similar to the StaticHttp API would work nicely. You tell it at the top of the page if you need to grab any metadata. You can tell it exactly what metadata you want, and it will only include that metadata, and will bundle it into the content.json file for that page.

It might look something like this:

getBlogPostMetadata : Metadata -> Bool
getBlogPostMetadata metadata =
    case metadata of
        Metadata.Article articleMetadata ->
            True

        _ ->
            False

This is a nice pattern because user's are already doing exactly this... so it's ideal to have what you would naturally want to do just result in nice performance optimizations (rather than having to just write arbitrary code to optimize things).

Here's an example of some code that does something similar:

elm-pages/examples/docs/src/Index.elm

Lines 22 to 31 in 4ea2e47

 case metadata of 

 Metadata.Article meta -> 

 if meta.draft then 

 Nothing 

 else 

 Just ( path, meta ) 

 _ -> 

 Nothing

It would be great to provide an API that also let's you return a Maybe yourCustomType rather than a Bool so that you can map over the items and narrow down the types (say, to only include blog posts and therefore narrow down the data type to that specific variant's data).

Allow StaticHttp to make requests for something other than JSON

Currently elm-pages requires you to use JSON, I'd like to parse an RSS feed which uses XML. How do you feel about making this independent of JSON? I can help with this if you want me to.

Render empty shell for service worker navigation route (not root page) to prevent flash

Right now when the service worker is registered and you open up an elm-pages site, it will load the root page's HTML and flash it briefly before the Elm code takes over. It should serve up the shell HTML (empty body) to use the shell service worker pattern: https://developers.google.com/web/fundamentals/architecture/app-shell.

Files is 'static' folder don't get copied to the dist folder in develop mode, only on build

Files is 'static' folder don't get copied to the dist folder in develop mode, only on build. Once copied by build they work with development mode.

Plugin architecture brainstorm

I'm thinking about ways to extend elm-pages for plugin authors.

Goals

Prevent users from having to wire in several pieces to "install" a plugin - for example, if a plugin adds a global <head> and generates a file - should be able to pipeline that easily with the user's config and with other plugins
Configuration should all be in pure Elm
Plugin builder API should be secure, which means that it's probably not the exact same API as users are using to build up their Pages.Platform.application
The functions for users to init a Platform.Pages.application should be simple, intuitive, and make it natural to do the right thing. So for example, we want API authors to be able to add global <head> tags, and different plugins (maybe a sitemap and RSS plugin) might both add global head tags. But for the user, it probably makes sense to define their head tags from a central location, rather than being able to arbitrarily chain it and then have to trace where in their code it came from.

It might look something like this:

Pages.init { ... }
|> Pages.withPlugin (Rss.plugin rssOptions)
|> Pages.withPlugin (SiteMap.plugin sitemapOptions)

Security

StaticHttp
Accessing secrets

Potential Strategies

User accepts through config file or CLI

The user could generate a list of files

{
    "dillonkearns/elm-pages-rss": {
    },
    "dillonkearns/elm-pages-sanity": {
        "domains": ["https://*.sanity.io/*"],
        "secrets": ["SANITY_TOKEN"]
    }
}

This seems reasonable for domains. But for secrets, what if the user wants to call it something else? Not very configurable (though the user might want to configure it for domains as well).

User passes a value in that restricts possible values

For example, if you want to allow a plugin to use a secret, the only way a plugin author can access it is if you pass it in to the plugin:

Pages.GoogleCalendar.install (Secrets.get "GOOGLE_CALENDAR_TOKEN")

Use cases

Generate files (access to StaticHttp)
Add global head tag (access to StaticHttp)
Generate a config or JavaScript file that is run by something like an admin CMS page to configure it
Generate a config file for services like Netlify

File generators

Typically just generate files where the user may depend on StaticHttp, but they don't directly call it, only pass through the user's StaticHttp.

RSS reader (generate files - user may depend on StaticHttp)
Sitemap (generate files - user may depend on StaticHttp)
Netlify CMS routing rules (would need access to all page paths)

Example:

plugin
    |> RssPlugin.generate
        { siteTagline = siteTagline
        , siteUrl = canonicalSiteUrl
        , title = "elm-pages Blog"
        , builtAt = Pages.builtAt
        , indexPage = Pages.pages.blog.index
        }
        metadataToRssItem

metadataToRssItem :
    { path : PagePath Pages.PathKey
    , frontmatter : Metadata
    , body : String
    }
    -> Maybe Rss.Item
metadataToRssItem page =
    case page.frontmatter of
        Metadata.Article article ->
            if article.draft then
                Nothing

            else
                Just
                    { title = article.title
                    , description = article.description
                    , url = PagePath.toString page.path
                    , categories = []
                    , author = article.author.name
                    , pubDate = Rss.Date article.published
                    , content = Nothing
                    }

        _ ->
            Nothing


RssPlugin.generate :
    { siteTagline : String
    , siteUrl : String
    , title : String
    , builtAt : Time.Posix
    , indexPage : PagePath pathKey
    }
    ->
        ({ path : PagePath pathKey
         , frontmatter : metadata
         , body : String
         }
         -> Maybe Rss.Item
        )
    -> Builder pathKey userModel userMsg metadata view 
    -> Builder pathKey userModel userMsg metadata view 
RssPlugin.generate options metadataToRssItem builder =
    let
        feedFilePath =
            (options.indexPage
                |> PagePath.toPath
            )
                ++ [ "feed.xml" ]
    in
    builder
        |> Pages.Platform.withFileGenerator
            (\siteMetadata ->
                { path = feedFilePath
                , content =
                    Rss.generate
                        { title = options.title
                        , description = options.siteTagline

                        -- TODO make sure you don't add an extra "/"
                        , url = options.siteUrl ++ "/" ++ PagePath.toString options.indexPage
                        , lastBuildTime = options.builtAt
                        , generator = Just "elm-pages"
                        , items = siteMetadata |> List.filterMap metadataToRssItem
                        , siteUrl = options.siteUrl
                        }
                }
                    |> Ok
                    |> List.singleton
                    |> StaticHttp.succeed
            )
        |> Pages.Platform.addGlobalHeadTags [ Head.rssLink (feedFilePath |> String.join "/") ]

Source Plugin

StaticHttp is core
Likely will provide a Json.Decode.Exploration.Decoder to encapsulate the data that the service gives you (what if you want to strip out the data you don't use? Do we need a builder-style syntax that allows you to just add in the pieces you want, and store it in your preferred data type? Would be similar to how elm-graphql provides you decoders but you piece them together)
Could encapsulate reading from the filesystem
Might specify how to parse file body or metadata?

Examples:

Contentful source
Google Calendar source
File system source - Parse markdown content from content folder (or let user specify)
Sanity.io source - not much to this, though, it just points to a GraphQL endpoint, so maybe there's no need for this

Things that could just wrap the core APIs

With the designs I have and am imaginging for features, I think plugin authors can just wrap them if they want a different API from these primitives:

Encapsulate image optimizations
Manifest configuration

Tracing errors

Let's say you have some build-time error, like:

Generating files with duplicate names
Failed StaticHttp request or decoder

The error message should clearly state that the conflicting generated file is conflicting with the plugin named X, or that the HTTP failure came from the plugin making HTTP call Y.

Support sites hosted not at the top level

Currently it is possible to create elm-pages sites at URLs like example.net but not example.net/some/subdir, since functions like ImagePath.toString and PagePath.toString return URLs of the form

/images/some_image.png
/some_page

instead of

/some/subdir/images/some_image.png
/some/subdir/some_page

https://example.net/some/subdir/images/some_image.png
https://example.net/some/subdir/some_page

(Specifically, I was unable to create a working elm-pages site at https://ianmackenzie.github.io/elm-geometry/guide.)

YAML frontmatter decoder

Many thanks for the great project, Dillon!

By my understanding, it should not be too difficult to plug e.g. https://package.elm-lang.org/packages/terezka/yaml/latest/ as a frontmatter decoder, which would allow for much cleaner frontmatter on each page. The only snag is that there is no fitting constructor for Pages.Document.DocumentHandler - parse expects a Json decoder.

Would that be an option that you'd be willing to add, e.g. with an additional parserWithFrontmatter that takes a String -> Result String metadata as an argument? Happy to provide a pull-request if you do!

Static asset routes not recognized

When I try to open an image rendered within a post/page in a new tab, I get the Page not found error, saying the url is not a valid route. To reproduce:

Navigate to https://elm-pages.com/images/article-covers/extensible-markdown-parsing.jpg

Expected:
The image shows in the browser
Actual:
Page not found error
EDIT:
Hard-refreshing the image url does show the image.

~~I wonder if this line should also include allImages in the list of routes.~~
Per conversation with @dillonkearns it seems like it could be related to the service worker fallback url.

Elm application initialized after each link click

It would be great if the navigation would work without page reloads so that application state is kept.

Some API that allows the application to switch to another page would be good. And the links in Markdown pages should use this API instead of the default link behavior.

Is this a reasonable request?

Bug: links break when deploying to subdomains

All links work fine if I deploy it to example.com, but they break if I deploy it to example.com/blog, e.g. a repository using github or gitlab pages.

Update favicons-webpack-plugin dependency

Currently I am not able to use elm-pages on macos with node 13, because the version of favicons-webpack-plugin it depends on in turn depends on sharp v0.22.1. The first version supporting node 13 is v0.23.2, which is depended on by the current version of favicons-webpack-plugin, so a dependency update should fix this.

Blog post has broken Ellie

The blog post https://elm-pages.com/blog/extensible-markdown-parsing-in-elm#markdownwithinhtmltags has a broken Ellie that doesn't compile.

I guess that dillonkearns/elm-markdown had a breaking update where the Renderer now expects both unordered and ordered lists instead of a generic list.

I managed to finally fix some issues where the types weren't right but the parser's error message didn't really help... Here's the fixed Ellie: https://ellie-app.com/7PKpvc4brDCa1 I might open a PR later, when I find the time.

Developer loop breaks when a code gen error happens.

When running in dev mode if you get a compile error the dev loop keeps running. If you get a codegen error, say cannot decode your metadata, it breaks out of the dev loop and has to be manually restarted.

For example:

-- METADATA DECODE ERROR ----------------------------------------------------- elm-pages

I ran into a problem when parsing the metadata for the page with this path: /

Problem with the given value:

{
        "type": "news",
        "author": "Rupert Smith",
        "title": "Authentication in Elm",
        "description": "About and authentication API I designed for Elm.",
        "image": "/images/article-covers/hello.jpg",
        "published": "2020-01-21"
    }

Unexpected page type news

and then the dev loop terminates.

===

Would be nice if the dev loop just kept running - the error printed on the console is helpful in taking me right to the problem that needs to be fixed.

Bonus - if the error would also show in the browser.

Support for Elm debugger

Showing the Elm debugger in dev mode would be very helpful.

Problem setting page height with elm-ui

Hi! First of all thank you for making this! I am excited to use it for my personal page.

I've run into an small issue when using elm-pages with elm-ui.
I wanted to make Element.layout take the full height of the page using Element.width Element.fill. This is something I've done many times with elm-ui.

Digging a bit I found elm-pages I found it is wrapping the user provided view with a div here:

elm-pages/elm-package/src/Pages/Platform.elm

Lines 124 to 129 in 4b9b3a5

 [ Html.div 

 [ Html.Attributes.attribute "data-url" (Url.toString model.url) 

 ] 

 [ body 

 |> Html.map UserMsg 

 ]

elm-ui is setting the height to 100%, but having a parent element makes it only taking the content height.

I am not sure what's the best way to proceed or even if it's something you should consider fixing, but I thought it was good to let you know!

Thank you again!

Files aren't generated for extensions besides .md and .emu

The problem is that currently the extension is hardcoded for .md files here:

elm-pages/generator/src/elm-pages.js

Lines 46 to 65 in eb5b7b7

 const content = glob.sync(contentGlobPath, {}).map(unpackMarkup); 

 const staticRoutes = generateRecords(); 

 const markdownContent = glob 

 .sync("content/**/*.md", {}) 

 .map(unpackFile) 

 .map(({ path, contents }) => { 

 return parseMarkdown(path, contents); 

 }); 

 const images = glob 

 .sync("images/**/*", {}) 

 .filter(imagePath => !fs.lstatSync(imagePath).isDirectory()); 

 let app = Elm.Main.init({ 

 flags: { 

 argv: process.argv, 

 versionMessage: version, 

 content, 

 markdownContent, 

 images

.emu files need special treatment so they can remain separate. Removing the hardcoded .md extension should fix the problem.

Compiler error on Windows when using directories due to slash

As initially reported in dillonkearns/elm-pages-starter#1, I'd like to open the issue against this repo, because I assume that the bug is in this code.

To reproduce, try to run examples/docs on Windows with npm start. You will see the following error message:

> elm-pages develop --port 1234

Running elm-pages...
Dependencies ready!
Detected problems in 1 module.
-- PROBLEM IN RECORD ------------------------------------------------- Pages.elm

I am partway through parsing a record, but I got stuck here:

79|     { blog/extensibleMarkdownParsingInElm = (buildPage [ "blog/extensible-markdown-parsing-in-elm" ])
              ^
I just saw a field name, so I was expecting to see an equals sign next. So try
putting an = sign here?

Note: If you are trying to define a record across multiple lines, I recommend
using this format:

    { name = "Alice"
    , age = 42
    , height = 1.75
    }

Notice that each line starts with some indentation. Usually two or four spaces.
This is the stylistic convention in the Elm ecosystem.

(node:8612) UnhandledPromiseRejectionWarning: Compilation failed
(node:8612) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:8612) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Dillon suspected that the bug is in a part of the code splitting on / instead of being platform-dependent.

I would like to provide more detailed information, but I'm not sure how to enable logging or how to debug the code. Any suggestions of how to get more information are welcome.

Provide API for service worker caching strategies

It would be nice to create, essentially, a type-safe wrapper for the Workbox API to give users control over the service worker caching strategy.

It should use PagePaths and ImagePaths, specifically, to make it as safe as possible.

It might be nice to have a function that passes in each static route or image, and then you can return a cache strategy.

cacheStrategy : PagePath Pages.PathKey -> CacheStrategy
cacheStrategy path =
  if  path |> Directory.includes Pages.pages.blog.directory then
    Precache
  else
    NetworkFirst

That Elm code can be used to just create a hardcoded list of routes and what caching strategy should be used. Although, what about forward-compatibility, since service workers could be out of date? Ideally, elm-pages would refresh the service worker when it's out of date, but it's something to consider.

Content file names can conflict with content directory names

The error as it appears:

$ elm-pages develop --port 3001
Running elm-pages...
/Users/antoine/Documents/projects/elm/elm-pages-tryout/node_modules/elm-pages/generator/src/generate-records.js:202
      if (name in obj) {
               ^

TypeError: Cannot use 'in' operator to search for 'src' in (buildPage [ "static-git", "files" ])
    at captureRouteRecord (/Users/antoine/Documents/projects/elm/elm-pages-tryout/node_modules/elm-pages/generator/src/generate-records.js:202:16)
    at generate (/Users/antoine/Documents/projects/elm/elm-pages-tryout/node_modules/elm-pages/generator/src/generate-records.js:85:7)
    at wrapper (/Users/antoine/Documents/projects/elm/elm-pages-tryout/node_modules/elm-pages/generator/src/generate-records.js:9:10)
    at run (/Users/antoine/Documents/projects/elm/elm-pages-tryout/node_modules/elm-pages/generator/src/elm-pages.js:47:24)
    at Object.<anonymous> (/Users/antoine/Documents/projects/elm/elm-pages-tryout/node_modules/elm-pages/generator/src/elm-pages.js:126:1)
    at Module._compile (module.js:652:30)
    at Object.Module._extensions..js (module.js:663:10)
    at Module.load (module.js:565:32)
    at tryModuleLoad (module.js:505:12)
    at Function.Module._load (module.js:497:3)
    at Function.Module.runMain (module.js:693:10)
    at startup (bootstrap_node.js:191:16)
    at bootstrap_node.js:612:3
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

My (simplified) content directory structure:

✔ $ tree content
content
├── index.projects
└── static-git
    ├── files
    │   └── src
    │       ├── Commit.file
    │       └── Common.file
    ├── files.files
    └── static-git.history

3 directories, 5 files

Removing the content/static-git/files.files file makes it compile.

develop loop not running when /content folder changes for non .md file.

I have been using elm-markup and had an index.emu file under /content.

I notived that under elm-pages develop changes to this file do not trigger the build loop, but they do for an index.md file.

Some suggested fixes:

Trigger the build loop on any change under /content. Not as refined as 2, but should be perfectly effective and not unreasonable.
Trigger the build loop on any change to files with a DocumentHandler for that file type. I guess this is harder to do.

Workaround: Just call your files .md, even if you are using elm-markup.

Add flags support to pagesInit

I would like to be able to pass flags to pagesInit that would eventually get passed to the elm app's init function. ~~In the meantime I am trying to use ports to accomplish something similar~~ (strike that I don't think I can do that either now that I think about it), but was curious if this was possible or if it would cause issues with the pre-rendering phase.

reStructuredText

I see that Markdown is supported:

https://elm-pages.com/blog/extensible-markdown-parsing-in-elm

Is reStructuredText supported?

https://docutils.sourceforge.io/docs/user/rst/quickref.html

Invalidate service-worker-cached app shell and content

#52 makes sure that the app shell is cached. But if there are changes, then ideally we should tell the service worker whenever we find out it is stale.

We can have the content.json file include a hash that tells use whether the service worker cache is up-to-date with what we're fetching in content.json.

Linking to "#anchor-name" no longer works with <base> tag

When using <base> tags, people seem to often work around anchor links navigating to the root path (instead of the current path) with some JavaScript. For example, https://rogerkeays.com/blog/using-base-href-with-anchors. Maybe we can have a similar workaround at the framework-level that makes this behavior of anchor tag links work as users would expect?

Duplicated text in 'types over conventions' blog post

It looks like the first half of the types over conventions blog post is duplicated. Search for "Rails started a movement of simplifying project setup" and you'll see that the paragraph (and following text) is repeated.

Colorize "elm-pages build" output

Looks a little blunt right now 😉

Instructions to setup and test the repo without resorting to npm link

Hi again and thanks for this great work!

I'd be interested in trying to tweak things a bit and I was wondering what you are using to test elm-pages locally and if there are some setup steps required for it.

Cheers

Feature request: asset management

I know elm-pages uses webpack, so it would be nice to have access (even if limited) to it. Currently, I have a set of specific fonts that I've bought from a designer studio, so I need to serve it myself. Right now, it's being moved to /dist rather than a more convenient directory like /dist/assets/fonts.

The easy fix is to expect the elm-pages user to add all assets (including subdirectories) to /assets and it later gets compiled to /dist/assets. A more elegant solution would be to have it configurable, but the former is good enough.

Content file names can't start with a digit

E.g. content/001.txt won't compile

It produces this elm error message

-- PROBLEM IN RECORD ------------------------------------------------- Pages.elm

I just started parsing a record, but I got stuck here:

89|     { 001 = (buildPage [ "001" ])
          ^
I was expecting to see a record field defined next, so I am looking for a name
like userName or plantHeight.

Note: Field names must start with a lower-case letter. After that, you can use
any sequence of letters, numbers, and underscores.

Sure enough in gen/Pages.elm it's using the filename as a record's field name.

pages =
    { 001 = (buildPage [ "001" ])
    ,  ...

I noticed that dashes in the file names are sanitized here. I suppose something similar could be done.

Image optimization API

I’ve been thinking about image optimization. My longer-term vision is to have fine-grained control over the image optimization (rather than a single rule for all images). I’m imagining an API similar to the StaticHttp design. So you could do something like this:

OptimizedImage.srcset { includeWebp: True, defaultFormat: OptimizedImage.jpeg, widths = [ 2000, 1200, 500 ] } Pages.images.hero

That’s just totally a back-of-the-envelope sketch, but the idea is that, like StaticHttp, this would be something that has multiple stages:

Build stage
elm-pages goes through and turns all of the OptimizedImage calls into actual optimized images (exactly analagous to the lifecycle of StaticHttp fetching all the static http content).

Runtime stage
It simply refers to the built image by filename, where the filename is generated based on the minification options and filename.

Similar to the StaticHttp, OptimizedImage would be something you would chain and get the data back in the view function, see: https://elm-pages.com/blog/static-http/#let'sseesomecode!

And it would give you something you could use directly to create a srcset (which if you haven’t heard of it, it’s a very cool browser API that allows you to progressively enhance your page with performant images that download the right dimensions based on the device you’re using).

Prior Art

For reference, here are the docs for Gatsby’s image optimization https://www.gatsbyjs.org/packages/gatsby-plugin-sharp/

In the case of Gatsby, it’s done through GraphQL fragments. With elm-pages, this StaticHttp pattern of getting data is the equivalent of Gatsby’s GraphQL layer. It will probably evolve over time, but IMO it’s a lot simpler and more type-safe doing it without the extra layer of abstraction of using GraphQL 😄

Responsive Image reference material

Getting to the low-level details of how to build the actual HTML for the responsive images, here are some handy resources:

https://css-tricks.com/responsive-images-youre-just-changing-resolutions-use-srcset/

https://developer.mozilla.org/en-US/docs/Learn/HTML/Multimedia_and_embedding/Responsive_images

I think that the elm-pages API for responsive images should give you a function that turns the value into actual HTML directly. When using something like elm-ui, there is no high-level API for srcsets, and even if there was there is more that can go wrong if you give the user the list of paths to the image resources and their metadata instead of directly giving the proper HTML for the responsive image. So unless there's a good reason not to, I think that's the ideal design.

Content edits are not reloaded in dev mode

When you make a change in a content file (markdown for example), you can see in the console `elm-pages is watching the files and running:

Rerunning for content/posts/post.md
Running elm-pages...
Done!
elm-pages DONE
ℹ ｢wdm｣: Compiling...
Started compiling Elm..
ℹ ｢wdm｣: Compiled successfully.

The sequence in the output suggest elm-pages is doing its thing, then webpack is noticing the change and building.

Unfortunately if you reload the page in the browser, the changes are not there, you only see a blank page (not even the old state).

You need to restart elm-pages to see the content.

NOTES:

Changing an .elm file and reloading the page works fine.
According to the discussion in slack, @dillonkearns thinks this is probably
webpack and elm-pages stepping on each other's toes.

@dillonkearns thank you again for the great work you are doing! If you want help I'd love to give it a try, pointers welcome because I am definitely not a webpack expert :)

elm-pages build crashes on Windows due to invalid path

Unfortunately the fun from #81 continues. However, let's first deal with merging #82 and then come back to this.

When I run the patched version (i.e. the one from #82) with elm-pages build, it first builds everything correctly but then crashes:

$ elm-pages build
Running elm-pages...
Dependencies ready!
Success! Compiled 9 modules.

    Main ---> C:\Users\y0hy0h\AppData\Local\Temp\2020215-10588-1imhixo.aik1.js

10% buildingelm-pages DONE
100%
ERROR in closure-compiler: java.nio.file.InvalidPathException: Illegal char <:> at index 78: C:\Users\y0hy0h\Code\elm-pages-fix\node_modules\style-loader\dist\index.js!C:\Users\y0hy0h\Code\elm-pages-fix\node_modules\css-loader\dist\cjs.js!C:\Users\y0hy0h\Code\test-elm-pages\node_modules\highlight.js\styles\github.css
        at sun.nio.fs.WindowsPathParser.normalize(WindowsPathParser.java:182)
        at sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:153)
        at sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:77)
        at sun.nio.fs.WindowsPath.parse(WindowsPath.java:94)
        at sun.nio.fs.WindowsFileSystem.getPath(WindowsFileSystem.java:255)
        at sun.nio.fs.AbstractPath.resolveSibling(AbstractPath.java:66)
        at com.google.javascript.jscomp.SourceMapResolver.getRelativePath(SourceMapResolver.java:102)
        at com.google.javascript.jscomp.Compiler.addSourceMapSourceFiles(Compiler.java:2824)
        at com.google.javascript.jscomp.Compiler.initBasedOnOptions(Compiler.java:537)
        at com.google.javascript.jscomp.Compiler.initModules(Compiler.java:517)
        at com.google.javascript.jscomp.AbstractCommandLineRunner.doRun(AbstractCommandLineRunner.java:1248)
        at com.google.javascript.jscomp.AbstractCommandLineRunner.run(AbstractCommandLineRunner.java:554)
        at com.google.javascript.jscomp.CommandLineRunner.main(CommandLineRunner.java:2158)
Duration: 2.4s

(Note that this is a subsequent run. In the first run, I saw all the images being generated. In this run, I guess they were already cached. In every run I encountered the error.)

No ability to change behavior depending on the current page

It is useful to trigger certain actions only upon loading a particular page/page type. My current thought would be to add an extra onPageChange field to the generated Pages.application config, where you could supply an optional msg that would be called after a page has loaded, similar to onUrlChange in Browser.application. Additionally, the init function would take an extra argument indicating the currently loaded page. Does this seem like a good/feasible idea, or am I misunderstanding something?

HTML from build sometimes doesn't include content

Hey 👋 Thanks for this great project!

I noticed that sometimes when I make a build, the resulting static HTML says Missing content (ie. the content isn't there). Also, sometimes the static–html content is wrong, instead of missing. For example, sometimes I'm getting content for the index page instead of the blog article.

It appears to happen randomly, also tested it with removing the .cache directory.

elm-pages develop throws error in v1.2.8

Hello Dillon!

I saw that you had closed an issue relating to the missing content flash and decided to bump my elm-pages version from 1.2.5 (the version hardcoded into the elm-pages-starter project) to the latest and greatest.

When I ran npm start I got the following error.

elm-pages develop

module.js:550
throw err;
^

Error: Cannot find module '../node-elm-compiler/index.js'
at Function.Module._resolveFilename (module.js:548:15)
at Function.Module._load (module.js:475:25)
at Module.require (module.js:597:17)
at require (internal/module.js:11:18)
at Object. (/myproject/node_modules/elm-pages/generator/src/compile-elm.js:1:91)
at Module._compile (module.js:653:30)
at Object.Module._extensions..js (module.js:664:10)
at Module.load (module.js:566:32)
at tryModuleLoad (module.js:506:12)
at Function.Module._load (module.js:498:3)
at Module.require (module.js:597:17)
at require (internal/module.js:11:18)
at Object. (/myproject/node_modules/elm-pages/generator/src/generate-elm-stuff.js:2:16)
at Module._compile (module.js:653:30)
at Object.Module._extensions..js (module.js:664:10)
at Module.load (module.js:566:32)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! Exit status 1

Not a huge problem. I dropped back to 1.2.7 and everything seems to work fine there. Just thought you should have the heads up.

Thank again for all your work!

Add access to StaticHttp data in more places

One thing I'm thinking about right now is changing generateFiles to have access to StaticHttp.Request data.

I'm feeling pretty good about that direction because you can share StaticHttp.Requests by just extracting them in your Elm code. For example, you might have:

type alias Event = { title : String, {- ... -} }
Event.staticRequest : StaticHttp.Request (List Event)

Then re-use the same one in generateFiles, or in your view function for specific pages (or all pages).

One interesting thing about StaticHttp.Requests in the context of generateFiles is that the raw responses don't end up in your final bundle, only the resulting files that are generated. So there's no need to make optimized requests there (so if you wanted to do StaticHttp.unoptimizedRequest in that context, it wouldn't be any different).

Access to `StaticHttp.Request` data for static results

There are currently 2 other places where we can just use StaticHttp.Requests as a helper to fetch data to get your final results, but the responses are not bundled into your data, only the final result is.

Pages.Platform.application {
  - ...
    , canonicalSiteUrl : String
    , manifest : Pages.Manifest.Config pathKey
    , generateFiles :
        List
            { path : PagePath pathKey
            , frontmatter : metadata
            , body : String
            }
        -> List (Result String { path : List String, content : String })
}

So giving these access to StaticHttp data could look like this.

Pages.Platform.application {
  - ...
    , canonicalSiteUrl : StaticHttp.Request String
    , manifest : StaticHttp.Request (Pages.Manifest.Config pathKey)
    , generateFiles :
        List
            { path : PagePath pathKey
            , frontmatter : metadata
            , body : String
            }
        -> 
        StaticHttp.Request (List
             (Result String { path : List String, content : String })
        )
}

Access to `StaticHttp` data in more dynamic places

I think it makes sense to have access to StaticHttp data in dynamic places.

It could be in init, or in update, or in onPageChange (or some combination). A few things to consider:

init is called for all pages, so any StaticHttp.Requests in that context would be global data (need to download for any page you visit, whether it's used there or not)

Giving `onPageChange` access to `StaticHttp.Request` data

If we give onPageChange access to StaticHttp data, then you can access data for a specific page (and potentially fetch different data based on the PagePath and the page's metadata)

We would also need to change the arguments so that you don't have access to the query and fragment when you make the StaticHttp.Request, since that's dynamic and not known at build-time.

Currently, this is the function signature:

    , onPageChange :
        { path : PagePath pathKey
        , query : Maybe String
        , fragment : Maybe String
        }

Giving input

This is an early stage idea and I'm still doing some thinking on the design. Ideas and use cases are welcome!

I'm also going to be working on a builder-style way to add to your Pages.Platform.application config to build it, adding one piece at a time. So that design could work nicely with adding StaticHttp data in more places because we could have helpers for building that have StaticHttp data, or that don't.

Should `elm-pages` have a way to not generate a page

elm-pages doesn't have any special frontmatter directives (unlike Jekyll, which will look for a draft tag specifically: https://jekyllrb.com/docs/posts/#drafts). This is by design, as elm-pages tries to be agnostic to those details so there is no magic, and leave those things up to the user.

On the other hand, it does seem reasonable to be able to not generate certain pages under certain conditions.

I think there's a nice solution here, though I haven't thought through all the implications. My idea is to have a special type that you can return if you want to ignore a page. Although that could become cumbersome since you have to return the same type everywhere in Elm (you can't just return null in places you want to ignore). So that's something to consider.

Let's brainstorm some potential ideas and what code could look like here.

Add a parameter to set the port of the development server

The port 3000 tends to be a bit crowded and it would be nice to either pic another if it's taken or have the ability to set which port you'd like to use.

Decouple routes from content folder

I’m working on the design now for decoupling the elm-pages routes from the content/ folder. My goal is to 1) have a helper that sets up elm-pages exactly as it is now with mapping content/ files to routes. But, 2) give you the flexibility to create routes either based on files, StaticHttp response data… or just a hardcoded path with no associated file or static data! Or a route based on data from 2 different files!

My current thinking is that the design will look something like this.

Give a helper that allows you to iterate over files and turn those files into Static data.
Return a CreatePage record that represents how to pull in from data sources, decode, reduce down the JSON, and then create a route with that view and metadata

type alias CreatePage metadata pageView =
    Result String 
    { path : List String
    , metadata : metadata
    , content : pageView
    }

createPages =
Static.findFiles { extension = “md”, base = [ “content” ] }
|> Static.map
  (List.map (\file -> 
    Static.map2 (\meta body ->
      { route = file.path, metadata = meta, body = body }
    )
    (file
      |> Static.frontmatter 
      (Decode.map2 Metadata.Page
         (Decode.field “title” Decode.string)
         (Decode.field “tags” (Decode.list Decode.string))
      )
    )
    (file
      |> Static.body {- StaticHttp String -}
      |> Static.map parseMarkdown
    )

  )
)

Error running elm-pages build

Hello!

I was trying to play around with elm-pages and ran into an error with the build command.

I set up a project with a directory structure like

content
images
src
- Main.elm

I dropped in a couple short test .md files into the content folder and some images into the image folder.

I then went to set up a Pages.Platform.application in my main function. I realized that I would need the generated elm-pages information to complete the record and the Manifest config. So, I threw an Html.text node in my main function and ran elm-pages build. I got the following error:

Running elm-pages...
(node:12200) UnhandledPromiseRejectionWarning: Error: What node should I take over? In JavaScript I need something like:

Elm.Main.init({
    node: document.getElementById("elm-node")
})

You need to do this with any Browser.sandbox or Browser.element program.
at _Debug_crash (eval at (/home/glenn/.npm-global/lib/node_modules/elm-pages/generator/src/compile-elm.js:14:7), :550:10)
at eval (eval at (/home/glenn/.npm-global/lib/node_modules/elm-pages/generator/src/compile-elm.js:14:7), :3940:56)
at _Platform_initialize (eval at (/home/glenn/.npm-global/lib/node_modules/elm-pages/generator/src/compile-elm.js:14:7), :1879:16)
at eval (eval at (/home/glenn/.npm-global/lib/node_modules/elm-pages/generator/src/compile-elm.js:14:7), :3928:9)
at Object.eval [as init] (eval at (/home/glenn/.npm-global/lib/node_modules/elm-pages/generator/src/compile-elm.js:14:7), :20:33)
at /home/glenn/.npm-global/lib/node_modules/elm-pages/generator/src/compile-elm.js:15:28
at /home/glenn/.npm-global/lib/node_modules/elm-pages/generator/src/compile-elm.js:31:7
at
(node:12200) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:12200) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

I realize it's very likely I'm doing something wrong here. So, if this isn't a problem with elm-pages and is an issue with how I'm setting up my project, please let me know!

Thanks in advance!

Query Params for Current Page

Hey @dillonkearns I'd like to grab the query params for the current page being viewed. The use case I'm thinking through is that there are a list of items (credit cards e.g.) on a page, and clicking on an item will take the user to another page which will have actions the user may take associated with the item on the previous page. I was thinking one approach to this might be to pass an identifier in the query params. Enjoy your vacation, and I'll catch you when you're back!

Roadmap

In https://elm-pages.com/blog/introducing-elm-pages there was a "Next Steps" section that mentioned some super exciting features. In particular I think

Allow users to pass a set of HTTP requests to fetch during the build step (for making CMS or API data available statically in the build)

.. is super exciting! Very much like gatsby! Would be cool to connect to Contentful or other headless CMS!

Secondly, do you intend on publishing a prioritized roadmap?

Elm-markup files not parsed succesfully

I have a basic content/elm-markup.emu file which works with 1.0.39 (npm) and 1.0.0 (elm) but which cannot be rendered with elm-pages 1.0.40 (npm) and 1.0.1 (elm)

The page shows up like this

/elm-markup
I couldn't parse the frontmatter in this page. I ran into this error with your JSON decoder:
Failure

Prefetch only links with content.json file available

Currently all links going to the same site are prefetched which prevents links to downloadable files to work as well as <iframe> elements injected in Markdown files.

Anchor links are causing a page reload after <base> tag change

Problem

I also tried changing the link from #mathematics to blog/functional-graphics#mathematics, but for some reason that doesn't cause firefox to jump to the correct place. Instead, it jumps to the top of the page

Context

This workaround:

https://github.com/dillonkearns/elm-pages/blob/master/src/Pages/Internal/Platform.elm#L450-L461

No longer works with the <base> tag change. Perhaps that can be removed now?

Content files can't contain """

If any content file contains triple double quotes """ elm-pages will fail to compile.

Example content file:

---
title: txt format
type: page
---

Just some plain text
"""
some more plain text

I was partway through parsing a record, but I got stuck here:

11812|     , { frontMatter = """{"title":"txt format","type":"page"}
11813| """ , body = Just """
11814| Just some plain text
11815| """
          ^
I was expecting to see a closing curly brace next. Try putting a } next and see
if that helps?

Some context for my use case: I'm making a site to show my personal git repositories, a bit like github/gitlab does, but in a static/prerendered way. This will include the code and the diffs as well. As you can imagine some Elm files have """ in them.

TOC links broken

In https://elm-pages.com/docs and https://elm-pages.com/docs/directory-structure, both table of contents links are not navigating to their respective locations on the page.

Using Firefox Quantum 69.0.3 (64-bit)

Missing Content message flashes on initial load

Here are some ideas on how to approach this.

What's happening now

Browser loads and renders the HTML
Since the <script> tag that includes the Elm bundle is in the <head>, and doesn't have a defer on it, I believe it will block the rest of the HTML from rendering? (need to confirm this)
As the page is loading, we see this message briefly as it fetches the content.json file, which contains the body of our content, as well as any StaticHttp data for the page:

elm-pages/src/Pages/Internal/Platform.elm

Line 156 in 7eaea5d

{ title = "elm-pages error", body = Html.text "Missing content" }
The page fully loads and displays correctly

Possible solution

Render the HTML content immediately (ensure that the Elm bundle doesn't block HTML rendering, but ideally it should also be fetched concurrently as the HTML is rendered... possibly using a <script defer ...>?).
Pass in some flags to the low-level elm-pages init that contains already-fetched content.json data (could either hardcode that data into the generated HTML file, but probably best and most performant to fetch it once the HTML is rendered, and then callback once it's fetched to init the elm app with that data as a flag).
Now, when the Elm app takes over, it has all the data it needs so the "Missing content" message won't flash because the content is present on init from the flag

Old solution idea

~~What could happen instead (this is just one solution path)~~ I thought this would be a good approach, but thought of the simpler solution above

Render the HTML content immediately (ensure that the Elm bundle doesn't block HTML rendering, but ideally it should also be fetched concurrently as the HTML is rendered... possibly using a <script defer ...>?).
Render the Elm view off in an invisible <div> (will this cause rendering to differ at all? Probably not because it's Elm, right, so it should render reliably even off in an invisible div?).

Listen for the Elm view function in the invisible <div> to be called... can use the MutationObserver technique similar to how elm-pages currently listens for page changes in order to know when the URL has changed

elm-pages/index.js

Lines 72 to 84 in 5754bfb

 function observeUrlChanges( 

 /** @type {MutationRecord[]} */ mutationList, 

 /** @type {MutationObserver} */ _theObserver 

 ) { 

 for (let mutation of mutationList) { 

 if ( 

 mutation.type === "attributes" && 

 mutation.attributeName === "data-url" 

 ) { 

 setupLinkPrefetchingHelp(); 

 } 

 } 

 }

Once the view has been rendered, remove the visible, server-rendered view, and show the hidden, client-rendered view (will this cause any reflow or other kinds of jank? Is there a way to prevent that?)

Error mockup for frontmatter parsing issues

Goal

Let's brainstorm, as a community, on how we can make a specific error message as helpful as possible! We can figure out this process for refining error messages "on paper" before we do so in code, to think about it from the user's perspective and then think about how the message will serve them.

See the notes and philosophy in the elm-pages Error Message Style Guide.

How to pitch in

You can simply write a comment giving your thoughts and suggestions, or you can try writing out an error message yourself, taking the error below as a starting point!

Situation

This mockup is for the error the user is presented with when there is no DocumentHandler registered for the file extension .md (only one for .txt), but there is a .md file in the content folder.

Current State

This Elm code:

markdownDocument : ( String, Pages.Document.DocumentHandler Metadata () )
markdownDocument =
    Pages.Document.parser
        { extension = "txt"
        , metadata = JD.succeed ()
        , body = \_ -> Ok ()
        }

And this markdown file (no frontmatter, no body, just an empty content/index.md file):

---
---

Gives this error:

-- METADATA DECODE ERROR ----------------------------------------------------- elm-pages

I ran into a problem when parsing the metadata for the page with this path: /

Could not find extension 'md'

New Error Mockup

CONTEXT

I was trying to parse the file `content/index.md`

But I didn't find a `Pages.DocumentHandler` for `.md` files in your `Pages.Platform.application` config.

You have document parsers for the following extensions:
.txt

SUGGESTION

So you could either:
1) Create your new file with one of those extensions, like `content/index.txt`
2) Move the file `content/index.md` outside of the `content` folder, or delete it
3) Add a `Pages.DocumentHandler` for `.md` files, something like

    Pages.Document.parser
        { extension = "md"
        , metadata = Decode.succeed ()
        , body = \_ -> Ok ()
        }

EDUCATIONAL MATERIAL

You can learn more about how elm-pages deals with parsing different document types,
and some techniques and best practices for managing your content files and metadata types
at elm-pages.com/docs/document-handlers

	case metadata of
	Metadata.Article meta ->
	if meta.draft then
	Nothing

	else
	Just ( path, meta )

	_ ->
	Nothing

	[ Html.div
	[ Html.Attributes.attribute "data-url" (Url.toString model.url)
	]
	[ body
	\|> Html.map UserMsg
	]

	const content = glob.sync(contentGlobPath, {}).map(unpackMarkup);
	const staticRoutes = generateRecords();

	const markdownContent = glob
	.sync("content/*/.md", {})
	.map(unpackFile)
	.map(({ path, contents }) => {
	return parseMarkdown(path, contents);
	});
	const images = glob
	.sync("images/*/", {})
	.filter(imagePath => !fs.lstatSync(imagePath).isDirectory());

	let app = Elm.Main.init({
	flags: {
	argv: process.argv,
	versionMessage: version,
	content,
	markdownContent,
	images

	function observeUrlChanges(
	/** @type {MutationRecord[]} */ mutationList,
	/** @type {MutationObserver} */ _theObserver
	) {
	for (let mutation of mutationList) {
	if (
	mutation.type === "attributes" &&
	mutation.attributeName === "data-url"
	) {
	setupLinkPrefetchingHelp();
	}
	}
	}