Giter VIP home page Giter VIP logo

hugo-importer's Introduction

Hugo-to-WriteFreely Migration Tool

This command-line tool imports a content-source directory from a Hugo site into a WriteFreely instance, including Write.as.

Summary

Getting Started

These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. See deployment for notes on how to deploy the project on a live system.

Prerequisites

You'll need to have Go v1.15 or higher installed and available on your system's $PATH. Please refer to the Go installation instructions for detailed steps.

Installing

First, start by cloning the project to your local machine:

git clone [email protected]:writefreely/hugo-importer.git

or, if you prefer cloning via HTTPS rather than SSH:

git clone https://github.com/writefreely/hugo-importer.git

Then, install the migration tool:

cd hugo-importer
go install

Usage

Run the tool from the root directory of your Hugo site.

hugo-importer --user matt --blog my-great-blog --content-dir posts

The migration tool is interactive; it will prompt you for your password and to confirm that you're ready to proceed, then output each step of the parsing and publishing process as they occur.

Your password is not saved anywhere; it's used once to fetch an access token that is used for subsequent requests to the WriteFreely/Write.as API.

Please review the limitations below to confirm the migration tool will work for your Hugo site.

Parameters

Some parameters are required, and others only pertain to specific cases.

--user (or -u)

Required

The username for logging into your WriteFreely (or Write.as) account.

--blog (or -b)

Required

The alias of the WriteFreely/Write.as blog that the Hugo posts will be migrated to; you can find this from the Blogs page on WriteFreely/Write.as, clicking on the "Customize" button for that blog, and checking the bold portion in the "URL" section ("Preferred URL" on Write.as).

--content-dir (or -c)

Required

The source directory for the posts you wish to migrate, under content in your Hugo site directory. For example, to import your-hugo-site/content/posts, you would set --content-dir posts.

--instance (or -i)

Required for WriteFreely only

The WriteFreely instance URL (e.g., --instance https://pencil.writefree.ly); leave this out if you're importing into a Write.as account.

--images

Only available to Write.as Pro users with Snap.as

By adding this flag, the migration tool will attempt to find local images when parsing your source content, and upload them from your Hugo site's static directory to Snap.as.

Commands

help

The above information is available via the help command:

hugo-importer help

Output

Posts in the source directory are parsed, added to a queue, and then published to the destination blog from the queue. The server's response for each post is sanitized of the modification token and saved to a publishing log file in the root directory of your Hugo site.

If the --images flag is set, and the migration tool encounters errors trying to upload an image to Snap.as, it will note them in the console output and add them to an upload-error.log file in the root directory of your Hugo site.

Limitations

The migration tool expects a standard structure where content and static directories are one level below the Hugo site's root directory, and that local images are served out of the static directory.

The migration tool will convert most built-in Hugo shortcodes into URLs, specifically:

  • gist
  • instagram
  • tweet
  • vimeo
  • youtube

By converting these to URLs, gist, tweet, and youtube URLs are shown as rich embeds for Write.as Pro users.

Contributing

Please read the CONTRIBUTING documentation for details on how to report bugs, submit feature requests, and for submitting pull requests.

Versioning

We use SemVer for versioning. See the tags on this repository for versions that are currently available.

Authors

See also the list of contributors who participated in this project.

hugo-importer's People

Contributors

angelostavrow avatar thebaer avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

Forkers

hulb

hugo-importer's Issues

Add project documentation

This project requires, at a minimum, the following documentation:

  • Read Me
  • License
  • Contribution Guidelines
  • Code of Conduct

Add unit-test suite

The project should include a suite of unit tests that ensure everything's working as expected.

Tests should be included for the following files:

  • utilities.go
  • parser.go
  • logger.go

Import local images to Snap.as

Feature Overview

When a command-line flag (--images) is set, the hugo-importer tool should upload local images to Snap.as and update image links in the source content.

Constraints

This feature is only available to Write.as Pro subscribers. At a minimum, if the --instance flag is set, the tool should return an error.

There are two main ways to include an image in the source content, and the tool should handle both:

1. Using Markdown (!["Alt text here"](/link/to/image.jpeg))
2. Using HTML (<img src="/link/to/image.jpeg" alt="Alt text here" />)

Implementation

If the --images flag is set, the tool shall do the following when parsing a source file:

  1. Search the content of the file for an image (either Markdown or HTML)
  2. Determine if the image is local or remote (more on this below)
  3. If image is remote, continue searching; otherwise
  4. Confirm the image exists in the filesystem at /static/link/to/image.jpeg
  5. Upload the image to Snap.as and get the new URL
  6. Replace the original image source URL with the Snap.as URL
  7. Continue searching the source file for more images

Image Source: Absolute vs Relative Paths

Related to step 2 in the implementation details above, a local image's source may be represented via an absolute path (https://example.com/link/to/image.jpeg), or a relative path (/link/to/image.jpeg). In the latter case, it's clear that the image is local; in the former, however, the tool must determine if the base URL (in this scenario, example.com) is the URL of the Hugo site being migrated. Fortunately, the baseURL parameter in the Hugo site configuration file lets us make this comparison.

Migrate built-in Hugo shortcodes as URLs

Hugo provides built-in shortcodes that "represent very common usage" for convenience. The intention is to keep Markdown clean, but because they are proprietary to Hugo, they don't migrate nicely, so a Tweet shortcode ends up looking like:

{{< tweet 877500564405444608 >}}

Where possible, these would make more sense as plain URLs:

https://twitter.com/twitter/status/877500564405444608

For a subset (e.g. gist, tweet, and youtube), converting shortcodes to URLs allows Pro users to get proper embeds via embed.as; in all cases, special care must be taken to craft a valid URL from the shortcode, so we'll limit ourselves to the following shortcodes:

Media Hugo Shortcode URL Embed.as?
GitHub Gist {{< gist username gistID >}} https://gist.github.com/username/gistID
Instagram {{< instagram photoID >}} https://www.instagram.com/p/photoID/ ⛔️
Twitter {{< tweet tweetID >}} https://twitter.com/twitter/status/tweetID
Vimeo {{< vimeo videoID >}} https://player.vimeo.com/video/videoID ⛔️
YouTube {{< youtube videoID >}} https://www.youtube.com/watch?v=videoID

This doesn't cover all built-in shortcodes, nor does it cover all options for the shortcodes listed above, but I believe it's a pretty solid set of shortcodes to cover for launch. We can include a note in the README about this limitation, and extend it if the demand exists.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.