simonm0 / ghost-static-site-generator Goto Github PK

Hi, gssg has been a lifesaver, so first and foremost thank you very much sir!

The issue I'm seeing is that the photos for social cards (like Twitter) are not pulled into the content folder.

The built directory inside the assets is empty. No styling and JS functionality in the generated static site.

I've created a setup so that when I want to deploy ghost to github pages, I just run have to run ghostdeploy in cmd, then git commit -m "some message" and git push. This is the result: https://itsignacioportal.github.io/. No broken links, no broken images, no errors :)

Environment:
Windows 10

• node v16.14.2
• Ghost-CLI version: 1.19.2
• Ghost version: 4.41.3 (at ~\AppData\Local\Ghost_Server)
• ghostdeploy.bat in a folder that's part of %PATH% //This is the script we launch
• python 3.9
• replacer3.py //Used mainly to fix links
• ImageMagick (Used to resize images, for more performance on github pages)

WSL:

• Debian
• node v16.14.2 installed via NVM
• gssg installed. //gssg doesn't work on Windows with Cygwin, but it does work on WSL, idk why
• gssgsummoner.sh at $HOME //A wrapper for gssg
• gitcleaner.sh at $HOME //deletes all files except ".git" folder, and my google search console ownership file
• /etc/hosts modified to redirect localhost to 192.168.50.21 (beware of the warning on the first line of the file) //modification done because otherwise you would need to have Ghost running directly on WSL, and that's just bothersome for many reasons

The /etc/hosts redirection was made persistant using a scheduled task that runs "with the highest privileges" on user log-in (See: MicrosoftDocs/WSL#418):

@echo off
wsl -d Debian -u root ip addr add 192.168.50.11/24 broadcast 192.168.50.255 dev eth0 label eth0:1

netsh interface ip add address "vEthernet (WSL)" 192.168.50.21 255.255.255.0
@echo on
exit

You'll obviously have to modify some paths to make this work for you, but hey, it's better than starting at nothing. I spent three days making all of these scripts :p

ghostdeploy.bat:

@echo off

REM Summon the Ghost Static Page Generator
wsl.exe -d Debian cd /mnt/c/Users/Sherman/Documents/GitHub/ItsIgnacioPortal.github.io/ ^&^& /home/pinkdev1/gitcleaner.sh ^&^& echo "[+] Running the Ghost Static Site Generator (gssg)..." ^&^& /home/pinkdev1/gssgsummoner.sh

REM Fix picture links
echo [+] Fixing picture links and dangling hrefs...
py -3.9 "C:\Program Files\HackingSoftware\ghost\replacer3.py"

REM Fix profile picture
echo [+] Fixing profile picture...
cd C:\Users\Sherman\Documents\GitHub\ItsIgnacioPortal.github.io
mkdir content\images\size\w100\2022\03
REM Make sure to modify the Circle-2.png filename to match the actual filename of your uploaded profile picture
magick convert content/images/2022/03/Circle-2.png -resize 100x100! content/images/size/w100/2022/03/Circle-2.png

echo [+] Commiting files...
git config --global core.safecrlf false
git add .

echo [+] The blog is now ready for deployment!
echo [+] Just pick a commit message, and push it to the repo.

cd C:\Users\Sherman\Documents\GitHub\ItsIgnacioPortal.github.io

(file on WSL) gitcleaner.sh

#!/bin/bash

shopt -s extglob
cd /mnt/c/Users/Sherman/Documents/GitHub/ItsIgnacioPortal.github.io/
echo "[+] (WSL): Deleting everything except .git folder on \"/mnt/c/Users/Sherman/Documents/GitHub/ItsIgnacioPortal.github.io/\"..."
rm -rf !(.git|googleeef7c96830aeb56d.html)

replacer3.py

import os

root_dir = 'C:\\Users\\Sherman\\Documents\\GitHub\\ItsIgnacioPortal.github.io'
githubPagesDomain = "itsignacioportal.github.io"
fourLetterAllowedExtensions = ["html"]
threeLetterAllowedExtensions = ["xml", "css", "txt", "xsl"]
twoLetterAllowedExtensions = ["js"]

#There's probably a better way of doing this, but works well enough
def hasAllowedExtension(file):
	if file[-4:] in fourLetterAllowedExtensions:
		return True

	if file[-3:] in threeLetterAllowedExtensions:
		return True

	if file[-2:] in twoLetterAllowedExtensions:
		return True

	return False

def replaceStrings(searchText, replaceText, file):
	print("[+] Replacing \"" + searchText +"\" in \"" + file + "\" ...")
	#https://stackoverflow.com/a/17141572/11490425
	# Read in the file
	with open(file, 'r') as iofile :
		filedata = iofile.read()

	# Replace the target string
	filedata = filedata.replace(searchText, replaceText)

	# Write the file out again
	with open(file, 'w') as iofile:
		iofile.write(filedata)

def replaceStringsInDir(searchText, replaceText, folder):
	for file in os.listdir(folder):
		#print(folder + "\\" + file)
		if os.path.isdir(folder + "\\" +file) and file != ".git":
			file = folder + "\\" + file
			print("[+][DEBUG] Recursing to: " + file)
			replaceStringsInDir(searchText, replaceText, file)
		elif hasAllowedExtension(file):
			replaceStrings(searchText, replaceText, folder + "\\" + file)


def replaceRecursively(searchText, replaceText, mainfolder):
	for file in os.listdir(root_dir):
		if file != ".git":
			if os.path.isdir(mainfolder + file):
				replaceStringsInDir(searchText, replaceText, mainfolder + "\\" + file)
			else:
				replaceStringsInDir(searchText, replaceText, mainfolder)

os.chdir(root_dir)
replaceRecursively("http://localhost:2368", "https://" + githubPagesDomain, root_dir)

#Fix sitemaps
replaceStrings("href=\"//localhost:2368", "href=\"//" + githubPagesDomain, root_dir + "\\" + "sitemap-authors.xml")
replaceStrings("href=\"//localhost:2368", "href=\"//" + githubPagesDomain, root_dir + "\\" + "sitemap-pages.xml")
replaceStrings("href=\"//localhost:2368", "href=\"//" + githubPagesDomain, root_dir + "\\" + "sitemap-posts.xml")
replaceStrings("href=\"//localhost:2368", "href=\"//" + githubPagesDomain, root_dir + "\\" + "sitemap-tags.xml")
replaceStrings("href=\"//localhost:2368", "href=\"//" + githubPagesDomain, root_dir + "\\" + "sitemap.xml")

(file on WSL) gssgsummoner.sh

#!/bin/bash
cd /mnt/c/Users/Sherman/Documents/GitHub/ItsIgnacioPortal.github.io/
gssg --productionDomain "https://itsignacioportal.github.io" --dest "/mnt/c/Users/Sherman/Documents/GitHub/ItsIgnacioPortal.github.io/"

When i am generating the static site from ghost, it does not create page folder with the paginate items.

if the tag/homepage has pagination it does not work..

At the moment everything is done sync, spawning from https://github.com/Fried-Chicken/ghost-static-site-generator/blob/master/src/commands/generateStaticSite.js#L32

Having some parallelism would speed things up drastically, 10 or 20 simultaneous requests for example.

A library such as https://www.npmjs.com/package/run-parallel-limit could help with that.

When executing gssg command I get following error:

Error: Command failed: wget --help | grep "show-progress" || true
'grep' is not recognized as an internal or external command,
operable program or batch file.

at checkExecSyncError (child_process.js:629:11)
at execSync (child_process.js:666:13)
at shouldShowProgress (C:\Users\Me\AppData\Roaming\npm\node_modules\ghost-static-site-generator\src\constants\OPTIONS.js:15:32)
at Object.<anonymous> (C:\Users\Me\AppData\Roaming\npm\node_modules\ghost-static-site-generator\src\constants\OPTIONS.js:39:22)
at Module._compile (internal/modules/cjs/loader.js:778:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
at Module.load (internal/modules/cjs/loader.js:653:32)
at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
at Function.Module._load (internal/modules/cjs/loader.js:585:3)
at Module.require (internal/modules/cjs/loader.js:692:17)

Hey mate,
Thanks for this - works a treat when i run on my macbook however when trying to run it as part of an automated script on a centos machine; it doesn't:

wget: unrecognized option '--show-progress'
Usage: wget [OPTION]... [URL]...

Try `wget --help' for more options.

--show-progress feels like vanity anyway, so maybe it can just be removed?

The website's Domain/URL is not inserted in the link to the sitemap*.xsl files inside the sitemap*.xml files, e.g.:

<?xml version="1.0" encoding="UTF-8"?><?xml-stylesheet type="text/xsl" href="//localhost:2368/sitemap.xsl"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:image="http://www.google.com/schemas/sitemap-image/1.1"></urlset>

When generating the site, amp images are not being copied over

Hello,

I used ghosthunter for the search option. But after made the static site by using gssg, search not working.. Do you have any idea or tricks to show the search feature in static ghost site?

Thanks

There's an issue with sitemap url's not being replaced properly.

This is because some of the urls int he sitemap do not have 'http' but only '//'.
Fixed by removing 'http://' or 'https://' from both domain and the replacement url and replacing all domain instances with url provided by the url flag.

Hi and thanks for this nice utility!

I've noticed that if I want to publish the static site into a subdirectory, assets, content and public are loaded from web server root.

I would expect that issuing this command:
gssg --url "https://www.samplehostname.com/staticsite_dir01/" --dest staticsite_dir01
everything would be stored into and referenced from staticsite_dir01 ready to be dropped inside
the public_html of the web server.

Does an option for this exist/is in consideration?

Thanks in advance,

--R

It would be nice to know how to get gssg working on a Windows machine. And if not possible at all then at least have a short mention about this at the top of the README would be great.

Relative paths is replacing regex in js

As I generate this static content locally; it doesn't have https enabled.

When I come to run it from my bucket, i get:

Looks like we could do with something to rewrite this perhaps?

Post gallery images are not being saved correctly to static file folder.

OS: Mac OSX Big Sur 11.4
node version: v16.13.2
npm version: 8.1.2
wget version: 1.21.2

Navigate to my page: https://feedangrow.com/

You should see the broken image. If you open image in tab, or view page source it shows the URL: https://feedangrow.com/content/images/size/w150h150/2022/01/autopot_logo.png

If I go into my local files and tree that folder, it does not exist.

You can't see what you don't upload :)

Other than that all the other images seem to be working.

Title says most of it.

Example:
if I have --url http://localhost:8080 as a flag to gssg and I have http://localhost:8080 in the content of my site (not a link), gssg replaces it with the --domain parameter.

Ghost version: 5
NPM Version: 6.14.16
Node Version: v12.22.12
Ghost Static Site Generator version: v1.1.4
OS distro + version: Ubuntu 20.04.4 LTS

Description: When I try to launch gssg, it seems that the flag "--sourceDomain" is not used.
Instead, it performs multiple wget commands on http://localhost:2368.

Logs:

root@my-server:~# gssg --sourceDomain http://myghosthosted-onanotherserver.mydomain.org/ --productionDomain http://static.mydomain.org/ --dest myblog-static-folder
Fetching: http://localhost:2368/
ERROR: null
Using Command: wget -q --show-progress --recursive --timestamping --page-requisites --no-parent --no-host-directories --restrict-file-name=unix --directory-prefix myblog-static-folder --content-on-error  --trust-server-names http://localhost:2368/
Fetching: http://localhost:2368/sitemap.xsl
ERROR: null
Using Command: wget -q --show-progress --recursive --timestamping --page-requisites --no-parent --no-host-directories --restrict-file-name=unix --directory-prefix myblog-static-folder --content-on-error  --trust-server-names http://localhost:2368/sitemap.xsl
Fetching: http://localhost:2368/sitemap.xml
ERROR: null
Using Command: wget -q --show-progress --recursive --timestamping --page-requisites --no-parent --no-host-directories --restrict-file-name=unix --directory-prefix myblog-static-folder --content-on-error  --trust-server-names http://localhost:2368/sitemap.xml
myblog-static-folder/sitemap.xml http://localhost:2368/sitemap.xml Error: ENOENT: no such file or directory, open '/root/myblog-static-folder/sitemap.xml'
    at Object.openSync (fs.js:462:3)
    at Object.readFileSync (fs.js:364:35)
    at fetchUrlHelper (/usr/lib/node_modules/ghost-static-site-generator/src/helpers/fetchUrlHelper/fetchUrlHelper.js:47:31)
    at Array.forEach (<anonymous>)
    at generateStaticSite (/usr/lib/node_modules/ghost-static-site-generator/src/commands/generateStaticSite.js:35:10) {
  errno: -2,
  syscall: 'open',
  code: 'ENOENT',
  path: '/root/myblog-static-folder/sitemap.xml'
}
Fetching: http://localhost:2368/404
ERROR: null
Using Command: wget -q --show-progress --recursive --timestamping --page-requisites --no-parent --no-host-directories --restrict-file-name=unix --directory-prefix myblog-static-folder --content-on-error  --trust-server-names http://localhost:2368/404
Fetching: http://localhost:2368/public/ghost.css
ERROR: null
Using Command: wget -q --show-progress --recursive --timestamping --page-requisites --no-parent --no-host-directories --restrict-file-name=unix --directory-prefix myblog-static-folder --content-on-error  --trust-server-names http://localhost:2368/public/ghost.css
Fetching: http://localhost:2368/public/ghost.min.css
ERROR: null
Using Command: wget -q --show-progress --recursive --timestamping --page-requisites --no-parent --no-host-directories --restrict-file-name=unix --directory-prefix myblog-static-folder --content-on-error  --trust-server-names http://localhost:2368/public/ghost.min.css
Fetching: http://localhost:2368/public/404-ghost.png
ERROR: null
Using Command: wget -q --show-progress --recursive --timestamping --page-requisites --no-parent --no-host-directories --restrict-file-name=unix --directory-prefix myblog-static-folder --content-on-error  --trust-server-names http://localhost:2368/public/404-ghost.png
Fetching: http://localhost:2368/public/[email protected]
ERROR: null
Using Command: wget -q --show-progress --recursive --timestamping --page-requisites --no-parent --no-host-directories --restrict-file-name=unix --directory-prefix myblog-static-folder --content-on-error  --trust-server-names http://localhost:2368/public/[email protected]
Error: ENOENT: no such file or directory, copyfile '/root/myblog-static-folder/404/index.html' -> '/root/myblog-static-folder/404.html'
    at Object.copyFileSync (fs.js:1907:3)
    at copy404PageHelper (/usr/lib/node_modules/ghost-static-site-generator/src/helpers/copy404PageHelper/copy404PageHelper.js:15:8)
    at generateStaticSite (/usr/lib/node_modules/ghost-static-site-generator/src/commands/generateStaticSite.js:40:5) {
  errno: -2,
  syscall: 'copyfile',
  code: 'ENOENT',
  path: '/root/myblog-static-folder/404/index.html',
  dest: '/root/myblog-static-folder/404.html'
}
Domain: http://localhost:2368
Static site generated at: /root/myblog-static-folder
Site generated in: 229.252ms

thank for maintaining this package!

these are the versions i am using:

• gssg 1.0.1
• ghost 3.42.0
• ghost cli 1.16.0

when i execute gssg, it fails with ERROR: null.

$ ghost --version
Ghost-CLI version: 1.16.0
Ghost version: 3.42.0 (at /var/lib/ghost)

$ gssg --url https://mydomain.com --fail-on-error --silent
Fetching: http://localhost:2368/sitemap.xsl
Fetching: http://localhost:2368/sitemap.xml
Fetching: http://localhost:2368/sitemap-pages.xml
Fetching: http://localhost:2368/
ERROR: null
Using Command: wget -q --recursive --timestamping --page-requisites --no-parent --no-host-directories --restrict-file-name=unix --trust-server-names --directory-prefix static --content-on-error http://localhost:2368/

any clue what is wrong?

--preview fails to load images, because the URL for images is written starting with http://localhost:2368/ instead of http://localhost:8080/. This may be related to #65 ?

When I run
gssg --domain "http://localhost:2368" --url "myrepo.github.io" static --ignore-absolute-paths --preview

The preview on localhost:8083 works perfectly, however the static files themselves have broken paths for anything linked via the {{asset}} helper.
Static code:
<link rel="stylesheet" href="/assets/css/styles.css?v=6bbed80a06" />

Original ghost theme code:
<link rel="stylesheet" href="{{asset "css/styles.css"}}" />

So the Asset helper is being appended with

?v=6bbed80a06

osx 10.12

Error: Command failed: wget --help | grep "show-progress" || true
'grep' is not recognized as an internal or external command,
operable program or batch file.

at checkExecSyncError (node:child_process:885:11)
at execSync (node:child_process:957:15)
at shouldShowProgress (C:\Users\Eli\AppData\Roaming\npm\node_modules\ghost-static-site-generator\src\constants\OPTIONS.js:20:32)
at Object.<anonymous> (C:\Users\Eli\AppData\Roaming\npm\node_modules\ghost-static-site-generator\src\constants\OPTIONS.js:44:22)

Hey,
So when you use an image like the one on my first post: https://karlstoney.com, ghost will select an image size (600, 1000, 2000) to generate the page. The crawler will therefore only see one image (in my case it was the 1000w one), and that's the one that gets generated.

Then when the static content is uploaded and javascript tries to request the 600w version, it isn't there.

Not sure how best to tackle this! I've worked around it in my script by basically copying the content folder: https://github.com/Stono/ghost-static/blob/master/bin/generate_static_content.sh#L5

At the moment, I simply eyeball the output looking for errors.

It'd be nice to add a flag such as --fail-on-error which would stop the process and exit non-zero if there were any scrape errors rather than the current default in https://github.com/Fried-Chicken/ghost-static-site-generator/blob/master/src/helpers/crawlPageHelper/crawlPageHelper.js#L19 which is just to write to stdout.

It'd also be nice if that file output the wget command that failed in the log too, so you can run it manually and try and work out what's up.

After writing a blog post and trying to generate the website, I found that images are not loaded. After looking at the console for the live version, I found out that if I specify an url (The site is live on matfire.github.io), which in my case would be matfire.github.io, then all the urls are messed up.

If I click on the Home button, I am redirected to https://matfire.github.io/matfire.github.io/ (and if I click again another /matfire.github.io/ is appended to the url)

This is a screenshot of the medias that are loaded on an article on https://matfire.github.io

I also get this error while generating the static files

I am getting:

/bin/sh: fork: Resource temporarily unavailable
Error: Command failed: wget --help | grep "content-on-error" || true
/bin/sh: fork: Resource temporarily unavailable

    at genericNodeError (node:internal/errors:956:15)
    at wrappedFn (node:internal/errors:510:14)
    at checkExecSyncError (node:child_process:890:11)
    at execSync (node:child_process:962:15)
    at contentOnError (/Users/dmd/ghost-static-site-generator/src/helpers/crawlPageAsyncHelper/crawlPageAsyncHelper.js:6:34)
    at crawlPageAsyncHelper (/Users/dmd/ghost-static-site-generator/src/helpers/crawlPageAsyncHelper/crawlPageAsyncHelper.js:36:57)
    at /Users/dmd/ghost-static-site-generator/src/helpers/responsiveImagesHelper/responsiveImagesHelper.js:67:5
    at Array.forEach (<anonymous>)
    at responsiveImagesHelper (/Users/dmd/ghost-static-site-generator/src/helpers/responsiveImagesHelper/responsiveImagesHelper.js:61:12)
    at generateStaticSite (/Users/dmd/ghost-static-site-generator/src/commands/generateStaticSite.js:52:5) {
  status: 128,
  signal: null,
  output: [
    null,
    <Buffer >,
    <Buffer 2f 62 69 6e 2f 73 68 3a 20 66 6f 72 6b 3a 20 52 65 73 6f 75 72 63 65 20 74 65 6d 70 6f 72 61 72 69 6c 79 20 75 6e 61 76 61 69 6c 61 62 6c 65 0a>
  ],
  pid: 90699,
  stdout: <Buffer >,
  stderr: <Buffer 2f 62 69 6e 2f 73 68 3a 20 66 6f 72 6b 3a 20 52 65 73 6f 75 72 63 65 20 74 65 6d 70 6f 72 61 72 69 6c 79 20 75 6e 61 76 61 69 6c 61 62 6c 65 0a>
}

Hi, this issue has been brought up earlier but it has been closed and not re-opened for some reason, so I am creating a new one specifically for this issue:

My builds-folder is empty, thus no JS or CSS is loaded into the static site.

If you go to https://karlstoney.com/2018/02/23/nginx-ingress-modsecurity-and-secchatops/ you'll see:

Hi I tried this out and generated a static site from a Ghost 2 instance but the resulting files don't include any CSS, this that correct?

Hey,
I use formspree for "contact me" https://karlstoney.com/contact - which upon success redirects back to https://karlstoney.com/thanks.

/thanks exists in ghost as a page but it isn't being included in my static site content.

You can see it in my sitemap here: https://karlstoney.com/sitemap-pages.xml

Not sure what's going wrong, any ideas?

user@server:/site# ls static
404    assets  contact	contribute  grow	  portal   public      sell		    sitemap-pages.xml  sitemap-tags.xml  sitemap.xsl  welcome
about  author  content	design	    integrations  privacy  robots.txt  sitemap-authors.xml  sitemap-posts.xml  sitemap.xml	 tag	      write

This makes the static site un-usable as web hosts like Netlify can't find the start page for the domain

I have used GSSG to generate static site from my local Ghost application running at http://localhost:2368. It executed without any error but the Navigation URLs are not being converted. For example, there is an About link in the primary navigation bar which should be changed to /about or https://myproduction.com/about, but it remains http://localhost:2368/about in the generated static site. Command I have used:

gssg --productionDomain 'https://myproduction.com'

Also, the src of images also remains http://localhost:2368/content/images/2022/04/image.jpg in the generated static site. Even, same is happening for the homepage URL at the top navbar. This remains http://localhost:2368 instead of / or https://myproduction.com/.

I am using Ghost version 4.44.0.

Edit
I understand that, navigation bar URLs should be set manually from Ghost admin dashboard but the image src is still not getting changed to the production one, thus getting broken images all over the static site.

In one theme that I use I have an image of size w1140 stored in /content/images/size/w1140/2021/02/nodes-1.png but when I use gssg the only sizes that are being pulled are:

$ ls content/images/size/
w1000  w150  w600  w750  w960

It seems the sizes above w1000 are ignored. It's easy to replicate. One can use the theme "Dawn" available here then create a post in Ghost and in "post settings" change the post image (and use a large enough image).

Hi, there seems to be a case where images are missing, due to not being resized by Ghost. This is specifically for animated gifs. When the theme requests a resized image and that image is an animated GIF, Ghost will just serve the original image instead of a resized one. It doesn't save a resized GIF in any size folder, it just serves the original image from the server. This is leading to missing resized GIF images. Are you just not giving the script enough time for Ghost to serve the image? How do we fix this?

Originally posted by @LDubya in #8 (comment)

When I create my docker container for Ghost I would prefer to set the local URL as HTTP://serverip:2368 so I can work on it locally from another computer (this is why ghost defaulting to localhost:2368 wouldn't work). If I define this in the docker file though even though that I have --productionDomain www.mydomain.com defined, when I generate the static files it still defaults putting HTTP://serverip:2368 into all the links instead of mydomain.com. My current solution is to define mydomain.com in the docker container but this makes working on it locally virtually impossible as the server I have hosting it doesn't have a GUI.