dahie / caramelize Goto Github PK

Tool to migrate legacy wikis/documentation to markdown git-repository retaining history and syntax.

License: MIT License

Ruby 100.00%

wiki-migration gollum conversion redmine wiki markdown mediawiki obsidian-notes pandoc

caramelize's Issues

Improve markup configuration for input wikis

The config interface for wiki markups needs improvement. The current version handles markdown, but is not as flexible as should be.

Config option for Home-page

Wikis may have home pages that have a distinct name. It should be possible to define this name of the Home-page in the caramel.rb.
On conversion this Home-page gets renamed to "Home" to be used as gollum's default start page.

Use ActiveRecord for database-access

The config files could be much easier by using ActiveRecord.

Grit has encoding problems

Some migrations fail with a grit error pointing to incompatible character encodings. This issues has been reported with grit before and could be addressed by switching to a different git adapter.

.rvm/gems/ruby-2.1.5@caramelize/gems/gitlab-grit-2.7.3/lib/grit/index.rb:176:in `%': incompatible character encodings: UTF-8 and ASCII-8BIT (Encoding::CompatibilityError)
  from /Users/danielsenff/.rvm/gems/ruby-2.1.5@caramelize/gems/gitlab-grit-2.7.3/lib/grit/index.rb:176:in `block in write_tree'
  from /Users/danielsenff/.rvm/gems/ruby-2.1.5@caramelize/gems/gitlab-grit-2.7.3/lib/grit/index.rb:171:in `each'
  from /Users/danielsenff/.rvm/gems/ruby-2.1.5@caramelize/gems/gitlab-grit-2.7.3/lib/grit/index.rb:171:in `write_tree'
  from /Users/danielsenff/.rvm/gems/ruby-2.1.5@caramelize/gems/gitlab-grit-2.7.3/lib/grit/index.rb:123:in `commit'
  from /Users/danielsenff/.rvm/gems/ruby-2.1.5@caramelize/gems/gollum-grit_adapter-0.1.3/lib/grit_adapter/git_layer_grit.rb:241:in `commit'
  from /Users/danielsenff/.rvm/gems/ruby-2.1.5@caramelize/gems/caramelize-0.3.0/lib/caramelize/ext.rb:10:in `commit'
  from /Users/danielsenff/.rvm/gems/ruby-2.1.5@caramelize/gems/gollum-lib-4.0.2/lib/gollum-lib/wiki.rb:453:in `update_page'
  from /Users/danielsenff/.rvm/gems/ruby-2.1.5@caramelize/gems/caramelize-0.3.0/lib/caramelize/gollum_output.rb:26:in `commit_revision'
  from /Users/danielsenff/.rvm/gems/ruby-2.1.5@caramelize/gems/caramelize-0.3.0/lib/caramelize/gollum_output.rb:38:in `block in commit_history'
  from /Users/danielsenff/.rvm/gems/ruby-2.1.5@caramelize/gems/caramelize-0.3.0/lib/caramelize/gollum_output.rb:35:in `each'
  from /Users/danielsenff/.rvm/gems/ruby-2.1.5@caramelize/gems/caramelize-0.3.0/lib/caramelize/gollum_output.rb:35:in `each_with_index'
  from /Users/danielsenff/.rvm/gems/ruby-2.1.5@caramelize/gems/caramelize-0.3.0/lib/caramelize/gollum_output.rb:35:in `commit_history'
  from /Users/danielsenff/.rvm/gems/ruby-2.1.5@caramelize/gems/caramelize-0.3.0/lib/caramelize/content_transferer.rb:93:in `commit_history'
  from /Users/danielsenff/.rvm/gems/ruby-2.1.5@caramelize/gems/caramelize-0.3.0/lib/caramelize/content_transferer.rb:39:in `execute'
  from /Users/danielsenff/.rvm/gems/ruby-2.1.5@caramelize/gems/caramelize-0.3.0/lib/caramelize/cli.rb:69:in `commence_transfer'
  from /Users/danielsenff/.rvm/gems/ruby-2.1.5@caramelize/gems/caramelize-0.3.0/lib/caramelize/cli.rb:56:in `transfer_content'
  from /Users/danielsenff/.rvm/gems/ruby-2.1.5@caramelize/gems/caramelize-0.3.0/lib/caramelize/cli/run_command.rb:28:in `execute'

Detect if wiki.git by the same name exists

When running '$ caramelize run' multiple times, it should ask whether to overwrite the existing wiki.git folder.

Write DSL for configuration files

Instead of importing the config and running eval, it would be much better to create a simple Domain Specific Language to write config files.

Error extracting pandoc's information

When I run most caramelize functions, including help, I get this error:

$ caramelize new --trace
Error extracting pandoc's information: undefined method `[]' for nil:NilClass

        @scripting_engine = version_string.match(/Scripting engine: (.+)$/)[1]
                                                                           ^^^
Using made up values instead.
/var/lib/gems/3.1.0/gems/commander-5.0.0/lib/commander/user_interaction.rb:362:in `method_missing': undefined local variable or method `target_file' for main:Object (NameError)
        from /var/lib/gems/3.1.0/gems/caramelize-1.3.0/bin/caramelize:25:in `block (2 levels) in <top (required)>'
        from /var/lib/gems/3.1.0/gems/commander-5.0.0/lib/commander/command.rb:187:in `call'
        from /var/lib/gems/3.1.0/gems/commander-5.0.0/lib/commander/command.rb:157:in `run'
        from /var/lib/gems/3.1.0/gems/commander-5.0.0/lib/commander/runner.rb:444:in `run_active_command'
        from /var/lib/gems/3.1.0/gems/commander-5.0.0/lib/commander/runner.rb:58:in `run!'
        from /var/lib/gems/3.1.0/gems/commander-5.0.0/lib/commander/delegates.rb:18:in `run!'
        from /var/lib/gems/3.1.0/gems/commander-5.0.0/lib/commander/import.rb:7:in `block in <top (required)>'

I have pandoc 2.17.1.1:

pandoc 2.17.1.1
Compiled with pandoc-types 1.22.2.1, texmath 0.12.4, skylighting 0.12.3.1,
citeproc 0.6.0.1, ipynb 0.2
User data directory: /home/cswingley/.local/share/pandoc
Copyright (C) 2006-2022 John MacFarlane. Web:  https://pandoc.org
This is free software; see the source for copying conditions. There is no
warranty, not even for merchantability or fitness for a particular purpose.

This is Debian bookworm, Linux x86_64.

Support Trac wiki

As a user running a trac with a wiki
When I want to shutdown the trac instance
Then I can use caramelize to migrate my wiki to Gollum

Notes

https://trac.edgewall.org/wiki/WikiFormatting
commit 6ef1f2c has some starting points

Metrics for measuring migration quality

As a user
When I migrate a wiki
Then I'd like to know how successful it was

needs definition of KPIs
possible KPI: number of pages not linked in pages, number of links without corresponding page, each before and after conversion

Image support

As a wikka or redmine user
Given I have wiki pages with images
And I want to migrate the pages to Gollum
Then I want the images to be migrated correctly (syntax and files)

correctly migrate wikka image syntax
fetch images from remote sources and place them into gollum export

Possible to resume migration from certain point?

I tried to migrate a WikkaWiki database (the .sql file has a size of ~ 965 MB).

caramelize ran for 26 hours (in this time, it used all of my available RAM ~ 13 GB, and my complete swap ~ 14 GB), and it managed to migrate "(49637/63226)" before the process got killed.

Is there a way to resume the migration from this point?

I fear that the process will again get killed before completion if I’d have to start from the beginning.

dahie / caramelize Goto Github PK

caramelize's Issues

Improve markup configuration for input wikis

Config option for Home-page

Use ActiveRecord for database-access

Grit has encoding problems

Detect if wiki.git by the same name exists

Write DSL for configuration files

Error extracting pandoc's information

Support Trac wiki

Metrics for measuring migration quality

Image support

Possible to resume migration from certain point?

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent