minicomp / wax_tasks Goto Github PK
View Code? Open in Web Editor NEWUtility tasks for processing collection data with Wax π
Home Page: https://www.rubydoc.info/github/minicomp/wax_tasks/master
License: MIT License
Utility tasks for processing collection data with Wax π
Home Page: https://www.rubydoc.info/github/minicomp/wax_tasks/master
License: MIT License
it's currently hard to read/write tests, because rspec best practices are being followed; need to commit time to fixing
Is your feature request related to a problem? Please describe.
This happened twice to me already in completely different contexts, so I'm guessing it will be a relatively common problem. Terminal spits out a nil value error. We might be able to be more specific about the nature of the error so producers can quickly address.
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.
currently: warns that there is no record found (which is correct behavior) but then throws
NoMethodError: undefined method
pid' for nil:NilClass` when trying to update metadata
Learned the hard and painful way
Is your feature request related to a problem? Please describe.
Putting the layout in the front matter of a wax page is unnecessary since it is required to put in the _config.yml file and Jekyll will auto populate the layout from the _config.yml file. If you change it in the _config.yml file you have to reprocess all your pages file, which is unnecessary extra work for the user.
Describe the solution you'd like
Remove the piece of code that puts the layout in the front matter of every wax page.
Describe alternatives you've considered
There could be a setting that will put it in the front matter, but this should not be the default.
Additional context
Line 15
https://github.com/minicomp/wax/blame/e4dcaa10a28b427cd502ac960b8943952805ed7d/_qatar/obj10.md
Currently Wax works with .tiff but not .tif
^ copied from minicomp/wax#55
BUG: wax:iiif
includes source
dir specified in _config.yml
in path for image resources (e.g., thumbnail path, manifest path, etc.)
MUST FIX and write tests before release
... so that you can stop & start wax:derivatives
task and wax will keep track of progress made.
Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):
Smartphone (please complete the following information):
Additional context
Add any other context about the problem here.
This would require a decent amount of work to refactor but would be worth it.
Skips writing derivative (in memory) to disk if target file already exists
Should skip creating derivative at all if target file already exists.
This means collections need to know all* of their target derivative paths in advance.
If successful, derivative tasks would take the same amount of time the first time they are run, but would be exponentially quicker when rerun.
Is your feature request related to a problem? Please describe.
When debugging an issue in a site, it's important to be able to start over from a clean slate. 'wax:clobber' generally allows this, but as @mnyrop notes in the issue creating it, #46, it doesn't eliminate columns that have been added to _data/{collection}.csv
. So if you want to actually do a clean install, you need to maintain a backup copy of your csv and start copying it into _data.
Describe the solution you'd like
That wax overwrites a user-created file with an altered version makes me squeamish. I don't know Jekyll, but every static system I've used maintains a bright line between user generated files and program-generated ones. Like, you would never script something that adds new lines to _config.yml as part of a build process? I can't see a clearcut case where this would be disastrous, but it's nice, for instance, to have the modification date on the data csv be the date that the user actually edited it, not the last time they ran a wax build that updated it.
So just have a file at like _{collection}/{collection}.csv that wax creates which is the user csv from
_data/{collection}.csv but with the columns order,layout,collection,thumbnail,full,manifest
, and whatever else you're creating added.
Describe alternatives you've considered
Leave it as is? I don't know what you're thinking of to allow this to get clobbered.
I've occasionally used a solution of having something called catalog_derived.json
in the same folder, but using the collection folder seems more robust to me.
Additional context
Add any other context or screenshots about the feature request here.
Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):
Smartphone (please complete the following information):
Additional context
Add any other context about the problem here.
Describe the bug
When using source: my-src-dir
configuration option in Jekyll (see: https://jekyllrb.com/docs/configuration/options/) the derivative tasks will mistakenly include that value as part of the URL path.
Expected behavior
Wax_tasks should take note of the source:
directory set and drop it from the image resource paths & URIs
derivatives:simple
should create an array of images to add to collection page front matter IF the item is pagedderivatives:simple
should use the first image for main thumbnail
and full
banner πEdit: These changes in
wax_tasks
would be best written into a v2.0 that makes use of tmp and state-tracking json files so that the array of paged derivatives & their variants can be captured without writing that info back into a cannonical CSV.Time to rip the band-aid off! π©Ήβ²οΈ
_qatar
) and images (e.g., img/derivatives
), and search index (e.g., search/index.json
)_data/raw_images
and _data/qatar.csv
)manifest
from _data/qatar.csv
).gitignore
by default??Copied from minicomp/wax#54
spec.required_ruby_version = '>= 2.4'
copied from wax #93
Describe the bug
When running the rake wax:search qatar
task, rake
exists with an error:
rake aborted!
WaxTasks::Error::InvalidConfig:
...
To Reproduce
Steps to reproduce the behavior:
bundle install
bundle exec rake wax:search qatar
Expected behavior
The /search/index.json
file to be generated.
Error Trace Output
$bundle exec rake wax:search qatar --trace
rake aborted!
WaxTasks::Error::InvalidConfig:
/Users/<user>/.rvm/gems/ruby-2.6.3/gems/wax_tasks-1.0.2/lib/wax_tasks/config.rb:54:in `search'
/Users/<user>/.rvm/gems/ruby-2.6.3/gems/wax_tasks-1.0.2/lib/wax_tasks/site.rb:40:in `generate_static_search'
/Users/<user>/.rvm/gems/ruby-2.6.3/gems/wax_tasks-1.0.2/lib/tasks/search.rake:12:in `block (3 levels) in <top (required)>'
/Users/<user>/.rvm/gems/ruby-2.6.3/gems/wax_tasks-1.0.2/lib/tasks/search.rake:12:in `each'
/Users/<user>/.rvm/gems/ruby-2.6.3/gems/wax_tasks-1.0.2/lib/tasks/search.rake:12:in `block (2 levels) in <top (required)>'
/Users/<user>/.rvm/gems/ruby-2.6.3/gems/rake-12.3.3/lib/rake/task.rb:273:in `block in execute'
/Users/<user>/.rvm/gems/ruby-2.6.3/gems/rake-12.3.3/lib/rake/task.rb:273:in `each'
/Users/<user>/.rvm/gems/ruby-2.6.3/gems/rake-12.3.3/lib/rake/task.rb:273:in `execute'
/Users/<user>/.rvm/gems/ruby-2.6.3/gems/rake-12.3.3/lib/rake/task.rb:214:in `block in invoke_with_call_chain'
/Users/<user>/.rvm/rubies/ruby-2.6.3/lib/ruby/2.6.0/monitor.rb:230:in `mon_synchronize'
/Users/<user>/.rvm/gems/ruby-2.6.3/gems/rake-12.3.3/lib/rake/task.rb:194:in `invoke_with_call_chain'
/Users/<user>/.rvm/gems/ruby-2.6.3/gems/rake-12.3.3/lib/rake/task.rb:183:in `invoke'
/Users/<user>/.rvm/gems/ruby-2.6.3/gems/rake-12.3.3/lib/rake/application.rb:160:in `invoke_task'
/Users/<user>/.rvm/gems/ruby-2.6.3/gems/rake-12.3.3/lib/rake/application.rb:116:in `block (2 levels) in top_level'
/Users/<user>/.rvm/gems/ruby-2.6.3/gems/rake-12.3.3/lib/rake/application.rb:116:in `each'
/Users/<user>/.rvm/gems/ruby-2.6.3/gems/rake-12.3.3/lib/rake/application.rb:116:in `block in top_level'
/Users/<user>/.rvm/gems/ruby-2.6.3/gems/rake-12.3.3/lib/rake/application.rb:125:in `run_with_threads'
/Users/<user>/.rvm/gems/ruby-2.6.3/gems/rake-12.3.3/lib/rake/application.rb:110:in `top_level'
/Users/<user>/.rvm/gems/ruby-2.6.3/gems/rake-12.3.3/lib/rake/application.rb:83:in `block in run'
/Users/<user>/.rvm/gems/ruby-2.6.3/gems/rake-12.3.3/lib/rake/application.rb:186:in `standard_exception_handling'
/Users/<user>/.rvm/gems/ruby-2.6.3/gems/rake-12.3.3/lib/rake/application.rb:80:in `run'
/Users/<user>/.rvm/gems/ruby-2.6.3/gems/rake-12.3.3/exe/rake:27:in `<top (required)>'
/Users/<user>/.rvm/gems/ruby-2.6.3/bin/rake:23:in `load'
/Users/<user>/.rvm/gems/ruby-2.6.3/bin/rake:23:in `<main>'
/Users/<user>/.rvm/gems/ruby-2.6.3/bin/ruby_executable_hooks:24:in `eval'
/Users/<user>/.rvm/gems/ruby-2.6.3/bin/ruby_executable_hooks:24:in `<main>'
Tasks: TOP => wax:search
Is your feature request related to a problem? Please describe.
I am trying to hook WAX up with GitHub actions so you can create derivatives via GitHub and don't have to download the repo. The problem I am running into is splitting PDFs doesn't work on GitHub actions using an Ubuntu machine and if you use a Mac to create the derivatives it takes twice as long to create the derivatives. What I would like to do is split the images on a Mac and then create the derivatives on an Ubuntu machine. I thought I would get away with running a ruby script that imported the wax_tasks library and run the script but it looks like all the methods are made private or something because they are not showing up as a method I can run using the library.
Describe the solution you'd like
A rake task to create the image derivatives.
Describe alternatives you've considered
Making the methods public to use when importing the library to ruby.
Additional context
n/a
In the "Requirements" section of the Metadata file description, it notes that the pid
:
should follow βsnake caseβ, where letters are lowercase, special characters are removed, and spaces are replaced by underscores
Is the lowercase requirement actually a requirement? Out PIDs and corresponding image files include uppercase characters. We could rename the files... but that turns a seamless production workflow into a slightly more complicated / error-prone one.
Reference:
wax_tasks/lib/wax_tasks/utils.rb
Line 111 in 39f570d
Wax has constraints on metadata that users might not immediately notice--e.g., that ids cannot include a period or uppercase letters, and that certain fields have special meanings.
If these rules are violated, a site will build but break in unpredictable ways. Would you consider instead throwing an error that explains what needs to happen differently? I suspect this will save users time by helping them identify where things have gone wrong.
Custom variant image sizes specified in _config.yml
for a given collection are not being generated.
To Reproduce
_config.yml
, following the example in the sample sitebundle exec rake wax:derivatives:iiif
Expected behavior
An image such as obj12_00
ought to have the following full-region images in img/derivatives/iiif/images/obj12_00/full/
: 50, 250, 1400, full
. Instead, only the default sizes are found: 250, full
.
Desktop (please complete the following information):
Additional context
This is an important feature to make Wax IIIF images fully reusable, since Universal Viewer always requests 90,
thumbnails. An item without 90,
thumbnails always looks broken in UV.
Describe the bug
There are are additional slashes between the hostname and the path for IIIF resources in manifests, etc.
To Reproduce
Steps to reproduce the behavior:
bundle exec rake wax:derivatives:iiif
bundle exec rake wax:pagemaster
bundle exec jekyll build
Expected behavior
Something likehttps://example.com/img/derivatives/iiif/MKP_392_0032/manifest.json
should appear instead of https://example.com//img/derivatives/iiif/MKP_392_0032/manifest.json
this will make it easier to check, e.g., {% if page.manifest %}
, since checking for empty string values is more tedious in liquid than in ruby.
Describe the bug
While generating, the message should indicate that the process is happening, rather than wait for it to finish. This is important for big images.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):
Smartphone (please complete the following information):
Additional context
Add any other context about the problem here.
Is your feature request related to a problem? Please describe.
This is probably quite a difficult thing to do. But it is quite hard to continuously update a wax site right now. If a CSV or exhibit is edited, it's hard to know if those changes are getting pushed through to the final version, so it often involves:
This takes 30-45 minutes on our collection.
Describe the solution you'd like
In an ideal world, I want to be able to run bundle exec wax serve
which wraps jekyll serve with continuous updating of changed files
Describe alternatives you've considered
Additional context
This flurry of issues is being caused by my having borked up my open sea dragon displays somehow, and it being extremely slow to trace the flow of changes in the upstream metadata to the user-facing HTML. Nothing in there is Wax's fault, I'm pretty confident, but it would still be nice for the platform to be debuggable.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google β€οΈ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.