Comments (25)
I wanted to try wuffs to build a simple CLI tool to decode the very simple TFRecord format but I ran into a few challenges trying to get started. I'll list them here and then file independent issues.
- The documentation about how to write your own .wuffs code is hard to grok. It has an incredibly trivial hello example, but no more documentation showing how to write a decoder.
- The coroutines document didn't contain a single coroutine example
- The i/o document was low level about iobuffer but didn't offer any examples about how to actually use io_reader or io_writer.
- I looked for both an IntelliJ or VSCode language plugin to help me learn to write wuffs, but there's nothing. I'd want, at least, syntax highlighting to help me know if I've made a typo. Better would be code completion so when I type arg.src. it would tell me what methods I could call on the io_reader.
- I want to use wuffs in a Bazel based project, so I'd like to see a starlark library I could include that would paper over any toolchain stuff I don't want to learn to just get started.
- I'd love to be able to just write wuffs and compile it into a program to decode my file, rather than have to write a bunch of C code. Maybe what I need is just to start with the zcat c code and replace the gzip code with my decoder?
- The example/ directory doesn't contain any examples of wuffs code?! I don't want to have to read C code.
- No CRC32C support, as far as I can tell, even though the README is careful to point out that there are two common formats of CRC32.
It would be great if there was a tutorial example of a whole project step by step, much like what I'm trying to do. E.g. "create decoder.wuffs. paste in this tiny bit. create decoder.c. paste in this tiny bit. create Makefile. paste in this tiny bit. run make. try your program. see you've built cat. modify wuffs code adding this bit to verify x. run make. see your cat now requires uppercase starting letter. replace the last tiny bit with this tiny bit. see your cat now requires an md5 checksum to match the content before it will print it out" or some other not-quite trivial example that uses some features of the language progressively.
As it is the documentation seems mostly aimed to people who already know what they're doing here.
from wuffs.
Shouldn't come as a surprise, but I use it :) currently playing with it for https://github.com/mvdan/zstd. A bit rough and unstable, but now that I can have method calls, I should be able to implement more and more bits of zstd.
By the way - I know you created a mailing list, and I subscribed - but I created a #wuffs channel on gophers.slack.com too. Not much happening there yet, but just in case it wasn't on your radar.
from wuffs.
My main issue with the codebase is that although the documentation is well above average, it's also quite lacking in places, in particular around the C API, and it's very hard to locate the pieces that are present.
and
I've discovered the much simpler Auxiliary C++ API long time after I spent quite some time integrating with the C API and implementing robust metadata parsing procedure, so the docs might need some adjustment in that regard
Thanks for the feedback. The C API could still do with some more docs per se, but I've recently pushed 129d800 that should contain a few more signposts for anyone in the future in a similar situation.
from wuffs.
When using Wuffs, do you guys have any syntax highlighting? If so, which editor/extension do you use? I want to try Wuffs, but programming without syntax highlighting seems intimidating. Thanks!
Sorry, not a useful note about users (yet!) but this issue is the first hit for [vim syntax wuffs] so I wanted to drop a note about a quick hack vim syntax repo here https://github.com/sgraham/vim-wuffs .
It's "Christmas-tree style" though per my eye-preferences, and despite Nigel's protestations above. Patches welcome!
from wuffs.
Wuffs' standard library has a JPEG decoder, which supports progressive JPEGs. Its GIF and PNG decoders also support interlacing and progressive output (producing an intermediate image when not all of the source bytes have been received).
Wuffs has no JPEG-XL decoder yet.
Wuffs does not have a 16k pixel limit. You may be thinking of WebP instead (which does have that limit).
from wuffs.
Thanks to metallb/metallb#5 for the inspiration.
from wuffs.
The Wuffs GIF decoder is being trialled by Skia, the 2-D graphics library used by both the Android operating system and the Chromium web browser.
from wuffs.
When using Wuffs, do you guys have any syntax highlighting? If so, which editor/extension do you use? I want to try Wuffs, but programming without syntax highlighting seems intimidating. Thanks!
from wuffs.
I personally program without any editor support, which is acceptable given there is wuffsfmt.
from wuffs.
I use vim, but my use of syntax highlighting probably isn't what you're expecting. I wrote more about this at https://nigeltao.github.io/blog/2018/colorful-text.html
$ cat ~/.vim/syntax/wuffs.vim
" wuffs.vim: Nigel Tao's vim syntax file for Wuffs.
if exists("b:current_syntax")
finish
endif
let b:current_syntax = "wuffs"
syn case match
syn match wuffsSpaceError display excludenl "\s\+$"
syn match wuffsAssertion /^ *assert .*/
syn match wuffsAssertion /^ *inv .*/
syn match wuffsAssertion /^ *pre .*/
syn match wuffsAssertion /^ *post .*/
syn match wuffsComment /\/\/.*/
syn match wuffsDeclaration /^packageid .*/
syn match wuffsDeclaration /^pri .*/
syn match wuffsDeclaration /^pub .*/
syn match wuffsDeclaration /^use .*/
hi wuffsAssertion ctermfg=darkcyan
hi wuffsComment ctermfg=gray
hi wuffsDeclaration ctermfg=yellow
hi def link wuffsSpaceError Error
from wuffs.
I don't use Wuffs, but I've written a drop-in replacement for GIFLIB that I believe is even faster than Wuffs at LZW decoding. I use the output image as the dictionary and for images that compress well, it can decode > 20x faster than the original GIFLIB. I have only implemented the new API for most features, but if you need speed, it may be worth a look --> https://github.com/bitbank2/giflib-turbo
from wuffs.
@davidmankin thanks for the list. They're valid issues, but fixing all of them is also a sizable amount of work. As the trivial /hello-wuffs-c
example that you've already seen states, we've "prioritized users of Wuffs the Library over users of Wuffs the Language" for now. To set your expectations, that prioritization isn't likely to change in the short term without additional people to work on it.
from wuffs.
Thanks for the reply. I hoped that maybe since the caveat was written in 2019 that we'd be into the supporting users of the language. I'm not sure I understand what I'd use the library for. Maybe if I get far enough to understand the wuffs the language I could at least help contribute docs. As it is, I have no idea if what i'm doing is right or will work…
from wuffs.
My image viewer has reached general usefulness, and I'm inching towards something that could be called a stable version.
I need PNG metadata readout: sRGB/iCCP/gAMA (for display, definitely want to take care of applying these independently) and complete support of text chunks (for thumbnails). I've recently seen commits tackling the former, but until text is finished, I must use another library.
I probably won't use write support for that format, but it would be next on my list.
My main issue with the codebase is that although the documentation is well above average, it's also quite lacking in places, in particular around the C API, and it's very hard to locate the pieces that are present.
from wuffs.
I need PNG metadata readout: sRGB/iCCP/gAMA... and complete support of text chunks.
Let's move the sRGB/iCCP/gAMA discussion to #39 and the text chunk discussion to the freshly created #55.
from wuffs.
My image viewer has reached general usefulness,
@pjanx I don't have an account to file an issue on the project page, so I'll mention it here. Where fastiv-io.c
has this line:
#define WUFFS_IMPLEMENTATION
You might want to also copy/paste this recent addtion to the Wuffs example programs:
// Defining the WUFFS_CONFIG__STATIC_FUNCTIONS macro is optional, but when
// combined with WUFFS_IMPLEMENTATION, it demonstrates making all of Wuffs'
// functions have static storage.
//
// This can help the compiler ignore or discard unused code, which can produce
// faster compiles and smaller binaries. Other motivations are discussed in the
// "ALLOW STATIC IMPLEMENTATION" section of
// https://raw.githubusercontent.com/nothings/stb/master/docs/stb_howto.txt
#define WUFFS_CONFIG__STATIC_FUNCTIONS
This can noticably impact build times and binary sizes.
Also, there's a comment later:
// We might want to employ a more generic way of magic identification,
// and with some luck, it could even be integrated into Wuffs.
which might be obsolete, given that a few lines later, you say:
switch (wuffs_base__magic_number_guess_fourcc(prefix))
from wuffs.
It's fine, Gitea is sorely lacking the cross-account ability of GitLab, and I recently got persistently annoying spammers over direct registration. Mail would also be fine for contact.
I have actually tried the static functions some time ago and decided against them, as it didn't improve much for me, partly due to already using so much of the library. To the contrary, it produced -Wunused-function
warnings that I didn't want to disable. If memory serves me right.
Wuffs is currently limited in what it can identify. At some later point, I realised Wuffs' ‘registry’ is more or less arbitrary, and can be extended, so that comment could use some adjustments indeed. I actually might need to merge the common guesses of shared-mime-info (file-based), Wuffs (hardcoded), and gdk-pixbuf (plugin-based).
from wuffs.
It turns out I haven't had a good look at Gitea authentication settings, I should have GitHub OAuth sign-in now.
Anyway, I'm here to say that Abydos also has a Wuffs plugin for image loading: http://snisurset.net/code/abydos/
from wuffs.
I've got acquainted with Wuffs the Library when I was looking for a faster way to decode PNG images for the commercial computer vision product I develop at work. Previously it was using OpenCV (hence libpng with some wrapping code), and replacing it with Wuffs improved the decoding performance tremendously. I also look forward to the upcoming JPEG decoding support in Wuffs which seems very promising because as of now it shows performance comparable to libjpeg-turbo without even using SIMD.
What I found surprising then is that Wuffs is barely known by computer vision community while they care a lot about image decoding performance typically being a bottleneck in a neural network training pipeline or even in a production integration. For example, the NVIDIA's DALI library (a collection of highly optimized building blocks for loading and processing image) which is used heavily by CV engineers, still uses OpenCV (hence libpng with the default zlib under the hood) for PNG decoding.
From my perspective, one of the reasons for this unfair lack of attention and adoption of Wuffs the Library is its steep learning curve due to a complex low level API (btw, I've discovered the much simpler Auxiliary C++ API long time after I spent quite some time integrating with the C API and implementing robust metadata parsing procedure, so the docs might need some adjustment in that regard) and limited availability, programming language wise. So I decided to write a Python interface for Wuffs the Library starting with image decoding. The package is already available here, and I hope that it'll be useful for some folks and will help people to discover Wuffs - the hidden gem of image decoding.
from wuffs.
@nigeltao
Thank you for adjusting the docs and for adding pywuffs mention!
from wuffs.
Just enabled Wuffs for reading PNGs in glare-core: glaretechnologies/glare-core@bd7fc5e
Which is used in Substrata, a Metaverse project: https://substrata.info/
from wuffs.
Some benchmarks:
doDecodeFromBufferLibPNG took 14.877 ms
doDecodeFromBufferWithWuffs took 8.5301 ms
Not checking CRCs with either LibPNG or Wuffs.
Windows 10, Visual Studio 2022, x64, no enhanced instruction set (e.g. no AVX). AMD Ryzen 9 5950X.
Optimisations enabled (of course).
from wuffs.
Hello ! I would be interested in using wuffs to decode jpeg xl !
However I remember reading, I don't remember where, that wuffs can only work on size limited image (16k pixels or so), is that true ?
And before diving in, I would like to know if wuffs could handle progressiveness ?
from wuffs.
my bad, I found where I read about this limit, and it was indeed for webp https://github.com/nigeltao/qoir
As a matter of fact, I was thinking of using the wuffs language, not the library...
But I tried to gather documentation on the format and the topic is complex ! A jpeg XL decoder might be available in the library before I could successfully implement one myself 😄
from wuffs.
We integrated Wuffs into the Arm ASTC texture compressor about 18 months ago, primarily for faster PNG loading, but the improved robustness for bad images is a huge benefit. Thanks!
https://github.com/ARM-software/astc-encoder
from wuffs.
Related Issues (20)
- RGB/BGR 16 bit treated like RGBA/BGRA? HOT 1
- OSS-Fuzz issue 59018 HOT 1
- [JPEG] unsupported DQT after SOF markers HOT 1
- OSS-Fuzz issue 59182 HOT 1
- OSS-Fuzz issue 59540 HOT 1
- OSS-Fuzz issue 59966 HOT 1
- A question regarding auxiliary C++ API HOT 4
- What is the status of version 0.3? HOT 3
- Empty slice manipulation triggers UBSAN by offsetting from a null pointer. HOT 2
- error: conversion to ‘uint32_t’ {aka ‘unsigned int’} from ‘int’ may change the sign of the result HOT 3
- OSS-Fuzz issue 66816 HOT 1
- PNG's are stored in RGB order but Wuffs returns BGR/BGRA? HOT 1
- Decode PNG with gray+alpha as 2 channels (i.e. YA not BGRA) HOT 5
- Warning about always true comparison of integers HOT 1
- std/crc64 doesn't build for 32-bit x86 HOT 1
- Allowing LA and RGBA PNGs with a tRNS chunk HOT 2
- How to get the correct 'transparency' value in the DecodeImage API? HOT 2
- wuffs 0.4 significantly slower than 0.3 decoding PNGs HOT 27
- wuffs significantly slower than OpenCV 4.9.0 when decoding PNGs for 7680x4320 image HOT 9
- OSS-Fuzz issue 70340 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from wuffs.