base64 decoding function about jq HOT 20 CLOSED

As this is showing up on Google a lot, and good documentation on jq is sparse, here is to everybody who lands here:

Example:

echo '{"foo": "Ym9iIGxpa2VzIGFsaWNlCg=="}' | jq '.foo | @base64d'

Or even use it when building new objects:

echo '{"foo": "Ym9iIGxpa2VzIGFsaWNlCg=="}' | jq '{encoded: .foo, decoded: .foo | @base64d}'

from jq.

+1 for base64 decoding

from jq.

@ameissnersofi I believe @base64d isn't in 1.5 release, it might be in 1.6 though.

Edit: I built latest from source and confirmed it is there and working:

$ ./jq --version
jq-1.6rc1-10-g7fd9e86-dirty
$ echo '{"foo": "Ym9iIGxpa2VzIGFsaWNlCg=="}' | ./jq '.foo | @base64d'
"bob likes alice\n"

from jq.

@mterron -- jq is JSON-oriented with further limitations based on UTF-8 requirements. In particular, @base64 should only be expected to operate on UTF-8 strings. Similarly, @base64d has the necessary limitation that it has, as described in the documentation. Thus it's fair to say that both jq filters are appropriately named, despite these inherent limitations.

from jq.

Thanks for the report @mterron - I'd suggest starting a new issue so we can track a fix for it there. I'll try to take a look at this tomorrow.

Edit: Actually, I remember now that if the encoded value is not a UTF-8 string than the results of the decode are undefined.

Then it's not actually base64 and should be renamed to something else maybe @utfb64 or something along those lines?

I opened #1931 to track this.

from jq.

@stdeolan FYI there's several variants of base64 encoding :( Also, a decoder would be equally nice now that we have fromjson.

from jq.

I find myself also wanting a base64 decoder. Are there any updates on this feature?

from jq.

from jq.

Thank you @Overbryd! After building jq in a Docker container (fixing the Dockerfile first), this works for me.

from jq.

@Overbryd , when I run your example of decoding, I get an error.

$ jq --version
jq-1.5
$ echo '{"foo": "Ym9iIGxpa2VzIGFsaWNlCg=="}' | jq '.foo | @base64d'
jq: error (at <stdin>:1): base64d is not a valid format

However, if I run @base64 to encode things, that works just fine. Am I doing something wrong?

from jq.

@ameissnersofi My best guess is that you have to have a recent version of jq. @base64d was only added recently.

from jq.

@Overbryd oh I'm sorry. I thought you were actively developing jq. My bad. :/

@l8nite - can you verify my issue (assuming you are developing - the commit history seems to indicate so lol) @stedolan ?

Thanks!

from jq.

@l8nite awesome! I see that 1.6 is in rc1 right now, which is probably why brew didn't pick it up for me. Thanks so much!

from jq.

@ameissnersofi - Have you tried brew install --HEAD jq ?

from jq.

Great job all, on the feature and documenting its access.

from jq.

@base64d seems to have issues with binary values, or at least does not match base64 -d:

$ echo "V0MsL2hwbyCcGs2AMrFAKaSjPl8OuX4OWAEip+idGVU=" | jq -Rr '@base64d' | xxd -p -c64
57432c2f68706f20efbfbd1acd8032efbfbd4029efbfbdefbfbd3e5f0eefbfbd7e0e580122efbfbdefbfbd19550a
$ echo "V0MsL2hwbyCcGs2AMrFAKaSjPl8OuX4OWAEip+idGVU=" | base64 -d | xxd -p -c64
57432c2f68706f209c1acd8032b14029a4a33e5f0eb97e0e580122a7e89d1955
$ echo "V0MsL2hwbyCcGs2AMrFAKaSjPl8OuX4OWAEip+idGVU=" | base64 -d | hexdump -C
00000000  57 43 2c 2f 68 70 6f 20  9c 1a cd 80 32 b1 40 29  |WC,/hpo ....2.@)|
00000010  a4 a3 3e 5f 0e b9 7e 0e  58 01 22 a7 e8 9d 19 55  |..>_..~.X."....U|
$ echo "V0MsL2hwbyCcGs2AMrFAKaSjPl8OuX4OWAEip+idGVU=" | jq -Rr '@base64d' | hexdump -C
00000000  57 43 2c 2f 68 70 6f 20  ef bf bd 1a cd 80 32 ef  |WC,/hpo ......2.|
00000010  bf bd 40 29 ef bf bd ef  bf bd 3e 5f 0e ef bf bd  |..@)......>_....|
00000020  7e 0e 58 01 22 ef bf bd  ef bf bd 19 55 0a        |~.X.".......U.|

from jq.

Thanks for the report @mterron - I'd suggest starting a new issue so we can track a fix for it there. I'll try to take a look at this tomorrow.

Edit: Actually, I remember now that if the encoded value is not a UTF-8 string than the results of the decode are undefined.

from jq.

On my brand new centos 8 system, I yum installed jq, tried this and hit:
jq --version
jq-1.5
:(

from jq.

It works in jq-1.6

from jq.

Yup. but top of the line, epel is only jq-1.5, so to a bunch of users, it doesn't exist. :( Anyone maintain the jq version on epel?

from jq.