0xvavaldi / gramify Goto Github PK

please change this:
print("cat output_file.txt | sort | uniq -c | sort -rn | grep -oP '^ [0-9]+ \K.' > sorted_output.txt")
to this:
print("cat output_file.txt | sort | uniq -c | sort -rn | grep -oaP '^ [0-9]+ \K.' > sorted_output.txt")

For some workflows, it would be very helpful if gramify could operate on stdin and/or send output to stdout.

Some string may be in $HEX[] format and gramify treats them as words, without decoding them first, leading to some "unusable" results in the output.

I suggest updating this recommendation so that it is more likely to work as expected.
cut -c9- cuts at the 9th character position. But the size of each row can vary; uniq -c adds padding before the number and the number may be large depending on how large the gramify output is.

% echo "password" | uniq -c | sort -rn | awk '($1 >=1)' | cut -c9-
sword

A simple way to do this in awk would be to use {print $2}, but that would remove any ngrams with a space.

So the solution I found is to use a bit more complex awk command:

awk '{if ($1 >=1) {$1=""; print substr($0, index($0, $2))}}'

You can try it out with different variations with this test command:

% printf -- 'pass word\n%.0s' {1..99999} | uniq -c | sort -rn | awk '{if ($1 >=1) {$1=""; print substr($0, index($0, $2))}}'
pass word