zlin / wgetpaste Goto Github PK

The new dpaste API seems to be mangling spaces, for example: https://dpaste.com/G9USGHUC9.txt

I ran wgetpaste on an ebuild as an example and the result is:
#+Copyright+1999-2021+Gentoo+Authors
#+Distributed+under+the+terms+of+the+GNU+General+Public+License+v2

EAPI="7"

DESCRIPTION="Command-line+interface+to+various+pastebins"
HOMEPAGE="http://wgetpaste.zlin.dk/"

COMMIT_HASH="f9b08c50009485817e8b44300f79a92e37fe9f22"
SRC_URI="https://github.com/zlin/wgetpaste/archive/${COMMIT_HASH}.tar.gz+->+${P}.tar.gz"

LICENSE="public-domain"
SLOT="0"
KEYWORDS="~alpha+~amd64+~arm+~arm64+~hppa+~ia64+~mips+~ppc+~ppc64+~riscv+~s390+~sparc+~x86+~amd64-linux+~x86-linux+~x64-macos"
IUSE="%2bssl"

RDEPEND="net-misc/wget[ssl?]"

S="${WORKDIR}/${PN}-${COMMIT_HASH}"

src_prepare()+{
sed+-i+-e+"s:/etc:"${EPREFIX}"/etc:g"+wgetpaste+||+die
default
}

src_install()+{
dobin+${PN}
insinto+/usr/share/zsh/site-functions
doins+_wgetpaste
}

[Sun-13-Sep  4:43pm] michael@ds-miche2:pts/6:2074 ~/fabfile/gentoo-install $ wgetpaste -s dpaste packages
Apparently nothing was received. Perhaps the connection failed. Enable --verbose or
--debug to get the output from wget that can help diagnose it correctly.

Enabling --verbose proves that the transfer/exchange is fine, but wgetpaste is, again, failing to handle either the redirect or detect the resultant 'answer'/paste correctly.

Apparently nothing was received. Perhaps the connection failed.
--2020-09-13 16:43:58--  http://dpaste.com/
Resolving dpaste.com... 35.173.69.207
Connecting to dpaste.com|35.173.69.207|:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: /28GU7WLAR [following]
--2020-09-13 16:43:58--  http://dpaste.com/28GU7WLAR
Reusing existing connection to dpaste.com:80.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: https://dpaste.com/28GU7WLAR [following]
--2020-09-13 16:43:59--  https://dpaste.com/28GU7WLAR
Connecting to dpaste.com|35.173.69.207|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 10866 (11K) [text/html]
Saving to: 'STDOUT'

https://wgetpaste.zlin.dk/ looks unmaintained. Unfortunately it has no link to the active github repository.
I suggest to forward the page to this repo, or at least add a link.

Don't use any ~/.wgetpaste.d/wgetpaste.conf.

# whatever XXX, always receive nothing
$ hello | wgetpaste -s XXX
Apparently nothing was received. Perhaps the connection failed. Enable --verbose or
--debug to get the output from wget that can help diagnose it correctly.
$ hello | wgetpaste --debug 2>&1 | tee a.log > /dev/null

a.log

It would be nice if there was a config file to add custom pastebin services to wgetpaste. For example, bpa.st is open source and easy to self host, and it would be nice if users could configure this instead of editing the wgetpaste code.

Just tried to upload a larger log file to bpaste and it fails like so:

user@pentoo ~ $ wgetpaste -s bpaste /tmp/pentoo-updater.log --verbose                                                                                                                         
Apparently nothing was received. Perhaps the connection failed.                                
--2023-03-21 17:27:00--  https://bpa.st/api/v1/paste                                                                                                                                          
Resolving bpa.st... 65.109.61.99, 2a01:4f9:5a:5159::2                                                                                                                                         
Connecting to bpa.st|65.109.61.99|:443... connected.                                                                                                                                          
HTTP request sent, awaiting response... 413 Request Entity Too Large                           
2023-03-21 17:27:03 ERROR 413: Request Entity Too Large.          

Smaller log files upload without issue.

I did some digging and it looks like bpaste.net limits the upload size to 256K.

It would be helpful if wgetpaste could provide an informational message warning users of this size limit when the bpaste service is selected.

http://lpaste.net/
I have no background of web, sorta weak to implement it myself.

Would it be possible to support anonymous paste to https://paste.debian.net/

After following Creating a personal access token to create my Github token with a Gists permission, and then following the (meager) instructions at the end of "wgetpaste --help",

the Token does not work as described "HEADER_gists="Authorization: token ***".

However this does work:
my example wgetpaste.conf token file fix

I spent a (large) amount of time trying to get "Authorization: token" to work, and I was unable to.
Only various forms of "Authentication: Basic" work now, with modifications:

Option 1) echo -n ${GITHUB_TOKEN}:x-oauth-basic | base64
Option 2) echo -n ${DEFAULT_NICK}:${GITHUB_TOKEN} | base64

This b64 encodes the token along with a mandatory 2nd field to be sent as the header, albeit in "Basic" mode, not "Token" mode. I prefer option 1, where the nickname username is optional.

If you compare most examples on the web, most are using CURL and this is a more typical way of how the headers get sent, I was unable to find any evidence of "Authorization: token" even working at all. Perhaps its outdated, or the API Changed? Im not sure.

I also want to mention, to please create your own personal token for testing Gists, so you don't have to "HARD_SKIP" the Auth Test @ https://github.com/zlin/wgetpaste/blob/master/test/test.sh#L18 . You can store it in an env variable, so you dont have to publish the token, and the test script will remain functional for other users, but you will have the ability to sanity check the Auth Test on your computer at least every 6 months.

Extra Notes:
Using base64 in the .conf file is allowed, since its bash, and as per your documentation, coreutils is required, and base64 comes from coreutils. Theoretically this could be embedded in the script but adequate documentation would also suffice.
I am mostly looking for confirmation on the issue, and also to document my findings here in the meantime.

Thanks!,
Gentoo greatly appreciates your script, it's of essential importance.

I recently got blocked from dpaste.com ; I sent an e-mail to enquire as of why and the response was something like It looks like you were blocked for attempting to (wildly) exceed the maximum post size of 250K. They were really nice and removed the block once I asked.

This happened by mistake, probably using wgetpaste (forgive me if I'm wrong).

wgetpaste should not allow a user to (unknowingly) exceed post limits.

Hi Bo,

I'd like to be able to default to secret gists when using the github service.
I see in the API docs that "public:false" is the default and I also see in wgetpaste that "public: true" is hardcoded in the gist JSON request. Is there any way this could be exposed so I could customize it from a config file?

Thanks

I recommend doing the following:

	jq -Mnc ".expiry = \"${expiration}\" | .files[0] = { name: \"${description}\", lexer: \"${language}\", content: \"${content}\" }"

It will add a dependency on the jq utility; however, jq can also be used in geturl to parse JSON output instead of relying on regular expressions.

Using jq also enables the removal of some of the escape functions used below.

Originally posted by @nvinson in #16 (comment)

At gentoo support channel I was given an advise to use

# wgetpaste -ic 'emerge -uUDpv @world'

it says

Apparently nothing was received. Perhaps the connection failed. Enable --verbose or
--debug to get the output from wget that can help diagnose it correctly.

and then

# wgetpaste -ic --verbose 'emerge -uUDpv @world'
/usr/bin/wgetpaste: emerge -uUDpv @world No such file found.

If I tried to use dpaste.com service instead of gists.github.com (with -s dpaste command line parameter) everything works ok.

UPD:
<kurly> your second command tried to run a command called --verbose and paste a file called 'emerge -uUDpv @world'
<kurly> there cannot be other arguments between -c and the command

When bpaste (or more recently bpa.st) throws a 304 redirect, it confuses wgetpaste and it fails. However, running with --verbose or --debug you can see the request is sent successfully and a URL is generated, it just fails to be parsed correctly.

Failed with (default) bpaste. Portion of debug log follows:

+ WGETARGS=--post-file=/tmp/wgetpaste.mgxBZK
+ header=HEADER_bpaste
+ [[ -n '' ]]
+ WGETEXTRAHEADER=
+ additional_headers=ADDITIONAL_HEADERS_bpaste
+ WGETADDITIONALHEADERS=()
+ [[ -n '' ]]
+ WGETARGS='--tries=5 --timeout=60 --post-file=/tmp/wgetpaste.mgxBZK'
+ geturl needstdout
+ local regex location
+ regex=REGEX_URL_pinnwand
+ location=REGEX_LOC_pinnwand
+ [[ -n s|^.*"link": "\([^"]\+bpa.st[^"]\+\)".*$|\1|p ]]
+ [[ needstdout = needstdout ]]
+ return 0
+ tail -n1
++ LC_ALL=C
++ wget -O - --tries=5 --timeout=60 --post-file=/tmp/wgetpaste.mgxBZK https://bpa.st/api/v1/paste
+ OUTPUT='--2023-05-20 06:26:32--  https://bpa.st/api/v1/paste
Resolving bpa.st... 65.109.61.99, 2a01:4f9:5a:5159::2
Connecting to bpa.st|65.109.61.99|:443... connected.
HTTP request sent, awaiting response... 400 Bad Request
2023-05-20 06:26:33 ERROR 400: Bad Request.'
+ [[ -f /tmp/wgetpaste.mgxBZK ]]
+ [[ -n 0 ]]
+ echo 'Left temporary file: "/tmp/wgetpaste.mgxBZK" alone for debugging purposes.'
Left temporary file: "/tmp/wgetpaste.mgxBZK" alone for debugging purposes.
++ geturl '--2023-05-20 06:26:32--  https://bpa.st/api/v1/paste
Resolving bpa.st... 65.109.61.99, 2a01:4f9:5a:5159::2
Connecting to bpa.st|65.109.61.99|:443... connected.
HTTP request sent, awaiting response... 400 Bad Request
2023-05-20 06:26:33 ERROR 400: Bad Request.'
++ local regex location
++ regex=REGEX_URL_pinnwand
++ location=REGEX_LOC_pinnwand
++ [[ -n s|^.*"link": "\([^"]\+bpa.st[^"]\+\)".*$|\1|p ]]
++ [[ needstdout = --2023-05-20 06:26:32--  https://bpa.st/api/v1/paste
Resolving bpa.st... 65.109.61.99, 2a01:4f9:5a:5159::2
Connecting to bpa.st|65.109.61.99|:443... connected.
HTTP request sent, awaiting response... 400 Bad Request
2023-05-20 06:26:33 ERROR 400: Bad Request. ]]
++ sed -n -e 's|^.*"link": "\([^"]\+bpa.st[^"]\+\)".*$|\1|p'
++ tail -n1
+ URL=
+ verifyurl
+ local t
+ [[ -z '' ]]
+ [[ -n 0 ]]
+ die 'Apparently nothing was received. Perhaps the connection failed.
--2023-05-20 06:26:32--  https://bpa.st/api/v1/paste
Resolving bpa.st... 65.109.61.99, 2a01:4f9:5a:5159::2
Connecting to bpa.st|65.109.61.99|:443... connected.
HTTP request sent, awaiting response... 400 Bad Request
2023-05-20 06:26:33 ERROR 400: Bad Request.'
+ echo 'Apparently nothing was received. Perhaps the connection failed.
--2023-05-20 06:26:32--  https://bpa.st/api/v1/paste
Resolving bpa.st... 65.109.61.99, 2a01:4f9:5a:5159::2
Connecting to bpa.st|65.109.61.99|:443... connected.
HTTP request sent, awaiting response... 400 Bad Request
2023-05-20 06:26:33 ERROR 400: Bad Request.'
Apparently nothing was received. Perhaps the connection failed.
--2023-05-20 06:26:32--  https://bpa.st/api/v1/paste
Resolving bpa.st... 65.109.61.99, 2a01:4f9:5a:5159::2
Connecting to bpa.st|65.109.61.99|:443... connected.
HTTP request sent, awaiting response... 400 Bad Request
2023-05-20 06:26:33 ERROR 400: Bad Request.
+ exit 1

dpaste.com does not work correctly with DNSSEC enabled as mentioned in the PR and commit message. A new default service seems like a good idea!

All of the other services seem to work correctly from my testing. I think bpaste is the most user friendly paste service after dpaste so in the PR I set that to the new default, if this isn't an acceptable default hopefully we can find another one that isn't dpaste.

Thanks!

#35

When using strace in wgetpaste, wgetpaste returns "Apparently nothing was received. Perhaps the connection failed." Also tried with services other than dpaste and got similar results.

Verbose output returns 204 no content.

Example: wgetpaste -c 'strace echo hello'

I'm using Gentoo.

Would it be possible to support paste to Sourcehut Pastebin?

I think this line should be removed or at least omitted unless --verbose. It doesn't seem good practice for a simple tool, just as if cat wrote "Here is your data:" to stderr on every invocation. It would be nice to just be able to run /exec -o wgetpaste file.txt in Weechat, too.

This is a great little utility.
Well used in #gentoo on irc.
But the output can be really hard to read at times.
The output is quite hard to read due to the ansi stuff at times.
Was wondering how hard it would be to add the --color=n to the emerge command.
So it be like, wgetpaste -ic 'emerge --ask --color=n gnome-base/gnome'.
Instead of wgetpaste -ic 'emerge --ask gnome-base/gnome'

Currently when wgetpaste is called with multiple input files, it concatenates all output and uploads it as a single file with cat filename as beginning of file marks.

If the files are of different types or from different languages, when uploading to gist.github.com, the syntax highlighting engine there has a hard time and often makes the paste hard to read.

Since gists support upload of multiple files by passing multiple entries in the "files" section of the json payload, I'd like to request support for this in wgetpaste instead of the concatenating approach.

I am not familiar with the other pasting engines. Do they also have some kind of multiple file support?

Recent commits such as #26 contain important fixes on top of version 2.30. The Gentoo package maintainers don't wish to update the package by applying commits as patches and would rather see a new release (Bug 740762 Bug 788370).

Is the current master in a state where a new release could be made so that the updates will propagate into distros's repos?

As referenced on https://bugs.gentoo.org/674750

If i use wgetpaste -l bash on https://pastebin.com/raw/vNGc9vzw it generates https://codeload.github.com/PhoenicisOrg/phoenicis/tar.gz/v9999

https://i.imgur.com/CTIfBhX.png

Video: https://youtu.be/uV4A4vd-ZxQ

Reproducable: Always

Gitlab snippets have a documented API.

The API is similar to that of Gists so it should be relatively straightforward to implement it.