earwig / copyvios Goto Github PK

• Better presentation of URLs.
• Integrate results with checker; should just be done without fussing. Keep things simpler and avoids false positives like this one.

Traceback (most recent call last):
File "/data/project/copyvios/www/python/src/app.py", line 38, in inner
return func(*args, **kwargs)
File "/data/project/copyvios/www/python/src/app.py", line 103, in index
query = do_check()
File "./copyvios/checker.py", line 44, in do_check
_get_results(query, follow=not _coerce_bool(query.noredirect))
File "./copyvios/checker.py", line 58, in _get_results
page.get() # Make sure that the page exists before we check it!
File "/data/project/copyvios/git/earwigbot/earwigbot/wiki/page.py", line 587, in get
rvprop="content|timestamp", rvslots="main")
File "/data/project/copyvios/git/earwigbot/earwigbot/wiki/site.py", line 708, in api_query
return self._api_query(kwargs)
File "/data/project/copyvios/git/earwigbot/earwigbot/wiki/site.py", line 246, in _api_query
raise exceptions.APIError("API query failed: {0}".format(exc))
APIError: API query failed: 503 Server Error: Service Unavailable for url: https://en.wikipedia.org/w/api.php

Hi,

I repeatedly got an error on trying to use the Copyvio Detector on https://de.wikipedia.org/wiki/Benutzer:Munfarid1/Dinosaurier_von_Tendaguru. The following error message was shown:

Traceback (most recent call last):
  File "/data/project/copyvios/www/python/src/app.py", line 38, in inner
    return func(*args, **kwargs)
  File "/data/project/copyvios/www/python/src/app.py", line 103, in index
    query = do_check()
  File "./copyvios/checker.py", line 41, in do_check
    _get_results(query, follow=not _coerce_bool(query.noredirect))
  File "./copyvios/checker.py", line 80, in _get_results
    _perform_check(query, page, use_engine, use_links)
  File "./copyvios/checker.py", line 136, in _perform_check
    short_circuit=not query.noskip)
  File "/data/project/copyvios/git/earwigbot/earwigbot/wiki/copyvios/__init__.py", line 154, in copyvio_check
    workspace.enqueue(searcher.search(chunk), exclude)
  File "/data/project/copyvios/git/earwigbot/earwigbot/wiki/copyvios/search.py", line 157, in search
    result = self._open(url + urlencode(params))
  File "/data/project/copyvios/git/earwigbot/earwigbot/wiki/copyvios/search.py", line 61, in _open
    response = self.opener.open(*args)
  File "/usr/lib/python2.7/urllib2.py", line 431, in open
    response = self._open(req, data)
  File "/usr/lib/python2.7/urllib2.py", line 449, in _open
    '_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 1240, in https_open
    context=self._context)
  File "/usr/lib/python2.7/urllib2.py", line 1200, in do_open
    r = h.getresponse(buffering=True)
  File "/usr/lib/python2.7/httplib.py", line 1111, in getresponse
    response.begin()
  File "/usr/lib/python2.7/httplib.py", line 444, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python2.7/httplib.py", line 408, in _read_status
    raise BadStatusLine(line)
BadStatusLine: ''

Thx,
X

Certain templates ({{Free-content attribution}} on enwiki) are a good indication that something may not be a violation. Detect this and display a warning so people are aware.

Traceback (most recent call last):
File "/data/project/copyvios/env/local/lib/python2.7/site-packages/flask_mako.py", line 210, in _render
rv = template.render(**context)
File "/data/project/copyvios/env/local/lib/python2.7/site-packages/mako/template.py", line 443, in render
return runtime.render(self, self.callable, args, data)
File "/data/project/copyvios/env/local/lib/python2.7/site-packages/mako/runtime.py", line 783, in _render
**kwargs_for_callable(callable, data))
File "/data/project/copyvios/env/local/lib/python2.7/site-packages/mako/runtime.py", line 815, in _render_context
_exec_template(inherit, lclcontext, args=args, kwargs=kwargs)
File "/data/project/copyvios/env/local/lib/python2.7/site-packages/mako/runtime.py", line 841, in exec_template
callable(context, _args, *_kwargs)
File "/data/project/copyvios/git/copyvios/templates/index.mako", line 156, in render_body
<a href="${request.script_root | h}?lang=${query.lang | h}&project=${query.project | h}&oldid=${query.page.lastrevid}&action=${query.action}&${("use_engine=" + int(query.use_engine not in ("0", "false")) + "&use_links=" + int(query.use_links not in ("0", "false"))) if query.action == "search" else ("url=" + query.url) | h}">Permalink.
TypeError: cannot concatenate 'str' and 'int' objects

An error people have gotten:

Error !
IndexError: list index out of range
</ul> <table id="cv-chain-table"> <tr> <td class="cv-chain-cell">Article: <div class="cv-chain-detail"><p>${highlight_delta(result.article_chain, result.delta_chain)}</p></div></td> <td class="cv-chain-cell">Source: <div class="cv-chain-detail"><p>${highlight_delta(result.source_chain, result.delta_chain)}</p></div></td> </tr> </table> </div> </div>
./toolserver/copyvios/highlighter.py, line 29:
if highlights[i]:
pages/copyvios.mako, line 135:
<td class="cv-chain-cell">Article: <div class="cv-chain-detail"> <p>${highlight_delta(result.article_chain, result.delta_chain)}</p></div></td>
/home/earwig/.local/solaris/lib/python2.7/site-packages/Mako-0.7.2-py2.7.egg/mako/runtime.py, line 817:
callable_(context, *args, **kwargs)

I want to use this tool in my own wiki how do I set it up in my hosting and I want to create a tool like this for my wiki using a subdomain but how do I set it up

Hi, I am from the Odia-language Wikipedia and I was checking your amazing tool after a colleague mentioned about it. It's a tool many of us in the Wikipedia community might not know and it can come very handy. And I think, to make it more efficient, it would probably be a good idea to blacklist some of the sites that duplicate Wikipedia content as they are working against the whole purpose. An example here is "Wikipedia Top 100" list on Hatnote (e.g. see English list here) which lists the top 100 articles on a Wikipedia. It's ironic if an article is read more and makes it to this list and that duplication of content is flagged in the Copyvio Detector, isn't it. But it is not an issue of blacklisting just one site. I think a good idea would be to maintain, if that is possible given your time and other commitments, a list of all such sites that the tool should not search in. Thanks much again for bringing out this tool.

Would be useful when investigating extensive violations by a user where we have a large number of diffs to check. Link

Traceback (most recent call last):
File "/data/project/copyvios/git/copyvios/app.fcgi", line 36, in inner
return func(_args, *_kwargs)
File "/data/project/copyvios/git/copyvios/app.fcgi", line 82, in index
query = do_check()
File "/data/project/copyvios/git/copyvios/copyvios/checker.py", line 37, in do_check
_get_results(query, follow=not _coerce_bool(query.noredirect))
File "/data/project/copyvios/git/copyvios/copyvios/checker.py", line 84, in _get_results
_cache_result(page, query.result, conn, mode)
File "/data/project/copyvios/git/copyvios/copyvios/checker.py", line 193, in _cache_result
cursor.executemany(query3, data)
File "cursor.pyx", line 138, in oursql.Cursor.executemany (oursqlx/oursql.c:17073)
File "statement.pyx", line 409, in oursql._Statement.execute (oursqlx/oursql.c:10994)
File "util.pyx", line 91, in oursql._do_warnings_query (oursqlx/oursql.c:3969)
CollatedWarningsError: (None, 'query caused warnings', [(<class 'oursql.Warning'>, (u"Data truncated for column 'cdata_url' at row 1", 1265L))]

Goodbye TS.

(I should probably rename this repo...)

This can be used as a fallback when Google's quota gets exceeded. Suggested here.

The tool is not really popular enough for this to be necessary right now, but it is still a good security measure if someone spams the tool. DoS protection exists built-in to the checker itself, but there's nothing to stop a hundred people from checking a hundred different articles at once (even though this is probably a bad idea).

When using the tool to find possible blurbs of CV content in articles that are possibly salvageable without those blurbs, it would be nice to have a way to get a permalink to to the report based on the current revision of the article from inside the tool. Perhaps the best way to do this is to add a "link" link next the "compare" link for each source that will give the URL to replicate the exact results. Such links would be useful to be able to post in edit summaries when removing such content.

In this request
I'v got an error:
Traceback (most recent call last): File "/data/project/copyvios/www/python/src/app.py", line 38, in inner return func(*args, **kwargs) File "/data/project/copyvios/www/python/src/app.py", line 104, in index query = do_check() File "./copyvios/checker.py", line 38, in do_check _get_results(query, follow=not _coerce_bool(query.noredirect)) File "./copyvios/checker.py", line 74, in _get_results query.turnitin_result = search_turnitin(query.title, query.lang) File "./copyvios/turnitin.py", line 22, in search_turnitin return TurnitinResult(_make_api_request(page_title, lang)) File "./copyvios/turnitin.py", line 35, in _make_api_request parsed_api_result = literal_eval(result.text) File "/usr/lib/python2.7/ast.py", line 49, in literal_eval node_or_string = parse(node_or_string, mode='eval') File "/usr/lib/python2.7/ast.py", line 37, in parse return compile(source, filename, mode, PyCF_ONLY_AST) File "<unknown>", line 1 <body bgcolor="#f0f0f8"> ^ SyntaxError: invalid syntax

Traceback (most recent call last):
File "/data/project/copyvios/www/python/src/app.py", line 38, in inner
return func(*args, **kwargs)
File "/data/project/copyvios/www/python/src/app.py", line 103, in index
query = do_check()
File "./copyvios/checker.py", line 44, in do_check
_get_results(query, follow=not _coerce_bool(query.noredirect))
File "./copyvios/checker.py", line 58, in _get_results
page.get() # Make sure that the page exists before we check it!
File "/data/project/copyvios/git/earwigbot/earwigbot/wiki/page.py", line 587, in get
rvprop="content|timestamp", rvslots="main")
File "/data/project/copyvios/git/earwigbot/earwigbot/wiki/site.py", line 708, in api_query
return self._api_query(kwargs)
File "/data/project/copyvios/git/earwigbot/earwigbot/wiki/site.py", line 246, in _api_query
raise exceptions.APIError("API query failed: {0}".format(exc))
APIError: API query failed: ('Connection aborted.', error(104, 'Connection reset by peer'))

File checked https://en.wikipedia.org/w/index.php?title=Draft:Eimear_McGeown&oldid=1060342456
url checked against https://www.eimearmcgeown.com

Duplicate of #10? Whoops.

Traceback (most recent call last):
  File "/data/project/copyvios/env/local/lib/python2.7/site-packages/flask_mako.py", line 210, in _render
    rv = template.render(**context)
  File "/data/project/copyvios/env/local/lib/python2.7/site-packages/mako/template.py", line 443, in render
    return runtime._render(self, self.callable_, args, data)
  File "/data/project/copyvios/env/local/lib/python2.7/site-packages/mako/runtime.py", line 783, in _render
    **_kwargs_for_callable(callable_, data))
  File "/data/project/copyvios/env/local/lib/python2.7/site-packages/mako/runtime.py", line 815, in _render_context
    _exec_template(inherit, lclcontext, args=args, kwargs=kwargs)
  File "/data/project/copyvios/env/local/lib/python2.7/site-packages/mako/runtime.py", line 841, in _exec_template
    callable_(context, *args, **kwargs)
  File "/data/project/copyvios/git/copyvios/templates/index.mako", line 32, in render_body
    <form action="${environ['REQUEST_URI']}" method="get">
TypeError: 'Undefined' object has no attribute '__getitem__'

When visiting https://tools.wmflabs.org/copyvios/ after being redirected from the Toolserver.

Try to access this.

Error message was:

Traceback (most recent call last):
  File "/data/project/copyvios/www/python/src/app.py", line 38, in inner
    return func(*args, **kwargs)
  File "/data/project/copyvios/www/python/src/app.py", line 103, in index
    query = do_check()
  File "./copyvios/checker.py", line 38, in do_check
    _get_results(query, follow=not _coerce_bool(query.noredirect))
  File "./copyvios/checker.py", line 86, in _get_results
    short_circuit=not query.noskip)
  File "/data/project/copyvios/git/earwigbot/earwigbot/wiki/copyvios/__init__.py", line 127, in copyvio_check
    self._exclusions_db.sync(self.site.name)
  File "/data/project/copyvios/git/earwigbot/earwigbot/wiki/copyvios/exclusions.py", line 164, in sync
    self._update(sitename)
  File "/data/project/copyvios/git/earwigbot/earwigbot/wiki/copyvios/exclusions.py", line 138, in _update
    conn.execute(query7, (sitename, int(time())))
OperationalError: database is locked

Traceback (most recent call last):
File "/data/project/copyvios/www/python/src/app.py", line 38, in inner
return func(*args, **kwargs)
File "/data/project/copyvios/www/python/src/app.py", line 103, in index
query = do_check()
File "./copyvios/checker.py", line 41, in do_check
_get_results(query, follow=not _coerce_bool(query.noredirect))
File "./copyvios/checker.py", line 80, in _get_results
_perform_check(query, page, use_engine, use_links)
File "./copyvios/checker.py", line 123, in _perform_check
conn = get_db()
File "./copyvios/misc.py", line 63, in get_db
g._db = _connect_to_db(engine, args)
File "./copyvios/misc.py", line 51, in _connect_to_db
return oursql.connect(**args)
File "/data/project/copyvios/www/python/venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 1546, in connect
return self.get_pool(*args, **kw).connect()
File "/data/project/copyvios/www/python/venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 414, in connect
return _ConnectionFairy._checkout(self)
File "/data/project/copyvios/www/python/venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 811, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/data/project/copyvios/www/python/venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 548, in checkout
rec._checkin_failed(err)
File "/data/project/copyvios/www/python/venv/local/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 67, in exit
compat.reraise(exc_type, exc_value, exc_tb)
File "/data/project/copyvios/www/python/venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 545, in checkout
dbapi_connection = rec.get_connection()
File "/data/project/copyvios/www/python/venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 644, in get_connection
self.__connect()
File "/data/project/copyvios/www/python/venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 690, in __connect
connection = pool._invoke_creator(self)
File "/data/project/copyvios/www/python/venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 320, in
return lambda crec: creator()
File "/data/project/copyvios/www/python/venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 1523, in
lambda: self.module.connect(*args, **kw), **self.kw
File "oursqlx/connection.pyx", line 150, in oursql.Connection.cinit (oursqlx/oursql.c:6455)
File "oursqlx/connection.pyx", line 183, in oursql.Connection._raise_error (oursqlx/oursql.c:6984)
InterfaceError: (2003, "Can't connect to MySQL server on 'tools.labsdb' (113)", None)

Hello!

I've tried your tool for one article in ru.wikipedia ("Фишинг") and got the following error:

AttributeError: 'NoneType' object has no attribute 'find_all'

<%include file="/support/header.mako" args="environ=environ, cookies=cookies, title='Earwig's Copyvio Detector'"/>\ <%namespace module="copyvios" import="main, highlight_delta"/>\ <%namespace module="copyvios.misc" import="urlstrip"/>\ <% query, bot, all_langs, all_projects, page, result = main(environ) %>\ % if query.project and query.lang and query.title and not page:

https://www.uploady.com/#!/download/rRSTCPgwFN~/JcR~luRiN7H7YpUq on https://tools.wmflabs.org/copyvios/?lang=en&project=wikipedia&title=User%3ASijupm%2Fsandbox

Waiting the 12h period (or however long) is not ideal and not necessary when edit feeds exist.

• An optional box at the bottom of the input form, with one URL per line.
• A link after a confirmed violation check to re-run it with that URL added to the exclusion list.
• Same syntax as the exclusion page on-wiki (perhaps improve that syntax too; better wildcard support?)

Hi,
I've got following error. Reported as advised.

Traceback (most recent call last):
File "/data/project/copyvios/www/python/src/app.py", line 38, in inner
return func(*args, **kwargs)
File "/data/project/copyvios/www/python/src/app.py", line 104, in index
query = do_check()
File "./copyvios/checker.py", line 38, in do_check
_get_results(query, follow=not _coerce_bool(query.noredirect))
File "./copyvios/checker.py", line 86, in _get_results
short_circuit=not query.noskip)
File "/data/project/copyvios/git/earwigbot/earwigbot/wiki/copyvios/init.py", line 127, in copyvio_check
self._exclusions_db.sync(self.site.name)
File "/data/project/copyvios/git/earwigbot/earwigbot/wiki/copyvios/exclusions.py", line 164, in sync
self._update(sitename)
File "/data/project/copyvios/git/earwigbot/earwigbot/wiki/copyvios/exclusions.py", line 138, in _update
conn.execute(query7, (sitename, int(time())))
OperationalError: database is locked

Best,
Martin Urbanec@cswiki

For: http://tools.wmflabs.org/copyvios/api.json?action=search&project=wikipedia&lang=en&oldid=548209101
With code:

var CopyViosAPIQ = {
    action: 'search',
    project: mw.config.get('wgNoticeProject'),
    lang: mw.config.get('wgPageContentLanguage'),
    oldid: 548209101
};
$.get("//tools.wmflabs.org/copyvios/api.json", CopyViosAPIQ, function(CopyViosAPIR){
    console.log("%O", CopyViosAPIR);
});

I get:
Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at http://tools.wmflabs.org/copyvios/api.json?action=search&project=wikipedia&lang=en&oldid=548209101. This can be fixed by moving the resource to the same domain or enabling CORS.

I can't run the copyvios-tool on https://tools.wmflabs.org/copyvios/?lang=en&project=wikiversity&title=Decolonising+Science%2FReading+List&oldid=&use_engine=0&use_links=0&turnitin=0&action=compare&url=https%3A%2F%2Fmedium.com%2F%40chanda%2Fdecolonising-science-reading-list-339fb773d51f

It says "The given page doesn't seem to exist: Decolonising Science/Reading List." and links to beta.wikiversity.org, even though I've selected en.wikiversity

Websites must do so in order to operate in the EU

Running again didn't give the error, so reporting so that it's logged:

Traceback (most recent call last):
File "/data/project/copyvios/www/python/src/app.py", line 38, in inner
return func(*args, **kwargs)
File "/data/project/copyvios/www/python/src/app.py", line 103, in index
query = do_check()
File "./copyvios/checker.py", line 41, in do_check
_get_results(query, follow=not _coerce_bool(query.noredirect))
File "./copyvios/checker.py", line 46, in _get_results
page = query.page = _get_page_by_revid(query.site, query.oldid)
File "./copyvios/checker.py", line 104, in _get_page_by_revid
rvslots="main")
File "/data/project/copyvios/git/earwigbot/earwigbot/wiki/site.py", line 707, in api_query
return self._api_query(kwargs)
File "/data/project/copyvios/git/earwigbot/earwigbot/wiki/site.py", line 247, in _api_query
return self._handle_api_result(response, params, tries, wait, ae_retry)
File "/data/project/copyvios/git/earwigbot/earwigbot/wiki/site.py", line 338, in _handle_api_result
raise error
APIError: API query failed: got error "badinteger"; server says: "Invalid value "Akram Sheikh" for integer parameter "revids".".

Should be done by passing a general 'engines' parameter that is comma- or pipe-separated.

This query param change would also be nice to have in the regular tool itself, but because of the way HTML forms work, I'm not sure if we could get it to work without requiring JS.

Decide for real whether we want to use the dynamic image or the static tile.

Crashed with this traceback:

Traceback (most recent call last):
File "/data/project/copyvios/www/python/src/app.py", line 38, in inner
return func(*args, **kwargs)
File "/data/project/copyvios/www/python/src/app.py", line 103, in index
query = do_check()
File "./copyvios/checker.py", line 38, in do_check
_get_results(query, follow=not _coerce_bool(query.noredirect))
File "./copyvios/checker.py", line 80, in _get_results
page, conn, mode, _coerce_bool(query.noskip))
File "./copyvios/checker.py", line 144, in _get_cached_results
cursor.execute(query1)
File "oursqlx/cursor.pyx", line 122, in oursql.Cursor.execute (oursqlx/oursql.c:20330)
File "oursqlx/statement.pyx", line 402, in oursql._Statement.execute (oursqlx/oursql.c:13119)
File "oursqlx/statement.pyx", line 127, in oursql._Statement._raise_error (oursqlx/oursql.c:9947)
ProgrammingError: (1205, 'Lock wait timeout exceeded; try restarting transaction', None)

Got this error message:

Traceback (most recent call last):
File "/data/project/copyvios/www/python/src/app.py", line 38, in inner
return func(*args, **kwargs)
File "/data/project/copyvios/www/python/src/app.py", line 103, in index
query = do_check()
File "./copyvios/checker.py", line 41, in do_check
_get_results(query, follow=not _coerce_bool(query.noredirect))
File "./copyvios/checker.py", line 52, in _get_results
page.get() # Make sure that the page exists before we check it!
File "/data/project/copyvios/git/earwigbot/earwigbot/wiki/page.py", line 587, in get
rvprop="content|timestamp", rvslots="main")
File "/data/project/copyvios/git/earwigbot/earwigbot/wiki/site.py", line 716, in api_query
return self._api_query(kwargs)
File "/data/project/copyvios/git/earwigbot/earwigbot/wiki/site.py", line 254, in _api_query
return self._handle_api_result(response, params, tries, wait, ae_retry)
File "/data/project/copyvios/git/earwigbot/earwigbot/wiki/site.py", line 295, in _handle_api_result
raise exceptions.APIError(e)
APIError: API query failed: JSON could not be decoded.
Copyright © 2009–2019 Ben Kurtovic • API • Source Code • Background • Valid HTML5

Could be useful if an API was used to check a large number of articles en-masse.

http://stackoverflow.com/q/1848464/2712951 has some pointers on libraries for this.

This will occur for percentages below 50% but greater than, say, 30%.

This is instead of just showing the best match. Perhaps with AJAX and an API, clicking on each URL could update the comparison table. Also, an option to re-run the search at its full length without short-circuiting for a URL that exceeds the confidence threshold is an extension of this.

Got an API error when loading https://tools.wmflabs.org/copyvios/?lang=en&project=wikipedia&title=User:Lernglobaluk/sandbox&url=. The link was generated by {{AFC submission}}.

Traceback (most recent call last):
File "/data/project/copyvios/www/python/src/app.py", line 38, in inner
return func(*args, **kwargs)
File "/data/project/copyvios/www/python/src/app.py", line 103, in index
query = do_check()
File "./copyvios/checker.py", line 41, in do_check
_get_results(query, follow=not _coerce_bool(query.noredirect))
File "./copyvios/checker.py", line 52, in _get_results
page.get() # Make sure that the page exists before we check it!
File "/data/project/copyvios/git/earwigbot/earwigbot/wiki/page.py", line 587, in get
rvprop="content|timestamp", rvslots="main")
File "/data/project/copyvios/git/earwigbot/earwigbot/wiki/site.py", line 716, in api_query
return self._api_query(kwargs)
File "/data/project/copyvios/git/earwigbot/earwigbot/wiki/site.py", line 254, in _api_query
return self._handle_api_result(response, params, tries, wait, ae_retry)
File "/data/project/copyvios/git/earwigbot/earwigbot/wiki/site.py", line 295, in _handle_api_result
raise exceptions.APIError(e)
APIError: API query failed: JSON could not be decoded.

Using https://tools.wmflabs.org/copyvios/ worked normally.

http://toolserver.org/~earwig/earwigbot

How to add translations into Russian?

If I insert an article title with _ copyvios raises this error:

Traceback (most recent call last):
  File "/data/project/copyvios/www/python/src/app.py", line 38, in inner
    return func(*args, **kwargs)
  File "/data/project/copyvios/www/python/src/app.py", line 103, in index
    query = do_check()
  File "./copyvios/checker.py", line 41, in do_check
    _get_results(query, follow=not _coerce_bool(query.noredirect))
  File "./copyvios/checker.py", line 52, in _get_results
    page.get()  # Make sure that the page exists before we check it!
  File "/data/project/copyvios/git/earwigbot/earwigbot/wiki/page.py", line 587, in get
    rvprop="content|timestamp", rvslots="main")
  File "/data/project/copyvios/git/earwigbot/earwigbot/wiki/site.py", line 716, in api_query
    return self._api_query(kwargs)
  File "/data/project/copyvios/git/earwigbot/earwigbot/wiki/site.py", line 254, in _api_query
    return self._handle_api_result(response, params, tries, wait, ae_retry)
  File "/data/project/copyvios/git/earwigbot/earwigbot/wiki/site.py", line 295, in _handle_api_result
    raise exceptions.APIError(e)
APIError: API query failed: JSON could not be decoded.

When running http://tools.wmflabs.org/copyvios?lang=en&project=wikipedia&title=OK_Bazaars (before that article got deleted)

Traceback (most recent call last):
File "/data/project/copyvios/git/copyvios/app.fcgi", line 34, in inner
return func(_args, *_kwargs)
File "/data/project/copyvios/git/copyvios/app.fcgi", line 71, in index
def index():
File "/data/project/copyvios/git/copyvios/copyvios/checker.py", line 30, in do_check
if query.site:
File "/data/project/copyvios/git/copyvios/copyvios/checker.py", line 67, in _get_results
query.result = page.copyvio_check(
File "/data/project/copyvios/git/earwigbot/earwigbot/wiki/copyvios/init.py", line 181, in copyvio_check
max_time = self._workspace.until - time()
File "/data/project/copyvios/git/earwigbot/earwigbot/wiki/copyvios/search.py", line 87, in search
response = self.opener.open(req.to_url())
File "/usr/lib/python2.7/urllib2.py", line 406, in open
response = meth(req, response)
File "/usr/lib/python2.7/urllib2.py", line 519, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python2.7/urllib2.py", line 444, in error
return self._call_chain(_args)
File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain
result = func(_args)
File "/usr/lib/python2.7/urllib2.py", line 527, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 400: Bad Request

When attempting searches, the page crashes with error code InterfaceError: (2003, "Can't connect to MySQL server on 'tools.labsdb' (111)", None). This appears to occur regardless of which page is being searched, and whether or not "use search engine" and "use links in page" are selected.

This error occurred on Firefox 83.0, and also appears to be occurring within the NPP browser tool

Full error message:

Traceback (most recent call last):
  File "/data/project/copyvios/www/python/src/app.py", line 38, in inner
    return func(*args, **kwargs)
  File "/data/project/copyvios/www/python/src/app.py", line 103, in index
    query = do_check()
  File "./copyvios/checker.py", line 41, in do_check
    _get_results(query, follow=not _coerce_bool(query.noredirect))
  File "./copyvios/checker.py", line 80, in _get_results
    _perform_check(query, page, use_engine, use_links)
  File "./copyvios/checker.py", line 123, in _perform_check
    conn = get_db()
  File "./copyvios/misc.py", line 63, in get_db
    g._db = _connect_to_db(engine, args)
  File "./copyvios/misc.py", line 51, in _connect_to_db
    return oursql.connect(**args)
  File "/data/project/copyvios/www/python/venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 1546, in connect
    return self.get_pool(*args, **kw).connect()
  File "/data/project/copyvios/www/python/venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 414, in connect
    return _ConnectionFairy._checkout(self)
  File "/data/project/copyvios/www/python/venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 811, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/data/project/copyvios/www/python/venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 543, in checkout
    rec = pool._do_get()
  File "/data/project/copyvios/www/python/venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 1239, in _do_get
    self._dec_overflow()
  File "/data/project/copyvios/www/python/venv/local/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 67, in __exit__
    compat.reraise(exc_type, exc_value, exc_tb)
  File "/data/project/copyvios/www/python/venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 1236, in _do_get
    return self._create_connection()
  File "/data/project/copyvios/www/python/venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 361, in _create_connection
    return _ConnectionRecord(self)
  File "/data/project/copyvios/www/python/venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 488, in __init__
    self.__connect(first_connect_check=True)
  File "/data/project/copyvios/www/python/venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 690, in __connect
    connection = pool._invoke_creator(self)
  File "/data/project/copyvios/www/python/venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 320, in <lambda>
    return lambda crec: creator()
  File "/data/project/copyvios/www/python/venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 1523, in <lambda>
    lambda: self.module.connect(*args, **kw), **self.kw
  File "oursqlx/connection.pyx", line 150, in oursql.Connection.__cinit__ (oursqlx/oursql.c:6455)
  File "oursqlx/connection.pyx", line 183, in oursql.Connection._raise_error (oursqlx/oursql.c:6984)
InterfaceError: (2003, "Can't connect to MySQL server on 'tools.labsdb' (111)", None)