Giter VIP home page Giter VIP logo

moin's People

Contributors

abirke avatar ajgupta93 avatar akashsinha avatar ana-balica avatar ashu1461 avatar bgola avatar brunosmartin avatar bylsmad avatar dragooon avatar fabiomontefuscolo avatar inducer avatar jdmathur avatar kwatsen avatar lfagundes avatar maikumori avatar moloney avatar pkumar7 avatar psviderski avatar reimarbauer avatar rogerhaase avatar ronnypfannschmidt avatar sblondon avatar sharky93 avatar thomaswaldmann avatar ulrichb22 avatar vipul-sharma20 avatar wagner-intevation avatar walditm avatar xiaq avatar yask123 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

moin's Issues

jslint our js stuff

Original report by Reimar Bauer (Bitbucket: ReimarBauer, GitHub: ReimarBauer).


We should fix findings by http://www.jslint.com/ of our code.

The PEP8 fix should be separated from the others.

At some places we use == while we better use ===, same for != and !==.
Some functions may better use 'use strict' statement.
Some needs a var tag e.g. 'i' was used before it was defined.

fs backend may run out of files

Original report by Thomas Waldmann (Bitbucket: thomaswaldmann, GitHub: thomaswaldmann).


when running moin2 with pypy 1.5 and the fs backend, the process runs out of file handles rather quickly. See XXX comments in fs.py.

this is because fs backend keeps files open and pypy doesn't seem to garbage collect them as quickly as cpython does.

for pypy, we need to close() files and not rely on garbage collection.

just download

Original report by Reimar Bauer (Bitbucket: ReimarBauer, GitHub: ReimarBauer).


download should open the file dialog and not use the browser to show the content if it can be rendered by its engine. By this we have to use the browsers store method,

The browser is able to split item.data into different content types by using its own save method. Firefox for example splits svg from images if they were embedded. Afterwards you have a copy locally but you are not able to upload this easily into a different wiki.

This happens for example for svg if you use the svg to overlay/annotate something on an image. The svg item has on moin then a copy of the image in its item.data. If you currently download this item by firefox then you have a short svg file plus a folder with the png file.

We should keep the dependency and store it directly.

too much revisions

Original report by Reimar Bauer (Bitbucket: ReimarBauer, GitHub: ReimarBauer).


Currently every OK on an item in modify mode saves a new revision.

May be that can be solved by adding a checksum hash of the whole item data and meta to the querystring and compare it with the sended item.

Exception loading xml items file

Original report by Bruno Martin (Bitbucket: brunomartin, GitHub: brunomartin).

The original report had attachments: serilization.py


when loading xml file, if you have metadata like meta_data =

[["abc"],["asdf"," fsa"]] 

(list of lists), it raises:

#!python

Traceback (most recent call last):
  File "/home/bruno/devel/moin-head/moin-2.0/moin-2.0-env/bin/moin", line 9, in <module>
    load_entry_point('moin==2.0.0a0', 'console_scripts', 'moin')()
  File "/home/bruno/devel/moin-head/moin-2.0/MoinMoin/script/__init__.py", line 40, in main
    return manager.run(default_command=default_command)
  File "/home/bruno/devel/moin-head/moin-2.0/moin-2.0-env/lib/python2.6/site-packages/flaskext/script.py", line 684, in run
    sys.argv[2:])
  File "/home/bruno/devel/moin-head/moin-2.0/moin-2.0-env/lib/python2.6/site-packages/flaskext/script.py", line 653, in handle
    app = self.create_app(**app_namespace.__dict__)
  File "/home/bruno/devel/moin-head/moin-2.0/moin-2.0-env/lib/python2.6/site-packages/flaskext/script.py", line 432, in create_app
    return self.app(**kwargs)
  File "/home/bruno/devel/moin-head/moin-2.0/MoinMoin/app.py", line 47, in create_app
    return create_app_ext(flask_config_file=config)
  File "/home/bruno/devel/moin-head/moin-2.0/MoinMoin/app.py", line 148, in create_app_ext
    import_export_xml(app)
  File "/home/bruno/devel/moin-head/moin-2.0/MoinMoin/app.py", line 208, in import_export_xml
    unserialize(tmp_backend, xmlfile)
  File "/home/bruno/devel/moin-head/moin-2.0/MoinMoin/storage/serialization.py", line 209, in unserialize
    obj.unserialize(context)
  File "/home/bruno/devel/moin-head/moin-2.0/MoinMoin/storage/serialization.py", line 308, in unserialize
    xml_parse(context.xmlfile, MoinContentHandler(self, context))
  File "/usr/lib/python2.6/xml/sax/__init__.py", line 33, in parse
    parser.parse(source)
  File "/usr/lib/python2.6/xml/sax/expatreader.py", line 107, in parse
    xmlreader.IncrementalParser.parse(self, source)
  File "/usr/lib/python2.6/xml/sax/xmlreader.py", line 123, in parse
    self.feed(buffer)
  File "/usr/lib/python2.6/xml/sax/expatreader.py", line 207, in feed
    self._parser.Parse(data, isFinal)
  File "/usr/lib/python2.6/xml/sax/expatreader.py", line 304, in end_element
    self._cont_handler.endElement(name)
  File "/home/bruno/devel/moin-head/moin-2.0/MoinMoin/storage/serialization.py", line 59, in endElement
    self.unserialize('endElement', name)
  File "/home/bruno/devel/moin-head/moin-2.0/MoinMoin/storage/serialization.py", line 51, in unserialize
    self.unserializer.send(args)
  File "/home/bruno/devel/moin-head/moin-2.0/MoinMoin/storage/serialization.py", line 290, in _unserialize
    d = yield unserializer.send(d)
  File "/home/bruno/devel/moin-head/moin-2.0/MoinMoin/storage/serialization.py", line 290, in _unserialize
    d = yield unserializer.send(d)
  File "/home/bruno/devel/moin-head/moin-2.0/MoinMoin/storage/serialization.py", line 290, in _unserialize
    d = yield unserializer.send(d)
  File "/home/bruno/devel/moin-head/moin-2.0/MoinMoin/storage/serialization.py", line 290, in _unserialize
    d = yield unserializer.send(d)
  File "/home/bruno/devel/moin-head/moin-2.0/MoinMoin/storage/serialization.py", line 299, in _unserialize
    self.endElement()
  File "/home/bruno/devel/moin-head/moin-2.0/MoinMoin/storage/serialization.py", line 455, in endElement
    self._result_fn(value)
  File "/home/bruno/devel/moin-head/moin-2.0/MoinMoin/storage/serialization.py", line 517, in result_fn
    assert len(value) == 1 # entry is like a 1-tuple
AssertionError

Attached the diff file of MoinMoin/storage/serialization.py

#!python

517c517
<         assert len(value) == 1 # entry is like a 1-tuple
---
>         
519c519,529
<         self.target[key] = value[0]
---
>         
>         if len(value) > 1:
>             list_aux = []
>             for elem in value:
>                 list_aux.append(elem)
>             self.target[key] = list_aux
>         else:
>             self.target[key] = value[0]
> #        assert len(value) == 1 # entry is like a 1-tuple
>         
>         

remove mathjax

Original report by Thomas Waldmann (Bitbucket: thomaswaldmann, GitHub: thomaswaldmann).


mathjax should get removed from the moin2 standard install because of its vast size, time needed to download/install/upgrade it and extra overhead it causes:

$ time pip install --upgrade xstatic-mathjax
Downloading/unpacking xstatic-mathjax
  Downloading XStatic-MathJax-1.1.2.tar.gz (6.5Mb): 6.5Mb downloaded
  Running setup.py egg_info for package xstatic-mathjax
...
Installing collected packages: xstatic-mathjax
  Found existing installation: XStatic-MathJax 1.1.2
    Uninstalling XStatic-MathJax:
    
      Successfully uninstalled XStatic-MathJax
  Running setup.py install for xstatic-mathjax
...
    Installing /home/tw/w/moin-2.0/env/lib/python2.7/site-packages/XStatic_MathJax-1.1.2-py2.7-nspkg.pth

Successfully installed xstatic-mathjax
Cleaning up...

real	12m22.409s       <------ !!!
user	10m48.813s
sys	0m42.235s

$ du -sk mathjax 
124304	mathjax

$ find mathjax/ | wc -l
29872

Extra requests (as long as the js is not cached, but even if it is cached, it'll take some time because the js is called on every item view):

2011-08-06 19:14:31,381 INFO werkzeug:116 127.0.0.1 "GET /Home HTTP/1.1" 200 -
2011-08-06 19:14:31,484 INFO werkzeug:116 127.0.0.1 "GET /_themes/modernized/css/common.css HTTP/1.1" 304 -
2011-08-06 19:14:31,490 INFO werkzeug:116 127.0.0.1 "GET /+serve/jquery/jquery.min.js HTTP/1.1" 200 -
2011-08-06 19:14:31,496 INFO werkzeug:116 127.0.0.1 "GET /+serve/svgweb/svg.js HTTP/1.1" 200 -
2011-08-06 19:14:31,502 INFO werkzeug:116 127.0.0.1 "GET /static/js/common.js HTTP/1.1" 304 -
2011-08-06 19:14:31,538 INFO werkzeug:116 127.0.0.1 "GET /+serve/mathjax/MathJax.js HTTP/1.1" 200 -
2011-08-06 19:14:31,548 INFO werkzeug:116 127.0.0.1 "GET /static/logos/moinmoin.png HTTP/1.1" 304 -
2011-08-06 19:14:31,554 INFO werkzeug:116 127.0.0.1 "GET /static/logos/moinmoin_powered.png HTTP/1.1" 304 -
2011-08-06 19:14:31,564 INFO werkzeug:116 127.0.0.1 "GET /static/logos/python_powered.png HTTP/1.1" 304 -
2011-08-06 19:14:33,600 INFO werkzeug:116 127.0.0.1 "GET /_themes/modernized/img/white-clouds.jpg HTTP/1.1" 304 -
2011-08-06 19:14:33,806 INFO werkzeug:116 127.0.0.1 "GET /+serve/mathjax/jax/input/TeX/config.js HTTP/1.1" 200 -
2011-08-06 19:14:33,813 INFO werkzeug:116 127.0.0.1 "GET /+serve/mathjax/jax/output/HTML-CSS/config.js HTTP/1.1" 200 -
2011-08-06 19:14:33,828 INFO werkzeug:116 127.0.0.1 "GET /+serve/mathjax/extensions/tex2jax.js HTTP/1.1" 200 -

whoever wants to use mathjax can just pip install xstatic-mathjax and add the required fragment (that currently is in base.html template) to block head_scripts:

    <script type="text/x-mathjax-config">
    MathJax.Hub.Config({
        extensions: ["tex2jax.js"],
        jax: ["input/TeX","output/HTML-CSS"],
        tex2jax: {inlineMath: [["$","$"],["\\(","\\)"]]}
    });
    </script>
    <script src="{{ url_for('serve.files', name='mathjax', filename='MathJax.js') }}"></script>

For details, see:

http://static.moinmo.in/m2/html/admin/configure.html#adding-scripts

excessive memory usage

Original report by Thomas Waldmann (Bitbucket: thomaswaldmann, GitHub: thomaswaldmann).


i am just doing some tests with some real wiki. index directoy size is about 500MB.

after starting moin (no request yet), memory usage looks like this:

22818 tw        20   0  160m  36m 3924 S    1  0.5   0:01.25 python                                      

as long as one does some harmless requests, not much is happening. but if one requests the local history view of an wiki markup item with ~3000 revisions (revisions of up to ~200KB size), memory usage jumps to 1.4GB:

22818 tw        20   0 1602m 1.4g 4120 S  100 18.4   0:33.47 python                                      

after that request is finished, it drops again a bit (but still too high):

22818 tw        20   0 1122m 998m 4000 S    1 12.5   1:19.29 python                                      

Adding a check-filter to request.values.get() function

Original report by Akash Sinha (Bitbucket: sinha, GitHub: sinha).


In request.values.get('param', 0) function if the param value is None, it returns the default value i.e 0 here. What if we could also apply a check that if param value doesn't lie in a specified range, in that case also it may return the default one.

Something like this request.values.get('param', 0, ValueAtLeast(0))
So in above if the param value is -1, then also function should return the value 0.

use validation to protect against spambots

Original report by Thomas Waldmann (Bitbucket: thomaswaldmann, GitHub: thomaswaldmann).


often spambots just jump on every form field and submit button they can find and fill in what they believe that will result in links to their sites.

often this is just html, like

<a ... href="http://advertised-site.com" ...>

as we do not need (and should not accept) html input for many form fields we can use a validator that detects some generic html patterns and rejects the form submission then.

E.g.:

  • item name input fields
  • action comment fields
  • metadata (json)
  • quick search query input field (maybe - do we need to search for such stuff, do we need to lower the load?)

Exceptions where this (simple) validator should not be used:

  • the revision data edit field of text/html items (of course)
  • other revision data edit fields (html-like stuff could be a valid part of pre/code sections, so this gets too difficult to decide whether it is legitimate or not)

Note: sometimes it is not even needed to protect every input field, as they'll fill their crap in every form field anyway. :D

find a way to create static html dumps

Original report by Thomas Waldmann (Bitbucket: thomaswaldmann, GitHub: thomaswaldmann).


moin 1.9 had "moin dump" to create static html dumps, but moin2 has no such feature yet.

Problem: if one wants to load such html directly from the filesystem into a browser (without using a web server), the URLs in the html need to be relative. also, the filenames need to get some appropriate extension, because there is no web server to give a content-type header.

I tried Frozen-Flask a while ago, but was unable to create relative URLs. So we could use this just for html dumps that are served at some specific URL.

IE9 (and Chrome Incognito) hang the Built-In Web server on Windows 7

Original report by RogerHaase (Bitbucket: RogerHaase, GitHub: RogerHaase).


When using the built-in web server, IE9 may hang the server when accessing any web page. This happens frequently; usually on the first or second page although once I was able to successfully access 6 pages in a row. Once the built-in web server is hung, it is hung for all other browsers. The server must either be restarted or all running copies of IE9 must be stopped.

When Chrome is run in Incognito mode, it hangs on the first request. However, accessing the server with any other browser un-freezes the server and both the other browser and the Incognito window are served successfully.

The issue appears to be similar to reported incidents in other single process web servers:
http://stackoverflow.com/questions/5971081/ie9-loses-requests-to-a-local-catalyst-server
http://blog.plackperl.org/2011/05/ie9-reportedly-hangs-httpserverpsgi-issues-miyagawaplack-github.html

improve validation when saving items

Original report by Thomas Waldmann (Bitbucket: thomaswaldmann, GitHub: thomaswaldmann).


if a new revision gets stored into the storage, we should do some validation steps:

  • for text data, try to decode it to unicode - if it fails, do not accept it
    (use content-type metadata)
  • check meta fields for correctness

some checks are already in place, see storage validation layer, but just giving warnings, not raising exceptions.

heading style improvement

Original report by Thomas Waldmann (Bitbucket: thomaswaldmann, GitHub: thomaswaldmann).


content headings (h1, h2, h3, ...) of moin wiki pages (and likely also headings generated from other markups) look a bit too plain currently, they are just text in a bit bigger font.

esp. for smaller headings, this gets too similar to standard text, but even for bigger headings, this could look more pretty and could be more clearly identified as being a heading.

http://wiki.python.de/ <- maybe underlined like there (full width)?

The ¶ displayed on mouseover seems to have a different baseline as the heading, should be fixed also.

language attributes missing

Original report by Thomas Waldmann (Bitbucket: thomaswaldmann, GitHub: thomaswaldmann).


Warning: The html tag has no language attribute

Found by: http://validator.w3.org/i18n-checker/

We need to think about how to handle this.

Our internal default language is english, but (assuming that we have a complete xx translation) if the user requests xx, he'll get all user interface texts in xx.

Another thing is the content, which does not need to be in xx also, but could be just any other language. So I guess we need another lang attr for the content (comes from revision metadata).

Inside the content, there could be yet other languages, so we might need more lang attrs and a way to deal with that.

Transclusions must also be considered.

use of HTML5 <INPUT> attribute pattern

Original report by Reimar Bauer (Bitbucket: ReimarBauer, GitHub: ReimarBauer).


We should use html5 pattern attribute for browser based form data input

Specifes a pattern or format for the input field's value. Example: pattern="[0-9]" means that the input value must be a number between 0 an 9

By this html5 capable browsers can validate itself.

image converter doesn't care about revision

Original report by Thomas Waldmann (Bitbucket: thomaswaldmann, GitHub: thomaswaldmann).


if you have an image item with multiple (different) revisions and you go to its history view, you can "show" or "download" each revision of it.

problem: "show" always shows the current revision of the image, because it does not include the revno in the img src url.

"download" works correctly and uses the correct url.

other converters should also be checked whether they process the correct revision.

Problem rendering object tag on page

Original report by Bruno Martin (Bitbucket: brunomartin, GitHub: brunomartin).


When there is a image (or object, if html) on the text, nothing is displayed after the image (or object).
Seens that html_out isn't using to close the tag, but instead is using <object ... /> format, what is not acepted by html5
Tested with html and rst contenttype.

Here some example of genereted html:

#!html

<div class="page-object">
	<object data="+download/Patologia/MPT0220/Turma98A/Ac%C3%BAmulos%20e%20Calcifica%C3%A7%C3%B5es/burns1.png" />
		<div class="object-overlay-wrapper">
			<div class="object-overlay">
				<a href="+download/Patologia/MPT0220/Turma98A/Ac%C3%BAmulos%20e%20Calcifica%C3%A7%C3%B5es/burns1.png">→</a>
			</div>
		</div>
	</div>
<p>After image</p>
</div>

logging for tests

Original report by Thomas Waldmann (Bitbucket: thomaswaldmann, GitHub: thomaswaldmann).


currently, the tests use default logging, which generates quite a bit of useless output for the tests (like timing information).

The log output (on stderr) gets only visible if a test fails, but still is annoying.

logging should be configured more adequately for the tests.

py.test run with fs2 shows file leak

Original report by Thomas Waldmann (Bitbucket: thomaswaldmann, GitHub: thomaswaldmann).


If you configure a fs2 backend instead of memory backend (in MoinMoin/conftest.py -> init_test_app) and run py.test from the workdir (so it runs all the tests), it will run out of files:

INTERNALERROR> env/lib/python2.6/site-packages/py/_io/capture.py:227: 
INTERNALERROR> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
INTERNALERROR> 
INTERNALERROR> self = <py._io.capture.FDCapture instance at 0x336c23b0>
INTERNALERROR> 
INTERNALERROR> >   ???
INTERNALERROR> E   OSError: [Errno 24] Too many open files: '/dev/null'

py.test run leaks memory

Original report by Thomas Waldmann (Bitbucket: thomaswaldmann, GitHub: thomaswaldmann).


when running py.test from the toplevel directory, you can see how memory consumption steadily climbs up to about 1.5GB (on a 32bit system, likely more on a 64bit system - system with 2GB or less memory may run into trouble due to excessive paging).

I think the problem is in conftest.py and what it does (does not) in init_test_app() and deinit_test_app(), and also the nesting of that for module, class, method.

It might be also related to router_index ressources maybe not getting freed again.

delete respects acls but gives a traceback

Original report by Reimar Bauer (Bitbucket: ReimarBauer, GitHub: ReimarBauer).


With the following settings anonymous user can remove the rst page or any other.

diff -r [c61d7e7c6c6b (bb)](https://bitbucket.org/thomaswaldmann/moin-2.0/commits/c61d7e7c6c6b) wikiconfig.py
--- a/wikiconfig.py	Fri Aug 05 20:42:28 2011 +0200
+++ b/wikiconfig.py	Fri Aug 12 23:18:35 2011 +0200
@@ -38,7 +38,7 @@
                             backend_uri='fs2:%s/%%(nsname)s' % data_dir,
                             # XXX we use rather relaxed ACLs for the development wiki:
                             content_acl=dict(before=u'',
-                                             default=u'All:read,write,create,destroy,admin',
+                                             default=u'All:read,admin',
                                              after=u'', ),
                             user_profile_acl=dict(before=u'',
                                              default=u'All:read,write,create,destroy,admin',

Is removing write, create, destroy from the acl list what we can do? Should this protect an item for becoming moved to trash?

non-text items lose data when modifying

Original report by Thomas Waldmann (Bitbucket: thomaswaldmann, GitHub: thomaswaldmann).


if you have some non-text item (e.g. an image) and you go to modify view and modify its metadata and save, it will lose its data content.

it should reuse data from previous revision if no data is uploaded.

is there a difference between uploading a zero byte file and not uploading a file?

pre sections

Original report by Anonymous.


We can use

{{


text

Syntax for defining a pre section.

Currently we can also add some attribute also an undifined one do get no error.

e.g.

#!notexist

text

This renders different to a pre section.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.