Giter VIP home page Giter VIP logo

lexi-server's Introduction

Backend for Lexi software

Changelog

Version 0.3.1

  • restructured simplification pipeline: ranker and CWI use common scoring class
  • single-word requests always pass through CWI
  • featurizers still very POC, next step is to implement strong models

Version 0.3

  • no more pickling!
  • POS-based synonym selection

Version 0.2.5

  • more general database error handling

Version 0.2.4

  • bugfix in Database connection: rollback connection at error

Version 0.2.3

  • using synonym list for Danish
  • return simplification objects with unique sessionIds
  • accommodate for on-demand simplifications

Version 0.2.2

  • simplify HTML only between given start and end character offset

Version 0.2.1

  • blacklist words per user

Version 0.2.0

  • massive restructuring of source
  • marking if original word is displayed as first alternative

Version 0.1.2

  • log frontend_version in database

Version 0.1.1

  • small bugfix in database calls

Version 0.1

  • initial tagged release

lexi-server's People

Contributors

jbingel avatar

Stargazers

avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar

Forkers

shiraeisenberg xtreamtrader cocoxu yuluqinn davidheineman

lexi-server's Issues

Encoding problems in database calls

See for instance below error. Something wrong with invalid byte sequences.

2018-12-15 20:53:41,280 - lexi - {database.py:35} INFO - PSQL query: SELECT user_id FROM users WHERE email='[email protected]'
2018-12-15 20:53:41,281 - lexi - {run_lexi_server.py:130} ERROR - invalid byte sequence for encoding "UTF8": 0xe4 0x45 0x4c

2018-12-15 20:53:41,282 - lexi - {run_lexi_server.py:131} ERROR - Traceback (most recent call last):
  File "/var/www/lexi/lexi-venv/lib/python3.5/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/var/www/lexi/lexi-venv/lib/python3.5/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/var/www/lexi/lexi/server/run_lexi_server.py", line 269, in test_connection
    working_fine = db_connection.test_connection()
  File "/var/www/lexi/lexi/server/util/database.py", line 27, in test_connection
    self.execute_and_fetchone(test_query)
  File "/var/www/lexi/lexi/server/util/database.py", line 45, in execute_and_fetchone
    self.execute(query, log)
  File "/var/www/lexi/lexi/server/util/database.py", line 36, in execute
    self.cursor.execute(query)
psycopg2.DatabaseError: invalid byte sequence for encoding "UTF8": 0xe4 0x45 0x4c

Multilingual backend

Use the language ID information provided by frontend to enable simplification in various languages.

Ensure grammaticaiity through morphological analysis

stanfordnlp seems to do a decent job for Danish. We can use it to deal with potentially grammaticality-breaking ambiguities. Idea: analyse original sentence and the newly generated sentence containing the candidate simplification, then check if original word and simplification candidate receive same analysis. Alternative: check if dependency structure is maintained.

Better model serialization

For now, models (rankers etc.) are just pickled, which comes with the obvious and well-known problems. Serialize to some sensible (non-binary?) format if possible.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.