Giter VIP home page Giter VIP logo

nicolas-le / argumentretrieval Goto Github PK

View Code? Open in Web Editor NEW
0.0 2.0 0.0 13.32 MB

This git repository documents the code base used in a custom argument retrieval system. This git repository documents the code base used in a custom argument retrieval system. The system was build as a part of the Information Retrieval module at the University of Leipzig.

Python 100.00%
bias-detection stylometry stylometric-features elasticsearch argument-retrieval university-project

argumentretrieval's Introduction

Bias and Arguments

Using Natural Language Processing Approaches to Influence the Ranking of an Argument Retrieval Model.

This git repository documents the code base used in a custom argument retrieval system. The whole documentation can be found in the paper as quoted underneath.


Abstract

Relying on the background of document retrieval systems, this paper presents an argument retrieval system. It uses state of the art python libraries to automatically process a data set of arguments from the args.me corpus and store them in a custom index. The system can perform the process of ranking and ordering a list of relevant documents in response to a user's query by comparing the request to the produced index of documents. The special focus is on the use of several NLP-based analyses, including topic signal modeling, stylometric analysis, and primarily bias detection. These methods will be used to influence the ranking of the search engine.

Acknowledgements

EN: This work was realized as part of the course "Information Retrieval" (summer semester 2020) under the supervision of Jun.-Prof. Dr. Martin Potthast, Lukas Gienapp and Christopher Akiki at the University of Leipzig.

DE: Diese Arbeit wurde im Rahmen des Kurses "Information Retrieval" (Sommersemester 2020) unter der Leitung von Jun.-Prof. Dr. Martin Potthast, Lukas Gienapp und Christopher Akiki an der Universität Leipzig realisiert.


Use

git clone https://github.com/Nicolas-le/argumentRetrieval.git

cd argumentRetrieval

pip3 install -r requirements.txt

python3 main.py --i <source> --o <output-file>

source: Link to directory, has to be structured in the same way as the args.me corpus [https://webis.de/data/args-me-corpus.html]

output-file: Link to directory with run.txt file existing in it.

example: python3 main.py --i ./code_base/corpus --o ./code_base/run

requirements: Python 3.6 is necessary at the moment, because of the Pattern.en module and some unfixed issues with this package and the newer python versions.


Elastic Search

Elastic Search has to be running as a service before starting the retrieval process. If it's not starting automatically use this command:

sudo systemctl start elasticsearch.service

argumentretrieval's People

Contributors

abdulnaser98 avatar alqader avatar nicolas-le avatar simon-reich avatar

Watchers

 avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.