Giter VIP home page Giter VIP logo

alinajafi1998 / comstream Goto Github PK

View Code? Open in Web Editor NEW
26.0 5.0 5.0 1.7 MB

In this project, we implemented a topic detection system on Twitter. This system reads tweets from a data stream and assigns them to one of the existing clusters or a new one. Each cluster acts as an agent, which makes the proposed approach a multi-agent system. There is also a coordinator, who monitors the whole system and coordinates the agent.

License: MIT License

Python 82.00% Jupyter Notebook 18.00%
nlp topic-detection multi-agent data-stream elixir clustering-algorithm labse

comstream's Introduction

Hi there!

  • I am Ali Najafi, a Computer Science and Engineering master's student at Sabanci University, Turkey. ๐Ÿ“š ๐ŸŽ“
  • My field of research is mostly about Computational Social Science and Natural Language Processing. ๐ŸŽฏ
  • Always interested in Data and Real-World Challenges. ๐Ÿค—

Now I'm listening... ๐ŸŽง

Spotify


Get in touch

I am always open to new opportunities and collabrations. Do not hesitate to reach out to me!

๐ŸŒ www.najafi-ali.com

comstream's People

Contributors

alimpfard avatar alinajafi1998 avatar arazshilabin avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

comstream's Issues

Load model and continue training

Is there a way to continue training?
I saw that the part where the model is saved was commented in your code, so I uncommented it, saved a model a loaded it with the load_model( ) method, but when I call the method train( ), it starts to train from the beginning.
Am I missing something? Or is the feature not implemented?
Thanks a lot!

Number of agents becomes zero and returns only a cluster with one data point

Hey Ali, congratulations for your work!

I'm making some experiences with your solution with the data I'm working on, with the requirements described at README.md, but I think that I'm not getting the desired results...

I runned the main.py (with my own files) and I'm getting this error while fitting TfidfVectorizer on get_topics_of_agents method from Cordinator.py line 390:

empty vocabulary; perhaps the documents only contain stop words

If I comment the section where get_topics_of_agents is called to save topics, the logs on console show that the number of agents becomes zero, and the saved clusters (which are only one, or two in some runs) only appear to have one text:

Init_agents done : Number of agents : 2
2020-01-02 20:58:00 : Communicating -> Number of agents : 1
2020-01-02 20:58:00 : Save Model and Outputs -> Number of agents : 1
2020-01-02 20:59:00 : Communicating -> Number of agents : 0

Any idea what might be causing this behaviour?

Thank you so much in advance :)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.