Giter VIP home page Giter VIP logo

dublin-house-search's Introduction

Dublin House Search

Personal project created to keep searching houses in Dublin that matches different criteria. The motivation for the project came from situation of the house rental in Dublin where it is very hard to find a place. When you find something you like, you have to go on a viewing with hundreds with other people and hope you are the granted the grace of being selected by the landlord or agent.

The software keeps searching for houses on daft.ie and when somerhing is found it sends a notification that can be acted on potentially giving an advange by being one of the first applicants. It has a few advantages over the proprietary website and app:

  • It gives a more fine grained search options, allowing complex search criterias to be constructed.
  • Includes a geographical search that allow to specify exact areas of the city (by using latitude and longitude).

The project has a few learning purposes:

  • Use multithreading
  • Exercise modularity
  • Extensibility (by using Java's ServiceLoader)

Architecture overview

Core House Search Crawler

This module reads all the properties available on daft.ie and call the plugins to oprocess each property. This task is done in two steps:

  • First the website interface is used to get the list of properties
  • Each property page is retrieved to extract all the data

Different threads are responsible for doing this in the following way:

  • Producer threads retrieve the list of Rental, Sharing and Sale porperties in one thread for each property type and stores it on concurrent collections.
  • Multile consumer/procucer threads consume the list of properties and extract the data for each property and produce the collection of property data, again stored in concurrent colllections.
  • Multiple consumer threads now consume the property data and pass it to the plugins to be processed.

The data extraction grabs the data fro the webpages because there is no public API for using the data.

Plugins

Due to the ad-hoc nature of the architecture new plugins can be implemented and deployed without the need to change the core crawler. Each plugin is independent of each other.

dublin-house-search's People

Contributors

carvalhorr avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.