sjsufall2019-cmpe272 / wellpharma Goto Github PK

JavaScript 4.72% CSS 3.56% HTML 57.90% Python 0.36% Jupyter Notebook 33.46%

wellpharma's Introduction

Well Pharma

Team Members

Alaukika Diwanji
Dhruwaksh Dave
Panth Desai
Udit Marolia

Abstract

Many valuable information regarding the public health and welfare, disease outbreaks and their trend are available in the form of unstructured data lying in different news portals, Facebook, Twitter. It becomes important to become aware of the current diseases and to filter out relevant and correct information. This is especially important for commercial pharmacies as their need to be updated with the current outbreak in their region and also be ready stock-wise for the drugs needed to treat them. Our objective with Well-Pharma is to address this problem and built a system for the pharmacies which will analyse the disease outbreaks in all regions and carry out a disease-to-drug mapping and alert the pharmacist so as to keep the stock ready.

WellPharma will be a Web Application - built as an automated system for querying filtering and visualising the disease outbreak and to stock their respective drugs.

Persona

The main target user for the application will be the commercial pharmacies. The system will provide insights to the pharmacists on which drugs need to be stocked as per the current disease outbreak.

Hill Statement

The system will provide the pharmacists drugs recommendations as per the diseases prevelant in the selected region so that they can be prepared with those OTC drugs.

Process Flow

The main objective of the system will be to gather information regarding the disease outbreaks, location-wise through web crawling the CDC site to obtain disease data. Another crawler will crawl the internet for mapping drugs with diseases. This file will be fed into a python program that converts it into a timeseries form which will be fed into Facebook Prophet. It gives us the prediction about January 20. The top 5 diseases in January 20 are displayed on the website. The user(pharmacist) requests states his location, based on which the system provides the diseases prevalent in the region and the drugs corresponding to them. The data can help identify which drugs are needed to be stocked such that there is no shortage.

Technology Stack

Web Application

The Web Application will be the main interface to the users. The application will be built using the JS tools

Frontend: ReactJS will be used for the GUI.It will also take care of the authentication and token passing.

Backend: NodeJS and NPM Libraries will be used for backend. It will communicate with MySQL database to fetch the queried. Data

Cloud: The web application will be hosted on Amazon AWS using Docker container

Database

MySQL database will be used to store all data, which is : a) The data from the classification model, which will give information about location wise disease outbreak b) The disease to drug mapped data c) The web application data

Data Model

Data Cleaning and preprocessing: Python will be used to extract the data from various public web sources and clean the data.

NLP /NER: Python,NLTK, Tensorflow will be used for classification of the data location wise so as the location and the diseases prevalent there are identified and mapped.

Initial Feedback

Professor Ranjan's initial feedback

You must clearly define the personas. If this is for the health care provider or govt agency then gathering drug sale data from all pharmacy in a zipcode(for ex) is tricky. sales data of drugs at local level may not be available publicly. They wont share. large pharma companies may share their sales data but that may not tell you the ground truth. Also in developing countries like india majority of drug sales are not tracked due to tax evasion. Here is the twist to the problem I am giving you. You apply NLP and NER on all the public news in a geography or zipcode to figure out the health related problems and then give recommnedation to local pharmacist to stock those OTC drugs. You can also extend this to beauty products based on the beauty trends but beauty trends are more global than local. Weather data can play role as well.

wellpharma's People

Contributors

Watchers

Forkers

panthdesai18

wellpharma's Issues

Technical viability using social media: user reliability and misinformation

One concern is the reliability of user posts on social media. It is unlikely that a normal user will post about diseases on a regular basis, for example weekly. You are more likely to get info from dedicated groups such as a city’s health organizations or groups on social media related to a certain medical condition, for example diabetes or cancer. Then you could more easily spot specific trends. There may be a limited number of such groups in a certain city, however.

Furthermore, misinformation is rampant on social media, and even news sites may make mistakes. For example, there may be a health scare on a measles outbreak which make it look like the disease is happening more often than it actually is. There may be a number of posts with stories or untrue statistics on local diseases. How will the web app deal with this misinformation? Will there be a way to filter the content even further?

Technology Choice : APIs over Web Scraping

Web Scraping is a program or algorithm which is used to extract and process large amounts of data from web. It looks like that Well Pharma are going to use Python libraries like requests or beautifulsoup4 for Web Scraping. However this may not be the best choice. Most sites have a file called robots.txt which tells us about which parts of the site can be scraped and which parts are not allowed to scrape. Web Scraping on sites like Facebook may be risky as they may require written permission to do any scraping.

See. https://www.facebook.com/robots.txt

There are some ways to bypass the blocks but its a nasty thing to do. An alternative solution could be to use APIs given by sites to access their data. For ex: Twitter is the best example where Well Pharma can use the API instead of scraping. The good thing is that APIs are easier to use than scraping.

Business Value

How will the web app detect cases of diseases where there is not a large enough population to be noticed? Meaning if a single person gets sick, how will Wellpharma get the data? There may also be situations where there is no mention of a disease outbreak on social media. Then how will WellPharma be able to detect it?

Does Wellpharma fall short of business value?

Disease outbreak is a sudden increase in the occurrences of a disease in a particular place and time. For any disease to be coined as outbreak there has to be enough number of people already suffering from that disease. So by the time a news or social media platform inform the public of this outbreak the disease will have spread and people affected by this disease will already have been to the pharmacy for medicines.

Let's assume that if 100 people in a location suffer from a particular disease over a span of time then we can call it disease outbreak. Now let there be 10 pharmacies in an area and each pharmacy having enough medicines for 5 patients. The first 50 patients will be able to get the medicine, however the pharmacies will not be able to immediately provide for the other 49 patients. The purpose of Wellpharma is to solve this problem. But it will fail to do so. This happens because Wellpharma is trying to prevent the problem after the problem has already occurred. Therefore, I feel that Wellpharma fails to solve its purpose and falls short of business value.

Technology Choice: MySQL versus MongoDB, News APIs

MongoDB would improve performance over MySQL because information will be written to the database, the disease data and location, much more often than reading from the database itself. MongoDB allows unstructured documents and you can add new columns without suffering as much in speed as MySQL. See: https://blog.panoply.io/mongodb-and-mysql

Also, Google News API is deprecated so there may be problems later. You may need to use other news sites. Using city and regional newspaper sites may be more helpful since those will be local and close to the pharmacy, but these are unlikely to have any APIs to help with scraping. This will make it more difficult to collect trend data.