eleow / shopbot Goto Github PK

Create User Guide for new users to use or deploy the system

Provide explanation of terminology for the product (eg certain technical specifications that the user might not be familiar with)

• Scrape data from multiple websites to compile list of terms
• Create sample test script to do simple look-up of key, and retrieve value
• Create sample test script to do context-based FAQ
• Implement in system

Script should be able to take in the following inputs

• Input config file containing DialogFlow credentials (eg PROJECT_ID and credential json file)
• Input file containing test cases of utterance and expected intent and entity values

Script should invoke DialogFlow backend, and test if detected intent is correct, and provide a summary of test-cases that have passed and failed

Design Chatbot persona based on best practices.
Some links to get started:
link1
link2

Output should be a document describing

• our target audience/stereotypes for the chatbot (age group, typical interests) based on some statistics
• the chatbot's archetype (See brand archetypes) and how it would identify with our target audience
• name of chatbot and why this name
• avatar/picture (can be a cartoon of a human or animal) and why

Report needs to describe

• business problem,
• the datasets used,
• the solution approaches,
• test results demonstrating performance of system,
• conclusions,
• references to publicly available libraries and source codes

Just a thought.

Based on the amazon data-set extracted from Amazon, we will have the following data:

X : set of words representing the review by the user
Y: Rating given to the particular product

If we can built a deployable model based on the input text give sentiment/review of the user.

An encoder-decoder model can be thought of where encoder can encode the input the text and decoder will simply predicting the output(no attention hopefully required).

Benefits:

1. If implementable, then the model will be robust of the length of the text.

Issues:

1. Data has a review text of different length. (1 word simply good or bad) to large text.
2. The model can further be improved to give a prediction of the topic the sentiment is about (about a feature/service/product classes). Input here needs to be constructed for the model.
3. The model can have issues with the review containing many sentences some describing good features some describing why some features were bad. Rating given are generally generalised. On the positive side if we can segment it in form of sentences/paragraphs then a prediction can be given for each sentence. and weighted review would be the final review for the same. In such scenarios Issues with mixed sentiment can be resolved better by the system.
4. Model depends on the accuracy :( needing to be trained.

Request to let me know your comments on the same.

To showcase how the chatbot could be easily integrated into a website
For an easy demo of the chatbot functionalities

While parsing multiple websites for amazon we get high memory usage causing disk space to be filled

Typos in questions from the user will limit the ability of the chatbot to respond with answers that are actually helpful.

One possible way we can tackle this is to do the following for each user input

• Tokenize using spaCy and send each word to rule-based spelling correction library such as Hunspell or DeepPavLov spelling correction or SymSpell, etc or even ML/DL models. This will result in a list of suggested replacements
• Compute similarity score of replacements with the original word
• Compute rank score for each suggestion based on certain criteria (eg frequency in English language based on n-gram frequency analysis of Google's Trillion Word Corpus)
• Combine and normalise to provide final score for each replacement, and select top n for each misspelled word
• Create list of permutations of different replacements for the entire sentence
• Finally send each of these to our backend for processing, and return the result with the highest confidence

(see article for more details)

Of course, the common alternative is to simply have a list of synonyms (also containing misspelled words) for each word that we think is important to us. This feature is supported by both DialogFlow and rasa out-of-the-box, but will not scale up.

We would be recommending the different products to the user based on his/her interest.

This can be done through the intent classification where we can obtain the requirement of the user.

The data needs to be further mapped with the product. The level of mapping also determines how well the recommendation system can be.

A knowledge graph conceptually is an open ended data representation which can allow querying through different dimensions.

Goal for proposing a knowledge graph based approach is

1. Ease of allowing query based on the product features.
2. Getting list of products having similar product feature: allowing navigating from one product to another
3. Efficient data structuring

Amazon and certain webpages allow the parsing in a continuous format, however a proper format is not found in case of Shoppee and Lazada where none is returned.

Also searching a generalised format for searching through ASIN numbers this can help getting a general web page for a particular ASIN number and the process of parsing can then be automated. Currently web-search for ASIN number returns the website related to the product however looking for a more generalized platform like google shopping for making successful search

• Select existing training data
• Scrape/Obtain test data
• Perform preprocessing
• Perform LDA and obtain aspects