Giter VIP home page Giter VIP logo

node-webscrapper's People

Contributors

anekenonso avatar louicoder avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

node-webscrapper's Issues

Set up sms integration

Set up sms integration in the project. This task completely relies on the completion of Set up Twilio and whatsapp issue. Therefore, it's important that this task be attempted after the task mentioned above is completed.

This is because we shall be using the same Twilio account for both sms and whatsapp. Inside the social_integrations.js file create a pseudo function called sendTwilioSms for sending an sms and the functionality will implemented at a later stage.

Resources:
Sending sms using node and Twilio

Add telegram integration

Add telegram integration to the application. You should create a function and that function should be executed on an array of the collaborators telegram numbers sending each a message.

  • Create a new folder in ./App called Telegram
  • Inside that folder create a new file called telegram.js
  • Inside that file create called 'telegramSendMessage` which will be responsibe for sending messages to our telegram numbers or more flexible to our telegram group.
  • telegramSendMessage should receive have two parameters. one for the array of numbers to be messaged and the second for the message to be sent.

Resources:

Add read and write to file functionality

  • add new folder named Files in App folder
  • add new folder inside Files folder called FilesUtility
  • inside Files Utility, create a new file called readAndWriteToFiles.js
  • inside readAndWriteToFiles file add two functions one for writing to files called writeToTextFile and the other readFromTextFile
  • make sure both functions are to exported from this file to be used in other files.

Make sure to check all the requirements before raising a PR for this issue

Update project wiki

Update the project wiki with the relevant resources needed for the project to go on.

Create a cron-job

Create a cron-job in a new file called cronJob.js. In this file set up a cron-job console log any output/string every after a number of seconds you have set. For more on how to get started on cron-jobs in NodeJS click NodeJS cron-jobs

Make sure to run the cron-job in server.js file and not from your file.

Create PR template

Create a PR template that will be automated upon creating a new pull request.

Add a package.json

Add a package.json to the project. This can be automated using a command npm init -y for defaults which you will edit in the created package.json file and replace the following keys

  • url ( this should be the url of the git repository, get it from the project repo)
  • name, this should Node webscrapper
  • description , this should be A web scrapper built on NodeJS for data analytics

Run npm install <package name> --save so that the following packages are added to the package.json file

  • request
  • cheerio
  • apify
  • puppeteer

You can find more about setting up a package.json file here >> Link to resource

Make sure the above checklist is complete

Set up Twilio - whatsapp integration

Set up integration for Whatsapp and Twilio into our Node project. Create a new file and name it social_integrations.js. The checklist below should guide you on how this should be executed.

checklist

  • set up Twilio account with username - nodescrapper and password - nodescrapper45!
  • go through twilio account set up for whatsapp Example Here
  • go through this example on how to integrate Twilio and whatsapp in our application. Example here

Resources:
send whatsapp message with twilio and NodeJS

Add your name to collaborators

Add your name to the collaborators table in the readMe.md file. Make sure to add a link to your repository as well.

  • you should also create a new branch for this implementation.

Modify read and write file functionality

Modify the read and write functionality to be able to actually read from a file and also write to a file

  • The current implementation only added the functions without the body to be executed.

Related PR: #34

Host the application on Heroku

Host the application onto the Heroku platform. Make sure that all the environmental variables are set well
The variables include

  • PORT, this is the server address.
  • TWILIO_SID, this is the Twilio SID
  • TWILIO_TOKEN, this is the Twilio token
  • SENDER_EMAIL, this is the gmail address of the sender
  • SENDER_PASSWORD, this is the sender's password to login to gmail servers

Add username to readme file

You should add your username to the readme file along with the link to your profile. You can simply follow the table format. The username should be added to the collaborators/contributors table. All this should be done in the readme.md file

Modify readMe file

Modify readme file to add more details about what the project focuses on and what the end results will look like

Add gitignore file.

Add a gitignore file in the application root. make sure the checklist below is completed and tick completed task

  • add an entry of node_modules folder
  • add .vscode folder

The file should be of extension .gitignore

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.