Comments (10)
Hi, can you elaborate? Currently, the logic avoids any API calls over the network if the result is in the local cache.
Are you talking about a general rate-limiter for network traffic?
from safebrowsing.
I am talking about the network traffic in making the API call.
The scenario I am talking about is, if there are more than 10k unique partial hash hit and these are not in the local cache then we would make API call. At this point if the API key has a rate limit of 10K. It would exceed and get blocked.
from safebrowsing.
I see. In your usage of LookupURLs, are you passing in a batch of URLs every time you call it?
Before implementing any rate limiting logic, I would much rather see the TODO in that function be addressed. As it currently is, we make a network call for each URL, rather than batching them together.
On the other hand, if you have many goroutines calling LookupURLs independently with only a small handful of URLs, then some rate limiting mechanism may be needed.
from safebrowsing.
I am currently not batching the URLs to LookupURLs(which is on my TODO). Also I have a scenario where I have multiple goroutines/instance of the client running which would still need a mechanism for rate limiting across them.
from safebrowsing.
Batching of the URLs is going to be done any time soon or should I submit a PR?
from safebrowsing.
You're welcome to submit a PR (assuming you sign the CLA) and I can review it.
from safebrowsing.
I have a PR ready for the batching of the partial hashes. Can you please take a look at it @dsnet .
from safebrowsing.
@dsnet can you please let me know how to get this reviewed and merged in to master branch.
from safebrowsing.
Hi @rajnmithun, I apologize for the delay. I will try to take a look at this as soon as possible!
from safebrowsing.
@alexwoz thank you.
from safebrowsing.
Related Issues (20)
- How to query a historical snapshot? HOT 1
- Can you plz clarify the URL format that needs to be sent in the query? HOT 2
- Will this library be compatible with the new Web Risks API? HOT 1
- Why is the API marking Chinese cdns as unsafe while online/browser lookup aren't ? HOT 1
- project question / adding third-party threats
- Cripto
- Inconsistent Database: Threat List Mismatches HOT 7
- SafeSites URL filter - query filter?
- [email protected] HOT 1
- bnb1p5tus3kyl8euzu385pwzcj00wu7rjkpq4x5mqr
- Get the 'database is now healthy' every time.
- Please export the `safebrowing_proto` package
- I wonder what technique google safe browsing uses to update the API's list of malicious urls so that the API can detect the latest malicious urls, such as machine learning techniques? And how often is the API blacklist of malicious urls updated?
- digitalPhone issues
- Why does safe browsing remove some URLs? Then some of them re-added later on
- Is there an API to bulk report malicious websites to SafeBrowsing ?!
- New run
- Unsafe URL to test the Web Risk API HOT 1
- Ambulan Banjarmasin Jl.A.Yani KM.3,5 Komplek. Pandu Gang Damai No.53 RT.19 Kelurahan, Kebun Bunga, Kecamatan Banjarmasin Timur. Kota Banjarmasin, Kalimantan Selatan 70235 https://g.page/Ambulan-Banjarmasin , Indonesia AMBULAN BANJARMASIN SEBUAH PELAYANAN TRANSPORTASI KESEHATAN DARURAT DENGAN ARMADA TERBARU ANTAR KOTA ANTAR PROPINSI JUGA MEMBERIKAN PELAYANAN HOME CARE SEPERTI MERAWAT LUKA POST OPERASI, STOMA, GANGREN, DECUBITUS, DIABETES, Dll.. On Call 24 Jam : ☎️ 0812 5353 232 LAYANAN KU IBADAH KU MUDAHKAN MENUJU SURGA 👳🏻👳🏻👳🏻 Semoga Bermanfaat Bagi Umat Nabi Muhammad Amin Ya Allah Ya Rob 🤲🤲🤲 www.ambulanbanjarmasin.com https://g.page/Ambulan-Banjarmasin https://wa.me/c/628125353232 t.me/Ambulan_Banjarmasin tiktok.com/@ambulanbanjarmasin https://www.facebook.com/ambulanbanjarmasin https://www.linkedin.com/in/ambulan-banjarmasin https://github.com/Ambulanbanjarmasiin @Ambulan_232WD @AmbulanBanjarmasin #ambulanbanjarmasin
- Review requests for GitHub repositories
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from safebrowsing.