Comments (12)
时间控制感觉没有用唉....我设置时间后并没有监听我的c.Done() 之后就算到时间结束了也不会有什么操作啊,不太清楚,,,
from go-programming-tour-book-comments.
文中关于限流的中间件代码是有问题的:
GetBuckets
通过 key 获取 bucket 的时候,key 的获取方式是自定义的逻辑(文中是 uri),但 AddBuckets
注册的时候又是写死的,这就会导致中间件获取 bucket 的时候几乎永远都是 false(除非走了狗屎运😒),导致限流中间件没有效果。
下面是我这边调整后的:
// middleware/limiter.go
var (
IPLimiter = RateLimiter(limiter.NewIPLimiter(), limiter.BucketRule{
FillInterval: time.Minute,
Capacity: 1000,
Quantum: 1000,
})
RouteLimiter = RateLimiter(limiter.NewRouteLimiter(), limiter.BucketRule{
FillInterval: time.Minute,
Capacity: 1000,
Quantum: 1000,
})
)
func RateLimiter(l limiter.LimiterIface, rule limiter.BucketRule) gin.HandlerFunc {
return func(c *gin.Context) {
key := l.GetKey(c)
bucket, ok := l.GetBuckets(key)
if !ok {
rule.Key = key
l.AddBuckets(rule)
bucket, _ = l.GetBuckets(key)
}
count := bucket.TakeAvailable(1)
if count == 0 {
response := app.NewResponse(c)
response.ToErrorResponse(errcode.TooManyRequest)
c.Abort()
return
}
c.Next()
}
}
// pkg/limiter/ip-limiter.go
type IPLimiter struct {
Limiter *Limiter
}
func NewIPLimiter() *IPLimiter {
l := &IPLimiter{
Limiter: &Limiter{
Bucket: make(map[string]*ratelimit.Bucket),
},
}
return l
}
func (l *IPLimiter) GetKey(c *gin.Context) string {
ip := c.ClientIP()
return ip
}
func (l *IPLimiter) GetBuckets(key string) (*ratelimit.Bucket, bool) {
bucket, ok := l.Limiter.Bucket[key]
return bucket, ok
}
func (l *IPLimiter) AddBuckets(rules ...BucketRule) {
for _, rule := range rules {
l.Limiter.Bucket[rule.Key] = ratelimit.NewBucketWithQuantum(rule.FillInterval, rule.Capacity, rule.Quantum)
}
}
我上面定义的 RouteLimiter
和 IPLimiter
实现方式差不多,供参考。
from go-programming-tour-book-comments.
@forthespada 我们理解的有一丢丢的偏差,我指出的问题在于:「GetBuckets
是通过 key 获取 bucket,AddBuckets
时直接写入 /auth
」,如果应用中没有注册这个 /auth
路由的话,那这个中间件则没有效果。
另外我上面的调整也不会对全部请求都做限流,下面我一并贴一下实现和使用:
- 实现路由限流接口
// pkg/limiter/route-limiter.go
type RouteLimiter struct {
Limiter *Limiter
}
func NewRouteLimiter() *RouteLimiter {
return &RouteLimiter{
Limiter: &Limiter{
Bucket: make(map[string]*ratelimit.Bucket),
},
}
}
func (l *RouteLimiter) GetKey(c *gin.Context) string {
uri := c.Request.RequestURI
index := strings.Index(uri, "?")
if index == -1 {
return uri
}
return uri[:index]
}
func (l *RouteLimiter) GetBuckets(key string) (*ratelimit.Bucket, bool) {
// ... 和 ip-limiter 实现一样
}
func (l *RouteLimiter) AddBuckets(rules ...BucketRule) {
// ... 和 ip-limiter 实现一样
}
- 路由限流中间件的使用
// 2.1 实例化中间件
// middleware/limiter.go
RouteLimiter = RateLimiter(limiter.NewRouteLimiter(), limiter.BucketRule{
FillInterval: time.Minute,
Capacity: 5,
Quantum: 1000,
})
// 2.2 在路由中注册中间件
v1 := r.Group("/v1")
v1.GET("/verify/captcha", middleware.RouteLimiter, api.GetAuthCaptcha)
// 2.3 日志输出查看效果
2022-04-21 08:59:22 DEBUG middleware/logger.go:65 HTTP Access Log {"status": 200, "request": "GET /v1/verify/captcha", "query": "", "ip": "127.0.0.1", "time": "2.916ms"}
2022-04-21 08:59:25 DEBUG middleware/logger.go:65 HTTP Access Log {"status": 200, "request": "GET /v1/verify/captcha", "query": "", "ip": "127.0.0.1", "time": "2.299ms"}
2022-04-21 08:59:26 DEBUG middleware/logger.go:65 HTTP Access Log {"status": 200, "request": "GET /v1/verify/captcha", "query": "", "ip": "127.0.0.1", "time": "1.987ms"}
2022-04-21 08:59:27 DEBUG middleware/logger.go:65 HTTP Access Log {"status": 200, "request": "GET /v1/verify/captcha", "query": "", "ip": "127.0.0.1", "time": "2.123ms"}
2022-04-21 08:59:28 DEBUG middleware/logger.go:65 HTTP Access Log {"status": 200, "request": "GET /v1/verify/captcha", "query": "", "ip": "127.0.0.1", "time": "2.234ms"}
2022-04-21 08:59:28 WARN middleware/logger.go:63 HTTP Warn 429 {"status": 429, "request": "GET /v1/verify/captcha", "query": "", "ip": "127.0.0.1", "time": "0.173ms"}
from go-programming-tour-book-comments.
建议被依赖的结构体声明放到后面,避免在手动练习过程中报红
from go-programming-tour-book-comments.
LimiterIface 接口的AddBuckets() 按照源码是不应有返回值的, 所以上诉的 AddBuckets是错误的.
from go-programming-tour-book-comments.
type AccessLogWriter struct {
gin.ResponseWriter
body *bytes.Buffer
}
这个做法body不就有两份了吗,如果这个body比较大的话,感觉还是挺占内存的
from go-programming-tour-book-comments.
限流中间件的方法是没问题的。
煎鱼做的是需要针对某些路由方法来进行限流,而不是对全部的请求进行限流;评论区的那个代码做法会对全部的请求都做到限流。
from go-programming-tour-book-comments.
在限流那块,MethodLimiter结构体里面可嵌套一个LimiterIface
type MethodLimiter struct {
*Limiter
LimiterIface
}
from go-programming-tour-book-comments.
要是买了书,就能看完整版电子档内容就好了
from go-programming-tour-book-comments.
func RateLimiter(l limiter.LimiterIface) gin.HandlerFunc {
return func(c *gin.Context) {
key := l.Key(c)
//如果buckets中存在这个key -> 请求已经发生过了
if bucket, ok := l.GetBucket(key); ok {
//try take available token
count := bucket.TakeAvailable(1)
//no available token -> refuse request
if count == 0 {
response := app.NewResponse(c)
response.ToErrorResponse(errcode.TooManyRequests)
c.Abort()
return
}
} else {
l.AddBuckets(limiter.LimiterBucketRule{
Key: key,
FillInterval: time.Second,
Capacity: 10,
Quantum: 10,
})
}
c.Next()
}
}
这里加上key不存在的逻辑就不用在router.go中使用中间件的时候吧key写死了
from go-programming-tour-book-comments.
defer func() {
if err := recover(); err != nil {
global.Logger.WithCallersFrames().Errorf(c, "panic recover err: %v", err)
err := defailtMailer.SendMail(
global.EmailSetting.To,
fmt.Sprintf("异常抛出,发生时间: %d", time.Now().Unix()),
fmt.Sprintf("错误信息: %v", err),
)
if err != nil {
global.Logger.Panicf(c, "mail.SendMail err: %v", err)
}
app.NewResponse(c).ToErrorResponse(errcode.ServerError)
c.Abort()
}
}()
Cannot convert 'nil' to type 'any' 第一行的nil 报错啊 IDE
from go-programming-tour-book-comments.
@wqcstrong 其实限流这里本身就可以再追加多个限流规则的,具体的key写死却是不合适,通常可以加入到配置文件中,针对不同的key,做不同的限流规则,在以前开发java项目也是这样做的
var methodLimiters = limiter.NewMethodLimiter().AddBuckets(
limiter.LimiterBucketRule{
Key: "/auth",
FillInterval: time.Second,
Capacity: 10,
Quantum: 10,
},
limiter.LimiterBucketRule{
Key: "/api/v1",
FillInterval: time.Second,
Capacity: 10,
Quantum: 10,
},
)
甚至它是支持链式调用的
var methodLimiters = limiter.NewMethodLimiter().AddBuckets(limiter.LimiterBucketRule{
Key: "/auth",
FillInterval: time.Second,
Capacity: 10,
Quantum: 10,
},
limiter.LimiterBucketRule{
Key: "/api/v1/tags",
FillInterval: time.Second,
Capacity: 10,
Quantum: 55,
},
).AddBuckets(
limiter.LimiterBucketRule{
Key: "/...",
FillInterval: time.Second,
Capacity: 99,
Quantum: 10,
},
limiter.LimiterBucketRule{
Key: "/...",
FillInterval: time.Second,
Capacity: 100,
Quantum: 1000,
},
)
接口限流通常只针对部分接口,生成key的时候也可以生成多个,比如/api/v1/tags,你就可以生成一个api/v1,和api/v1/tags,再根据不同的给出不同规则
from go-programming-tour-book-comments.
Related Issues (20)
- 开启博客之路 | Go 语言编程之旅 HOT 5
- Go 大杀器之性能剖析 PProf(下) | Go 语言编程之旅 HOT 2
- gRPC 的使用和了解 | Go 语言编程之旅 HOT 5
- 同时提供 HTTP 接口 | Go 语言编程之旅 HOT 2
- Go 大杀器之性能剖析 PProf(上) | Go 语言编程之旅 HOT 6
- 基于 TCP 的聊天室 | Go 语言编程之旅 HOT 5
- 生成接口文档 | Go 语言编程之旅 HOT 13
- 缓存淘汰算法 | Go 语言编程之旅 HOT 1
- 缓存的性能和优化思路 | Go 语言编程之旅 HOT 2
- 进行项目设计 | Go 语言编程之旅 HOT 3
- 非核心功能 | Go 语言编程之旅 HOT 2
- 公开和发布度量指标 | Go 语言编程之旅 HOT 1
- gorm v2版本没有gorm解决方案
- 实现聊天室:广播器 | Go 语言编程之旅 HOT 3
- 附录 D:让 Go panic 的十种方法 | Go 语言编程之旅 HOT 2
- 服务注册和发现 | Go 语言编程之旅 HOT 1
- 用 GODEBUG 看调度跟踪 | Go 语言编程之旅 HOT 1
- 附录 B:Goroutine 与 panic、recover 的小问题 | Go 语言编程之旅 HOT 2
- WebSocket 介绍、握手协议和细节 | Go 语言编程之旅 HOT 13
- 在尝试blog-service中Logger部分出错
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from go-programming-tour-book-comments.