Giter VIP home page Giter VIP logo

Comments (12)

0RAJA avatar 0RAJA commented on May 26, 2024 2

时间控制感觉没有用唉....我设置时间后并没有监听我的c.Done() 之后就算到时间结束了也不会有什么操作啊,不太清楚,,,

from go-programming-tour-book-comments.

wqcstrong avatar wqcstrong commented on May 26, 2024 1

文中关于限流的中间件代码是有问题的:
GetBuckets 通过 key 获取 bucket 的时候,key 的获取方式是自定义的逻辑(文中是 uri),但 AddBuckets 注册的时候又是写死的,这就会导致中间件获取 bucket 的时候几乎永远都是 false(除非走了狗屎运😒),导致限流中间件没有效果。

下面是我这边调整后的:

// middleware/limiter.go
var (
	IPLimiter = RateLimiter(limiter.NewIPLimiter(), limiter.BucketRule{
		FillInterval: time.Minute,
		Capacity:     1000,
		Quantum:      1000,
	})

	RouteLimiter = RateLimiter(limiter.NewRouteLimiter(), limiter.BucketRule{
		FillInterval: time.Minute,
		Capacity:     1000,
		Quantum:      1000,
	})
)

func RateLimiter(l limiter.LimiterIface, rule limiter.BucketRule) gin.HandlerFunc {
	return func(c *gin.Context) {
		key := l.GetKey(c)
		bucket, ok := l.GetBuckets(key)
		if !ok {
			rule.Key = key
			l.AddBuckets(rule)
			bucket, _ = l.GetBuckets(key)
		}

		count := bucket.TakeAvailable(1)
		if count == 0 {
			response := app.NewResponse(c)
			response.ToErrorResponse(errcode.TooManyRequest)
			c.Abort()
			return
		}
		c.Next()
	}
}

// pkg/limiter/ip-limiter.go

type IPLimiter struct {
	Limiter *Limiter
}

func NewIPLimiter() *IPLimiter {
	l := &IPLimiter{
		Limiter: &Limiter{
			Bucket: make(map[string]*ratelimit.Bucket),
		},
	}
	return l
}

func (l *IPLimiter) GetKey(c *gin.Context) string {
	ip := c.ClientIP()
	return ip
}

func (l *IPLimiter) GetBuckets(key string) (*ratelimit.Bucket, bool) {
	bucket, ok := l.Limiter.Bucket[key]
	return bucket, ok
}

func (l *IPLimiter) AddBuckets(rules ...BucketRule) {
	for _, rule := range rules {
		l.Limiter.Bucket[rule.Key] = ratelimit.NewBucketWithQuantum(rule.FillInterval, rule.Capacity, rule.Quantum)
	}
}

我上面定义的 RouteLimiterIPLimiter 实现方式差不多,供参考。

from go-programming-tour-book-comments.

wqcstrong avatar wqcstrong commented on May 26, 2024 1

@forthespada 我们理解的有一丢丢的偏差,我指出的问题在于:「GetBuckets 是通过 key 获取 bucket,AddBuckets 时直接写入 /auth」,如果应用中没有注册这个 /auth 路由的话,那这个中间件则没有效果。

另外我上面的调整也不会对全部请求都做限流,下面我一并贴一下实现和使用:

  1. 实现路由限流接口
// pkg/limiter/route-limiter.go
type RouteLimiter struct {
	Limiter *Limiter
}

func NewRouteLimiter() *RouteLimiter {
	return &RouteLimiter{
		Limiter: &Limiter{
			Bucket: make(map[string]*ratelimit.Bucket),
		},
	}
}
func (l *RouteLimiter) GetKey(c *gin.Context) string {
	uri := c.Request.RequestURI
	index := strings.Index(uri, "?")
	if index == -1 {
		return uri
	}
	return uri[:index]
}
func (l *RouteLimiter) GetBuckets(key string) (*ratelimit.Bucket, bool) {
  // ... 和 ip-limiter 实现一样
}
func (l *RouteLimiter) AddBuckets(rules ...BucketRule) {
  // ... 和 ip-limiter 实现一样
}
  1. 路由限流中间件的使用
// 2.1 实例化中间件
// middleware/limiter.go
RouteLimiter = RateLimiter(limiter.NewRouteLimiter(), limiter.BucketRule{
  FillInterval: time.Minute,
  Capacity:     5,
  Quantum:      1000,
})

// 2.2 在路由中注册中间件
v1 := r.Group("/v1")
v1.GET("/verify/captcha", middleware.RouteLimiter, api.GetAuthCaptcha)

// 2.3 日志输出查看效果
2022-04-21 08:59:22     DEBUG   middleware/logger.go:65 HTTP Access Log {"status": 200, "request": "GET  /v1/verify/captcha", "query": "", "ip": "127.0.0.1", "time": "2.916ms"}
2022-04-21 08:59:25     DEBUG   middleware/logger.go:65 HTTP Access Log {"status": 200, "request": "GET  /v1/verify/captcha", "query": "", "ip": "127.0.0.1", "time": "2.299ms"}
2022-04-21 08:59:26     DEBUG   middleware/logger.go:65 HTTP Access Log {"status": 200, "request": "GET  /v1/verify/captcha", "query": "", "ip": "127.0.0.1", "time": "1.987ms"}
2022-04-21 08:59:27     DEBUG   middleware/logger.go:65 HTTP Access Log {"status": 200, "request": "GET  /v1/verify/captcha", "query": "", "ip": "127.0.0.1", "time": "2.123ms"}
2022-04-21 08:59:28     DEBUG   middleware/logger.go:65 HTTP Access Log {"status": 200, "request": "GET  /v1/verify/captcha", "query": "", "ip": "127.0.0.1", "time": "2.234ms"}
2022-04-21 08:59:28     WARN    middleware/logger.go:63 HTTP Warn 429   {"status": 429, "request": "GET  /v1/verify/captcha", "query": "", "ip": "127.0.0.1", "time": "0.173ms"}

from go-programming-tour-book-comments.

willnotlazy avatar willnotlazy commented on May 26, 2024

建议被依赖的结构体声明放到后面,避免在手动练习过程中报红

from go-programming-tour-book-comments.

willnotlazy avatar willnotlazy commented on May 26, 2024

LimiterIface 接口的AddBuckets() 按照源码是不应有返回值的, 所以上诉的 AddBuckets是错误的.

from go-programming-tour-book-comments.

TBXark avatar TBXark commented on May 26, 2024
type AccessLogWriter struct {
	gin.ResponseWriter
	body *bytes.Buffer
}

这个做法body不就有两份了吗,如果这个body比较大的话,感觉还是挺占内存的

from go-programming-tour-book-comments.

forthespada avatar forthespada commented on May 26, 2024

限流中间件的方法是没问题的。
煎鱼做的是需要针对某些路由方法来进行限流,而不是对全部的请求进行限流;评论区的那个代码做法会对全部的请求都做到限流。

from go-programming-tour-book-comments.

stephenzhang0713 avatar stephenzhang0713 commented on May 26, 2024

在限流那块,MethodLimiter结构体里面可嵌套一个LimiterIface

type MethodLimiter struct {
	*Limiter
	LimiterIface
}

from go-programming-tour-book-comments.

oYto avatar oYto commented on May 26, 2024

要是买了书,就能看完整版电子档内容就好了

from go-programming-tour-book-comments.

blkcor avatar blkcor commented on May 26, 2024
func RateLimiter(l limiter.LimiterIface) gin.HandlerFunc {
	return func(c *gin.Context) {
		key := l.Key(c)
		//如果buckets中存在这个key -> 请求已经发生过了
		if bucket, ok := l.GetBucket(key); ok {
			//try take available token
			count := bucket.TakeAvailable(1)
			//no available token -> refuse request
			if count == 0 {
				response := app.NewResponse(c)
				response.ToErrorResponse(errcode.TooManyRequests)
				c.Abort()
				return
			}
		} else {
			l.AddBuckets(limiter.LimiterBucketRule{
				Key:          key,
				FillInterval: time.Second,
				Capacity:     10,
				Quantum:      10,
			})
		}
		c.Next()
	}
}

这里加上key不存在的逻辑就不用在router.go中使用中间件的时候吧key写死了

from go-programming-tour-book-comments.

githublister avatar githublister commented on May 26, 2024

defer func() {
if err := recover(); err != nil {
global.Logger.WithCallersFrames().Errorf(c, "panic recover err: %v", err)

			err := defailtMailer.SendMail(
				global.EmailSetting.To,
				fmt.Sprintf("异常抛出,发生时间: %d", time.Now().Unix()),
				fmt.Sprintf("错误信息: %v", err),
			)
			if err != nil {
				global.Logger.Panicf(c, "mail.SendMail err: %v", err)
			}

			app.NewResponse(c).ToErrorResponse(errcode.ServerError)
			c.Abort()
		}
	}()

Cannot convert 'nil' to type 'any' 第一行的nil 报错啊 IDE

from go-programming-tour-book-comments.

wyming175533 avatar wyming175533 commented on May 26, 2024

@wqcstrong 其实限流这里本身就可以再追加多个限流规则的,具体的key写死却是不合适,通常可以加入到配置文件中,针对不同的key,做不同的限流规则,在以前开发java项目也是这样做的
var methodLimiters = limiter.NewMethodLimiter().AddBuckets(
limiter.LimiterBucketRule{
Key: "/auth",
FillInterval: time.Second,
Capacity: 10,
Quantum: 10,
},
limiter.LimiterBucketRule{
Key: "/api/v1",
FillInterval: time.Second,
Capacity: 10,
Quantum: 10,
},
)
甚至它是支持链式调用的
var methodLimiters = limiter.NewMethodLimiter().AddBuckets(limiter.LimiterBucketRule{
Key: "/auth",
FillInterval: time.Second,
Capacity: 10,
Quantum: 10,
},
limiter.LimiterBucketRule{
Key: "/api/v1/tags",
FillInterval: time.Second,
Capacity: 10,
Quantum: 55,
},
).AddBuckets(
limiter.LimiterBucketRule{
Key: "/...",
FillInterval: time.Second,
Capacity: 99,
Quantum: 10,
},
limiter.LimiterBucketRule{
Key: "/...",
FillInterval: time.Second,
Capacity: 100,
Quantum: 1000,
},
)
接口限流通常只针对部分接口,生成key的时候也可以生成多个,比如/api/v1/tags,你就可以生成一个api/v1,和api/v1/tags,再根据不同的给出不同规则

from go-programming-tour-book-comments.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.