Giter VIP home page Giter VIP logo

req's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

req's Issues

How can i get a http request with basic auth?

	authHeader := req.Header{
		//"Authorization": "Basic YWRtaW46YWRtaW4=",
		"Authorization": basicAuth("admin", "password"),
	}

	resp, _ := req.Get("http://localhost:9112", authHeader)
	println(resp.String())

How to finish the basicAuth function ?

req goroutine fail

goroutine 77828 [select]:
net/http.(*persistConn).roundTrip(0xc00030ac60, 0xc0005ec9f0, 0x0, 0x0, 0x0)
/usr/local/go/src/net/http/transport.go:2498 +0x756
net/http.(*Transport).roundTrip(0xc0001da280, 0xc0015b1d00, 0xf8, 0xc0013a2701, 0xc0015b1d00)
/usr/local/go/src/net/http/transport.go:565 +0xad9
net/http.(*Transport).RoundTrip(0xc0001da280, 0xc0015b1d00, 0xc0001da280, 0xbfc4b6ce03456f8c, 0x7ff4564d55e)
/usr/local/go/src/net/http/roundtrip.go:17 +0x35
net/http.send(0xc0015b1c00, 0xe4d440, 0xc0001da280, 0xbfc4b6ce03456f8c, 0x7ff4564d55e, 0x1746a60, 0xc001a6c318, 0xbfc4b6ce03456f8c, 0x1, 0x0)
/usr/local/go/src/net/http/client.go:252 +0x43e
net/http.(*Client).send(0xc0002be6f0, 0xc0015b1c00, 0xbfc4b6ce03456f8c, 0x7ff4564d55e, 0x1746a60, 0xc001a6c318, 0x0, 0x1, 0x11)
/usr/local/go/src/net/http/client.go:176 +0xfa
net/http.(*Client).do(0xc0002be6f0, 0xc0015b1c00, 0x0, 0x0, 0x0)
/usr/local/go/src/net/http/client.go:699 +0x44a
net/http.(*Client).Do(...)
/usr/local/go/src/net/http/client.go:567
github.com/imroc/req.(*Req).Do(0xc00009b350, 0xd163d6, 0x3, 0xd3a72d, 0x29, 0x0, 0x0, 0x0, 0x469e1c, 0x40, ...)
/root/go/pkg/mod/github.com/imroc/[email protected]/req.go:323 +0xa0e
github.com/imroc/req.(*Req).Get(...)
/root/go/pkg/mod/github.com/imroc/[email protected]/req.go:612
github.com/imroc/req.Get(0xd3a72d, 0x29, 0x0, 0x0, 0x0, 0x1, 0x11, 0xc0004b17f0)
/root/go/pkg/mod/github.com/imroc/[email protected]/req.go:647 +0x7e
reptile/utils.GetProxy(0xc000ed6e60, 0x4)
/root/go/src/reptile/utils/get_proxy.go:12 +0x4b
reptile/utils.GetActiveProxy(0xc000ed6e40, 0x24, 0xc001060270, 0xca6e01, 0x0)
/root/go/src/reptile/utils/set_proxy.go:26 +0x53
reptile/vio.GetCacheClient(0xc00116d740, 0x12, 0x9f90a0)
/root/go/src/reptile/vio/index.go:389 +0x300
reptile/vio.FindNumberViolation(0x0, 0x0, 0xc0010dc8a0, 0x10, 0x1, 0x0, 0xc0010dc8a0, 0x10, 0x0)
/root/go/src/reptile/vio/index.go:480 +0x1106
reptile/controllers.(*VioController).FindNumberViolation(0xc000566a50)
/root/go/src/reptile/controllers/vio_controller.go:551 +0x368
reflect.Value.call(0xd10d40, 0xc000566a50, 0x3a13, 0xd16fe8, 0x4, 0x17729a0, 0x0, 0x0, 0x4b8277, 0xd10d40, ...)
/usr/local/go/src/reflect/value.go:460 +0x8ab
reflect.Value.Call(0xd10d40, 0xc000566a50, 0x3a13, 0x17729a0, 0x0, 0x0, 0x17729a0, 0x0, 0x0)
/usr/local/go/src/reflect/value.go:321 +0xb4
github.com/astaxie/beego.(*ControllerRegister).ServeHTTP(0xc00014a000, 0xe5d040, 0xc000113dc0, 0xc0016a6300)
/root/go/pkg/mod/github.com/astaxie/[email protected]/router.go:853 +0x147c
net/http.serverHandler.ServeHTTP(0xc0001121c0, 0xe5d040, 0xc000113dc0, 0xc0016a6300)
/usr/local/go/src/net/http/server.go:2807 +0xa3
net/http.(*conn).serve(0xc0002a0140, 0xe5f7c0, 0xc00089c3c0)
/usr/local/go/src/net/http/server.go:1895 +0x86c
created by net/http.(*Server).Serve
/usr/local/go/src/net/http/server.go:2933 +0x35c

请问我是结构体参数如何转换成req.Param?

type OnLineUserParam struct {
	Start   int    `json:"start"`
	Limit   int    `json:"limit"`
	Sort    string `json:"sort"`
	Dir     string `json:"dir"`
	Grpid   string `json:"grpid"`
	Recflag string `json:"recflag"`
	Keystr  string `json:"keystr"`
	Type    string `json:"type"`
}

像上面的结构体 要如何转换成你的Param?

use proxy show error info malformed HTTP response "<html>"

My proxy ip can used .but when use SetProxyUrl ,the result is malformed HTTP response "<html>"

the test code:

// 测试代理
func TestX5(t *testing.T) {

	header := req.Header{
		"Accept-Encoding": "gzip, deflate, br",
		"Accept":          "*/*",
		"User-Agent":      "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36",
	}

	req.SetProxyUrl("http://27.203.219.181:8060")
	req.Debug = true


	r, err := req.Get("https://www.cnblogs.com/", header)
	if err != nil {
		t.Error(err)
	}
	t.Log(r.String())
}

result:

Get https://www.cnblogs.com/: malformed HTTP response "<html>"
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x68 pc=0x12a78bc]

use branch : master

为啥初始化没有设置默认属性这些功能呢?

使用时往往会有这样的场景,就是为当前模块创建一个独立的http client, 并且可以初始化一些默认参数
比方说:

defaultHeader := make(http.Header)
defaultHeader .Set("APPID", "xxxx")
r := req.New("https://api.github.com", defaultHeader)

res, _ := r.get("/users")

关于重定向

这个库有没有可以设置不跟随重定向的选项啊,我访问不想跟随302跳转

使用Context传入req.Do 会导致debug模式内存出错

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x45d9f48]

goroutine 124 [running]:
github.com/imroc/req.(*Resp).dumpReqHead(0xc00024a580, 0xc00058b3e8)
	/Users/xh/go/pkg/mod/github.com/imroc/[email protected]/dump.go:117 +0x128
github.com/imroc/req.(*Resp).dumpRequest(0xc00024a580, 0xc00058b3e8)
	/Users/xh/go/pkg/mod/github.com/imroc/[email protected]/dump.go:103 +0x18f
github.com/imroc/req.(*Resp).Dump(0xc00024a580, 0x4d92d16, 0x10)
	/Users/xh/go/pkg/mod/github.com/imroc/[email protected]/dump.go:206 +0xc8
github.com/imroc/req.(*Req).Do(0xc0001889a0, 0x4d89712, 0x4, 0xc0000d0000, 0x44, 0xc0005edc48, 0x4, 0x4, 0xc0005edaa8, 0x403ea3b, ...)
	/Users/xh/go/pkg/mod/github.com/imroc/[email protected]/req.go:355 +0xafe
github.com/imroc/req.(*Req).Post(...)
	/Users/xh/go/pkg/mod/github.com/imroc/[email protected]/req.go:621
......

调用代码

post, err := r.Post(domain, header, req.FileUpload{
		File:      file,
		FileName:  filename,
		FieldName: "file",
	}, req.UploadProgress(progress),ctx)
	if err != nil {
		return nil, err
	}

跟踪内存发现,已经释放了r.req, 但是debug模式还在调用r.req.URL,
改成非debug模式或者关闭ctx就无问题

req.Param{}不支持struct中json tag

我需要从struct转为req.Param{},但是req.Param{}不支持struct中json tag。
struct param {
UserId *string json:"userId"
}
reqParam := req.Param{}

for k, v := range param {
reqParam[k] = v
}

fmt.Println(reqParam) //reqParam是{"UserId": xxx} 需要的是{"userId": xxx}

BodyJSON和BodyXml的个人看法

你传参数的方式很好,只是没有具体研究过这种方式(v.(type))是否有些性能影响。

在工作中也在用这个package, 谈下我的个人想法:
在传实体(json,xml)时,能否通过header去判断而不是用 BodyJSON等方式
req.Post(url, req.BodyJSON(&foo))

如:
//数据实体
type ReqData struct {
v interface{}
}

//在外边用的时候,传入header
req.Header{
"Accept": "application/json", //可以写一些公共的,大家直接用就可以了
"Content-Type": "application/json",
}

//在POST提交的方法中,通过判断header去完成不同的序列化

//同时返回也可以通过header去判断完成反序列化,不过相应的结构会有些调整

上传文件设置header头中Content-Type无效

	req.Debug = true
    //打开文件句柄操作
	// file, _ := os.Open(filename)
	progress := func(current, total int64) {
		fmt.Println(float32(current)/float32(total)*100, "%")
	}
	authHeader := req.Header{
		"User-Agent": "Go-http-client/1.2",
		"Content-Type": "image/png",
	}
	req.Post(tinypngPushApi,authHeader, req.File(filename),req.UploadProgress(progress))
    return nil
}

debug

2.4035301 %
POST /web/shrink HTTP/1.1
Host: tinypng.com
User-Agent: Go-http-client/1.2
Transfer-Encoding: chunked
Content-Type: multipart/form-data; boundary=bbc4ac77ebd45833cb74d9971017228148c453e4bfa38de6bc1eb77c7bc3
Accept-Encoding: gzip

--d05faa4986965b6bed833f966fd22a11adeb86dd774685c41bd33996af04
Content-Disposition: form-data; name="media"; filename="e.png"
Content-Type: application/octet-stream


--d05faa4986965b6bed833f966fd22a11adeb86dd774685c41bd33996af04--

=================================

HTTP/2.0 415 Unsupported Media Type
Content-Length: 73
Alt-Svc: clear
Content-Type: application/json
Date: Wed, 14 Aug 2019 03:14:11 GMT
Via: 1.1 google

{"error":"unsupported_media_type","message":"File type is not supported"}

Dose it support Go Mod?

I've got response while compile if I refer it as a mod directly:

import "github.com/imroc/req"

cannot load github.com/imroc/req: cannot find module providing package github.com/imroc/req

proxy enhancement?

Hello,

Is the http/https proxy authentication in plan?

will socks5 proxy be supported?

it would be nice if cookiejar could be supported.

best regards

debug模式下存在短暂的协程泄露

debug模式下dumpReqHead方法中由于没有读取response内容或者关闭response,导致setRequestCancel方法中stopTimerCh一直得不到值,直到超时

抄了一系列viper的接口

是不是可以考虑给这个包添一些类似viper的操作 这样获取回json数据后直接可以操作里面的值。

例如
`package main

import (
"fmt"

"github.com/ruolinn/req"

)

/*
{
"id": 1,
"name": "Leanne Graham",
"username": "Bret",
"email": "[email protected]",
"address": {
"street": "Kulas Light",
"suite": "Apt. 556",
"city": "Gwenborough",
"zipcode": "92998-3874",
"geo": {
"lat": "-37.3159",
"lng": "81.1496"
}
},
"phone": "1-770-736-8031 x56442",
"website": "hildegard.org",
"company": {
"name": "Romaguera-Crona",
"catchPhrase": "Multi-layered client-server neural-net",
"bs": "harness real-time e-markets"
}
}
*/

func main() {
rsp, _ := req.Get("https://jsonplaceholder.typicode.com/users/1")

fmt.Println(rsp.GetFloat64("address.geo.lng"))

}
`
我fork作者的库有过修改, 不知道这样合不合法 因为也是自己用,如果有冒犯的作者 我这边后面删除。

net/http: HTTP/1.x transport connection broken: http: ContentLength=431 with Body length 667

I am trying to upload a file to my server, but when I try to upload the file I get this:

error-->  Post http://127.0.0.1:2047/ctrl/upload: net/http: HTTP/1.x transport connection broken: http: ContentLength=431 with Body length 667
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x68 pc=0x6bbc2c]

This is the client code:

package ctrl

import (
    "fmt"
    "os"
    "github.com/imroc/req"
    "strings"
    "strconv"
)

func UploadFile(file_up string){

   //get file info
   fileInfo, err := os.Stat(file_up)
   if err!=nil{
	 fmt.Println(err)
    }

   file, _ := os.Open(file_up)
   upload:= req.FileUpload{
	   	File:      file,
	   	FieldName: "file",
	   	FileName:  fileInfo.Name(),
   }

   //set header
   header := req.Header{
	   "Content-Type":  "multipart/form-data",
	   "Authorization": "BEARER "+readKey(), 
	   "Content-Length": strconv.FormatInt(fileInfo.Size(),10), //convert bytes to string
   }

   defer file.Close()

   //set progress
   progress := func(current, total int64) {
	   fmt.Println(float32(current)/float32(total)*100, "%")
   }

r,err:=req.Post("http://127.0.0.1:2047/ctrl/upload",upload,header,req.UploadProgress(progress))
   if err != nil {
	   fmt.Println("error--> ",err)
   }

fmt.Println(r.String())
   resp := r.Response()
   r.Response().Body.Close()
   fmt.Println(resp.StatusCode)

}

The line that causes the problem is:

"Content-Length": strconv.FormatInt(fileInfo.Size(),10),

when I remove that line everything works fine, but the server log tells me this for each file uploaded

Content-Length = -1

and I need real Content-Length of each file for the record in the db

support retry

here is my code, hope it could be helpful.

type RetryableClient struct {
	Client    *http.Client
	Retryable *Retryable
}

type Retryable struct {
	RetryableStatus []int
	RetryerTime     time.Duration
	RetryerCount    int
	Attempt         int
	Enable          bool
}

func NewRetryClient() *RetryableClient {
	return &RetryableClient{
		Client: &http.Client{
			Timeout: 2 * time.Second,
		},
		Retryable: &Retryable{
			RetryableStatus: []int{
				http.StatusBadRequest, http.StatusInternalServerError,
				http.StatusBadGateway, http.StatusRequestTimeout,
			},
			RetryerTime:  time.Second,
			RetryerCount: 5,
			Attempt:      0,
			Enable:       true,
		},
	}
}

func (rr *RetryableClient) Disable() {
	rr.Retryable.Enable = false
}

func (rr *RetryableClient) SetTimeout(timeout time.Duration) {
	rr.Client.Timeout = timeout
}

func (rr *RetryableClient) SetRetryStatus(statusCode ...int) {
	var validStatusCode []int
	for _, code := range statusCode {
		if len(http.StatusText(code)) != 0 {
			validStatusCode = append(validStatusCode, code)
		}
	}
	rr.Retryable.RetryableStatus = validStatusCode
}

func (rr *RetryableClient) Do(req *http.Request) (*http.Response, []byte, error) {
	var (
		resp *http.Response
		body []byte
		err  error
	)
	for {
		resp, err = rr.Client.Do(req)
		if err != nil {
			revel.WARN.Printf("do request error: %v Attempt count: %v Request: %v", err, rr.Retryable.Attempt+1, req)
			if rr.Retryable.Enable && rr.Retryable.Attempt < rr.Retryable.RetryerCount {
				rr.Retryable.Attempt++
				continue
			} else {
				return resp, body, err
			}
		}
		defer resp.Body.Close()

		if rr.Retryable.Enable && rr.Retryable.Attempt < rr.Retryable.RetryerCount && contains(resp.StatusCode, rr.Retryable.RetryableStatus) {
			rr.Retryable.Attempt++
			time.Sleep(rr.Retryable.RetryerTime)
			revel.WARN.Printf("Status code: %v Attempt count: %v Request: %v", resp.StatusCode, rr.Retryable.Attempt, req)
			continue
		} else {
			resp.Header.Set("Retry-Count", strconv.Itoa(rr.Retryable.Attempt))
		}

		body, err = ioutil.ReadAll(resp.Body)
		if err != nil {
			revel.WARN.Printf("read body from response failed: %v Attempt count: %v Request: %v", err, rr.Retryable.Attempt+1, req)
			if rr.Retryable.Enable && rr.Retryable.Attempt < rr.Retryable.RetryerCount {
				rr.Retryable.Attempt++
				continue
			} else {
				return resp, body, err
			}
		}
		break
	}
	if resp.StatusCode != http.StatusOK {
		return resp, body, errors.New("bad response status code :" + strconv.Itoa(resp.StatusCode))
	}
	return resp, body, err
}

func contains(respStatus int, statuses []int) bool {
	for _, status := range statuses {
		if status == respStatus {
			return true
		}
	}
	return false
}

Bug set correct Content-Length

Hi,

i want to upload an file to oboom com via API. And req dont set the correct content length header.
I get the following message: "missing or invalid Content-Length header"

I debug it and content length it always 0.

Is it possible to set correct header?

Flags 设置 req.Lcost 无效

	var r *req.Resp
	req.SetTimeout(10 * time.Second)
	req.SetFlags(req.LstdFlags | req.Lcost) // 输出格式显示请求耗时
	if config.APP.Debug {
		req.Debug = true
	}
	if method == "get" {
		r, err = req.Get("http://baidu.com", header)
	} else if method == "post" {
		r, err = req.Post("http://baidu.com", header, req.BodyJSON(&content))
	}
	if err != nil {
		return nil, err
	}
	log.Debug("a1111", r)

输出的日志里面有请求方法、请求URL、请求参数、response。但是没有看到请求耗时。

Is QueryParams is a map[string]interface variable type?

I try to make a request with query params. When i define params with map[string]interface{} variable type, req is not sending query params.

Here is the detail

var params map[string]interface{}
params = req.QueryParam{
	"key": "value",
}
req.Get("https://something.com/path/", params)

But it's work well when params variable type is auto assigned.

params := req.QueryParam{
	"key": "value",
}
req.Get("https://something.com/path/", params)

for循环提示runtime error

使用for循环执行req.Post(),提示

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x653685]

goroutine 1 [running]:
github.com/imroc/req.(*Resp).Response(0x0, 0x37)
        /root/go/src/github.com/imroc/req/resp.go:36 +0x5
main.main()

还有一个问题,如果请求的URL连接失败,需要做特殊处理吗?
还是

r.Response().Body.Close()
continue

即可。
感谢。

请求发起后,是否需要手动调用 resp.Body.Close()

demo:

header := make(http.Header)
header.Set("Accept", "application/json")
resp, _ := req.Get("https://www.baidu.com", header)
defer resp.Response().Body.Close()

请问每次发起请求后 我需要自行加上下面这行代码吗?

defer resp.Response().Body.Close()

req.POST throws EOF, when the previous request fails

I have a function which calls resp, err := req.Post(url, req.BodyJSON(requestJson)) for each row in CSV to ingest data. When one of the requests fails (for genuine reason), the subsequent post call (most of the time only 1st call after the failed request) throws EOF even though there is no issues with data. This is not consistent though.

But, while testing I saw if we use req.New() and use this to post then the above error is not reproducible.

Is this expect? and also are there any implications if we use req.New() for each request instead of req.Post() method?

Make possible to set Debug for specific client.

I have a few services in my project.
So I wanna set Debug for one of them.
What do you think of inject debug flag in the Req client?

But it seems hard for compatibility with current implementation.
I mean, if we have set one of them, what to do?

Now I need to change this value before and after each request, it doesn't thread-safe.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.