Giter VIP home page Giter VIP logo

agent's Introduction

We've moved!

Hi! This repository is no longer being used and has been archived for historical purposes.

For more information on the move, see our blog post about this transition, and this thread with additional questions. We look forward to seeing you in our new spaces!

agent's People

Contributors

dependabot[bot] avatar github-actions[bot] avatar lukekarrys avatar nlf avatar torbjorn-svensson avatar wraithgar avatar

Stargazers

avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

torbjorn-svensson shunf4 hashim21223445 morni-andoka qasura-h

agent's Issues

Code smells

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

The following code is slightly suspect. If written as intended, it could use a comment.

Due to the way once works, this expression throws an Array containing a single Error instead of the probably intended Error.

agent/lib/agents.js

Lines 126 to 128 in 4b2c4d1

(s) => once(socket, 'error', s).then((err) => {
throw err
}),

This code creates a non-resolving promise (maybe instead of an immediately-resolving Promise?).

agent/lib/util.js

Line 23 in c51096b

return signal ? new Promise(() => {}) : null

Initially reported when trying to figure out the root cause behind: npm/cli#6763

Expected Behavior

No response

Steps To Reproduce

  1. In this environment...
  2. With this config...
  3. Run '...'
  4. See error...

Environment

  • npm:
  • Node:
  • OS:
  • platform:

[BUG] proxy behavior change introduced in #59 – appears to always use HttpProxyAgent

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

Overview

I’ve noticed a behavior change in npm/agent v2.2.0 that shipped as part of npm/cli v10.2.0.

npm install appears to always use HttpProxyAgent instead of selecting between HttpProxyAgent and HttpsProxyAgent depending on the requested URL's protocol.

Observations

I think I've isolated the behavior change to #59.

Prior to #59, it looks like the library would select an HttpAgent or HttpsAgent based on the url.protocol of the request.

agent/lib/index.js

Lines 20 to 30 in 22ac4e5

const secure = url.protocol === 'https:'
const proxy = getProxy(url, { proxy: _proxy, noProxy })
const options = { ...normalizeOptions(_options), proxy }
return cacheAgent({
key: cacheOptions({ ...options, secure }),
cache: agentCache,
secure,
proxies: [HttpAgent, HttpsAgent],
}, options)
}

agent/lib/util.js

Lines 67 to 75 in 22ac4e5

const cacheAgent = ({ key, cache, secure, proxies }, ...args) => {
if (cache.has(key)) {
return cache.get(key)
}
const Ctor = (secure ? proxies[1] : proxies[0]) ?? proxies[0]
const agent = new Ctor(...args)
cache.set(key, agent)
return agent
}

After #59, the library selects an HttpProxyAgent or HttpsProxyAgent based on an options.secureEndpoint property, but this appears to (always?) be undefined and not based on the url.protocol of the request.

agent/lib/index.js

Lines 26 to 36 in e24eb6c

const cacheKey = cacheOptions({
...normalizedOptions,
secureEndpoint: url.protocol === 'https:',
})
if (agentCache.has(cacheKey)) {
return agentCache.get(cacheKey)
}
const newAgent = new Agent(normalizedOptions)
agentCache.set(cacheKey, newAgent)

agent/lib/agents.js

Lines 26 to 32 in e24eb6c

if (proxy) {
this.#proxy = {
proxy: urlify(proxy),
noProxy,
Agent: getProxyAgent(proxy),
}
}

agent/lib/proxy.js

Lines 26 to 37 in e24eb6c

const getProxyAgent = (url) => {
url = urlify(url)
const protocol = url.protocol.slice(0, -1)
if (SOCKS_PROTOCOLS.has(protocol)) {
return SocksProxyAgent
}
if (protocol === 'https' || protocol === 'http') {
return [HttpProxyAgent, HttpsProxyAgent]
}
throw new InvalidProxyProtocolError(url)

agent/lib/agents.js

Lines 64 to 72 in e24eb6c

let { Agent: ProxyAgent } = this.#proxy
if (Array.isArray(ProxyAgent)) {
ProxyAgent = options.secureEndpoint ? ProxyAgent[1] : ProxyAgent[0]
}
const proxyAgent = new ProxyAgent(proxy, this.#options)
proxyCache.set(cacheKey, proxyAgent)
return proxyAgent

Expected Behavior

  • I would expect npm install to use HttpProxyAgent when the requested url has an http protocol.
  • I would expect npm install to use HttpsProxyAgent when the requested url has an https protocol.

Steps To Reproduce

  1. Use the latest node and npm, for example: node v21.3.0 (npm v10.2.4)
  2. Configure .npmrc to use a proxy, for example:
https-proxy=<some proxy>
http-proxy=<some proxy>
  1. Ensure an empty cache and install a package like uuid
$ npm cache clean --force
$ npm install -g uuid
  1. Observe HttpProxyAgent is used to fetch an https url: 'https://registry.npmjs.org/uuid'
  2. Observe the request's path is modified when using an HttpProxyAgent to setRequestProps for an https url. For example, the request.path is modified from '/uuid' to 'http://registry.npmjs.org:443/uuid'.

agent/lib/agents.js

Lines 161 to 173 in e24eb6c

addRequest (request, options) {
const proxy = this.#getProxy(options)
// it would be better to call proxy.addRequest here but this causes the
// http-proxy-agent to call its super.addRequest which causes the request
// to be added to the agent twice. since we only support 3 agents
// currently (see the required agents in proxy.js) we have manually
// checked that the only public methods we need to call are called in the
// next block. this could change in the future and presumably we would get
// failing tests until we have properly called the necessary methods on
// each of our proxy agents
if (proxy?.setRequestProps) {
proxy.setRequestProps(request, options)
}

  1. Note that the installation succeeds in the example above, but other registries appear to be less forgiving of the path change and this can cause installation issues in some cases.

Environment

  • npm: 10.2.4
  • Node: 21.3.0
  • OS: macOS 14.1.1
  • platform: macOS

[BUG] Fails to connect http url using https proxy

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

I try to fetch a http:// webpage using HttpAgent (or getAgent()) initialized with https:// proxy url:

const agent = new HttpAgent({proxy:'https://some-proxy-domain:8888'}) 
const response = await fetch('http://some-url', {...opts, agent})

But this fails with InvalidProxyResponseError:

{
  code: 'EINVALIDRESPONSE',
  status: 403,
  errno: 'EINVALIDRESPONSE',

It seems that something wrong is being sent to proxy, so I'm getting such headers from it:

rawHeaders: [
    'Server',
    'squid/4.10',
...
    'X-Squid-Error',
    'ERR_ACCESS_DENIED 0',
...
    'Connection',
    'close'
  ]

Expected Behavior

I can successfully access "http://some-url" via browser or curl using the same proxy.

I tried different agents, and found that http-proxy-agent works fine.

Expected, that agent can do it too.

Steps To Reproduce

See above...

Environment

  • "@npmcli/agent": "^1.1.0"
  • npm: -
  • Node: v16.20.1
  • OS: macos 10.15
  • platform: mac mini

[BUG] tests fail in node 20

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

No response

Expected Behavior

No response

Steps To Reproduce

  1. In this environment...
  2. With this config...
  3. Run '...'
  4. See error...

Environment

  • npm:
  • Node:
  • OS:
  • platform:

[BUG] windows 18.8.0 tests fail

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

No response

Expected Behavior

No response

Steps To Reproduce

  1. In this environment...
  2. With this config...
  3. Run '...'
  4. See error...

Environment

  • npm:
  • Node:
  • OS:
  • platform:

feat: more granular timeout configuration

Implement the following timeout types:

Connection - socket

Starts when connection is initially attempted (the call to net/tls.connect()), ends when the connection is established (the connect event). When timeout occurs, the socket is destroyed and removed from the pool. Default value should be something relatively small.

Idle - socket

Starts when data flow stops, ends when data flow resumes. When timeout occurs, the socket is destroyed and removed from the pool. Default value should be something relatively small. This one just uses the node native timeout property on the socket as that is an idle timeout.

Response - request

Starts after the request has been sent (the finish event, specifically), ends when the first chunk of data is received (the response event). When timeout occurs, the request errors and the socket is returned to the pool. Default value will be 0.

Total transfer - request

Starts when the first data chunk is received (the response event), ends when the response has been completely received (the close event). When timeout occurs, the request errors and the socket is returned to the pool. Default value will be 0.

feat: implement socks proxy

we currently lack a socks proxy entirely causing us to fall back to socks-proxy-agent which has the same pitfalls we're trying to move away from. in order to eliminate our usage on this module, we need to implement our own socks support

[BUG] agent.js caches proxy w/o respect for credentials

What

getAgent() in agent.js serializes the proxy based only on the proxy protocol, hostname, and port, disregarding the credentials (username and password). This causes a bug that prevents the library from using the same proxy multiple times with different credentials.

How

Current Behavior

agent.js, lines 30-42:

  const key = [
    `https:${isHttps}`,
    pxuri
      ? `proxy:${pxuri.protocol}//${pxuri.host}:${pxuri.port}`
      : '>no-proxy<',
    `local-address:${opts.localAddress || '>no-local-address<'}`,
    `strict-ssl:${isHttps ? !!opts.strictSSL : '>no-strict-ssl<'}`,
    `ca:${(isHttps && opts.ca) || '>no-ca<'}`,
    `cert:${(isHttps && opts.cert) || '>no-cert<'}`,
    `key:${(isHttps && opts.key) || '>no-key<'}`,
    `timeout:${agentTimeout}`,
    `maxSockets:${agentMaxSockets}`
  ].join(':')

Steps to Reproduce

fetch(url, {proxy: "https://user1:[email protected]:1234"}); // uses https://user1:[email protected]:1234
fetch(url, {proxy: "https://user2:[email protected]:1234"}); // ALSO uses https://user1:[email protected]:1234

Expected Behavior

fetch(url, {proxy: "https://user1:[email protected]:1234"}); // uses https://user1:[email protected]:1234
fetch(url, {proxy: "https://user2:[email protected]:1234"}); // uses https://user2:[email protected]:1234

Recommended Patch

  const key = [
    `https:${isHttps}`,
    pxuri
      ? `proxy:${pxuri.href}` // or 
      : '>no-proxy<',
    `local-address:${opts.localAddress || '>no-local-address<'}`,
    `strict-ssl:${isHttps ? !!opts.strictSSL : '>no-strict-ssl<'}`,
    `ca:${(isHttps && opts.ca) || '>no-ca<'}`,
    `cert:${(isHttps && opts.cert) || '>no-cert<'}`,
    `key:${(isHttps && opts.key) || '>no-key<'}`,
    `timeout:${agentTimeout}`,
    `maxSockets:${agentMaxSockets}`
  ].join(':')

If it is a security risk to include the username and password in the agent cache key, then a hashed version of pxuri.href can be used instead.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.