Clamav file size limit - throw custom error about clamscan HOT 10 CLOSED

No problem. I will make PR with proposition.

from clamscan.

Now I start thinking that it can be clamd problem. I've performed some tests on v0.103.6 clamd and it looks like clamscan is detecting socket error (write EPIPE), before any information is send back by clamd (that stream size limit is exceeded) almost every time (much, much more often than for v0.104.x or v0.105.x).

I'm starting my holidays so I will get back to this when I'll be back.

from clamscan.

I have investigated it a little bit more, and it looks like it can be a problem on clamscan side (don't know where exactly).
Calmav is calling function handle_stream to process the network payload (link). If the payload exceeds StreamMaxLength the handle_stream function:

1. calls conn_reply_error which sends INSTREAM size limit exceeded. to the socket (link)
2. set error to 1
3. return -1

After that the socket is closed (link)

Here is the debug information from calmd side when StreamMaxLength is exceeded:

WARNING: Thu Jul 28 12:32:11 2022 -> INSTREAM: Size limit reached, (requested: 104857600, max: 26214400)
Thu Jul 28 12:32:11 2022 -> Moved partial command: 4
Thu Jul 28 12:32:11 2022 -> mode == MODE_STREAM
Thu Jul 28 12:32:11 2022 -> Got chunksize: 104857600
Thu Jul 28 12:32:11 2022 -> Shutting down socket after error (FD 10)
Thu Jul 28 12:32:11 2022 -> Number of file descriptors polled: 1 fds

It looks like clamscan does not every time receive/process the information that the payload is too big and even if this information is properly received/processed clamscan does not properly handle such situation (it does not reject but resolve with NodeClamError)

from clamscan.

@benzino77 If you have any spare time to look into how to resolve the issue of clamscan not receiving/processing the info consistently and submit a PR, I'd be very grateful. At the moment, I'm just a bit overloaded with a million other projects. Also, thank you so much for the investigative work you've done so far--it's been very helpful and informative.

from clamscan.

No problem. I will make PR with proposition.

Definitely, this approach going to help us. therefore we can notify such error in our services & then we'll throw based that to my HTTP client request

@benzino77 @kylefarris
Once Again - Thank you guys ( For your Support ).

Looking forward to this PR 🏳

from clamscan.

Thanks for the update @benzino77. Enjoy your holidays!

from clamscan.

Hi @kylefarris,

Please can you help me

from clamscan.

Hi @SaltwaterC, @genio, @ngraef

Please can you help me
it's urgent to me 😊

from clamscan.

I'm observing the same situation (clamd v0.104 and clamd v0.105). What I can say is that sometimes it is rejected on socket.error and sometimes it is resolved with NodeClamError:

Reject example:

node-clam: Provided stream is readable.
node-clam: Attempting to establish socket/TCP connection for "scanStream"
node-clam: using remote server: 127.0.0.1:3310
node-clam: Received final data from stream.
node-clam: The input stream has dried up.
node-clam: _initSocket Socket/Host connection failed: Error: write EPIPE
    at WriteWrap.onWriteComplete [as oncomplete] (internal/stream_base_commons.js:94:16) {
  errno: 'EPIPE',
  code: 'EPIPE',
  syscall: 'write'
}
node-clam: scanStream Error emitted from ClamAV socket:  Error: write EPIPE
    at WriteWrap.onWriteComplete [as oncomplete] (internal/stream_base_commons.js:94:16) {
  errno: 'EPIPE',
  code: 'EPIPE',
  syscall: 'write'
}
node-clam: _initSocket Socket/Host connection closed.
node-clam: scanStream ClamAV socket has been closed! true

and here is Resolve with NodeClamError example:

node-clam: Provided stream is readable.
node-clam: Attempting to establish socket/TCP connection for "scanStream"
node-clam: using remote server: 127.0.0.1:3310
node-clam: Received final data from stream.
node-clam: The input stream has dried up.
node-clam: scanStream Received output from ClamAV Socket.
node-clam: scanStream ClamAV is done scanning.
node-clam: scanStream Raw Response:  INSTREAM size limit exceeded. ERROR
node-clam: Error Response:  INSTREAM size limit exceeded.
node-clam: File may be INFECTED!
CUSTOM DEBUG scanStream before resolve:  NodeClamError: An error occurred while scanning the piped-through stream: INSTREAM size limit exceeded.
    at NodeClam._processResult (/home/benzino/Dokumenty/clamav-rest-api/node_modules/clamscan/index.js:781:20)
    at Socket.<anonymous> (/home/benzino/Dokumenty/clamav-rest-api/node_modules/clamscan/index.js:2283:45)
    at Socket.emit (events.js:326:22)
    at endReadableNT (_stream_readable.js:1241:12)
    at processTicksAndRejections (internal/process/task_queues.js:84:21) {
  data: { error: 'INSTREAM size limit exceeded. \u0000' },
  date: 2022-07-27T10:07:14.898Z
}
node-clam: _initSocket Socket/Host connection failed: Error: write EPIPE
    at WriteWrap.onWriteComplete [as oncomplete] (internal/stream_base_commons.js:94:16) {
  errno: 'EPIPE',
  code: 'EPIPE',
  syscall: 'write'
}
node-clam: scanStream Error emitted from ClamAV socket:  Error: write EPIPE
    at WriteWrap.onWriteComplete [as oncomplete] (internal/stream_base_commons.js:94:16) {
  errno: 'EPIPE',
  code: 'EPIPE',
  syscall: 'write'
}
node-clam: _initSocket Socket/Host connection closed.
node-clam: scanStream ClamAV socket has been closed! true

from clamscan.

I have investigated it a little bit more, and it looks like it can be a problem on clamscan side (don't know where exactly). Calmav is calling function handle_stream to process the network payload (link). If the payload exceeds StreamMaxLength the handle_stream function:

1. calls conn_reply_error which sends INSTREAM size limit exceeded. to the socket (link)
2. set error to 1
3. return -1

After that the socket is closed (link)

Here is the debug information from calmd side when StreamMaxLength is exceeded:

WARNING: Thu Jul 28 12:32:11 2022 -> INSTREAM: Size limit reached, (requested: 104857600, max: 26214400)
Thu Jul 28 12:32:11 2022 -> Moved partial command: 4
Thu Jul 28 12:32:11 2022 -> mode == MODE_STREAM
Thu Jul 28 12:32:11 2022 -> Got chunksize: 104857600
Thu Jul 28 12:32:11 2022 -> Shutting down socket after error (FD 10)
Thu Jul 28 12:32:11 2022 -> Number of file descriptors polled: 1 fds

It looks like clamscan does not every time receive/process the information that the payload is too big and even if this information is properly received/processed clamscan does not properly handle such situation (it does not reject but resolve with NodeClamError)

@benzino77 - Thanks for spending some time here - to help us! 😊

from clamscan.