benzino77 / clamav-rest-api Goto Github PK

View Code? Open in Web Editor NEW

106.0 106.0 39.0 7.76 MB

ClamAV REST API. Scan files using simple POST request.

License: MIT License

Dockerfile 1.82% JavaScript 98.18%

api-server clamav nodejs rest-api virus-scanning

clamav-rest-api's People

Contributors

Stargazers

Watchers

clamav-rest-api's Issues

I do not know why, but during startup, the container is displayed as active and disappears after 2-3 seconds, this error is found in the logs
Cannot initialize clamav object: Error: connect ECONNREFUSED 127.0.0.1:3310
netstat -lnp | grep -E "(clam|3310|docker)" ->

what is host name when i use clamd-rest-api within the container.On juypter host name is localhost and its working, but when i run python code within the container its not working . Please help

Open vulnerabilities for express-fileupload

Currently, at the latest master e107592, I've observed that express-fileupload using version 1.4.0, which exposes vulnerabilities CVE-2022-27140 (critical) and CVE-2022-27261 (high).

Despite upgrading to version 1.5.0, both vulnerabilities persist in the Express-fileupload library.

Details:

CVE-2022-27140 (CRITICAL): being disputed in the NIST database
CVE-2022-27261 (HIGH): still open, might pose a risk for file overwrite

Previous Discussions:

Issue #312: Link
Issue #316: Link

Do we assess the risks associated with these vulnerabilities, given that we are using express-fileupload: 1.4.0?

CRA returns `"is_infected": null`

I ran docker-compose (simple docker-compose up) and CRA returns result as in the title. No matter what files I scanned, every time it returned null.
When I changed image from benzino77/clamav-rest-api to benzino77/clamav-rest-api:1.1.2 it works fine - hence, there must be some regression done lately.

❯ curl -s -XPOST http://localhost:8080/api/v1/scan -F [email protected] | jq
{
  "success": true,
  "data": {
    "result": [
      {
        "name": "versions.sh",
        "is_infected": null,
        "viruses": []
      }
    ]
  }
}

logs from containers:

synerise-base-images-api-1    | 
synerise-base-images-api-1    | > [email protected] start /clamav-rest-api
synerise-base-images-api-1    | > node src/app.js
synerise-base-images-api-1    | 
synerise-base-images-api-1    | Cannot initialize clamav object: Error: connect ECONNREFUSED 172.18.0.2:3310
synerise-base-images-api-1 exited with code 0
Socket for clamd not found yet, retrying (18/1800) ...Tue Feb 14 00:09:35 2023 -> Limits: Global time limit set to 120000 milliseconds.
synerise-base-images-clamd-1  | Tue Feb 14 00:09:35 2023 -> Limits: Global size limit set to 104857600 bytes.
synerise-base-images-clamd-1  | Tue Feb 14 00:09:35 2023 -> Limits: File size limit set to 26214400 bytes.
synerise-base-images-clamd-1  | Tue Feb 14 00:09:35 2023 -> Limits: Recursion level limit set to 17.
synerise-base-images-clamd-1  | Tue Feb 14 00:09:35 2023 -> Limits: Files limit set to 10000.
synerise-base-images-clamd-1  | Tue Feb 14 00:09:35 2023 -> Limits: MaxEmbeddedPE limit set to 10485760 bytes.
synerise-base-images-clamd-1  | Tue Feb 14 00:09:35 2023 -> Limits: MaxHTMLNormalize limit set to 10485760 bytes.
synerise-base-images-clamd-1  | Tue Feb 14 00:09:35 2023 -> Limits: MaxHTMLNoTags limit set to 2097152 bytes.
synerise-base-images-clamd-1  | Tue Feb 14 00:09:35 2023 -> Limits: MaxScriptNormalize limit set to 5242880 bytes.
synerise-base-images-clamd-1  | Tue Feb 14 00:09:35 2023 -> Limits: MaxZipTypeRcg limit set to 1048576 bytes.
synerise-base-images-clamd-1  | Tue Feb 14 00:09:35 2023 -> Limits: MaxPartitions limit set to 50.
synerise-base-images-clamd-1  | Tue Feb 14 00:09:35 2023 -> Limits: MaxIconsPE limit set to 100.
synerise-base-images-clamd-1  | Tue Feb 14 00:09:35 2023 -> Limits: MaxRecHWP3 limit set to 16.
synerise-base-images-clamd-1  | Tue Feb 14 00:09:35 2023 -> Limits: PCREMatchLimit limit set to 100000.
synerise-base-images-clamd-1  | Tue Feb 14 00:09:35 2023 -> Limits: PCRERecMatchLimit limit set to 2000.
synerise-base-images-clamd-1  | Tue Feb 14 00:09:35 2023 -> Limits: PCREMaxFileSize limit set to 26214400.
synerise-base-images-clamd-1  | Tue Feb 14 00:09:35 2023 -> Archive support enabled.
synerise-base-images-clamd-1  | Tue Feb 14 00:09:35 2023 -> AlertExceedsMax heuristic detection disabled.
synerise-base-images-clamd-1  | Tue Feb 14 00:09:35 2023 -> Heuristic alerts enabled.
synerise-base-images-clamd-1  | Tue Feb 14 00:09:35 2023 -> Portable Executable support enabled.
synerise-base-images-clamd-1  | Tue Feb 14 00:09:35 2023 -> ELF support enabled.
synerise-base-images-clamd-1  | Tue Feb 14 00:09:35 2023 -> Mail files support enabled.
synerise-base-images-clamd-1  | Tue Feb 14 00:09:35 2023 -> OLE2 support enabled.
synerise-base-images-clamd-1  | Tue Feb 14 00:09:35 2023 -> PDF support enabled.
synerise-base-images-clamd-1  | Tue Feb 14 00:09:35 2023 -> SWF support enabled.
synerise-base-images-clamd-1  | Tue Feb 14 00:09:35 2023 -> HTML support enabled.
synerise-base-images-clamd-1  | Tue Feb 14 00:09:35 2023 -> XMLDOCS support enabled.
synerise-base-images-clamd-1  | Tue Feb 14 00:09:35 2023 -> HWP3 support enabled.
synerise-base-images-clamd-1  | Tue Feb 14 00:09:35 2023 -> Self checking every 600 seconds.
synerise-base-images-clamd-1  | Tue Feb 14 00:09:35 2023 -> Set stacksize to 1048576
synerise-base-images-clamd-1  | socket found, clamd started.
synerise-base-images-clamd-1  | Starting Freshclamd
synerise-base-images-clamd-1  | ClamAV update process started at Tue Feb 14 00:09:36 2023
synerise-base-images-clamd-1  | daily database available for update (local version: 26644, remote version: 26811)
synerise-base-images-clamd-1  | WARNING: downloadFile: file not found: https://database.clamav.net/daily-26645.cdiff
synerise-base-images-clamd-1  | WARNING: downloadPatch: Can't download daily-26645.cdiff from https://database.clamav.net/daily-26645.cdiff
synerise-base-images-clamd-1  | WARNING: downloadFile: file not found: https://database.clamav.net/daily-26645.cdiff
synerise-base-images-clamd-1  | WARNING: downloadPatch: Can't download daily-26645.cdiff from https://database.clamav.net/daily-26645.cdiff
synerise-base-images-clamd-1  | WARNING: downloadFile: file not found: https://database.clamav.net/daily-26645.cdiff
synerise-base-images-clamd-1  | WARNING: downloadPatch: Can't download daily-26645.cdiff from https://database.clamav.net/daily-26645.cdiff
synerise-base-images-clamd-1  | WARNING: Incremental update failed, trying to download daily.cvd
synerise-base-images-api-1    | 
synerise-base-images-api-1    | > [email protected] start /clamav-rest-api
synerise-base-images-api-1    | > node src/app.js
synerise-base-images-api-1    | 
synerise-base-images-api-1    | Server started on PORT: 3000
synerise-base-images-clamd-1  | Testing database: '/var/lib/clamav/tmp.0ea496f986/clamav-75a151af4898ac4094f65ca92323f264.tmp-daily.cvd' ...
synerise-base-images-clamd-1  | Database test passed.
synerise-base-images-clamd-1  | daily.cvd updated (version: 26811, sigs: 2020779, f-level: 90, builder: raynman)
synerise-base-images-clamd-1  | main.cvd database is up-to-date (version: 62, sigs: 6647427, f-level: 90, builder: sigmgr)
synerise-base-images-clamd-1  | bytecode.cvd database is up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia2)
synerise-base-images-clamd-1  | Clamd successfully notified about the update.
synerise-base-images-clamd-1  | Tue Feb 14 00:09:46 2023 -> Reading databases from /var/lib/clamav
synerise-base-images-clamd-1  | Tue Feb 14 00:10:05 2023 -> Database correctly reloaded (8652711 signatures)
synerise-base-images-clamd-1  | Tue Feb 14 00:10:05 2023 -> Activating the newly loaded database...
synerise-base-images-api-1    | ::ffff:172.18.0.1 - - [14/Feb/2023:00:10:29 +0000] "POST /api/v1/scan HTTP/1.1" 200 91 "-" "curl/7.85.0"
synerise-base-images-api-1    | ::ffff:172.18.0.1 - - [14/Feb/2023:00:11:58 +0000] "POST /api/v1/scan HTTP/1.1" 200 91 "-" "curl/7.85.0"
synerise-base-images-api-1    | ::ffff:172.18.0.1 - - [14/Feb/2023:00:12:05 +0000] "POST /api/v1/scan HTTP/1.1" 200 91 "-" "curl/7.85.0"
synerise-base-images-api-1    | ::ffff:172.18.0.1 - - [14/Feb/2023:00:12:11 +0000] "POST /api/v1/scan HTTP/1.1" 200 91 "-" "curl/7.85.0"
synerise-base-images-api-1    | ::ffff:172.18.0.1 - - [14/Feb/2023:00:12:19 +0000] "POST /api/v1/scan HTTP/1.1" 200 91 "-" "curl/7.85.0"
synerise-base-images-api-1    | ::ffff:127.0.0.1 - - [14/Feb/2023:00:14:33 +0000] "GET /api/v1/dbsignatures HTTP/1.1" 200 98 "-" "curl/7.64.0"
synerise-base-images-api-1    | ::ffff:127.0.0.1 - - [14/Feb/2023:00:15:20 +0000] "POST /api/v1/scan HTTP/1.1" 200 92 "-" "curl/7.64.0"
synerise-base-images-api-1    | ::ffff:127.0.0.1 - - [14/Feb/2023:00:17:53 +0000] "POST /api/v1/scan HTTP/1.1" 200 92 "-" "curl/7.64.0"
synerise-base-images-api-1    | ::ffff:172.18.0.1 - - [14/Feb/2023:00:19:31 +0000] "POST /api/v1/scan HTTP/1.1" 400 59 "-" "curl/7.85.0"
synerise-base-images-api-1    | ::ffff:172.18.0.1 - - [14/Feb/2023:00:19:34 +0000] "POST /api/v1/scan HTTP/1.1" 200 91 "-" "curl/7.85.0"
synerise-base-images-clamd-1  | Tue Feb 14 00:19:48 2023 -> SelfCheck: Database status OK.

Why forcing filenames to have APP_FORM_KEY name ?

Hello @benzino77,

I was wondering if there is any specific reason to only scan files with APP_FORM_KEY name ?

There is a use case of your project where we could use it to scan uploaded files to a web (HTTP) app. A reverse proxy can intercept the request and send it back to the CRA if there is at least one uploaded file. But then we need to edit the requests on the fly to replace filenames with APP_FORM_KEY (that will consume resources on the reverse proxy). A pragmatic way of doing it would be to copy (or even better stream) the client request directly to CRA.

Something like that should do the trick :

for (file in req.files) {
    ...
    const r = await scanFile(req.files[file], av);
    ...
}

Let me know what you think.

Changelog available

Hello, do you have any changelog? I cannot find version history or changelog.

File storage path

Hi Chaps,

Where can i find the Files/Path that are passed from API in docker clam av machine?
Will the files gets automatically deleted in docker clam av machine?

k8s setup communication problem - ECONNREFUSED

Hi,
I have a problem with running rest api in k8s cluster based on your examples. Sometimes i get response like this.
{ "success": false, "data": { "error": { "errno": "ECONNREFUSED", "code": "ECONNREFUSED", "syscall": "connect", "address": "10.97.205.26", "port": 3310 } } }

Generally, 50% calls fail. With docker-compose setup it works perfectly fine. Is it a bug? or could you give me some advice how to avoid this?

Thank you
Jakub

Error when calling /

HI there, When calling the service with curl like:

curl -s clamav-api-service.clamav.svc.cluster.local:3000

I get back:

{"success":false,"data":{"error":"Not allowed."}}

Should't that give me the version? I mean the API is up and running.

clamav-rest-api/src/routes/version.js

Line 28 in e107592

router.route('/').get(async (req, res, next) => {

Thanks
Martin

"is_infected":null returned on any request, even if virus is found

Hi folks,

I would like to mention, that issue #45 doesn't seem to be resolved.
Today I pulled clamav:latest and clamav-rest-api:latest from dockerhub and started both.

$ docker run --detach --publish 8080:8080 \
--name rest-api \
-e NODE_ENV=production \
-e APP_FORM_KEY=files \
-e APP_MAX_FILE_SIZE=1073741824 \
-e APP_PORT=8080 \
-e CLAMD_IP=192.168.4.54 \
benzino77/clamav-rest-api

ce995ce2b7d043d84864dde83ceba71f6704254cefd5efdbb8dc2f09ba997d19

When firing the curl request with three eicar files it returns "is_infected":null everytime:

$ curl -X POST http://localhost:8080/api/v1/scan -F [email protected] -F [email protected] -F files=@eicarcom2/eicar_com.zip
{"success":true,"data":{"result":[{"name":"eicarcom2.zip","is_infected":null,"viruses":[]},{"name":"eicarcom2.zip","is_infected":null,"viruses":[]},{"name":"eicar_com.zip","is_infected":null,"viruses":[]}]}}

but clamav is detecting it right (logentry):
Mon Aug 7 12:38:37 2023 -> instream(172.17.0.1@33300): Win.Test.EICAR_HDB-1 FOUND

[Question] Could I use this API to scan another pod?

Hello, would it be possible to scan another pod in the cluster through this API?

Like for example with the C# Kubernetes client

a few suggestions and fixes

tried to push a branch, for you to evaluate, no permissions - so ill post it here:

dockerfile
i've succesfully managed to update and run the rest api image under node:19, add below to dockerfile

FROM node:19.8.1-buster-slim

you should strongly consider using the official npm installer for idempotent images, replace your

npm install --production

with:

npm ci --only=prodution

note: why - here https://docs.npmjs.com/cli/v9/commands/npm-ci
note: this will also fail on the current node version, so do upgrade to the latest node:19

reason to why you had to rollback the previous commit/image
if you attempt to run docker compose without the proper env variable APP_MAX_FILE_SIZE set, clamd will response with garbage.
This is also why it was working when running it "locally"

example, if you enable node-clam debug flag and run without this env variable set:

node-clam: Socket/Host connection closed.
examples-api-1    | node-clam: Provided stream is readable.
examples-api-1    | node-clam: Attempting to establish socket/TCP connection for "scanStream"
examples-api-1    | node-clam: using remote server: 172.25.0.2:3310
examples-api-1    | node-clam: Received final data from stream.
examples-api-1    | node-clam: The input stream has dried up.
examples-api-1    | node-clam: Received output from ClamAV Socket.
examples-api-1    | node-clam: ClamAV is done scanning.
examples-api-1    | node-clam: Raw Response:  UNKNOWN COMMAND
examples-api-1    |
examples-api-1    | node-clam: Error Response:  UNKNOWN COMMAND
examples-api-1    | node-clam: File may be INFECTED!

fix, add to docker-compose.yml, (it already exists in your .env.example):

   - APP_MAX_FILE_SIZE=26214400

in any case - if this config is ommitted - it will yield a 'null'-result returned from the scanner endpoint

updated packages
you can freely bump the following packages, as it will build and run:

axios to 1.3.4
express-fileupload to 1.4.0

Limited File Types

""is_infected": null" is for almost all types of file except text and gif. Am i missing any configuration

Testing Endpoints /version works /scan doesn't

Running inside of Kubernetes. The pods are running and I'm able to access the endpoint to retrieve the version but I'm unable scan a file.

Working

http --form GET http://clamav-rest-service.default.svc.cluster.local:3000/api/v1/version
HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Connection: keep-alive
Content-Length: 83
Content-Type: application/json; charset=utf-8
Date: Sun, 16 Jun 2024 23:17:38 GMT
ETag: W/"53-ih6cCd8a32HdUVUvezFSoGXOtCE"
Keep-Alive: timeout=5
X-Powered-By: Express

{
"data": {
"version": "ClamAV 1.3.1/27308/Sun Jun 16 08:28:55 2024\n"
},
"success": true
}

Not working

http --form POST http://clamav-rest-service.default.svc.cluster.local:3000/api/v1/scan FILES@/tmp/testfile.txt
HTTP/1.1 400 Bad Request
Access-Control-Allow-Origin: *
Connection: keep-alive
Content-Length: 79
Content-Type: application/json; charset=utf-8
Date: Sun, 16 Jun 2024 23:21:47 GMT
ETag: W/"4f-PIl2ikdI5Iaf4TJBR0V5ATKtLss"
Keep-Alive: timeout=5
X-Powered-By: Express

{
"data": {
"error": "The request should have only undefined key"
},
"success": false
}

cat /tmp/testfile.txt
This is a test file for ClamAV scanning.

env from clamav pod

$ env
KUBERNETES_PORT=tcp://10.96.0.1:443
KUBERNETES_SERVICE_PORT=443
CLAMAV_REST_SERVICE_SERVICE_PORT=3000
CLAMAV_REST_SERVICE_PORT=tcp://10.108.68.111:3000
CLAMAV_SERVICE_PORT_3310_TCP_ADDR=10.103.127.244
API_GATEWAY_SERVICE_PORT=80
API_GATEWAY_PORT=tcp://10.106.133.31:80
NODE_VERSION=18.20.3
HOSTNAME=clamav-rest-6fd98596c6-gmlmk
CLAMAV_SERVICE_PORT_3310_TCP_PORT=3310
YARN_VERSION=1.22.19
CLAMD_HOST=clamav-service
CLAMAV_SERVICE_PORT_3310_TCP_PROTO=tcp
HOME=/home/node
OLDPWD=/
API_GATEWAY_PORT_80_TCP_ADDR=10.106.133.31
CLAMAV_SERVICE_SERVICE_HOST=10.103.127.244
API_GATEWAY_PORT_80_TCP_PORT=80
CLAMD_PORT=3310
API_GATEWAY_PORT_80_TCP_PROTO=tcp
CLAMAV_SERVICE_PORT_3310_TCP=tcp://10.103.127.244:3310
CLAMAV_SERVICE_PORT=tcp://10.103.127.244:3310
CLAMAV_SERVICE_SERVICE_PORT=3310
TERM=xterm
KUBERNETES_PORT_443_TCP_ADDR=10.96.0.1
CLAMAV_REST_SERVICE_PORT_3000_TCP_ADDR=10.108.68.111
API_GATEWAY_PORT_80_TCP=tcp://10.106.133.31:80
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
KUBERNETES_PORT_443_TCP_PORT=443
CLAMAV_REST_SERVICE_PORT_3000_TCP_PORT=3000
KUBERNETES_PORT_443_TCP_PROTO=tcp
CLAMAV_REST_SERVICE_PORT_3000_TCP_PROTO=tcp
KUBERNETES_SERVICE_PORT_HTTPS=443
KUBERNETES_PORT_443_TCP=tcp://10.96.0.1:443
CLAMAV_REST_SERVICE_PORT_3000_TCP=tcp://10.108.68.111:3000
CLAMAV_REST_SERVICE_SERVICE_HOST=10.108.68.111
KUBERNETES_SERVICE_HOST=10.96.0.1
PWD=/tmp
API_GATEWAY_SERVICE_HOST=10.106.133.31

[Feature] Check if the database signature its up to update in the version endpoint

{"success":true,"data":{"version":"ClamAV 0.104.3/26598/Mon Jul 11 07:56:26 2022\n", updated:true}}
``

Typo on the readme

The following is missing a "" after "FILES"

docker run -d -p 8080:8080 \
-e NODE_ENV=production \
-e APP_PORT=8080 \
-e APP_FORM_KEY=FILES
-e CLAMD_IP=192.168.10.10 \
benzino77/clamav-rest-api

TypeError: req.files.hasOwnProperty is not a function

Environment : NodeJS v16.18.1 on CentOS 7

We are getting the error when postng a file. Any idea how to fix this ?

/home/xvision/node/clamav-rest-api/src/utils/checkParams.js:20
const hasProperKey = req.files.hasOwnProperty(process.env.APP_FORM_KEY);

TypeError: req.files.hasOwnProperty is not a function
at checkParams (/home/xvision/node/clamav-rest-api/src/utils/checkParams.js:20:34)
at /home/xvision/node/clamav-rest-api/src/routes/scan.js:10:24
at Layer.handle [as handle_request] (/home/xvision/node/clamav-rest-api/node_modules/express/lib/router/layer.js:95:5)
at next (/home/xvision/node/clamav-rest-api/node_modules/express/lib/router/route.js:144:13)
at Route.dispatch (/home/xvision/node/clamav-rest-api/node_modules/express/lib/router/route.js:114:3)
at Layer.handle [as handle_request] (/home/xvision/node/clamav-rest-api/node_modules/express/lib/router/layer.js:95:5)
at /home/xvision/node/clamav-rest-api/node_modules/express/lib/router/index.js:284:15
at Function.process_params (/home/xvision/node/clamav-rest-api/node_modules/express/lib/router/index.js:346:12)
at next (/home/xvision/node/clamav-rest-api/node_modules/express/lib/router/index.js:280:10)
at Function.handle (/home/xvision/node/clamav-rest-api/node_modules/express/lib/router/index.js:175:3)

benzino77 / clamav-rest-api Goto Github PK

clamav-rest-api's People

Contributors

Stargazers

Watchers

Forkers

clamav-rest-api's Issues

Working

Not working

env from clamav pod

Recommend Projects

Recommend Topics

Recommend Org