Giter VIP home page Giter VIP logo

Comments (8)

cyw3 avatar cyw3 commented on May 2, 2024

麻烦更新下CodeAnalysis代码库，执行./compose_init.sh，然后重试下

from codeanalysis.

wushifeng avatar wushifeng commented on May 2, 2024

更新后的确可以运行的，不再提示Cobra的问题
不过运行一直不结束，无法看结果呢？

2022-01-13 08:36:48,356-INFO-util.logutil: Task_36 is done.
2022-01-13 08:36:48,718-INFO-util.logutil: Task_37 (PMD) starts ...
2022-01-13 08:36:58,830-INFO-util.logutil: Task_37 is done.
2022-01-13 08:36:59,199-INFO-util.logutil: Task_38 (Cpd) starts ...
2022-01-13 08:37:09,312-INFO-util.logutil: Task_38 is done.
2022-01-13 08:37:09,682-INFO-util.logutil: Task_39 (Lizard) starts ...
2022-01-13 08:37:19,795-INFO-util.logutil: Task_39 is done.
2022-01-13 08:37:20,161-INFO-util.logutil: Task_40 (CodeCount) starts ...
2022-01-13 08:37:30,267-INFO-util.logutil: Task_40 is done.
2022-01-13 08:37:31,130-INFO-util.logutil: Get code line data...
2022-01-13 08:37:31,131-INFO-util.logutil: ==================================================
2022-01-13 08:37:31,131-INFO-util.logutil: 本次扫描的代码量:
2022-01-13 08:37:31,131-INFO-util.logutil: code_line_num : 68570
2022-01-13 08:37:31,131-INFO-util.logutil: comment_line_num : 10108
2022-01-13 08:37:31,131-INFO-util.logutil: blank_line_num : 8472
2022-01-13 08:37:31,131-INFO-util.logutil: total_line_num : 87150
2022-01-13 08:37:31,131-INFO-util.logutil: filtered_code_line_num : 56048
2022-01-13 08:37:31,131-INFO-util.logutil: filtered_comment_line_num : 9458
2022-01-13 08:37:31,131-INFO-util.logutil: filtered_blank_line_num : 7525
2022-01-13 08:37:31,132-INFO-util.logutil: filtered_total_line_num : 73031
2022-01-13 08:37:31,132-INFO-util.logutil: filtered_lang_count : {'XML': {'file_num': 17, 'code_line_num': 1847, 'comment_line_num': 251, 'blank_line_num': 178, 'total_line_num': 2276}, 'TypeScript': {'file_num': 20, 'code_line_num': 2454, 'comment_line_num': 99, 'blank_line_num': 178, 'total_line_num': 2731}, 'Shell': {'file_num': 10, 'code_line_num': 112, 'comment_line_num': 30, 'blank_line_num': 39, 'total_line_num': 181}, 'SQL': {'file_num': 1, 'code_line_num': 14, 'comment_line_num': 13, 'blank_line_num': 6, 'total_line_num': 33}, 'gitignore': {'file_num': 5, 'code_line_num': 79, 'comment_line_num': 4, 'blank_line_num': 3, 'total_line_num': 86}, 'JavaServer Pages': {'file_num': 21, 'code_line_num': 942, 'comment_line_num': 4, 'blank_line_num': 110, 'total_line_num': 1056}, 'JavaScript': {'file_num': 54, 'code_line_num': 28019, 'comment_line_num': 3057, 'blank_line_num': 2099, 'total_line_num': 33175}, 'Plain Text': {'file_num': 2, 'code_line_num': 7, 'comment_line_num': 0, 'blank_line_num': 6, 'total_line_num': 13}, 'Java': {'file_num': 49, 'code_line_num': 2281, 'comment_line_num': 549, 'blank_line_num': 455, 'total_line_num': 3285}, 'TypeScript Typings': {'file_num': 19, 'code_line_num': 2689, 'comment_line_num': 4016, 'blank_line_num': 1454, 'total_line_num': 8159}, 'Markdown': {'file_num': 1, 'code_line_num': 27, 'comment_line_num': 0, 'blank_line_num': 4, 'total_line_num': 31}, 'JSON': {'file_num': 6, 'code_line_num': 1284, 'comment_line_num': 0, 'blank_line_num': 1, 'total_line_num': 1285}, 'CSS': {'file_num': 34, 'code_line_num': 16218, 'comment_line_num': 1418, 'blank_line_num': 2981, 'total_line_num': 20617}, 'HTML': {'file_num': 1, 'code_line_num': 59, 'comment_line_num': 0, 'blank_line_num': 11, 'total_line_num': 70}, 'Properties File': {'file_num': 5, 'code_line_num': 16, 'comment_line_num': 17, 'blank_line_num': 0, 'total_line_num': 33}}
2022-01-13 08:37:31,132-INFO-util.logutil: ==================================================
2022-01-13 08:37:32,679-INFO-util.logutil: query result from server ...

卡在最后一句，一直没有结果

config.ini配置

[COMMON]
TOOL_CONFIG_URL=https://hub.fastgit.org/TCATools/puppy-tools-config.git
PASSWORD_KEY=a6x4c7esudcv396w
DEBUG=
USE_LOCAL_TOOL=True

[SERVER_URL]
URL=http://localhost/server/main/

[FILE_SERVER]
URL=http://localhost/server/files/

from codeanalysis.

Lingghh avatar Lingghh commented on May 2, 2024

麻烦发一下analysis-worker的执行日志哈

docker-compose exec analysis-worker /bin/bash

tail -F log/codedog.log

from codeanalysis.

wushifeng avatar wushifeng commented on May 2, 2024

codedog.log中日志
-2022-01-12 18:26:22,925-DEBUG-asyncio: Using selector: EpollSelector
-2022-01-13 08:21:10,873-DEBUG-asyncio: Using selector: EpollSelector

root@591fe31dd2ae:/var/www/django/codedog/log# tail -5 celery.log 日志
-2022-01-13 10:52:53,895-WARNING-util.retrylib: method <download_file> exception: [116] download file exception: HTTPConnectionPool(host='localhost', port=80): Max retries exceeded with url: /server/files/taskdata/projects/1/task_56/e6cab0ee741a11ecb7286651e7c0f790/tmp_dir.zip (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f72cec04110>: Failed to establish a new connection: [Errno 111] Connection refused'))
-2022-01-13 10:52:53,895-INFO-util.retrylib: retrying method <download_file> on 3 time
-2022-01-13 10:52:58,901-INFO-util.fileserver: Downloading file from http://localhost/server/files/taskdata/projects/1/task_56/e6cab0ee741a11ecb7286651e7c0f790/tmp_dir.zip to /var/www/django/codedog/tempdata/0a0f404ffcbd5c89ce0ae35d11b6983a27f391da/tmp_dir.zip
-2022-01-13 10:52:58,901-WARNING-urllib3.connectionpool: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f72ceb60390>: Failed to establish a new connection: [Errno 111] Connection refused')': /server/files/taskdata/projects/1/task_56/e6cab0ee741a11ecb7286651e7c0f790/tmp_dir.zip
-2022-01-13 10:53:02,904-WARNING-urllib3.connectionpool: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f72cec04d90>: Failed to establish a new connection: [Errno 111] Connection refused')': /server/files/taskdata/projects/1/task_56/e6cab0ee741a11ecb7286651e7c0f790/tmp_dir.zip

  • docker-compose up的server日志截取
analysis-server_1  | [13/Jan/2022:10:50:17 +0800] <47> 172.18.0.11 'GET /api/projects/1/scans/12/ HTTP/1.1' 200 0.016400 603 -' 'python-urllib3/1.26.7'
main-server_1      | -2022-01-13 10:50:17,746-DEBUG-urllib3.connectionpool: http://analysis-server:8002 "GET /api/projects/1/scans/12/ HTTP/1.1" 200 603
main-server_1      | [13/Jan/2022:10:50:17 +0800] <53> 172.18.0.12 'GET /api/orgs/4dXNupGrLPa/teams/demo/repos/1/projects/1/scans/12/ HTTP/1.0' 200 0.050256 603 -' 'Python-urllib/3.7'
analysis-worker_1  | [2022-01-13 10:50:20,425: WARNING/ForkPoolWorker-2] Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f72cebb8b10>: Failed to establish a new connection: [Errno 111] Connection refused')': /server/files/taskdata/projects/1/task_56/e6cab0ee741a11ecb7286651e7c0f790/tmp_dir.zip
analysis-worker_1  | -2022-01-13 10:50:20,425-WARNING-urllib3.connectionpool: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f72cebb8b10>: Failed to establish a new connection: [Errno 111] Connection refused')': /server/files/taskdata/projects/1/task_56/e6cab0ee741a11ecb7286651e7c0f790/tmp_dir.zip
analysis-worker_1  | [2022-01-13 10:50:20,426: WARNING/ForkPoolWorker-2] method <download_file> exception: [116] download file exception: HTTPConnectionPool(host='localhost', port=80): Max retries exceeded with url: /server/files/taskdata/projects/1/task_56/e6cab0ee741a11ecb7286651e7c0f790/tmp_dir.zip (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f72ceb4c290>: Failed to establish a new connection: [Errno 111] Connection refused'))
analysis-worker_1  | -2022-01-13 10:50:20,426-WARNING-util.retrylib: method <download_file> exception: [116] download file exception: HTTPConnectionPool(host='localhost', port=80): Max retries exceeded with url: /server/files/taskdata/projects/1/task_56/e6cab0ee741a11ecb7286651e7c0f790/tmp_dir.zip (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f72ceb4c290>: Failed to establish a new connection: [Errno 111] Connection refused'))
analysis-worker_1  | [2022-01-13 10:50:20,426: INFO/ForkPoolWorker-2] retrying method <download_file> on 2 time
analysis-worker_1  | -2022-01-13 10:50:20,426-INFO-util.retrylib: retrying method <download_file> on 2 time
main-server_1      | [13/Jan/2022:10:50:22 +0800] <53> 172.18.0.12 'POST /api/jobs/12/tasksbeat/ HTTP/1.0' 200 0.023438 82 -' 'Python-urllib/3.7'
analysis-worker_1  | [2022-01-13 10:50:25,432: INFO/ForkPoolWorker-2] Downloading file from http://localhost/server/files/taskdata/projects/1/task_56/e6cab0ee741a11ecb7286651e7c0f790/tmp_dir.zip to /var/www/django/codedog/tempdata/0a0f404ffcbd5c89ce0ae35d11b6983a27f391da/tmp_dir.zip
analysis-worker_1  | -2022-01-13 10:50:25,432-INFO-util.fileserver: Downloading file from http://localhost/server/files/taskdata/projects/1/task_56/e6cab0ee741a11ecb7286651e7c0f790/tmp_dir.zip to /var/www/django/codedog/tempdata/0a0f404ffcbd5c89ce0ae35d11b6983a27f391da/tmp_dir.zip
analysis-worker_1  | [2022-01-13 10:50:25,434: WARNING/ForkPoolWorker-2] Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f72cebb8410>: Failed to establish a new connection: [Errno 111] Connection refused')': /server/files/taskdata/projects/1/task_56/e6cab0ee741a11ecb7286651e7c0f790/tmp_dir.zip
analysis-worker_1  | -2022-01-13 10:50:25,434-WARNING-urllib3.connectionpool: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f72cebb8410>: Failed to establish a new connection: [Errno 111] Connection refused')': /server/files/taskdata/projects/1/task_56/e6cab0ee741a11ecb7286651e7c0f790/tmp_dir.zip
analysis-worker_1  | [2022-01-13 10:50:29,442: WARNING/ForkPoolWorker-2] Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f72ceb4c510>: Failed to establish a new connection: [Errno 111] Connection refused')': /server/files/taskdata/projects/1/task_56/e6cab0ee741a11ecb7286651e7c0f790/tmp_dir.zip
analysis-worker_1  | -2022-01-13 10:50:29,442-WARNING-urllib3.connectionpool: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f72ceb4c510>: Failed to establish a new connection: [Errno 111] Connection refused')': /server/files/taskdata/projects/1/task_56/e6cab0ee741a11ecb7286651e7c0f790/tmp_dir.zip

猜测分析客户端配置的是localhost，而docker-compose中analysis肯定是不能使用localhost的

from codeanalysis.

wushifeng avatar wushifeng commented on May 2, 2024

另外客户端分析工具从github下载，国内使用的体验太差了
我在gitee上建了部分java相关的分析的镜像，地址如下：

https://gitee.com/organizations/TCATools/projects

客户端的配置设置为：
[COMMON]
; [必填]工具配置库git地址
#TOOL_CONFIG_URL=https://hub.fastgit.org/TCATools/puppy-tools-config.git
TOOL_CONFIG_URL=https://gitee.com/TCATools/puppy-tools-config.git

puppy-tools-config.git这个库中修改工具地址到https://gitee.com/TCATools
至少下载工具这块好很多

from codeanalysis.

wushifeng avatar wushifeng commented on May 2, 2024

原因已查到
分析客户端写成如下地址时不行
URL=http://localhost/server/main/
URL=http://localhost/server/files/

写如下这种具体的地址后，结果就可以上报了，估计代码有bug
URL=http://172.18.0.1/server/main/
URL=http://172.18.0.1/server/files/

但检查结果，查看需要修改的源代码时，又出问题【待会单独提个问题吧】
{"status_code":400,"code":1000,"msg":"未知错误: <class 'Exception'>:{method: cat_file, error_message: error: could not lock config file /var/www/django/codedog/sourcedirs/a12a4dc1c38f18898e86c1db745fb33fdc457ce3/.git/config: No such file or directory\nfatal: could not set 'core.repositoryformatversion' to '0'}"}

from codeanalysis.

yipeikk avatar yipeikk commented on May 2, 2024

你好 这边也遇到了这个问题 但是config.ini配置是正确的server地址 还是卡在 query result这一步

image

from codeanalysis.

cyw3 avatar cyw3 commented on May 2, 2024

原因已查到 分析客户端写成如下地址时不行 URL=http://localhost/server/main/ URL=http://localhost/server/files/

写如下这种具体的地址后，结果就可以上报了，估计代码有bug URL=http://172.18.0.1/server/main/ URL=http://172.18.0.1/server/files/

但检查结果，查看需要修改的源代码时，又出问题【待会单独提个问题吧】 {"status_code":400,"code":1000,"msg":"未知错误: <class 'Exception'>:{method: cat_file, error_message: error: could not lock config file /var/www/django/codedog/sourcedirs/a12a4dc1c38f18898e86c1db745fb33fdc457ce3/.git/config: No such file or directory\nfatal: could not set 'core.repositoryformatversion' to '0'}"}

好的，麻烦单独提个issue哈

from codeanalysis.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.