webraybtl / codeqlpy Goto Github PK
View Code? Open in Web Editor NEWCodeQLpy是一款基于CodeQL实现的半自动化代码审计工具,目前仅支持java语言。实现从源码反编译,数据库生成,脆弱性发现的全过程,可以辅助代码审计人员快速定位源码可能存在的漏洞。
CodeQLpy是一款基于CodeQL实现的半自动化代码审计工具,目前仅支持java语言。实现从源码反编译,数据库生成,脆弱性发现的全过程,可以辅助代码审计人员快速定位源码可能存在的漏洞。
像ruoyi这种打包好的jar包,lib下面还是有jar包是网站代码
Initializing database at /Users/code/python/CodeQLpy/out/database/XXEDemo.
Running build command: [/bin/bash, -c, /Users/code/python/CodeQLpy/out/decode/run.sh]
[2023-01-30 16:36:17] [build-stdout] [INFO] Scanning for projects...
[2023-01-30 16:36:18] [build-stdout] [INFO]
[2023-01-30 16:36:18] [build-stdout] [INFO] ---------------------< io.github.talelin:latticy >----------------------
[2023-01-30 16:36:18] [build-stdout] [INFO] Building latticy 0.2.1-RELEASE
[2023-01-30 16:36:18] [build-stdout] [INFO] --------------------------------[ jar ]---------------------------------
[2023-01-30 16:36:19] [build-stdout] [INFO]
[2023-01-30 16:36:19] [build-stdout] [INFO] --- maven-clean-plugin:3.1.0:clean (default-clean) @ latticy ---
[2023-01-30 16:36:19] [build-stdout] [INFO] Deleting /Users/code/python/CodeQLpy/target
[2023-01-30 16:36:19] [build-stdout] [INFO]
[2023-01-30 16:36:19] [build-stdout] [INFO] --- maven-resources-plugin:3.2.0:resources (default-resources) @ latticy ---
[2023-01-30 16:36:19] [build-stdout] [INFO] Using 'UTF-8' encoding to copy filtered resources.
[2023-01-30 16:36:19] [build-stdout] [INFO] Using 'UTF-8' encoding to copy filtered properties files.
[2023-01-30 16:36:19] [build-stdout] [INFO] skip non existing resourceDirectory /Users/code/python/CodeQLpy/src/main/resources
[2023-01-30 16:36:19] [build-stdout] [INFO] skip non existing resourceDirectory /Users/code/python/CodeQLpy/src/main/java
[2023-01-30 16:36:19] [build-stdout] [INFO]
[2023-01-30 16:36:19] [build-stdout] [INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ latticy ---
[2023-01-30 16:36:20] [build-stdout] [INFO] No sources to compile
[2023-01-30 16:36:20] [build-stdout] [INFO]
[2023-01-30 16:36:20] [build-stdout] [INFO] --- maven-resources-plugin:3.2.0:testResources (default-testResources) @ latticy ---
[2023-01-30 16:36:20] [build-stdout] [INFO] Using 'UTF-8' encoding to copy filtered resources.
[2023-01-30 16:36:20] [build-stdout] [INFO] Using 'UTF-8' encoding to copy filtered properties files.
[2023-01-30 16:36:20] [build-stdout] [INFO] skip non existing resourceDirectory /Users/code/python/CodeQLpy/src/test/resources
[2023-01-30 16:36:20] [build-stdout] [INFO]
[2023-01-30 16:36:20] [build-stdout] [INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ latticy ---
[2023-01-30 16:36:20] [build-stdout] [INFO] No sources to compile
[2023-01-30 16:36:20] [build-stdout] [INFO]
[2023-01-30 16:36:20] [build-stdout] [INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ latticy ---
[2023-01-30 16:36:20] [build-stdout] [INFO] Tests are skipped.
[2023-01-30 16:36:20] [build-stdout] [INFO]
[2023-01-30 16:36:20] [build-stdout] [INFO] --- maven-jar-plugin:3.2.0:jar (default-jar) @ latticy ---
[2023-01-30 16:36:20] [build-stdout] [WARNING] JAR will be empty - no content was marked for inclusion!
[2023-01-30 16:36:20] [build-stdout] [INFO] Building jar: /Users/code/python/CodeQLpy/target/latticy-0.2.1-RELEASE.jar
[2023-01-30 16:36:20] [build-stdout] [INFO]
[2023-01-30 16:36:20] [build-stdout] [INFO] --- spring-boot-maven-plugin:2.5.2:repackage (repackage) @ latticy ---
[2023-01-30 16:36:21] [build-stdout] [INFO] ------------------------------------------------------------------------
[2023-01-30 16:36:21] [build-stdout] [INFO] BUILD FAILURE
[2023-01-30 16:36:21] [build-stdout] [INFO] ------------------------------------------------------------------------
[2023-01-30 16:36:21] [build-stdout] [INFO] Total time: 3.989 s
[2023-01-30 16:36:21] [build-stdout] [INFO] Finished at: 2023-01-30T16:36:21+08:00
[2023-01-30 16:36:21] [build-stdout] [INFO] ------------------------------------------------------------------------
[2023-01-30 16:36:21] [build-stdout] [ERROR] Failed to execute goal org.springframework.boot:spring-boot-maven-plugin:2.5.2:repackage (repackage) on project latticy: Execution repackage of goal org.springframework.boot:spring-boot-maven-plugin:2.5.2:repackage failed: Unable to find main class -> [Help 1]
[2023-01-30 16:36:21] [build-stdout] [ERROR]
[2023-01-30 16:36:21] [build-stdout] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[2023-01-30 16:36:21] [build-stdout] [ERROR] Re-run Maven using the -X switch to enable full debug logging.
[2023-01-30 16:36:21] [build-stdout] [ERROR]
[2023-01-30 16:36:21] [build-stdout] [ERROR] For more information about the errors and possible solutions, please read the following articles:
[2023-01-30 16:36:21] [build-stdout] [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
[2023-01-30 16:36:22] [build-stdout] Eclipse Compiler for Java(TM) v20160829-0950, 3.12.1
在config.ini中配置qlpath后,是不是程序会同时使用ql 库中的插件和程序目录plugins中的插件进行扫描。
扫个项目等待的时间太长了,能否把结果实时写入到文件
(windows写入的时候打开貌似会占用导致报错,linux和mac不会)
你好,我在大项目中使用,出现了如下问题
database.py INFO Found 2047 jsp files to decode
后面就一直 functions.py WARNING
最终结果就是
database.py WARNING Decode jsp file 0/2047 success
database.py ERROR Auto decompiler error, no java file found.
请问这个错误要怎么解决?
作者您好!感谢您卓越的工作!
我查看了插件目录中的部分 java 插件,发现与 ql 库中的插件不完全一致,请问您是从哪里收集到的这些插件?插件的作用经过您的验证了吗?
FileNotFoundError: [WinError 3] 系统找不到指定的路径。: '/Users/pang0lin/CodeQL/ql/java/ql/test/tmpbstpho93'
PS E:\codeql\CodeQLpy> codeql database create out/database/webroot --language=java --command="E:\codeql\CodeQLpy\out\decode/run.cmd" --overwrite
Initializing database at E:\codeql\CodeQLpy\out\database\webroot.
Running build command: [E:\codeql\CodeQLpy\out\decode/run.cmd]
[2023-03-14 19:10:13] [build-stdout] offjava -jar E:\codeql\CodeQLpy\lib\ecj-4.6.1.jar -extdirs "E:\codeql\CodeQLpy\out\decode\lib;lib/spring_mvc_lib;lib/common_lib;lib/java8_lib;lib/tomcat_lib" -sourcepath E:\codeql\CodeQLpy\out\decode\classes -encoding UTF-8 -8 -warn:none -proceedOnError -noExit @E:\codeql\CodeQLpy\out\decode/file.txt
Finalizing database at E:\codeql\CodeQLpy\out\database\webroot.
CodeQL detected code written in Java but could not process any of it. This can occur if the specified build commands failed to compile or process any code.
- Confirm that there is some source code for the specified language in the project.
- For codebases written in Go, JavaScript, TypeScript, and Python, do not specify
an explicit --command.
- For other languages, the --command must specify a "clean" build which compiles
all the source code files without reusing existing build artefacts.
纯class文件或者java文件跑不起来,得需要web.xml? 或者是poc.xml,建议加个判断
插件的里的ql,如何包含自己编写的qll库?
因为想查看具体的每一步的节点
看到很多关于windows得bug修复,不知道Mac能否正常兼容
2023/03/30 10:46:44 functions.py WARNING
2023/03/30 10:46:44 functions.py WARNING
2023/03/30 10:46:46 functions.py WARNING A fatal error occurred: Error reading /Users/xxxxx/tools/codeqldemo/ql/java/ql/test/qlpack.yml: Unexpected '$'.
${workspace}
^ (through reference chain: com.semmle.frontend.packs.QlPack["dependencies"]->java.util.TreeMap["codeql/java-all"])
2023/03/30 10:46:46 check.py ERROR database or codeql is error.
qlpath error,check it at config/config.ini or close the Visual Studio Code.
WEB-INF/lib为jar包,关键代码都在jar里面,jsp文件只有几个,没啥用主要文件都在几个lib文件夹的jar里,我该怎么执行命令呢?
python main.py -t C:\Users\john\Desktop\oa\ -c -j oa-*?.jar 失败了
错误信息:
`2023/09/25 17:02:43 database.py WARNING Decode jsp file 0/11 success
2023/09/25 17:02:43 database.py ERROR Auto decompiler error, no java file found.
`
2023/02/16 12:44:02 database.py ERROR Target SourceCode doesn't have WEB-INF/web.xml file,exit
2023/01/22 00:19:06 database.py ERROR Target SourceCode doesn't found any jsp fi
le,exit
Could not resolve module java. There should probably be a qlpack.yml file declaring dependencies
太好用了!!希望支持codeql所支持的其他语言
2023/01/20 00:47:40 functions.py WARNING
2023/01/20 00:47:40 functions.py WARNING
Traceback (most recent call last):
File "/Users/xxx/CodeQL/CodeQLpy/main.py", line 71, in
createDB(parse_args.target, parse_args.compiled, version, parse_args.jar)
File "/Users/xxx/CodeQL/CodeQLpy/compiler/database.py", line 402, in createDB
return createDir(source, compiled, version, jars)
File "/Users/xxxx/CodeQL/CodeQLpy/compiler/database.py", line 139, in createDir
compile_cmd = ecjcompile(qlConfig("decode_savedir"), source)
File "/Users/xxx/CodeQL/CodeQLpy/compiler/ecjcompiler.py", line 98, in ecjcompile
jar_libs = dirFiles(os.path.join(save_path, "lib"))
File "/Users/xxx/CodeQL/CodeQLpy/utils/functions.py", line 25, in dirFiles
for filename in os.listdir(dirpath):
FileNotFoundError: [Errno 2] No such file or directory: '/Users/xxx/CodeQL/CodeQLpy/out/decode/lib'
python3 main.py -t ***.jar -c
decoding e.jar...
Command '/Library/Java/JavaVirtualMachines/jdk-11.0.15.jdk/Contents/Home/bin/java -jar lib/jd-cli.jar --outputDir out/decode/xboot-module.jar /.jar' timed out after 240 seconds
Traceback (most recent call last):
File "/Users//Security/Web/CodeQLpy/main.py", line 75, in
createDB(parse_args.target, parse_args.compiled, version, parse_args.jar, parse_args.root)
File "/Users//Security/Web/CodeQLpy/compiler/database.py", line 408, in createDB
return createJar(source, compiled, version)
File "/Users//Security/Web/CodeQLpy/compiler/database.py", line 46, in createJar
copyJavaFile(java_file, os.path.join(qlConfig("decode_savedir"), "classes"))
File "/Users//Security/Web/CodeQLpy/utils/functions.py", line 77, in copyJavaFile
with open(srcpath, 'rb') as r:
IsADirectoryError: [Errno 21] Is a directory: 'out/decode/**.jar/META-INF/maven/com.github.spullara.mustache.java'
Traceback (most recent call last):
File "C:\code\CodeQLpy\main.py", line 71, in
createDB(parse_args.target, parse_args.compiled, version, parse_args.jar)
File "C:\code\CodeQLpy\compiler\database.py", line 398, in createDB
return createDir(source, compiled, version, jars)
File "C:\code\CodeQLpy\compiler\database.py", line 139, in createDir
compile_cmd = ecjcompile(qlConfig("decode_savedir"), source)
File "C:\code\CodeQLpy\compiler\ecjcompiler.py", line 103, in ecjcompile
source_java_path = getSourcePath(source_path)
File "C:\code\CodeQLpy\compiler\ecjcompiler.py", line 78, in getSourcePath
pack_loc = java_file.index(packname.replace(b".", b"/").decode("utf-8"))
ValueError: substring not found
UnicodeDecodeError: 'gbk' codec can't decode byte 0xae in position 38: illegal multibyte sequence
utils/option.py
line8替换成:
conn.read("config/config.ini",encoding="utf-8")
qlpath是什么路径呢?是指存放了codeql.exe 的目录吗?
有一些项目是war或者jar包,但是里面没有web.xml,进行第一步时会报找不到web.xml,这种怎么办
所有漏洞都跑一遍时间太长了,能不能单独设置跑哪些漏洞
python3 main.py -d /Users/xxxxxxxx/tools/CodeQL/CodeQLpy/out/database/ruoyi_demo
Welcome to
.. _________ . ________ .
_ __ | | _ ___ \ ____ | /_ _____ \ | |
| // | | / \ / / _ \ / __ |/ __ \ / / \ | |
| | _ | \ _( <> ) // \ // _/. \ |
|| / | _ /_/_ |__ >_____\ _/_______ / / / / __> /
2023/03/29 14:57:26 functions.py WARNING
2023/03/29 14:57:26 functions.py WARNING
2023/03/29 14:57:28 functions.py WARNING Compiling query plan for /Users/xxxxxx/tools/CodeQL/tmpjfw8i5b8/4a24a317f6b24508ac45368607e75799.ql.
ERROR: Could not resolve module java. There should probably be a qlpack.yml file declaring dependencies in /Users/xxxxxx/tools/CodeQL/tmpjfw8i5b8 or an enclosing directory. (/Users/xxxxxxx/tools/CodeQL/tmpjfw8i5b8/4a24a317f6b24508ac45368607e75799.ql:1,8-12)
2023/03/29 14:57:28 check.py ERROR database or codeql is error.
qlpath error,check it at config/config.ini or close the Visual Studio Code.
ecj使用@符号读取生成的file.txt(由于项目太大)作为参数展开 导致传递给codeql创建数据库会触发标题错误
-- Initializing database at /home/john/Desktop/codeql_analysis/out/databases/seeyon.
Running build command: [/usr/lib/jvm/java-8-openjdk-amd64/bin/java, -jar, /home/john/Desktop/codeql_analysis/plugins/ecj-4.20.jar, -extdirs, /home/john/Desktop/codeql_analysis/tmp/decompiler/dir/seeyon/lib:/home/john/Desktop/codeql_analysis/plugins/java8_lib, -sourcepath, /home/john/Desktop/codeql_analysis/tmp/decompiler/dir/seeyon/classes, -encoding, UTF-8, -8, -warn:none, -proceedOnError, -noExit, @/home/john/Desktop/codeql_analysis/tmp/decompiler/dir/seeyon/file.txt]
A fatal error occurred: IOException while executing process with args: [/home/john/CodeQL/codeql/tools/linux64/preload_tracer, /usr/lib/jvm/java-8-openjdk-amd64/bin/java, -jar, /home/john/Desktop/codeql_analysis/plugins/ecj-4.20.jar, -extdirs, /home/john/Desktop/codeql_analysis/tmp/decompiler/dir/seeyon/lib:/home/john/Desktop/codeql_analysis/plugins/java8_lib, -sourcepath, /home/john/Desktop/codeql_analysis/tmp/decompiler/dir/seeyon/classes, -encoding, UTF-8, -8, -warn:none, -proceedOnError, -noExit, /home/john/Desktop/codeql_analysis/tmp/decompiler/dir/seeyon/classes/code2/www/seeyon/com/system/auth/SeeyonDog.java, /home/john/Desktop/codeql_analysis/tmp/decompiler/dir/seeyon/classes/com/alibaba/fastjson/IOUtils.java, /home/john/Desktop/codeql_analysis/tmp/decompiler/dir/seeyon/classes/com/alibaba/fastjson/JSONArray.java, /home/john/Desktop/codeql_analysis/tmp/decompiler/dir/seeyon/classes/com/alibaba/fastjson/JSONException.java, /home/john/Desktop/codeql_analysis/tmp/decompiler/dir/seeyon/classes/com/alibaba/fastjson/JSONObject.java, /home/john/Desktop/codeql_analysis/tmp/decompiler/dir/seeyon/classes/com/alibaba/fastjson/TypeReference.java, /home/john/Desktop/codeql_analysis/tmp/decompiler/dir/seeyon/classes/com/alibaba/fastjson/TypeUtils.java, ...]
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.