networksecurity's Introduction
networksecurity's People
Forkers
alan125999networksecurity's Issues
Input Format Problem
Question about restarting service of winlogbeat & packetbeat
Hello TAs,
I managed to change fields.hostname after reading #16
When I restart services of winlogbeat and packetbeat,
Kibana create a new index pattern, yet the number of the fields seems different.
My winlogbeat.yml
#---------fields---------
fields:
hostname: _309551108
#----------kibana---------
setup.kibana:
host: "192.168.66.1:5601"
username: "admin"
username: "admin"
#------Logstash Output ----------
output.logstash:
hosts: ["192.168.66.1:5044"]
username: "admin"
password: "admin"
my first winlogbeat patterns: 679 fields
my first packetbeat patterns: 93 fields
After restarting service , new index patterns: 340 fields
(changing hostname of both winlogbeat and packet)
The Problems
-
the old winlogbeat pattern still receive some winlogbeat data(the hostname is still unknown)
-
new index patterns seems to only accept packetbeat's data(the hostname shows correctly)
My questions
My question is as follows:
- Do winlogbeat & packetbeat's data can send to the same index pattern? If so, how to distinguish them?
- Why the hostname is still unknown after changing fields in winlogbeat.yml?
- the cmd
./winlogbeat.exe setup -e
will prompt out error(like the pictures below), but cmd./winlogbeat.exe -e
or "Start-Service winlogbeat" works properly, did the system reject winlogbeat.yml after changing the hostname or did anything I miss?
Reply or discussion is appreciated, thanks!
How to read test data in efficient?
As title,
I tyied to use pd.read_json(json_file,lines=True)
, but it raise Memory error
because I need to extract key,value from the json_file
so, I used json.loads(line)
(line is one line in json_file) to read one by one, and extract the key/value I need.
it work, but I found it too slow to read(40k lines/5 mins).
Is there any better way to read?
thanks!
Question about Logstash Connection
Dear TAs,
I just finish my setting following the instruction you gave in previous issue question https://www.elastic.co/guide/en/beats/winlogbeat/7.11/winlogbeat-installation-configuration.html
Unfortunately, although I can successfully log into Kibana but there is no data related to Winlogbeat
my Kibana now looks like this
I tried to set index pattern winlogbeat-*
as the instruction but there's no such data
My winlogbeat.yml
# =================================== Kibana ===================================
setup.kibana:
# Kibana Host
host: "192.168.66.1:5601"
username: "admin"
password: "admin"
and I commented the Elasticsearch Output
#----------------------------- Logstash output --------------------------------
output.logstash:
# The Logstash hosts
hosts: ["192.168.66.1:5044"]
My question is
- Do I need to do any command on the ubuntu side or I just need to open and sign in?
- Do I need to change any config and yml file in ubuntu side?
- Since I also set the logging.files in winlogbeat.yml , I got the log and the error message is like this
I cannot connent to Logstash but I don't know why
I think I must missed something, sorry for bothering you🥺 🥺
The DEMO testcase has 5 files,too?
As title,
and will each attack appear exactly once?
Thanks!
setting group policy editor
請問要如何正確設定這次作業的group policy editor?
我感覺是在user configuration->windows settings->scripts(logon/logoff)裡輸入想要觀察的指令(新增刪除檔案 開啟關閉小算盤)
但是我仍在在event viewer與kibana上沒看到相關log
group policy editor problem
Stuck in pj2
Can classmates or TAs give me some suggestions or tips for project2?
I think I am in trouble T^T
Currently I have tried to import log into elasticsearch by using elasticdump
But I have no idea how to visualize and analyze in KIBANA
And I didn’t have any machine learning experience before
A little difficult for me
There are too many packages, but I don’t know which one is suitable for this job
(multi-class classification?)
So I came up to ask for help, hoping someone could give me directions to complete the assignment QQ
Thanks
Windows VM keeps restarting again and again
Hi TA, I would like to ask the Question of environment:
I used the VM released by TA, but it turned out that whenever I started up the Windows VM, the machine just kept lagging and restarting itself again and again. So I'm wondering if it's OK for me to install a Windows 10 VM by myself? Or there is some configuration already setted up in the one released by TA., thank you.
Winlogbeat error connecting to Kibana
Dear TAs:
What I have done
-
According to the issues, I did
port forwarding
, but I'm not sure it is correct or not.
Problems
I'm wondering that I didn't correct winlogbeat.yml as the right way, or my window just can't connect to my virtual box.
Thanks in advance. 😭
Question
Question about Winlogbeat & Packetbeat
-
When I tried to start
.\winlogbeat.exe setup -e
&.\packetbeat.exe setup -e
in Powershell (select run as Administrator), I would get an error message. ("error connecting to Elasticsearch at http://ubuntu_id:9200") I can't find new log in Kibana so I guess the reason behind is this issue. Does any suggestion for this issue?? -
I tried to use
ping
(windows->linux & linux->windows) to check and I found only linux can ping to windows. I got a research about this issue and they said Network setup of NAT that they can't ping to each other. But using the Bridge, they can ping to each other.
import json to ES
Can implement this project in another programming language other than Python?
What does this command do?
$ sudo sysctl -w vm.max_map_count=262144
I saw this command on the slide, why is this command needed, and why do we need to execute it every time after rebooting?
winlogbeat config setup
Hi TA,
I have two questions to ask:
1. I want to set up the configuration of the winlogbeat and I am on step 3, but it says "more than one configured accessing
output, what is the problem I might have met?
2. Is the elastic.zip file need to be launched on the Ubuntu VM? If so, I am not sure how to install the docker first before
docker-compose up
Question on Kibana
About event action
I have a problem about different format of event action!
First, it appear like this
Then, after I reboot both Windows and Ubuntu, it somehow change to this
However, there's no different between the code and setting.
Furthermore, some fields are't available now! e.g. process.name ,that's really bothering.
Any idea about this differences? Appreciate any response!
Hints for the project
import data problem
what the meaning of got permission denied when I check docker version
Question about Environment & Tools
Hello TA,
Just want to make sure something.
- I want to deploy Elasticsearch in Docker on my machine directly, the VM is way too fat. In order to do so, I want to make sure that
- Does the Ubuntu VM only contain the deployed Elasticsearch?
- If so, did you add other configuration on it? Can I just
docker-compose up
the files in theelastic.zip
?
- Since you also provide us a Windows 10 VM, if my host machine is Windows 10 already, am I required to use the Windows 10 VM you provided? (Maybe you've configure something in the Windows 10 VM already?)
Thank you.
Weird host name
Logon success and Logoff scenario
Hello,
Does anyone know that how can I logout windows without closing the winlogbeat?
I thought that if I logout windows, the winlogbeat will close, and no log will be transport to ELK stack.
elasticdump
Hi TA, if I want to set up elasticdump, do I need to set up elk which is set up in the previous project, or I just need to install node.js and directly input json file
Kibana server is not ready yet
How to modify field.hostname as student id?
Hi all,
Does anyone know how to modify field.hostname on kibana? I get "unknown" as default, but TAs' mentioned that it should be set as student id.
Thanks very much for any suggestion.
Hint by TA
Kibana connecting problem
- If you have trouble connect to your Kibana.
Check if you are using Virtual Box. If that's the case, you should survey the port forwarding problem.
Kibana Login account
- The default account is like this below,
username: admin
password: admin
The output of your beats
- The output should be Logstash, not the ElasticSearch.
Winlogbeat issue
- You should run winlogbeat as
admin
, otherwise, some events may not be found in kibana.
Screensaver issue
- Your windows should be
education
version notindividual/home
.
Notice
- That's correct if you don't find event.code field in packetbeat log. You don't need to contain event.code field for the packetbeat log.
- Google is your friend, try to google error message
- Even if some scenarios you fail to accomplish, you can still try the best to write down the process and your attempts. We will give some points after full consideration.
Is there has the testcase without any attack?
Hello, is there has a test case without any attack?
I think Test_4
doesn't have any attack.
Information about C&C attack
Hello, TAs.
According to ppt, logs for C&C attack is about "the attacker exfiltrate some files from the victim on an unusual port", but after some investigation with Kibana, I still could not found lots of information with those packetbeat logs. Is there any further information or tips we can get about the attack? Thanks.
invalid argument
elasticsearch: port 9200 connectivity issue
Dear TAs,
Here are what work for me so far
- Ubuntu -
ELK
is up and running (I can log intokibana
with admin/admin) - Windows 10 - Installed
winlogbeat
and successfully started the service.
The problem
No logs from winlogbeat
were sent to elasticsearch (?)
What I've tried
I've read the official documentation of elastic and, if I'm not mistaken, winlogbeat
should send the logs via port 9200.
The official doc suggests that the user should run the following command to verify if the nodes are running:
$ curl -X GET "localhost:9200/_cat/nodes?v&pretty"
However, I received this output:
$ curl -X GET "localhost:9200/_cat/nodes?v&pretty"
curl: (52) Empty reply from server
Visiting this URL via a browser gives:
My winlogbeat.yml
Using the following config, I can start the winlogbeat
service successfully.
# =================================== Kibana ===================================
setup.kibana:
# Kibana Host
host: "192.168.131.137:5601"
# ---------------------------- Elasticsearch Output ----------------------------
output.elasticsearch:
# Array of hosts to connect to.
hosts: ["192.168.131.137:9200"]
# Protocol - either `http` (default) or `https`.
#protocol: "https"
# Authentication credentials - either API key or username/password.
#api_key: "id:api_key"
username: "admin"
password: "admin"
The problem is that there isn't any log from winlogbeat
showing up in elasticsearch.
Did I miss something? Thanks in advance! 🥺 🥺
"feilds.hostname" attribute is still unknown
Kibana can't find any index pattern like 'logstash*'
Dear TAs,
as title, as I apply the following settings:
My 'winlogbeat.yml':
setup.kibana:
host: "192.168.50.89:5601"
username: "admin"
password: "admin"
# ------------------------------ Logstash Output -------------------------------
output.logstash:
hosts: ["192.168.50.89:5044"]
Port forwarding:
I can load into the dashboard successfully in Windows, but I can't find any index that contains 'logstash*' in Kibana and create it as an index pattern.
Did I miss anything? 🥺
Are no repetitive attacks in the cases correct?
Hello, I find Attack_1,3,5 have CVE-2020-0796-RCE-POC
AccessList in winlogbeat
.
It is mean one of Attack_1,3,5 has Malicious Attachment
?
If no repetitive attacks, Why winlogbeat
have its accessList log?
Hint by TA
Test4 case
- We provided an additional test case for Attack4 in newe3 platform, it has more obvious behavior in it.
Elasticdump problem
- You can refer to #24, they have some possible solutions
Output
- For the output, you only need to output Attack_ID. You don't need to output the attack name.
Format issue
- About the format issue, always follow the spec.
Notice
- Make sure you have read old(closed) issue before you post issues.
- We might directly close the issue if it is a duplicate problem.
command not found
/elastic& sudo docker-compose up 打出這行後
sudo: docker-compose: Command not found 回應是這樣
Question about field hostname
Hi all,
I can receive "login" log if my field.hostname is "unknown" as default. But after I modify the hostname to student id in winlogbeat.yml, I can't receive "login" log but only "user account management". I've tried to restart winlogbeat and all docker containers in ubuntu but still not working.
Does anyone know how to solve this problem? Or is there any other way to modify field.hostname without modifing winlogbeat.yml?
Thanks for any suggestion.
error fetching fields for index pattern winlogbeat: Not Found
不知道有沒有人跟我遇到一樣的問題
已經好幾個小時卡在這了
winlogbeat就照著官網改,然後改用logstash連線
start-service winlogbeat之後在kibana上怎麼跑都一樣
唯一想到可能的bug是官網說要連至logstash下這條
.\winlogbeat.exe setup --template -E output.logstash.enabled=false -E 'output.elasticsearch.hosts=["192.168.153.133:9200"]'
可是他只會噴
而且我做完的朋友跟我說不用下 .\winlogbeat setup的指令(?)
Training Data & Test Data
PDF上面似乎沒有說明哪個data對應到哪種攻擊,但是這樣我在做test的時候就沒辦法確定我的model判斷的是否準確。
想請問助教能否提供這方面的資訊,謝謝。
問題,kibana server is not ready yet
How to upload the file to data visualizer with exceeding 100 MB size?
I have referred to closed issue #24
But,I have problem,
How to upload the file packetbeat.json to data visualizer with exceeding 100 MB size?
I had google it, but still not found solution.(or maybe no solution?)
Or, I need to use elasticdump to upload file with large size?
thanks.
the whole step question
可以請問助教整個步驟是如何進行的嗎?
第一個步驟下載完our docker files後已經是包含了甚麼文件?
我看裡面有docker-compose.yml 、 internal-use.yml 、logstash.yml 、logstash.conf
(1)而裡面這個docker-compose跟從網路上載下來的有什麼不一樣?
(2)哪ELK Stack又去哪裡了?Elasticsearch是包含docker-compose.yml internal-use.yml 就是Elasticsearch ?Logstash裡面有了就是logstash.yml 、logstash.conf,需要設定什麼開啟甚麼嗎? Kibana是Winlogbeat、Packetbeat嗎?
(3)sudo docker compose up跑完已經是跑了Elasticsearch、Logstash嗎?最後一步驟聯網才是開啟Kibana跑Kibana嗎?
(4)跑Elasticsearch、Logstash是在虛擬機上跑,所以只有Kibana需要處理port問題,也就是5601、5044?如果不是的話這還有少甚麼?
(5) VMnet1為主機ip,enp0s為客機ip?
(6)前面問題說要確認能不能連在windows上打ping 10.0.2.15,不過我的顯示這樣是沒有連到?
Will the test case in demo have either more than 1 attack scenario or no attack scenario?
As title.
If so, what should we output?(ex. testcase1: sql injection and ddos?)
or maybe they all only have 1 attack scenario?
winlogbeat secure module not support event 4802-4803 4667-4668
official docs :
https://www.elastic.co/guide/en/beats/winlogbeat/current/winlogbeat-module-security.html
I think 4802-4803 4667-4668 is essential to this project, but winlogbeat doesn't support it,
I wonder if this is expected,
If yes, I think I need some hints : ( (maybe filebeat((? , I almost tried to modify and recompile winlogbeat ......)
Is the Log file has correct format?
Scenario:
when I try to use "elasticdump" to import log data to my elastic search,I got an error.
error log like below:
{
_index: 'my_index',
_type: '_doc',
_id: '4mwlHnYBWeK1m2KP0ZrY',
status: 400,
error: {
type: 'mapper_parsing_exception',
reason: 'failed to parse',
caused_by: {
type: 'not_x_content_exception',
reason: 'Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes'
}
}
}
Project2 Submission Format
In ppt, the submission format is
<student_id>.zip
|- project_code/
|- <student_id>.pdf
In spec, the submission format is
<student_id>.zip
|- <student_id>/
|- project_code/
|- report.pdf
Which is the correct submission format?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.