Comments (19)
micheloosterhof
commented on June 19, 2024
The JSON log is enabled, but it's failing. Which version of Cowrie are you using? Do you use the Docker image, a git clone or a release download?
from docker-cowrie.
devamtech
commented on June 19, 2024
thank you so much for your response,
I am using the below yaml file on K8s 1.21 to deploy Cowrie, Can you please tell me where to provide
apiVersion: v1
kind: Service
metadata:
name: cowrie-lb
labels:
app: cowrie-app
spec:
selector:
app: cowrie-app
tier: honeypot
ports:
- protocol: "TCP"
port: 22
targetPort: 2222
type: LoadBalancer
apiVersion: apps/v1
kind: Deployment
metadata:
name: cowrie-app
labels:
app: cowrie-app
spec:
replicas: 4
selector:
matchLabels:
app: cowrie-app
tier: honeypot
strategy:
type: Recreate
template:
metadata:
labels:
app: cowrie-app
tier: honeypot
spec:
# runtimeClassName: gvisor # ADD THIS LINE
containers:
- image: cowrie/cowrie:latest
name: cowrie-app
# env:
# - name: WORDPRESS_DB_HOST
# value: cowrie-app-mysql
# - name: WORDPRESS_DB_PASSWORD
# valueFrom:
# secretKeyRef:
# name: mysql-pass
# key: password
ports:
- containerPort: 2222
name: cowrie-app
# volumeMounts:
# - name: cowrie-app-persistent-storage
# mountPath: /var/www/html
# volumes:
# - name: cowrie-app-persistent-storage
# persistentVolumeClaim:
# claimName: wp-pv-claim
from docker-cowrie.
devamtech
commented on June 19, 2024
I am using Cowrie Version 2.2.0. I like this version because of below output but only problem is, getting this as a "String"
eventid': 'cowrie.login.success', 'username': b'root', 'password': b'kumar', 'message': "login attempt [b'root'/b'kumar'] succeeded", 'sensor': 'cowrie-app-667d44f77-kcdzb', 'timestamp': '2021-05-18T14:26:24.763361Z', 'src_ip': '192.0.2.1', 'session': '24221d408f3c'}'
from docker-cowrie.
devamtech
commented on June 19, 2024
Hi team,
can you please help me on this issue, If I have json format, its easy to create kibana dashboard
from docker-cowrie.
micheloosterhof
commented on June 19, 2024
I just checked the Docker container at hub.docker.com/cowrie/cowrie and that successfully logs to JSON.
To get that on persistent storage, assign volumes for VOLUME [ "/cowrie/cowrie-git/var", "/cowrie/cowrie-git/etc" ]
The var/log/cowrie/cowrie.json should contain the log in JSON format.
If you get persistent errors with JSON serialisation failing, please ensure all your components are up to date and please share which event is causing the serialisation failure.
from docker-cowrie.
devamtech
commented on June 19, 2024
thank you so much Michel for your response.
I am using the cowrie 2.2 version and
Only for login even I am getting error
jsonlog: Can't serialize: '{'eventid': 'cowrie.login.success', 'username': b'root', 'password': b'deva', 'message': "login attempt [b'root'/b'deva'] succeeded", 'sensor': 'cowrie-app-5bcb76dd6b-8s7wn', 'timestamp': '2021-05-22T07:12:08.262918Z', 'src_ip': '10.144.0.2', 'session': '92fcfba6d9b2'}'
I have tested on multiple Kubernetes versions k8s 1.18 and K8s 1.19 and K8s 1.21, on environments, getting same above logs output.
on new cowrie, I am not seeing the error, but the problem is, In the new version, I am not getting src_ip and username and password fields like the above output, that's the reason I am using cowrie 2.2
new cowrie logs
2021-05-20T16:05:48+0000 [cowrie.ssh.factory.CowrieSSHFactory] New connection: 10.163.0.2:9054 (10.163.1.8:2222) [session: 7240a1ba5a5e]
2021-05-20T16:05:48+0000 [HoneyPotSSHTransport,0,10.163.0.2] Remote SSH version: b'SSH-2.0-OpenSSH_7.8'
2021-05-20T16:05:48+0000 [HoneyPotSSHTransport,0,10.163.0.2] SSH client hassh fingerprint: 93726f9c35049b4369578bad80d3a24a
2021-05-20T16:05:48+0000 [cowrie.ssh.transport.HoneyPotSSHTransport#debug] kex alg=b'ecdh-sha2-nistp256' key alg=b'ssh-rsa'
2021-05-20T16:05:48+0000 [cowrie.ssh.transport.HoneyPotSSHTransport#debug] outgoing: b'aes128-ctr' b'hmac-sha1' b'none'
2021-05-20T16:05:48+0000 [cowrie.ssh.transport.HoneyPotSSHTransport#debug] incoming: b'aes128-ctr' b'hmac-sha1' b'none'
2021-05-20T16:05:50+0000 [cowrie.ssh.transport.HoneyPotSSHTransport#debug] NEW KEYS
2021-05-20T16:05:50+0000 [cowrie.ssh.transport.HoneyPotSSHTransport#debug] starting service b'ssh-userauth'
2021-05-20T16:05:50+0000 [cowrie.ssh.userauth.HoneyPotSSHUserAuthServer#debug] b'root' trying auth b'none'
2021-05-20T16:05:52+0000 [cowrie.ssh.userauth.HoneyPotSSHUserAuthServer#debug] b'root' trying auth b'password'
2021-05-20T16:05:52+0000 [HoneyPotSSHTransport,0,10.163.0.2] Could not read etc/userdb.txt, default database activated
2021-05-20T16:05:52+0000 [HoneyPotSSHTransport,0,10.163.0.2] login attempt [b'root'/b'deva'] succeeded
I am looking for Old cowrie logs format.
Can you please help
Many thanks in advance.
from docker-cowrie.
micheloosterhof
commented on June 19, 2024
I'm very confused by what version you are using. If you are running on k8s, you are using cowrie/cowrie:latest container from Docker Hub? Or did you create your own container from an old version?
Changing k8s versions is not going to change a single thing, it's only responsible for starting your container. Your root cause will be somewhere inside the container.
Cowrie has 2 log files. cowrie.log and cowrie.json. If you are looking at cowrie.log you will see the text based logs. Inside the .json file are the JSON logs. Exceptions/errors will go into the .log file, not in the .json files.
from docker-cowrie.
devamtech
commented on June 19, 2024
10 days before cowrie/cowrie:latest was having a different image, the same image I pushed to my local repository and using the same
from docker-cowrie.
devamtech
commented on June 19, 2024
can you please help me to get the below kind of output in cowrie/cowrie:latest
eventid': 'cowrie.login.success', 'username': b'root', 'password': b'deva', 'message': "login attempt [b'root'/b'deva'] succeeded", 'sensor': 'cowrie-app-5bcb76dd6b-8s7wn', 'timestamp': '2021-05-22T07:12:08.262918Z', 'src_ip': '10.144.0.2', 'session': '92fcfba6d9b2'}'
from docker-cowrie.
micheloosterhof
commented on June 19, 2024
Asking the same question over and over will not give different results.
from docker-cowrie.
micheloosterhof
commented on June 19, 2024
So I think you are not looking at the file output, you are looking at standard output from the container?
That's not in JSON. that's text. The JSON log goes into a file on the file system. If you want to put it elsewhere, use an output plugin. There are elastic search output plugins that can send directly to Elasticsearch.
from docker-cowrie.
devamtech
commented on June 19, 2024
The same format its coming in elastic search also, because the below logs is considering a string, if I use parsers and filter in fluent bit, its not working.
log": [
"2021-05-22T12:21:57.553505608Z stdout F 2021-05-22T12:21:57+0000 [stdout#info] jsonlog: Can't serialize: '{'eventid': 'cowrie.login.success', 'username': b'root', 'password': b'maatv', 'message': "login attempt [b'root'/b'maatv'] succeeded", 'sensor': 'cowrie-app-bcd99c59f-pdcz6', 'timestamp': '2021-05-22T12:21:57.553068Z', '@kibana-highlighted-field@src_ip@/kibana-highlighted-field@': '192.0.6.1', 'session': '44530278b1e6'}'"
]
from docker-cowrie.
micheloosterhof
commented on June 19, 2024
Please share your configuration. What you are looking at is an error message. This is not the message you want to parse. It's the error message it can't write that data to JSON. Please share your exact configuration, how you are sending to Elastic, etc.
from docker-cowrie.
devamtech
commented on June 19, 2024
Thank you Michel for your response.
This is my fluentbit Configuration file.
apiVersion: v1
kind: ConfigMap
metadata:
name: fluent-bit-config
namespace: logging
labels:
k8s-app: fluent-bit
data:
# Configuration files: server, input, filters and output
# ======================================================
fluent-bit.conf: |
[SERVICE]
Flush 1
Log_Level info
Daemon off
Parsers_File parsers.conf
HTTP_Server On
HTTP_Listen 0.0.0.0
HTTP_Port 2020
@INCLUDE input-kubernetes.conf
@INCLUDE filter-kubernetes.conf
@INCLUDE output-elasticsearch.conf
input-kubernetes.conf: |
[INPUT]
Name tail
Tag kube.*
Path /var/log/containers/*.log
Parser docker
DB /var/log/flb_kube.db
Mem_Buf_Limit 5MB
Skip_Long_Lines On
Refresh_Interval 10
filter-kubernetes.conf: |
[FILTER]
Name kubernetes
Match kube.*
Kube_URL https://kubernetes.default.svc:443
Kube_CA_File /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
Kube_Token_File /var/run/secrets/kubernetes.io/serviceaccount/token
Kube_Tag_Prefix kube.var.log.containers.
Merge_Log On
Merge_Log_Key log_processed
K8S-Logging.Parser On
K8S-Logging.Exclude Off
output-elasticsearch.conf: |
[OUTPUT]
Name es
Match *
Host 10.98.41.40
Port 9200
Logstash_Format On
Replace_Dots On
Retry_Limit False
parsers.conf: |
[PARSER]
Name apache
Format regex
Regex ^(?<host>[^ ]*) [^ ]* (?<user>[^ ]*) \[(?<time>[^\]]*)\] "(?<method>\S+)(?: +(?<path>[^\"]*?)(?: +\S*)?)?" (?<code>[^ ]*) (?<size>[^ ]*)(?: "(?<referer>[^\"]*)" "(?<agent>[^\"]*)")?$
Time_Key time
Time_Format %d/%b/%Y:%H:%M:%S %z
[PARSER]
Name apache2
Format regex
Regex ^(?<host>[^ ]*) [^ ]* (?<user>[^ ]*) \[(?<time>[^\]]*)\] "(?<method>\S+)(?: +(?<path>[^ ]*) +\S*)?" (?<code>[^ ]*) (?<size>[^ ]*)(?: "(?<referer>[^\"]*)" "(?<agent>[^\"]*)")?$
Time_Key time
Time_Format %d/%b/%Y:%H:%M:%S %z
[PARSER]
Name apache_error
Format regex
Regex ^\[[^ ]* (?<time>[^\]]*)\] \[(?<level>[^\]]*)\](?: \[pid (?<pid>[^\]]*)\])?( \[client (?<client>[^\]]*)\])? (?<message>.*)$
[PARSER]
Name nginx
Format regex
Regex ^(?<remote>[^ ]*) (?<host>[^ ]*) (?<user>[^ ]*) \[(?<time>[^\]]*)\] "(?<method>\S+)(?: +(?<path>[^\"]*?)(?: +\S*)?)?" (?<code>[^ ]*) (?<size>[^ ]*)(?: "(?<referer>[^\"]*)" "(?<agent>[^\"]*)")?$
Time_Key time
Time_Format %d/%b/%Y:%H:%M:%S %z
[PARSER]
Name json
Format json
Time_Key time
Time_Format %d/%b/%Y:%H:%M:%S %z
[PARSER]
Name docker
Format json
Time_Key time
Time_Format %Y-%m-%dT%H:%M:%S.%L
Time_Keep On
[PARSER]
# http://rubular.com/r/tjUt3Awgg4
Name cri
Format regex
Regex ^(?<time>[^ ]+) (?<stream>stdout|stderr) (?<logtag>[^ ]*) (?<message>.*)$
Time_Key time
Time_Format %Y-%m-%dT%H:%M:%S.%L%z
[PARSER]
Name syslog
Format regex
Regex ^\<(?<pri>[0-9]+)\>(?<time>[^ ]* {1,2}[^ ]* [^ ]*) (?<host>[^ ]*) (?<ident>[a-zA-Z0-9_\/\.\-]*)(?:\[(?<pid>[0-9]+)\])?(?:[^\:]*\:)? *(?<message>.*)$
Time_Key time
Time_Format %b %d %H:%M:%S
from docker-cowrie.
micheloosterhof
commented on June 19, 2024
I just checked the latest docker container and JSON logging seems okay. I suspect your JSON log can't write its data because you haven't defined the volume.
Disable the JSON logging, and enable the Elasticsearch output plugin instead.
You can use environment variables.
COWRIE_OUTPUT_JSONLOG_ENABLED=false
COWRIE_OUTPUT_ELASTICSEARCH_ENABLED=true
COWRIE_OUTPUT_ELASTICSEARCH_HOST=hostname
from docker-cowrie.
alexey-kleymenov
commented on June 19, 2024
Hi @micheloosterhof, nice to meet you! I experience the same issue with broken authentication records using the cowrie/cowrie:latest image. The issue persists for at least 1 month (when I deployed it the first time) and up until now.
Here is the command line I'm using:
docker run -p 22:2222 -p 23:2223 -e COWRIE_TELNET_ENABLED=yes -v /home/ubuntu/honeypot/logs:/cowrie/cowrie-git/var/log/cowrie/ -v /home/ubuntu/honeypot/samples:/cowrie/cowrie-git/var/lib/cowrie/downloads cowrie/cowrie
Here is an example stdout log snippet with 2 errors (the first and the last lines):
2021-06-11T10:51:33+0000 [stdout#info] jsonlog: Can't serialize: '{'eventid': 'cowrie.client.kex', 'hassh': '2f300334eb474e4d5ef932343447dd80', 'hasshAlgorithms': '[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1;aes128-ctr,aes192-ctr,aes256-ctr,[email protected],arcfour256,arcfour128,aes128-cbc,arcfour,3des-cbc;[email protected],hmac-sha2-256,hmac-sha1,hmac-sha1-96;none', 'kexAlgs': [b'[email protected]', b'ecdh-sha2-nistp256', b'ecdh-sha2-nistp384', b'ecdh-sha2-nistp521', b'diffie-hellman-group14-sha1', b'diffie-hellman-group1-sha1'], 'keyAlgs': [b'[email protected]', b'[email protected]', b'[email protected]', b'[email protected]', b'[email protected]', b'[email protected]', b'ecdsa-sha2-nistp256', b'ecdsa-sha2-nistp384', b'ecdsa-sha2-nistp521', b'ssh-rsa', b'ssh-dss', b'ssh-ed25519'], 'encCS': [b'aes128-ctr', b'aes192-ctr', b'aes256-ctr', b'[email protected]', b'arcfour256', b'arcfour128', b'aes128-cbc', b'arcfour', b'3des-cbc'], 'macCS': [b'[email protected]', b'hmac-sha2-256', b'hmac-sha1', b'hmac-sha1-96'], 'compCS': [b'none'], 'langCS': [b''], 'message': 'SSH client hassh fingerprint: 2f300334eb474e4d5ef932343447dd80', 'sensor': 'dda70ef8e80a', 'timestamp': '2021-06-11T10:51:33.923804Z', 'src_ip': '78.128.113.150', 'session': '79d7d0a46cee'}'
2021-06-11T10:51:33+0000 [cowrie.ssh.transport.HoneyPotSSHTransport#debug] kex alg=b'[email protected]' key alg=b'ssh-rsa'
2021-06-11T10:51:33+0000 [cowrie.ssh.transport.HoneyPotSSHTransport#debug] outgoing: b'aes128-ctr' b'hmac-sha1' b'none'
2021-06-11T10:51:33+0000 [cowrie.ssh.transport.HoneyPotSSHTransport#debug] incoming: b'aes128-ctr' b'hmac-sha1' b'none'
2021-06-11T10:51:34+0000 [cowrie.ssh.transport.HoneyPotSSHTransport#debug] NEW KEYS
2021-06-11T10:51:34+0000 [cowrie.ssh.transport.HoneyPotSSHTransport#debug] starting service b'ssh-userauth'
2021-06-11T10:51:34+0000 [cowrie.ssh.userauth.HoneyPotSSHUserAuthServer#debug] b'root' trying auth b'none'
2021-06-11T10:51:34+0000 [cowrie.ssh.userauth.HoneyPotSSHUserAuthServer#debug] b'root' trying auth b'password'
2021-06-11T10:51:34+0000 [HoneyPotSSHTransport,37,78.128.113.150] Could not read etc/userdb.txt, default database activated
2021-06-11T10:51:34+0000 [stdout#info] jsonlog: Can't serialize: '{'eventid': 'cowrie.login.success', 'username': b'root', 'password': b'admin', 'message': "login attempt [b'root'/b'admin'] succeeded", 'sensor': 'dda70ef8e80a', 'timestamp': '2021-06-11T10:51:34.241754Z', 'src_ip': '78.128.113.150', 'session': '79d7d0a46cee'}'
And here are a few of the many broken JSON entries in the cowrie.json. As you can see, they don't follow the JSON format and the username and password values are not available:
{"eventid":"cowrie.client.kex","hassh":"2f300334eb474e4d5ef932343447dd80","hasshAlgorithms":"[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1;aes128-ctr,aes192-ctr,aes256-ctr,[email protected],arcfour256,arcfour128,aes128-cbc,arcfour,3des-cbc;[email protected],hmac-sha2-256,hmac-sha1,hmac-sha1-96;none","kexAlgs":[{"eventid":"cowrie.login.success","username":{"eventid":"cowrie.direct-tcpip.request","dst_ip":"ya.ru","dst_port":80,"src_ip":"92.118.36.4","src_port":0,"message":"direct-tcp connection request to ya.ru:80 from 0.0.0.0:0","sensor":"dda70ef8e80a","timestamp":"2021-06-11T10:43:02.863361Z","session":"fc8657d24108"}
{"eventid":"cowrie.login.success","username":{"eventid":"cowrie.session.params","arch":"linux-x64-lsb","message":[],"sensor":"05944c27d909","timestamp":"2021-06-11T00:00:17.447110Z","src_ip":"60.171.154.30","session":"bf66dfc101ed"}
from docker-cowrie.
micheloosterhof
commented on June 19, 2024
This is really strange, like it's mixing 2 events together. In both cases it's the username field containing another event.
@alexey-kleymenov your issue is not the same as the previous one. the ticket opener does not have malformed JSON entries but issues writing to a file. Could you open a new ticket for this, please?
from docker-cowrie.
alexey-kleymenov
commented on June 19, 2024
@micheloosterhof, sure! Here we go: #28. Originally, I put it here because I saw that the topic starter had the same jsonlog serialization error message in logs when parsing cowrie.login.success records.
from docker-cowrie.
micheloosterhof
commented on June 19, 2024
I just checked the
latestdocker container and JSON logging seems okay. I suspect your JSON log can't write its data because you haven't defined the volume.
Disable the JSON logging, and enable the Elasticsearch output plugin instead.You can use environment variables.
COWRIE_OUTPUT_JSONLOG_ENABLED=false COWRIE_OUTPUT_ELASTICSEARCH_ENABLED=true COWRIE_OUTPUT_ELASTICSEARCH_HOST=hostname
@devamtech are you still experiencing issues or did this work for you?
from docker-cowrie.
Related Issues (20)
- Forbidden access to S3-compatible Object Storage HOT 14
- How to use mysql to log in docker-cowrie?
- Problem with unable to modify configuration file HOT 4
- Locale US.UTF-8 missing HOT 2
- Read only mount for /cowrie/cowrie-git/etc is not supported
- line 25: DEPLOY_KEY: unbound variable HOT 2
- MySQL output engine won't work within docker HOT 4
- Needs Update on Docker Image on Docker Hub HOT 1
- SQL logging error HOT 10
- Failing to build wheels for cryptography package on arm device HOT 7
- output_mysql generates incorrect SQL queries HOT 20
- Corrupted log entries when username is involved in latest cowrie docker image HOT 5
- No more FTP when connected
- Failed to load output engine: hpfeeds HOT 7
- Cowrie log files with docker-compose HOT 1
- How do you use playlog and other features if it's a container? HOT 1
- No attackers ip in the log HOT 2
- Needs HEALTHCHECK
- Raspberry Pi support HOT 9
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from docker-cowrie.