THeHive5 LDAP Requests Issue[Bug] about thehive-feedback HOT 37 CLOSED

Hello,

Did you configure the LDAP synchronisation in TheHive (Admin > Platform management > LDAP servers ) ?
This will keep TheHive users in sync with your ldap (it will create users, lock former users, update user organisations ...)
The sync interval is every hour: so a burst of requests every hour.

If you want just the authentification with ldap (TH4 behavior), you can configure your ldap server in Platform Management > Authentication > Directories Authentication

from thehive-feedback.

Hello,

Did you configure the LDAP synchronisation in TheHive (Admin > Platform management > LDAP servers ) ? This will keep TheHive users in sync with your ldap (it will create users, lock former users, update user organisations ...) The sync interval is every hour: so a burst of requests every hour.

If you want just the authentification with ldap (TH4 behavior), you can configure your ldap server in Platform Management > Authentication > Directories Authentication

Thanks for your reply.

Having tried both of your suggestions, unfortunately the 'LDAP Servers' option is disabled with no way to enable it, and the 'Directories Authentication' option is also disabled with no way to change it. Screenshots below:

Any ideas? :)

from thehive-feedback.

If your configuration file does not mention ldap config, I don't think that the issue is coming from TheHive, as it seems that no server is configured here.

TheHive only uses ldap with those settings.

from thehive-feedback.

to be 100% sure of that, you could try running tcpdump on the TheHive server or on your ldap to see where the connections are coming from.

from thehive-feedback.

If your configuration file does not mention ldap config, I don't think that the issue is coming from TheHive, as it seems that no server is configured here.

TheHive only uses ldap with those settings.

So, I went into our application.conf file and confirmed there isn't any config in there for LDAP currently. We have it in our old conf file for TH4 before we upgraded, but that is no longer used and stored as backup with a different filename.

I went and ran a tcpdump on the server that runs thehive (both with and without thehive service running). When it's running, it appears to be flooding our LDAP server with logs into the hundreds every second. Once I stopped the thehive service and ran tcpdump again, these logs were gone - so unfortunately, it looks like its thehive causing it somehow.

If there's any other screenshots or information I can provide, please just let me know. Happy to help where necessary.

from thehive-feedback.

There might be something in the configuration somewhere then.
Here could be a few ways to explore the issue further:

• Get the configuration from the server: connect to http://<thehive_url>/api/v1/admin/config/auth with an admin account.
  This sould get you the json object configuration. Could you remove your sensitive information and provide me the value here ?
  It should look like this:

{
  "providers": [
    {
      "name": "session"
    },
    {
      "name": "basic",
      "realm": "thehive"
    },
    {
      "name": "local"
    },
    {
      "name": "key"
    },
    {
      "name": "header",
      "userHeader": "X-USER"
    },
    {
      "authorizationHeader": "Bearer",
      "authorizationUrl": "http://localhost:8080/auth/realms/TheHive/protocol/openid-connect/auth",
      "clientId": "thehive-app",
      "clientSecret": "xxx",
      "grantType": "authorization_code",
      "name": "oauth2",
      ...
    }
  ],
  "defaults": {
    "ad": {
      "useSSL": false
    },
    "basicSession": {
      "timeout": "1 hour",
      "warning": "5 minutes"
    },
    "header": {
      "userHeader": "X-USERID"
    },
    "ldap": {
      "useSSL": false
    },
    "local": {
      "passwordPolicy": {
        "cannotContainUsername": true,
        "enabled": false,
        "minDigit": 1,
        "minLength": 8,
        "minLowerCase": 1,
        "minSpecial": 1,
        "minUpperCase": 1
      }
    },
    "oauth2": {},
    "pki": {
      "certificateField": "cn"
    },
    "session": {
      "maxSession": "10 hours",
      "timeout": "1 hour",
      "warning": "5 minutes"
    },
    "sso": {}
  },
  "availableProviders": [
    {
      "name": "session"
    },
    {
      "name": "htpasswd"
    },
    {
      "name": "ad"
    },
    {
      "name": "ldap"
    },
    {
      "name": "local"
    },
    {
      "name": "key"
    },
    {
      "name": "basic"
    },
    {
      "name": "header"
    },
    {
      "name": "pki"
    },
    {
      "name": "basicSession"
    },
    {
      "name": "oauth2"
    }
  ],
  "multifactor": {
    "enabled": true
  },
  "user": {
    "autoCreateOnSso": true,
    "profileFieldName": "profile",
    "organisationFieldName": "organisation",
    "defaults": {
      "profile": "analyst",
      "organisation": "myorg"
    }
  }
}

• You can activate the logs in the application: in logback.xml, add the loggers and restart the service:

<logger name="org.thp.scalligraph.auth" level="DEBUG" />
<logger name="org.thp.thehive.enterprise.services.LdapSync" level="DEBUG" />

• Activate the logs from java.jndi: https://docs.oracle.com/javase/jndi/tutorial/ldap/connect/config.html
  You need to pass the flag -Dcom.sun.jndi.ldap.connect.pool.debug=all to the java process.
  To do this, I think the easiest way is to update the systemd unit file for thehive. You can do this by modifying the systemd unit file:

ExecStart=/opt/thehive/bin/thehive \
	-Dconfig.file=/etc/thehive/application.conf \
	-Dlogger.file=/etc/thehive/logback.xml \
	-Dpidfile.path=/dev/null \
        -Dcom.sun.jndi.ldap.connect.pool.debug=all

This setting will produce logs in stdout and stderr of the process, so should be visible with journalctl

from thehive-feedback.

Hi, so I've retrieved the logs you requested, these are below (I've removed server names etc from some of the logs, just in case you think they are missing):

Server configuration with admin account output:

{"providers":[{"name":"session"},{"name":"basic","realm":"thehive"},{"name":"local"},{"name":"key"}],"defaults":{"ad": 
{"useSSL":false},"basicSession":{"timeout":"1 hour","warning":"5 minutes"},"header":{"userHeader":"X-USERID"},"ldap": 
{"useSSL":false},"local":{"passwordPolicy":{"cannotContainUsername":true,"enabled":false,"minDigit":1,"minLength":8,"minLowerCase":1,"minSpecial":1,"minUpperCase":1}},"oauth2":{},"pki":{"certificateField":"cn"},"session":{"maxSession":"10 hours","timeout":"1 hour","warning":"5 minutes"},"sso":{}},"availableProviders":[{"name":"session"},{"name":"htpasswd"},{"name":"local"},{"name":"key"},{"name":"basic"},{"name":"basicSession"}],"multifactor":{"enabled":true},"user":{"autoCreateOnSso":false,"profileFieldName":"profile","organisationFieldName":"organisation","defaults":{"profile":"admin","organisation":"admin"}}}

LDAP Log output using journalctl:

/usr/lib/systemd/system> sudo journalctl -u thehive.service
-- Logs begin at Thu 2022-08-11 14:06:28 BST, end at Fri 2022-08-12 09:47:56 BST. --
Aug 11 14:06:30 systemd[1]: Started Scalable, Open Source and Free Security Incident Response Solutions.
Aug 11 14:09:58 systemd[1]: Stopping Scalable, Open Source and Free Security Incident Response Solutions...
Aug 11 14:09:59 systemd[1]: thehive.service: Succeeded.
Aug 11 14:09:59 systemd[1]: Stopped Scalable, Open Source and Free Security Incident Response Solutions.
Aug 11 15:25:21 systemd[1]: Started Scalable, Open Source and Free Security Incident Response Solutions.
Aug 11 15:32:58 systemd[1]: Stopping Scalable, Open Source and Free Security Incident Response Solutions...
Aug 11 15:32:59 systemd[1]: thehive.service: Succeeded.
Aug 11 15:32:59 systemd[1]: Stopped Scalable, Open Source and Free Security Incident Response Solutions.
Aug 12 09:30:42 systemd[1]: Started Scalable, Open Source and Free Security Incident Response Solutions.
Aug 12 09:38:59 systemd[1]: Stopping Scalable, Open Source and Free Security Incident Response Solutions...
Aug 12 09:39:00 systemd[1]: thehive.service: Succeeded.
Aug 12 09:39:00 systemd[1]: Stopped Scalable, Open Source and Free Security Incident Response Solutions.
Aug 12 09:39:19 systemd[1]: /usr/lib/systemd/system/thehive.service:12: Unknown lvalue '-Dcom.sun.jndi.ldap.connect.pool.debug' in section 'Service', ignoring
Aug 12 09:42:15 systemd[1]: Started Scalable, Open Source and Free Security Incident Response Solutions.

We are also currently looking into retrieving some of our LDAP logs just to see exactly what they are reporting, but we don't have this information just yet.

from thehive-feedback.

We've gone through the LDAP logs on our end, and it looks like the system is enumerating all users in our DC - the account are real, but not valid for thehive. A sample of the logs are;

2022-08-11T14:06:43.182056+01:00 theta slapd[179299]: conn=26687 op=17 SRCH attr=uid uniqueMember objectClass
2022-08-11T14:06:43.182193+01:00 theta slapd[179299]: conn=26687 op=17 SEARCH RESULT tag=101 err=0 nentries=1 text=
2022-08-11T14:06:43.182464+01:00 theta slapd[179299]: conn=26687 op=18 SRCH base="uid={real UID},ou=People,dc={dc}" scope=0 deref=0 filter="(objectClass=*)"
2022-08-11T14:06:43.182610+01:00 theta slapd[179299]: conn=26687 op=18 SRCH attr=uid uniqueMember objectClass
2022-08-11T14:06:43.182752+01:00 theta slapd[179299]: conn=26687 op=18 SEARCH RESULT tag=101 err=0 nentries=1 text=
2022-08-11T14:06:43.183156+01:00 theta slapd[179299]: conn=26687 op=19 SRCH base="uid={real UID},ou=People,dc={dc}" scope=0 deref=0 filter="(objectClass=*)"
2022-08-11T14:06:43.183302+01:00 theta slapd[179299]: conn=26687 op=19 SRCH attr=uid uniqueMember objectClass
2022-08-11T14:06:43.183453+01:00 theta slapd[179299]: conn=26687 op=19 SEARCH RESULT tag=101 err=0 nentries=1 text=
2022-08-11T14:06:43.183657+01:00 theta slapd[179299]: conn=26687 op=20 SRCH base="uid={real UID},ou=People,dc={dc}" scope=0 deref=0 filter="(objectClass=*)"
2022-08-11T14:06:43.183791+01:00 theta slapd[179299]: conn=26687 op=20 SRCH attr=uid uniqueMember objectClass
2022-08-11T14:06:43.183948+01:00 theta slapd[179299]: conn=26687 op=20 SEARCH RESULT tag=101 err=0 nentries=1 text=
2022-08-11T14:06:43.184388+01:00 theta slapd[179299]: conn=26687 op=21 SRCH base="uid={real UID},ou=People,dc={dc}" scope=0 deref=0 filter="(objectClass=*)"
2022-08-11T14:06:43.184523+01:00 theta slapd[179299]: conn=26687 op=21 SRCH attr=uid uniqueMember objectClass
2022-08-11T14:06:43.184663+01:00 theta slapd[179299]: conn=26687 op=21 SEARCH RESULT tag=101 err=32 nentries=0 text=
2022-08-11T14:06:43.184852+01:00 theta slapd[179299]: conn=26687 op=22 SRCH base="uid={real UID},ou=People,dc={dc}" scope=0 deref=0 filter="(objectClass=*)"
2022-08-11T14:06:43.185005+01:00 theta slapd[179299]: conn=26687 op=22 SRCH attr=uid uniqueMember objectClass
2022-08-11T14:06:43.185371+01:00 theta slapd[179299]: conn=26687 op=22 SEARCH RESULT tag=101 err=32 nentries=0 text=
2022-08-11T14:06:43.185537+01:00 theta slapd[179299]: conn=26687 op=23 SRCH base="uid={real UID},ou=People,dc={dc}" scope=0 deref=0 filter="(objectClass=*)"

from thehive-feedback.

From our code there are only 3 instances where we search in the LDAP servers:

• Ad Auth service configured => not configured apparently
• Ldap auth configured => not configured apparently
• ldap sync service => not in your license, but to make sure, what is the response of <the_hive_url>/api/config/syncUser.ldap.servers made with an admin account ?

from thehive-feedback.

From our code there are only 3 instances where we search in the LDAP servers:

• Ad Auth service configured => not configured apparently
• Ldap auth configured => not configured apparently
• ldap sync service => not in your license, but to make sure, what is the response of <the_hive_url>/api/config/syncUser.ldap.servers made with an admin account ?

Tested the response of <the_hive_url>/api/config/syncUser.ldap.servers, see below the result:

{"path":"syncUser.ldap.servers","description":"","defaultValue":[],"value":[]}

from thehive-feedback.

So no server configured here neither.

Could you share your application.conf file ? (remove your credentials from it)

Also could you try adding an iptable rule to disallow the access to the ldap server to TheHive ?
Maybe it will trigger an exception in the logs that we'll be able to analyze

from thehive-feedback.

I've dropped our application.conf below. With regards to the iptable rule suggestion, unfortunately implementing a rule like this is not possible with our current setup.

# TheHive configuration - application.conf
#
#
# This is the default configuration file.
# This is prepared to run with all services locally:
# - Cassandra for the database
# - Elasticsearch for index engine
# - File storage is local in /opt/thp/thehive/files
#
# If this is not your setup, please refer to the documentation at:
# https://docs.strangebee.com/thehive/
#
#
# Secret key - used by Play Framework
# If TheHive is installed with DEB/RPM package, this is automatically generated
# If TheHive is not installed from DEB or RPM packages run the following
# command before starting thehive:
#   cat > /etc/thehive/secret.conf << _EOF_
#   play.http.secret.key="$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 64 |#   head -n 1)"
#   _EOF_
include "/etc/thehive/secret.conf"


# Database and index configuration
# By default, TheHive is configured to connect to local Cassandra 4.x and a
# local Elasticsearch services without authentication.
db.janusgraph {
  storage {
    backend = cql
    hostname = ["127.0.0.1"]
    # Cassandra authentication (if configured)
    # username = "thehive"
    # password = "password"
    cql {
      cluster-name = thp
      keyspace = thehive
    }
  }
  index.search {
    backend = elasticsearch
    hostname = ["127.0.0.1"]
    index-name = thehive
  }
}

# Attachment storage configuration
# By default, TheHive is configured to store files locally in the folder.
# The path can be updated and should belong to the user/group running thehive service. (by default: thehive:thehive)
storage {
  provider = localfs
  localfs.location = /opt/thp/thehive/files
}

# Define the maximum size for an attachment accepted by TheHive
play.http.parser.maxDiskBuffer = 1GB
# Define maximum size of http request (except attachment)
play.http.parser.maxMemoryBuffer = 10M

# Service configuration
application.baseUrl = "http://localhost:9000"
play.http.context = "/"

# Additional modules
#
# TheHive is strongly integrated with Cortex and MISP.
# Both modules are enabled by default. If not used, each one can be disabled by
# commenting the configuration line.
scalligraph.modules += org.thp.thehive.connector.cortex.CortexModule
scalligraph.modules += org.thp.thehive.connector.misp.MispModule

from thehive-feedback.

It does not seem to be anything wrong with your configuration.

Maybe the queries do not come from TheHive but from Cortex with the LdapQuery Analyzer ? https://thehive-project.github.io/Cortex-Analyzers/analyzers/LdapQuery/

from thehive-feedback.

I’ve looked and it doesn’t appear that we even have that LDAP analyzer in our directory, so I don’t think it can be that. When starting Cortex, I did some across these logs, however Cortex is on the same version it was before we upgraded thehive and hasn't been touched since, so I doubt Cortex is the culprit...

2022-08-18 10:41:09,931 [INFO] from module in main - Loading authentication module class org.thp.cortex.services.OAuth2Srv
2022-08-18 10:41:09,931 [INFO] from module in main - Loading authentication module class org.elastic4play.services.auth.ADAuthSrv
2022-08-18 10:41:09,931 [INFO] from module in main - Loading authentication module class org.elastic4play.services.auth.LdapAuthSrv
2022-08-18 10:41:09,932 [INFO] from module in main - Loading authentication module class org.thp.cortex.services.KeyAuthSrv
2022-08-18 10:41:09,932 [INFO] from module in main - Loading authentication module class org.thp.cortex.services.LocalAuthSrv

from thehive-feedback.

Does Cortex has some configuration with your LDAP ? See: https://github.com/TheHive-Project/CortexDocs/blob/master/admin/admin-guide.md#authentication

Maybe when TheHive tries to connect to Cortex, it triggers some auth attempts in your ldap

from thehive-feedback.

Hi - This is something I already checked. We have no LDAP configuration in place in application.conf for Cortex, so I doubt it could be that.

As a suggestion - we have a test server we used to have for thehive4 just for new version upgrade testing before releasing to our live server. We could attempt installing thehive5 on there and see if the same issue occurs - it might be interesting to know if the issue replicates or not, especially if we are limited to solutions now.

from thehive-feedback.

Hi there - just wondering if there's any updates on this one @vdebergue? Thanks for your support!

from thehive-feedback.

Hello, I launched TheHive on my side connected to a local LDAP but I didn't see any suspicious activity on it. Other than when users are attempting to log in using login + password.

from thehive-feedback.

Thanks for the update. We have tried over the past week or so performing a fresh install (with no previous hive configs) on our test server and whenever we start thehive, it starts relentlessly hitting our LDAP server looking for accounts. Its so noisy, it acts and a Denial of Service on our LDAP storage - we just can't run it like this. For some reason, its occurring on both servers and the LDAP hits only occur when the service is being run (we've observed it immediately stopping upon the service being halted).

from thehive-feedback.

We've done some more testing on this, and found that if we move the files;

/etc/libnss-ldap.conf
/etc/pam_ldap.conf

Out of the way, then thehive will start and not flood our LDAP servers (as it can't find them!). This does mean that all our LDAP authentication on the server is broken though, so its a workaround at best.
We don't have access to the LDAP functionality on thehive, so we can't change / affect how thehive looks at these files (see attached screenshot)
It looks like LDAP is 'on' even though we aren't using it, is there a way it can be explicitly turned off? That might be the easiest approach.

from thehive-feedback.

With the latest version 5.0.13, the ldap sync is turned off if the license does not enable it (which is your case)
You can try to upgrade to this version and see what happens.

But it's really weird that moving those files has an impact, because TheHive does not specifically read those. TheHive will only read the content of application.conf and not look at other configuration files on the system.

from thehive-feedback.

Thanks - we have upgraded. With the LDAP server unavailable, if we run;

service thehive status
Sep 14 09:11:30 [server] getent[1825]: nss_ldap: could not search LDAP server - Server is unavailable
Sep 14 09:11:30 [server] getent[1827]: nss_ldap: could not connect to any LDAP server as (null) - Can't contact LDAP server

If we run with the LDAP files in place - it begins flooding again (no LDAP messages on status)

It looks like it is still looking for

/etc/libnss-ldap.conf

from thehive-feedback.

Looking at this, we've narrowed it down to

/etc/libnss-ldap.conf

If we change ldap.conf and pam_ldap.com, it makes no difference to thehive. As soon and libnss-ldap.conf is enabled, thehive picks up the LDAP server and starts to query.

from thehive-feedback.

Seems like nss-ldap and getent are responsible for the calls but I don't see how it is used by TheHive.
Do you have something that is using getent ?

from thehive-feedback.

getentwill be called by the OS, and it looks like thehive is asking for passwords and then enumerating the LDAP service. Our nsswitch.conf says

password files LDAP

(And for group and shadow) - so it should be searching the local files first. Thehive running user is thehive which is a local account (as expected).

We use LDAP for other accounts such as SSH, so the config is 'correct' in as much as the system works, aside from thehive doing huge numbers of LDAP lookups.

from thehive-feedback.

• Does your /etc/passwd file contains the user thehive ?
• What version of the jvm are you using ? (use java -version to display it)

from thehive-feedback.

Yes, /etc/passwd contains thehive user and
java -version
openjdk version "11.0.16" 2022-07-19

from thehive-feedback.

We ran nss in debug mode (using this https://github.com/gmjosack/nss_debug) with TheHive and did not see many calls. Just a few at the beginning of the application.
Maybe you can try the debug also and see what user it's looking for.

Does the group thehive also exists in /etc/groups ?

from thehive-feedback.

thehive does exist in /etc/group

Your behaviour is very close to ours:
If no LDAP server is defined - a few LDAP calls at the beginning, then just works
If LDAP is defined, but not available - thehive notes that it is unable to query, then starts fine
If LDAP is defined (valid nss-ldap) and available - then it starts sending out lots of requests

We'll start to see if we can run nss_debug as well

from thehive-feedback.

We've got nss_debug running however we can't see any logs for it. Can you tell us where the logs go/what directory they are stored in?

from thehive-feedback.

logs go to stderr: https://github.com/gmjosack/nss_debug/blob/master/nss_debug-passwd.c#L32
So if the service is started with systemd, you should get the logs from journalctl -u thehive.service

from thehive-feedback.

Extract from our nsswitch.conf;

`passwd:         debug ldap

group:          debug ldap

shadow:         debug ldap`

Sudo systemctl start thehive

sudo journalctl -u thehive.service

`Sep 14 16:02:33 d-beehaus systemd[1]: Started Scalable, Open Source and Free Security Incident Response Solutions.

Sep 14 16:02:39 d-beehaus systemd[1]: Stopping Scalable, Open Source and Free Security Incident Response Solutions...

Sep 14 16:02:39 d-beehaus systemd[1]: thehive.service: Succeeded.

Sep 14 16:02:39 d-beehaus systemd[1]: Stopped Scalable, Open Source and Free Security Incident Response Solutions.`

I am not ruling out that we are not running nss_debug badly, but from this we can't see anything (it was only running for 6 seconds total - we did see the LDAP queries in tcpdump).

from thehive-feedback.

You could try to run the process manually ?

sudo -u thehive -g thehive -- /opt/thehive/bin/thehive \
	-Dconfig.file=/etc/thehive/application.conf \
	-Dlogger.file=/etc/thehive/logback.xml \
	-Dpidfile.path=/dev/null

With the default logs configuration, TheHive should log some outputs to stdout when starting

from thehive-feedback.

That gives us a lot of logs - is there anything that you are looking for? Immediately we notice this one;

`[info] a.c.s.ClusterSingletonManager [|] Singleton manager starting singleton actor [akka://application/system/singletonManagerLdapSync/LdapSync]

[info] a.c.s.ClusterSingletonManager [|] ClusterSingletonManager state change [Start -> Oldest]

[info] a.c.s.ClusterSingletonProxy [|] Singleton identified at [akka://application/system/singletonManagerLdapSync/LdapSync]`

from thehive-feedback.

The actor will start and then check if the config is set and the feature is enabled with the current license. This is a normal behavior.

We are looking for logs from NSS:

NSS DEBUG: Called _nss_debug_getpwuid_r with args (uid: 1000)   

from thehive-feedback.

We don't see that entry in the logs we have.

At this point, we are at the limit of our knowledge, though and we can't be sure that nss_debug is actually running properly

from thehive-feedback.

Unfortunately we have had to just remove the LDAP aspects of our server - the issue was completely repeatable, but we just couldn't find out what is happening. The workaround for us was removing libnss-ldap file and it stopped sending requests.

TCPDump would show the requests start and finish every time we started (or stopped) thehive.

Thanks for all the help in tracking this down.

from thehive-feedback.