Giter VIP home page Giter VIP logo

domain-monitor's Introduction

domain-monitor

Test Build Go Project GitHub go.mod Go version GitHub package.json version

Self-hosted server to monitor WHOIS records for specified domains, to help alert you of unwanted/unexpected changes to your domains and to help remind you when they need renewed.

If you use domain-monitor, make sure to abide by the terms of service of the TLD WHOIS server. Most forbid high-volume queries, marketing usage, and automated queries that are more than reasonably needed to register domain names.

To help you abide by WHOIS server TOS, domain-monitor barely queries the whois databases. domain-monitor acts in this way:

  1. Creates a local cache of the whois info for the domain, including time queried
  2. Updates a whois info only if one of these conditions is met:
  • No cached whois info for the domain
  • Reference info becomes 9 months old
  • It's 3 months until the domain expiry date
  • It's 2 months until the domain expiry date
  • It's 1 month until the domain expiry date
  • It's 2 weeks until the domain expiry date
  • It's 1 week until the domain expiry date
  • Manual update is requested
  • A DNS query of the FQDN returns a different set of name servers than the cached whois info

Installation

Docker

The docker image has a single volume for the files generated by the server.

Image Mount Contains
/app/data config.yaml, domain.yaml and WHOIS cache
Exposed Ports Used for
3124/tcp WEB GUI and API

docker run -p 127.0.0.1:3214:3214 -v ./data:/app/data ghcr.io/nwesterhausen/domain-monitor:1

Example docker-compose:

services:
  dm:
    image: ghcr.io/nwesterhausen/domain-monitor:1
    ports:
      - 127.0.0.1:3124:3124/tcp
    volumes:
      - ./data:/app/data:rw
version: "3.9"

Image is just nwesterhausen/domain-monitor, latest tag will be the most recent version, or pull by tagged version. The most recent old (non-go) version is tagged with 0.3.4.

docker run -p 127.0.0.1:3124:3124 -v ./data:/app/data docker.io/nwesterhausen/domain-monitor

Running Locally

Should by OS agnostic. Requires go and nodejs. Clone this repo and follow the build instructions. Then you can run the binary alongside the assets folder.

Configuration can be done via the configuration page of the web gui (default http://localhost:3124)

Config

There are two config files which you can edit yourself if you so choose.

config.yaml

A sample is provided as sample.config.yaml and on first run if you don't have an existing config.yaml, domain-monitor will create one with all default values. Any changes you made in the webgui persist in config.yaml.

App Settings

Port

Set the port used by the http server

Automated Whois Lookups

Enable or disable automated whois lookups. If disabled, whois lookups will only be done when manually requested.

Sample App Config
app:
  port: 3124
  automateWHOISRefresh: yes

Alerts

admin

Set what email should receive alerts from domain-monitor

sendalerts

Boolean, if false prevents all alerts from being sent.

Send Alert at 2 Months to Expiry

Boolean, if true, an alert will be sent when a domain is 2 months from expiry.

Send Alert at 1 Month to Expiry

Boolean, if true, an alert will be sent when a domain is 1 month from expiry.

Send Alert at 2 Weeks to Expiry

Boolean, if true, an alert will be sent when a domain is 2 weeks from expiry.

Send Alert at 1 Week to Expiry

Boolean, if true, an alert will be sent when a domain is 1 week from expiry.

Send Alert at 3 Days to Expiry

Boolean, if true, an alert will be sent when a domain is 3 days from expiry.

Send Daily Expiry Alerts

Boolean, if true, an alert will be sent every day for domains that expire within a week.

Sample Alerts Config
alerts:
  admin: [email protected]
  sendalerts: true
  send2MonthAlert: false
  send1MonthAlert: true
  send2WeekAlert: false
  send1WeekAlert: false
  send3DayAlert: true
  sendDailyExpiryAlert: false

SMTP

Set smtp settings for domain-monitor to use to send email alerts.

Host

The hostname of the SMTP server

Port

The port of the SMTP server

Secure

Boolean, if true, will force the SMTP connection to use TLS. If false, will use opportunistic TLS (so fallback to plaintext if the server doesn't support TLS)

Authuser

The username to use to authenticate with the SMTP server

Authpass

The password to use to authenticate with the SMTP server

Enabled

Boolean, if false, domain-monitor will not send any email alerts

FromName

The name to use in the "from" field of the email alerts

FromEmail

The email address to use in the "from" field of the email alerts

Sample SMTP Config
smtp:
  host: localhost
  port: 25
  secure: false
  authuser: [email protected]
  authpass: SECRET-PASS
  enabled: false
  fromName: Domain Monitor
  fromEmail: [email protected]

Scheduler

Set some schedule options for the WHOIS lookups.

WHOIS Cache Stale Interval

The number of days after which the WHOIS cache is considered stale and a new lookup will be done.

Use Standard WHOIS Refresh Schedule

Boolean, if true, domain-monitor will use a standard schedule for WHOIS lookups. If false, it will still perform the automated WHOIS lookup for stale, new domains and DNS changes, but will not perform regular lookups.

Sample Scheduler Config
scheduler:
  whoisCacheStaleInterval: 190
  useStandardWHOISRefreshSchedule: true

domain.yaml

Contains a single object (domains) which is a list of domains to monitor. Each domain has the following properties:

Property Type Description
name string Descriptive name for the domain entry
fqdn string FQDN for the domain in question. This is just host.tld
alerts bool If true, email alerts will be sent for this domain
enabled bool If true, whois lookups will be done (on the schedule described above) for this domain.

Development

Requirements:

  • golang
  • nodejs
  • pnpm
  • air go install github.com/cosmtrek/air@latest
  • templ go install github.com/a-h/templ/cmd/templ@latest

Development server

To run the development server, use the following command:

air

Air will take care of all the build steps whenever a change is detected.

Build steps:

  1. Install npm dependencies and "build" (copy the JS libraries to the assets folder).
pnpm i && pnpm build
  1. Compile the templates
templ generate
  1. Build the go binary
go build cmd/main.go

The binary requires the assets folder to be in the same directory as the binary.

domain-monitor's People

Contributors

connorlinfoot avatar dependabot[bot] avatar nwesterhausen avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

domain-monitor's Issues

Modify Docker Image Action

Update the docker creation GitHub action to also upload the created image to docker.io and then releases will be totally automated as far as container images go.

Promises

Change from callbacks to promises

Rename included config files

Instead of using default.yaml for the config file, use sample.config.yaml and sample.domain.yaml for the config files. On run check for "normal" versions (i.e. config.yaml and domain.yaml) and if they don't exist, copy over the samples.

Modifications made in the web gui can then affect these config files and the samples will be there for reference if needed.

Storage of WHOIS data

Solution to how to keep track of the whois data:

serialize it to JSON and store it in files. when needed it can then be easily read and sent to clients. also would allow an interval between whois requests to remain consistent if the server has an issue and restarts.

alerts for whois changes/problems should be looked at when the whois data is gathered. compared to the previous data / considered all new

[req] add sorting options to dashboard

Is your feature request related to a problem? Please describe.
It would be nice to be able to sort the dashboard in different ways besides just alphabetically.

Describe the solution you'd like
Sort dropdown with options like A-Z, Z-A, Expiring Soonest, Expiring Latest

Privacy issues

Hi Nicholas, thanks a lot for this great tool, I've been looking for something like it for a long time.

I just have a few questions that I'd love to get clarification on.

Firstly, is there any issue with the tool being used to monitor domains that you don't yet own? Is any verification needed from the domain owner for a domain to be monitored?

Secondly, do you or DomainMonitor have any awareness of the domains being monitored? Or put another way, does the tool ever call back to any servers other than the WHOIS APIs needed for the lookups? I've had a quick scan through the source and can't find anything that would, but just wanted to get an assurance of this. If this is not currently the case, will it ever change?

Thanks in advance and thanks again for your work on this, it's much appreciated.

No expiration and nameservers for .nl-TLD's

Al my domains that have a .nl-TLD do not show the expiration and nameservers.

Domains with other TLDs work as expected.

SIDN, the local registrar, does provide this information their whois.

Reminder alert and whois update schedule

Describe the bug

  1. Based on the wording in the readme, specifically the part and to help remind you when they need renewed in the first sentence led me to believe there would me email reminders when we got closer to the registration expiration. We are now within 60 days of expiration for one of my domains, and I have not received an email. I was wondering if there are really reminder emails and if so what the schedule for those would be.

  2. I also noticed that the whois query time is listed as 2022-04-05 for that domain. However, the readme suggests the whois reference would be updated if one of the listed conditions are met, and one of those conditions is It's 3 months until the domain expiry date. That does not seem to have happened here as we are 60 days from expiration, but the last query date is many months ago.

To Reproduce
Steps to reproduce the behavior:

  1. Create a docker container as described in the readme (from ghcr)
  2. Populate the yaml config and confirm the info in the webgui for various domains
  3. Wait until close to expiration on one of the domains

Expected behavior

  1. Whois query would be updated 3 months prior to expiration
  2. Expiration reminder email would be sent (at some point I'm not sure)

Screenshots
Screenshot 2023-03-05 203628

Desktop (please complete the following information):

  • OS: Unraid/Docker
  • Browser: Chrome
  • Version: 110

Smartphone (please complete the following information):
N/A

Additional context
I set the container up some months ago and it seems to be working and I see the info in the webgui for a number of domains. However, there was no test email function so I was not able to confirm whether email alerts were set up properly or working.

Thanks

Domain Cards

Domain Cards should give the following information:

  • registrar
  • name servers
  • registered date
  • registry expiration date
  • WHOIS db update time

The card header should reflect the status:

GREEN - all good
YELLOW - within 2 months of expiration
RED - WHOIS errors OR expired domain
(NO COLOR) - no WHOIS data yet

errors after adding domains and going to dashboard

Describe the bug
After adding new domains, going to the dashboard produces malformed cards and logs errors. I suspect this is do to how the dashboard is pulling domain info. Note that this same error tends to happen after restarting the server, when new WHOIS data is pulled for the domain in question.

To Reproduce
Steps to reproduce the behavior:

  1. Go to Configuration -> Domains
  2. Add a new domain
  3. Click save
  4. Go to Dashboard

Expected behavior
Expect to see new card for the domain with "No data yet" and no errors.

Instead it's just the title and errors are logged.

Screenshots
If applicable, add screenshots to help explain your problem.

image

Additional context
Errors logged:

2024/04/09 16:37:53 🆕 Adding domain: {Name: FQDN:nwest.wiki Alerts:true Enabled:true}
2024/04/09 16:37:53 🆕 Added domain nwest.wiki
2024/04/09 16:37:53 💾 Flushed domain table to domain.yaml
{"time":"2024-04-09T16:37:53.167350321Z","id":"","remote_ip":"::1","host":"127.0.0.1:3124","method":"POST","uri":"/domain/new","user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:124.0) Gecko/20100101 Firefox/124.0","status":200,"error":"","latency":622104,"latency_human":"622.104µs","bytes_in":46,"bytes_out":0}
{"time":"2024-04-09T16:37:54.865353526Z","id":"","remote_ip":"::1","host":"127.0.0.1:3124","method":"GET","uri":"/dashboard","user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:124.0) Gecko/20100101 Firefox/124.0","status":200,"error":"","latency":8567,"latency_human":"8.567µs","bytes_in":0,"bytes_out":0}
{"time":"2024-04-09T16:37:54.945633353Z","id":"","remote_ip":"::1","host":"127.0.0.1:3124","method":"GET","uri":"/domain/cards","user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:124.0) Gecko/20100101 Firefox/124.0","status":200,"error":"","latency":42960,"latency_human":"42.96µs","bytes_in":0,"bytes_out":0}
{"time":"2024-04-09T16:37:55.033027668Z","id":"","remote_ip":"::1","host":"127.0.0.1:3124","method":"POST","uri":"/whois/","user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:124.0) Gecko/20100101 Firefox/124.0","status":200,"error":"","latency":45174,"latency_human":"45.174µs","bytes_in":15,"bytes_out":0}
{"time":"2024-04-09T16:37:55.07976855Z","id":"","remote_ip":"::1","host":"127.0.0.1:3124","method":"POST","uri":"/whois/","user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:124.0) Gecko/20100101 Firefox/124.0","status":200,"error":"","latency":63940,"latency_human":"63.94µs","bytes_in":16,"bytes_out":0}
{"time":"2024-04-09T16:37:55.082577746Z","id":"","remote_ip":"::1","host":"127.0.0.1:3124","method":"POST","uri":"/whois/","user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:124.0) Gecko/20100101 Firefox/124.0","status":200,"error":"","latency":40405,"latency_human":"40.405µs","bytes_in":14,"bytes_out":0}
{"time":"2024-04-09T16:37:55.086241723Z","id":"","remote_ip":"::1","host":"127.0.0.1:3124","method":"POST","uri":"/whois/","user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:124.0) Gecko/20100101 Firefox/124.0","status":200,"error":"","latency":30447,"latency_human":"30.447µs","bytes_in":16,"bytes_out":0}
2024/04/09 16:37:55 WHOIS entry cache miss for nwest.wiki
echo: http: panic serving [::1]:44206: runtime error: invalid memory address or nil pointer dereference
goroutine 37 [running]:
net/http.(*conn).serve.func1()
        /usr/local/go/src/net/http/server.go:1898 +0xbe
panic({0x7630c0?, 0xa81ec0?})
        /usr/local/go/src/runtime/panic.go:770 +0x132
github.com/nwesterhausen/domain-monitor/handlers.(*WhoisHandler).GetCard.WhoisDetail.func1({0x85ccb0?, 0xc00008f0e0?}, {0x85a920, 0xc0002e4d20?})
        /workspaces/domain-monitor/views/domains/domain-dashboard_templ.go:185 +0x21f
github.com/a-h/templ.ComponentFunc.Render(0xa833e0?, {0x85ccb0?, 0xc00008f0e0?}, {0x85a920?, 0xc0002e4d20?})
        /go/pkg/mod/github.com/a-h/[email protected]/runtime.go:41 +0x37
github.com/nwesterhausen/domain-monitor/handlers.View({0x862378, 0xc000183720}, {0x85a120, 0xc000195c00})
        /workspaces/domain-monitor/handlers/routes.go:76 +0x170
github.com/nwesterhausen/domain-monitor/handlers.(*WhoisHandler).GetCard(0xc00009c348, {0x862378, 0xc000183720})
        /workspaces/domain-monitor/handlers/whois.handler.go:31 +0x172
github.com/labstack/echo/v4.(*Echo).add.func1({0x862378, 0xc000183720})
        /go/pkg/mod/github.com/labstack/echo/[email protected]/echo.go:582 +0x4b
github.com/labstack/echo/v4/middleware.LoggerWithConfig.func2.1({0x862378, 0xc000183720})
        /go/pkg/mod/github.com/labstack/echo/[email protected]/middleware/logger.go:126 +0xd8
github.com/labstack/echo/v4.(*Echo).ServeHTTP(0xc0000ec908, {0x85c5f0, 0xc0002e4d20}, 0xc0003010e0)
        /go/pkg/mod/github.com/labstack/echo/[email protected]/echo.go:669 +0x327
net/http.serverHandler.ServeHTTP({0x85b6f0?}, {0x85c5f0?, 0xc0002e4d20?}, 0x6?)
        /usr/local/go/src/net/http/server.go:3137 +0x8e
net/http.(*conn).serve(0xc00007ca20, {0x85cc78, 0xc00026a8d0})
        /usr/local/go/src/net/http/server.go:2039 +0x5e8
created by net/http.(*Server).Serve in goroutine 1
        /usr/local/go/src/net/http/server.go:3285 +0x4b4

Cannot read property 'domains' of undefined

Describe the bug
When adding a domain with whois caching, the page just keeps saying: No WHOIS cache for domain.tld
Running Docker Image: nwesterhausen/domain-monitor

domain-monitor    | (node:1) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'domains' of undefined
domain-monitor    |     at sendCachedWHOISData (/app/lib/sockets.js:25:14)
domain-monitor    |     at /app/lib/sockets.js:120:9
domain-monitor    | (node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 8)

To Reproduce

  • Run Docker image nwesterhausen/domain-monitor
  • Add a 2 domains.
  • For each add, the error above will be thrown and neither domains will update their whois data / status.

Expected behavior
The domain status should update.

Screenshots
Screenshot 2021-07-25 at 10 26 34

Desktop (please complete the following information):

  • Docker on Ubuntu 20.04 LTS

WHOIS cache miss for fubar.com

Describe the bug
No matter what domain I enter, I get "WHOIS cache miss for fubar.com". I have tried waiting 24 hours based on a different ticket.
Running via docker
whois data folder set to 777

To Reproduce
Steps to reproduce the behavior:

  1. Add a domain
  2. Get a error
  3. 8/18/2021, 9:31:10 AM: Socket Connection Established
    8/18/2021, 9:31:10 AM: Configuration data received.
    8/18/2021, 9:31:10 AM: Domain configuration received.
    8/18/2021, 9:31:10 AM: WHOIS cache miss for bartdyer.com

Expected behavior
Whois to load

Screenshots
image

Test alerts from Web UI

There is no way for me to check if the alerts will work if needed, It would be really useful and assuring if we had a way to verify the SMTP connection by sending a test email from the Web UI.

Most apps have this feature these days, like NextCloud, ValutWarden etc.

How does this sound ?

Support for ARM64

Most of my Docker containers run on Raspberry Pi or in a free instance of Oracle Cloud. Unfortunately, they only support ARM64, so your software always exits with this error: exec /usr/local/bin/docker-entrypoint.sh: exec format error.

I've found some instructions for adding multiple platform support:
https://docs.docker.com/build/building/multi-platform/

Do you think it'd be possible to tweak your software to play nice with different architectures like ARM64? That'd be a huge help!

WSL - manual config directory

Under WSL the config folder must be created manually.
Node has no authorization to create folders on its own due to the lack of access rights.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.