Compilation failures are observed on FreeBSD, due to missing syscall symbols:

laitos/main_util.go:26:13: undefined: syscall.Mlockall
laitos/main_util.go:175:58: undefined: syscall.TCGETS
laitos/main_util.go:185:65: undefined: syscall.TCSETS

Yeah.. So. Progressive web applications are a thing on phones nowdays, been for a while. And desktops support same functionality to a certain extent.

So I had this feature idea: Laitos hosted notification service done with javascript using:

• Progressive web application technology on phones to register persistent service workers on phones
• Service workers on desktop
• http(S) as the rest backend
• http(S) upgrade connections - websockets as the backend.

ideas of features of these notifications:

• Send & forget
• send & require confirmation
  IE: send notification -> user clicks it -> notification launches another action (laitos http page or something) that includes ticking off the notification, otherwise keep sending it periodically to the user
• Other applications should be able to send these notifications using rest/websockets (ie, bash script that can just fire off curl requests 😁)
• Notifications should have importance priority (no vibrating phones or playing notification sounds at night if low priority)
• SIMPLE this needs to be veeeeery simple to the end user and developers sending these notifications

Needed testing:

• websockets vs time.tick-like XHR requests - Mainly for the convenience/speed and the battery consumption on mobile, on desktop websockets are great. We can even let the user decide which one to pick in fact
• other things?

Should this be a stand alone thing or included in laitos? (I'm eventually going to make a standalone service out of this idea, but Laitos would be great place to actually test this out)

If this gets wind under its wings, I wouldn't mind implementing the clientside stuff (JS, progressive app, everything related to the client side), would someone be interested in implementing the laitos side of things?

👍 / 👎?

stuff related to this.
https://developers.google.com/web/progressive-web-apps/
https://developers.google.com/web/fundamentals/primers/service-workers/
https://developers.google.com/web/fundamentals/codelabs/offline/

The browser-in-browser window height input resets itself to 1024 no matter what value the user specifies.

Looks like I forgot to do stats for SNMP requests.

If the password web server is used to launch laitos from encrypted data archive, laitos doesn't automatically release its ramdisk and remove its temporary directory after exiting.

I need a way to do "atexit".

In health check output, present the total count and average processing time of

• Succeeded HTTP requests (2xx)
• Failed HTTP requests
• DNS requests in both TCP and UDP server
• SMTP conversations
• .. other daemons that could present this information

It would be nice to see environment info, especially memory consumption, after unlock page has successfully done its duty.

It is often quite inconvenient to investigate a laitos panic right away, but then its log file can be gone in a day or two. Therefore, tell supervisor to buffer the latest ~4KB of program output, and send it via the recovery notification email.

Would it be possible to make some nice rpms packages for Laitos?

After today's work, skype bot doesn't seem to work anymore. No error is logged on server side.

the health report already has load and memory usage info, which is very useful, and it would be even better if complemented by disk usage as well, that will help identifying even more resource issues before they cause more trouble.

Several daemons such as DNS server need to know server's public IP address, this is conducted only once when program starts up. However, if server IP changes, the daemons will continue to operate based on stale invalid IP address.

Correct the behaviour so that every daemon that needs public IP address will fetch it periodically.

Implement the following behaviour in supervisor:

• Upon start, fork laitos main process that runs all daemons.
• When the main process crashes, restart the main process up to two times.
• Should more crashes occur, shed daemons one by one in consecutive crashes, in the following order:
  • maintenance
  • dnsd
  • sockd
  • smtpd
  • httpd
  • insecurehttpd
  • telegram
  • plainsocket
• If the last remaining daemon still crashes, then cease shedding daemons and only relaunch main process with all daemons enabled in case of further crash.

The rationale works very well for aircraft checklists, I suppose it should also work in this case.

Small feature request:

sudo ./laitos -gomaxprocs 16 -config sample-config.json.txt -daemons dnsd,httpd,insecurehttpd,maintenance,plainsocket,serialport,simpleipsvcd,smtpd,snmpd,sockd,telegram

is rather long, but breaking it up with \ newlines and spaces broke it. Making the command more "whitespace-tolerant" would be nice.

In the likely case of system upgrade triggered reboot, or the unlikely case of fault triggered reboot, a server with encrypted program data will need to be manually unlocked by entering decryption password. It is a small hassle.

If there are more than one instances of laitos deployed and online at a time, it would be quite convenient to tell an online laitos instance to decrypt an instance awaiting password input.

hi,

according to https://github.com/HouzuoGuo/laitos/blob/c8c46a1901d6eb72027c005c950cdaeaebbfcf15/daemon/smtpd/smtp/smtp.go, daemon/smtpd/smtp contains code from @siebenmann project https://github.com/siebenmann/smtpd, released under GPL-v3.

thus, laitos can't be released under MPL. it needs to be GPL.

Hi @HouzuoGuo,

Would you write a how to for Outgoing mail configuration for SMTP server (via SSL Mail) please?

Azure does not block SSL Mail port 465. I can test it out after you've written it.
If you need Go code that can send SSL Mail, I can look it up for you, but it's the most common one that I used and connect to SMTP server via SSL Mail and send mail just fine.

thanks

The default selinux policy on centos prevents slimerjs from functioning. Disable selinux for that case.

i wanted to share some small points from my pov about laitos doc, that could be improved afaik.

• putting documentation in a wiki, from my pov can decrease contributors to improve the documentation of the project. At least i would prefer the doc and or Readme.md ( each pr can improve documentation directly in this way)

• on the getting started, would be nice to have at least a minimal to go example/muster that would not require the full reading of doc/conf files, just a simple quickstart

In that case, replace the command processor by a non-working variant and log the error.

🎉 to @HouzuoGuo and all

• As of now laitos works the same way for all Twilio phone number connected with it. Think of a way to distinguish between the connected phone numbers, and allow configured app command shortcuts to execute on backup numbers (See also #44)
• Allow certain configured app command shortcuts to execute even if global lockdown is in effect (see also #43)
• Extend the existing "recurring app command" feature to execute alarm detection parameters, and sends out notifications when an alarm is raised (see also #36)
• Create a new daemon for generating push notifications, each notification entry can be a piece of text collected from web API invocation, a generated alarm, response from a recurring app command, or something else (see also #29)

Tonight Unity office in Helsinki kindly hosted the October golang meet-up, during which I gave a short introduction and live demo of couple of laitos features. A fellow geek suggested that self destruct command shall remain accessible even when lockdown is in effect, which is indeed a good idea. Find a way to get it done 👍

The error occurs on windows exclusively and happens rarely. There is no clue as to what triggers the error - it may happen as soon as daemons are started or as late as after hundreds of hours of uptime. The error leads to a panic which is subsequently recovered by supervisor.

Unfortunately supervisor for some reason won't capture the entire stack trace, and the only clue so far is an error simply stating "string with NUL passed to StringToUTF16".

• The total processing seconds accounted for daemon stats appear to be abnormally high during a benchmark run. Is this an indication of some sort of FD leak?
  (Comment: more probably the daemons may not have closed client socket early enough, and they had to wait for IO timeout)

• Benchmark isn't using UDP as much as it uses TCP, very strange.

Nowadays telegram bot API gives an error HTTP response when more than one servers are polling on the same account:

HTTP 409: {"ok":false,"error_code":409,"description":"Conflict: terminated by other getUpdates request; make sure that only one bot instance is running"}

Nevertheless, the API succeeds in retrieving pending message in the next round. Therefore, suppress logging of the conflict HTTP API response.

systemd-resolved listens on port 53 on 127.0.0.53, which gives dnsd a startup failure.

Due to rather hideous issues deep inside laitos or one of the popular cloud platforms, laitos DNS and sock daemon appears to grow "tired" after unpredictable amount of uptime, generally between 2-7 days.

Symptom being that both daemons begin to perform very slowly at around the same time, yet none of the daemon reports rate limit being exceeded. One very rare case even caused laitos to crash - and subsequently supervisor did its proper job to relaunch laitos. Unfortunately the program log was lost before it could be inspected.

So while this issue demands more investigation, implement a temporary measure in supervisor that relaunches laitos regularly, interval of which is defined by user in configuration.

Provide RSS reader via the toolbox feature interface, and by default configure the reader with reputable sources such as BBC.

not sure what took place, in one instance apt-get upgrade did not install any package upgrade.

Several HTTP endpoints also have API credentials that would be nice to validate regularly.

ElasticBeanstalk is quite weird, it periodically clears /tmp directory and leaves nothing but supervisor socket behind, this causes the utility program directory to disappear. Therefore, periodically re-copy those bundled programs similar to how the pseudo-random number generator is reseeded.

The following crash was captured recently and has never been seen before:

github.com/HouzuoGuo/laitos/toolbox.(*WolframAlpha).SelfTest(0xc4200017e8, 0x12031, 0x0)
/home/howard/gopath/src/github.com/HouzuoGuo/laitos/toolbox/wolframalpha.go:30 +0x30a
github.com/HouzuoGuo/laitos/toolbox.(*FeatureSet).SelfTest.func1(0xc4204b1800, 0xc42036f300, 0xc4204b1810, 0xaea440, 0xc4200017e8)
/home/howard/gopath/src/github.com/HouzuoGuo/laitos/toolbox/feature_set.go:70 +0x49
created by github.com/HouzuoGuo/laitos/toolbox.(*FeatureSet).SelfTest
/home/howard/gopath/src/github.com/HouzuoGuo/laitos/toolbox/feature_set.go:69 +0x190

The information HTTP endpoint does way too many things, consequently it is way too slow to load.

It will be better to remove configuration validation from the info output, so that the output solely focuses on general stats.

As noticed and recovered by supervisor:

panic: http: multiple registrations for /
goroutine 384 [running]:
net/http.(*ServeMux).Handle(0xc4203a94d0, 0xc4200136a8, 0x1, 0x9bd820, 0xc4202b5460)
/home/howard/go/src/net/http/server.go:2270 +0x627
net/http.(*ServeMux).HandleFunc(0xc4203a94d0, 0xc4200136a8, 0x1, 0xc4202b5460)
/home/howard/go/src/net/http/server.go:2302 +0x55
github.com/HouzuoGuo/laitos/daemon/httpd.(*Daemon).Initialise(0xc4201326e0, 0xc42047a6f0, 0xc4203263c0)
/home/howard/gopath/src/github.com/HouzuoGuo/laitos/daemon/httpd/httpd.go:175 +0x409
github.com/HouzuoGuo/laitos/launcher.Config.GetHTTPD.func1()
/home/howard/gopath/src/github.com/HouzuoGuo/laitos/launcher/config.go:308 +0x512
sync.(*Once).Do(0xc420419ab0, 0xc4201e2690)
/home/howard/go/src/sync/once.go:44 +0xbe
github.com/HouzuoGuo/laitos/launcher.Config.GetHTTPD(0xc420000f00, 0xc420013840, 0xb, 0xc42005f400, 0x11, 0x9dd, 0xc420013830, 0x6, 0xc42008c140, 0x45, ...)
/home/howard/gopath/src/github.com/HouzuoGuo/laitos/launcher/config.go:206 +0x89
main.main.func2(0xc4203e9140, 0xc420081b00)
/home/howard/gopath/src/github.com/HouzuoGuo/laitos/main.go:264 +0x57
created by main.main

Azure is similar to GCE and does not allow outgoing connections to port 25.

The new daemon collects configuration input on alarm name, methods of notification (telephone, SMS, email, satellite, or any combination), and trigger conditions such as:

• Disk usage on / (built-in)
• Free memory (built-in)
• Shell script that normally exits with status 0 (customised input)

Then, the daemon runs in background and periodically checks the trigger conditions, interval of which is also customised by configuration input.

Consecutive failures in trigger conditions will not cause more notifications to be delivered, however, when trigger conditions are no longer present, notifications will tell that alarm conditions are no longer triggered.

For example:

An alarm called "disk usage threshold exceeded" is configured to send notification via telephone call, SMS, and email, the alarm checks disk usage on / at 3 minutes interval and triggers when the usage exceeds 95%.

This crash has never been observed in the past:

crypto/cipher.(*ctr).refill(0xc4210b2eb0)
/home/howard/go/src/crypto/cipher/ctr.go:54 +0x1bf
crypto/cipher.(*ctr).XORKeyStream(0xc4210b2eb0, 0xc42110f081, 0x1ccd, 0x1ccd, 0xc421159581, 0x1ccd, 0x1ccd)
/home/howard/go/src/crypto/cipher/ctr.go:76 +0x1db
github.com/HouzuoGuo/laitos/daemon/sockd.(*Cipher).Encrypt(0xc420773a40, 0xc42110ea00, 0x234e, 0x234e, 0xc421158f00, 0x234e, 0x234e)
/home/howard/gopath/src/github.com/HouzuoGuo/laitos/daemon/sockd/sockd.go:193 +0x6f
github.com/HouzuoGuo/laitos/daemon/sockd.(*TCPCipherConnection).Write(0xc420be10a0, 0xc421158f00, 0x234e, 0x234e, 0x0, 0x0, 0xb25600)
/home/howard/gopath/src/github.com/HouzuoGuo/laitos/daemon/sockd/tcp.go:135 +0x111
github.com/HouzuoGuo/laitos/daemon/sockd.PipeTCPConnection(0xaebdc0, 0xc420cec1c8, 0xaebd00, 0xc420be10a0, 0xaebd00)
/home/howard/gopath/src/github.com/HouzuoGuo/laitos/daemon/sockd/tcp.go:246 +0x1f8
github.com/HouzuoGuo/laitos/daemon/sockd.(*TCPCipherConnection).HandleTCPConnection(0xc420be10a0)
/home/howard/gopath/src/github.com/HouzuoGuo/laitos/daemon/sockd/tcp.go:234 +0x41a
created by github.com/HouzuoGuo/laitos/daemon/sockd.(*Daemon).StartAndBlockTCP
/home/howard/gopath/src/github.com/HouzuoGuo/laitos/daemon/sockd/tcp.go:58 +0x46e

Create a new self destruct command that executes the following:

1. Connect the incoming caller to a number specified in the input self-destruct command.
2. Execute self-destruct command in the meanwhile.
3. The outgoing call is sustained by Twilio infrastructure, the server peacefully RIPs as the final call is made.

Thanks to our lovely colleague Riku for the idea!