Comments (49)
NICE it is finally working !!!!!
After 4 days of troubleshooting
THANKS @jasonbosco and @britisharmy for your assistance :)
Also I noticed that it fails to write empty arrays
Is this as intended?
from firestore-typesense-search.
Sure!
So the process goes like this for me:
Firebase extension -> my SSL (by certbot) "domain.xxx" -> Reverse proxy to my VM local IP address
My reverse proxy settings on my domain's xxx-le-ssl.conf file
ProxyPreserveHost On
ProxyPass "/" "http://local IP:8108/"
ProxyPassReverse "/" "http://local IP:8108/"
AllowEncodedSlashes On
The config for my typesense
api-address: local IP
port: 8108
api-key: value here
enable-cors: true
Then on the firebase extension, use the ssl domain name like "sampledomain.xxx"
Make sure to enable the necessary modules in apache for reverse proxying
https://www.digitalocean.com/community/tutorials/how-to-use-apache-http-server-as-reverse-proxy-using-mod_proxy-extension
Apache docs about reverse proxying that might help understand this stuff
https://httpd.apache.org/docs/2.4/howto/reverse_proxy.html
Also, if you are using the domain for other subdirectories like domain.xxx/otherpath
You want to make an exception for that path to not be included in reverse proxy requests
ex.
ProxyPass /sample/path/ !
https://stackoverflow.com/questions/39666686/proxypass-but-exclude-certain-sub-directory/39666874
from firestore-typesense-search.
I made it to work with letsencrypt, email me i cann share my config,it encrypts but you shall have to switch to localhost for inserts and curl only for upserts and reading remotely.
from firestore-typesense-search.
Thanks!
I emailed you at [email protected]
from firestore-typesense-search.
The extension expects Typesense to be running on port 443 with https enabled.
So you need to start the Typesense server with the ssl key, ssl cert and api port (443) specified in the configs. More info about these params here: https://typesense.org/docs/overview/benchmarks.html
You can definitely use certbot / LetsEncrypt SSL certs. Self-signed certs don’t work.
from firestore-typesense-search.
@britisharmy Curious why you had to do this:
but you shall have to switch to localhost for inserts and curl only for upserts and reading remotely.
from firestore-typesense-search.
The extension expects Typesense to be running on port 443 with https enabled.
So you need to start the Typesense server with the ssl key, ssl cert and api port (443) specified in the configs. More info about these params here: https://typesense.org/docs/overview/benchmarks.html
You can definitely use certbot / LetsEncrypt SSL certs. Self-signed certs don’t work.
I see. Now my problem is that I have a virtual host in my server which has certbot SSL installed and I tried to use it as my API address with its certificates
api-address = domain.com
api-port = 8108
data-dir = /var/lib/typesense
api-key = value here
log-dir = /var/log/typesense
enable-cors = true
ssl-certificate = /etc/letsencrypt/live/domain.com/fullchain.pem
ssl-certificate-key = /etc/letsencrypt/live/domain.com/privkey.pem
I can now curl with https in port 8108
Now when I change the port to 443, my typesense server fails to start when I restart it :(
I am guessing that I'm doing something wrong in setting up my SSL but I don't know how exactly to set it up in this case
from firestore-typesense-search.
@britisharmy Curious why you had to do this:
but you shall have to switch to localhost for inserts and curl only for upserts and reading remotely.
I am almost done, updating the multiple catalogs i have and then i shall revisit the ssl issue and email you. I don't recall why, but i made some notes somewhere, once the entire catalog is up, i am going to shoot you an email.
from firestore-typesense-search.
@echo-slam-jam You want to use an IP address for api-adddress, not domain name.
Could you also post the logs from /var/log/typesense/typesense.log?
from firestore-typesense-search.
Do I need to post the whole log file?
from firestore-typesense-search.
Just since the last restart. You should see a line saying Starting Typesense when you restart Typesense.
from firestore-typesense-search.
from firestore-typesense-search.
@echo-slam-jam You want to use an IP address for api-adddress, not domain name.
Could you also post the logs from /var/log/typesense/typesense.log?
OHH so that's one thing I needed to know haha
Actually I tried to input the IP address on the host at the firebase extension and the error was about not being recognized by the certificate
ext-firestore-typesense-search-indexToTypesenseOnFirestoreWrite
Request #1628516055899: Request to Node 0 failed due to "ERR_TLS_CERT_ALTNAME_INVALID Hostname/IP does not match certificate's altnames: IP: my public IP is not in the cert's list: "
from firestore-typesense-search.
Will continue to reply tomorrow since it's midnight here in the Philippines
I guess the problem I have now is setting up SSL for this case?
from firestore-typesense-search.
Request #1628516055899: Request to Node 0 failed due to "ERR_TLS_CERT_ALTNAME_INVALID Hostname/IP does not match certificate's altnames: IP: my public IP is not in the cert's list: "
It sounds like you've generated your certificate using a domain name, but you've used your IP address when configuring the Firebase extension. Instead, you want to use the domain name in the Firebase extension - the same domain name you used when generating the SSL cert.
Separately, those logs seem like they're from before you had changed the port to 443. If the above still doesn't work, could you change the port, restart Typesense and then capture the logs from that point on?
from firestore-typesense-search.
My config
; Typesense Configuration
[server]
api-address = xxx.xx.xxx.xxx
api-port = 443
data-dir = /var/lib/typesense
api-key = value here
log-dir = /var/log/typesense
enable-cors = true
ssl-certificate = /etc/letsencrypt/live/domain/cert.pem
ssl-certificate-key = /etc/letsencrypt/live/domain/privkey.pem
from firestore-typesense-search.
May I know which cloud provider you’re running this on? It looks like the public IP address is not directly accessible on the instance. May be try using the private IP of the instance for api-address?
from firestore-typesense-search.
May I know which cloud provider you’re running this on? It looks like the public IP address is not directly accessible on the instance. May be try using the private IP of the instance for api-address?
The virtual machine is actually owned by the university where I am studying at.
They are also the ones responsible for giving me the public DNS which I used.
I tried the private IP which also didn't work
What might be the cause of this accessibility problem? I can try to bring this up to the manager of the server.
from firestore-typesense-search.
The server is actually hosted by the university where I study at.
from firestore-typesense-search.
Notice how the log here says "Cannot assign requested IP". That's the underlying issue.
- When you run
ifconfig
does that show you the public IP? - Does
ifconfig
show you an address like10.x.x.x
or192.168.x.x
?
from firestore-typesense-search.
from firestore-typesense-search.
Could you post the logs, once you update the IP address in the Typesense configs to 10.207.9.46
and restart the Typesense process?
from firestore-typesense-search.
Ohhhh I can see that it says that the address is already in use
from firestore-typesense-search.
Yup, so some other process is already using port 443. You can try lsof -i :443
to see which process that is.
from firestore-typesense-search.
Hmmm the directory is the same as the directory where I hosted the file storage that I use for serving the files from this server
So does this mean that I cannot host this service anymore due to that?
from firestore-typesense-search.
Ah, if you're already using 443 for Apache, another thing you could do is run Typesense with http
protocol on say port 8108
(the default). Then setup Apache to handle HTTPS on port 443, and then have Apache reverse proxy to Typesense on port 8108.
from firestore-typesense-search.
Ohh nice. This is what @britisharmy also suggested.
Are these steps for elasticsearch applicable?
https://devdocs.magento.com/guides/v2.4/install-gde/prereq/es-config-apache.html
from firestore-typesense-search.
That link doesn't seem to cover https. This looks more recent and talks about https setup as well: https://www.digitalocean.com/community/tutorials/how-to-use-apache-http-server-as-reverse-proxy-using-mod_proxy-extension
The key thing to change in that guide is:
ProxyPass / http://0.0.0.0:8080/
ProxyPassReverse / http://0.0.0.0:8080/
becomes
ProxyPassReverse / http://0.0.0.0:8108/
from firestore-typesense-search.
I see.
Do I need to follow the whole steps starting from Modifying The Default Configuration
Or just Enabling SSL Reverse-Proxy Support
from firestore-typesense-search.
Everything except "Enabling Load-Balancing"
from firestore-typesense-search.
ok I'll get back to you after I do it
from firestore-typesense-search.
That link doesn't seem to cover https. This looks more recent and talks about https setup as well: https://www.digitalocean.com/community/tutorials/how-to-use-apache-http-server-as-reverse-proxy-using-mod_proxy-extension
The key thing to change in that guide is:
ProxyPass / http://0.0.0.0:8080/ ProxyPassReverse / http://0.0.0.0:8080/
becomes
ProxyPassReverse / http://0.0.0.0:8108/
This is my config for the 000-default
<VirtualHost :>
ProxyPreserveHost On
# Servers to proxy the connection, or;
# List of application servers:
# Usage:
# ProxyPass / http://[IP Addr.]:[port]/
# ProxyPassReverse / http://[IP Addr.]:[port]/
# Example:
ProxyPass / http://10.207.9.46:8108/
ProxyPassReverse / http://10.207.9.46:8108/
ServerName localhost
Listen 443
NameVirtualHost *:443
<VirtualHost *:443>
SSLEngine On
# Set the path to SSL certificate
# Usage: SSLCertificateFile /path/to/cert.pem
SSLCertificateFile /etc/letsencrypt/live/my domain/cert.pem
# Servers to proxy the connection, or;
# List of application servers:
# Usage:
# ProxyPass / http://[IP Addr.]:[port]/
# ProxyPassReverse / http://[IP Addr.]:[port]/
# Example:
ProxyPass / http://10.207.9.46:8108/
ProxyPassReverse / http://10.207.9.46:8108/
# Or, balance the load:
# ProxyPass / balancer://balancer_cluster_name
I cannot restart apache2 because of the error
Line 16 is Listen 443
from firestore-typesense-search.
The certificate cant work on a naked ip. You shall need a sub domain or a tld.
from firestore-typesense-search.
Hmmm
So I managed to successfully restart apache2.
- I removed Listen 443
- I removed *NameVirtualHost :443
This is now my config for 000-default
<VirtualHost :>
ProxyPreserveHost On
# Servers to proxy the connection, or;
# List of application servers:
# Usage:
# ProxyPass / http://[IP Addr.]:[port]/
# ProxyPassReverse / http://[IP Addr.]:[port]/
# Example:
ProxyPass / http://10.207.9.46:8108/
ProxyPassReverse / http://10.207.9.46:8108/
ServerName localhost
SSLEngine On
# Set the path to SSL certificate
# Usage: SSLCertificateFile /path/to/cert.pem
SSLCertificateFile /etc/letsencrypt/live/my-domain/cert.pem
SSLCertificateKeyFile /etc/letsencrypt/live/my-domain/privkey.pem
# Servers to proxy the connection, or;
# List of application servers:
# Usage:
# ProxyPass / http://[IP Addr.]:[port]/
# ProxyPassReverse / http://[IP Addr.]:[port]/
# Example:
ProxyPass / http://10.207.9.46:8108/
ProxyPassReverse / http://10.207.9.46:8108/
# Or, balance the load:
# ProxyPass / balancer://balancer_cluster_name
And here is my config for typesense
GNU nano 4.8 /etc/typesense/typesense-server.ini ; Typesense Configuration
[server]
api-address = 10.207.9.46
api-port = 8108
data-dir = /var/lib/typesense
api-key = value here
log-dir = /var/log/typesense
enable-cors = true
Now both services are running
Are these configs correct?
I can now curl "HTTP://10.207.9.46:8108/health"
I can't curl https version
What should I be able to curl?
How do I verify if my reverse proxy works?
from firestore-typesense-search.
You only need
ProxyPassReverse / http://10.207.9.46:8108/
and NOT
ProxyPass / http://10.207.9.46:8108/
.
Essentially, the idea is that you're running multiple web-servers behind Apache, one is your existing site, and the other is Typesense. Depending on what hostname shows up in the HTTP headers, Apache will either send the request to your existing site or to Typesense. HTTPS/SSL is handled by Apache and the connection to Typesense uses regular http on port 8108.
In this setup, the idea is that if you've configured say typesense.domain.com to reverse proxy to http://10.207.9.46:8108
, then, you should be able to curl https://typesense.domain.com/health and Apache will send that request to Typesense internally.
from firestore-typesense-search.
You only need
ProxyPassReverse / http://10.207.9.46:8108/
and NOT
ProxyPass / http://10.207.9.46:8108/
.Essentially, the idea is that you're running multiple web-servers behind Apache, one is your existing site, and the other is Typesense. Depending on what hostname shows up in the HTTP headers, Apache will either send the request to your existing site or to Typesense. HTTPS/SSL is handled by Apache and the connection to Typesense uses regular http on port 8108.
In this setup, the idea is that if you've configured say typesense.domain.com to reverse proxy to
http://10.207.9.46:8108
, then, you should be able to curl https://typesense.domain.com/health and Apache will send that request to Typesense internally.
If I understand correctly, do I need to create a subdomain for my domain which I will call, say, "typesense.domain" which I will reverse proxy to http://10.207.9.46:8108
or can I just use 000-default like what digital ocean did?
from firestore-typesense-search.
Or can I also do it like this
I will just assign the specific /typesense path to reverse proxy to port 8108
from firestore-typesense-search.
Yeah, either of those should work. You might have to do some URL re-writing so Typesense doesn't see the /typesense/
portion of the URL... Not sure if that's how Apache reverse-proxying works out of the box, or if more configuration is needed.
from firestore-typesense-search.
Ok
So I have now successfully configured my reverse proxy and curl "https://mydomain/typesense/health" now returns ok:true
I can also see it through web browser so I assume that this is already working
Now I went to the firebase extension and
typesense host: mydomain/typesense
so I changed it to
typesense host: mydomain/typesense/collections
because that is where my collection is
What should be the correct host?
is there still something missing from my server setup?
This is my reverse proxy setup and I think it works fine
from firestore-typesense-search.
Oh hang on, my bad. While Typesense itself can be hosted under a path, the Firebase extension requires Typesense to be hosted under the root domain (and not in a sub-path). So you'd need to create a new virtual host in Apache and move the reverse proxy config under that, and setup a different sub-domain just for Typesense.
from firestore-typesense-search.
Hmmm let me clarify.
Currently, the process goes like this
extension -> my SSL domain/typesense subpath (is this what you were talking about?) -> reverse proxy to 10.207.9.46:8108
which does not work
Can you illustrate in terms of this process chain?
from firestore-typesense-search.
You'd have to set it up this way:
Extension -> https://typesense-sub.domain (setup in Apache as a separate Virtual Host with its own SSL cert) -> reverse proxy to 10.207.9.46:8108
from firestore-typesense-search.
Hmm can I still use my ssl domain like this?
extension -> https://mydomain/typesense -> reverse proxy to newly created https://typesense -> reverse proxy to 10.207.9.46:8108
So in this way I wont have to request for another public DNS from the university
from firestore-typesense-search.
That won’t work with the Firebase extension. So you need a new domain/sub-domain just for Typesense, but it can still run on the same server.
from firestore-typesense-search.
Ok I just checked with the server handler and I can fully use the root of mydomain
So now I don't need the subpath anymore and my process now is this:
extension -> my SSL domain (no more subpath) -> reverse proxy to 10.207.9.46:8108
from firestore-typesense-search.
NICE it is finally working !!!!!
After 4 days of troubleshootingTHANKS @jasonbosco and @britisharmy for your assistance :)
Also I noticed that it fails to write empty arrays
Is this as intended?
Idk why but now it accepts empty arrays haha
from firestore-typesense-search.
Amazing! Kudos to you for your persistence! 🙌 🙌
It would be awesome if you're able to share the final Apache configs that worked for you, for the benefit of other folks who might have a need to reverse proxy through Apache in the future.
from firestore-typesense-search.
Is there a reason why Typesense extension doesn't allow to specify a custom port? Is this a Firebase requirement?
from firestore-typesense-search.
@braincomb It's not a Firebase requirement, but a security best practice: #22 (comment)
from firestore-typesense-search.
Related Issues (20)
- Document flattening throws error with deeper nested structures HOT 2
- Return unflattened objects from search HOT 3
- `indexToTypesenseOnFirestoreWrite.js` does not delete documents when the docId has a hash `#` HOT 2
- Typesense v0.24.+ Nested fields support HOT 1
- Collections with ids that contain spaces cause errors in the emulated environment. HOT 1
- Add option to map field names
- Update documentation on how to contribute to this package
- Typesense Firebase Extension issue HOT 14
- Can we make most of the traffic go though a certan node? HOT 1
- Feature Request add collection parameter to indexed document to allo filtering
- Port selection
- Add support for multiple api keys HOT 1
- Support multiple collections HOT 2
- Support Node.js 18 HOT 6
- Function Names Generated by Typesense Extensions Are Too Long, Causing Setup Issues in Firebase Locally HOT 4
- Getting started with Cloud offering
- Filter records which are indexed HOT 1
- Fresh plugin installation fails with 401 - a valid `x-typesense-api-key` header must be sent. HOT 6
- Warning on deleted documents HOT 3
- Added field in under Extension configuration did not trigger Typesense update HOT 9
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from firestore-typesense-search.