hashicorp-forge / hermes Goto Github PK

Right now the README appears mostly geared to running on localhost. There is a small bit for production setup but it says "More to come here..."

Looking forward to the more to come!

Node.js 14 EOL is coming soon (April 2023 for Security release, active support is already over).

Consider to update it to 16 or 18

If you're looking at this, thanks for your interest in Hermes! We (HashiCorp Labs, the maintainers of Hermes) are a very small team. We will do our best to respond to issues created here in a timely manner, but please understand that it currently may take us up to a week depending on our internal commitments. Thanks for your patience! 🙏

Discussed in https://github.com/hashicorp-forge/hermes/discussions/58

For use cases where Hermes will be used for sensitive documents, it may not be preferred to send the full text of the documents off of the box to an indexing service (even if Algolia promises SOC2 compliance and so on). In these cases, it might be acceptable to give up document search.

Is Algolia already optional if you don't want search? If not, that would be a useful option.

There is currently a bug with the Google Docs DeleteContentRange API where deleting content that contains a suggestion fails with this error:

googleapi: Error 500: Internal error encountered.

This affects all Hermes actions that rewrite the document header, including indexing updated documents, changing document state (e.g., requesting a review), and approving documents.

We have reported this to Google and they confirmed the bug, but there is no estimated time for a resolution. Unfortunately, while this bug exists, Hermes functionality to rewrite document headers fails if a user puts a suggestion in the header, and there is no way to automatically remediate this (working with suggestions is also unsupported by Google Docs' public API). Until this bug is fixed, we should lock documents in this state and provide some visual feedback for users for how to remediate, which is to remove all suggestions in the document header. Once all suggestions are removed from the document header, the Hermes indexer will automatically re-index the document and unlock it for editing.

1. Download and build Hermes
2. Setup Google Workspace, Algolia, Google Drive
3. Copy credentials.json
4. set HERMES_WEB_GOOGLE_OAUTH2_CLIENT_ID
5. ./hermes server -config=config.hcl

First I got the error from Google: Error 400 redirect_uri_mismatch. Redirect_uri: http://localhost:9999/callback
I tried adding that redirect_uri to the credentials via https://console.cloud.google.com/. Now when I try to start up Hermes the browser tries to open "http://localhost:9999/callback" which of course doesn't work.

Any suggestions? Thanks

Congrats on the launch! 🎉

What would it take for Hermes to support “backends” other than Google Workspace? Say, something open source like ONLYOFFICE. From the looks of it, Google-related code is decoupled from the rest of the system, but I can't say on the first glance how tightly it's integrated.

Would you be interested in a PR implementing something like ONLYOFFICE support?

Hi guys,

I want to say that this project is amazing, and in my company, we are creating a proof of concept to use Hermes.

The idea here is remove this hardedcoded getting by configuration.

	// TODO: remove hardcoded port.
	s := http.Server{Addr: ":9999", Handler: m}
	config.RedirectURL = "http://localhost:9999/callback"

Something like it:

// base_url is the base URL used for building links. This should be the public
// URL of the application.
base_url = "http://localhost:8000"


server_port = "9090"

What do you think?

It would be useful to offer a way to onboard existing docs so they can be brought under Hermes management.

Ex: User wishes to onboard a doc (shared with everyone in the organization) -> In Hermes, user provides the document url, description, contributors -> Hermes sets up the link, updates / creates headers -> 🏁

Hi, I followed the instructions to run hermes locally on my machine but when I try to create a document I get this error in the terminal

"error copying file: googleapi: Error 404: File not found: drafts., notFound" template=1Oz_7FhaWxdFUDEzKCC5Cy58t57C4znmC_Qr80BORy1U drafts_folder=my-drafts-folder-id

I tried changing my-drafts-folder-id to drafts and creating a drafts directory in my shared drive but it didn't work.

The UI stays at this URL http://localhost:8000/new/doc?docType=RFC without any error message.

Could you help me to get this up and running?

It looks like the current implementation heavily depends on golang binary itself, while frontend components are just built from yarn command and will be hosted in a static folder. Considering for performance and scalability I think that it should be able to hosted by an independent web server such as nginx or some cloud components like S3/GCS + CDN.

You write that the documents are stored like /l/rfc/lab-123
Is there a way to access the files directly, via file server? Or would that corrupt the database?
Our company stores client data in projects, so file structure right now is /projects/PROJECTNUMBER/PROJECT-RELATED-Files
Could that be incorporated?

Proposed changes:

• spelling (usin -> using)
• remove verbiage

Related PR

#4

I'm trying to setup Hermes, need few clarifications on security aspects of Hermes. I read that we need to ensure below OAuth scopes for Production, which is essentially full access to all of Google Drive and Google Docs. Is my assumption right on that? Is that a security risk?
https://www.googleapis.com/auth/directory.readonly,https://www.googleapis.com/auth/documents,https://www.googleapis.com/auth/drive,https://www.googleapis.com/auth/gmail.send

I also see only below OAuth scope is required for localhost, whereas prod we need more OAuth scopes than this. Why is that?
https://www.googleapis.com/auth/drive.readonly

I see Hermes uses Algolia Search. Can we use Hermes without Algolia, or can we configure something else in place of Algolia, such as self hosted Elastic Search? Does configuring Algolia mean all of the Google Docs content is parsed to Algolia servers and is indexed to serve search capability? What are the endpoints of Algolia that Hermes hits to parse data to Algolia to index and search? Are these the only APIs?

<MY_APP_ID>-dsn.algolia.net, <MY_APP_ID>.algolianet.com

Thank you!

I configured Hermes as per the docs, but I seem to be getting an error with Algolia. The full error looks like this:

error initializing Algolia write client: error setting settings for docs index: all hosts have been contacted unsuccessfully, it can either be a server or a network err
or or wrong appID/key credentials were used. You can use opt.ExposeIntermediateNetworkErrors(true) to investigate.

As far as I can tell, my Algolia section of the config looks to be correct. The only part I was a bit unclear on was the write_api_key, I assume this is meant to be the Admin API Key?

I've tested connectivity to various Algolia endpoints with netcat, and seem to be able to connect over port 443 fine:

• <MY_APP_ID>-dsn.algolia.net
• <MY_APP_ID-1>.algolianet.com
• <MY_APP_ID-2>.algolianet.com
• <MY_APP_ID>-3.algolianet.com

I didn't see any options related to opt.ExposeIntermediateNetworkErrors(true) in the config, so I assume thats something that needs to be passed to the Go client?

Any advice on this would be greatly appreciated.

Hi!

First of all, I believe this kind of solution on top on existing document provider is great! It allows centralizing templates, status and sharing in one place, without being dependent on the document provider.

With that in mind, is there any plan to allow another document provider like MS SharePoint or even open source one like NextCloud?

And if it's planned, do you accept PR for that kind of proposal?

Regards

Hello!
I see there is not currently a plan for a pluggable search engine (#449).

Would you be open to accepting a change that creates go search engine interface (but the only and default implementation is Algolia)? I could make this at your direction or I could propose an interface and we could go from there.

Thanks for the work on this great project.

I get the following error with the latest yarn v1, which I believe most of developers who don't care much about next gen yarn have.

Expected version "~3.3.0" should be documented as well as node version.

$ make build
cd web; \
	yarn install; \
	rm -rf dist/; \
	yarn build;
yarn install v1.22.19
[1/5] 🔍  Validating package.json...
error [email protected]: The engine "yarn" is incompatible with this module. Expected version "~3.3.0". Got "1.22.19"
error Found incompatible module.
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
yarn run v1.22.19
error [email protected]: The engine "yarn" is incompatible with this module. Expected version "~3.3.0". Got "1.22.19"
error Commands cannot run with an incompatible environment.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
make: *** [build-ui] Error 1

For someone who just want to try Hermes real quick I think it's nice to have a Docker image officially.

In the config, there's this:

// document_types configures document types. Currently this block should not be
// modified, but Hermes will support custom document types in the near future.
// *** DO NOT MODIFY document_types ***

That's great! Looking forward to new document types. However, in advance of that, it might be useful to add support specifically for blank documents.

1. Download and build Hermes
2. Setup Google Workspace, Algolia, Google Drive
3. Copy credentials.json
4. set HERMES_WEB_GOOGLE_OAUTH2_CLIENT_ID
5. ./hermes server -config=config.hcl

First I got the error from Google: Error 400 redirect_uri_mismatch. Redirect_uri: http://localhost:9999/callback
I tried adding that redirect_uri to the credentials via https://console.cloud.google.com/. Now when I try to start up Hermes the browser tries to open "http://localhost:9999/callback" which of course doesn't work.

Any suggestions? Thanks

There is already a quite popular open-source project called "Hermes" in Java community: https://hermes.allegro.tech/ https://github.com/allegro/hermes which is more than 6 years old, and actively developed (not to mention yet another Hermes project from Facebook?). It's not a big issue, but have you researched the usage of the name before choosing it? What about users' confusion?

Expanding on #62 with a specific implementation request that maintains the search functionality. It would be nice to have an alternative full text search engine backend. This tool would preferably something that can be hosted internally or is already supported by various cloud vendors with their own hosted flavors like typesense or elasticsearch.

Algolia does not do BAA's or other industry-required arrangements for their free tier and we can't ship docs to third party vendors without that level of agreement even for a POC. We are married to google docs for documentation over better tools in the first place because of these kinds of data requirements that google is able to meet. Putting hermes in our org has turned from a "setup hermes" to "onboard a secondary vendor" because of this Algolia requirement.

#62 seeks to make the system operate in any degree without Algolia. This issue is to specifically enable search functionality without Algolia. Full text search is a pretty significant part of our reason to adopt Hermes in the first place.

Whenever I try to view a private google doc via hermes, I get two 302 redirects: first to https://accounts.google.com/ServiceLogin and then to https://docs.google.com/accounts/SetOSID which returns a 400

Is there any piece of setup that I am missing? Everything else works - including creating the docs

The README.md and the config.hcl have references to creating a domain-wide delegated service account, and using it for production access. The instructions aren't clear though. Once the key is downloaded, how do I reference it in the config.hcl?

Thank you for the fantastic work done here. Hashicorp opensource has been overall wonderful. I recently needed to start keeping track of documents and ideas and I found hermes.

I'd like to make a suggestion as regards the search engine. Would we be interested in switching or providing the option of letting the user pick the search engine ? I'd like to recommend meilisearch.

1. it's open source and can be self-hosted
2. the syntax and spirit is pretty much the same as Algolia

Hi! We want to use an env var to set the database password for security reasons.

What do you think? I'm willing to submit a PR if you're agree.

Thank you!

Hi everyone,

I am getting the following error when executing indexer:

➜  hermes git:(main) ✗ ./hermes indexer -config=config.hcl
starting indexer...
2023-06-06T18:41:04.747-0300 [INFO]  hermes.indexer: indexing documents folder: folder_id=docs last_indexed_at=1970-01-01T00:00:00Z
2023-06-06T18:41:05.076-0300 [WARN]  backoff error (retrying): error="error listing files: googleapi: Error 404: File not found: ., notFound" delay=423.413661ms
2023-06-06T18:41:05.724-0300 [WARN]  backoff error (retrying): error="error listing files: googleapi: Error 404: File not found: ., notFound" delay=570.766486ms
2023-06-06T18:41:06.588-0300 [WARN]  backoff error (retrying): error="error listing files: googleapi: Error 404: File not found: ., notFound" delay=793.352999ms
2023-06-06T18:41:07.587-0300 [WARN]  backoff error (retrying): error="error listing files: googleapi: Error 404: File not found: ., notFound" delay=2.155192318s
2023-06-06T18:41:09.968-0300 [WARN]  backoff error (retrying): error="error listing files: googleapi: Error 404: File not found: ., notFound" delay=1.700310086s

All index has been created:

Server running:

➜  hermes git:(main) ✗ ./hermes server -config=config.hcl 
2023-06-06T18:43:01.395-0300 [INFO]  hermes: listening on 127.0.0.1:8000...
2023-06-06T18:43:12.453-0300 [WARN]  backoff error (retrying): error="error searching people directory: googleapi: Error 403: The G Suite domain admin has disabled external directory sharing. See more details at https://support.google.com/a/answer/6343701, forbidden" delay=566.29779ms
2023-06-06T18:43:13.230-0300 [WARN]  backoff error (retrying): error="error searching people directory: googleapi: Error 403: The G Suite domain admin has disabled external directory sharing. See more details at https://support.google.com/a/answer/6343701, forbidden" delay=863.326168ms
2023-06-06T18:43:15.151-0300 [WARN]  backoff error (retrying): error="error searching people directory: googleapi: Error 403: The G Suite domain admin has disabled external directory sharing. See more details at https://support.google.com/a/answer/6343701, forbidden" delay=1.103591023s
2023-06-06T18:43:16.572-0300 [WARN]  backoff error (retrying): error="error searching people directory: googleapi: Error 403: The G Suite domain admin has disabled external directory sharing. See more details at https://support.google.com/a/answer/6343701, forbidden" delay=1.108521232s
2023-06-06T18:43:18.447-0300 [WARN]  backoff error (retrying): error="error searching people directory: googleapi: Error 403: The G Suite domain admin has disabled external directory sharing. See more details at https://support.google.com/a/answer/6343701, forbidden" delay=3.243996113s

Directory config:

There is another configuration to do?

Hello,

I would like to know if it is possible to use Hermes without a Google Workspace account, but only with a shared folder in Google Drive?

On running make build it shows: Killed Usage Error: Couldn't find the node_modules state file - running an install might help (findPackageLocation)

I am using latest yarn and tried deleting node_modules and re installed with yarn install, tried with previous node LTS versions also. Couldn't fix :(

This is my package.json file:

Hi,

I'm kindly requesting to add Microsoft integration in addition to Google.

Regards,

AC.

I set up hermes as per the setup guide, Snipped of google_workspace section as follows:
`// google_workspace configures Hermes to work with Google Workspace.
google_workspace {

create_doc_shortcuts = true
docs_folder = "hermes-test"
drafts_folder = "hermes-draft-folder"
shortcuts_folder = "hermes-shortcuts-folder"
temporary_drafts_folder = "hermes-temporary-drafts-folder"
}`

While creating a new document via hermes ui i keep on getting following error:

2024-01-31T15:18:16.667-0500 [ERROR] hermes: error creating draft: error="error copying file: googleapi: Error 404: File not found: hermes-draft-folder., notFound" method=POST path=/api/v1/drafts template=1Oz_7FhaWxdFUDEzKCC5Cy58t57C4znmC_Qr80BORy1U drafts_folder=hermes-draft-folder 2024-01-31T15:19:17.155-0500 [ERROR] hermes: error creating draft: error="error copying file: googleapi: Error 404: File not found: hermes-draft-folder., notFound" method=POST path=/api/v1/drafts template=1Oz_7FhaWxdFUDEzKCC5Cy58t57C4znmC_Qr80BORy1U drafts_folder=hermes-draft-folder 2024-01-31T15:19:19.738-0500 [ERROR] hermes: error creating draft: error="error copying file: googleapi: Error 404: File not found: hermes-draft-folder., notFound" method=POST path=/api/v1/drafts template=1Oz_7FhaWxdFUDEzKCC5Cy58t57C4znmC_Qr80BORy1U drafts_folder=hermes-draft-folder

I created 'hermes-draft-folder' manually in my GDrive still getting same error.

Any idea?

Build Error (Inserter)

ember-auto-import could not find a place to insert app styles in index.html.