everruler12 / roam2github Goto Github PK

I am think you for your excellent work, it help me a lot. work so fine and new never got failed messages as before.

today i tried the new function to backup multigraph, and proceeded show this

2021-02-04 10:05:56.034 R2G - Saving Markdown 2021-02-04 10:05:56.936 R2G Open graph F*********l 2021-02-04 10:05:56.936 R2G - (Wait 1 second) 2021-02-04 10:05:57.938 R2G - Navigating away to 404 (workaround) 2021-02-04 10:05:58.047 R2G - (Wait 1 second) 2021-02-04 10:05:59.048 R2G - Navigating to graph 2021-02-04 10:05:59.949 R2G - astrolabe spinning... Error: The operation was canceled.

just let you know.

After a successful login and loading at the Astrolabe logo, for whatever reason this thing drops off after 10 minutes. I've never had it take 10 minutes on my own browser, and my graph is 14 MB right now. Log is as follows:

2021-02-26 04:22:16.695 R2G Detected GitHub Actions path
2021-02-26 04:22:16.698 R2G Create browser
2021-02-26 04:22:18.041 R2G Login
2021-02-26 04:22:18.107 R2G - Navigating to login page
2021-02-26 04:22:19.347 R2G - Checking for email field
2021-02-26 04:22:19.458 R2G - (Wait for auto-refresh)
2021-02-26 04:22:21.656 R2G - Filling email field
2021-02-26 04:22:21.746 R2G - Filling password field
2021-02-26 04:22:21.852 R2G - Checking for "Sign In" button
2021-02-26 04:22:21.857 R2G - Clicking "Sign In"
2021-02-26 04:22:22.743 R2G Login successful!
2021-02-26 04:22:22.802 R2G Open graph ***
2021-02-26 04:22:22.802 R2G - Navigating to graph
2021-02-26 04:22:23.695 R2G - astrolabe spinning...
Error: The operation was canceled.

Specific error is:

The job running on runner Hosted Agent has exceeded the maximum execution time of 10 minutes.

Can roam2github give backups of offline graphs?

I am using your script. But I want to extract markdown to my _notes folder of Jekyll based project. Is it possible ?

No formatting needed. Just extract or move extracted markdown files to specific location.

Thanks a lot in advance.

Looks like for now you can specify which backup type to download by editing roam2github.js and modifying backup_types.

Easy in the short term but would be nice to have a more friendly way to specify which backups to make. Could help some people if they hit the minute long limit. Perhaps another secret? Something like R2G_BACKUP_TYPES = "json, edn, markdown"

Hi Eric, my backup run started failing around 8 hours ago for no apparent reason. I am not aware of any changes that might have led to this. Any ideas about how to troubleshoot? I have it set to run every 30 mins. Thanks in advance

After many successful runs, this error happened today:

2021-02-12 18:27:05.568 R2G GitHub Actions path found
2021-02-12 18:27:05.571 R2G Create browser
2021-02-12 18:27:06.308 R2G Login
2021-02-12 18:27:06.362 R2G - Navigating to login page
2021-02-12 18:27:07.647 R2G - Checking for email field
2021-02-12 18:27:10.035 R2G ERROR - Error: Protocol error (DOM.resolveNode): Node with given id does not belong to the document
    at /tmp/roam2github/node_modules/puppeteer/lib/cjs/puppeteer/common/Connection.js:208:63
    at new Promise (<anonymous>)
    at CDPSession.send (/tmp/roam2github/node_modules/puppeteer/lib/cjs/puppeteer/common/Connection.js:207:16)
    at ExecutionContext._adoptBackendNodeId (/tmp/roam2github/node_modules/puppeteer/lib/cjs/puppeteer/common/ExecutionContext.js:299:47)
    at ExecutionContext._adoptElementHandle (/tmp/roam2github/node_modules/puppeteer/lib/cjs/puppeteer/common/ExecutionContext.js:314:21)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at async Frame.waitForSelector (/tmp/roam2github/node_modules/puppeteer/lib/cjs/puppeteer/common/FrameManager.js:846:24)
    at async /tmp/roam2github/roam2github.js:173:13
R2G Exit after: 4.473s
Error: Process completed with exit code 1.

I'm hoping the next run will be fine, but figured you would want to see it anyway.

Hi,

Thanks for creating a stable backup solution for Roam. Here is a specific issue that I have encountered recently: When the Roam subscription lapses, the backup script will start failing on every run. This is probably due to a modal popup with a subscription reminder that appears while opening the Roam database in question. However, it is still possible to export the database, so the problem can likely be worked around. I plan to renew my subscription imminently, but I would be happy to provide the information needed to handle this scenario.

Kind regards,
Ketil

I followed the setup instructions but when I run my workflow I get this: " Error: fatal: couldn't find remote ref refs/heads/main" attached is a zip of my logs
logs_5.zip

I have been seeing roam2git fail for two consecutive nights with the error:

R2G ERROR - EDN formatting error: mismatch with original

I’m reporting this per the instructions in Common error causes.

The backup in question is for a group, the Yak Collective, that believes in working in public. As such, both our Roam graph (https://roamresearch.com/#/app/ArtOfGig) and the backup (https://github.com/The-Yak-Collective/backups) are publicly accessible (note that the repo backs up two graphs; the failure is occurring on the ArtOfGig graph).

Happy to try provide additional information or help test things, though I’m hoping that the public nature of both the graph and the backup will make it easier to diagnose this issue.

I walked through the step-by-step instructions and it seemed all straight forward.

my understanding cron “0 * * * *” means top of the hour - and nothing happened.

so I went into the workflow and found the error:

Run actions/checkout@v2 
 with: 
   ssh-strict: true 
   persist-credentials: true 
   clean: true 
   fetch-depth: 1 
   lfs: false 
   submodules: false 
Error: Input required and not supplied: token

in main.yml:

- uses: actions/checkout@v2
   with: 
     repository: ${{ secrets.BACKUP_REPO }}
     token: ${{ secrets.ACCESS_TOKEN }}

I was only prompted to insert 3 keys during setup: ROAM_{EMAIL, PASSWORD, GRAPH}

why is my ACCESS_TOKEN null?
who defines BACKUP_REPO and ACCESS_TOKEN?

I found something called GITHUB_TOKEN - not sure if this is something completely different as I’m a bit of a noob when it comes to workflows (and am finding it neat :)

It looks to me like the npm install is failing on Chromium, but I haven't spent any time digging into it.

Started 16 hours ago, or around when 016ce8c was merged.

I'm seeing these logs in the Run Backup step

Run cd ..
  cd ..
  git clone -q https://github.com/everruler12/roam2github-test.git roam2github
  cd $_
  npm i
  npm run start -s
  shell: /usr/bin/bash -e {0}
  env:
    R2G_EMAIL: ***
    R2G_PASSWORD: ***
    R2G_GRAPH: ***

> [email protected] install /home/runner/work/roam-backups/roam2github/node_modules/puppeteer
> node install.js


Chromium (818858) downloaded to /home/runner/work/roam-backups/roam2github/node_modules/puppeteer/.local-chromium/linux-818858
added 66 packages from 83 contributors and audited 66 packages in 10.564s
found 0 vulnerabilities

Error: Process completed with exit code 1.

Pretty sure my secrets are correct - the secrets page looks like this

And none of those values have changed in Roam in the last day.

Is it possible to use this if I use my google account to sign in to Roam instead of using email + password?

Hi, Erik, thanks to your script roam is now backed up successfully, and I am relieved from losing data.
Github says yesterday disk usage is coming near to the upper limit.
I am wondering if it is possible to change and reduce the back-up timing of a day or delete old backup automatically.
Should I periodically delete old files by manual?

Hi. Thank you very much for making such a useful library.

There seems to be a problem with backup. If there are any Quick Capture Notes that you have not synced, a pop-up will appear on your Roam.

This caused the backup to fail. When I close this window, the backup now succeeds.

Every job was stopped by Checking for "Export All" option.

Logs

Run cd ..

> [email protected] install /home/runner/work/roambackup/roam2github/node_modules/puppeteer
> node install.js


Chromium (818858) downloaded to /home/runner/work/roambackup/roam2github/node_modules/puppeteer/.local-chromium/linux-818858
added 66 packages from 83 contributors and audited 66 packages in 10.854s
found 0 vulnerabilities

2021-02-07 05:05:01.067 R2G Create browser
2021-02-07 05:05:01.364 R2G Login
2021-02-07 05:05:01.367 R2G - Navigating to login page
2021-02-07 05:05:02.537 R2G - Checking for email field
2021-02-07 05:05:02.672 R2G - (Wait 10 seconds)
2021-02-07 05:05:12.676 R2G - Filling email field
2021-02-07 05:05:12.753 R2G - Filling password field
2021-02-07 05:05:12.830 R2G - Checking for "Sign In" button
2021-02-07 05:05:12.836 R2G - Clicking "Sign In"
2021-02-07 05:05:13.575 R2G Login successful!
2021-02-07 05:05:13.575 R2G Open graph s******a
2021-02-07 05:05:13.575 R2G - (Wait 1 second)
2021-02-07 05:05:14.576 R2G - Navigating away to 404 (workaround)
2021-02-07 05:05:14.628 R2G - (Wait 1 second)
2021-02-07 05:05:15.628 R2G - Navigating to graph
2021-02-07 05:05:16.193 R2G - astrolabe spinning...
2021-02-07 05:05:26.431 R2G - astrolabe spinning stopped
2021-02-07 05:05:26.478 R2G Graph loaded!
2021-02-07 05:05:26.478 R2G Export JSON
2021-02-07 05:05:26.483 R2G - Clicking "..." button
2021-02-07 05:05:26.531 R2G - Checking for "Export All" option
Error: The operation was canceled.

roam2github runs dutifully for 60 days and because I don’t make any changes to the the actions repo, the workflow gets disabled.

The message in my actions tab:

“This scheduled workflow is disabled because there hasn’t been activity in this repository for at least 60 days”.

Is there an easy to prevent this from happening?

I missed an email notification I was supposed to get 7d prior to workflow deactivation - that was 8 months ago and just noticing this now! grin

(sorry I don’t have a screenshot, I thought of this after reenabling the workflow!)