Giter VIP home page Giter VIP logo

logseq-readwise-plugin's Introduction

💝 Sponsor this project on Github or ☕ Get me a coffee if you like this plugin!

README below is outdated since the major change on 14/2/2022. Pending a rewrite.

Overview

This is a simple Readwise plugin to:

  1. Pull in all your highlights from Readwise
  2. For subsequent pulls, it only pulls in those not in the graph

There is now a FAQ that you may find helpful to read before you use this plugin.

Samples

Sample of a book highlight

Sample of a tweet highlight

Sample of a tweet thread

Random Highlights

Book View

If you would like to view all your imported books as cards, you can use the Book Renderer function. Simply go to any block and trigger it by typing /Book Renderer.

Disclaimer

If you have multiple sources (e.g. books, tweets, instapaper) and thousands of highlights, the initial pull can take a while. You will have a progress bar to keep track on what's happening, and can terminate the pull process at any time.

Each source will have its own page in Logseq. If there has been an error, just remove the necessary pages and refresh your graph. Assuming that you do not have any filenames containing (Readwise), you can use the following command in MacOS to remove all the pages added by the plugin. Be sure to refresh your graph in Logseq before attempting any new synchronisation.

find . -name "*(Readwise)*" -delete

New highlights are found by comparing the date of the highlight against the date of the latest highlight in your last synchronisation. When using the plugin for the first time, the initial date is set to 1970-01-01T00:00:00Z.

Usage

Migrating from manual loading to marketplace

BEFORE YOU INSTALL FROM THE MARKETPLACE, please follow the instructions below to avoid synchronising duplicate highlights:

  1. Go to the settings folder of the manually loaded plugin (Windows: C:\Users\Peter\.logseq\settings or MacOS: `~/.logseq/settings).
  2. Open the file logseq-readwise-plugin.json and copy the contents of the file somewhere.
  3. Uninstall the manually loaded plugin.
  4. Install the plugin from the marketplace.
  5. Click on the settings icon in the plugins page and click Open settings. .
  6. Copy the contents in Step 2 and paste it in the file that opens up. Save and close the file.
  7. Restart Logseq.
  8. You can start to use the plugin after!

First time (from the marketplace - preferred)

  1. Go to your Readwise Access Token page and obtain a new token. Keep this token somewhere safe.
  2. Download the logseq-readwise-plugin from the Logseq marketplace.
  3. Click on the icon (📖) in the plugins bar.
  4. If you are using the plugin for the first time, do remember to click the button Click here if you are using this plugin for the first time.
  5. Key in the token that you obtained in (1) and click Save Token.
  6. Review the number of sources and highlights that you have.
  7. Click button to sync highlights.

Subsequent times

  1. Click refresh to retrieve the recent number of changes.
  2. Click button to sync highlights.

Detailed Instructions

How do I reset the export of my whole library, e.g. to start afresh?

As this plugin is still new, you may encounter situations where you want to reset your export. Firstly, thank you so much for trying this plugin out and reporting the bugs that I've missed! Secondly, you can reset by one of 2 approaches:

Please backup your graph before attempting any of the below

  1. Using a script to find all files ending with (Readwise) and deleting them. Naturally, this **assumes that you do not have any other pages whose name includes (Readwise) if not they will be deleted as well. After deleting the files, please restart Logseq and refresh your graph. A MacOS script example would be:

find . -name "*(Readwise)*" -delete

  1. Using File Explorer (Windows) or Finder (MacOS) to find these files and delete them manually. Files created by this plugin have (Readwise) added to the end of their filenames.

What happens when I take new highlights? Do they sync automatically with Logseq?

Not really. When you open Logseq and click on the plugin button (📖), you will see the new number of sources that you took highlights from since your last sync. If you would like to sync those sources, you can proceed to click on the Sync New Sources button.

What is a source?

A source is basically a book, a twitter account, etc that contains highlights.

Why are new pages being created?

If the new highlight(s) is from a new source, a new page in Logseq will be created. If it is from an existing source, the highlight(s) will be appended to the top of your current highlights list, right under the Readwise Highlights block.

Certain services such as Amazon Kindle, Instapaper do not provide automatic synchronisation with Readwise. In these cases, you can either wait for it to appear as a new source in the plugin, or proceed to your Readwise dashboard to manually sync them, and then initiate the sync from the plugin button (📖).

How do I trigger a new sync from Logseq?

By default, the plugin will automatically sync when you open the Logseq app and look for new highlights. You can then click the Sync New Sources button to initiate the sync.

Without clicking Sync New Sources button, the plugin will not automatically create pages or pull highlights in for you.

What happens when I update highlights in Readwise? Will those changes automatically sync with Logseq (or vice versa)?

Unfortunately not. It will be technically challenging to look for that specific highlight within a page, and make changes to it without the possibility to accidentally removing edits made by the user.

Can I rename the page in Logseq?

Unfortunately not as well. The plugin uses the original name given to the source to find and add subsequent highlights. If you rename it, when there are new highlights, a new page will be created for the source (with the old name). Hence, you should only rename when you do not expect any more highlights from that source, e.g. a book that you know you will not make any more highlights for it.

Can I edit the page that the plugin created for a source?

Yes and no. As long as you do not change any of the below, you may edit the page, e.g. add in your own thoughts as child blocks under the highlights.

  • Rename the block [[Readwise Highlights]].
  • Convert the block [[Readwise Highlights]] to a child block.

The above is because [[Readwise Highlights]] is used when syncing new highlights to that source. If you have done any of the above, a new block called [[Readwise Highlights]] will be created and the highlights added under it. You will then need to clean it up after.

Where does the Location link in each Kindle highlight take me?

If you have the Kindle app installed on your desktop, you will be brought directly to the highlight in the Kindle app when you click on the link.

What do I do if I have other Feature Requests to suggest or bugs to report?

Feel free to look for me on Discord, or just opening an issue in this repository.

Thanks for trying out the plugin!

Adapted from Readwise's help article for Obsidian

Future

  • Change style of popup.
  • Fix issue of pulling new highlights removing old blocks from existing pages.
  • Fix source of non-Kindle highlights.
  • Account for cases that have more than 1000 cases.
  • Refactor code.
  • Will be incorporating the possibility of only pulling highlights for specific sources.

Credits

Big thanks to @MattHulse for helping to contributing to the code (duplicate source with same name, but has no highlights)!

logseq-readwise-plugin's People

Contributors

ashfaqfarooqui avatar benjypng avatar luhmann avatar matthulse avatar pstuifzand avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

logseq-readwise-plugin's Issues

Empty URLs after import/sync

Hi after importing I see entries like this:

And I have to add that Medium platform is bad for write, the editor is too much simple that became very complicate to use, for example there’re a lot of shortcust to insert Mardown sintax but they’re not the standard Mardown shortcuts, are not well
              link:: [null](null)
              on:: [[Jan 23rd, 2022]]
              tags::

Note the link:: [null](null)
I was expecting it would contain the link to the article.
Any ideas? Thanks for this great tool

Error "Sync in progress initiated by different client"

Hi,
when I try to sync I get "Sync in progress initiated by different client" error.

Recently, I've reset my Logseq cache. After that, the error started to appear.

I've tried to reset Readwise plugin but it didn't fix the error.

If I watch the developer console, an error is throwed

image

Books with no highlights abort sync

Thank you for the great plugin!

When trying to use it I ran into the same issue that was reported with #2 and solved via this PR #3.

I just wanted to open a PR with the fix when I found the original issue.

The problem might be more prevalent now because the offending source in my case was the "Quick passages"-document that Readwise seems to add for all new users as an instruction manual. It does not have any highlights and when encountering it the sync freezes. As the document seems to have the date of the user signup and the newest sources are imported first, only one source was imported in my case before the import stopped. Deleting the document also does not help, because the API Responses seem to still contain ist, even though it does not show up in the UI any longer.

It looks like the fix mentioned above is not part of the source code anymore? Would you be open to add it again so that the plugin works in the mentioned case? I opened a PR that does that, but as the issue was solved before the credit should go to @MattHulse

Retrieved timestamp creates references to hh:mm: pages

Each imported book contains a properties section including retrieved:: .
This has a link to the journal date for when the highlight was imported, but also a hh:mm timestamp. This ends up creating odd pages in my graph like the following:

1:15:
1:16:
1:17:

The link to the date the items are imported is nice, and the time may be interesting in the context of the highlight (more interesting if it was the actual 24 hour time). However it is not something I want to link to.

Here is a sample header for an imported highlight:

retrieved:: [[Dec 28th, 2021]], 1:15:
              author:: [[Psychology Today]]
              category:: [[articles]]
              source:: [[reader]]

Book View not displaying

When I use the /Book renderer command, nothing displays.

image

image

I have the latest version of the both the plugin and Logseq.

Properties display and function

Great plugin; thanks so much for creating it.
I'm having a couple of small issues with properties: the content of the properties bullet is invisible when it is not active, and changing the order of the properties lines deletes content. Here are two screenshots that show the properties bullet when active and not active:
Screen Shot 2022-01-15 at 9 23 59 AM
Screen Shot 2022-01-15 at 12 57 18 PM

With other properties bullets in Logseq, the properties bullet has a background and the properties are shown in bold. I've shown this default situation below, in case I am not describing it well:
Screen Shot 2022-01-15 at 12 58 27 PM

So, there is something different about these Readwise properties that is preventing the background and bold effects and also making the bullet appear empty when it is not.
(In the first screenshot above I have removed the 14 spaces that the plugin inserts, for some reason, to each property line. I prefer the properties to be left-aligned. This change seems to make no difference to the issue I'm seeing. Also, if I move a property such as tags to the top of the list, that property is deleted when I click out of the bullet. I don't mind the order of the properties as it is; I just thought I would experiment with placing the tags at the top, to see if doing so would fix the issue, and that's when I noticed the deletion.)

The plugin stucks frequently

In Step 3 when I start the sync the plugin gets stuck in x% (I'm not referring to the "Please wait for Readwise's cooling off period to lapse." message) and refuses to sync more highlights. If I manually activate "Stop Syncing", and restart de plugin it starts from the beginning and advances a little bit more, but eventually gets stuck again and I need to restart it multiple times.

Initial Sync Keeps Hanging, Dev Tools shows exception

My initial sync keeps hanging. It seems to stop at the same document every time. The last document processed appears to have been created successfully. Unfortunately I can't tell what the next document is as it is likely the one causing trouble. Sometimes article titles have characters that are not valid as filenames (I've noticed this in the Obsidian Readwise Export). So maybe one of the sources has a character that is invalid for a filename and that is causing the import to stop?

The exception I see in the console is this:

index.ddbc8bca.js:2 Uncaught (in promise) TypeError: Cannot read properties of null (reading '$cljs$core$IFn$_invoke$arity$1$')
    at $clojure$zip$node$$ (zip.cljs:64)
    at $frontend$modules$outliner$core$get_node_tree_topmost_last_loc$$ (core.cljs:355)
    at $frontend$modules$outliner$core$insert_nodes$$ (core.cljs:377)
    at $frontend$handler$editor$paste_block_vec_tree_at_target$$ (editor.cljs:2205)
    at $frontend$handler$editor$paste_block_tree_after_target$$ (editor.cljs:2267)
    at api.cljs:430
    at k (helpers.ts:141)
    at Ae.<anonymous> (LSPlugin.core.ts:186)
    at Ae.a.emit (index.js:181)
    at B.<anonymous> (LSPlugin.caller.ts:202)

Sync crash

My syncing no longer works. It just stops at some percentage.

I've tried downgrading to Logseq 0.6.5 because I say that in another issue. Same result though.

With 0.6.5 I get the following error in the console:
Screenshot 2022-05-14 at 11 27 04

Seems to always happen with the same book.

Didn't find a good way to debug, any tips on how to proceed?

Duplicate data

Every time i synchronize data from readwise, there will find that duplicate blocks are created

Twitter threads are ordered in the wrong order inside highlights

When syncing a thread, In the readwise highlights section, the threads are synced in the reverse order, which means it's basically unreadable.

The linked reference that appears in the Journal daily page, has sorting which is even weirder, it does not seem to follow any kind of logical pattern of sorting.

Example in the linked image.

image

Images are too big

There should be an option to specify max image height/width and then scale down the images appropriately with this syntax

![book_image](link){:height 180, :width 120}

Consistency with the built-in Zotero system

An entry from the built-in Zotero connection creates properties like so:

tags:: [[bias]], [[research]], [[uncertainty]]
date:: 2020
extra:: OCLC: 1061820225
isbn:: 978-0-425-28464-3
title:: @Skin in the game: hidden asymmetries in daily life
item-type:: [[book]]
original-title:: Skin in the game: hidden asymmetries in daily life
language:: English
short-title:: Skin in the game
authors:: [[Nassim Nicholas Taleb]]
library-catalog:: Open WorldCat
links:: [Local library](zotero://select/library/items/JG44VHNX), [Web library].
(https://www.zotero.org/users/5457446/items/JG44VHNX)

For a Readwise quotation from the publication above, the Readwise plugin creates properties like so:

retrieved:: [[Jan 15th, 2022]], :04:3
author:: [[Nassim Nicholas Taleb]]
category:: [[books]]
source:: [[kindle]]
tags:: [[uncertainty]], [[bias]], [[research]]

The two sets of properties refer to the same thing in two different ways: in this case, [[book]] vs. [[books]] (there may be other such instances in other types of references). The category (aka item-type) entries are not aligned, with the result that two pages are created (book and books) for the same type of thing.

Furthermore, each Zotero reference creates a separate page (with title of the form @Skin in the game: hidden asymmetries in daily life, and the Readwise plugin creates its own page of the form Skin in the Game (Readwise). It would be good to have a single unified page for each publication, showing both the Zotero citation and its accompanying Readwise quotations — instead of two different pages for the same publication. This might be tough to do, as some integration with the Zotero plugin would (probably) be required.

prefer date format

The plugin does not comply with prefer date format, i.e., all its dates are "MMM do, yyyy".

I'd wish to the plugin to honor the logseq prefer date format or have a config to set it up for itself.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.