Backup System about src.next HOT 28 OPEN

Since #845 was just closed with the message:

Closing and linking this to #122

I would like to request that whatever backup method is added also support exporting to a browser-neutral format (such as .html or txt) for the currently opened tabs as the main ask in #845 was to be able to have something that could allow exporting currently opened tabs from kiwi in a format that could be opened on desktop browsers without requiring cloud services.

from src.next.

In case anybody may ever need it, here I leave the "opened tabs" path, which has recently been tested by me.

opened tabs:
'/data/data/com.kiwibrowser.browser/app_tabs/'
copy the whole folder and replace the app_tabs/0/tab_state0 file or edit it adding the URLs listed there

Moreover I'd say we don't need the bookmarks path since we have the bookmark im/export feature.

from src.next.

history:
'/data/data/com.kiwibrowser.browser/app_chrome/Default/History'

bookmarks:
'/data/data/com.kiwibrowser.browser/app_chrome/Default/Bookmarks'

chrome flags & settings:
'/data/data/com.kiwibrowser.browser/app_chrome/Local State'
'/data/data/com.kiwibrowser.browser/app_chrome/Default/Preferences'
'/data/data/com.kiwibrowser.browser/shared_prefs/com.kiwibrowser.browser_preferences.xml'

from src.next.

An (relatively) easy way this could be implemented is to save all your data to /sdcard/Kiwi Browser. Then there would be no need to implement a backup system, as you could just copy the folder to wherever you like.

from src.next.

Thank you @gdgsdg123 and @NotesOfReality. Luckily I have root.

For saved passwords, /data/data/com.kiwibrowser.browser/app_chrome/Default/Login Data

from src.next.

Is there a way to backup the extensions list or the extensions with their configs collectively ?

from src.next.

@Nichtraucher I did not know about this flag, I'm not sure it really exists since a long time, the post that was linked to is from 2013. Either way, it is unfortunately not a future-proof or even present-proof method, since modern Android releases make it very difficult for apps to store their data outside of their isolated folder, and its very difficult to access them, you need root or to use adb. A flag is certainly not sufficient anymore to allow external access to an app's files, I know I worked on several apps to implement backup features that got broken since Android 10.

from src.next.

Thank you very much for the precious information, if only I had known a bit of programming languages I would have implemented the backup system by myself and then created a pull request.

from src.next.

Just manually back them up/restore with any adequate explorer, actually...

from src.next.

Yeah, I already know that, I simply think that adding such a feature with a proper UI could make the app go in the top 5 or something like that

from src.next.

Thanks. This is good for helping back up an existing Kiwi config and restoring it. But doesn't address importing data from another browser.

Please don't fall into the trap so many developers do of bringing out a new product and expecting people to migrate to it without being able to bring their existing data over from your rivals, who they've been [more or less] happily using for years.

Doesn't directly affect me as I have my bookmarks stored on pinboard. But a lot of folks will have years worth of bookmarks and history stored in $CurrentBrowser and are much more likely to move to $NewBrowser if they can bring that stuff with them.

from src.next.

Thanks. This is good for helping back up an existing Kiwi config and restoring it. But doesn't address importing data from another browser.

Please don't fall into the trap so many developers do of bringing out a new product and expecting people to migrate to it without being able to bring their existing data over from your rivals, who they've been [more or less] happily using for years.

Doesn't directly affect me as I have my bookmarks stored on pinboard. But a lot of folks will have years worth of bookmarks and history stored in $CurrentBrowser and are much more likely to move to $NewBrowser if they can bring that stuff with them.

I think this is a quite worth of attention denotation, it now is so more than ever since Kiwi Browser developer even has brought Chrome extensions support.

from src.next.

Moreover I'd say we don't need the bookmarks path since we have the bookmark im/export feature.

Prefer doing so by modifying the files anyway...

from src.next.

While this may work, it does require you to have root. I tried using the "Extensions" tab to manually go inside its folder (since it would have access), but it didn't work.

from src.next.

An (relatively) easy way this could be implemented is to save all your data to /sdcard/Kiwi Browser. Then there would be no need to implement a backup system, as you could just copy the folder to wherever you like.

This would be really useful.

from src.next.

history: '/data/data/com.kiwibrowser.browser/app_chrome/Default/History'

bookmarks: '/data/data/com.kiwibrowser.browser/app_chrome/Default/Bookmarks'

chrome flags & settings: '/data/data/com.kiwibrowser.browser/app_chrome/Local State' '/data/data/com.kiwibrowser.browser/app_chrome/Default/Preferences' '/data/data/com.kiwibrowser.browser/shared_prefs/com.kiwibrowser.browser_preferences.xml'

But without root you don't have access there. Please implement feature like Termux has so you can access all of it's files even without root

from src.next.

This appears (to me) very similar to the #728 issue I had submitted quite some time ago without any resolution thus far.

~Ibuprophen

from src.next.

Very important. Also necessary for switching from a non-rooted to a new phone. Without proper data portability, it is de-facto impossible to bring the tabs and history and bookmarks and (depending on browser) saved pages to a new phone.

Individually copying and pasting a thousand URLs or depending on cloud services that can go defunct any day without warning are obviously impractical.

I believe that one can only really own a smartphone, and a tablet for that matter, if it is rooted. Without root access, there are so many crippling restrictions, especially regarding file access. The data can not even be backed up properly! And backups are highly important!

https://beepb00p.xyz/sad-infra.html

from src.next.

I'm not sure why the devs decided to not allow backups. If data safety is a concern, there is a way to back up data safely if it's encrypted. .

Many thanks to the developers for their efforts! :-D

from src.next.

In case anybody may ever need it, here I leave the "opened tabs" path, which has recently been tested by me.

opened tabs: '/data/data/com.kiwibrowser.browser/app_tabs/' copy the whole folder and replace the app_tabs/0/tab_state0 file or edit it adding the URLs listed there

Moreover I'd say we don't need the bookmarks path since we have the bookmark im/export feature.

I have looked into the files. They have some gibberish when I tried using cat directly.
How could I extract the URLs only from that
Also I noticed that there were two directories. Namely, 0 and custom_tabs

from src.next.

FYI, both bookmarking all tabs and sharing all tabs as a simple text list are implemented in Brave for Android (I tested, both work). It seems this is simply because Brave merged in the latest chromium release. Note that exporting to bookmarks seem to work with an arbitrary number of tabs, whereas exporting to a list is limited and may crash (likely because of a memory overflow since it attempts to store in the clipboard). Brave does not support extensions.

Until an on-device solution is implemented by the developer, I think the most reliable non-root approach is by using the remote debug tools (but note I did not try this myself, but I can't see why it would not work).

from src.next.

FYI I could get a list of opened tabs using the method described here. Summary:

• install a "dev" release from the github releases
• Inside kiwi, access: chrome://process-internals (note: if you do not have a dev release, it will output an error saying that dev tools are not included in the installed release)
• On the left, go to Frame Trees
• Here you have all your tabs and groups. To export, you need a HTML exporter, such as this one, but this is not even necessary, you can simply save the whole page, or even explore it with the included Devtools in the menu of Kiwi, you will be able to see all the tabs URLs, and likely it's possible to extract them as a nice list with an XPath or XQuery statement. Or simply select all then do a simple copy/paste, then you can likely use a GPT LLM model to easily extract without programming, or a regex (each URL first starts with url: ).

/EDIT: this only displays the recent tabs of the current session that are still in-memory, this does NOT solve the issue of exporting past tabs that are not loaded in-memory.

Here are a list of extensions that can do the same (exporting currently opened tabs) that work in Kiwi:

• Tab Groups Exporter, export as JSON (can import too). https://github.com/parthpower/chrome-tab-group-exporter
• OpenedTabs, can export in JSON or in text format, akin to Tab Groups Exporter but it cannot import https://github.com/anaghvj/OpenedTabs
• Tab Porter exports to a html file and can import too! https://chromewebstore.google.com/detail/tab-porter-export-restore/hjbdgionekajacbeocbgopledpkegkmo
• TabsToMarkdown can export everything to markdown and can also serve as a tree-style tabs manager (can close and switch to other tabs - it's the only tree-style tabs manager that is functional on Kiwi so far). https://chromewebstore.google.com/detail/tabstomarkdown/iddmkgomojbcbdhfgbijihjggkgkimdb
• Export Tabs URLs, can export in custom format hence can easily support markdown! But no import. By default it filters special urls such as extension urls but can be disabled in options. Cleanest export so far. https://github.com/clach04/export-tabs-urls-phoenix/tree/main - should support imports in theory but not in practice.
• tabs2markdown: can export in 4 formats: markdown, html, wysiwyg (plain text), jira. Opensource. https://github.com/mac2000/tabs2markdown

from src.next.

After some fiddling, I found the really working and future-proof way to export really all tabs.

The future-proof non-root method to export all tabs from Kiwi browser, including past, memory-unloaded ones, is to enable Developer Mode on the Android phone, then enable USB Debugging on the phone, then connect it to a computer via USB, and then on the computer launche Chrome and open the page chrome://inspect/#devices , then tap on the phone to enable USB debugging for this session, and after a few seconds you should see a list of all your tabs, including ones in private session / incognito mode, and you can simply copy/paste everything as plain text (there are both the pages titles and URLs). This method works for all chromium based browsers, and is described in more details here and as first mentioned by another Kiwi user here. Note that it is likely that this method only works if you have installed a "dev" release of kiwi, with the debug flags enabled (I only tried with a dev release).

Now if Kiwi could just get synced with the latest chromium for Android features, it would support selecting all tabs in the tabs menu and then bookmarking/export as Brave and Vivaldi browsers can. But personally this solution will allow me to use Kiwi in the meantime as I can backup all my tabs this way, even if it is a bit cumbersome.

from src.next.

There is (perhaps) a backup method - limited to Android versions 11 and below though. I didn't quite figure it out and asked about it on reddit here
Perhaps some readers here are able to chime in and explain carefully in small steps how this modification can be done successfully?

from src.next.

To export all the URLs of open browser tabs in Kiwi on your Android phone using the adb utility on Windows, you can follow these steps.

I tested this procedure with Kiwi Browser running under Android 14 on my Samsung Galaxy S22 Ultra, connected via USB to my Windows 10 PC.

Prerequisites:

1. Android Device: Ensure USB debugging is enabled on your Android device.
2. ADB Installed: Ensure you have the Android Debug Bridge (ADB) installed on your Windows computer.
3. Kiwi Browser Installed: Ensure Kiwi browser is installed and running on your Android device.
4. USB Cable: Ensure your Android device is connected to your Windows computer via USB.

Steps:

1. Enable Developer Options and USB Debugging on Android:

   • Go to Settings > About phone.
   • Tap on Build number seven times to enable Developer options.
   • Go to Settings > System > Developer options.
   • Enable USB debugging.

2. Connect Android Device to Windows Computer:

   • Connect your Android device to your Windows computer using a USB cable.
   • Allow USB debugging authorization on your Android device when prompted.

3. Open Command Prompt on Windows:

   • Press Win + R, type cmd, and press Enter.

4. Verify ADB Connection:

   • In the Command Prompt, type:

     adb devices

   • Ensure your device is listed. If not, troubleshoot the USB connection and authorization.

5. Use Chrome Remote Debugging:

   • Open Kiwi browser on your Android device.
   • In the Command Prompt, type:

     adb forward tcp:9222 localabstract:chrome_devtools_remote
     

   • Open a web browser on your Windows computer and go to:

     http://localhost:9222/json
     

   • This will display a JSON file containing information about all open tabs in the Kiwi browser.
   • Copy the content of the JSON file displayed in your web browser.
   • Save it as a file named kiwi_tabs.json.

6. Extract URLs from JSON:

   • There are many ways you can extract URLs and page titles from the JSON file. Here is a Windows PowerShell script that will do the job:

# Load the JSON file
$json = Get-Content -Raw -Path "kiwi_tabs.json" | ConvertFrom-Json

# Filter out entries where 'id' is not a numerical value and sort by 'id' numerically
$filteredSortedJson = $json | Where-Object { $_.id -match '^\d+$' } | Sort-Object { [int]$_.id }

# Initialize a counter for the incrementing number
$counter = 1

# Process each entry and format the output
$output = $filteredSortedJson | ForEach-Object {
  "$counter"
  $_.title
  $_.url
  ""
  $counter++
}

# Output the result to the console or save to a file
$output | Out-File -FilePath "kiwi_tabs.txt" -Encoding utf8

Explanation:

1. Load the JSON file: The script reads the JSON file and converts it into a PowerShell object using ConvertFrom-Json.

2. Filter and Sort: It filters out entries where the id is not numerical using a regular expression (-match '^\d+$'). Then it sorts the remaining entries numerically by id.

3. Auto-increment Counter: A counter is initialized to start from 1, which will be used to number the entries.

4. Process Each Entry: The script processes each entry, outputting the incrementing number on the first line, the title on the second line, and the URL on the third line. An empty line is added after each entry.

5. Output: The result is saved to a file named kiwi_tabs.txt with UTF-8 encoding.

Running the Script:

1. Save the script in a .ps1 file, for example, process-json.ps1.
2. Run the script in PowerShell:

   .\process-json.ps1

This will create an kiwi_tabs.txt file in the same directory, with the following format:

1
Page Title 1
URL 1

2
Page Title 2
URL 2

The entries with non-numerical id values will be skipped, and the remaining entries will be numbered and formatted as specified.

7. Finishing up:
   -To stop ADB forwarding, you can use the adb forward --remove command followed by the local port you forwarded. Here is how you can do it:

8. Open Command Prompt:

   • Press Win + R, type cmd, and press Enter.

9. Remove the Forwarding:

   • To remove the specific port forwarding you set up (e.g., tcp:9222), type:

     adb forward --remove tcp:9222

   • If you want to remove all port forwarding, type:

     adb forward --remove-all

These commands will stop ADB from forwarding the specified ports, effectively stopping the remote debugging connection you established earlier.

from src.next.

@vlasky did you try yourself the instructions you gave? So far all approaches involving adb only exported opened tabs during the current session, but not tabs open during previous sessions when the browser got closed after, before the attempt to backup. I have found no way to achieve this so far on a non-rooted phone, it seems newer versions of Android isolate apps databases including past browser history, even from browser extensions, maybe as a way to safeguard privacy.

from src.next.

@vlasky did you try yourself the instructions you gave? So far all approaches involving adb only exported opened tabs during the current session, but not tabs open during previous sessions when the browser got closed after, before the attempt to backup. I have found no way to achieve this so far on a non-rooted phone, it seems newer versions of Android isolate apps databases including past browser history, even from browser extensions, maybe as a way to safeguard privacy.

@lrq3000 I certainly did. I exported 3000 tabs.

from src.next.

@vlasky did you try yourself the instructions you gave? So far all approaches involving adb only exported opened tabs during the current session, but not tabs open during previous sessions when the browser got closed after, before the attempt to backup. I have found no way to achieve this so far on a non-rooted phone, it seems newer versions of Android isolate apps databases including past browser history, even from browser extensions, maybe as a way to safeguard privacy.

I used auto clicker to swipe throught all the tabs and then exported it using session buddy.

from src.next.