Giter VIP home page Giter VIP logo

pixel-backup-gang's Introduction

pixel backup gang

mount ext4 drives into the google pixel's internal storage

WARNING: this code is experimental and there is no guarantee that it works. rooting your phone or running any commands as root can be very dangerous. you have been warned.

anyway here is a demo image of a portable SSD enclosure mounted into the internal storage on my Pixel XL. the data is readable & writable in the Google Photos app. image

why?

the main goal is to allow for easy backup of external media via Google Photos on original Google Pixel phones, which which have an unlimited storage benefit. the "usual" method is to copy every file to the device's internal storage, which can incur terabytes of unnecessary writes to the limited lifetime of the internal flash memory.

i first tried using FUSE (filesystem in user space) based solutions like bindfs (via termux root-packages) and fuse-nfs (complicated to compile for android so i built my own minimal version in Rust). this works and is especially good at sidestepping android's selinux policies. however the performance was not acceptable. (note: i have not tried fbind but i don't think that works out of the box here without using FUSE)

this method is basically a hack around the selinux policies + app permissions using the plain old mount command.

(if you don't care about using these scripts and just want to see how it's done, take a look at mount_ext4.sh)

the good

  • works with the stock kernel
  • backs up external files larger than 4gb (stock OS only supports FAT32 for external drives)
  • reduces wear on internal flash storage
  • prevents device from overheating - the external drive gets hot instead
  • makes 32gb pixels viable for mass backup

the bad

  • phone needs to be rooted
  • there's currently no way to auto-mount when the disk is connected

the ugly

  • there's no GUI, you need to execute shell scripts

prerequisites

  • a Google Pixel (sailfish) or Google Pixel XL (marlin) on Android 10, rooted with Magisk. may work on other phones.
  • a USB storage drive formatted with an ext4 filesystem. other filesystems not currently supported.

installation

installation is essentially just copying the scripts to the device & making them executable. you can do this manually, or use one of the automated steps below.

via pixel terminal

  1. start a terminal application and navigate to the directory where you want to install the scripts
  2. run the following command:

sh -c "$(curl -fSs https://raw.githubusercontent.com/master-hax/pixel-backup-gang/master/install.sh)"

via linux pc using adb

  1. install Android Debug Bridge (adb) & connect the pixel
  2. clone this repository
  3. run make mobile-install. this installs the scripts to /data/local/tmp by default.
    • if your pixel has Termux installed, you can install the scripts to the Termux home directory with make mobile-install DEVICE_INSTALL_DIRECTORY=/data/data/com.termux/files/home
    • if you are running these steps on WSL, you should use the adb executable from windows (which has USB support) with make mobile-install HOST_ADB_COMMAND=/mnt/c/Users/someone/AppData/Local/Android/Sdk/platform-tools/adb.exe

usage

setup

  1. start a shell on the device & navigate to the installation directory
    • from the device
      • launch Terminal, Termux, JuiceSSH, or some other terminal app
      • run su then allow sudo access to your terminal app in Magisk
    • from a PC
      • run adb shell
      • run su then allow sudo access to the shell process in Magisk
  2. run ./start_global_shell.sh to enter the global mount namespace
    • you may not need this step if you use Magisk to force the global mount namespace

mounting

  1. connect the ext4 formatted external drive to the pixel. you should get an os notification that says the drive is not supported. clear or ignore this notification.
    • this notification directs you to format the drive in FAT32 - don't do that
  2. find the block device that you want to mount. it is usually found at /dev/block/sdg1 but changes when devices are connected and disconnected.
    • if you don't know the filesystem UUID, use ./show_devices.sh
    • if you know the filesystem UUID, use ./find_device.sh
  3. run ./mount_ext4.sh <BLOCK_DEVICE> e.g. ./mount_ext4.sh /dev/block/sdg1

everything located under /the_binding on the external drive should now be visible by apps at /the_binding in the internal storage

unmounting

  1. make sure nothing important is reading from or writing to the drive
  2. run ./unmount.sh

everything located under /the_binding in the internal storage should now be gone. you can disconnect the drive if you're sure all pending writes have been flushed.

notes

TODO

pixel-backup-gang's People

Contributors

master-hax avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

pixel-backup-gang's Issues

Trouble with make mobile-install

Hey guys
Really excited about this project.

I'm having some basic trouble with the adb command "make mobile-install"
It doesn't seem to be in anyway recognised as a command with platform-tools.
I'm definitely doing something silly, would greatly appreciate a point int he right direction.

Make mobile-install on windows

Hi,
I have downloaded make using choco on windows. Then running 'make mobile-install' gives me:
tar: Option --transform=s,^scripts/,pixel-backup-gang-snapshot/, is not supported
Usage:
List: tar -tf
Extract: tar -xf
Create: tar -cf [filenames...]
Help: tar --help
make: *** [Makefile:42: pixel-backup-gang-snapshot.tar.gz] Error 1

(Could this be a limitation of the tar command on windows, no '--transform' option?)

I also have no idea on how to do the process on the pixel phone. I'm just really new to this, and probably need a step by step process. I just find the current process a bit difficult to wrap my head around. Especially where to copy the scripts and files, what I need and where to run the files. Like an ELI5 would be amazing for amateurs like me who want to get the most out of my pixel phone.

Thanks for the help

Trigger Uploads

Problem

I find that "background uploads" is unreliable and it prevents new stuff to be transferred to the SSD since it will be full when upload doesn't happen. Since my pixel is away most of the time, its not that feasible to trigger physically.

What I've tried

  1. Use https://github.com/Genymobile/scrcpy manually go to the app and force upload using desktop/laptop. (Works but Not automated)
  2. Tasker + AutoInput to do the press commands (This is unreliable since the phone sometimes close tasker due to deep sleep or lack of ram?)

Question

Is there an adb command or root app to "Trigger upload"? Would be great if its adb since we can use home assistant to do it.

Suggestion or workaround for adb push on Pixel 2 running LineageOS

Firstly congrats for this idea. Inspired by your idea... I did the following for pixel 2. Install lineageos (with microg). Enable adb root (in developer settings) and adb WiFi. Then connect via wifi and connect the appropriate USB storage (using fat32 usb ssd)

Now, connect via cable

adb tcpip 5555
adb connect 192.168.1.26:5555

disconnect USB cable to phone

adb root
adb shell
df -h  (snippet attached)
/dev/fuse                   50G  5.3G   45G  11% /storage/emulated
/dev/block/vold/public:8,1 128G  7.9G  119G   1% /mnt/media_rw/45E3-1D10

seems crude but... this works

mount  /mnt/media_rw/45E3-1D10/  /mnt/pass_through/0/emulated/0/DCIM
am broadcast -a android.intent.action.MEDIA_SCANNER_SCAN_FILE -d file:///storage/emulated/0/DCIM

(Note that all files in DCIM natively will disappear as it became mount point. They dont get deleted but just temporarily disappear). After the broadcast command files in the USB drive automatically are visible in the photos app recognises everything and starts uploading.

TODO: push things directly from adb push into the external device. At the moment if I try

adb push  foo.png /mnt/media_rw/45E3-1D10/
remote fchown failed: Operation not permitted

If that works then I can leave this setup forever at my router and RW only the USB disk but WIP.

The way to directly mount smb share to internal storage

Download rclone mount from https://github.com/AvinashReddy3108/rclone-mount-magisk
replace rclone to newer version for SMB support
Install through magisk

adb shell su -c rclone --config="/mnt/sdcard/rclone.conf" config
(create config file. smb, sftp etc. are support)
adb shell su -c rclone --config="/mnt/sdcard/rclone.conf" mount remotename:d/ /mnt/runtime/default/emulated/0/mount --allow-other
(“remotename“ is the name of the remote created in config, "d/" is my windows smb share drive, in Windows File Explorer looks like \\192.168.x.x\d, mount point folder need to manually create)
Tested on pixel and pixel xl, QP1A.191005.007.A3

Doubts I've! Thanks in Advance :D

Doubts!

I have a few questions regarding the project. Your assistance in answering these or updating the repository's README file would be greatly appreciated:

  1. Can you please suggest a video/link to root the device?

    • Is unlimited storage available if we root a Pixel 1 device?
  2. Main Question: Can we charge a Pixel Device and connect to a hard disk at the same time?

    • If possible, could you please suggest a link to Amazon/Flipkart for purchasing the connecting device you are using?

Thank you for your support!

Hub

Thank you :) what hub is that in the picture?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.