Giter VIP home page Giter VIP logo

sdat2img's Introduction

sdat2img

Convert sparse Android data image (.dat) into filesystem ext4 image (.img)

Requirements

This binary requires Python 2.7 or newer installed on your system. It currently supports Windows, Linux, MacOS & ARM architectures.

Note: newer Google's Brotli format (system.new.dat.br) must be decompressed to a valid sparse data image before using sdat2img binary.

Usage

sdat2img.py <transfer_list> <system_new_file> [system_img]
  • <transfer_list> = input, system.transfer.list from rom zip
  • <system_new_file> = input, system.new.dat from rom zip
  • [system_img] = output ext4 raw image file (optional)

Example

This is a simple example on a Linux system:

~$ ./sdat2img.py system.transfer.list system.new.dat system.img

OTAs

If you are looking on decompressing system.patch.dat file or .p files, therefore reproduce the patching system on your PC, check imgpatchtools out by @erfanoabdi.

Info

For more information about this binary, visit http://forum.xda-developers.com/android/software-hacking/how-to-conver-lollipop-dat-files-to-t2978952.

sdat2img's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

sdat2img's Issues

Warn/error if file has .br extension

Yes, it's written in the readme.. but of course people are very lazy.
And for some reason the conversion doesn't crash badly if you give it the wrong file format.

And brotli hasn't a magic number, so you can only check for the file extension.

img file can't be mount: wrong fs type, bad option, bad superblock on ...

Following the tutorial from Lineage on how to extract proprietary blobs from zip files, I downloaded 2 zip archives from Lineage nightly builds repo: lineage-20.0-20230609-nightly-beyondx-signed.zip and lineage-20.0-20230602-nightly-beyondx-signed.zip.
Finally I ended up with 2 different vendor.img that can't be mount:

# mount vendor.img system/vendor/
mount: wrong fs type, bad option, bad superblock on /dev/loop27,
       missing codepage or helper program, or other error

       In some cases useful info is found in syslog - try
       dmesg | tail or so.

Image corrupted

Hi!
I've finished to convert from .dat to .img and the image is always corrupt. i tried with a lot of versions of lineage, but i get the same, image corrupt. Can someone help me!?

Error Win 10

Hello,

i am getting this Error in Windows 10 x64.

Traceback (most recent call last): File "C:\Users\marku\Downloads\Tool Unpack Repack DAT + IMG v3 x64 For Windows BY MTK THAI Developers (Thailand)\Android-Tool\sdat2img.py", line 143, in <module> main(TRANSFER_LIST_FILE, NEW_DATA_FILE, OUTPUT_IMAGE_FILE) File "C:\Users\marku\Downloads\Tool Unpack Repack DAT + IMG v3 x64 For Windows BY MTK THAI Developers (Thailand)\Android-Tool\sdat2img.py", line 68, in main version, new_blocks, commands = parse_transfer_list_file(TRANSFER_LIST_FILE) File "C:\Users\marku\Downloads\Tool Unpack Repack DAT + IMG v3 x64 For Windows BY MTK THAI Developers (Thailand)\Android-Tool\sdat2img.py", line 38, in parse_transfer_list_file version = int(trans_list.readline()) File "C:\Users\marku\AppData\Local\Programs\Python\Python38-32\lib\encodings\cp1252.py", line 23, in decode return codecs.charmap_decode(input,self.errors,decoding_table)[0] UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 4225: character maps to <undefined>

Encoding problem

I am on Win7 SP1 x64 with python 3.5.0 x64.
I get error: File "D:\bin\sdat2img.py", line 2 SyntaxError: encoding problem: utf8

I replaced second line: # -*- coding: utf-8 -*- and no more error!

Clarify License

Is this published under some LICENSE?
I might package this for NixOS (a linux distribution), but that will require some form of official license on the source code.

Syntax error

With latest commit, you have syntax error when running file..

sdat2img.py line 77 print('\nDetected system version: ', end=''), ^ SyntaxError: invalid syntax

Support `bsdiff` command

Hi,

tried to use the script on my MacbookPro without success.

installed bsdiff using lowbrow. However when i run the script i get:

./sdat2img/sdat2img.py ./update/system.transfer.list ./update/system.new.dat system.img
No valid command: bsdiff.

Installed python3.5.1 and run it with python3 -> same error

If you care to fix this and need any additional info i would be glad to help :)

Regards

I tried the command below, and It occurs error 'No such file or directory: 'system.transfer.list'',Why?

-> % ./sdat2img.py system.transfer.list '/Users/yangjianfei/Downloads/miui_PYXIS_V11.0.1.0.QFCCNXM_ed59cac77c_10.0/system.new.dat' system.img
sdat2img binary - version: 1.2

Traceback (most recent call last):
File "./sdat2img.py", line 143, in
main(TRANSFER_LIST_FILE, NEW_DATA_FILE, OUTPUT_IMAGE_FILE)
File "./sdat2img.py", line 68, in main
version, new_blocks, commands = parse_transfer_list_file(TRANSFER_LIST_FILE)
File "./sdat2img.py", line 35, in parse_transfer_list_file
trans_list = open(TRANSFER_LIST_FILE, 'r')
IOError: [Errno 2] No such file or directory: 'system.transfer.list'

"invalid sparse file format" converting a cm12.1 nightly and flashing on titan

I tried to convert cm12.1-20150511 for motorola G 2014 (titan board), and get a "Invalid sparse file format at header magi" [sic] error:

 $ unzip cm-12.1-20150511-NIGHTLY-titan.zip system.\*
Archive:  cm-12.1-20150511-NIGHTLY-titan.zip
signed by SignApk
 extracting: system.patch.dat        
  inflating: system.new.dat          
  inflating: system.transfer.list    

 $ sdat2img system.transfer.list system.new.dat system.img | fmt -s
Reading 32767 blocks... Writing to 0... Reading 2 blocks... Writing
to 32768... Reading 2 blocks... Writing to 32841... Reading
32201 blocks... Writing to 33334... Reading 2 blocks... Writing
to 65536... Reading 2 blocks... Writing to 98304... Reading 2
blocks... Writing to 98377... Reading 31020 blocks... Writing
to 98870... Reading 2 blocks... Writing to 131072... Reading 2
blocks... Writing to 163840... Reading 2 blocks... Writing to
163913... Reading 2 blocks... Writing to 196608... Reading 2
blocks... Writing to 229376... Reading 2 blocks... Writing to
229449... Reading 2 blocks... Writing to 262144... Reading 19986
blocks... Writing to 262637...
Done! Output image: /home/scott/motog/unzip/system.img

 $ fastboot flash system system.img
target reported max download size of 536870912 bytes
Invalid sparse file format at header magi
sending sparse 'system' (443721 KB)...
OKAY [ 27.842s]
writing 'system'...
OKAY [ 13.262s]
finished. total time: 41.104s

this appears to work, but I'm uncertain if the flash was incomplete or if the error was a real one to be concerned about (or, why the error was emitted).

Unable to unpack rom with UnicodeDecodeError

Hi,

I tried to unpack ASUS' rom WW_14.0400.1709.165 from here.

But when I input python sdat2img.py system.new.dat system.transfer.list system.img, I got an error message as below:-

sdat2img binary - version: 1.0
Traceback (most recent call last):
  File "sdat2img.py", line 142, in <module>
    main(sys.argv)
  File "sdat2img.py", line 87, in main
    version, new_blocks, commands = parse_transfer_list_file(TRANSFER_LIST_FILE)
  File "sdat2img.py", line 58, in parse_transfer_list_file
    version = int(trans_list.readline())
  File "/usr/lib64/python3.6/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 1024-1025: invalid continuation byte

I tried to change line 58 from trans_list = open(TRANSFER_LIST_FILE, 'r') to trans_list = open(TRANSFER_LIST_FILE, 'r', encoding = "ISO-8859-1"), but then I got a new error message as below:-

Traceback (most recent call last):
  File "sdat2img.py", line 142, in <module>
    main(sys.argv)
  File "sdat2img.py", line 87, in main
    version, new_blocks, commands = parse_transfer_list_file(TRANSFER_LIST_FILE)
  File "sdat2img.py", line 58, in parse_transfer_list_file
    version = int(trans_list.readline())
ValueError: invalid literal for int() with base 10: '\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x0

Any ideas?

Thanks in advance!

Support `move` command

When trying to apply an OTA I get:
No valid command: move.
any chance to implement this command to, or maybe share some insight/give some directions how it could be implemented?

Close file handler

We have a strange issue on some vps on Ubuntu server 16.04 x64.

After execution of your script we have about 90% of memory used as cache.
Normally OS automatically claim memory from cache when needed but in these vps this doesn't happen and so soon we get out of memory.

We tried to analize the problem and it seems due to a lot of file handlers used by this script. The cached memory amount in fact grows while script is executing.

I don't have enough skills to patch it so I'm asking you an help.

Command "bsdiff" is not valid.

download sdat2img, and it work my ubuntu.
try:
./sdat2img.py system.transfer.list system.new.dat system.img
but it is no working.

sdat2img binary - version: 1.2

Command "bsdiff" is not valid.

I installed pkg of bsdiff in my OS,and try again, but it's no work

give me a hand,tks

files:
system.transfer.list
system.patch.dat
system.new.dat

Can't mount on Mac

I can't seem to mount de image file in Mac OS X...

how can I do this?

Can't extract Marshmallow ROMs

Hi !
here my "extract log"
Skipping zero command.
Copying 32767 blocks into position 0...
Copying 2 blocks into position 32768...
Copying 2 blocks into position 32825...
Copying 32226 blocks into position 33307...
Copying 2 blocks into position 65536...
Copying 2 blocks into position 98304...
Copying 2 blocks into position 98361...
Copying 6452 blocks into position 98843...
Copying 2 blocks into position 131072...
Copying 2 blocks into position 163840...
Copying 2 blocks into position 163897...
Copying 2 blocks into position 196608...
Copying 17949 blocks into position 197090..

When I try to mount system.img :
mount: wrong fs type, bad option, bad superblock on /dev/loop0,
missing codepage or helper program, or other error

In some cases useful info is found in syslog - try
dmesg | tail or so.
On my dmseg
[13992.612662] EXT4-fs (loop0): bad geometry: block count 215040 exceeds size of device (215039 blocks)
[13994.338220] EXT4-fs (loop0): bad geometry: block count 215040 exceeds size of device (215039 blocks)
[13999.630333] EXT4-fs (loop0): bad geometry: block count 215040 exceeds size of device (215039 blocks)
[14080.991588] EXT4-fs (loop0): bad geometry: block count 215040 exceeds size of device (215039 blocks)
[14084.488430] EXT4-fs (loop0): couldn't mount as ext2 due to feature incompatibilities
[14119.141050] EXT4-fs (loop0): mounted filesystem with ordered data mode. Opts: (null)

It's on a 6.0 firmware.. Any clue ?

To fix this I have to use resiz2fs and resize it to a one block less image

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.