Giter VIP home page Giter VIP logo

sms-tools's Introduction

SMS Tools

Multipurpose import / export / merge tool for your text message history. (formally Android-SMS-DB-importer)

Now on PyPi (the python package index) and available using pip! pip install smstools puts smstools in your path, you're ready to roll.

Convert your message history between:

  • iOS 5, 6, and 7 databases directly (from backup or from your jailbroken phone directly)
  • Android mmssms.db database (directly from phone)
  • Android XML from the SMS Backup & Restore app
  • CSV files
  • JSON files
  • google voice data dump (see more details below)

Get all of your old messages onto your android phone.

Why?

  • Leaving Google Voice?
  • Getting a new iPhone or Android phone?
  • Want a searchable CSV, JSON, or XML file of your conversations?
  • Want to move all your messages from your past into a new, date-sorted, database?

##Howto:

Now on PyPi (the python package index) and available using pip! pip install smstools puts smstools in your path, you're ready to roll.

usage: smstools [-h] [--type {xml,json,android,csv,ios5,ios7,ios6}]
                    infiles [infiles ...] outfile

##Where do I get my files from?

  • iPhone:
  • Pull from your iTunes backup: See below!
  • If you're jailbroken: pull down /private/var/mobile/Library/SMS/sms.db
  • Android:
  • mmssms.db pulled from your phone: See below!
  • SMS Backup & Restore to get an XML file of your messages.
  • Google Voice: Work in progress

####Getting your iPhone messages from iTunes backup

  1. Open the right folder:
  • On Mac OS X open "~/Library/Application Support/MobileSync/Backup/"
  • On Windows 7/Vista open "C:\Users[USERNAME]\AppData\Roaming\Apple Computer\MobileSync\Backup"
  1. Open the most recent folder (the most recent backup)
  2. Get the file named "3d0d7e5fb2ce288813306e4d4636395e047a3d28" and rename it to sms.db

####What is the Android mmssms.db file? This is the sqlite file where your Android phone stores messages. To read or write it you'll need root. It's located at /data/data/com.android.providers.telephony/databases/mmssms.db

It may be possible to read it directly using ADB by running the adb pull command as com.android.providers.telephony. Otherwise use SMS Backup & Restore or something similar.

So why use this option?

  • It's orders of magnitude faster. Perfect for load testing mms applications with different databases (why I created this)
  • Much better database performance. After importing the output.xml file with SMSBackupRestore.apk my Messaging.apk was left completely unusable. SMSBackupRestore is great, but it doesn't handle tens of thousands of messages.

##My results When I run this tool on my Google Voice data it processes 6675 messages into 149 conversations in 15 seconds, which is 435 average entries/second. Not bad!

sms-tools's People

Contributors

flopraden avatar t413 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

sms-tools's Issues

sms.db not recognized

I'm trying to "convert" an Android mmssms.db database to an iOS one.
This is the output of the command:

$ smstools --type ios6 mmssms.db out.db
5086 messages read in 0 seconds from mmssms.db using parser smstools.android.Android
  latest text: ###########:
  #############
sorting all 5086 texts by date
Creating empty iOS 6 SQLITE db
built handles table with 99, chat with 144, chat_handle_join with 144 entries
built messages table with 5086 entries
changes saved to out.db
saved messages to out.db

After moving the file to /private/var/mobile/Library/SMS/ on an iOS 7 device (iPhone 4) and checking the permissions, the Messages app doesn't show any message. Am I doing something wrong?

Thank you.

Support group messages in iOS

Right now group messages are processed as standard messages.

I wrote this originally to work with Google Voice mainly, and it doesn't support group messages or MMS. It wasn't even on my radar.

Now if you have group messages from your iPhone or Android phone and export back to one of those phones it shouldn't loose your messages.

detailed documentation

I think it could be usefull to write a doc of this tool to help users to identify issues :

  • some of them are just due to a misunderstanding of the usage message
  • some of them are due to a bad configuration
  • some others are probably due to some bugs

I think this doc should gather :

  • an installation guide (i.e "pip install sms-tools")
  • the steps to follow to get the infile and to test outfile, and what should be the result
  • how to use runalltests.py and give the results with conveniant information to undersand what occured
  • a list of the error messages to understand better the script outputs

Here are some firsts thoughs which could probably help with the use of this script and relieve you for basic questions.
t413 : could you just say us if you are OK with giving us these informations to be able to use your work? Even if you do not have time actually, just to know if we should have hope that we can use it in a few weeks/months

UnboundLocalError: local variable 'new_texts' referenced before assignment

hello,

I get an error with the following command:
$ python sms_db_importer.py data/mmssms.db data/android.db
is Android
Traceback (most recent call last):
File "sms_db_importer.py", line 297, in
sms_main()
File "sms_db_importer.py", line 50, in sms_main
texts += new_texts
UnboundLocalError: local variable 'new_texts' referenced before assignment

By the way, the new_texts variable is not set if the code moves to the "elif "sms" in tableNames" case.

regards,
Damien

Readme.md: what is 'hon'?

Hello,

Your readme does not explaing what 'hon' is in your howto:

Howto:

Just run hon sms_db_importer.py sms-iOS-5.db gv.csv android.xml out.db, sit back, and relax.

regards,
Damien

Exported iOS messages have inverted sender & receiver

  • Application version: 0.2.1
  • Python version: 2.7.6
  • iOS: 8.1.1

Produced XML has sender & receivers inverted.
Observed after importing into Android & verified by looking at the XML.

I am reporting the issue in case some one else faces it. I will try to find some free time in the days following to investigate it and possibly provide a patch.

missing file used in runalltests.py

When I run runalltests.py, I get this output :

./runalltests.py
..test_parse_all_real_files() can't run, REAL_TEST_FILES_DIR does not exist.
..built handles table with 2, chat with 2, chat_handle_join with 2 entries
built messages table with 3 entries
processed 2 entries, 2 convos, (0 entries/sec)
finished in 0.00315690040588 seconds (average 949/second)

..

Ran 7 tests in 0.091s

OK

When I read smstools/core_tests.py it appears that REAL_TEST_FILES_DIR is defined to "/Users/timo/Downloads/testing_smstools/test_files/" and so it can not be read.

I think this file should be included into SMSTools package or a part of unit test can be deleted.

suggestion: add support to windows phone format

It is just a suggestion, for those who are interested in implementing it.

Since SMS-Tools tries to be "universal app", why not try to add support to Windows Phone 8.1? :-)

Windows Phone 8.1 supports exporting and importing sms to and from SD card via official app Transfer my Data. And here is a tutorial.

can't find '__main__' module

C:\Users\Alok\AppData\Local\Programs\Python\Python36\python.exe: can't find 'main' module in 'C:\Users\Alok\AppData\Local\Programs\Python\Python36\Lib\site-packages\smstools'

solved

running smoothly in debian

issues with readme: explain `type` and correct `ios7`

Here are two issues with README.md and README.rst.

In the beginning, I didn't know whether type parameter is used for specifying the type of input files or output files. Only after trying did I know that it is used for specifying the type of output file. Maybe you could add some explanation to it.

And in your readme in github, you write [--type {...ios7..}]. But it seems that you do not write any relevant code. And the one I installed using pip even do not support this parameter. I suggest adding some code, treating ios7 and ios8 as the alias of ios6, although I AM NOT sure whether the database format has changed or not.

Crash SMS app under CyanogenMode 11 - KitKat 4.4.4

Hello,

I have copied my sms.db from my rooted iphone 4 ( iOS 7 ).
I used your last Git version like that : ./bin/smstools --type android ~/sms.db ~/mmssms.db

38624 messages read in 0.580560922623 seconds from ~/sms.db
sorting all 38624 texts by date
Creating empty Android SQLITE db
processed 38623 entries, 695 convos, (115 entries/sec)
finished in 157.396391153 seconds (average 245/second)
changes saved to /home/guillaume/mmssms.db

I have keep a copy of my mmssms.db of my rooted Android phone ( I have direct ssh access )
I have set the correct owner/privileges on the copied file ( chown radio:radio / chmod 660 )
I have restarted the phone.

When I launch the SMS app, I have a popup to said the app have crashed.
I can't start the app :(

Any idea ?

If this could help, I give you the empty DB created by my Android version. May be the DB format have changed ?
http://demo.ovh.eu/fr/0c27f9a58507da9ac539c8e32b99f63e/

Regards,

Need some help. Newbie in python language.

In your readme.md I've found this suggestion:
usage: smstools [-h] [--type {xml,json,android,csv,ios5,ios7,ios6}]
infiles [infiles ...] outfile
Well, executing this command line gives me an error on "{".
I'm sorry but I need some help. I'm a newbie in python.
Basically, I have two files: an Android .xml file and an iPhone .db file. I want to merge them in a unique iphone .db file. What is the correct command line I have to execute?
Thank you in advance for your kind attention and patience.

Unicode encoding for sms

Hi,
thank you first for your script, a script like this one should be very helpfull to me.

Trying to convert my ios6 sms.db to android, I get this output when I launch "./bin/smstools --type android sms.db mmssms.db" :

Traceback (most recent call last):
File "./bin/smstools", line 59, in
print " " + term.blue(smstools.truncate(new_texts[-1].body))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 42: ordinal not in range(128)

Some sms contains french characters like accents.

SMS-Tools stops with invalid syntax error

Hi - Thanks for creating this tool! I installed SMStools with pip3, but when I try to run it, it gives up with the following syntax error. I would appreciate any help in getting it running!

$ smstools Hangouts.json test.xml File "/usr/local/bin/smstools", line 54
print term.black_on_blue("%d messages read in %d seconds from %s using parser %s") %
^
SyntaxError: invalid syntax

Handle (or at least fudge &/ memoize) the msg_group.hash value in iOS.

In iOS the threads table, called msg_group, has a 32-bit column called 'hash'. From what I can tell it's not the crc32, adler32, or fletcher32 checksum hash of the number field directly.

It's doubtful I'll find the exact hashing function it's using. The fallback is to memoize that value, the original number, and a 'cleaned' version of the number all together with the messages. If you're outputting to another iOS format I'll use it if it exists and fudge another hash using CRC if it doesn't.

Existing conversion support is flaky

When populating a mmssms.db that already has data, existing conversations are not updated, they're added anew.

Fix:
Either populate the convoMap object (which is a lookup table to go from a csv entry contactId to a thread id) with threads that already exist in the mmssms.db (very tricky) or use the existing table canonical_addresses for this. (less tricky).

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.