In IDA version 7, idaapi.dt_3byte == idaapi.dt_byte. This is causing all byte operands to be listed as size 3 instead of size 1.

I believe the issue could be fixed by reversing the order to idaapi.dt_byte and idaapi.dt_3byte in the sark.code.base.DTYP_TO_SIZE dictionary.

As an aside, there is no dt_3byte in the ida_ua module, which I believe is the module containing all the dt_* fields in the idaapi reorganization HexRays did.

I've sent an email to HexRays about this issue as well. I can share any updates they give me here.

Getting errors when importing in IDA after pip installation (IDA 6.9.151221, OS X 10.11.4):

Python>import sark
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Library/Python/2.7/site-packages/sark/__init__.py", line 28, in <module>
    from . import (core,
  File "/Library/Python/2.7/site-packages/sark/codeblocks.py", line 2, in <module>
    import networkx
  File "/Library/Python/2.7/site-packages/networkx/__init__.py", line 68, in <module>
    import networkx.utils
  File "/Library/Python/2.7/site-packages/networkx/utils/__init__.py", line 2, in <module>
    from networkx.utils.decorators import *
  File "/Library/Python/2.7/site-packages/networkx/utils/decorators.py", line 7, in <module>
    from decorator import decorator
ImportError: No module named decorator

Python>import sark
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Library/Python/2.7/site-packages/sark/__init__.py", line 28, in <module>
    from . import (core,
  File "/Library/Python/2.7/site-packages/sark/codeblocks.py", line 2, in <module>
    import networkx
  File "/Library/Python/2.7/site-packages/networkx/__init__.py", line 43, in <module>
    from networkx import release
ImportError: cannot import name release

Not sure how to fix this one! Looks like it changes on subsequent attempts to import.

Create scripts to dump as many types of information as possible from the test binaries.
Those can be used in Approval Tests to ensure that the output does not change as the code changes.

So, we have the Line. Which can be (in IDA) a code or a data or unknown.
So, it would be natural to write:

l = sark.Line(ea)
if l.is_code:
     i = l.code 
     if i.is_call:
         aaa

or perhaps:

 l = sark.Line(ea)
 if l.is_data:
     d = l.data
     if d.is_string:
         xxxx
     elif d.is_qword:
         yyy
     elif d.is_struct:
         zzz

while we have the Instruction type, which we could return on line.code property, we don't have a Data type to wrap the line.data.

So, how about adding it?
What do you think about it?

Small, simple binaries must be created to demonstrate Sark capabilities.
They should be small enough for IDA to analyze quickly, and have interesting parts so that they can show more of Sark.

Add function coloring to sark.Function() object

file: ui.py
317: node_ids = {node: self.AddNode(node) for node in self._graph.nodes_iter()}
319: for frm, to in self._graph.edges_iter():
say error

https://stackoverflow.com/questions/33734836/graph-object-has-no-attribute-nodes-iter-in-networkx-module-python
You are probably using the pre-release version of networkx-2.0 which has removed the nodes_iter() method and now provides the the nodes() method with the same functionality. See this for details on the networkx-2.0 changes.

you just should change
nodes_iter() -> nodes()
edges_iter() -> edges()

Best Regards

I'm getting the following error when trying to install using pip:

Requirement already satisfied (use --upgrade to upgrade): awesomelib in c:\python27\lib\site-packages (from -r require
nts.txt (line 1))
Collecting yapsy (from -r requirements.txt (line 2))
  Using cached Yapsy-1.11.223.tar.gz
Collecting attrdict (from -r requirements.txt (line 3))
  Using cached attrdict-2.0.0.tar.gz
Collecting PyYAML (from -r requirements.txt (line 4))
  Using cached PyYAML-3.11.tar.gz
Obtaining Sark from git+https://github.com/tmr232/Sark.git#egg=Sark (from -r requirements.txt (line 5))
  Cloning https://github.com/tmr232/Sark.git to c:\tools\die\src\sark
Requirement already satisfied (use --upgrade to upgrade): six in c:\python27\lib\site-packages (from attrdict->-r requ
ements.txt (line 3))
Collecting clipboard (from Sark->-r requirements.txt (line 5))
  Downloading clipboard-0.0.4.tar.gz
Requirement already satisfied (use --upgrade to upgrade): networkx in c:\python27\lib\site-packages (from Sark->-r req
rements.txt (line 5))
Collecting wrapt (from Sark->-r requirements.txt (line 5))
  Downloading wrapt-1.10.5.tar.gz
Collecting pyperclip>=1.3 (from clipboard->Sark->-r requirements.txt (line 5))
  Downloading pyperclip-1.5.24.zip
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 20, in <module>
      File "c:\docume~1\user\locals~1\temp\pip-build-3yjab7\pyperclip\setup.py", line 8, in <module>
        version=__import__('pyperclip').__version__,
      File "pyperclip\__init__.py", line 103, in <module>
        copy, paste = determine_clipboard()
      File "pyperclip\__init__.py", line 53, in determine_clipboard
        return init_windows_clipboard()
      File "pyperclip\windows.py", line 75, in init_windows_clipboard
        wcscpy_s = ctypes.cdll.msvcrt.wcscpy_s
      File "c:\python27\lib\ctypes\__init__.py", line 378, in __getattr__
        func = self.__getitem__(name)
      File "c:\python27\lib\ctypes\__init__.py", line 383, in __getitem__
        func = self._FuncPtr((name_or_ordinal, self))
    AttributeError: function 'wcscpy_s' not found

    ----------------------------------------

I put ptvsd_enable.py in the IDA plugins directory and, it opens without error. But when I want to attach IDA Pro to Visual Studio I get an error with this sentence "Unable to attach to the process. General Exception". Both process are running with administrator privilege.

How I should solve this problem?

Issue in codeblocks.get_nx_graph. The graph is created by adding all the edges between blocks, so blocks without edges are not included.
For example, when calling codeblocks.get_nx_graph on a function with a single block, it returns an empty graph.

It should be possible to do sark.Line() == sark.Line() properly.

I am iterating through a list of addresses and colors for functions, lines, and codeblocks. I use identical code for each with the only difference being whether sark.Function(address)/sark.Line(address) is called or sark.CodeBlock(address). The issue is that when called on my test files the line and function loops take only .05 seconds. The loop that calls codeblock takes 6.5 and has dramatically decreased the speed of my program.

The following miscellaneous documentation page contains incorrect module names for the mentioned functions.
https://github.com/tmr232/Sark/blob/master/docs/api/Miscellaneous.rst

e.g. The get_ea function is written in the sark.core module. However, it's not imported to the init module of the sark package. Therefore, it cannot be accessed without using the absolute module name of the function (which should be sark.core.get_ea).

It can be solved by importing specific functions from the core module (in the init module) or modifying the Misc's documentation page.

Hi,

First off. Sark is what I've always wished the IDA API looked like. Thank you so much for reigniting the joy and excitement of binary analysis!! The graph API looks extremely powerful, and the concepts of Sark feel very well thought out. Hats off!

Now, for the issue I happened to stumble upon today, when exploring what the Sark world had to offer.

It seems there is currently a discrepancy between how IDA and Sark handles is_code and is_tail (I did not check is_data, but it would make sense to check that too as part of this issue).

The proof of concept is as follows:

Python>is_code(idc.GetFlags(0x4785A4))
True
Python>is_code(idc.GetFlags(0x4785A5))
False
Python>is_tail(idc.GetFlags(0x4785A4))
False
Python>is_tail(idc.GetFlags(0x4785A5))
True
Python>sark.Line(ea=0x4785A4).is_code
True
Python>sark.Line(ea=0x4785A5).is_code
True
Python>sark.Line(ea=0x4785A4).is_tail
False
Python>sark.Line(ea=0x4785A5).is_tail
False
Python>print(sark.Line(ea=0x4785A4))
[004785A4]    test    esi, esi
Python>print(sark.Line(ea=0x4785A5))
[004785A4]    test    esi, esi

I can't share this database, however, I hope this should be easy to reproduce. If not, just ping me and we'll do some troubleshooting together.

Oh, and for the record, this is at IDA 7.3, using the 6.x branch of Sark (downloaded today, so should be the latest).

Cheers,
Robin

When I debug the sample idapython script followed the "Debugging IDAPython Scripts" chapter using Python Tools for Visual Studio, it does not work and the error message is "ImportError: No module named idaapi". How can i fix this bug?

I propose to add a property to a line object.
A property that will return the bytes of the object.
something like:
return idaapi.get_many_bytes(self.ea, self.size)

Hi,

I successfully installed sark with Windows 64-bit + IDA 6.8 + Python 2.7 32-bit in the past but cannot make it work with IDA 6.9.

I get the following error::

autostruct.py: cannot import name QtCore
Traceback (most recent call last):
  File "C:\Program Files (x86)\IDA 6.9\python\idaapi.py", line 601, in IDAPython_ExecScript
    execfile(script, g)
  File "C:/.../Sark/plugins/autostruct.py", line 4, in <module>
    import sark
  File "C:\Python27\lib\site-packages\sark-0.1.0-py2.7.egg\sark\__init__.py", line 28, in <module>
    from . import (core,
  File "C:\Python27\lib\site-packages\sark-0.1.0-py2.7.egg\sark\code\__init__.py", line 9, in <module>
    from . import line
  File "C:\Python27\lib\site-packages\sark-0.1.0-py2.7.egg\sark\code\line.py", line 8, in <module>
    from ..ui import updates_ui
  File "C:\Python27\lib\site-packages\sark-0.1.0-py2.7.egg\sark\ui.py", line 6, in <module>
    from .qt import MenuManager
  File "C:\Python27\lib\site-packages\sark-0.1.0-py2.7.egg\sark\qt.py", line 4, in <module>
    from cute import QtCore, QtWidgets, QtGui, use_qt5, connect, disconnect, form_to_widget
ImportError: cannot import name QtCore

This is because I didn't manage to install PyQt5. And Sark relies on cute which relies on PyQt5 [1] for my setup.

Any hint on how to make sark work?

Thanks,
saidelike

[1] https://github.com/tmr232/Cute/blob/master/cute.py#L45

sark.CodeBlock(0)

results in

AttributeError: 'NoneType' object has no attribute 'dict'

It should be a descriptive error instead.

When traversing the pointer table, for example, it would make sense to traverse it with a native pointer size for the platform. This would allow to write code that would work for both 32 and 64 bit ARM platforms without modification.

The instruction at 'sz_ea' is MOV W3, #0x50
The following test code should print the same operands twice.

            print sark.Line(sz_ea)
            opns = sark.Line(sz_ea).insn.operands
            print opns
            print "------------------------"
            print opns[0]
            print opns[1]
            print "------------------------"
            print sark.Line(sz_ea).insn.operands[0]
            print sark.Line(sz_ea).insn.operands[1]

The output is:

[FFFFFF80206CDFD0]    MOV             W3, #0x50
[<Operand(n=0, text='W3')>, <Operand(n=1, text='#0x50')>]
------------------------
W3
W3
------------------------
W3
#0x50

I've observed other strange behavior related to operands, which is a bit harder to reproduce.
However, I believe that it might be a manifestation of the same problem.

A Lines object should be added to allow operations like reversing the line order. Would make possible lines like:

sark.CodeBlock(0x1234).lines.reverse

import sark
graph = sark.get_nx_graph(sark.Function().ea)
viewer = sark.ui.NXGraph(graph, title="My Graph", handler=sark.ui.AddressNodeHandler())
viewer.Show()
creates the following error message:

In IDA's SDK, all instruction fields are pre-defined and documented asside for phrase values, which can be used in whatever way the processor module writer deems fit.
While this makes a lot of sense for IDA, it does make it harder to parse. So getting the registers for an operand that looks like [eax + ebx + 4] (or any phrase with 2 registers) does not currently work.
This will probably be resolved soon, but be aware.

Hi,

I tried to debug IDAPython code according to your blog post.
http://sark.readthedocs.io/en/latest/debugging.html

I could install required packages and attach to IDA but I cannot break the execution.
For example, I set the breakpoint in the following code but it was ignored and print the message.

Any thought?
The attaching looks working and message is also displayed in VS output window.
I use IDA 7.0/6.95 and PTVS 2.2.6.

Hello,

Almost in every class there are many iterator/generator properties and methods.
Sometimes it's a method (e.g. CodeBlock.succs() is a generator of succeeding blocks) and sometimes it's a property (e.g. CodeBlock.prev). I failed to understand when a property is used and when a method is used which makes the library less usable. It's quite confusing and forces the user to check the documentation more frequently than necessary (which reduces usability and makes the library 'surprising' in a bad way).

If there is any logic behind these decisions, can you share and document them?
If not, can you align it please?

Thanks

Allow the following:

function = sark.Function(sark.Line())

class IdaTestPlugin(idaapi.plugin_t):
flags = idaapi.PLUGIN_UNL

test code as above,setting flags to idaapi.PLUGIN_UNL ,that makes the plugin unloaded after running,and the plugin will be reloaded on the next time.however i can only debug at the first time.the visual studio does not break the script after reloaded.

Sometimes it's useful to be able to iterate over all function's code blocks. Just like lines.
What do you think?

Currently, Sark does not provide information about ARM pre-index and post-index options.

This can be done using insn_t.auxpref:

The main issue is whether to add it as a property of the instruction (probably), the operand (less likely), or both.

I'm not sure if this already exists, but I didn't find it. The following is not probably the best implementation but represents the goal.

def get_symbolic_name(ea):
    """Returns function offset string.
    """
    function = sark.Function(ea)
    if function.ea == ea:
        return function.name
    else:
        return "{0.name}+{1:X}".format(function, ea - function.ea)

In [203]: for xref in sark.Function(name='SomeFunction').xrefs_to:
     ...:     print get_symbolic_name(xref.frm)
     ...:     
sub_4060B7+17
SomeFunctionName+16

In an attempt to remove jumps from the xrefs_from property, I accidentally ended up removing recursion altogether (by removing all xrefs that are internal to a function).
This is not the desired behavior and should be fixed.
I will probably add some new property for either the recursion-enabled version or the current one to make it more friendly.

Hi,

When you write:
block_list = [sark.CodeBlock()]
sark.CodeBlock() in block_list

you get False.
It would be very nice to have equality of CodeBlocks so it won't be necessary to write my own equality function.

• https://github.com/nirizr/pytest-idapro
• https://github.com/approvals/ApprovalTests.Python

in case an anterior or posterior comment has an empty line anywhere,
the text after the empty line doesn't appear.

seems like the empty line is treated as None and
return "\n".join(iter(lines.next, None))
stops when it gets to it

not so trivial and could save time.

client should have python3.x installed (that's trivial), rename the ~/ida7.4/plugins/idapython*.disabled and run ~/ida7.4/idapyswitch.

btw, great project.

As an example use xrefsgraph.py and execute show_xref_graph(SOME_ADDRESS) try to create a group of nodes, then just click on it and you will notice that the new id (group node) is not available in self.__graph.node[self[node_id]].

In my use of Sark, I am often checking function flags to see if a function is a recognized library or is a thunk function.

It would be quite convenient to have a properties such as Function.is_library or Function.is_thunk. This would make my code much more readable.

Is this a feature you would consider including? If so, I don't mind doing the work and creating a PR.

Ida 7.5 + python3

[PluginLoader] Loading plugins from system-wide and user-specific lists:
[PluginLoader]   System-wide List:      C:\Program Files\IDA Pro 7.5\cfg\plugins.list
[PluginLoader]   User-specific List:    C:\Users\stephen.eckels\AppData\Roaming\Hex-Rays\IDA Pro\plugins.list
[PluginLoader]   Project-specific List: C:\Users\stephen.eckels\Desktop\labs\plugins.list
[PluginLoader] Failed creating system plugin list at C:\Program Files\IDA Pro 7.5\cfg\plugins.list
C:\Users\stephen.eckels\source\repos\Sark\plugins\highlight_calls.py: string argument without an encoding
Traceback (most recent call last):
  File "C:\Program Files\IDA Pro 7.5\python\3\ida_idaapi.py", line 604, in IDAPython_ExecScript
    exec(code, g)
  File "C:/Users/stephen.eckels/source/repos/Sark/plugins/highlight_calls.py", line 4, in <module>
    from ida_settings import IDASettings
  File "C:\Users\stephen.eckels\AppData\Local\Programs\Python\Python37\lib\site-packages\ida_settings\__init__.py", line 1, in <module>
    from .ida_settings import IDASettings, PermissionError
  File "C:\Users\stephen.eckels\AppData\Local\Programs\Python\Python37\lib\site-packages\ida_settings\ida_settings.py", line 859, in <module>
    VALUE_1 = bytes("hello")
TypeError: string argument without an encoding

When running sark scripts via idat, sark causes IDA to abort because of qt imports
(I'm guessing the qt DLL can't be loaded into a console process? Maybe it's looking for user32 functions?)

Specifically, IDA aborts the script's execution with the error:
DLL load failed: The specified procedure could not be found

Workaround
Comment out any code that uses Sark's ui module:
__init__.py Lines 36, 48
function.py Line 9 (and subsequent uses of updates_ui)
line.py Line 8 (and subsequent uses of updates_ui)

Steps to reproduce (Windows)
Run from command prompt:
"C:\Program Files\IDA 7.2\idat.exe" -S"somesarkscript.py" someidb.idb

I've seen that the actual Registers implementation doesn't follows the line of sark library (human and pythonic ida usage). I still find it as ugly as the actual ida c++ interface to get registers.

So I came up with this implementation, it doesn't walks the deepness of chained registers, but it wouldn't be hard to implement it.

class Registers(object):
    def __init__(self):
        self._schema = idaapi.dbg_get_registers()
        self._registers = [s[0].lower() for s in schema]

    def __getattr__(self, name):
        return idc.GetRegValue(name)

    def __setattr__(self, name, value):
        if hasattr(self, "_registers") and name in self._registers:
            idc.SetRegValue(value, name)
        else:
            self.__dict__[name] = value

    def __dir__(self):
        return self.__dict__.keys() + self._registers

This allows you to easily access registers, and assign values. Also you can TAB on your ipython ida instance and you will get all the possible registers to work with.

Tell me what do you think about it and if we can improve this section.

if I want to write a script that applies a function prototype I cannot do this from sark and have to to fall back to IDAPython with SetType / ApplyType.

See http://reverseengineering.stackexchange.com/questions/12791/how-to-get-cross-references-to-a-struct-type-in-ida-by-idapython-and-add-comment/12799#12799

Hi,

I am running IDA Pro 6.8 and I want to try Sark. So I installed it with the stated pip command and wanted to import it in IDA. Ida tells me: "No module named sark".

When pulling the sark repo and copying the sark folder into IDA's python directory, it finds the sark modules but not the dependencies which I also installed via pip.

I am new to IDA scripting and I don't know what I am doing wrong. Can anybody help me with the installation from sark? That would be gread!

Thanks in advance

Currently, the workflow of AutoEnum is a bit annoying. It would be nice to just rename a constant in the code, not caring where it is saved. This can be done using a different hotkey and a default enum.

Also, it would be nice to add a move-to-enum option, to later move the desired constants into new enums, preserving all uses.

Should exist, the just like the Line.has_name.

This is not an issue, but just to put it in the queue. I find very useful the relationship between the decompilation line and his dissasembly address.

Taking this code as an example.

def get_decompiled_line(source, ea):
    if ea not in source.eamap:
        print 'strange, %x is not in %x eamap' % (ea, source.entry_ea)
        return
    insnvec = source.eamap[ea]
    lines = []
    for stmt in insnvec:
        qp = idaapi.qstring_printer_t(source.__deref__(), False)
        stmt._print(0, qp)
        s = qp.s.split('\n')[0]        
        lines.append(s)
    return '\n'.join(lines)

ea = idaapi.get_screen_ea()
source = idaapi.decompile(ea)
print get_decompiled_line(source, ea)

Imagine that for each ea in the meaningful plugin you add this information, sounds interesting right?

The bad part is that the layer of this "source" object isn't the friendliest possible. So I think that the powerful of this relationships (editing pseudocode and so on) would be amazing if Sark makes it Pythonic.

 Could not find a version that satisfies the requirement idacute (from sark) (from versions: )
No matching distribution found for idacute (from sark)

Using Python 3.6.2

Lets consider adding the make_string() to the data.py
For example, something like this:

def make_string(start, len=0):
    return idc.MakeStr(start, idaapi.BADADDR if len is 0 else start + len)

This needs to be fixed with backwards compatibility 😢
And I need to start reading SDK history. Not sure how I missed this one.

The phrase update to Sark broke support for ARM instructions, as ARM phrases are not properly parsed, and raise an exception. This should be resolved ASAP. Even if only to avoid exceptions.