angr / angr-dev Goto Github PK

View Code? Open in Web Editor NEW

113.0 17.0 96.0 362 KB

Some helper scripts to set up an environment for angr development.

License: BSD 2-Clause "Simplified" License

Shell 76.48% Python 8.48% Batchfile 0.76% Dockerfile 2.58% Nix 1.99% PowerShell 9.72%

angr-dev's Introduction

angr-dev

This is a repository to make installing a development version of angr easier.

Install

To set up angr for development, automatically install dependencies, and automatically create a python virtualenv, do:

./setup.sh -i -e angr

This will grab and install angr. You can launch it with:

$ workon angr
(angr) $ ipython
[1] import angr

MacOS

Mojave seems to be working with the current version, so this fixing might not be necessary. Will need to verify.

If you are working on macOS, you have to run the fix_macOS.sh script while in your virtualenv to fix the native libraries in angr. This is necessary, since macOS introduced restrictions for relative paths in dynamic libraries.

./fix_macOS.sh

Install (docker)

Alternatively, you can use the dockerfile:

$ docker build -t angr angr-dev
$ docker run -it angr

Updating

To update angr, simply pull all the git repositories.

./git_all.sh pull

For repositories that have C components, you might have to rebuild.

pip install -e ./pyvex && pip install -e ./angr

Issues

I want to use my github username and password via https

Comment out the GIT_ASKPASS=true line. Or, just use ssh.

angr-dev's People

Contributors

Stargazers

Watchers

Forkers

chubbymaggie zhuyue1314 0xbc juliusdedekind hira-a lisabahler gynophage shadown windhl zardus tiffanyb r1ceplus gitcollect damnya pwntester joeleong peperunas tudorciochina aoikonomopoulos chadbrewbaker manouchehri bannsec m1ghtym0 tyb0807 codemaxx fmagin johnnylchang lixuan2416 mineralwater3 techlord-rce whsz6 lordmoses rainoftime qq332982511 bennofs bspar perce7al afaerber plusls f0829 kyle-kyle lldder fiberx fengxl3320 kevorr mephi42 beenjaminmb amazingghost cvvt yuzhouheike twizmwazin joostrijneveld sraboy balbassam xylearn nyamisty boshijingang alexeyan cquaker lmkalg mborgerson 5l1v3r1 jiahao42 stevenjiang1110 hwu71 fjh1997 rjt-gupta aos002 biancaguo shingarov kompanek shitongzhu jeffreyfairbanks dlocasc sandwich125 muqi-zou imikr4m ice-tong ma5ker doctorscrad yuzeming ulugbekna pascal-0x90 edmcman novafacing wntress r0x0d flounderk pbhdk qwestduck draperlaboratory karzanwang voodoochild99 yuethub brayand9106 johenanli

angr-dev's Issues

Incompatible versions of some packages due to updates in the last three days

Because you updated some packages three days ago, the versions of 'aliment', 'angr', 'angr management', 'clarity', 'cle', 'pyvex', and 'archr' packages are inconsistent, leading to incompatibilities. Although I manually adjusted these packages to version 9.2.18, there are still errors in use. How to solve this problem?

about usage of the feature ReachingDefinitions engine

hi there!

I've read the post A reaching definition engine for binary analysis built-in in angr, and finished the experiments of the ASU class CSE545 Guest Lecture: Binary Analysis. everything works well on the early edition angr 9.0.6885

however in the latest angr-dev, it seems like no function get_objects_by_offset() to call inside register_definitions. is that a deprecated feature? or just change the way to use it?

thanks!

`./setup.sh -i -e angr` due to `archr` requiring `--no-build-isolation` when `pip` installing

Description

Below cmake cannot find Ninja, which we install via pip as declared as a build dependency by pyproject.toml

Building wheels for collected packages: archr
  Building editable for archr (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building editable for archr (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [62 lines of output]
      running editable_wheel
      creating /tmp/pip-wheel-2doyw0dp/tmp68sh4sno/archr.egg-info
      writing /tmp/pip-wheel-2doyw0dp/tmp68sh4sno/archr.egg-info/PKG-INFO
      writing dependency_links to /tmp/pip-wheel-2doyw0dp/tmp68sh4sno/archr.egg-info/dependency_links.txt
      writing requirements to /tmp/pip-wheel-2doyw0dp/tmp68sh4sno/archr.egg-info/requires.txt
      writing top-level names to /tmp/pip-wheel-2doyw0dp/tmp68sh4sno/archr.egg-info/top_level.txt
      writing manifest file '/tmp/pip-wheel-2doyw0dp/tmp68sh4sno/archr.egg-info/SOURCES.txt'
      reading manifest file '/tmp/pip-wheel-2doyw0dp/tmp68sh4sno/archr.egg-info/SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      adding license file 'LICENSE'
      writing manifest file '/tmp/pip-wheel-2doyw0dp/tmp68sh4sno/archr.egg-info/SOURCES.txt'
      creating '/tmp/pip-wheel-2doyw0dp/tmp68sh4sno/archr-9.2.25.dev0.dist-info'
      adding license file "LICENSE" (matched pattern "LICENSE")
      creating /tmp/pip-wheel-2doyw0dp/tmp68sh4sno/archr-9.2.25.dev0.dist-info/WHEEL
      /home/zwimer/.virtualenvs/test2/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
        warnings.warn(
      running build
      Building Archr support libraries
      CMake Deprecation Warning at CMakeLists.txt:1 (cmake_minimum_required):
        Compatibility with CMake < 2.8.12 will be removed from a future version of
        CMake.

        Update the VERSION argument <min> value or use a ...<max> suffix to tell
        CMake that the project does not need compatibility with older versions.


      CMake Error: CMake was unable to find a build program corresponding to "Ninja".  CMAKE_MAKE_PROGRAM is not set.  You probably need to select a different build tool.
      CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage
      CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
      -- Configuring incomplete, errors occurred!
      See also "/home/zwimer/bar/archr/archr/implants/udp_tcp_convert/build/CMakeFiles/CMakeOutput.log".
      Traceback (most recent call last):
        File "/home/zwimer/.virtualenvs/test2/lib/python3.10/site-packages/setuptools/command/editable_wheel.py", line 138, in run
          self._create_wheel_file(bdist_wheel)
        File "/home/zwimer/.virtualenvs/test2/lib/python3.10/site-packages/setuptools/command/editable_wheel.py", line 284, in _create_wheel_file
          files, mapping = self._run_build_commands(dist_name, unpacked, lib, tmp)
        File "/home/zwimer/.virtualenvs/test2/lib/python3.10/site-packages/setuptools/command/editable_wheel.py", line 257, in _run_build_commands
          self.run_command("build")
        File "/home/zwimer/.virtualenvs/test2/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
          self.distribution.run_command(command)
        File "/home/zwimer/.virtualenvs/test2/lib/python3.10/site-packages/setuptools/dist.py", line 1217, in run_command
          super().run_command(command)
        File "/home/zwimer/.virtualenvs/test2/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
          cmd_obj.run()
        File "<string>", line 38, in run
        File "/home/zwimer/.virtualenvs/test2/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 340, in execute
          util.execute(func, args, msg, dry_run=self.dry_run)
        File "/home/zwimer/.virtualenvs/test2/lib/python3.10/site-packages/setuptools/_distutils/util.py", line 343, in execute
          func(*args)
        File "<string>", line 23, in _build_support_libraries
        File "/usr/lib64/python3.10/subprocess.py", line 524, in run
          raise CalledProcessError(retcode, process.args,
      subprocess.CalledProcessError: Command '['cmake', '.', '-Bbuild']' returned non-zero exit status 1.
      error: Support for editable installs via PEP 660 was recently introduced
      in `setuptools`. If you are seeing this error, please report to:

      https://github.com/pypa/setuptools/issues

      Meanwhile you can try the legacy behavior by setting an
      environment variable and trying to install again:

      SETUPTOOLS_ENABLE_FEATURES="legacy-editable"
      [end of output]
      
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building editable for archr
Failed to build archr
ERROR: Could not build wheels for archr, which is required to install pyproject.toml-based projects
[!!] 13:44:31 pip failure (--no-build-isolation -e archr).

Steps to reproduce the bug

./setup.sh -i -e angr

Environment

A clean clone of master: 059e9bd

Additional context

No response

pypy environment setup failed

In the pypy_venv.sh, it tries to execute the following command:

mkvirtualenv -p "$PWD/"pypy3.6-*/bin/pypy3 $NAME

However, it failed because there's no pypy3.6-* but only pypy3.7-v7.3.3-linux64.
So I changed the command from 3.6 to 3.7 and it worked.

Segmentation fault

I installed angr-dev.But how to do with segmentation fault when i use it?

autogenerated protobuf type error

SO: Windows 10, 1903, 64 bit
Python: 3.5 64 bit

To reproduce:

import angr
Traceback (most recent call last):
File "", line 1, in
File "c:\users\sym\desktop\angr-dev-master\angr-dev-master\angr\angr_init_.py", line 30, in
from .sim_procedure import SimProcedure
File "c:\users\sym\desktop\angr-dev-master\angr-dev-master\angr\angr\sim_procedure.py", line 5, in
from cle import SymbolType
File "c:\users\sym\desktop\angr-dev-master\angr-dev-master\cle\cle_init_.py", line 18, in
from .loader import *
File "c:\users\sym\desktop\angr-dev-master\angr-dev-master\cle\cle\loader.py", line 14, in
import claripy
File "c:\users\sym\desktop\angr-dev-master\angr-dev-master\claripy\claripy_init_.py", line 21, in
from . import backends as backends_module
File "c:\users\sym\desktop\angr-dev-master\angr-dev-master\claripy\claripy\backends_init.py", line 785, in
from .backend_smtlib import BackendSMTLibBase
File "c:\users\sym\desktop\angr-dev-master\angr-dev-master\claripy\claripy\backends\backend_smtlib.py", line 3, in
from pysmt.shortcuts import Symbol, String, StrConcat, NotEquals,
File "C:\Python35\lib\site-packages\pysmt\shortcuts.py", line 38, in
import pysmt.smtlib.parser
File "C:\Python35\lib\site-packages\pysmt\smtlib\parser_init_.py", line 91, in
pyxbuild_dir=build_dir)
File "C:\Python35\lib\site-packages\pyximport\pyximport.py", line 191, in build_module
reload_support=pyxargs.reload_support)
File "C:\Python35\lib\site-packages\pyximport\pyxbuild.py", line 102, in pyx_to_dll
dist.run_commands()
File "C:\Python35\lib\distutils\dist.py", line 955, in run_commands
self.run_command(cmd)
File "C:\Python35\lib\distutils\dist.py", line 974, in run_command
cmd_obj.run()
File "C:\Python35\lib\site-packages\Cython\Distutils\old_build_ext.py", line 186, in run
_build_ext.build_ext.run(self)
File "C:\Python35\lib\distutils\command\build_ext.py", line 339, in run
self.build_extensions()
File "C:\Python35\lib\site-packages\Cython\Distutils\old_build_ext.py", line 194, in build_extensions
self.build_extension(ext)
File "C:\Python35\lib\distutils\command\build_ext.py", line 533, in build_extension
depends=ext.depends)
File "C:\Python35\lib\distutils_msvccompiler.py", line 304, in compile
self.initialize()
File "C:\Python35\lib\distutils_msvccompiler.py", line 197, in initialize
vc_env = _get_vc_env(plat_spec)
File "C:\Python35\lib\distutils_msvccompiler.py", line 85, in _get_vc_env
raise DistutilsPlatformError("Unable to find vcvarsall.bat")
distutils.errors.DistutilsPlatformError: Unable to find vcvarsall.bat

However, i can see that th efile "vcvarsall.bat" exists in "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build"

Any workaround to fix it?

"libvex failed to initialize" error when import angr

Hi, the error occurs when I just git_all.sh pull the latest angr-dev branch.

The detailed error info is as below:

>>> import angr
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/dsk/Tools/Labs/angr-dev/angr/angr/__init__.py", line 30, in <module>
    from .sim_procedure import SimProcedure
  File "/home/dsk/Tools/Labs/angr-dev/angr/angr/sim_procedure.py", line 5, in <module>
    from cle import SymbolType
  File "/home/dsk/Tools/Labs/angr-dev/cle/cle/__init__.py", line 18, in <module>
    from .loader import *
  File "/home/dsk/Tools/Labs/angr-dev/cle/cle/loader.py", line 1099, in <module>
    from .backends import MetaELF, ELF, PE, Soot, Blob, ALL_BACKENDS, Backend
  File "/home/dsk/Tools/Labs/angr-dev/cle/cle/backends/__init__.py", line 415, in <module>
    from .elf import ELF, ELFCore, MetaELF
  File "/home/dsk/Tools/Labs/angr-dev/cle/cle/backends/elf/__init__.py", line 1, in <module>
    from .elf import ELF
  File "/home/dsk/Tools/Labs/angr-dev/cle/cle/backends/elf/elf.py", line 15, in <module>
    from .metaelf import MetaELF, maybedecode
  File "/home/dsk/Tools/Labs/angr-dev/cle/cle/backends/elf/metaelf.py", line 1, in <module>
    import pyvex
  File "/home/dsk/Tools/Labs/angr-dev/pyvex/pyvex/__init__.py", line 44, in <module>
    pvc = _find_c_lib() # type: Any # This should be properly typed, but this seems non trivial
  File "/home/dsk/Tools/Labs/angr-dev/pyvex/pyvex/__init__.py", line 39, in _find_c_lib
    raise ImportError("libvex failed to initialize")
ImportError: libvex failed to initialize

Have releaser.sh generate version requirements for the various packages

setup.sh no longer works for me

It seems the setup.sh script is not working properly anymore. Normally, I would successfully build all angr-dev stuff with docker build -t angr - < angr-dev/Dockerfile, but now I am getting the following:

[+] 14:25:29 All done! Execute "workon angr" to use your new angr virtual
[+] 14:25:29 environment. Any changes you make in the repositories will reflect
[+] 14:25:29 immediately in the virtual environment, with the exception of things
[+] 14:25:29 requiring compilation (i.e., pyvex). For those, you will need to rerun
[+] 14:25:29 the install after changes (i.e., "pip install -e pyvex").
[+] 14:25:29 Checking dependencies...
[+] 14:25:29 Enabling virtualenvwrapper.
[+] 14:25:30 Creating pypy virtualenv angr-pypy...
[!!] 14:25:32 An error occurred on line 342. Saved output:
[+] Checking dependencies...
[+] Enabling virtualenvwrapper.
Requirement already satisfied: virtualenvwrapper in /usr/lib/python3/dist-packages (4.8.4)
[+] Creating pypy virtualenv angr-pypy...
--2020-08-26 14:25:30--  https://bitbucket.org/pypy/pypy/downloads/
Resolving bitbucket.org (bitbucket.org)... 18.205.93.1, 18.205.93.0, 18.205.93.2, ...
Connecting to bitbucket.org (bitbucket.org)|18.205.93.1|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2020-08-26 14:25:31 ERROR 403: Forbidden.

--2020-08-26 14:25:31--  https://bitbucket.org/pypy/pypy/downloads/
Resolving bitbucket.org (bitbucket.org)... 18.205.93.1, 18.205.93.0, 18.205.93.2, ...
Connecting to bitbucket.org (bitbucket.org)|18.205.93.1|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2020-08-26 14:25:32 ERROR 403: Forbidden.


bzip2: Compressed file ends unexpectedly;
	perhaps it is corrupted?  *Possible* reason follows.
bzip2: Inappropriate ioctl for device
	Input file = (stdin), output file = (stdout)

It is possible that the compressed file(s) have become corrupted.
You can use the -tvv option to test integrity of such files.

You can use the `bzip2recover' program to attempt to recover
data from undamaged sections of corrupted files.

tar: Child returned status 2
tar: Error is not recoverable: exiting now
[!!] An error occurred on line 342. Saved output:
The command '/bin/sh -c su - angr -c "git clone https://github.com/angr/angr-dev && cd angr-dev && ./setup.sh -w -e angr && ./setup.sh -w -p angr-pypy"' returned a non-zero code: 1

Arch linux QT4 depency dropped

It appears as though in the setup.sh script qt4 is being pulled in as a dependency with the -i option on arch linux. Arch has dropped qt4 support officially but it is available in the AUR.
As qt4 cannot be pulled in from the official repos, the setup fails.

Looking at the source it seems as though it qt4 is only needed for angr-management.

Prefer `angr` repos to external repos?

In extremely-simple-setup.py at

angr-dev/extremely-simple-setup.sh

Line 51 in 0f43373

git clone https://github.com/angr/wheels.git

These repos contain links to things like https://github.com/salls/angrop.git; should we instead prefer https://github.com/angr/angrop.git and such (if they exist)? That one, like rex, just redirects to angr's anyway.

Development install "./setup.sh -i -p angr" is not working

Under the current version (June 23, 2021) angr-dev (cfad4e1)
angr development install is not working, the following error occurs.
I'm using Ubuntu 20.04.2 LTS.

Requirement already satisfied: virtualenvwrapper in /usr/lib/python3/dist-packages (4.8.4)
created virtual environment PyPy3.7.10.final.0-64 in 77ms
  creator PyPy3Posix(dest=/home/alan/.virtualenvs/angr.ta, clear=False, global=False)
  seeder FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, pkg_resources=latest, via=copy, app_data_dir=/home/alan/.local/share/virtualenv/seed-app-data/v1.0.1.debian.1)
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
virtualenvwrapper.user_scripts creating /home/alan/.virtualenvs/angr.ta/bin/predeactivate
virtualenvwrapper.user_scripts creating /home/alan/.virtualenvs/angr.ta/bin/postdeactivate
virtualenvwrapper.user_scripts creating /home/alan/.virtualenvs/angr.ta/bin/preactivate
virtualenvwrapper.user_scripts creating /home/alan/.virtualenvs/angr.ta/bin/postactivate
virtualenvwrapper.user_scripts creating /home/alan/.virtualenvs/angr.ta/bin/get_env_details
Traceback (most recent call last):
  File "/home/alan/.virtualenvs/angr.ta/bin/pip", line 5, in <module>
    from pip._internal.cli.main import main
  File "/home/alan/.virtualenvs/angr.ta/site-packages/pip/_internal/cli/main.py", line 10, in <module>
    from pip._internal.cli.autocompletion import autocomplete
  File "/home/alan/.virtualenvs/angr.ta/site-packages/pip/_internal/cli/autocompletion.py", line 9, in <module>
    from pip._internal.cli.main_parser import create_main_parser
  File "/home/alan/.virtualenvs/angr.ta/site-packages/pip/_internal/cli/main_parser.py", line 7, in <module>
    from pip._internal.cli import cmdoptions
  File "/home/alan/.virtualenvs/angr.ta/site-packages/pip/_internal/cli/cmdoptions.py", line 24, in <module>
    from pip._internal.exceptions import CommandError
  File "/home/alan/.virtualenvs/angr.ta/site-packages/pip/_internal/exceptions.py", line 10, in <module>
    from pip._vendor.six import iteritems
ModuleNotFoundError: No module named 'pip._vendor.six'
[!!] An error occurred on line 332. Saved output:

Trim native depedencies in setup.sh

Currently we have multiple lists of native dependencies for different OSes that people have installed angr-dev on. Some of these lists are quite long, others are quite short, and for all but Ubuntu it is not especially clear if they are maintained or verified on a regular basis. I think it would be a good idea to trim down the native dependency lists to only what is strictly required to install and make basic use of the core angr packages. For distros other than Ubuntu, it may be more maintainable to simply have a list of binaries and libraries we require and promt the user to sort out where those come from.

move some scripts into special folders

tests folder:

angr-test
appveyor_test.bat
lint.py
test.sh
travis-*.sh

admin (?) folder:

bdist.bat
bdist.sh
prsync.py
releaser.sh

Of course this is gonna require updating links in a lot of places. The root is a little cluttered right now.

ERROR: No matching distribution found for angr==9.2.5.dev0

I got an install error when I ran the install command: ./setup.sh -i -e angr

ERROR: Ignored the following versions that require a different python version: 7.8.9.26 Requires-Python <3.0
ERROR: Could not find a version that satisfies the requirement angr==9.2.5.dev0 (from angrop) (from versions: 0.8.0, 0.8.1, 4.5.9.9b0, 4.5.9.9, 4.5.9.13, 4.5.9.13.post1, 4.5.9.14, 4.5.9.22, 4.5.9.29, 4.5.10.14, 4.5.10.15, 4.5.11.23, 4.5.12.12, 4.5.12.21, 4.5.12.28, 4.6.1.4, 4.6.1.12, 4.6.1.18, 4.6.1.30, 4.6.2.25, 4.6.3.15, 4.6.3.15.post1, 4.6.3.18, 4.6.3.28, 4.6.4.28, 4.6.5.27, 4.6.5.27.post1, 4.6.6.4, 4.6.6.4.post1, 4.6.6.10, 4.6.6.15, 4.6.6.16, 4.6.6.28, 4.6.6.29, 5.6.8.22, 5.6.10.5, 5.6.10.7, 5.6.10.12, 5.6.12.3, 6.7.1.13, 6.7.1.13.post2, 6.7.1.31, 6.7.3.26, 6.7.4.12, 6.7.6.9, 6.7.7.27, 6.7.7.31, 7.7.9.8, 7.7.9.8.post1, 7.7.9.14, 7.7.9.21, 7.7.12.16, 7.8.2.21, 7.8.6.16, 7.8.6.23, 7.8.7.1, 7.8.8.1, 8.18.10.5, 8.18.10.25, 8.19.2.4, 8.19.4.5, 8.19.7.25, 8.19.10.29, 8.19.10.30, 8.20.1.7, 8.20.5.27, 8.20.6.1, 8.20.6.8, 8.20.7.6, 8.20.7.27, 9.0.4378, 9.0.4446, 9.0.4495, 9.0.4663, 9.0.4885, 9.0.4940, 9.0.5002, 9.0.5034, 9.0.5171, 9.0.5326, 9.0.5327, 9.0.5376, 9.0.5405, 9.0.5450, 9.0.5609, 9.0.5610, 9.0.5739, 9.0.5811, 9.0.5903, 9.0.6136, 9.0.6281, 9.0.6421, 9.0.6488, 9.0.6588, 9.0.6642, 9.0.6790, 9.0.6852, 9.0.6885, 9.0.7293, 9.0.7491, 9.0.7833, 9.0.7912, 9.0.8021, 9.0.8761, 9.0.9031, 9.0.9166, 9.0.9297, 9.0.9355, 9.0.9438, 9.0.9506, 9.0.9572, 9.0.9684, 9.0.9792, 9.0.9947, 9.0.10010, 9.0.10055, 9.0.10072, 9.0.10159, 9.0.10281, 9.0.10339, 9.0.10409, 9.0.10534, 9.0.10576, 9.0.10651, 9.0.10689, 9.0.10730, 9.1.10913, 9.1.11508, 9.1.11611, 9.1.11752, 9.1.12332, 9.2.1, 9.2.2, 9.2.3, 9.2.4)
ERROR: No matching distribution found for angr==9.2.5.dev0
[!!] pip failure (--no-build-isolation -e angrop). Check /tmp/setup-50845 for details, or read it here:

commit id: c4e422b

Does it seem that angrop and angr-management are dependent on an invalid version of angr?

Dockerfile is broken

docker build -t angr-dev . is a lie.

Processing triggers for libgdk-pixbuf2.0-0:amd64 (2.40.0+dfsg-3ubuntu0.2) ...
E: Could not configure 'libc6:i386'. 
E: Could not perform immediate configuration on 'libgcc-s1:i386'. Please see man 5 apt.conf under APT::Immediate-Configure for details. (2)
The command '/bin/sh -c apt-get update &&								apt-get install -y virtualenvwrapper python3-dev python3-pip build-essential libxml2-dev libxslt1-dev git libffi-dev cmake libreadline-dev libtool debootstrap debian-archive-keyring libglib2.0-dev libpixman-1-dev qtdeclarative5-dev binutils-multiarch nasm libc6:i386 libgcc1:i386 libstdc++6:i386 libtinfo5:i386 zlib1g:i386 vim libssl-dev openjdk-8-jdk' returned a non-zero code: 100

pypy_venv.sh fails on Ubuntu 1804

Hi,

I am having problems using the recent version of pypy_venv.sh script on Ubuntu 18.04.

The script fails when executing the command python3 -m pip install virtualenvwrapper (this should be https://github.com/angr/angr-dev/blob/master/pypy_venv.sh#L68)

The error is Could not find an activated virtualenv (required).

I am not sure how this script is supposed to work. Do I have to create and activate a virtualenv before starting it and the script then generates a second pypy based virtualenv?

Document native dependencies in setup.sh

Somewhat related to #134, we have many native packages that are installed by setup.sh. Some, such as python3, are fairly obvious as to why they are included, but others, such as qtdeclarative5-dev and debootstrap, are less clear.

Officially support making a development install on macOS

The primary issue is that libunicorn.dylib cannot load libpyvex.dylib on macOS. Possible solutions are:

Setting rpath to $ORIGIN. Thanks @dfraze for this suggestion (if it works).
Running install_name_tool on libunicorn.dylib as suggested in this issue.

error in ipdb setup command: Invalid environment marker: python_version >= "3.3"

I'm installing angr in development mode, hower, while after the step "./setup.sh -e driller" (I use the "-e" option to create a virtualenv)，I got some error as below:

[+] Installing some other helpful stuff (logging to /tmp/setup-6381).
Collecting ipython
/home/jshuang/.virtualenvs/driller/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
  SNIMissingWarning
/home/jshuang/.virtualenvs/driller/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
  Using cached ipython-5.5.0-py2-none-any.whl
Collecting pylint
  Using cached pylint-1.8.1-py2.py3-none-any.whl
Collecting ipdb
  Using cached ipdb-0.10.3.tar.gz
    Complete output from command python setup.py egg_info:
    error in ipdb setup command: Invalid environment marker: python_version >= "3.3"
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-nAD0Lf/ipdb/
[!!] Something failed to install. Check /tmp/setup-6381 for details, or read it here:

it seems like the python version is not correct ,but angr is bulid on pyhton2. The machine I build angr is a
64bit ubuntu(14.04 desktop).
How can I fix this?

setup script simplification

We should simplify the current setup scripts. Currently we have 4. Let's reduce this down to two: setup.ps1 for windows, setup.sh for everything else. extremely-simple-setup.sh serves a purpose of documenting how this works because setup.sh is not obvious, so steps should be taken to simplify setup.sh accordingly. Things to remove include wheels, multiple remotes, separate support for downloading pypy and virtualenvwrapper integration. When this is complete, setup.sh should be understandable after a quick skim to anyone with a passing familiarity with shell scripts, as extremely-simple-setup.sh is.

Docker not building again

This time i'm not sure why. I can build it just fine, but it's been 10 days since the latest autobuild on angr/angr.

Maybe something to do with this: https://success.docker.com/article/i-have-issues-triggering-autobuilds

Cannot install in Windows, pyvex error

C:\Users\sym\Desktop\angr-dev-master\angr-dev-master>setup.bat

pip fails in pyvex installation
...
C:\Users\sym\Desktop\angr-dev-master\angr-dev-master>pip install -e .\pyvex || goto :error
Obtaining file:///C:/Users/sym/Desktop/angr-dev-master/angr-dev-master/pyvex
Requirement already satisfied: pycparser in c:\python35\lib\site-packages (from pyvex==8.19.7.25) (2.18)
Requirement already satisfied: cffi>=1.0.3 in c:\python35\lib\site-packages (from pyvex==8.19.7.25) (1.11.2)
Requirement already satisfied: archinfo==8.19.7.25 in c:\users\sym\desktop\angr-dev-master\angr-dev-master\archinfo (from pyvex==8.19.7.25) (8.19.7.25)
Requirement already satisfied: bitstring in c:\python35\lib\site-packages (from pyvex==8.19.7.25) (3.1.6)
Requirement already satisfied: future in c:\python35\lib\site-packages (from pyvex==8.19.7.25) (0.16.0)
Installing collected packages: pyvex
Found existing installation: pyvex 8.19.7.25
Uninstalling pyvex-8.19.7.25:
Successfully uninstalled pyvex-8.19.7.25
Running setup.py develop for pyvex
ERROR: Command errored out with exit status 1:
command: 'c:\python35\python3.exe' -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\sym\Desktop\angr-dev-master\angr-dev-master\pyvex\setup.py'"'"'; file='"'"'C:\Users\sym\Desktop\angr-dev-master\angr-dev-master\pyvex\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' develop --no-deps
cwd: C:\Users\sym\Desktop\angr-dev-master\angr-dev-master\pyvex
Complete output (3 lines):
running develop
Building libVEX
error: Unable to build libVEX.
----------------------------------------
Rolling back uninstall of pyvex
Moving to c:\python35\lib\site-packages\pyvex-8.19.7.25.dist-info
from c:\python35\lib\site-packages~yvex-8.19.7.25.dist-info
Moving to c:\python35\lib\site-packages\pyvex
from c:\python35\lib\site-packages~yvex
ERROR: Command errored out with exit status 1: 'c:\python35\python3.exe' -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\sym\Desktop\angr-dev-master\angr-dev-master\pyvex\setup.py'"'"'; file='"'"'C:\Users\sym\Desktop\angr-dev-master\angr-dev-master\pyvex\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.

C:\Users\sym\Desktop\angr-dev-master\angr-dev-master>echo "Development install failed!"
"Development install failed!"

Windows version: 10, 1903
Python version: 3.5
Git: 2.23.0

Exception has occurred: ArchNotFound - Can't find architecture info for architecture image_file_machine_unknown with '' bits and unsure endness

Dear Sir / Madam,

Thank you for creating angr tool. I am using it for analyzing some binary files. However, for some "disarmed" binary, the tool throw this exception: Exception has occurred: ArchNotFound - Can't find architecture info for architecture image_file_machine_unknown with '' bits and unsure endness

I am using up-to-date angr-dev master branch.

This is the full tracelog:

File "/angr-dev/archinfo/archinfo/arch.py", line 858, in arch_from_id
    raise ArchNotFound("Can't find architecture info for architecture %s with %s bits and %s endness" % (ident, repr(bits), endness))
  File "/angr-dev/cle/cle/backends/pe/pe.py", line 40, in __init__
    self.set_arch(archinfo.arch_from_id(pefile.MACHINE_TYPE[self._pe.FILE_HEADER.Machine]))
  File "/angr-dev/cle/cle/loader.py", line 850, in _load_object_isolated
    result = backend_cls(binary, binary_stream, is_main_bin=self.main_object is None, loader=self, **options)
  File "/angr-dev/cle/cle/loader.py", line 668, in _internal_load
    obj = self._load_object_isolated(main_spec)
  File "/angr-dev/cle/cle/loader.py", line 133, in __init__
    self.initial_load_objects = self._internal_load(main_binary, *preload_libs, *force_load_libs, preloading=(main_binary, *preload_libs))
  File "/angr-dev/angr/angr/project.py", line 137, in __init__
    self.loader = cle.Loader(self.filename, concrete_target=concrete_target, **load_options)
  File "/Angr_CFG/utils/AngrHelper.py", line 18, in load_project
    p = angr.Project(f'{self.source_dir}/{project_path}', load_options={'auto_load_libs': False})

The root cause I find out is because /angr-dev/cle/cle/backends/pe/pe.py can't detect the binary file architecture.

self._pe = pefile.PE(self.binary, fast_load=True) # This line of code return self._pe.FILE_HEADER.Machine = 0 ==> image_file_machine_unknown
            self._parse_pe_non_reloc_data_directories()
            if not self.is_main_bin:
                # only cache shared libraries, the main binary will not be reused
                self._pefile_cache[self.binary] = self._pe

The binary is attached. 0a0a38fa5884a8b109bad5572b2ab2d568a6fe4621b5f86641c42f7d0d713e79.exe.zip

Could you please suggest the solution or workaround for this issue?

Thank you

cle.backends.elf.ELF AttributeError

elf = cle.backends.elf.ELF('/bin/true')

gives:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/user/workspace/angr-dev/cle/cle/backends/elf.py", line 214, in __init__
    self.__register_segments()
  File "/home/user/workspace/angr-dev/cle/cle/backends/elf.py", line 397, in __register_segments
    self._load_segment(seg_readelf)
  File "/home/user/workspace/angr-dev/cle/cle/backends/elf.py", line 411, in _load_segment
    self._load_segment_memory(seg)
  File "/home/user/workspace/angr-dev/cle/cle/backends/elf.py", line 425, in _load_segment_memory
    if ph.p_align & (self.loader.page_size - 1) != 0:
AttributeError: 'NoneType' object has no attribute 'page_size'

But elf = cle.backends.elf.ELF('/bin/true', loader=cle.Loader("/bin/true")) works fine.

(on Ubuntu 16.10 64bit)

setup script fails on Ubuntu 22.04 due to changed package name(libgcc1 -> libgcc-s1)

setup.sh fails when run on Ubuntu 22.04 because it is fails to verify that package libgcc1:i386 is installed. This package is virtual in Ubuntu 22.04 and libgcc-s1:i386 is installed instead. Replacing libgcc1 with libgcc-s1 in the setup script seems to fix the issue.

Failing `clone_repo` do not display what went wrong

During the setup.sh, in the clone_repo function, the following code is hit if a repo fails for whatever reason:

error "Failed to clone $NAME. Error was:"
cat $CLONE_LOG
rm -f $CLONE_LOG
return 1

However, the error function ending with an exit 1, the script exits before the display of the log ever happens (because the three lines after the error call are never executed).

Note that for the same reason, the /tmp/clone-... files are not cleaned up.

setup.sh does not install angr-targets

The module is used on angr/angr/engines/concrete.py and some angr tests. Is there a reason for this?

AppVeyor tests raise errors related to missing avatar2, which is a dependency of angr-targets.

Overarching thing: documentation

An angr meeting on how to get documentation to a state that makes sense would be cool.

missed blocks in functions

This piece of code iterates over functions and their blocks, and flags whenever a block has both an inner succesor (i.e. inside the function) and an outer one, which basically means the outer block was wrongly assigned to a new function:

def check_function(path):
    b = angr.Project(path, load_options={'auto_load_libs': False})
    cfg = b.analyses.CFGFast(resolve_indirect_jumps=False, normalize=True)
    for procedure_address in cfg.kb.functions:
        angr_procedure = cfg.kb.functions[procedure_address]
        for block in angr_procedure.blocks:
            node = cfg.get_any_node(block.addr)
            if not node.has_return:
                inner_successors = filter(lambda s: s.addr in [b.addr for b in angr_procedure.blocks], node.successors)
                outer_successors = filter(lambda s: s not in inner_successors, node.successors)
                if len(inner_successors) > 0 and len(outer_successors) > 0:
                    # this means that Angr missed a block
                    print inner_successors
                    print outer_successors
                    for b in angr_procedure.blocks:
                        print b
                        b.pp()
                    exit()

When i run it on a simple /bin/ls i get:

[<CFGNode 0x404a6aL[2]>]
[<CFGNode 0x404a59[9]>]
<Block for 0x404a50, 9 bytes>
0x404a50:       cmp     byte ptr [rip + 0x219bf1], 0
0x404a57:       jne     0x404a6a
<Block for 0x404a6a, 2 bytes>
0x404a6a:       ret

Which clearly shows that the block at 0x404a59 was missed.

This seems like a bug. I can try and fix it with a pull request if you'd like, but you'll need to give me a pointer to where in the code this happens.

ls.zip

Call plot_cg(cfg.kb, 'out.png') call use demangled_name for function node names?

The function callgraph generated by plot_cg() does not use demanged_names. (demanged_names appear to be available in non-stripped ELF binary files, and possibly for windows PE files w/ accompanying .pdb file.)

Looks like in angr-dev/bingraphvis/bingraphvis/base.py in VisPipeLine.process() perhaps this can be handled? (I may have this totally wrong, I just looked around code a bit.)

Nose is going out of style

nose will no longer be supported as of python 3.9. nose2, which we have started using in CI, is a little more unweildy to use, and notably does not include an equivalent to nose.tools.assert_*. The proposed fix for this is to simply remove all the nose-based asserts and replace them with vanilla asserts, though if anyone wanted to take up the project of converting all our testcases to unittest-style classes, that would be cool as well.

Installing angr-dev using ./setup.sh errors out

We are trying to install angr-dev version following the instructions in the install guide at https://docs.angr.io/INSTALL.html

We follow the steps in the section titled "Development Install". However, we are seeing the following errors:

ubuntu@ubuntu:~/code/angr-dev$ git clone https://github.com/angr/angr-dev.git
Cloning into 'angr-dev'...
remote: Counting objects: 949, done.
remote: Compressing objects: 100% (6/6), done.
remote: Total 949 (delta 1), reused 4 (delta 1), pack-reused 942
Receiving objects: 100% (949/949), 160.01 KiB | 0 bytes/s, done.
Resolving deltas: 100% (581/581), done.
Checking connectivity... done.

ubuntu@ubuntu:~/code/angr-dev$ cd angr-dev/

ubuntu@ubuntu:~/code/angr-dev/angr-dev$ mkvirtualenv angr
New python executable in /home/ubuntu/co/virtual_env/angr/bin/python
Installing setuptools, pip, wheel...done.

(angr) ubuntu@ubuntu:~/code/angr-dev/angr-dev$ ls
admin Dockerfile git_all.bat git_all.sh LICENSE pylintrc pypy_venv.sh README.md setup.bat setup.sh standard_logging.py tests

(angr) ubuntu@ubuntu:~/code/angr-dev/angr-dev$ ./setup.sh
[+] 13:51:56 Checking dependencies...
dpkg: no packages found matching virtualenvwrapper
dpkg: no packages found matching python-pip
dpkg: no packages found matching libxml2-dev
dpkg: no packages found matching libxslt1-dev
dpkg: no packages found matching libreadline-dev
dpkg: no packages found matching debootstrap
dpkg: no packages found matching debian-archive-keyring
dpkg: no packages found matching libglib2.0-dev
dpkg: no packages found matching libpixman-1-dev
dpkg: no packages found matching graphviz-dev
dpkg: no packages found matching nasm
dpkg: no packages found matching libc6:i386
dpkg: no packages found matching libgcc1:i386
dpkg: no packages found matching libstdc++6:i386
dpkg: no packages found matching libtinfo5:i386
dpkg: no packages found matching zlib1g:i386
[!!] 13:51:56 Please install the following packages: virtualenvwrapper python-pip python2.7-dev build-essential libxml2-dev libxslt1-dev git libffi-dev cmake libreadline-dev libtool debootstrap debian-archive-keyring libglib2.0-dev libpixman-1-dev libqt4-dev graphviz-dev binutils-multiarch nasm libc6:i386 libgcc1:i386 libstdc++6:i386 libtinfo5:i386 zlib1g:i386
[+] Checking dependencies...
[!!] Please install the following packages: virtualenvwrapper python-pip python2.7-dev build-essential libxml2-dev libxslt1-dev git libffi-dev cmake libreadline-dev libtool debootstrap debian-archive-keyring libglib2.0-dev libpixman-1-dev libqt4-dev graphviz-dev binutils-multiarch nasm libc6:i386 libgcc1:i386 libstdc++6:i386 libtinfo5:i386 zlib1g:i386

(angr) ubuntu@ubuntu:~/code/angr-dev/angr-dev$ pip install virtualenvwrapper python-pip python2.7-dev build-essential libxml2-dev libxslt1-dev git libffi-dev cmake libreadline-dev libtool debootstrap debian-archive-keyring libglib2.0-dev libpixman-1-dev libqt4-dev graphviz-dev binutils-multiarch nasm libc6:i386 libgcc1:i386 libstdc++6:i386 libtinfo5:i386 zlib1g:i386
Invalid requirement: 'libc6:i386'
Traceback (most recent call last):
File "/home/ubuntu/co/virtual_env/angr/local/lib/python2.7/site-packages/pip/req/req_install.py", line 82, in init
req = Requirement(req)
File "/home/ubuntu/co/virtual_env/angr/local/lib/python2.7/site-packages/pip/_vendor/packaging/requirements.py", line 96, in init
requirement_string[e.loc:e.loc + 8]))
InvalidRequirement: Invalid requirement, parse error at "':i386'"

(angr) ubuntu@ubuntu:~/code/angr-dev/angr-dev$ sudo pip install virtualenvwrapper python-pip python2.7-dev build-essential libxml2-dev libxslt1-dev git libffi-dev cmake libreadline-dev libtool debootstrap debian-archive-keyring libglib2.0-dev libpixman-1-dev libqt4-dev graphviz-dev binutils-multiarch nasm libc6:i386 libgcc1:i386 libstdc++6:i386 libtinfo5:i386 zlib1g:i386
The directory '/home/ubuntu/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/ubuntu/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Invalid requirement: 'libc6:i386'
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/pip/req/req_install.py", line 82, in init
req = Requirement(req)
File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/packaging/requirements.py", line 96, in init
requirement_string[e.loc:e.loc + 8]))
InvalidRequirement: Invalid requirement, parse error at "':i386'"

setup.sh installs incompatible PyPy binary for Debian stable

Problem:

On stable Debian (currently Stretch), when setup.py is told to use PyPy (-p or -P flag), pypy_venv.sh tries to fetch the latest official PyPy release, which is currently pypy3-v6.0.0-linux64. Unfortunately, the PyPy binary in this release fails to link against libssl.so.1.0.0, meaning pip cannot use SSL/TLS, ultimately causing the setup to fail.

Possible Solutions:

Fetch portable PyPy instead.

The official PyPy install documentation mentions an unofficial portable build of PyPy. Since this version provides its own libssl library, it avoids the above issue. I've tested this solution and it works.

Wait for Debian to offer a PyPy3 package.

Debian has a pypy package, but it's currently PyPy 5.6 and implements Python 2.x, not 3.x. The upcoming stable (Buster) will package PyPy 6.0, but it's unclear if this binary is going to support 2.x and 3.x, or just 2.x. There's currently an open ticket for this issue.

Symlink libssl to hotfix the issue.

Debian Stretch has libssl packages that provide libssl.so.1.0.2 and libssl.so.1.1. Symlinking libssl.so.1.0.0 to one of these will probably fix the issue, but this is an ad-hoc fix and there may be other linking errors beyond libssl.

angr.analyses.vfg results in an uninitialized access (used as addr)

related python test code :

    # __main__

   binary = angr.Project(target_binary_path, 
                         load_options={"auto_load_libs": False})
    analyze(binary)

   #  analyze (binary)

    cfg = binary.analyses.CFG(context_sensitivity_level=3, keep_state=True)    
    vfg = binary.analyses.VFG(cfg, interfunction_level=150)
    vfg_ddg = binary.analyses.VSA_DDG(vfg, context_sensitivity_level=3, keep_data=True)

Here is the stack trace (target binary is attached):

ERROR   | 2016-02-26 00:13:24,853 | angr.analyses.vfg | Found an uninitialized access (used as addr) at expression <BV64 Reverse(global_7ffffffffff0010_51_64)>.
ERROR   | 2016-02-26 00:13:24,854 | angr.analyses.vfg | Variable global_7ffffffffff0010_51_64 does not exist in self._uninitialized_access. Please report it to Fish.
Traceback (most recent call last):
  File "analyze.py", line 88, in <module>
    analyze(binary)
  File "analyze.py", line 63, in analyze
    vfg = binary.analyses.VFG(cfg, interfunction_level=150)
  File "/home/osboxes/angr-dev/angr/angr/analysis.py", line 84, in make_analysis
    oself.__init__(*args, **kwargs)
  File "/home/osboxes/angr-dev/angr/angr/analyses/vfg.py", line 130, in __init__
    self._construct(initial_state=initial_state)
  File "/home/osboxes/angr-dev/angr/angr/analyses/vfg.py", line 254, in _construct
    self._ai_analyze(initial_state)
  File "/home/osboxes/angr-dev/angr/angr/analyses/vfg.py", line 320, in _ai_analyze
    tracing_times, retn_target_sources
  File "/home/osboxes/angr-dev/angr/angr/analyses/vfg.py", line 436, in _handle_entry
    simrun, error_occured, restart_analysis = self._get_simrun(input_state, current_path, addr)
  File "/home/osboxes/angr-dev/angr/angr/analyses/vfg.py", line 982, in _get_simrun
    sim_run = self.project.factory.sim_run(current_path.state, jumpkind=jumpkind)
  File "/home/osboxes/angr-dev/angr/angr/factory.py", line 114, in sim_run
    r = sim_proc_class(state, addr=addr, sim_kwargs=kwargs)
  File "/home/osboxes/angr-dev/simuvex/simuvex/s_procedure.py", line 62, in __init__
    r = run_func(*args, **self.kwargs)
  File "/home/osboxes/angr-dev/simuvex/simuvex/procedures/libc___so___6/fopen.py", line 32, in run
    p_strlen = strlen(self.state, inline=True, arguments=[p_addr])
  File "/home/osboxes/angr-dev/simuvex/simuvex/s_procedure.py", line 62, in __init__
    r = run_func(*args, **self.kwargs)
  File "/home/osboxes/angr-dev/simuvex/simuvex/procedures/libc___so___6/strlen.py", line 32, in run
    self.max_null_index = max(self.max_null_index + i)
TypeError: unsupported operand type(s) for +: 'int' and 'list'

wordcount.tar.gz

Add OS dependencies check to each setup.py

As per giovanni, it would be nice if each package explicitly checked for the libraries and binaries it expects to find, and if it doesn't, make an attempt to show the user a friendly error message with the OS-appropriate install command for them.

cvc4-solver dependency

angr-dev/setup.sh

Line 495 in 9f95e8d

 if pip3 install ipython pylint ipdb nose nose-timer coverage flaky sphinx sphinx_rtd_theme recommonmark cvc4-solver 'requests[security]' >> $OUTFILE 2>> $ERRFILE 

Should cvc4-solver still be in here? It was removed here angr/claripy@3a6c5ae.

angr/angr docker is broken: "workon command not found"

Workaround: source /usr/share/virtualenvwrapper/virtualenvwrapper.sh && workon angr

I want to be able to `pip install -e .` angr/archr/angr-management/etc

I want to be able to pip install -e . angr/archr/angr-management/etc.

This does not work because version 9.0.gitrolling is a lie. It does not actually exist.
I think the gitrolling concept makes sense to me, but should be implemented differently so that setup.py still works.
This could be done for instance, via a __gitrolling__ = ["angr"] assignment that gets parsed out during CI release.

Here is a rough prototype of what could be used to accomplish this:

import ast


def patch_gitrolling(version):
    with open("setup.py") as f:
        setup = ast.parse(f.read())

    for node in setup.body:
        if not isinstance(node, ast.Assign):
            continue
        if not any(target.id == "__gitrolling__" for target in node.targets):
            continue
        if not hasattr(node.value, "elts"):
            continue
        gitrolling = [e.value for e in node.value.elts]
        break
    else:
        gitrolling = []

    for node in setup.body:
        if not isinstance(node, ast.Expr):
            continue
        if not isinstance(node.value, ast.Call):
            continue
        if not isinstance(node.value.func, ast.Name):
            continue
        if not node.value.func.id == "setup":
            continue
        for keyword in node.value.keywords:
            if keyword.arg != "install_requires":
                continue
            if not hasattr(keyword.value, "elts"):
                continue
            for e in keyword.value.elts:
                if e.value in gitrolling:
                    e.value = f"{e.value}=={version}"

    return ast.unparse(setup)


print(patch_gitrolling(version="9.0.gitrolling"))

Installation Error

While compiling pyvex, the following error occurs. Feel free if you need more debug information.

    x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DPYVEX_STATIC=1 -I./vex_include -I/usr/include/python3.5m -I/home/jmh/.virtualenvs/tz/include/python3.5m -c pyvex/pyvex_irsb.c -o build/temp.linux-x86_64-3.5/pyvex/pyvex_irsb.o --std=c99
    In file included from pyvex/pyvex_enums.h:7:0,
                     from pyvex/pyvex_irsb.c:7:
    pyvex/pyvex_irsb.c: In function ‘pyIRSB_dealloc’:
    pyvex/pymare.h:44:83: error: ‘pyIRSB {aka struct <anonymous>}’ has no member named ‘ob_type’
     #define PYMARE_DEALLOC(type) static void py##type##_dealloc(py##type* self) { self->ob_type->tp_free((PyObject*)self); }

Overarching thing: engine specialization for static analysis

The next angr meeting should be about the movement of parts of the static analysis into engines, similar to the "atomic" SimEngine.process for symbolic execution.

Dependency-Problem through angrop

Hello,

when installing angr for development, with ./setup.sh -i -e repo, angrop is automatically cloned and installed. The setup.py file froom angrop has as its dependency angr itself, which gets installed via pip install angr - getting the lattest pip version from angr overriding any changes made to the cloned angr repository itself.

Would be good to get angrop as "optional" or angr be listed as setup_requires instead of install_requires where its getting installed automatically.

Greetings,
Andy

Travis Testing with Dependent PRs

I think it would be nice to have a way to test multiple PRs that are dependent on each other. For instance, I submitted 1 2 3 PRs for a new feature's logic, testing, and binary for testing, respectively. However, they won't pass Travis testing until they are either merged or a configuration is changed to checkout the respective branch.

I did name the branches the same in each component, so it would be nice if there was way for Travis to automatically pull or checkout the dependent branches in each of the components for testing based on the branch name.

Obviously if two branches are named the same (think master) in different PRs that aren't related, then some errors may result. Contributors would also have to name their branches the same when submitting PRs that depend on each other. Another potential issue would be the age of the PRs and how up to date they are with the current revisions of each component, but this might be easily solved by updating the PR with current component's master. The order in which a PR is submitted might also mess things up, but that job could be manually restarted.

I tried looking for an existing solution from other projects, but I'm not very familiar with Travis or its configuration. I found this article that might be helpful in determining the name of the branch in the PR which could then be used to checkout the respective dependant’s branch : https://graysonkoonce.com/getting-the-current-branch-name-during-a-pull-request-in-travis-ci/

Maybe I don't quite understand how Travis works or this is too complicated or there are other issues that I haven't thought about. I'd be interested in your thoughts.

Thank you!

error after angr import

I get following syntax error, when I try to import angr. I am using wsl and latest angr pull from repo.

  File "/mnt/d/Ruturaj/binary_research/angr/angr-dev/cle/cle/backends/tls/elf_tls.py", line 55
    self.tcb_offset: int = None
                   ^
SyntaxError: invalid syntax

Daily Status CI

Once a day, we should have a script that installs everything, runs all the examples, including the really really long ones, and runs @salls' speed tests and logs the results.

Potentially, if everything passes, the script could also make a release, but that would be a lot of releases. Do we care about constantly pushing these huge new releases? Is that detrimental to the health of the pypi platform?

We also need to figure out where this should be running. I think a cron job on an internal server would be sufficient tbh, but we could look at how to do that with travis/appveyor.

Typo in setup.sh when setting venvwrapper_loc

venvwrapper_loc is misspelled as venvwrapepr_loc on line 268 of setup.sh.

Overarching thing: environment model abstraction

We probably want to have an angr meeting about how to approach OS/environment abstractions (as also discussed here: angr/simuvex#71).

No tracer repo inside setup.sh

Maybe not an issue, but one of the tests inside angr doesnt work without a tracer

======================================================================
ERROR: Test identification of common comparison functions
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/amatter/.virtualenvs/angr.io/local/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/amatter/angr-dev/angr/tests/test_identifier.py", line 18, in test_comparison_identification
    idfer = p.analyses.Identifier(require_predecessors=False)
  File "/home/amatter/angr-dev/angr/angr/analysis.py", line 99, in make_analysis
    oself.__init__(*args, **kwargs)
  File "/home/amatter/angr-dev/angr/angr/analyses/identifier/identify.py", line 53, in __init__
    self._runner = Runner(self.project, self._cfg)
  File "/home/amatter/angr-dev/angr/angr/analyses/identifier/runner.py", line 28, in __init__
    from tracer.simprocedures import FixedOutTransmit, FixedInReceive
ImportError: No module named tracer.simprocedures

Docker

Could you please help me out with the error below? I tried to build a Docker container. Could someone repeat the steps noted in README.md?

./pypy_venv.sh: line 30: curl: command not found
--2018-03-23 14:28:28--  https://bitbucket.org/pypy/pypy/downloads/
Resolving bitbucket.org (bitbucket.org)... 104.192.143.1, 104.192.143.3, 104.192.143.2, ...
Connecting to bitbucket.org (bitbucket.org)|104.192.143.1|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 343753 (336K) [text/html]
Saving to: 'STDOUT'

     0K .......... .....bzip2: (stdin) is not a bzip2 file.
..... .......... .tar: Child died with signal 13
tar: Error is not recoverable: exiting now
                      9%  232K=0.1s


Cannot write to '-' (No such file or directory).
[!!] An error occurred on line 232. Saved output:
The command '/bin/sh -c su - angr -c "git clone https://github.com/angr/angr-dev && cd angr-dev && ./setup.sh -w -e angr && ./setup.sh -w -p angr-pypy"' returned a non-zero code: 1

Fails to setup pyvex on NixOS

After I nix-shell, I use ./setup.py -p venv, which triggers the following error after some time:

[...]
ERROR: Command "/home/pamplemousse/.virtualenvs/venv/bin/pypy3 -c 'import setuptools, tokenize;__file__='"'"'/home/pamplemousse/Workspace/angr/pyvex/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps" failed with error code 1 in /home/pamplemousse/Workspace/angr/pyvex/
Exception information:
Traceback (most recent call last):
  File "/home/pamplemousse/.virtualenvs/venv/site-packages/pip/_internal/cli/base_command.py", line 178, in main
    status = self.run(options, args)
  File "/home/pamplemousse/.virtualenvs/venv/site-packages/pip/_internal/commands/install.py", line 414, in run
    use_user_site=options.use_user_site,
  File "/home/pamplemousse/.virtualenvs/venv/site-packages/pip/_internal/req/__init__.py", line 58, in install_given_reqs
    **kwargs
  File "/home/pamplemousse/.virtualenvs/venv/site-packages/pip/_internal/req/req_install.py", line 910, in install
    install_options, global_options, prefix=prefix,
  File "/home/pamplemousse/.virtualenvs/venv/site-packages/pip/_internal/req/req_install.py", line 766, in install_editable
    cwd=self.setup_py_dir,
  File "/home/pamplemousse/.virtualenvs/venv/site-packages/pip/_internal/utils/misc.py", line 776, in call_subprocess
    % (command_desc, proc.returncode, cwd))
pip._internal.exceptions.InstallationError: Command "/home/pamplemousse/.virtualenvs/venv/bin/pypy3 -c 'import setuptools, tokenize;__file__='"'"'/home/pamplemousse/Workspace/angr/pyvex/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps" failed with error code 1 in /home/pamplemousse/Workspace/angr/pyvex/

Here are the version of some components (inside the nix-shell):

$ nixos-version
19.03.173068.9ec7625cee5 (Koi)
$ pip --version
pip 19.1.1 from /home/pamplemousse/.virtualenvs/venv/site-packages/pip
$ python3 --version
Python 3.5.3 (fdd60ed87e941677e8ea11acf9f1819466521bf2, Jun 23 2019, 18:51:05)
[PyPy 6.0.0 with GCC 7.4.0]
$ gcc --version
gcc (GCC) 7.4.0