Giter VIP home page Giter VIP logo

s2e-env's Introduction

s2e-env

A command-line tool for creating and administering isolated development environments for S2E. Each environment contains all the tools required to run S2E plus one or more "projects". A project is essentially an analysis target. For example, one project might be the analysis of a CGC binary, while another project might be the analysis of the file program from Coreutils.

Prerequisites

We assume that you are working on an Ubuntu 18.04 LTS 64-bit OS. You will need the following packages:

$ sudo apt-get install git gcc python3 python3-dev python3-venv

Some commands (such as basic block coverage) require a disassembler. Supported disassemblers include:

Install

We recommend installing s2e-env into a virtual environment.

git clone https://github.com/S2E/s2e-env.git
cd s2e-env
python3 -m venv venv
. venv/bin/activate
pip install .

# Note: if you use pip earlier than version 19, you must add --process-dependency-links:
pip install . --process-dependency-links

Tests

Run tests with the following command:

$ ./test.sh

This will create a fresh virtual environment venv-test, install all requirements, run pylint, the tests, and record coverage.

Configuring

s2e-env is configurable in two ways. Firstly, there is a global YAML configuration file located in s2e_env/dat/config.yaml. This configuration file controls how all environments are created. You are not normally required to modify the settings in this file. If you wish to customize how environments are created, you should edit this file before running pip install to install s2e-env.

For example, you may want to clone the S2E source repos via SSH rather than HTTPS, in which case you would set the repos, url option to [email protected]:S2E.

A second YAML configuration file, s2e.yaml, is created in each S2E environment. This contains settings that are local to each S2E environment. For example, if you want to generate basic block coverage, you will also have to set the ida, path option.

Usage

The package can be installed via pip, thus making the s2e command available.

To list the available commands:

s2e help --commands

To get help on a particular command:

s2e <subcommand> --help

Most commands use the S2EDIR environment variable so that commands can be run from any directory. S2EDIR can be set by sourcing s2e_activate in your environment directory. Sourcing this file also makes s2e_deactivate available, which unsets the S2E environment variables.

Alternatively, most commands take an optional --env /path/to/env argument. This argument can be used to specify the path to the S2E environment you want to execute the command in.

Note that one of the S2EDIR environment variable or --env option must be used.

Workflow

Each command follows the Unix philosophy that each command ("tool") consists of a small program designed to accomplish a single, particular task, rather than trying to develop monolithic commands to do a number of tasks.

A typical workflow is therefore:

  1. Run s2e init $DIR to create a new S2E environment in $DIR. This will create the environment, install dependencies (unless --skip-dependencies is used) and fetch all of the S2E engine code.
  2. Activate the environment via . $DIR/s2e_activate.
  3. Look around the source code, make some modifications, etc. Then when you are ready to build run s2e build.
  4. You'll need some images to analyze your software in! See what images are available with s2e image_build.
  5. Run s2e image_build $TEMPLATE to build one of the images listed in the previous step. This will create the image in the images directory.
  6. Use s2e new_project to create a new analysis project. This will create all the launch scripts, configuration files and bootstrap scripts necessary to perform the analysis on a given target. Currently Linux ELF executables, Decree CGC binaries, Windows PE executables and Windows DLLs can be targeted with the new_project command.
  7. Change into the project directory and run the S2E analysis with the launch-s2e.sh script.
  8. After your analysis has finished, a number of subcommands exist to analyze and summarize your results, e.g. the coverage and execution_trace subcommands.

Other useful commands:

  • s2e info can be used to display a summary of the S2E environment.
  • To download the latest changes from the git repositories, run s2e update.
  • Projects can be shared using s2e export_project and s2e import_project.

Environment structure

s2e init generates the following directory structure in your S2E environment.

.
├── build/
├── images/
├── install/
├── projects/
├── s2e.yaml
├── source/
  • build: Staging directory for builds
  • images: Images created with s2e image_build go here
  • install: Installed executables, libraries, header files, etc.
  • projects: Analysis projects created with s2e new_project go here
  • s2e.yaml: A per-environment configuration file. This file is also used to "mark" the directory as an S2E environment, so please do not delete it!
  • source: Source code repositories

Extending

Extending with new commands is relatively simple. s2e-env is heavily influenced by Django's command subsystem, so there is a wealth of documentation already available (for example, here).

For example, to create a command foo:

  1. Create a new Python module s2e_env/commands/foo.py

  2. In foo.py define a Command class that extends

    • s2e_env.command.BaseCommand - The base class. Probably not that useful to inherit directly from this class
    • s2e_env.command.EnvCommand - For commands that operate on an existing S2E environment
    • s2e_env.command.ProjectCommand - For commands that operate on an existing analysis project
  3. The only method required in your Command class is handle(self, *args, **options). This method contains your command logic

  4. You may optionally define an add_arguments(self, parser) method for parsing command-line arguments specific to the foo command. The parser argument is essentially an ArgumentParser from the argparse library.

    If you extend from EnvCommand you must call the super add_arguments, i.e.:

    def add_arguments(self, parser):
        super(Command, self).add_arguments(parser)
        # Add your arguments/options here
  5. On error, an s2e_env.command.CommandError should be raised

  6. Use the logging module for printing messages. When calling logging.getLogger the command name should be provided as the logger name.

Running commands from your code

Like Django's command subsystem (see here), s2e-env also allows you to call commands programatically via the call_command function.

Example:

from s2e_env.commands.new_project import Command as NewProjectCommand
from s2e_env.manage import call_command


def create_s2e_project(target_path, s2e_env_path):
    call_command(NewProjectCommand(), target_path, env=s2e_env_path, force=True)

Custom projects

Occasionally the default analysis projects (e.g., Windows driver, Linux application, etc.) may not meet your requirements. In these cases, a custom project may be created by extending the s2e_env.commands.project_creation.abstract_project.AbstractProject class. This child class must implement the following methods:

  • _configure: Generates a configuration dictionary that describes the project. The contents of this dictionary are up to the user; and
  • _create: Creates the actual project on disk. This should including, making the project directory, and creating the files necessary to run the project in this project directory. The project creation is guided by the configuration dictionary generated in _make_config. The path to the project should be returned from this method.

Optionally, the child class may also implement:

  • _get_instructions: Return a string that is displayed to the user upon successful creation of a project; and
  • _is_valid_image: If an image is not specified, this method is used as a predicate when automatically selecting an image.

Currently, custom projects can only be used programmatically as follows:

import os

from s2e_env.commands.new_project import Command as NewProjectCommand
from s2e_env.commands.project_creation import AbstractProject
from s2e_env.manage import call_command


class MyProject(AbstractProject):
    def _configure(self, target, *args, **kwargs):
        return dict(project_dir='/path/to/my/project')

    def _create(self, config, force=False):
        os.mkdir(config['project_dir'])

        return config['project_dir']

    def _get_instructions(self, config):
        return 'Your project has been successfully created in %s' % config['project_dir']


call_command(NewProjectCommand(), env='/path/to/s2e', project_class=MyProject)

s2e-env's People

Contributors

adrianherrera avatar ameily avatar bannsec avatar bmorgan1296 avatar dependabot[bot] avatar gkso avatar humeafo avatar insuyun avatar loverics avatar lzto avatar michaelbrownuc avatar peng-hui avatar u1f383 avatar vitaly-cyberhaven avatar vitalych avatar vpaulv avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

s2e-env's Issues

"custom memory allocation vtable not supported"

So, I've managed to get this at least somewhat installed. Performed the following steps:

  1. Setup a new Ubuntu:16.04 docker image
  2. s2e init command
  3. s2e build command
  4. s2e image_build -d command
  5. s2e new_project --image debian-8.7.1-x86_64 /bin/ls

However, when I perform the following command:

  1. s2e run ls

The TUI pops open for a second, then goes away. I've found some error logs in the project directory:

$ cat stderr.txt 

(process:837): GLib-WARNING **: /build/glib2.0-prJhLS/glib2.0-2.48.2/./glib/gmem.c:483: custom memory allocation vtable not supported
qemu-system-x86_64: -n: invalid option
$ cat warnings.txt
WARNING: Cannot get configuration value 'pluginsConfig['TestCaseGenerator'].logLevel':
    value of type nil can not be converted to string
WARNING: Cannot get configuration value 'pluginsConfig['ProcessExecutionDetector'].logLevel':
    value of type nil can not be converted to string

WARNING: Cannot get configuration value 'pluginsConfig['ModuleExecutionDetector'].logLevel':
    value of type nil can not be converted to string

WARNING: Cannot get configuration value 'pluginsConfig['ModuleExecutionDetector'].trackAllModules':
    value of type nil can not be converted to boolean

WARNING: Cannot get configuration value 'pluginsConfig['ModuleExecutionDetector'].configureAllModules':
    value of type nil can not be converted to boolean

WARNING: Cannot get configuration value 'pluginsConfig['ModuleExecutionDetector'].trackExecution':
    value of type nil can not be converted to boolean

WARNING: Cannot get configuration value 'pluginsConfig['StaticFunctionModels'].logLevel':
    value of type nil can not be converted to string

WARNING: Cannot get configuration value 'pluginsConfig['StaticFunctionModels'].count':
    value of type nil can not be converted to integer

WARNING: Cannot get configuration value 'pluginsConfig['CUPASearcher'].batchTime':
    value of type nil can not be converted to integer

WARNING: Cannot get configuration value 'pluginsConfig['CUPASearcher'].enabled':
    value of type nil can not be converted to boolean

WARNING: Cannot get configuration value 'pluginsConfig['TranslationBlockCoverage'].logLevel':
    value of type nil can not be converted to string

WARNING: Cannot get configuration value 'pluginsConfig['ModuleTracer'].logLevel':
    [string "return pluginsConfig['ModuleTracer'].logLevel..."]:1: attempt to index field 'ModuleTracer' (a nil value)

WARNING: Cannot get configuration value 'pluginsConfig['CorePlugin'].logLevel':
    [string "return pluginsConfig['CorePlugin'].logLevel"]:1: attempt to index field 'CorePlugin' (a nil value)

3 [State 0] LinuxMonitor: Module load not yet implemented
3 [State 0] LinuxMonitor: Module load not yet implemented
3 [State 0] LinuxMonitor: Module load not yet implemented
3 [State 0] LinuxMonitor: Module load not yet implemented
3 [State 0] LinuxMonitor: Module load not yet implemented
3 [State 0] LinuxMonitor: Module load not yet implemented
3 [State 0] LinuxMonitor: Module load not yet implemented
3 [State 0] LinuxMonitor: Module load not yet implemented
3 [State 0] LinuxMonitor: Module load not yet implemented
3 [State 0] LinuxMonitor: Module load not yet implemented
3 [State 0] BaseInstructions: Message from guest (0x7f14aa34ad36): S2E_SYM_ARGS is not set. All arguments will be concrete
All states were terminated
Terminating node 0 (instance slot 0)

Also this:

BEGIN searcher description
DFSSearcher
END searcher description
1 [State 0] Created initial state
Adding CPU (addr = 0x7fa7a6c8c010, size = 0x36e70)
Initializing periodic timer
Adding memory block (startAddr = 0xffffffffffffffff, size = 0x10000000, hostAddr = 0x7fa773e00000, isSharedConcrete=0, name=pc.ram)
Adding memory block (startAddr = 0xffffffffffffffff, size = 0x20000, hostAddr = 0x55bf72f39000, isSharedConcrete=1, name=pc.bios)
Adding memory block (startAddr = 0xffffffffffffffff, size = 0x20000, hostAddr = 0x55bf72f8a000, isSharedConcrete=1, name=pc.rom)
Adding memory block (startAddr = 0xffffffffffffffff, size = 0x800000, hostAddr = 0x7fa764200000, isSharedConcrete=1, name=vga.vram)
Adding memory block (startAddr = 0xffffffffffffffff, size = 0x10000, hostAddr = 0x55bf73a27000, isSharedConcrete=1, name=cirrus_vga.rom)
Adding memory block (startAddr = 0xffffffffffffffff, size = 0x20000, hostAddr = 0x55bf73a95000, isSharedConcrete=1, name=e1000.rom)
Warning: vlan 0 is not connected to host network
Could not initialize SDL(No available video device) - exiting
Terminating node 0 (instance slot 0)

Can't download PDB with symchk

$ ../scripts/symchk.py  en_windows_xp_professional_with_service_pack_3_x86_cd_x14-80428_40f8880122a030a7e9e1fedea833b33d_ntkrnlmp.exe
Unpacking to en_windows_xp_professional_with_service_pack_3_x86_cd_x14-80428_40f8880122a030a7e9e1fedea833b33d_ntkrnlmp.pdb
Traceback (most recent call last):
  File "../scripts/symchk.py", line 208, in <module>
    main()
  File "../scripts/symchk.py", line 204, in main
    handle_pe(args.exe, not args.verbose)
  File "../scripts/symchk.py", line 173, in handle_pe
    unpack_file(saved_file, new_file)
  File "../scripts/symchk.py", line 182, in unpack_file
    Archive(source).extractall(dirname)
  File "/c/Users/Vitaly/Cyberhaven/guest-tools/windows/venv/lib/python2.7/site-packages/pyunpack/__init__.py", line 90, in extractall
    self.extractall_patool(directory, patool_path)
  File "/c/Users/Vitaly/Cyberhaven/guest-tools/windows/venv/lib/python2.7/site-packages/pyunpack/__init__.py", line 62, in extractall_patool
    raise PatoolError('patool can not unpack\n' + str(p.stderr))
pyunpack.PatoolError: patool can not unpack
patool error: error extracting /c/Users/Vitaly/Cyberhaven/guest-tools/windows/kernels/ntkrnlmp.pd_: unknown archive format for file `/c/Users/Vitaly/Cyberhaven/guest-tools/windows/kernels/ntkrnlmp.pd_'

Debian 8.7.1 i386 image build failed (jigdo jigdo jigdo)

Recent I'm trying to rebuild S2E from scratch using s2e-env, while I was building the Debian 8.7.1 guest image, I encountered the following error:

INFO: [image_build] The following images will be built:
INFO: [image_build] * debian-8.7.1-i386
INFO: [image_build] Kernel repository already exists in /home/alan/s2e-work/s2e/source/s2e-linux-kernel
WARNING: [image_build] Image creation will run in headless mode. Use --gui to see graphic output for debugging.
make: Entering directory `/home/alan/s2e-work/s2e/images'
[Fri Dec 15 14:52:28 PST 2017] [/home/alan/s2e-work/s2e/images/.tmp-output/debian-8.7.1-i386/debian-8.7.1-i386.iso] Downloading disk image...
wget --no-use-server-timestamps -O /home/alan/s2e-work/s2e/images/.tmp-output/debian-8.7.1-i386/debian-8.7.1-i386.iso https://cdimage.debian.org/mirror/cdimage/archive/8.7.1/i386/iso-cd/debian-8.7.1-i386-netinst.iso
--2017-12-15 14:52:28-- https://cdimage.debian.org/mirror/cdimage/archive/8.7.1/i386/iso-cd/debian-8.7.1-i386-netinst.iso
Resolving cdimage.debian.org (cdimage.debian.org)... 194.71.11.173, 194.71.11.165, 2001:6b0:19::173, ...
Connecting to cdimage.debian.org (cdimage.debian.org)|194.71.11.173|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2017-12-15 14:52:29 ERROR 404: Not Found.

make: *** [/home/alan/s2e-work/s2e/images/.tmp-output/debian-8.7.1-i386/debian-8.7.1-i386.iso] Error 8
make: Leaving directory `/home/alan/s2e-work/s2e/images'
ERROR: [image_build]

RAN: /usr/bin/make --directory=/home/alan/s2e-work/s2e/images --file=/home/alan/s2e-work/s2e/source/guest-images/Makefile -j 2 debian-8.7.1-i386

STDOUT:

STDERR:

After some investigation, I found Debian has removed the archived ISO images from its website, and using jigdo files instead. Currently I have to manually download the image. But it would be perfect if s2e-env can automatically do it.

s2e build error

Install the project...
-- Install configuration: ""
-- Up-to-date: /home/s2e/s2e/install/bin/guest-tools64/include/s2e/s2e.h
-- Up-to-date: /home/s2e/s2e/install/bin/guest-tools64/include/s2e/opcodes.h
-- Installing: /home/s2e/s2e/install/bin/guest-tools64/./s2ecmd.exe
-- Installing: /home/s2e/s2e/install/bin/guest-tools64/./s2eget.exe
-- Installing: /home/s2e/s2e/install/bin/guest-tools64/./s2eput.exe
make[1]: Leaving directory '/home/s2e/s2e/build/s2e/guest-tools64-win'
rm clang+llvm-3.9.0-x86_64-linux-gnu-ubuntu-16.04make: unlink: clang+llvm-3.9.0-x86_64-linux-gnu-ubuntu-16.04: Is a directory

make: Leaving directory '/home/s2e/s2e/build/s2e'
SUCCESS: [build] S2E built

Dump incremental coverage data

For now coverage is written at state termination, which may be suboptimal if S2E is killed before the paths terminate.

Debian image build failing

I've applied to join the google group, but while i wait i didn't want to forget about this one.

Attempting to build a Debian image using the s2e program, I'm getting errors about dependencies not being there anymore. Sounds like the version of Debian that is being built might be old?

(s2e) s2e@a005e5eef4ca:~/s2e$ s2e image_build debian-8.7.1-x86_64
INFO: [image_build] The following images will be built:
INFO: [image_build]  * debian-8.7.1-x86_64
INFO: [image_build] Kernel repository already exists in /home/s2e/s2e/source/s2e-linux-kernel
WARNING: [image_build] Image creation will run in headless mode. Use --gui to see graphic output for debugging.
make: Entering directory '/home/s2e/s2e/images'
mkdir -p /home/s2e/s2e/images/.tmp-output
[Sun Jul  9 00:16:04 UTC 2017] [/home/s2e/s2e/images/.stamps/linux-build-x86_64] Building docker image...
mkdir -p /home/s2e/s2e/images/.stamps
cd /home/s2e/s2e/source/guest-images//Linux/docker && docker build -t linux-build-x86_64 -f Dockerfile.x86_64 .
Sending build context to Docker daemon 11.78 kB
Step 1 : FROM debian
latest: Pulling from library/debian
c75480ad9aaf: Pull complete
Digest: sha256:7d067f77d2ae5a23fe6920f8fbc2936c4b0d417e9d01b26372561860750815f0
Status: Downloaded newer image for debian:latest
 ---> a2ff708b7413
Step 2 : MAINTAINER Vitaly Chipounov <[email protected]>
 ---> Running in 0945ee1202e3
 ---> 8ec762525745
Removing intermediate container 0945ee1202e3
Step 3 : RUN apt-get update &&     DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends     sudo apt-file texinfo flex bison patch python unzip git bc     bzip2 wget less nano g++ gcc file libc6-dev  make     fakeroot build-essential devscripts kernel-package libncurses5-dev &&     apt-get clean &&     apt-file update
 ---> Running in 822b008455f8
Ign:1 http://deb.debian.org/debian stretch InRelease
Get:2 http://security.debian.org stretch/updates InRelease [62.9 kB]
Get:3 http://deb.debian.org/debian stretch-updates InRelease [88.5 kB]
Get:4 http://security.debian.org stretch/updates/main amd64 Packages [68.7 kB]
Get:5 http://deb.debian.org/debian stretch Release [113 kB]
Get:6 http://deb.debian.org/debian stretch Release.gpg [3108 B]
Get:7 http://deb.debian.org/debian stretch/main amd64 Packages [9497 kB]
Fetched 9833 kB in 4s (2174 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
Package kernel-package is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'kernel-package' has no installation candidate
The command '/bin/sh -c apt-get update &&     DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends     sudo apt-file texinfo flex bison patch python unzip git bc     bzip2 wget less nano g++ gcc file libc6-dev  make     fakeroot build-essential devscripts kernel-package libncurses5-dev &&     apt-get clean &&     apt-file update' returned a non-zero code: 100
/home/s2e/s2e/source/guest-images//Makefile.linux:120: recipe for target '/home/s2e/s2e/images/.stamps/linux-build-x86_64' failed
make: *** [/home/s2e/s2e/images/.stamps/linux-build-x86_64] Error 100
make: Leaving directory '/home/s2e/s2e/images'

S2E runs out of memory even if there is plenty of it available

Looks like vm.max_map_count is too small on some systems. Increasing it solves the issue.
The problem is osAlloc in ConcreteBuffer.h which allocates one page at a time. Need to have a pool of pages instead.

Use this command to increase the count:

sysctl -w vm.max_map_count=655350

Switch libcpu/libtcg to C++

Mostly useful for namespaces and templates.
This will make it much easier to add support for multiple architectures without having to do crazy stuff with macros and build scripts.

makefile: list target may not work on non-english setups

Hi, when I want to compile a plugin, the s2e command not working.

I installed the s2e using s2e-env.

(venv) ➜  s2e-proj s2e build --rebuild-components libs2e
INFO: [sh.command] <Command u'/usr/bin/make --directory=/home/xxx/s2e-proj/build/s2e --file=/home/xxx/s2e-proj/source/s2e/Makefile list'>: starting process
INFO: [sh.command] <Command u'/usr/bin/make --directory=/home/xxx/s2e-proj/build/s2e --file=/home/xxx/s2e-proj/source/s2e/Makefile list', pid 87347>: process started
INFO: [sh.command] <Command u'/usr/bin/make --directory=/home/xxx/s2e-proj/build/s2e --file=/home/cxm/xxx/source/s2e/Makefile list', pid 87347>: process completed
INFO: [sh.command] <Command u'/usr/bin/make --directory=/home/xxx/s2e-proj/build/s2e --file=/home/cxm/xxx/source/s2e/Makefile list', pid 87347>: process completed
INFO: [sh.command] <Command u'/usr/bin/make --directory=/home/xxx/s2e-proj/build/s2e --file=/home/cxm/xxx/s2e-proj/source/s2e/Makefile list', pid 87347>: process completed
ERROR: [build] Component libs2e is not valid. Valid components are: 

BTW, when I use s2e build, the s2e-env downloaded some binaries from github then show me that build success, without compiling any of the components.

Looking forward to your reply!

windows: add fault injection support for device drivers

Write annotations to inject faults into kernel APIs. E.g., to test error recovery code.
This is better than using Driver Verifier, because symbolic fault injection allows systematic testing of recovery code (vs. probabilistic injection in Driver Verifier).

The annotations will go into s2e.sys and look like this:

PVOID Hook_ExPoolAllocateWithTag() {
  if (DecideInjectFault()) { // Fork 2 paths here (one faulty, one not faulty)
    return NULL;
  }
  return ExPoolAllocateWithTag()
}

Automating the generation of these annotations is out of scope for this issue.
We could parse WDK docs to extract possible errors codes from each API and use clang python bindings in order to parse WDK headers to get function prototypes.

libs2e is missing KVM debug features

Hi, I used ./launch-s2e.sh debug, and run inside gdb, but it gives me

s2e-block: dirty sectors on close:0
s2e-block: dirty after restore: 640 (ro=0)
s2e-block: wasted sectors: 1792
KVM: entry failed, hardware error 0x80000021

If you're running a guest on an Intel machine without unrestricted mode
support, the failure can be most likely due to the guest entering an invalid
state for Intel VT. For example, the guest maybe running in big real mode
which is not supported on less recent Intel processors.

EAX=00000000 EBX=00000002 ECX=ffffffff EDX=b7772878
ESI=bfa07f94 EDI=bfa08f14 EBP=00000000 ESP=bf9f7ee0
EIP=080487d0 EFL=00000246 [---Z-P-] CPL=3 II=0 A20=1 SMM=0 HLT=0
ES =007b 00000000 ffffffff 00c0f300 DPL=3 DS   [-WA]
CS =0073 00000000 ffffffff 00c0fb00 DPL=3 CS32 [-RA]
SS =007b 00000000 ffffffff 00c0f300 DPL=3 DS   [-WA]
DS =007b 00000000 ffffffff 00c0f300 DPL=3 DS   [-WA]
FS =0000 00000000 00000000 00000000
GS =0033 b75c7940 ffffffff 00d0f300 DPL=3 DS   [-WA]
LDT=0000 00000000 00000000 00008200 DPL=0 LDT
TR =0080 cfdf07c0 0000206b 00008900 DPL=0 TSS32-avl
GDT=     cfded000 000000ff
IDT=     fffba000 000007ff
CR0=8005003b CR2=b76352c0 CR3=0f8e1000 CR4=000006b0
DR0=00000000 DR1=00000000 DR2=00000000 DR3=00000000 
DR6=ffff0ff0 DR7=00000400
EFER=0000000000000000
Code=50 8e 04 08 e8 ab fd ff ff 83 c4 10 90 90 90 90 90 90 90 90 <31> c0 0f 3f 00 00 00 00 00 00 00 00 85 c0 74 f0 83 ec 0c 68 70 8e 04 08 e8 83 fd ff ff 83

I used qemu's -s option to attach gdb to s2e, as written in this doc.

I modified launch-s2e.sh script as follows, and remotely attach it.

LD_PRELOAD=$LIBS2E $QEMU -S -s $DRIVE \
    -k en-us $GRAPHICS -monitor null -m 256M -enable-kvm \
    -serial file:serial.txt -net none -net nic,model=e1000  \
    -loadvm ready $*

But it gives me

libs2e: unknown KVM VCPU IOCTL vcpu 234238 request=0x4048ae9b arg=0x7ffc4a80b5a0 ret=0xffffffff
libs2e: unknown KVM VCPU IOCTL vcpu 234238 request=0x4048ae9b arg=0x7ffc4a80b5a0 ret=0xffffffff

The first problem looks like that it is related to kernel.
Currently, I am using Ubuntu 14.04.
What's OS in the development machine? 16.04?

Thanks.

Add more debug logs

  • Periodic screenshots of the guest
  • Hash of every S2E component in debug.txt
  • Revision of the guest images

Build guest images without KVM

KVM should be optional when building guest images - it is only needed to improve performance. Some systems may not have KVM available - e.g. the Windows Subsystem for Linux (WSL) - but guest images should still be build-able on these systems.

Factor out InvokePlugin() types

Currently, structures passed by the guest to S2E are duplicated in libs2eplugins and guest-tools repos. They shall all go to the guest tools repo. Also, figure out how to avoid duplicating types between Windows and Linux guest tools.

Extract kernels from KB files

E.g., Windows6.1-KB3033929-x64.msu installs a new kernel.
Update the script to extract kernels from KBs in addition to ISO files.

CROMU_00001 testing

Hi, I want to test s2e with other CGC binary, CROMU_00001.
I used the s2e docker image, and use /demo/run.sh to create a new project for the CROMU_00001.

But it gives me a lot of warnings and error.
I attached my debug.txt.
Do you have any idea what the problem is?

Also, if I want to add a seed file, do I have to just put it to the seeds folder?
In that case, should I use XML format or just binary?

Thanks.

image_build errors on i386 and x86-64

Hi,

I am attempting to build on a 16.04 machine, and I get these errors when I run s2e image_build debian-8.7.1-i386:

Working dir: /nas/raywang/testing/images/.tmp-output/linux-4.9.3-i386/linux-4.9.3
/bin/bash: /nas/raywang/testing/source/guest-images/Linux/docker/make-kernel.sh: No such file or directory
/nas/raywang/testing/source/guest-images//Makefile.linux:151: recipe for target '/nas/raywang/testing/images/.stamps/linux-4.9.3-i386' failed

and s2e image_build debian-8.7.1-x86_64:

E: Package 'kernel-package' has no installation candidate
The command '/bin/sh -c apt-get update &&     DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends     sudo apt-file texinfo flex bison patch python unzip git bc     bzip2 wget less nano g++ gcc file libc6-dev  make     fakeroot build-essential devscripts kernel-package libncurses5-dev &&     apt-get clean &&     apt-file update' returned a non-zero code: 100

more s2e build errors

So manually setting the llvm download in issue #33, i was able to get past that. However, after clang finished building I got more errors:

make[3]: Entering directory '/home/user/opt/s2e/build/s2e/libvmi-release'
[  5%] Building CXX object src/CMakeFiles/vmi.dir/DecreeFile.cpp.o
[ 10%] Building CXX object src/CMakeFiles/vmi.dir/ExecutableFile.cpp.o
[ 21%] Building CXX object src/CMakeFiles/vmi.dir/FileProvider.cpp.o
[ 21%] Building CXX object src/CMakeFiles/vmi.dir/ElfDwarf.cpp.o
[ 26%] Building CXX object src/CMakeFiles/vmi.dir/PEFile.cpp.o
[ 31%] Building CXX object src/CMakeFiles/vmi.dir/WindowsCrashDumpGenerator.cpp.o
[ 36%] Building CXX object src/CMakeFiles/vmi.dir/Vmi.cpp.o
[ 42%] Building CXX object src/CMakeFiles/vmi.dir/WinKernDumpFile.cpp.o
/home/user/opt/s2e/source/s2e/libvmi/src/ElfDwarf.cpp:9:10: fatal error: 'dwarf.h' file not found
#include <dwarf.h>
         ^
In file included from /home/user/opt/s2e/source/s2e/libvmi/src/Vmi.cpp:8:
/home/user/opt/s2e/source/s2e/libvmi/include/vmi/ElfDwarf.h:13:10: fatal error: 'libdwarf.h' file not found
#include <libdwarf.h>
         ^
1 error generated.
src/CMakeFiles/vmi.dir/build.make:182: recipe for target 'src/CMakeFiles/vmi.dir/Vmi.cpp.o' failed
make[3]: *** [src/CMakeFiles/vmi.dir/Vmi.cpp.o] Error 1
make[3]: *** Waiting for unfinished jobs....
1 error generated.
src/CMakeFiles/vmi.dir/build.make:86: recipe for target 'src/CMakeFiles/vmi.dir/ElfDwarf.cpp.o' failed
make[3]: *** [src/CMakeFiles/vmi.dir/ElfDwarf.cpp.o] Error 1
make[3]: Leaving directory '/home/user/opt/s2e/build/s2e/libvmi-release'
CMakeFiles/Makefile2:270: recipe for target 'src/CMakeFiles/vmi.dir/all' failed
make[2]: *** [src/CMakeFiles/vmi.dir/all] Error 2
make[2]: Leaving directory '/home/user/opt/s2e/build/s2e/libvmi-release'
Makefile:127: recipe for target 'all' failed
make[1]: *** [all] Error 2
make[1]: Leaving directory '/home/user/opt/s2e/build/s2e/libvmi-release'
/home/user/opt/s2e/source/s2e/Makefile:155: recipe for target 'stamps/libvmi-release-make' failed
make: *** [stamps/libvmi-release-make] Error 2
rm clang+llvm-3.9.0-x86_64-linux-gnu-ubuntu-16.04make: unlink: clang+llvm-3.9.0-x86_64-linux-gnu-ubuntu-16.04: Is a directory

make: Leaving directory '/home/user/opt/s2e/build/s2e'
ERROR: [build]

  RAN: /usr/bin/make --directory=/home/user/opt/s2e/build/s2e --file=/home/user/opt/s2e/source/s2e/Makefile install

  STDOUT:


  STDERR:

Perhaps you have missing dependencies? It would seem possibly libdwarf-dev needs to be installed.

s2e build failing

I'm attempting to get S2E running, however S2E build is failing with the following:

ar rcu liblua.a lapi.o lcode.o lctype.o ldebug.o ldo.o ldump.o lfunc.o lgc.o llex.o lmem.o lobject.o lopcodes.o lparser.o lstate.o lstring.o ltable.o ltm.o lundump.o lvm.o lzio.o lauxlib.o lbaselib.o lbitlib.o lcorolib.o ldblib.o liolib.o lmathlib.o loslib.o lstrlib.o ltablib.o loadlib.o linit.o
ar: `u' modifier ignored since `D' is the default (see `U')
ranlib liblua.a
gcc -o lua   lua.o liblua.a -lm -Wl,-E -ldl
gcc -o luac   luac.o liblua.a -lm -Wl,-E -ldl
make[3]: Leaving directory '/home/user/opt/s2e/build/s2e/lua-5.2.4/src'
make[2]: Leaving directory '/home/user/opt/s2e/build/s2e/lua-5.2.4/src'
make[1]: Leaving directory '/home/user/opt/s2e/build/s2e/lua-5.2.4'
touch stamps/lua-make
mkdir -p libvmi-release
mkdir -p llvm-release
wget http://llvm.org/releases/3.9.0/clang+llvm-3.9.0-x86_64-linux-gnu-ubuntu-16.10.tar.xz
--2017-07-06 20:45:47--  http://llvm.org/releases/3.9.0/clang+llvm-3.9.0-x86_64-linux-gnu-ubuntu-16.10.tar.xz
Resolving llvm.org (llvm.org)... 54.67.122.174
Connecting to llvm.org (llvm.org)|54.67.122.174|:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: http://releases.llvm.org/3.9.0/clang+llvm-3.9.0-x86_64-linux-gnu-ubuntu-16.10.tar.xz [following]
--2017-07-06 20:45:47--  http://releases.llvm.org/3.9.0/clang+llvm-3.9.0-x86_64-linux-gnu-ubuntu-16.10.tar.xz
Resolving releases.llvm.org (releases.llvm.org)... 151.101.32.204
Connecting to releases.llvm.org (releases.llvm.org)|151.101.32.204|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2017-07-06 20:45:48 ERROR 404: Not Found.

/home/user/opt/s2e/source/s2e/Makefile:164: recipe for target 'clang+llvm-3.9.0-x86_64-linux-gnu-ubuntu-16.10.tar.xz' failed
make: *** [clang+llvm-3.9.0-x86_64-linux-gnu-ubuntu-16.10.tar.xz] Error 8
make: Leaving directory '/home/user/opt/s2e/build/s2e'
ERROR: [build]

  RAN: /usr/bin/make --directory=/home/user/opt/s2e/build/s2e --file=/home/user/opt/s2e/source/s2e/Makefile install

  STDOUT:


  STDERR:

For reference, 16.10 is the Ubuntu version I am running, however it is not a valid download for LLVM (http://releases.llvm.org/download.html#3.9.0)

Arm support required

Thanks for your work! The s2e official website said that

S²E runs unmodified x86, x86-64, or ARM software stacks

I checked the s2e-env but didn't find arm support. Does or Will the project support ARM architecture?

Thanks!

Parallel S2E Not Working

So I'm attempting to follow these instructions: https://github.com/S2E/docs/blob/master/src/Howtos/Parallel.rst

I set the S2E_MAX_PROCESSES variable to 8 in my case in the launcher.sh script for the project. Then, I have tried both running the launcher script as well as using s2e run. Neither end up using more than a single core.

A side note as well, when I use the s2e run command, it appears to just get stuck at "Waiting for analysis". I have yet to get past that point.

I am still getting a bunch of warnings like this in "warnings.txt" file

WARNING: Cannot get configuration value 'pluginsConfig['LinuxMonitor'].terminateOnSegfault':
    value of type nil can not be converted to boolean

WARNING: Cannot get configuration value 'pluginsConfig['ProcessExecutionDetector'].logLevel':
    value of type nil can not be converted to string

WARNING: Cannot get configuration value 'pluginsConfig['ModuleExecutionDetector'].logLevel':
    value of type nil can not be converted to string

WARNING: Cannot get configuration value 'pluginsConfig['ModuleExecutionDetector'].trackAllModules':
    value of type nil can not be converted to boolean

WARNING: Cannot get configuration value 'pluginsConfig['ModuleExecutionDetector'].configureAllModules':
    value of type nil can not be converted to boolean

WARNING: Cannot get configuration value 'pluginsConfig['ModuleExecutionDetector'].trackExecution':
    value of type nil can not be converted to boolean

WARNING: Cannot get configuration value 'pluginsConfig['StaticFunctionModels'].logLevel':
    value of type nil can not be converted to string

WARNING: Cannot get configuration value 'pluginsConfig['StaticFunctionModels'].count':
    value of type nil can not be converted to integer

WARNING: Cannot get configuration value 'pluginsConfig['CUPASearcher'].batchTime':
    value of type nil can not be converted to integer

WARNING: Cannot get configuration value 'pluginsConfig['CUPASearcher'].enabled':
    value of type nil can not be converted to boolean

WARNING: Cannot get configuration value 'pluginsConfig['TranslationBlockCoverage'].logLevel':
    value of type nil can not be converted to string

WARNING: Cannot get configuration value 'pluginsConfig['ModuleTracer'].logLevel':
    [string "return pluginsConfig['ModuleTracer'].logLevel..."]:1: attempt to index field 'ModuleTracer' (a nil value)

WARNING: Cannot get configuration value 'pluginsConfig['CorePlugin'].logLevel':
    [string "return pluginsConfig['CorePlugin'].logLevel"]:1: attempt to index field 'CorePlugin' (a nil value)

Cannot get Linux binary tutorial to execute symbolically

I'm following the instructions at https://github.com/S2E/docs/blob/master/src/Tutorials/Coreutils.rst
but when I run launch-s2e.sh, the program terminates within seconds.

2 [State 0] LinuxMonitor: Module load not yet implemented
2 [State 0] LinuxMonitor: Module load not yet implemented
2 [State 0] LinuxMonitor: Module load not yet implemented
2 [State 0] LinuxMonitor: Module load not yet implemented
2 [State 0] LinuxMonitor: Module load not yet implemented
2 [State 0] LinuxMonitor: Module load not yet implemented
2 [State 0] BaseInstructions: Message from guest (0xb77907d1): S2E_SYM_ARGS is not set. All arguments will be concrete

I can't get it to recognize S2E_SYM_ARGS anywhere. Is this tutorial out of date?

S2E gets stuck when switching to seed state

Can be reproduced in release build using CROMU_00001 and a sample pov.
Does not seem to occur in debug build.

4 [State 0] Forking state 0 at pc = 0x80497d9 at pagedir = 0xf343000
    state 0
    state 1
BEGIN searcher description
DFSSearcher
END searcher description
4 [State 0] BaseInstructions: Message from guest (0x804a10a): Going to next seed loop iteration

Image Build Error

I been trying to build any of linux s2e image using s2e image_build but all seem to fail unsuccessfully.

cd /home/jtumina/s2e/images/.tmp-output/decree-cgc-cfe-i386/decree-cgc-cfe && mv config-i386 .config
/home/jtumina/s2e/source/guest-images//Linux/docker/run-docker.sh linux-build-i386 /home/jtumina/s2e/images/.tmp-output/decree-cgc-cfe-i386/decree-cgc-cfe /home/jtumina/s2e/source/guest-images//Linux/docker/make-kernel.sh /home/jtumina/s2e/source/s2e-linux-kernel/include 98323 98323
Working dir: /home/jtumina/s2e/images/.tmp-output/decree-cgc-cfe-i386/decree-cgc-cfe
/bin/bash: /home/jtumina/s2e/source/guest-images/Linux/docker/make-kernel.sh: No such file or directory
/home/jtumina/s2e/source/guest-images//Makefile.linux:150: recipe for target '/home/jtumina/s2e/images/.stamps/decree-cgc-cfe-i386' failed
make: *** [/home/jtumina/s2e/images/.stamps/decree-cgc-cfe-i386] Error 127
make: *** Waiting for unfinished jobs....

Yet, I checked and make-kernel.sh does exist in that directory. Why would it be unable find such file?

I did see an earlier issue #30, who had a similar error and the response was either to try in their home directory or use s2e image_build -d. This is in my home directory and unfortunately -d option will not work for me as I am on a corporate network with required proxy servers that I am unable to seem to configure to be used with the google api call to download the image.

s2e got stuck infinitely when using seeds support

to test a simple program ./proga on debain 8.7.1 x64 built image, s2e got stuck infinitely.
reproduce:
sys: ubuntu 16.04
steps:

  1. s2e new_project -s -i debian-8.7.1-x86_64 -n progra ./proga @@
  2. put seeds to proga/seeds dir
  3. s2e run proga

the latest std outputs is the following:

3 [State 0] BaseInstructions: BaseInstructions: s2e_end_atomic
3 [State 0] BaseInstructions: Message from guest (0x7ffdf76ae510): s2e_seed_get_file: ret=-1 should_fork=1 seed_file=
3 [State 0] BaseInstructions: Inserted symbolic data @0x7ffdf76ae720 of size 0x4: seed_fork='\x00\x00\x00\x00' pc=0x401bfa
3 [State 0] Forking state 0 at pc = 0x401c09 at pagedir = 0xde3f000
state 0
state 1
BEGIN searcher description
DFSSearcher
END searcher description
3 [State 0] BaseInstructions: Message from guest (0x40261e): Going to next seed loop iteration
3 [State 0] ProcessExecutionDetector: Unloading process 0x526
3 [State 0] ModuleExecutionDetector: Process 0x1e08000 (pid=0x526) is unloaded
3 [State 0] LinuxMonitor: Removing task (pid=0x526, cr3=0xde3f000, exitCode=256) record from collector.
3 [State 0] LinuxMonitor: Process /bin/sleep loaded entry_point=0x7f0098343130 pid=0x527 start_code=0x400000 end_code=0x4063dc start_data=0x606e10 end_data=0x607234 start_stack=0x7fffee810560
3 [State 0] LinuxMonitor: ModuleDescriptor Name=sleep Path=/bin/sleep NativeBase=0x400000 LoadBase=0x400000 Size=0x207234 AddressSpace=0xdd16000 Pid=0x527 EntryPoint=0x7f0098343130 Checksum=0xe06f4a90
3 [State 0] ModuleExecutionDetector: Module sleep loaded - Base=0x400000 NativeBase=0x400000 Size=0x207234 AS=0xdd16000
3 [State 0] ModuleExecutionDetector:

s2e init error

So i've gone back and am now running the s2e tool inside 16.04.2 TLS. I mentioned the setup problems over in the docs section (probably should have opened that ticket here).

Once I got past that problem, now init is having problems:

s2e init /home/s2e/s2e
INFO: [init] Creating environment in /home/s2e/s2e
INFO: [init] Installing S2E dependencies
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Note, selecting 'libsdl1.2-dev' instead of 'libsdl-dev'
E: Unable to locate package libncurses5-dev:i386
E: Unable to locate package libglib2.0-0:i386
E: Couldn't find any package by glob 'libglib2.0-0'
E: Couldn't find any package by regex 'libglib2.0-0'
ERROR: [init] 

  RAN: /usr/bin/sudo -S apt-get install build-essential cmake wget git texinfo flex bison python-dev libguestfs-tools genisoimage python-pip xz-utils docker.io p7zip-full pxz fuse libhivex-bin libdwarf-dev libelf-dev libiberty-dev binutils-dev libreadline-dev libboost-dev zlib1g-dev libjemalloc-dev nasm pkg-config libmemcached-dev libvdeplug-dev libpq-dev libc6-dev-i386 libboost-system-dev libboost-serialization-dev libboost-regex-dev libprotobuf-dev protobuf-compiler libbsd-dev libsigc++-2.0-dev libsdl-dev libglib2.0-dev python-docutils qemu mingw-w64 lcov libprocps4-dev lib32ncurses5 lib32ncurses5-dev libncurses5-dev libx32ncurses5-dev libncurses5-dev:i386 libglib2.0-0:i386

  STDOUT:


  STDERR:

For reference:

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.2 LTS
Release:        16.04
Codename:       xenial

is windows 10 supported?

After installing windows 10 1703, it seems when restore the snap, win10 restarted and then s2e coredumped... historically s2e1 has the same problem.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.