NILI: A Tool For Network Scan, Man in the Middle, Protocol Reverse Engineering And Fuzzing

NILI: A Tool For Network Scan, Man in the Middle, Protocol Reverse Engineering And Fuzzing

Installing

Here is some Instructions for Installing Prerequisites, Select Proper Instructions for your Operating System.

Unix-like

1- Install Python3 and pip:

$ sudo apt-get install python3
$ sudo apt-get install python3-pip

2- Install Scapy:

$ cd /tmp
$ git clone https://github.com/phaethon/scapy

$ cd scapy
$ sudo python3 setup.py install

3- Install Netzob:

$ git clone https://dev.netzob.org/git/netzob
$ cd ./netzob/
$ sudo apt-get install python3 python3-dev python3-setuptools build-essential
$ python3 setup.py install
$ python3 -m pip install bintrees --upgrade

Windows

1- Install python3

2- Install Scapy:

2-1- Install Winpcap
2-2- Install Scapy3k

python -m pip install scapy-python3

3- Install Netzob

Download

PyREBox A Python Scriptable Reverse Engineering Sandbox

Python scriptable Reverse Engineering Sandbox, a Virtual Machine instrumentation and inspection framework based on QEMU

PyREBox is a Python scriptable Reverse Engineering sandbox. It is based on QEMU, and its goal is to aid reverse engineering by providing dynamic analysis and debugging capabilities from a different perspective. 

PyREBox allows to inspect a running QEMU VM, modify its memory or registers, and to instrument its execution, by creating simple scripts in python to automate any kind of analysis.

QEMU (when working as a whole-system-emulator) emulates a complete system (CPU, memory, devices...). By using VMI techniques, it does not require to perform any modification into the guest operating system, as it transparently retrieves information from its memory at run-time.

Several academic projects such as DECAF, PANDA, S2E, or AVATAR, have previously leveraged QEMU based instrumentation to overcome reverse engineering tasks. These projects allow to write plugins in C/C++, and implement several advanced features such as dynamic taint analysis, symbolic execution, or even record and replay of execution traces. With PyREBox, author aim to apply this technology focusing on keeping the design simple, and on the usability of the system for threat analysts.

Goals

• Provide a whole system emulation platform with a simple interface for inspecting the emulated guest system.

1. Fine grained instrumentation of system events.
2. Integrated Virtual Machine Introspection (VMI), based on volatility. No agent or driver needs to be installed into the guest.
3. An IPython based shell interface.
4. A Python based scripting engine, that allows to integrate into the scripts any of the security tools based on this language (one of the biggest ecosystems).

• Have a clean design, de-coupled from QEMU. Many projects that are built over QEMU do not evolve when QEMU gets upgraded, missing new features and optimizations, as well as security updates. In order to achieve this, PyREBox is implemented as an independent module that can be compiled together with QEMU requiring a minimal set of modifications.
• Support for different architectures. Currently, PyREBox only supports Windows for x86 and x86-64 bit architectures, but its design allows to support other architectures such as ARM, MIPS, or PowerPC, and other operating systems as well.

IPython shell

Starting a PyREBox shell is as easy as typing the sh command on QEMU’s monitor. It will immediately start an IPython shell. This shell records the command history as well as the defined variables. For instance, you can save a value and recover it later at a different point of the execution, when you start the shell again. PyREBox takes advantage of all the available features in IPython such as auto-completion, command history, multi-line editing, and automated command help generation.

PyREBox will allow you to debug the system (or a process) in a fairly stealthy way. Unlike traditional debuggers which stay in the system being debugged (even modifying the memory of the debugged process to insert breakpoints), PyREBox stays completely outside the inspected system, and it does not require the installation of any driver or component into the guest.

REBox offers a complete set of commands to inspect and modify the state of the running VM. Just type list_commands to obtain a complete list. You can run any volatility plugin just by typing vol and the corresponding volatility command. For a complete list of available volatility plugins, you can type list_vol_commands. This list is generated automatically, so it will also show any volatility plugin you install on PyREBox's volatility/ path.

You can also define your own commands! It is as simple as declaring a function in a script, and loading it.

If you need something more expressive than a command, you can write a Python snippet leveraging the API. For a detailed description of the API, see Documentation or type help(api) in the shell.

Scripting

PyREBox allows to dynamically load scripts that can register callback functions that are called when certain events occur, like instructions executed, memory read/written, processes created/destroyed, and so on.

Given that PyREBox is integrated with Volatility, it will let you take advantage of all the volatility plugins for memory forensics in your python scripts. Many of the most famous reverse engineering tools are implemented in Python or at least have Python bindings. Our approach allows to integrate any of these tools into a script.

Finally, given that python callbacks can introduce a performance penalty on frequent events such as instructions executed, it is also possible to create triggers. Triggers are native-code plug-in’s (developed in C/C++) that can be inserted dynamically at run-time on any event just before the Python callback is executed. This allows to limit the number of events that hit the python code, as well as to precompute values in native code.

In this repository you will find example scripts that can help you to write your owncode . Contributions are welcome!

Install

A build script is provided. For specific details about dependencies, please see BUILD. We also provide a Dockerfile.

Starting a VM

PyREBox is based on QEMU, so in order to start a VM withing PyREBox, you need to run it exactly as you were booting up a QEMU VM. A couple of example scripts are provided: start_i386.sh, start_x86_64.sh, you can use them as an example.

The only QEMU monitor option supported currently is stdio (-monitor stdio).

Roadmap

• VM image configuration and management console.
• Support for ARM, MIPS, and other architectures.
• Support for GNU/Linux guest systems.

Download PyREBox

BARF: Binary Analysis And Reverse Engineering Framework

A Multi-platform open source Binary Analysis and Reverse engineering Framework.

The analysis of binary code is a crucial activity in many areas of the computer sciences and software engineering disciplines ranging from software security and program analysis to reverse engineering.

Manual binary analysis is a difficult and time-consuming task and there are software tools that seek to automate or assist human analysts. However, most of these tools have several technical and commercial restrictions that limit access and use by a large portion of the academic and practitioner communities.

BARF is an open source binary analysis framework that aims to support a wide range of binary code analysis tasks that are common in the information security discipline. It is a scriptable platform that supports instruction lifting from multiple architectures, binary translation to an intermediate representation, an extensible framework for code analysis plugins and inter operation with external tools such as debuggers, SMT solvers and instrumentation tools. The framework is designed primarily for human-assisted analysis but it can be fully automated.

The BARF project includes BARF and related tools and packages. So far the project is composed of the following items:

BARF : A multiplatform open source Binary Analysis and Reverse engineering Framework
PyAsmJIT : A JIT for the Intel x86_64 and ARM architecture.

Tools built upon BARF:

BARFgadgets : Lets you search, classify and verify ROP gadgets inside a binary program.

BARFcfg : Lets you recover the control-flow graph of the functions of a binary program.
BARFcg : Lets you recover the call graph of the functions of a binary program.

BARF

BARF is a Python package for binary analysis and reverse engineering. It can:

Load binary programs in different formats (ELF, PE, etc),
It supports the Intel x86 architecture for 32 and 64 bits,
It supports the ARM architecture for 32 bits,
It operates on an intermediate language (REIL) thus all analysis algorithm are architecture-agnostic,
It has integration with Z3 and CVC4 SMT solvers which means that you can express fragments of code as formulae and check restrictions on them.
It is currently under development.

Installation

 

BARF depends on the following SMT solvers:

Z3 : A high-performance theorem prover being developed at Microsoft Research.
CVC4 : An efficient open-source automatic theorem prover for satisfiability modulo theories (SMT) problems.

The following command installs BARF on your system:

$ sudo python setup.py install

You can also install it locally:

$ sudo python setup.py install --user

Notes

Only one SMT solver is needed in order to work. You may choose between Z3 and CVC4 or install both. You can use the barf-install-solver.sh script which downloads and install both solver.
To run some tests you need to install PyAsmJIT first.

Download

 

Reverse Engineering Toolset And Fuzzing Protobuf-based Apps

Reverse Engineering Toolset And Fuzzing Protobuf-based Apps

Protobuf is a serialization format developed by Google and used in an increasing number of Android, web, desktop and more applications. It consists of a language for declaring data structures, which is then compiled to code or another kind of structure depending on the target implementation.

pbtk (Protobuf toolkit) is a full-fledged set of scripts, accessible through an unified GUI, that provides two main features:

• Extracting Protobuf structures from programs, converting them back into readable .protos, supporting various implementations:All the main Java runtimes (base, Lite, Nano, Micro, J2ME), with full Proguard support,
  Binaries containing embedded reflection metadata (typically C++, sometimes Java and most other bindings),
  Web applications using the JsProtoUrl runtime.
  
  • Editing, replaying and fuzzing data sent to Protobuf network endpoints, through a handy graphical interface that allows you to edit live the fields for a Protobuf message and view the result.

  Installation

  
  PBTK requires Python ≥ 3.6, PyQt 5, Python-Protobuf 3, and a handful of executable programs (chromium, jad, dex2jar...) for running extractor scripts.
  
  Archlinux users can install directly through the package:
  
  $ yaourt -S pbtk-git
  $ pbtk    On most distributions, you'll want to run the cx_freeze executable (bundling Python 3.6 along with all modules):
  
  $ wget https://github.com/marin-m/pbtk/releases/download/1.0.1/pbtk-1.0.1-linux-cxfreeze.tar.gz
  $ tar xvf pbtk-1.0.1-linux-cxfreeze.tar.gz && cd pbtk-1.0.1-linux-cxfreeze
  $ ./gui
  
  Or run it directly if you have the required dependencies:
  
  $ git clone https://github.com/marin-m/pbtk
  $ cd pbtk
  $ ./gui.py
  
  Windows support is untested yet. Once you run the GUI, it should warn you on what you are missing depending on what you try to do.

  Command line usage

  The GUI can be lanched through the main script:
  
  ./gui.py
  
  The following scripts can also be used standalone, without a GUI:
  
  ./extractors/jar_extract.py [-h] input_file [output_dir]
  ./extractors/from_binary.py [-h] input_file [output_dir]
  ./extractors/web_extract.py [-h] input_url [output_dir]
  
  Scripts don't have an extension and are at root of archive when you use the cx_freeze image.
  
  Download 

Binary Code Static Analyser With IDA Integration

Binary code static analyser, with IDA integration. Performs value and taint analysis, type reconstruction.

What is BinCAT?

BinCAT is a static Binary Code Analysis Toolkit, designed to help reverse engineers, directly from IDA.

It features:

• value analysis (registers and memory)
• taint analysis
• type reconstruction and propagation
• backward and forward analysis

Quick FAQ

Supported Platforms:

• IDA plugin: all, version 6.9 or later
• analyzer (local or server): Linux, macOS (maybe)

Supported CPUs (for now):

x86-32

Installation

Analyzer

The analyzer is only supported on Linux, but can expose a Web service for use from a Windows IDA.

Using Docker: Docker installation instructions
Manual: Manual installation instructions

IDA Plugin

Only IDA v6.9 or later are supported

Install for Windows

• In IDA, click on "File -> Script File..." menu (or type ALT-F7)
• Select python\windows_install_plugin.py
• the BinCAT plugin is now installed in your IDA user dir

Or install manually.

Linux install

Installation instructions

Using BinCAT

Now you can run analyses (Ctrl-Shift-A)

Quick start

Load the plugin by using the Ctrl-Shift-B shortcut, or using the Edit -> Plugins -> BinCAT menu

Select an instruction in any IDA view, then use the Ctrl-Shift-A shortcut, or the BinCAT -> Analyze from here context menu

Configuration

Global options can be configured through the Edit/BinCAT/Options menu.

Default config and options are stored in $IDAUSR/idabincat/conf.

Options

• Use remote bincat: select if you are running docker in a Docker container
• Remote URL: http://localhost:5000 (or the URL of a remote BinCAT server)
• Autostart: autoload BinCAT at IDA startup
• Save to IDB: default state for the save to idb checkbox

Analyzer configuration files

Default config for analyzer.

Documentation

A manual is provided.

A tutorial is provided to help you try BinCAT's features. It makes use of a sample binary and screenshots.

Log levels

• basic info
• more info
• debug
• advanced debug

Download

HookCase A Tool For Reverse Engineering MacOS/OS X

HookCase A Tool For Reverse Engineering MacOS/OS X

HookCase is a tool for debugging and reverse engineering applications on macOS (aka OS X), and the operating system itself. 

It re-implements and extends Apple's DYLD_INSERT_LIBRARIES functionality. It can be used to hook any method defined in any module's symbol table, whether or not that method is exported. In a single operation, it can be applied to a parent process and all its child processes, whether or not the child processes inherit their parent's environment.

So HookCase is considerably more powerful than DYLD_INSERT_LIBRARIES. It also doesn't have the restrictions Apple has placed on DYLD_INSERT_LIBRARIES. So, for example, HookCase can be used with applications that have entitlements.

HookCase runs on OS X 10.9 (Mavericks) through macOS 10.12 (Sierra).

Installing

On OS X 10.10 (Yosemite) and up, to load HookCase.kext you'll need to turn off Apple's protection against loading unsigned or "inappropriately" signed kernel extensions. (From Apple's point of view, the only "appropriately" signed kernel extensions are those signed with a special kernel extension signing certificate -- in practice almost exclusively Apple's own kernel extensions.) On OS X 10.11 (El Capitan) and up, to do this you'll need to turn off "rootless mode". Rootless mode will also prevent you from copying the HookCase.kext binary to its final destination.

On OS X 10.10:

From the command line run nvram boot-args to see if you already have some boot-args.
Then run the following command:

sudo nvram boot-args="<existing-boot-args> kext-dev-mode=1"

Reboot your computer.

On OS X 10.11 and up:

Boot into your Recovery partition by restarting your computer and pressing Command-R immediately after you hear the Mac startup sound. Release these keys when you see the Apple logo.

Choose Utilties : Terminal, then run the following at the command line:

csrutil disable

Quit Terminal and reboot your computer.

Now copy HookCase.kext to the /usr/local/sbin/ directory. One way to do this is with the following command:

sudo cp -R HookCase.kext /usr/local/sbin

To load HookCase.kext into the kernel, do the following on the command line:

sudo kextutil /usr/local/sbin/HookCase.kext

Because it won't have been signed using a kernel extension signing certificate, you'll see the following error (or something like it):

    Diagnostics for HookCase.kext:
    Code Signing Failure: code signature is invalid
    kext-dev-mode allowing invalid signature -67050 0xFFFFFFFFFFFEFA16
      for kext "HookCase.kext"
    kext signature failure override allowing invalid signature -67050
      0xFFFFFFFFFFFEFA16 for kext "/usr/local/sbin/HookCase.kext"

Run kextstat to see that it did load.

To unload HookCase.kext from the kernel, run the following command:

sudo kextunload -b org.smichaud.HookCase

HookCase supports the release, development and debug kernels. But if you use it with the debug kernel, we recommend increasing the kernel stack size. One way to do this is as follows. kernel_stack_pages defaults to 4.

Copy kernel.debug (from the appropriate Kernel Debug Kit) to /System/Library/Kernels.

 sudo nvram boot-args="kcsuffix=debug kernel_stack_pages=6"

Reboot your computer.

Without this change, you sometimes get kernel panics using the debug kernel, at least on OS X 10.11 (El Capitan). These are usually double-faults with CR2 set to an address on the stack (indicating a stack underflow).


Download HookCase