Reflash flash research framework

Reflash is a proof-of-concept framework for analysing flash files. It produces a SQL database of flash VM stack trace by injecting dynamically generated instrumentation to flash files. The SQL database can later be analyzed with various tools.

Main features of Reflash:

• Extract embedded flash images
• Fetch and analyze the flash stack trace
• Run YARA over the stack values and disassembly
• Browse the database with a GUI tool, which works like a tradtional debugger
• Fake flash player version

Reflash (capital 'R') refers to an overall framework, and reflash to a standalone instrumentation tool.

Disclamer: Reflash is a proof-of-concept. So no proper packaging or system-wide installation, or anything like that. It might crash and burn every now and then. If it works for you, I'm very happy for you. Please give me feedback on any problems you face, and of course please do share your success stories as well.

reflash

reflash is the actual tool for flash file manipulation. Together with the Instrument library (in directory "instrument"), it forms an instrumentation environment much like Intel pin or Dynamorio.

reflash has the capability to insert hooks for arbitrary opcodes (in practice, not all opcodes are supported). There are three types of hooks supported by reflash:

• Method entry hook for collecting method arguments
• Generic opcode hook for collecting stack trace and manipulating function call arguments prior to execution
• Post opcode hook for manipulating stack values (only "getproperty" supported in current version)

In addition to inserting instrumentation, reflash can be used for disassembling and assembling any flash file.

For more information, please read the reflash documentation.

Instrument

Instrument library is responsible for analyzing and manipulating arguments sent by the instrumentation hooks. It is an standalone SWF file, but it is required to implement a specific API corresponding to the hook types described above. Instrument is configured statically prior to execution with a recompiler tool included in the distribution.

Please see the directory "instrument/instrument_package" for more details.

Framework

Framework is built around mitmproxy, the python man-in-the-middle proxy. In addition to HTTP(S) proxy, Framework collects stack trace from Instrument. Collected stack trace is put to a SQL database for later inspection.

In the "framework" directory, a collection tools can be found:

• proxy.py: main proxy module, also a standalone command line tool
• dbtool.py: module for manipulating SQL database, also a standalone command line tool
• run_file.sh: frontend script for the proxy
• run_live.sh: frontend script for the proxy
• logserver.py: log server module, also a standalone command line tool
• recompile.py: module for manipulating Instrument SWF file, also a standalone command line tool
• replay: graphical frontend for the database
• monitor: graphical frontend for the proxy

Replay

replay is a graphical frontend for the database. Main features include:

• Disassembly
• Stack trace
• Stack data hex view
• Step back and forth, set breakpoints
• Search data
• Run YARA

Installation

Source installation is described here only for 64-bit Ubuntu 14.04/16.04 LTS. Other systems might have a bit different set of prerequisities, but the instructions should be applicable to most debian-based systems with minor tweaks.

Install D-compiler

For compiling reflash, D compiler is required. Download and install the .deb package corresponding to your environment (Ubuntu/Debian x86_64 in this example).

Install the package (for example 2.0.73.0):

$ sudo dpkg -i dmd_2.073.0-0_amd64.deb

Compile reflash

For compression support, install liblzma development package:

$ sudo apt-get install liblzma-dev

With liblzma and DMD installed, compile reflash with the following commands:

$ cd framework
$ make

This should produce a compiled binary reflash and a symbolic link to it in directory "framework".

Python tools and modules

For installing rest the python modules, you first need to install pip and few other dependencies:

$ sudo apt-get install git python-qt4 python-pip python-dev libffi-dev libssl-dev libtiff5-dev libjpeg8-dev zlib1g-dev libwebp-dev libxml2-dev libxslt1-dev

misc. packages with pip

$ sudo pip install psutil
$ sudo pip install yara-python
$ sudo pip install setuptools --upgrade

pyamf

$ git clone https://github.com/fmoo/pyamf.git
$ cd pyamf
$ python setup.py build
$ sudo python setup.py install

Note: this is an unofficial pyamf module for extended AMF datatype support. Do not install the official pyamf with pip.

mitmproxy

Reflash python modules are written in python2, so you need a latest 0.18.x version of mitmproxy (0.17.x is not compatible with reflash). For default pip (pip2), this should be straightforward as this:

$ sudo pip install mitmproxy==0.18.3

Mitmproxy tries hard to get rid of python2, so the latest versions (1.0+) support only python3. If pip fails to install version 0.18.x, you need to get the source from github with 0.18.x tag and do manual build (python setup.py build etc.)

selenium (optional)

If you wish to use Selenium framework (see notes on Automated browsing below):

$ sudo pip install selenium

Usage

The most typical usage work flow is to run Reflash as a proxy for remote machine, typically sitting inside a VM on same host machine. If this is your plan, please make sure you have the target VM guest up and running at this point.

All usage is done from command line, in the directory "framework". So please "cd" back there, in case you ever went away.

All the tools support at least command line option -h or --help for getting help. Frontend scripts run_file.sh and run_live.sh redirect command line options to proxy.py so all these scripts accept the same command line options. Only exception to this rule is that run_file.sh requires at least one argument: the actual SWF to be analyzed.

Before going into typical use cases, let's first finish the installation by setting up the environment.

Setting up the evironment with unittest.swf

unittest.swf (in directory "framework") is a simple SWF file for testing out the installation. After few test rounds, it displays a message Hello Reflash! on browser screen.

Configure you browser settings

Now go to your flash analysis target machine (it can be also on the same machine for testing) and configure the browser proxy settings to match the following:

• IP address or name: where your proxy runs
• Port: by default, mitmproxy uses 8080

The browser also needs to have flash player installed (version 9+). If you want to test on a local machine, make sure to use Chrome (flash player included by default) or install flashplayer-plugin package for firefox.

Now go back to your proxy machine, again to directory "framework".

Test unittest.swf with run_file.sh

Please make sure that the interface variables in frontend scripts run_file.sh and run_live.sh match to your setup:

interface='eth0'

IP address in above interface should match to the proxy server name/address configured to the target analysis machine proxy settings.

Execute the following command:

$ ./run_file.sh unittest.swf

If everything is set up correctly, you should see something like this in the console:

  dump dir: /tmp/tmp.c8JRweb6ED, database: unittest.swf.db, landing page: index.html
  Proxy started.
  Hit ENTER to abort.

Next:

• Open browser on your target machine and type in any url followed by index.html for example: http://example.com/index.html
• Expect to see something along these lines in the console:

  << Landing page request: http://example.com/index.html
  << Payload request: http://example.com/unittest.swf
  << Reflash request: http://example.com/dHsdKJab/loadBytes
     Content saved as /tmp/tmp.c8JRweb6ED/13b00a897d2314552f7816376535045f.swf
  << Reflash request: http://example.com/dHsdKJab/loadBytes
     Content saved as /tmp/tmp.c8JRweb6ED/0a7fb7924a02518a6d320ca6d62ac1c2.swf
  << Reflash request
     Content saved as /tmp/tmp.c8JRweb6ED/855e7a3e2fb1d05b02218b34d50de0d3.swf
  << Flash trace data

• After you see "Flash trace data" message, hit ENTER.
• Run replay and open the database unittest.swf.db.
• Mess around with the database.
• Note: by default, run_file.sh sets proxy timeout for 120 seconds.

Install mitmproxy CA cert (optional)

First run of the proxy produces a CA certificate directory, by default, "cadir" under "framework". For smooth mitmproxy experience, copy the file mitmproxy-ca-cert.cer from directory "cadir" to your target machine and install the CA certificate (example for IE11 on Windows 7):

• Right-click mitmproxy-ca-cert.cer
• "Install cerificate"
• Choose "Place all cerificates in the following store"
• Choose "Trusted Root Certification Authorities"
• Next, finish, Yes

Example for firefox:

• Open Preferences->Advanced->Certificates
• Click "View Certificates"
• Click "Import"
• Select mitmproxy-ca-cert.cer
• Choose "Trust this CA to identify websites"
• Press OK

CA cert installation is not absolutely needed for Reflash, but for live sessions it is good to have.

Test live connection with run_file.sh

From command line, run:

$ ./run_live.sh

Now go back to your target browser and type in URL http://www.adobe.com/software/flash/about/ (or some other site you know hosting some simple flash files). You should see few files coming in to your console. After "Flash trace data" message, press ENTER and open the file live.db (default file se by run_live.sh) with replay:

$ ./replay -f live.db

Console messages

Proxy started

Proxy is running.

Landing page request

Proxy is serving the supplied landing page in "file" mode.

Payload request

Proxy is serving the supplied payload in "file" mode.

Flash content detected

Proxy detected SWF in "live" mode

Reflash request

Embedded SWF is sent back to proxy for instrumentation.

Flash trace data

Trace data received. This is the most important console message to monitor (if you want to abort the trace before timeout) because if it is missing, there is no data put to SQL database.

If there is no data, message "Nothing was logged." is written to console after trace termination.

Analyzing standalone files

Standalone files are simple to analyze because there are no complicating factors, like javascript interaction with a live server. All the action happens in a closed circuit between Reflash proxy and the browser. This also means it limits the targets down to a simple, standalone files. Most modern Exploit Kits require some sort of interaction from a live server. Some go even so far that they cannot be correctly replayed using web replay dumps, such as Fiddler.

If your file is suitable for standalone analysis (most likely just a random file from VirusTotal is not), you can try out running it like unittest.swf in the above example. If you suspect the file requires arguments, you can supply them to the example index.html (see FlashVars). You can naturally build whatever supporting interaction is required in the landing page, but that is out of scope of this document.

Analyzing live web traffic

For most cases like live Exploit Kits, live mode is what you want to use. Using Reflash in live mode is simple: run run_live.sh or just leave on "live" option in monitor and make sure the timeout is long enough. When you think you captured all interesting exection (look for "Flash trace data" messages on the console), just press "Stop" (or ENTER when using command line tools).

It should noted that all flash execution is monitored and logged in a single database during a live session. This can be observed nicely by browsing to Adobe flash player test page in live mode. It loads (at least) three simple SWF files.

Advanced topics

YARA

Reflash has integrated support for running YARA in the context of stack values and disassembly text. YARA scan can be activated automatically after the trace using any of the command line tools with command line option --yara or with dbtool.py by directly manipulating ready-made database. See the usage of dbtool.py with --help.

In the GUI tool replay, YARA rules can be loaded with Ctrl+Y (Search->Run yara).

Example rules file misc.yara contains few rules detecting entities such as embedded SWF and some simple Exploit Kit shellcodes.

Upstream proxy

Proxy tools can be instructed to forward all requests to an upstream proxy, for example Fiddler.

• Run Fiddler as upstream proxy for Reflash: start proxy with command line option -U proxy_URL (for example -U http://localhost:9999)
• If you plan to run Fiddler on a same machine, you need to configure Fiddler as a standalone proxy. Open Tools->Fiddler Options->Connections

• Use some other port than 8080 (Reflash proxy default) or 8888 (Reflash log server default), for example 9999
• Uncheck "Act as a system proxy on startup"
• Uncheck "Monitor all connections"
• Check "Allow remote computers to connect"

Automated browsing

There is a rudimentary support for running automated browsing with the proxy, using Selenium. In order to activate that, please first read the Selenium documentation for setting up Selenium hub and node. In short, you need to run hub, most likely in the proxy machine (could be remote as well) and a node in target machine.

If you want to play with this, please use proxy command line options --browse and --hub or JSON settin "hub" in "proxyConfig" (see below). By default, proxy sets up the webdriver connection with Internet Explorer default capabilities. For tweaking the capabilities, you need to edit function run_browser in proxy.py.

Running hub on the local machine is as simple as downloading Selenium standalone and running the jar file:

$ java -jar selenium-server-standalone-3.0.1.jar -role hub

Setting up Selenium node for running Internet Explorer is described in this document.

For even more automated configuration, please see the example script run_vm.sh in directory "framework". It uses VirtualBox for running preconfigured Selenium nodes.

config.json

Some tools also use a JSON configuration file in addition to command line options. This can be supplied with cmdline options -c. JSON file has two sections: "proxyConfig" for proxy tools and "reflashConfig" for reflash.

Supported settings in config.json:

proxyConfig:

• "address" - IP address of the proxy
• "logdir" - directory for storing logs and other temporary data
• "logport" - log server port
• "proxyport" - proxy server port
• "cadir" - CA cert directory
• "dumpdir" - directory for storing dumps
• "upstream" - upstream proxy URL, for example Fiddler
• "landing_page" - landing page file
• "timeout" - timeout in seconds
• "tag" - loadBytes tag used by Instrument
• "namespace" - javascript namespace used by Instrument
• "package" - AS3 package name used by Instrument
• "noInstrument" - array of MD5 hashes for files not to be instrumented
• "version" - Fake flash player version (see notes below on faking flash version)
• "player" - Fake flash player type (see notes below on faking flash version)
• "os" - Fake flash player OS type (see notes below on faking flash version)
• "flash_in" - Flash instrument template file
• "flash_out" - Generated flash instrument file name
• "hub" - Selenium hub URL (something like http://localhost:4444/wd/hub)

reflashConfig:

• "input" - input file
• "output" - output file
• "work_dir" - temporary working directory (relative for 'i'-mode, absolute for 'd' and 'a')
• "inject" - instrument to inject
• "inject_pkg" - instrument package name
• "stream" - store stream disassembly to file
• "id" - session identifier in stream disassembly
• "quiet" - be quiet
• "dbg" - produce debug output, keep temp files etc.
• "opcodeHooks" - array of opcode hooks (please read the reflash documentation)

Typically, you don't need to touch config.json because proxy automates for example template flash file generation. But you could for example set up a series of configuration files for different fake flash player versions etc.

Faking flash version

Flash version information faking is a powerful feature implemented with the reflash post-opcode hooking mechanism. It is very useful for automated triggering of flash exploits that depend on specific flash version.

This feature can be configured with proxy command line options --version, --os and --player, or alternatively with config.json settings "version", "os" and "player". There is some basic checking on the format of these setting:

• Version string must be an OS identification string and comma separated list of numbers indicating Major and Minor versions, according to this document For example: "WIN 21,0,0,192".
• OS type string typically consist of two strings, OS and version, example: "Windows XP". Please read this document for correct version format.
• Player type string must be one of these, for example "ActiveX".

Hacking / debugging tips

If you bump into application that doesn't like to be reflashed, it is a good practice to run it under debug flash player and use something like Vizzy for logging flash debug messages.

Each instrument hook prints out a trace message like this:

  InstrumentStack: callproperty:0-1:195:249

In the event of crash, the debug flash player most likely prints out an error message before stopping. Usually it is possible to get an idea of the bug by following the InstrumentStack trace messages right before the crash. You can then manually instrument the problematic file and disassemble it:

$ reflash i --input problem.swf --config config.json
$ reflash d --input problem.swf.reflash --dir <disasm_dir>

Then try to "grep" the hook identifier trace message "callproperty..." for locating the exact place where the trace message is coming from and work your way from there.

It might also be a good approach to start debugging problems by limiting Reflash features. For example, for making sure the proxy framework works in general, try first running the session with "noflash" flag (-x in command line tools). Next level of intrusiveness could be running reflash with non-matching/bogus "opcodeHooks" in config.json. Next, start adding hooks one by one, first "method_entry", followed by "callpropvoid" etc.

If you think the logging is the problem, create Instrument.swf without logging (recompile -T).

Note that most Exploit Kit flash files refuse to run under debug flash player.

Good luck!

Licence

GPL v3 or later.