My First Volatility Plugin with Unified Output

MyFirstVolatilityPluginWithUnifiedOutput.md

My First Volatility Plugin with Unified Output

Introduction

Although there are many excellent resources for learning Volatility available (The Art of Memory Forensics book, the vol-users mailing list, the Volatility Labs blog, and the Memory Analysis training course to name a few), I've never really seen a good absolute beginners guide to writing your first plugin. So if you find yourself needing that, hopefully this will help.

Also, it's worth checking out @jameshabben's post on the topic.

The Aim

Write a plugin that lists the running processes from a Win7SP1x64 sample. (Yes, you can already do this using the pslist plugin, but we've got to start somewhere.)

The Objectives

1. Download the latest version of Volatility from GitHub.
2. Write a new plugin called 'MyPlugin'.
3. Successfully execute 'MyPlugin'.

Btw, I'll be doing all this on my Linux machine, running Ubuntu 16.04.

Off we go..!

1. Download Volatility

$ git clone https://github.com/volatilityfoundation/volatility

Ok, that was easy. You should now have a volatility directory.

2. Working Directory and Plugin File

It's a good idea to have a directory to contain the files for your plugin, so let's create one within our home directory:

$ cd ~
$ mkdir myplugin
$ cd myplugin

Now we need to create the python file which will be our plugin:

$ touch myplugin.py

3. The Absolute Minimum

When learning new languages and new frameworks I always think its useful to know what's the absolute least I can write and still have something work. That way you have a minimum to build on. So, the following is what I understand to be the least code you can write and have a plugin that can be called and not cause any errors.

import volatility.plugins.common as common
class MyPlugin(common.AbstractWindowsCommand):
  def render_text(self, outfd, data):
    print "Hello world!"

Let's make sure it runs:

$ python vol.py --plugins="/home/btg/myplugin/" -f /path/to/memory.img myplugin
Volatility Foundation Volatility Framework 2.5
Hello world!

Let me explain:

• python vol.py
  • Run Volatility
• --plugins="/home/btg/myplugin/"
  • Search this folder for plugins
• -f /path/to/memory.img
  • The memory image to analyse
• myplugin
  • The plugin we want to run.

Some Gotchas

Two things that always catch people out:

i) --plugins must come first.

Owing to the way the Volatility framework is set up, if you're using the --plugins parameter it must be the first parameter you specify, so immediately after python vol.py.

ii) The path used with --plugins must be absolute.

In the example above I used /home/btg/myplugin/, if I try ~/myplugin/ it doesn't work. I haven't bothered to investigate whether this is a Volatility thing or a shell thing or a python thing.

Let's Explain the Code

import volatility.plugins.common as common

Import the common library which is part of the Volatility Framework.

class MyPlugin(common.AbstractWindowsCommand):

Each plugin is a python class. The name of the class becomes the name of the plugin. This class subclasses the AbstractWindowsCommand class, which is in the common library.

Note: It is the inheriting of one of the Command classes that actually makes it a plugin. If the plugin didn't inherit one of the Command classes Volatility wouldn't recognise it as a plugin when it scanned the directory.

def render_text(self, outfd, data):

Volatility will look for a method named render_X where X matches the value of the --output parameter. The default for --output is text, hence because we didn't specify anything else, Volatility looks for render_text. The parameters:

• self
  • In python, self refers to the instance of the class in question. It is always the first parameter of a method that belongs to a class.
• outfd
  • I assume this stands for output file descriptor. This is the output stream to which Volatility will write. This is defined by the --output-file parameter. The default for --output-file is stdout.
• data
  • This contains the data returned by the class's calculate method. If our plugin doesn't define it's own calculate method, it'll be inherited from the superclass (in our example, AbstractWindowsCommand).

print "Hello world!"

This of course prints the string "Hello world!" to stdout. Now, if you've been paying attention you'll realise we shouldn't really do this. If the user has specified a value for --output-file, this line wouldn't write to their value, it'd always write to stdout. We'll fix that shortly.

4. A Better Minimum

Although our absolute minimum above works, let's make a couple of improvements:

import volatility.plugins.common as common

class MyPlugin(common.AbstractWindowsCommand):
  """My First Volatility Plugin"""

  def render_text(self, outfd, data):
  
    outfd.write("Hello world!\n")

i) """My First Volatility Plugin"""

This docstring is what Volatility uses to describe the plugin when it lists the plugins with the --help parameter. For example:

$ python vol.py --plugins="/home/btg/myplugin/" --help
Volatility Foundation Volatility Framework 2.5
Usage: Volatility - A memory forensics analysis platform.

Options:
  -h, --help            list all available options and their default values.
--SNIP--
  mutantscan     	Pool scanner for mutex objects
  myplugin       	My First Volatility Plugin
  notepad        	List currently displayed notepad text
--SNIP--

ii) outfd.write("Hello world!\n")

We've replaced print with outfd.write(...). This means our plugin will now respect the user's --output-file value. write doesn't automatically append a linebreak, so if we want one, we have to add it ourselves: \n

5. Get the Processes

Ok, so we have the very basic framework of a plugin. Let's do something useful: a list of running processes. (As already mentioned, we are essentially doing the same as the pslist plugin, the code for which you can find in taskmods.py.)

As briefly mentioned above, the render_text method has a data parameter which is popualted by the calculate method. So let's write a calculate method to get the tasks (processes):

def calculate(self):

  addr_space = utils.load_as(self._config)
  tasks = win32.tasks.pslist(addr_space)
  
  return tasks

Let me explain:

• def calculate(self):
  • Declare the calculate method. Has the self parameter so that it becomes a class method.
• addr_space = utils.load_as(self._config)
  • Declare a variable named addr_space which contains the address space identified by self._config. self._config is an object which identifies various things about this run of Volatility. The address space would be generated from the file you passed via -f.
• tasks = win32.tasks.pslist(addr_space)
  • Declare a variable named tasks which contains a list (technically a generator) of process objects parsed from the address space.
• return tasks
  • Return the generator of process objects to the render_text method (becomes data).

Note: You will need to import the relevant libraries for utils and win32:

import volatility.utils as utils
import volatility.win32 as win32

6. Print the Tasks

Now that our data parameter has been populated in the render_text method, we can do something with it. Let's look at a full plugin with render_text updated:

import volatility.plugins.common as common
import volatility.utils as utils
import volatility.win32 as win32

class MyPlugin(common.AbstractWindowsCommand):
  """My First Volatility Plugin"""

  def calculate(self):
      
    addr_space = utils.load_as(self._config)
    tasks = win32.tasks.pslist(addr_space)
      
    return tasks
      
  def render_text(self, outfd, data):
  
    for task in data:
      outfd.write("{0}\n".format(task))

If you run this plugin you'll get something like this:

$ python vol.py --plugins="/home/btg/myplugin/" -f /path/to/memory.img --profile Win7SP1x64 myplugin
Volatility Foundation Volatility Framework 2.5
18446738026421482992
18446738026436541232
18446738026449392304
--SNIP--

So what's happening? render_text is iterating each of the tasks and printing it to stdout. The value you are seeing is the virtual offset in memory of the _EPROCESS structure.

If we look at the definition for _EPROCESS (in Win7SP1x64) we can see there are any number of properties we might want, for example:

• UniqueProcessId
• CreateTime
• ImageFileName

So let's make our plugin a bit better and include some of these values:

def render_text(self, outfd, data):
  
  for task in data:
    outfd.write("{0} {1} {2}\n".format(
      task.UniqueProcessId,
      task.CreateTime,
      task.ImageFileName)
    )

And run it:

$ python vol.py --plugins="/home/btg/myplugin/" -f /path/to/memory.img --profile Win7SP1x64 myplugin
Volatility Foundation Volatility Framework 2.5
4 2016-07-13 17:19:16 UTC+0000 System
280 2016-07-13 17:19:16 UTC+0000 smss.exe
364 2016-07-13 17:19:16 UTC+0000 csrss.exe
404 2016-07-13 17:19:16 UTC+0000 wininit.exe
416 2016-07-13 17:19:16 UTC+0000 csrss.exe

7. Moving to Unified Output

So the above is all well and good, but it's so pre Volatility 2.5. In Volatility 2.5, the capability for unified output was introduced. The reason is simple: a user of a plugin may want the output in various formats, for example, text, csv, json or SQLite. But to write a render_X method for each possible output becomes annoying for the plugin developer. The answer: a unified_output method.

The unified_output method must return a TreeGrid object.

The TreeGrid object requires two parameters:

1. A list of tuples each of which defines a title and a type for the value you are returning.
2. A method call which will populate the data.

For example:

return TreeGrid([
  ("PID", int),
  ("Created", str),
  ("Image", str)],
  self.generator(data)
)

It's easiest to think of a TreeGrid as a table. The list of tuples defines the columns of the table:

1. ("PID", int) - this column will have a title of "PID" and the values in the column will be of type int.
2. ("Created", str) - this column will have a title of "Created" and the values in the column will be of type str.
3. ("Image", str) - this column will have a title of "Image" and the values in the column will be of type str.

The method then needs to return the same number of objects of corresponding types. In our example, an int followed by two strings. For example:

def generator(self, data):

  for task in data:
    yield (0, [
      int(task.UniqueProcessId),
      str(task.CreateTime),
      str(task.ImageFileName)
      ])

Note: Volatility contains some extra types as defined in volatility.renderers.basic.

So in a finihed plugin:

• unified_output gets its data by calling self.generator(data).
• The data operated on by self.generator(data) is provided by calculate.

8. The Finished Plugin

So, yes, it's far from elegant, but that's not the point: it works. And hopefully it's given you a bit of an understanding as to how a plugin works.

import volatility.plugins.common as common
import volatility.utils as utils
import volatility.win32 as win32

from volatility.renderers import TreeGrid
from volatility.renderers.basic import Address, Hex

class MyPlugin(common.AbstractWindowsCommand):
	"""My First Volatility Plugin"""

	def calculate(self):

		addr_space = utils.load_as(self._config)
		tasks = win32.tasks.pslist(addr_space)

		return tasks

	def generator(self, data):

		for task in data:
			yield (0, [
				int(task.UniqueProcessId),
				str(task.CreateTime),
				str(task.ImageFileName)
			])

	def unified_output(self, data):

		return TreeGrid([
			("PID", int),
			("Created", str),
			("Image", str)],
			self.generator(data))

<3