add VMI AddressSpace

[Wenzel]

Hi !
This PR aims to bring the support of a new address space based on virtual machine introspection.

ping @asabellico since she looked at the issue I opened a few weeks ago.

It uses libvmi to access the physical memory of a guest via the hypervisor's API.

The supported hypervisors at the moment are Xen and KVM.

Note: I would need more information about the read and zread methods.
Can they fail ? Because I'm raising a RuntimeError from read, and having discussed with Michael Cohen on the Rekall integration, the read wasn't supposed to fail, but fill the buffer with zeroes instead.

Usage:

./vol.py -l vmi:///nitro_win7x64 --profile=Win7SP0x64 pslist

Libvmi will attempt to detect the hypervisor, but you can also specify it in the URL:

./vol.py -l vmi://kvm/nitro_win7x64 --profile=Win7SP0x64 pslist
./vol.py -l vmi://xen/nitro_win7x64 --profile=Win7SP0x64 pslist

Example run:

$ ./vol.py -l vmi://kvm/nitro_win7x64 --profile=Win7SP0x64 pslist
Volatility Foundation Volatility Framework 2.6
LibVMI Version 0.11.0
LibVMI Driver Mode 1
--completed driver init.
--got id from name (nitro_win7x64 --> 2)
**set image_type = nitro_win7x64
--libvirt version 1003001
--qmp: virsh -c qemu:///system qemu-monitor-command nitro_win7x64 '{"execute": "pmemaccess", "arguments": {"path": "/tmp/vmictdOHL"}}'
--kvm: using custom patch for fast memory access
**set allocated_ram_size = 59700000, max_physical_address = 0x59700000
--qmp: virsh -c qemu:///system qemu-monitor-command nitro_win7x64 '{"execute": "human-monitor-command", "arguments": {"command-line": "info registers"}}'
Offset(V)          Name                    PID   PPID   Thds     Hnds   Sess  Wow64 Start                          Exit                          
------------------ -------------------- ------ ------ ------ -------- ------ ------ ------------------------------ ------------------------------
0xfffffa8001108890 System                    4      0     76      452 ------      0 2018-03-19 20:10:34 UTC+0000                                 
0xfffffa8002329950 smss.exe                248      4      2       29 ------      0 2018-03-19 20:10:34 UTC+0000                                 
0xfffffa8002963060 csrss.exe               316    308      8      285      0      0 2018-03-19 20:10:36 UTC+0000                                 
0xfffffa8002a4b7c0 wininit.exe             364    308      3       74      0      0 2018-03-19 20:10:36 UTC+0000                                 
0xfffffa80011837c0 csrss.exe               376    356      7      134      1      0 2018-03-19 20:10:36 UTC+0000                                 
0xfffffa8002ba0280 winlogon.exe            416    356      3      108      1      0 2018-03-19 20:10:36 UTC+0000                                 
0xfffffa8002bcc320 services.exe            460    364      5      178      0      0 2018-03-19 20:10:37 UTC+0000                                 
0xfffffa8002bde450 lsass.exe               468    364      6      508      0      0 2018-03-19 20:10:37 UTC+0000                                 
0xfffffa8002bd9830 lsm.exe                 476    364      9      135      0      0 2018-03-19 20:10:37 UTC+0000                                 
0xfffffa8002be3b30 svchost.exe             580    460      9      334      0      0 2018-03-19 20:10:37 UTC+0000                                 
0xfffffa8002984780 svchost.exe             652    460      5      201      0      0 2018-03-19 20:10:37 UTC+0000                                 
0xfffffa8002c78700 svchost.exe             696    460     18      428      0      0 2018-03-19 20:10:37 UTC+0000                                 
0xfffffa8002cc89e0 svchost.exe             804    460     15      400      0      0 2018-03-19 20:10:37 UTC+0000                                 
0xfffffa8002ceab30 svchost.exe             844    460     23      830      0      0 2018-03-19 20:10:37 UTC+0000                                 
0xfffffa8002d40290 svchost.exe             996    460     12      301      0      0 2018-03-19 20:10:38 UTC+0000                                 
0xfffffa8002d78b30 svchost.exe             524    460     15      397      0      0 2018-03-19 20:10:38 UTC+0000                                 
0xfffffa8002dcc200 dwm.exe                1076    804      3       68      1      0 2018-03-19 20:10:38 UTC+0000                                 
0xfffffa8002dd95a0 explorer.exe           1088   1068     19      675      1      0 2018-03-19 20:10:38 UTC+0000                                 
0xfffffa8002e0d420 spoolsv.exe            1164    460     12      260      0      0 2018-03-19 20:10:39 UTC+0000                                 
0xfffffa8002e278e0 taskhost.exe           1196    460      7      142      1      0 2018-03-19 20:10:39 UTC+0000                                 
0xfffffa8002e33b30 svchost.exe            1220    460     19      305      0      0 2018-03-19 20:10:39 UTC+0000                                 
0xfffffa8002eb9060 svchost.exe            1380    460      8      154      0      0 2018-03-19 20:10:39 UTC+0000                                 
0xfffffa8002effb30 sppsvc.exe             1476    460      4      146      0      0 2018-03-19 20:10:39 UTC+0000                                 
0xfffffa8002fe7b30 wlms.exe               1752    460      4       41      0      0 2018-03-19 20:10:41 UTC+0000                                 
0xfffffa8002ffe170 svchost.exe            1828    460      5       89      0      0 2018-03-19 20:10:41 UTC+0000                                 
0xfffffa8002ec73f0 svchost.exe             292    460     13      336      0      0 2018-03-19 20:12:41 UTC+0000  

Please give some feedback !

Thanks.

[asabellico]

Hello @Wenzel ,
trying to test it both ways:

1. vol.py -l vmi://xen/test1 --profile=Win7SP0x86 pslist
2. vol.py -l vmi://test1 --profile=Win7SP0x86 pslist

In case 1) (ran as root) it says:
VMI_ERROR: Could not find a live guest VM or file to use.
VMI_ERROR: Opening a live guest VM requires root access.

i think you are not handling the "xen/" prefix in domain name, isn't it?

In case 2) I get this error:
File "/usr/local/lib/python2.7/dist-packages/volatility-2.5-py2.7.egg/volatility/addrspace.py", line 283, in zread
return self._read(addr, length, True)
File "/usr/local/lib/python2.7/dist-packages/volatility-2.5-py2.7.egg/volatility/addrspace.py", line 258, in _read
data = read(paddr, datalen)
File "/usr/local/lib/python2.7/dist-packages/volatility-2.5-py2.7.egg/volatility/plugins/addrspaces/vmi.py", line 69, in zread
buffer, bytes_read = self.vmi.read_pa(addr, length)
File "/usr/local/lib/python2.7/dist-packages/libvmi-3.0-py2.7-linux-x86_64.egg/libvmi/libvmi.py", line 390, in read_pa
check(status)
File "/usr/local/lib/python2.7/dist-packages/libvmi-3.0-py2.7-linux-x86_64.egg/libvmi/libvmi.py", line 118, in check
raise LibvmiError(error)
libvmi.libvmi.LibvmiError: VMI_FAILURE

i'm testing it with Volatility 2.5. Should I try with 2.6?

Thank you

[Wenzel]

@asabellico thanks for trying the address space !

for the case 1)
try to enable some debug output on libvmi;
edit libvmi/debug.h and recompile/install libvmi.

Xen should be handled properly.
i haven't tested this address space on Xen though, but i did with Rekall.

Note that in the case 2) you are missing a /:
it should be vol.py -l vmi:///test1 --profile=Win7SP0x86 pslist

I'm using urllib.parse.urlparse to parse the url.
The netloc part is the hypervisor, if specified.
The path part is the domain; that's why it should start with /domain.

i'm testing it with Volatility 2.5. Should I try with 2.6?

No it doesn't matter.

[asabellico]

Maybe I'm missing something as neither vmi:///[dom name] nor vmi://xen/[dom name] is working for me.

Using this location: vmi://[dom name] at least it finds the domain and then fails with:
[omissis]
--MEMORY cache hit 0xe4f00000
--MEMORY cache hit 0x185000
--MEMORY cache hit 0x188000
--MEMORY cache hit 0xe5006000
--MEMORY cache hit 0xe4f00000
--MEMORY cache hit 0x185000
--MEMORY cache hit 0x188000
--MEMORY cache hit 0xe5006000
--MEMORY cache set 0xa0000
--xen_get_memory_pfn failed on pfn=0xa0

(I enabled the debug for libvmi)

[Wenzel]

You are hitting this issue i reported on libvmi when i was testing Rekall on Xen:
libvmi/libvmi#579

I "fixed" it by calling a new API called read_pa_padded, which cannot fail, and will just pad the rest of the buffer with zeros if the call to read the physical memory failed to read all the bytes requested.

I already implemented this call for zread(); but what about read() ?
I need more information here. (it's what i asked at the beginning)

Are these methods supposed to never fail ?
Look at the comment @scudette gave to me when i was implementing the address space for Rekall:

A read() method raises RuntimeError() which is strange since read() should always succeed (it may return null padding). The read() method is always expected to return the number of bytes requested. This is analogous to real machine memory - you can do a memcpy from a physical memory range which does not exist and you would just get zeros (or a BSOD but we assume this is handled :-).

[asabellico]

Well, actually the fail happen on a zread(). Tomorrow i'll investigate more on the topic

Thank you!

[Wenzel]

Ok,
i'm wondering why your zread uses read_pa.
in the code that i pushed: https://github.com/volatilityfoundation/volatility/pull/500/files#diff-27e6b7a9f0c1c6ac8f1e83ddecb6b330R86

[asabellico]

you are right! i was using another vmi.py version! now it works

[Wenzel]

@asabellico good news :)

What about my questions about the implementation details ?

[asabellico]

Sorry @Wenzel i can't help with your question..

[Wenzel]

Maybe @asabellico do you know any Volatility core developers that could help me ?

[Wenzel]

Ping,
is there a core developer to review this PR ?

[Wenzel]

Ping ?

[mintancy]

Hi, did anyone meet the same error with me like this? #libvmi/python#15

# python vol.py -l vmi://winxpsp3x86 --profile=WinXPSP2x86 psxview
AttributeError: 'PyVmiAddressSpace' object has no attribute 'translate'

I checked the code in psxview plugin(volatility/plugins/malware/psxview.py) and found that there is a function call which translates the memory's virtual address into a physical address. This function is provided by address space plugin.
see volatility/plugins/addresspace/lime.py

    def translate(self, addr):
        """Find the offset in the file where a memory address can be found.
        @param addr: a memory address
        """
        firstram = self.runs[0][0]

        if addr < firstram:
            addr = firstram + addr

        return addrspace.AbstractRunBasedMemory.translate(self, addr)

However, this "translate" function is not provided in vmi.py, so I met this error.
But the translation function in libvmi is availible, so I tried to add translate_kv2p(addr) in vmi.py,

    Def translate(self, addr):
        Return self.vmi.translate_kv2p(addr)

then I got results:

Offset(P)  Name                    PID pslist psscan thrdproc pspcid csrss session deskthrd ExitTime
---------- -------------------- ------ ------ ------ -------- ------ ----- ------- -------- --------
0x065e3da0 HelpSvc.exe            1580 True   True   True     True   True  True    True     
0x06589da0 lsass.exe               556 True   True   True     True   True  True    True     
0x0658d988 svchost.exe             876 True   True   True     True   True  True    True     
0x065c0da0 svchost.exe             924 True   True   True     True   True  True    True     
0x0664c558 wpabaln.exe            1788 True   True   True     True   True  True    True     
0x0660f390 msmsgs.exe             1472 True   True   True     True   True  True    True     
0x06625c10 ctfmon.exe             1444 True   True   True     True   True  True    True     
0x064c5020 spoolsv.exe            1252 True   True   True     True   True  True    True     
0x06621768 services.exe            544 True   True   True     True   True  True    True     
0x06626da0 (?
             ???w.exe            248 True   True   True     True   True  True    True     
0x065fe3c0 wscntfy.exe             276 True   True   True     True   True  True    True     
0x066e93d0 svchost.exe             712 True   True   True     True   True  True    True     
0x066337e8 wmiprvse.exe           1032 True   True   True     True   True  True    True     
0x06624180 svchost.exe             768 True   True   True     True   True  True    True     
0x064e2da0 svchost.exe             828 True   True   True     True   True  True    True     
0x064df9f8 alg.exe                 976 True   True   True     True   True  True    True     
0x066383c0 wuauclt.exe            1360 True   True   True     True   True  True    True     
0x065b0898 winlogon.exe            500 True   True   True     True   True  True    True     
0x064be958 explorer.exe           1300 True   True   True     True   True  True    True     
0x066234d8 logon.scr              1228 True   True   True     True   True  True    True     
0x06618928 taskmgr.exe             896 True   True   True     True   True  True    True     
0x067b7a00 System                    4 True   True   True     True   False False   False    
0x06577128 smss.exe                308 True   True   True     True   False False   False    
0x065b9020 csrss.exe               476 True   True   True     True   False True    True     

But I'm not sure is it perfect.
Are these the correct results?

[aoshiken]

Any news about?