Custom Profile not Recognised #521
Comments
|
Hiya, could you please check that the symbol file you made is listed in the output of the |
|
The symbol file I made does not show in the output for isfinfo |
|
If the symbol file isn't even listed in the output for |
|
The JSON file contains a banner: |
|
And this is the output from using -vvvv flag: |
|
Hmmm, no issues with the linux caches, and the constant_data looks to be present. It's very strange that
We need to figure out why it's not got that file listed, and isn't classing it as a file it can't process in the verbose logging. 5:S |
|
Hmmmm, so it looks like it was finding the mac.zip file in but the isfinfo output was giving a lot more files/directories than just those. I don't know whether sansforensics alters the installation in any way, but it might be worth checking with a copy from git? |
|
I don't think sansforensics does alter anything because I have pulled the latest version of vol3 from GitHub already - vol3 does not come preinstalled. I have tried the same steps as above on a clean Ubuntu 18.04 host and I am still getting errors. Perhaps I have created the profile incorrectly? |
|
Then I'm really not sure what's going on? It's clearly crawling all the JSON files it can find (and under a bunch of directories I wouldn't have expected it to)? The output.txt didn't have the logging output (which the The symbol files need to follow the JSON schema specified by their version type in the metadata. I don't have one matching your kernel exactly, but here's an older one that should show you what it should look like. Hopefully you can compare the contents and check that the file is right (my guess is the file just isn't even being looked at for some reasons, but it's not yet clear why)... |
|
Thank you - this is the JSON I have created - please let me know of any errors |
|
That's very interesting. The schema validator says the contents of the file is fine, but dumping it in the right place and running |
|
Ok, that was my bad, it wasn't showing up because the file wasn't actually there. 5:P When it's there I get the following: So it looks like it should be fine, and again, it suggests the symbol path isn't being read correctly, or it doesn't have permission to read that file for some reason? |
|
Could you please suggest a solution because I don't know what more information I can provide. Verifying permissions Result of running isfinfo (removed several lines for sake of brevity) Result of running Plugin |
|
Ok, so This is the banner you originally found using the None of the JSON files you have, have matching banners, so volatility will not class them as matched? The closest you've got is the following which suggests (due to the diference in the gcc version) that it was a debug kernel from a different version of ubuntu (and compiled about 7 hours earlier). We take a very strict policy on matching kernel versions, because we aim for accuracy. Since every kernel can be compiled with options that may affect the symbols, if we accepted "close" matches we'd potentially have different kernel structures and we'd then be answering bugs about those all the time. Since we'll be answering bugs one way or another, we decided to require an exact match between the banners to provide the highest probability of success and accuracy. So far, volatility has been behaving exactly as expected. I'm happy to leave this bug open for a little bit, but the only advice I can offer at this point is to find the exact debug kernel for the distribution and version of ubuntu that you're using. The |
|
I understand the explanation but I am still not convinced volatility3 is working as expected. I then searched for the relevant debug symbol here: http://ddebs.ubuntu.com/ubuntu/pool/main/l/linux/ and pulled down linux-image-4.15.0-112-generic-dbgsym_4.15.0-112.113_arm64.ddeb Installed, compiled and ran dwarf2json to create the debug symbol: Target HostAnd I have rerun the correct commands with the x86_64 symbol table and I still get an error - have reattached the correct symbol table also |
|
In this particular instance you pulled down the symbols for the ARM architecture, which will be different to those for x64: and the output from More generally though, the gcc compiler may change between the same version of the package compiled for 16.04 and 18.04 (for example). If Continuing to leave this open for a few days, but I'll close this off after that unless an actual bug is identified. |
|
Updated my previous comment. Please review as I am still getting an error. |
|
So, there's a whole bunch of version numbers flying around here, the kernel version (4.15.0) the ubuntu package version (112) and then the version of the compiler that was used to compile it (gcc version 5.4.0) as well as the ubuntu version (Ubuntu 5.4.0-6ubuntu1~16.04.12), all of which need to match exactly. The linux kernel is extremely configurable, and the same version can be configured with support for many different options, some of which alter the structures that the kernel uses. Since we can't guarantee that two kernels of the same kernel version number were compiled with the same configuration options, we require the entire banner string to match. The banner string you need to match is one of the ones found by the It is mismatching because the kernel with debugging symbols that you're generating the ISF file from is not the same as the running kernel you're trying to analyze. The entire banner has to match and at the moment you haven't produced an ISF file that does so. There is work being done on a mechanism to allow a file to be compiled on a system to produce most of the common symbols required for volatility, but this presents its own issues (such JSON files won't have all the symbols that the kernel should contain, which means future plugins wouldn't be able to rely on those symbol names). You can read more over at volatilityfoundation/dwarf2json#12. Please note that volatility is simply a data analysis tool, often used for forensics purposes. For this reason, we've made a conscious decision in volatility 3 to aim for accuracy over simply returning results and letting the user figure out how valid they are. This means that if you don't provide volatility the correct symbol information that it requires, it will fail and any results you get may be questionable. That is not a bug, that's the intended behaviour. In no way is the fact that there isn't a JSON file matching the kernel you're trying to analyze, a bug in volatility. |
|
I totally understand that but how can I ensure the exact kernel version I obtain from http://ddebs.ubuntu.com/ubuntu/pool/main/l/linux/ matches the kernel of the system I am analysing? |
|
How distributions distribute their debug kernel versions isn't in our control, but for most systems, you can check which package the package manager would install and where it would download it from. I'd imagine it would be listed under the specific operating system's package list (something like http://ddebs.ubuntu.com/ubuntu/dists/xenial-updates/), but it's also possible that they don't keep all old kernels and you won't be able to get the package. I'm sorry, it's not always possible to get everything easily. I'm going to close this bug now because it's very clearly not a volatility issue. Good luck with your quest to find the symbols, if you need further help I suggest asking on the slack channel... |
Describe the bug
I have created my own symbol table for Ubuntu 18.04 (5.4.0-42-generic) and imported it using the commands listed below but I get the following error in volatility3 (screenshot also attached):
Context
Volatility Version: Volatility 3 Framework 1.0.1
Operating System: Ubuntu 20.04 (SIFT Workstation)
Python Version: 3.8.5
Suspected Operating System: Ubuntu 18.04
Command: sudo /usr/local/lib/python3.8/dist-packages/volatility3/vol.py -f /Ubuntu18.04.mem linux.pslist.PsList
To Reproduce
Steps to reproduce the behavior:
Expected behavior
I expect volatility3 to be able to read the custom symbol table I have created and as a result read the memory image I have captured
The text was updated successfully, but these errors were encountered: