Showing posts with label Maltego. Show all posts
Showing posts with label Maltego. Show all posts

Monday, January 01, 2018

toolsmith #130 - OSINT with Buscador

First off, Happy New Year! I hope you have a productive and successful 2018. I thought I'd kick off the new year with another exploration of OSINT. In addition to my work as an information security leader and practitioner at Microsoft, I am privileged to serve in Washington's military as a J-2 which means I'm part of the intelligence directorate of a joint staff. Intelligence duties in a guard unit context are commonly focused on situational awareness for mission readiness. Additionally, in my unit we combine part of J-6 (command, control, communications, and computer systems directorate of a joint staff) with J-2, making Cyber Network Operations a J-2/6 function. Open source intelligence (OSINT) gathering is quite useful in developing indicators specific to adversaries as well as identifying targets of opportunity for red team and vulnerability assessments. We've discussed numerous OSINT offerings as part of toolsmiths past, there's no better time than our 130th edition to discuss an OSINT platform inclusive of previous topics such as Recon-ng, Spiderfoot, Maltego, and Datasploit. Buscador is just such a platform and comes from genuine OSINT experts Michael Bazzell and David Wescott. Buscador is "a Linux Virtual Machine that is pre-configured for online investigators." Michael is the author of Open Source Intelligence Techniques (5th edition) and Hiding from the Internet (3rd edition). I had a quick conversation with him and learned that they will have a new release in January (1.2), which will address many issues and add new features. Additionally, it will also revamp Firefox since the release of version 57. You can download Buscador as an OVA bundle for a variety of virtualization options, or as a ISO for USB boot devices or host operating systems. I had Buscador 1.1 up and running on Hyper-V in a matter of minutes after pulling the VMDK out of the OVA and converting it with QEMU. Buscador 1.1 includes numerous tools, in addition to the above mentioned standard bearers, you can expect the following and others:
  • Creepy
  • Metagoofil
  • MediaInfo
  • ExifTool
  • EmailHarvester
  • theHarvester
  • Wayback Exporter
  • HTTrack Cloner
  • Web Snapper
  • Knock Pages
  • SubBrute
  • Twitter Exporter
  • Tinfoleak 
  • InstaLooter 
  • BleachBit 
Tools are conveniently offered via the menu bar on the UI's left, or can easily be via Show Applications.
To put Buscador through its paces, using myself as a target of opportunity, I tested a few of the tools I'd not prior utilized. Starting with Creepy, the geolocation OSINT tool, I configured the Twitter plugin, one of the four available (Flickr, Google+, Instagram, Twitter) in Creepy, and searched holisticinfosec, as seen in Figure 1.
Figure 1:  Creepy configuration




The results, as seen in Figure 2, include some good details, but no immediate location data.

Figure 2: Creepy results
Had I configured the other plugins or was even a user of Flickr or Google+, better results would have been likely. I have location turned off for my Tweets, but my profile does profile does include Seattle. Creepy is quite good for assessing targets who utilize social media heavily, but if you wish to dig more deeply into Twitter usage, check out Tinfoleak, which also uses geo information available in Tweets and uploaded images. The report for holisticinfosec is seen in Figure 3.

Figure 3: Tinfoleak
If you're looking for domain enumeration options, you can start with Knock. It's as easy as handing it a domain, I did so with holisticinfosec.org as seen in Figure 4, results are in Figure 5.
Figure 4: Knock run
Figure 5: Knock results
Other classics include HTTrack for web site cloning, and ExifTool for pulling all available metadata from images. HTTrack worked instantly as expected for holisticinfosec.org. I used Instalooter, "a program that can download any picture or video associated from an Instagram profile, without any API access", to grab sample images, then ran pyExifToolGui against them. As a simple experiment, I ran Instalooter against the infosec.memes Instagram account, followed by pyExifToolGui against all the downloaded images, then exported Exif metadata to HTML. If I were analyzing images for associated hashtags the export capability might be useful for an artifacts list.
Finally, one of my absolute favorites is Metagoofil, "an information gathering tool designed for extracting metadata of public documents." I did a quick run against my domain, with the doc retrieval parameter set at 50, then reviewed full.txt results (Figure 6), included in the output directory (home/Metagoofil) along with authors.csv, companies.csv, and modified.csv.

Figure 6: Metagoofil results

Metagoofil is extremely useful for gathering target data, I consider it a red team recon requirement. It's a faster, currently maintained offering that has some shared capabilities with Foca. It should also serve as a reminder just how much information is available in public facing documents, consider stripping the metadata before publishing. 

It's fantastic having all these capabilities ready and functional on one distribution, it keeps the OSINT discipline close at hand for those who need regular performance. I'm really looking forward to the Buscador 1.2 release, and better still, I have it on good authority that there is another book on the horizon from Michael. This is a simple platform with which to explore OSINT, remember to be a good citizen though, there is an awful lot that can be learned via these passive means.
Cheers...until next time.

Saturday, September 10, 2016

Best toolsmith tool of the last ten years

As we celebrate Ten Years of Toolsmith and 120 individual tools covered in detail with the attention they deserve, I thought it'd be revealing to see who comes to the very top of the list for readers/voters.
I've built a poll from the last eight Toolsmith Tools of the Year to help you decide, and it's a hell of a list.
 Amazing, right? The best of the best.

You can vote in the poll to your right, it'll be open for two weeks.

Thursday, February 17, 2011

OSINT: large email address list imports with Maltego



Fans of OSINT are inevitably fans of Maltego; I count myself amongst the dedicated.
Given the recent HBGary debacle, you'll soon see where the following discussion may prove useful for discovery of relationships between entries in a large list of email addresses. Consider the prospect of grepping through the HBGary emails, culling out a list of unique entries, then transforming them as an entirety via Maltego to determine what other relationships may exist between entrants.

I've been hoping for some large list import functionality via Maltego local transforms, and Andrew at Paterva immediately provided upon request.
Imagine similar functionality as found in transforms discussed earlier where a CSV inclusive of IP addresses is imported (this older method was via Phrase entity pointed directly to the full path of the CSV), then unique IP address entities are populated to the Maltego UI workspace.
For our current scenario, Andrew has provided me (and thus you) with local transforms that will allow import of a CSV, now using EasyDialogs (Linux, Windows) inclusive of multiple email addresses, and populate them each as unique Email Address entities to the Maltego UI workspace.

Making sense?
I'll walk you through it.

First, ensure that you grab EasyDialogs as mentioned above and embed it properly with your Python interpreter.
Second, grab getEmailAddresses.py, the above mentioned local transform for email address list imports, and configure it for use with your Maltego instance.

Now, let's start with a googledork.
email addresses filetype:csv senator

The second hit yields a CSV of Virginia state delegates affiliated with the Hampton Roads Partnership.
Looks like fun.
NOTE: This is entirely benign OSINT, simply a good object model for validation of our new local transform.

After cleaning up the CSV to include only a column inclusive of the delegates email addresses, drag a Phrase entity onto the Maltego workspace; I named mine Virgina Delegates.
Right-click the Phrase entity, select Run Transforms, then Other Transforms, then getEmailAddresses. A pop-up window will appear (EasyDialogs) and ask you "Which file do you want to use?"
Give it the path (I used the shell extension CopyPath) to your CSV file and click OK.
Results will be populated as seen in Figure 1.


Figure 1

Highlight all the resulting Email Address entities that are now populated in the Maltego workspace, right-click the selection, choose Run Transforms, then Other Transforms, then To Website [using Search Engine]. It'll take a few minutes to run as there's some crawling to be done.

The Mining View of the results can be a bit of kluge as seen in Figure 2.


Figure 2

This transform is best viewed as Edge Weighted (Figure 3).


Figure 3

See the commonality regardless of view?
Both result sets point out the fact that one of the most significant relationships all these delegates share is...wait for it...the website for the firm owned by the person I can only imagine is...ta-da...their lobbyist!
Politics as usual. ;-)

Enjoy this transform, and stay tuned for more discussion of similar transforms.

Cheers.

del.icio.us | digg | Submit to Slashdot

Please support the Open Security Foundation (OSVDB)

Thursday, April 01, 2010

Malware behavior analysis: studying PCAPs with Maltego local transforms

In recent months I've made regular use of Maltego during security data visualization efforts specific to investigations and analysis.
While Maltego includes numerous highly useful entities and transforms, it does not currently feature the ability to directly manipulate native PCAP files.
This is not entirely uncommon amongst other tools, particularly those specific to visualization; often such tools consume CSV files.

With thanks to Andrew MacPherson of Paterva for creating these for me upon request for recent presentations, I'm pleased to share with you Maltego local transforms that will render CSVs created from PCAP files. Simple, but extremely useful.

I'll take you step by step through the process, starting with creating CSVs from PCAPs.
For those of you already comfortable with PCAP to CSV conversion and/or using local transforms with Maltego, here are the pyCSV transforms:

GetSourceClients.py
GetDestinationClients.py

All others, read on.
Raffael Marty's AfterGlow (version 1.6 just released) includes tcpdump2csv.pl which uses tcpdump/windump to read a PCAP file and parse it into parametrized CSV output.

Windows users, assuming that Perl is installed and all files and scripts reside in the same directory, execute:
windump -vttttnnelr example.pcap | perl tcpdump2csv.pl "sip dip dport" > example.csv.

Linux users:
tcpdump -vttttnnelr example.pcap | tcpdump2csv.pl "sip dip dport" > example.csv

To integrate the pyCSV local transforms with your Maltego instance:
1. Click Tools, then Manage Transforms.
2. Click New Local Transforms.
3. Define the Display name as the name of the local transform. Example: GetSourceClients
4. Each transform must map to an entity. Do so as follows for
each transform as you create it:
getSourceClients to Phrase
getDestinationClients to IP Address
5. Click Next.
6. The Command field should point to Python binary (C:\Python25\python.exe on Windows, /usr/bin/python
on Ubuntu 9.10).
7. The Parameters field should refer only to the transform name. Example: GetSourceClients.py
8. Work Directory should be the complete path to the directory where you keep the Nmap local transform Python scripts (suggest C:\localTransforms\pyCSV for Windows users).
9. Finish, then Save.



You'll also need a copy MaltegoTransform.py in your local transforms directory (included with the Maltego python lib during installation).

During a recent Zeus bot investigation, I used GetSourceClients.py and GetDestinationClients.py as follows:

1) Convert zeus.pcap, captured during malware analysis in a virtual ennvironment,
to zeus.csv: tcpdump -vttttnnelr zeus.pcap | tcpdump2csv.pl "sip dip dport" > zeus.csv

2) Drag a Phrase entity into the Maltego workspace, and using CopyPath, pasted the full path to the zeus.csv into the Phrase entity.

3) Right-clicked the Phrase entity, chose All, then getSourceClients.



4) Right-clicked the IP entity created for my infected host, chose All, then getDestinationClients.



5) Egress traffic to a likely malicious host immediately jumped out of the Maltego workspace at me. I right-clicked (after removing the port reference in the IP entity label) and selected AllTransforms.



6) Maltego's results were swift and validated my immediate assumption. 115.100.250.105 is a malicious Chinese (omg, really?) Zeus C&C server. Nice.
Highlighting a Website entity then choosing Detail View will tell you everything you need to know.



"Size matters not. Look at me. Judge me by my size, do you? Hmm? Hmm. And well you should not. For my ally is Maltego, and a powerful ally it is."

Yoda's right. ;-)

If you have any questions or would like saved transforms, PCAPs, or binary samples, ping me at russ at holisticinfosec dot org.

Cheers.

del.icio.us | digg | Submit to Slashdot

Please support the Open Security Foundation (OSVDB)

Thursday, December 03, 2009

Maltego is the 2009 Toolsmith Tool of the Year

Maltego: transform and correlate

December 2009's edition of the ISSA Journal's toolsmith discusses Maltego at length with specific attention to transforming RFI and scareware attributes.
Maltego is an open source intelligence and forensics application.
While researching and writing for December's article I fell completely for this tool.
It was a difficult decision having covered some brilliant and remarkable tools in 2009, but only one can come out on top.

The 2009 Toolsmith Tool of the Year is Maltego.
Congratulations to Andrew MacPherson and his team.

As an example, I used Maltego to analyze remote file include (RFI) attacks against my website and found it to be an extraordinary addition to my toolkit.
RFI attack URL strings often end with a common script name with a .txt or .gif extension.
I grabbed five such file names as most often seen in my logs from October:
zfxid1.txt
id1.txt
fx29id1.txt
idxx.txt
crespon1.txt
fxid1.txt

I fed these to Maltego and one of the URLS revealed showed results for a U.S. IP address, further showing that it had been flagged seven times for RFI attacks. This IP address has been identified as hijacked host/automated scanning drone due to the fact, that the host at this IP address has tried to injected a malicious script (RFI attack): http://www.ciasoftwares.com/fxid1.txt [show script].
Clicking the show script link then revealed that the script has a hash of a05dfd7cca7771a7565a154d65f05ea2 with all the attack details including script locations (RFI URLs), related IPs, and RFI script details. The figure below just begins to highlight how powerful Maltego really is.



The ISSA Journal December 2009 toolsmith article is here.
Again, congratulations to Maltego and Andrew MacPherson.

del.icio.us | digg | Submit to Slashdot

Please support the Open Security Foundation (OSVDB)

Moving blog to HolisticInfoSec.io

toolsmith and HolisticInfoSec have moved. I've decided to consolidate all content on one platform, namely an R markdown blogdown sit...