Toolsmith #127: OSINT with Datasploit

I was reading an interesting Motherboard article, Legal Hacking Tools Can Be Useful for Journalists, Too, that includes reference to one of my all time OSINT favorites, Maltego. Joseph Cox's article also mentions Datasploit, a 2016 favorite for fellow tools aficionado, Toolswatch.org, see 2016 Top Security Tools as Voted by ToolsWatch.org Readers. Having not yet explored Datasploit myself, this proved to be a grand case of "no time like the present."
Datasploit is "an #OSINT Framework to perform various recon techniques, aggregate all the raw data, and give data in multiple formats." More specifically, as stated on Datasploit documentation page under Why Datasploit, it utilizes various Open Source Intelligence (OSINT) tools and techniques found to be effective, and brings them together to correlate the raw data captured, providing the user relevant information about domains, email address, phone numbers, person data, etc. Datasploit is useful to collect relevant information about target in order to expand your attack and defense surface very quickly.
The feature list includes:

• Automated OSINT on domain / email / username / phone for relevant information from different sources
• Useful for penetration testers, cyber investigators, defensive security professionals, etc.
• Correlates and collaborate results, shows them in a consolidated manner
• Tries to find out credentials,  API keys, tokens, sub-domains, domain history, legacy portals, and more as related to the target
• Available as single consolidating tool as well as standalone scripts
• Performs Active Scans on collected data
• Generates HTML, JSON reports along with text files

Resources
Github: https://github.com/datasploit/datasploit
Documentation: http://datasploit.readthedocs.io/en/latest/
YouTube: Quick guide to installation and use

Pointers
Second, a few pointers to keep you from losing your mind. This project is very much work in progress, lots of very frustrated users filing bugs and wondering where the support is. The team is doing their best, be patient with them, but read through the Github issues to be sure any bugs you run into haven't already been addressed.
1) Datasploit does not error gracefully, it just crashes. This can be the result of unmet dependencies or even a missing API key. Do not despair, take note, I'll talk you through it.
2) I suggest, for ease, and best match to documentation, run Datasploit from an Ubuntu variant. Your best bet is to grab Kali, VM or dedicated and load it up there, as I did.
3) My installation guidance and recommendations should hopefully get you running trouble free, follow it explicitly.
4) Acquire as many API keys as possible, see further detail below.

Installation and preparation
From Kali bash prompt, in this order:

1. git clone https://github.com/datasploit/datasploit /etc/datasploit
2. apt-get install libxml2-dev libxslt-dev python-dev lib32z1-dev zlib1g-dev
3. cd /etc/datasploit
4. pip install -r requirements.txt
5. mv config_sample.py config.py
6. With your preferred editor, open config.py and add API keys for the following at a minimum, they are, for all intents and purposes required, detailed instructions to acquire each are here:
1. Shodan API
2. Censysio ID and Secret
3. Clearbit API
4. Emailhunter API
5. Fullcontact API
6. Google Custom Search Engine API key and CX ID
7. Zoomeye Username and Password

If, and only if, you've done all of this correctly, you might end up with a running instance of Datasploit. :-) Seriously, this is some of the glitchiest software I've tussled with in quite a while, but the results paid handsomely. Run python datasploit.py domain.com, where domain.com is your target. Obviously, I ran python datasploit.py holisticinfosec.org to acquire results pertinent to your author. 

Datasploit rapidly pulled results as follows:

211 domain references from Github:

Github results

Luckily, no results from Shodan. :-)

Four results from Paste(s): 

Pastebin and Pastie results

Datasploit pulled russ at holisticinfosec dot org as expected, per email harvesting.

Accurate HolisticInfoSec host location data from Zoomeye:

Details regarding HolisticInfoSec sub-domains and page links:

Sub-domains and page links

Finally, a good return on DNS records for holisticinfosec.org and, thankfully, no vulns found via PunkSpider. 

DataSploit can also be integrated into other code and called as individual scripts for unique functions. I did a quick run with python emailOsint.py russ@holisticinfosec.org and the results were impressive:

Email OSINT

I love that the first query is of Troy Hunt's Have I Been Pwned. Not sure if you have been? Better check it out. Reminder here, you'll really want to be sure to have as many API keys as possible or you may find these buggy scripts crashing. You'll definitely find yourself compromising between frustration and the rapid, detailed results. I put this offering squarely in the "shows much promise category" if the devs keep focus on it, assess for quality, and handle errors better.

Give Datasploit a try for sure.

Cheers, until next time...