Prerequisites/dependencies
Python interpreter-enabled system, Kali Linux utilized
for this review
Introduction
The community of tools and developers converges again
this month as we explore Tim Tomes’ Recon-ng. Jeremy Druin, whose NOWASP
Mutillidae we explored in August 2012’s toolsmith,
introduced me to Tim, having recognized another great tool worthy of
exploration and sharing with toolsmith nation. Recon-ng is optimized for use
during the reconnaissance phase of web application penetration testing. You’ll
note convergence again given that we described managing web application
penetration testing phases in last month’s toolsmith regarding Redmine. Tim
says it best on his Recon-ng site: “Recon-ng is not intended to compete with
existing frameworks, as it is designed exclusively for web-based open source
reconnaissance. If you want to exploit, use the Metasploit Framework. If you
want to Social Engineer, us the Social Engineer Toolkit.
If you want to conduct reconnaissance, use Recon-ng!”
More from Tim on Recon-ng, shared exclusively with
toolsmith:
“Recon-ng is
commonly seen as being most useful in the role of supporting Social Engineering
engagements, but the real power of the framework lies in its ability to perform
all steps of the traditional penetration testing methodology, except
exploitation, within the context of reconnaissance. What does that mean? It
means that we can do scope validation through host discovery, server
enumeration, vulnerability discovery, and gain access to authentication
credentials, all without sending a single packet to the target application or
network. Recon-ng does this by leveraging powerful, 3rd party, web-based
resources that do all of this stuff for us, and provide access to the results.
It is important to keep in mind that there are caveats to this. Using 3rd
parties to collect data on clients may be in direct violation of Non-Disclosure
Agreements (NDA) or contracts. It is up to the tester to make sure that the client
specifically approves this activity as part of the testing agreement.
While the framework
is named for its focus on reconnaissance, the intent is not to limit its
functionality to only recon. Python developers have been waiting a long time
for a fun, easy, and useful project to contribute to. They now have that in
Recon-ng. Therefore, when contributors come up with new ideas for modules that
cross the boundary of reconnaissance into active discovery and exploitation,
they are encouraged to submit them for review. The several discovery modules
included in the framework are good examples of this.
I get asked quite
often, "How does Recon-ng fit into your testing methodology?" The
answer is simple. It's the first tool I use on every engagement, and often
during the scoping process. Do I run every module in the framework? No. It
largely depends on the type of assessment. But there are several things I
always do. I always harvest hosts from Google, Shodan and IP-Neighbors and
enumerate with the Resolve, BuiltWith and PunkSPIDER modules. I always harvest
contacts using Jigsaw, LinkedIn, PGP and Whois, and mangle them into email
addresses with the Mangle module. And I always check for compromised accounts
and harvest any available credentials using the various PwnedList modules.”
We’ve provided much detail on the web application
penetration testing methodology as describe by SANS in earlier toolsmiths, so
in order to broaden our horizons a bit, I’ll plug Recon-ng use into the various
phases of the OWASP Testing Guide v4. Version
4 is the draft version, version 3 (2008) is considered stable. The Information
Gathering section of the guide is a ten-part contribution to Section 4 of the
guide, Web Application Penetration Testing. Immediately relevant steps from the
draft TOC include:
·
4.2.1 Testing for Web Server Fingerprint
(OWASP-IG-004)
·
4.2.2 Review Webserver Metafiles (OWASP-IG-001)
·
4.2.5 Identify application entry points
(OWASP-IG-003)
We’ll also use reconnaissance methods to lend to section 4.4.2
Testing for User Enumeration and Guessable User Account (OWASP-AT-002) from 4.4
Authentication Testing.
We’ll use Recon-ng to realize the goals of a few of these
OWASP Testing Guide steps as we explore further below.
Recon-ng Installation
Recon-ng installs with ease on any Python and Git enabled
system. On Kali, running as root, it’s as simple as:
cd recon-ng./recon-ng.py
Figure 1 represents the initiated Recon-ng shell and its
57 recon, 6 discovery, 1 exploitation, and 2 reporting modules.
 |
| FIGURE 1: Getting underway with Recon-ng |
The dependencies on dnspython,
httplib2, and python-oauth2 are already met in the
recon-ng lib directory. If you’re familiar with Metasploit you’ll be right at
home with Recon-ng. Refer to the wiki for a
usage overview. I worked with both the stable version 1.20, as well as the beta
of 1.30, which should be a stable release by the time you read this. 1.30
includes major updates including what @LaNMaSteR53 Tweeted is a badly needed
API key handling system.
Putting Recon-ng to use
A few quick use pointers may
help you get under way with Recon-ng. Command completion is handy as you
consider typing commands such as
use recon/contacts/enum/http/should_change_password.
Hitting tab while keying will
complete based on options for command or parameter. Also extraordinarily useful
is the smart load feature which allows loads modules when you refer to a
keyword unique to the desired module's name. As an example, for the first
module we’ll test I simply typed
. With a particular
module loaded, use info for
name, author, description, and options details. Then use set based on the options defined
followed by the run command.
That’s all there is to it. You can define individual workspaces or other global
options as well. I ran show options
then set workspace holisticinfosec
for our efforts here. You can also set proxy settings here if you wish to
record your sessions with the like of Burp Suite. The resulting report from
Burp is a nice output product for your pentest engagements. Equally useful
might be the use of an anonymizing proxy.
use xssed which loaded the recon/hosts/enum/http/web/xssed module.
This works well without the full path as it is the only module containing the
string xssed but if multiple
modules share the same keyword you’ll receive a list of possible modules. Use is also, in reality, an alias for
the load command; they work
identically. Apparently, overly sensitive Metasploit users bugged Tim until
created command alignment. From a Recon-ng prompt the best way to see all modules
available to you is to pass the show
modules command, and don’t forget to use the ? command when you need more information. As an example show ? reveals your usage options are show
[modules|options|workspaces|schema|]
Use the recon/hosts/enum/http/api/builtwith module for 4.2.1 Testing for Web Server Fingerprint
(OWASP-IG-004). As the guidance states “knowing the version and type of a
running web server allows testers to determine known vulnerabilities and the
appropriate exploits to use during testing” you can imagine why. I loaded the
module, passed set host
holisticinfosec.org, and followed with run,
resulting in Figure 2.
 |
| FIGURE 2: Recon-ng establishes server details |
For 4.2.2 Review Webserver Metafiles (OWASP-IG-001) an
ideal module is discovery/info_disclosure/http/interesting_files.
This is not a passive module; it will reach out and touch the defined source,
and download discovered files such as robots.txt, sitemap.xml, crossdomain.xml,
and phpinfo.php. The discovered and downloaded files are written to the
workspace directory in which you are operating. The /recon-ng/workspace/default workspace is the default if
none is specified in the global options.
The xssed module relates nicely to section 4.2.5 Identify application entry points
(OWASP-IG-003) which describes the process to identify application entry
points. OWASP’s brief overview of this phase states that “enumerating the
application and its attack surface is a key precursor before any attack should
commence. This section will help you identify and map out every area within the
application that should be investigated once your enumeration and mapping phase
has been completed.” Parameters vulnerable to cross-site scripting (XSS) via
GET or POST requests certainly fall in the “worthy of investigation” category
as variables exhibiting XSS vulnerabilities are sometime vulnerable to other
issues such as SQL injection or directory traversal. Of course, XSS in and of
itself represents a number of opportunities for the attacker and should be paid
close attention as such.
The xssed module as written
by Micah Hoffman (@WebBreacher) checks XSSed.com for XSS records for the given
domain and displays the first 20 results. From the Recon-ng prompt I passed the
use xssed command followed by set domain microsoft.com. Given that I
work there and my attack & penetration testing may have a Microsoft domain
in scope for a penetration test, this module could prove a logical first step.
Note that all the returned results for this effort have been fixed even if
results state otherwise. After setting
the domain parameter one need only issue a run
command to kick off the module. Figure 3 shows the results.
 |
| FIGURE 3: Recon-ng XSSed module results |
The result advises us that,
had it not been fixed, the search parameter would have been ideal for further
exploration or use in packaging XSS payloads during an exploitation phase.
Recon-ng’s LinkedIn
Authenticated Contact Enumerator is a great way to gather possible social
engineering or bruteforcing targets, ideal during the 4.4.2 Testing for User Enumeration and Guessable User Account
(OWASP-AT-002) phase. You’ll need a LinkedIn API key; just login with you
LinkedIn cred and visit the LinkedIn Developer Network. Note: a few Recon-ng modules require
API keys. Keep in mind that the Pwnedlist API has a rather high cost associated
with it, but if your organization has already purchased API access, you can
leverage it with Recon-ng for the Pwnedlist modules account_creds, api_usage,
domain_creds, domain_ispwned, leak_lookup, and leaks_dump. Tim pointed out that, as a
Pwnedlist customer, he extremely fond of the domain_creds
module in particular as it returns actual domain credentials. Nothing like
walking in to a customer penetration testing engagement already in possession
of domain creds. For the LinkedIn module run use
linkedin, followed by set
company
Keep in mind that you can
always query the native Recon-ng SQLite database with the query command followed by common SQL
syntax. As an example query select *
from hosts returns data populated in the columns host, ip_address, region,
country, latitude, and longitude during module runs. The database schema is
included in Figure 4.
 |
| FIGURE 4: Recon-ng database schema |
Finally, you will definitely want to take advantage of the reporting modules.
Tim mentioned that the reporting/csv_file module is great for
importing into Excel then massaging the data while reporting/html_report module is optimal for producing
reports for customers. Figure 5 shows my reporting run against all date I’d
written so for the db.
 |
| FIGURE 5: Recon-ng database schema |
There are, as is often the
case with great toolsmith topics, too many features and killer use case
scenarios to cover here. I even suggested to Tim he write the Recon-ng book.
Yes, I think it’s that good.
In Conclusion
I’m really excited about Recon-ng and wish Tim great
success. My two favorite phases are reconnaissance and exploitation and
Recon-ng fits the bill to dominate the first and contribute greatly to the
second. Setting it up and getting started is a sixty second proposition and
leaves you no room for excuses. Get cracking with this tool STAT, run it
against entities specific to your organizations, and immediately benefit. Or
there’s always the alternative of waiting and having the hackers do it for you.
Ping me via email if you have questions or suggestions
for topic via russ at holisticinfosec dot org or hit me on Twitter @holisticinfosec.
Cheers…until
next month.
