tag:blogger.com,1999:blog-200119602024-03-09T18:46:21.745-08:00HolisticInfoSec™Russ McRee's HolisticInfoSec™ includes articles and research, as well as feedback and an occasional rant.
HolisticInfoSec™ promotes standards, simplicity, tooling and efficiency in achieving holistic information security.Russ McReehttp://www.blogger.com/profile/05647342839278416757noreply@blogger.comBlogger314125tag:blogger.com,1999:blog-20011960.post-47724483082692149112018-08-06T22:15:00.000-07:002018-08-06T22:36:56.338-07:00Moving blog to HolisticInfoSec.io<div dir="ltr" style="text-align: left;" trbidi="on">
toolsmith and HolisticInfoSec have moved.<br />
I've decided to consolidate all content on one platform, namely an R <a href="https://bookdown.org/yihui/rmarkdown/" target="_blank">markdown</a> <a href="https://bookdown.org/yihui/blogdown/" target="_blank">blogdown</a> site running with Hugo for static HTML creation. My frustration with Blogger/Blogspot met its limit when a completed draft of toolsmith #134 vanished in to thin air, with no prospect of recovery. I'm not a fan of losing hours and hours of work in the simple act of an accidental tab refresh.<br />
<div>
As such, I've been meaning to do this for a while now, so I bought<a href="https://holisticinfosec.io/" target="_blank"> holisticinfosec.io</a> and mastered <a href="https://bookdown.org/yihui/blogdown/" target="_blank">blogdown</a> as fast as possible.<br />
<div>
toolsmith will continue to publish on a regular basis and will be the focal content for <a href="https://holisticinfosec.io/" target="_blank">holisticinfosec.io</a>.<br />
Old content will remain right here, I'm not pulling it down or republishing it.</div>
<div>
Update your feed readers, favorites, and bookmarks accordingly (holisticinfosec.org redirects to holisticinfosec.io), and see you at holisticinfosec.io for a far more modern, social experience.<br />
Do note that the site overall has a bit of ongoing work but will be finished soon.<br />
Regardless, <a href="https://holisticinfosec.io/post/shodan-as-a-verb-find-the-fail-before-it-finds-you/" target="_blank">toolsmith #134 Shodan As A Verb - Find The Fail Before It Finds You</a> is ready for your reading pleasure.</div>
</div>
<div>
Cheers, see you there.<br />
Russ</div>
</div>
Russ McReehttp://www.blogger.com/profile/05647342839278416757noreply@blogger.com3tag:blogger.com,1999:blog-20011960.post-44588734508472548122018-06-03T17:37:00.000-07:002018-06-16T11:02:48.077-07:00toolsmith #133 - Anomaly Detection & Threat Hunting with Anomalize When, in <a href="https://holisticinfosec.blogspot.com/2017/10/toolsmith-128-dfir-redefined-deeper.html" target="_blank">October</a> and <a href="https://holisticinfosec.blogspot.com/2017/11/toolsmith-129-dfir-redefined-deeper.html" target="_blank">November</a>'s toolsmith posts, I redefined DFIR under the premise of <b>D</b>eeper <b>F</b>unctionality for <b>I</b>nvestigators in <b>R</b>, I discovered a "tip of the iceberg" scenario. To that end, I'd like to revisit the concept with an additional discovery and opportunity. In reality, this is really a case of DFIR (<b>D</b>eeper <b>F</b>unctionality for <b>I</b>nvestigators in <b>R) </b>within the general practice of the original and paramount DFIR (<b>D</b>igital <b>F</b>orensics/<b>I</b>ncident <b>R</b>esponse).<br />
As discussed here before, those of us in the DFIR practice, and Blue Teaming at large, are overwhelmed by data and scale. Success truly requires algorithmic methods. If you're not already invested here I have an immediately applicable case study for you in tidy anomaly detection with anomalize.<br />
First, let me give credit where entirely due for the work that follows. Everything I discuss and provide is immediately derivative from Business Science (<a href="https://twitter.com/bizScienc" target="_blank">@bizScienc</a>), specifically Matt Dancho (<a href="https://twitter.com/mdancho84" target="_blank">@mdancho84</a>). He created <a href="https://business-science.github.io/anomalize/" target="_blank">anomalize</a>, "<i>a tidy anomaly detection algorithm that’s time-based (built on top of tibbletime) and scalable from one to many time series</i>," when a client asked Business Science to build an open source anomaly detection algorithm that suited their needs. I'd say he responded beautifully, when his <a href="http://www.business-science.io/code-tools/2018/04/08/introducing-anomalize.html" target="_blank">blogpost</a> hit my radar via <a href="https://www.r-bloggers.com/" target="_blank">R-Bloggers</a> it lived as an open tab in my browser for more than a month until generating this toolsmith. Please consider Matt's post a mandatory read as step one of the process here. I'll quote Matt specifically before shifting context: "<i>Our client had a challenging problem: detecting anomalies in time series on daily or weekly data at scale. Anomalies indicate exceptional events, which could be increased web traffic in the marketing domain or a malfunctioning server in the IT domain. Regardless, it’s important to flag these unusual occurrences to ensure the business is running smoothly. One of the challenges was that the client deals with not one time series but thousands that need to be analyzed for these extreme events.</i>"<br />
Key takeaway: <b>Detecting anomalies in time series on daily or weekly data at scale. Anomalies indicate exceptional events.</b><br />
Now shift context with me to security-specific events and incidents, as the pertain to security monitoring, incident response, and threat hunting. In my November 2017 <a href="https://holisticinfosec.blogspot.com/2017/11/toolsmith-129-dfir-redefined-deeper.html" target="_blank">post</a>, recall that I discussed Time Series Regression with the Holt-Winters method and a focus on seasonality and trends. Unfortunately, I couldn't share the code for how we applied TSR, but pointed out alternate methods, including Seasonal and Trend Decomposition using Loess (STL):<br />
<ul>
<li>Handles any type of seasonality ~ can change over time</li>
<li>Smoothness of the trend-cycle can also be controlled by the user</li>
<li>Robust to outliers</li>
</ul>
<div>
Here now, Matt has created a means to immediately apply the STL method, along with the Twitter method (<a href="https://business-science.github.io/anomalize/reference/time_decompose.html" target="_blank">reference page</a>), as part of his <span style="font-family: "courier new" , "courier" , monospace;">time_decompose()</span> function, one of three functions specific to the anomalize package. In addition to <span style="font-family: "courier new" , "courier" , monospace;">time_decompose()</span><span style="font-family: inherit;">, which sep</span>arates the time series into seasonal, trend, and remainder components,<span style="font-family: "courier new" , "courier" , monospace;"> </span><a href="https://github.com/business-science/anomalize" target="_blank">anomalize</a> includes:</div>
<div>
<div>
<ul>
<li><span style="font-family: "courier new" , "courier" , monospace;">anomalize()</span>: Applies anomaly detection methods to the remainder component.</li>
<li><span style="font-family: "courier new" , "courier" , monospace;">time_recompose()</span>: Calculates limits that separate the “normal” data from the anomalies</li>
</ul>
</div>
</div>
<div>
The methods used in <span style="font-family: "courier new" , "courier" , monospace;">anomalize()</span>, including IQR and GESD are described in Matt's <a href="https://business-science.github.io/anomalize/reference/anomalize.html" target="_blank">reference page</a>. Matt ultimately set out to build a scalable adaptation of Twitter's AnomalyDetection package in order to address his client's challenges in dealing with not one time series but thousands needing to be analyzed for extreme events. You'll note that Matt describes anomalize using a dataset of the daily download counts of the 15 <a href="https://www.tidyverse.org/" target="_blank">tidyverse</a> packages from <a href="https://cran.r-project.org/" target="_blank">CRAN</a>, relevant as he leverages the tidyverse package. I initially toyed with tweaking Matt's demo to model downloads for security-specific R packages (yes, there are such things) from CRAN, including <a href="https://cran.r-project.org/web/packages/RAppArmor/index.html" target="_blank">RAppArmor</a>, <a href="https://cran.r-project.org/web/packages/net.security/index.html" target="_blank">net.security</a>, <a href="https://cran.r-project.org/web/packages/securitytxt/index.html" target="_blank">securitytxt</a>, and <a href="https://cran.r-project.org/web/packages/cymruservices/index.html" target="_blank">cymruservices</a>, the latter two courtesy of Bob Rudis (<a href="https://twitter.com/hrbrmstr" target="_blank">@hrbrmstr</a>) of our beloved <a href="https://holisticinfosec.blogspot.com/2014/09/toolsmith-jay-and-bob-strike-back-data.html" target="_blank">Data-Driven Security: Analysis, Visualization and Dashboards</a>. Alas, this was a mere rip and replace, and really didn't exhibit the use of anomalize in a deserving, varied, truly security-specific context. That said, I was able to generate immediate results doing so, as seen in <b>Figure 1</b>. </div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgVNlBc976rdSti20sD4CeA-UvcMHOWfAwhyphenhyphenjl-dHcyayfwGhQhdcwB9Xaxs2LW4txQ2mIgy4EAzd3GgLdaCbPE3_42nK12_PkuX_ImZyQtoh1K0FzO8UOrkJ1Uea_GIdoe0eKZtg/s1600/Figure1.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="980" data-original-width="1341" height="465" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgVNlBc976rdSti20sD4CeA-UvcMHOWfAwhyphenhyphenjl-dHcyayfwGhQhdcwB9Xaxs2LW4txQ2mIgy4EAzd3GgLdaCbPE3_42nK12_PkuX_ImZyQtoh1K0FzO8UOrkJ1Uea_GIdoe0eKZtg/s640/Figure1.PNG" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 1:</b> Initial experiment</td></tr>
</tbody></table>
<div>
As an initial experiment you can replace packages names with those of your choosing in <a href="https://github.com/business-science/anomalize/blob/master/data-raw/tidyverse_cran_downloads.R" target="_blank">tidyverse_cran_downloads.R</a>, run it in R Studio, then tweak variable names and labels in the code per Matt's <a href="https://github.com/business-science/anomalize" target="_blank">README page</a>. </div>
<div>
I wanted to run anomalize against a real security data scenario, so I went back to the dataset from the original DFIR articles where I'd utilized counts of 4624 Event IDs per day, per user, on a given set of servers. As utilized originally, I'd represented results specific to only one device and user, but herein is the beauty of anomalize. We can achieve quick results across multiple times series (multiple systems/users). This premise is but one of many where time series analysis and seasonality can be applied to security data.<br />
I originally tried to write log data from <span style="font-family: "courier new" , "courier" , monospace;">log.csv</span> straight to an <span style="font-family: "courier new" , "courier" , monospace;">anomalize.R</span> script with <span style="font-family: "courier new" , "courier" , monospace;">logs = read_csv("log.csv")</span> into a <a href="https://tibble.tidyverse.org/index.html" target="_blank">tibble</a> (ready your troubles with tibbles jokes), which was not being parsed accurately, particularly time attributes. To correct this, from Matt's Github I grabbed <span style="font-family: "courier new" , "courier" , monospace;">tidyverse_cran_downloads.R</span>, and modified it as follows:<br />
<script src="https://gist.github.com/holisticinfosec/46e4dbb87d5c7b150d164c8769a91024.js"></script>
This helped greatly thanks to the tibbletime package, which is "is an extension that allows for the creation of time aware tibbles. Some immediate advantages of this include: the ability to perform time based subsetting on tibbles, quickly summarising and aggregating results by time periods. Guess what, Matt wrote tibbletime too. :-)<br />
I then followed Matt's sequence as he posted on Business Science, but with my logs defined as a function in <span style="font-family: "courier new" , "courier" , monospace;">Security_Access_Logs_Function.R</span>. Following, I'll give you the code snippets, as revised from Matt's examples, followed by their respective results specific to processing my Event ID 4624 daily count log.<br />
First, let's summarize daily login counts across three servers over four months.<br />
<script src="https://gist.github.com/holisticinfosec/4824415a4daef911d9bf053df2d73211.js"></script>
The result is evident in <b>Figure 2</b>.
<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhy1RIGgIW4ba6SPfGqcnwkShPg4dpYrRKaXKauN7VsQXMQEx0ZB5BT_nNPjLMplRfmdMMzk3PNtgTsJ9RHRb62mHXWZN6AUq5K9OzfBh9R1WJ3wia33CwZSJ2-XCFgvrEmOUO9yA/s1600/Figure2.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="593" data-original-width="823" height="459" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhy1RIGgIW4ba6SPfGqcnwkShPg4dpYrRKaXKauN7VsQXMQEx0ZB5BT_nNPjLMplRfmdMMzk3PNtgTsJ9RHRb62mHXWZN6AUq5K9OzfBh9R1WJ3wia33CwZSJ2-XCFgvrEmOUO9yA/s640/Figure2.PNG" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 2:</b> Server logon counts visualized</td></tr>
</tbody></table>
Next, let's determine which daily download logons are anomalous with Matt's three main functions, <span style="font-family: "courier new" , "courier" , monospace;">time_decompose()</span>, <span style="font-family: "courier new" , "courier" , monospace;">anomalize()</span>, and <span style="font-family: "courier new" , "courier" , monospace;">time_recompose()</span>, along with the visualization function, <span style="font-family: "courier new" , "courier" , monospace;">plot_anomalies()</span>, across the same three servers over four months.<br />
<script src="https://gist.github.com/holisticinfosec/6b6811f2df767b83e98f7ad690c33594.js"></script>
The result is revealed in <b>Figure 3</b>.
<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgNCWY4Mpgc9Q_VuYBSyuYvlr7UqyQ3u80J-miM2H364MexXLkzAJbJ2Z2U3EvGfHgM_c9Wpr9RZ7sO6t8nmiXeHJlccQRGOaRU-xI44MBxBCISJgkxysgArbcLj6MVXWXd4-hpIw/s1600/Figure3.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="621" data-original-width="880" height="449" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgNCWY4Mpgc9Q_VuYBSyuYvlr7UqyQ3u80J-miM2H364MexXLkzAJbJ2Z2U3EvGfHgM_c9Wpr9RZ7sO6t8nmiXeHJlccQRGOaRU-xI44MBxBCISJgkxysgArbcLj6MVXWXd4-hpIw/s640/Figure3.PNG" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 3:</b> Security event log anomalies</td></tr>
</tbody></table>
Following Matt's method using Twitter’s <span style="font-family: "courier new" , "courier" , monospace;">AnomalyDetection</span> package, combining <span style="font-family: "courier new" , "courier" , monospace;">time_decompose(method = "twitter")</span> with <span style="font-family: "courier new" , "courier" , monospace;">anomalize(method = "gesd")</span>, while adjusting the <span style="font-family: "courier new" , "courier" , monospace;">trend = "4 months"</span> to adjust median spans, we'll focus only on SERVER-549521.<br />
<script src="https://gist.github.com/holisticinfosec/5ac699647ab91fe568f7c62fcbfc615d.js"></script>
In <b>Figure 4</b>, you'll note that there are anomalous logon counts on SERVER-549521 in June.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjluPBn1yp5BdVp-bnia4lA1Bzjb5Ltc8jeFhL83VnIVnRbDu5szUZtHvCNH2K4DI_kONRqrPjkHgdKFuJqwjUNmjXMjv0k0vXbXRF_8Lvxa4fbgicZYpDqP6Nfc7Lq3HYXMxL3Jg/s1600/Figure4.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="629" data-original-width="881" height="456" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjluPBn1yp5BdVp-bnia4lA1Bzjb5Ltc8jeFhL83VnIVnRbDu5szUZtHvCNH2K4DI_kONRqrPjkHgdKFuJqwjUNmjXMjv0k0vXbXRF_8Lvxa4fbgicZYpDqP6Nfc7Lq3HYXMxL3Jg/s640/Figure4.PNG" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 4:</b> SERVER-549521 logon anomalies with Twitter & GESD methods</td></tr>
</tbody></table>
We can compare the <a href="https://business-science.github.io/anomalize/reference/decompose_methods.html" target="_blank">Twitter</a> (time_decompose) and <a href="https://business-science.github.io/anomalize/reference/anomalize_methods.html" target="_blank">GESD</a> (anomalize) methods with the <a href="https://business-science.github.io/anomalize/reference/decompose_methods.html" target="_blank">STL</a> (time_decompose) and <a href="https://business-science.github.io/anomalize/reference/anomalize_methods.html" target="_blank">IQR</a> (anomalize) methods, which use different <a href="https://business-science.github.io/anomalize/reference/time_decompose.html" target="_blank">decomposition</a> and <a href="https://business-science.github.io/anomalize/reference/anomalize.html" target="_blank">anomaly</a> detection approaches.<br />
<script src="https://gist.github.com/holisticinfosec/7a026502b10880b5596c156b67032764.js"></script>
Again, we note anomalies in June, as seen in <b>Figure 5</b>.
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2J4pbb3D4yEeoCuFSZPPSCEom6uzjPvIN7swtyqhWnDWMl1I_5MsUUJQoTN3e2A9YnPhcXTgUizzjuqbbIM88JjeDPbXqLhL6g2nvPoB_vFMGIhcwCEegr_UihtIVvwoPDpcj9A/s1600/Figure5.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="621" data-original-width="878" height="452" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2J4pbb3D4yEeoCuFSZPPSCEom6uzjPvIN7swtyqhWnDWMl1I_5MsUUJQoTN3e2A9YnPhcXTgUizzjuqbbIM88JjeDPbXqLhL6g2nvPoB_vFMGIhcwCEegr_UihtIVvwoPDpcj9A/s640/Figure5.PNG" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 5:</b> SERVER-549521 logon anomalies with STL & IQR methods</td></tr>
</tbody></table>
Obviously, the results are quite similar, as one would hope. Finally, let use Matt's <span style="font-family: "courier new" , "courier" , monospace;">plot_anomaly_decomposition()</span> for visualizing the inner workings of how algorithm detects anomalies in the <a href="https://business-science.github.io/anomalize/reference/anomalize.html" target="_blank">remainder</a> for SERVER-549521.<br />
<script src="https://gist.github.com/holisticinfosec/45e2a4263a6087ea690dda52808ca362.js"></script>
The result is a four part visualization, including observed, season, trend, and remainder as seen in <b>Figure 6</b>.
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhKWURVvXS5RKmyYTNby2PcOjkbBPorQLoxi4Drbljz8MpTuuZDMWHJJKp5_D_VYZX_5T9kVW6ziYiJ4DUfvIjloVd3FYvfazOh0DBqY2vRRo5Uvxy49R1m6DJt7_urai0NoRhMHg/s1600/Figure6.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="622" data-original-width="879" height="452" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhKWURVvXS5RKmyYTNby2PcOjkbBPorQLoxi4Drbljz8MpTuuZDMWHJJKp5_D_VYZX_5T9kVW6ziYiJ4DUfvIjloVd3FYvfazOh0DBqY2vRRo5Uvxy49R1m6DJt7_urai0NoRhMHg/s640/Figure6.PNG" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 6:</b> Decomposition for SERVER-549521 Logins</td></tr>
</tbody></table>
I'm really looking forward to putting these methods to use at a much larger scale, across a far broader event log dataset. I firmly assert that blue teams are already way behind in combating automated adversary tactics and problems of sheer scale, so...much...data. It's only with tactics such as Matt's anomalize, and others of its ilk, that defenders can hope to succeed. Be sure the watch Matt's <a href="https://www.youtube.com/watch?v=Gk_HwjhlQJs" target="_blank">YouTube video on anomalize</a>, Business Science is building a series of videos in addition, so keep an eye out there and on their GitHub for more great work that we can apply a blue team/defender's context to.</div>
<div>
All the code snippets are in my GitHubGist <a href="https://gist.github.com/holisticinfosec" target="_blank">here</a>, and the sample <a href="https://github.com/holisticinfosec/toolsmith_R/blob/master/anomalize/log.csv" target="_blank">log file</a>, a single <a href="https://github.com/holisticinfosec/toolsmith_R/blob/master/anomalize/anomalizeEventID.R" target="_blank">R script</a>, and a <a href="https://github.com/holisticinfosec/toolsmith_R/tree/master/anomalize" target="_blank">Jupyter Notebook</a> are all available for you on my GitHub under <a href="https://github.com/holisticinfosec/toolsmith_R" target="_blank">toolsmith_r</a>. I hope you find anomalize as exciting and useful as I have, great work by Matt, looking forward to see what's next from Business Science.</div>
<div>
Cheers...until next time.</div>
Russ McReehttp://www.blogger.com/profile/05647342839278416757noreply@blogger.com1tag:blogger.com,1999:blog-20011960.post-24693460633811520262018-04-03T00:01:00.001-07:002018-04-03T00:03:01.450-07:00toolsmith #132 - The HELK vs APTSimulator - Part 2<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: left; margin-right: 1em; text-align: left;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhWKthNvEl2J3Q3L7jwu9GuGmZ8kDKsxj5nScdi1F7uIblMbmMpz1TfYQRb5UOV03NMIhzNoZtHgQ30iPk8g_OAx3I24fU0wuJpFpnj9smDHQKlXrviI0Pp6FQ41dqUepDCE3H4yw/s1600/HELK.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="467" data-original-width="1355" height="135" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhWKthNvEl2J3Q3L7jwu9GuGmZ8kDKsxj5nScdi1F7uIblMbmMpz1TfYQRb5UOV03NMIhzNoZtHgQ30iPk8g_OAx3I24fU0wuJpFpnj9smDHQKlXrviI0Pp6FQ41dqUepDCE3H4yw/s400/HELK.PNG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><br /></td></tr>
</tbody></table>
<div>
Continuing where we left off in <a href="https://holisticinfosec.blogspot.com/2018/02/toolsmith-131-helk-vs-aptsimulator-part.html" target="_blank">The HELK vs APTSimulator - Part 1</a>, I will focus our attention on additional, useful HELK features to aid you in your threat hunting practice. HELK offers Apache Spark, GraphFrames, and Jupyter Notebooks as part of its lab offering. These capabilities scale well beyond a standard ELK stack, this really is where parallel computing and significantly improved processing and analytics truly take hold. This is a great way to introduce yourself to these technologies, all on a unified platform.<br />
<br />
Let me break these down for you a little bit in case you haven't been exposed to these technologies yet. First and foremost, refer to <a href="https://twitter.com/cyb3rWard0g" target="_blank">@Cyb3rWard0g</a>'s <a href="https://github.com/Cyb3rWard0g/HELK/wiki/Spark" target="_blank">wiki</a> page on how he's designed it for his HELK implementation, as seen in <b>Figure 1</b>.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj_ysR6MnloXOE8EfFxp_qqMUQ80sbOCCCcbRjQnHIVTRWq2C40RxC49S0t0hRaxUB_3528wUAySvChlqrAZKXFH-7kiBQWBKKn2gL7_OsKXef7nZ-L1WBATHeliI1Q72qlfWB78w/s1600/SPARK-Design.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="792" data-original-width="1600" height="315" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj_ysR6MnloXOE8EfFxp_qqMUQ80sbOCCCcbRjQnHIVTRWq2C40RxC49S0t0hRaxUB_3528wUAySvChlqrAZKXFH-7kiBQWBKKn2gL7_OsKXef7nZ-L1WBATHeliI1Q72qlfWB78w/s640/SPARK-Design.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 1:</b> HELK Architecture</td></tr>
</tbody></table>
First, Apache Spark. For HELK, "Elasticsearch-hadoop provides native integration between Elasticsearch and Apache Spark, in the form of an RDD (Resilient Distributed Dataset) (or Pair RDD to be precise) that can read data from Elasticsearch." Per the Apache Spark FAQ, "Spark is a fast and general processing engine compatible with Hadoop data" to deliver "lighting-fast cluster computing."<br />
Second, GraphFrames. From the GraphFrames <a href="https://graphframes.github.io/" target="_blank">overview</a>, "GraphFrames is a package for Apache Spark which provides DataFrame-based Graphs. GraphFrames represent graphs: vertices (e.g., users) and edges (e.g., relationships between users). GraphFrames also provide powerful tools for running queries and standard graph algorithms. With GraphFrames, you can easily search for patterns within graphs, find important vertices, and more." <br />
Finally, Jupyter Notebooks to pull it all together.<br />
From Jupyter.org: "The Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. Uses include: data cleaning and transformation, numerical simulation, statistical modeling, data visualization, machine learning, and much more." Jupyter Notebooks provide a higher order of analyst/analytics capabilities, if you haven't dipped your toe in that water, this may be your first, best opportunity.<br />
Let's take a look at using Jupyter Notebooks with the data populated to my Docker-based HELK instance as implemented in <a href="https://holisticinfosec.blogspot.com/2018/02/toolsmith-131-helk-vs-aptsimulator-part.html" target="_blank">Part 1</a>. I repopulated my HELK instance with new data from a different, bare metal Windows instance reporting to HELK with Winlogbeat, Sysmon enabled, and looking mighty compromised thanks to <a href="https://twitter.com/cyb3rops" target="_blank">@cyb3rops</a>'s <a href="https://github.com/NextronSystems/APTSimulator" target="_blank">APTSimulator</a>.<br />
To make use of Jupyter Notebooks, you need your JUPYTER CURRENT TOKEN to access the Jupyter Notebook web interface. It was presented to you when your HELK installation completed, but you can easily retrieve it via <span style="font-family: "courier new" , "courier" , monospace;">sudo docker logs helk-analytics</span>, then copy and paste the URL into your browser to connect for the first time with a token. It will look like this,<br />
<span style="font-family: "courier new" , "courier" , monospace;">http://localhost:8880/?token=3f46301da4cd20011391327647000e8006ee3574cab0b163</span>, as described in the <a href="https://github.com/Cyb3rWard0g/HELK/wiki/Installation" target="_blank">Installation wiki</a>. After browsing to the URL with said token, you can begin at <span style="font-family: "courier new" , "courier" , monospace;">http://localhost:8880/lab</span>, where you should immediately proceed to the <span style="font-family: "courier new" , "courier" , monospace;">Check_Spark_Graphframes_Integrations.ipynb</span> notebook. It's found in the hierarchy menu under <span style="font-family: "courier new" , "courier" , monospace;">training > jupyter_notebooks > getting_started</span>. This notebook is essential to confirming you're ingesting data properly with HELK and that its integrations are fully functioning. Step through it one cell at a time with the play button, allowing each task to complete so as to avoid errors. Remember the above mentioned Resilient Distributed Dataset? This notebook will create a Spark RDD on top of Elasticsearch using the <span style="font-family: "courier new" , "courier" , monospace;">logs-endpoint-winevent-sysmon-*</span> (Sysmon logs) index as source, and do the same thing with the <span style="font-family: "courier new" , "courier" , monospace;">logs-endpoint-winevent-security-*</span> (Window Security Event logs) index as source, as seen in <b>Figure 2</b>.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiw5ECYNkKJsxPvgRC41jUeKbPoSUAKlklzcFE8L5jp-SsuE8sDumDqKE-rU-xN3gCJLFkiBwUNkqcLuM9DJao0H5uwPPpm6vBPXu6dkDqtY6DBe90JKPX7046CuGPhxmHknnuQVg/s1600/WinSecEVTRDD.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="916" data-original-width="1600" height="228" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiw5ECYNkKJsxPvgRC41jUeKbPoSUAKlklzcFE8L5jp-SsuE8sDumDqKE-rU-xN3gCJLFkiBwUNkqcLuM9DJao0H5uwPPpm6vBPXu6dkDqtY6DBe90JKPX7046CuGPhxmHknnuQVg/s400/WinSecEVTRDD.PNG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 2:</b> Windows Security EVT Spark RDD</td></tr>
</tbody></table>
The notebook will also query your Windows security events via Spark SQL, then print the schema with:<br />
<span style="font-family: "courier new" , "courier" , monospace;">df = spark.read.format("org.elasticsearch.spark.sql").load("logs-endpoint-winevent-security-*/doc")</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">df.printSchema()</span><br />
The result should resemble <b>Figure 3</b>.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhYxeW6QKKOYSAyntlROoW9XhkudVcRN4dXIjlRIV-NgNqjOEYZuBg5Z4mps8BKoZmt4E7D1W-WuNvcl4PWf2nugqfaLcUvg8z8rVfA7qRsTl4zr0tUz_4JbLMVThl1m3emf85kqA/s1600/schema.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1132" data-original-width="894" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhYxeW6QKKOYSAyntlROoW9XhkudVcRN4dXIjlRIV-NgNqjOEYZuBg5Z4mps8BKoZmt4E7D1W-WuNvcl4PWf2nugqfaLcUvg8z8rVfA7qRsTl4zr0tUz_4JbLMVThl1m3emf85kqA/s400/schema.PNG" width="315" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 3: </b>Schema</td></tr>
</tbody></table>
Assuming all matches with relative consistency in your experiment, let's move on to the <span style="font-family: "courier new" , "courier" , monospace;">Sysmon_ProcessCreate_Graph.ipynb</span> notebook, found in <span style="font-family: "courier new" , "courier" , monospace;">training > jupyter_notebooks</span>. This notebook will again call on the Elasticsearch Sysmon index and create vertices and edges dataframes, then create a graph produced with GraphFrame built from those same vertices and edges. Here's a little walk-through.<br />
The <span style="font-family: "courier new" , "courier" , monospace;">v</span> parameter (yes, for vertices) is populated with:<br />
<span style="font-family: "courier new" , "courier" , monospace;">v = df.withColumn("id", df.process_guid).select("id","user_name","host_name","process_parent_name","process_name","action")</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">v = v.filter(v.action == "processcreate")</span><br />
Showing the top three rows of that result set, with <span style="font-family: "courier new" , "courier" , monospace;">v.show(3,truncate=False)</span>, appears as <b>Figure 4</b> in the notebook, with the data from my APTSimulator "victim" system, N2KND-PC.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEij7XdCqORqFSVOIgWRNvPpox6iDI9QKMK2Cv2qd1YudxA_qXIMFpJZExBpb4-VAwZ-B_dr9bHWToY5crHzbwHS9GCc29V-n-BE-6TG4ItEwhK5Twz9ztIMNoqXXYaU-dDHagOpNg/s1600/ProcessCreate.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="226" data-original-width="1600" height="89" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEij7XdCqORqFSVOIgWRNvPpox6iDI9QKMK2Cv2qd1YudxA_qXIMFpJZExBpb4-VAwZ-B_dr9bHWToY5crHzbwHS9GCc29V-n-BE-6TG4ItEwhK5Twz9ztIMNoqXXYaU-dDHagOpNg/s640/ProcessCreate.PNG" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 4:</b> WTF, Florian :-)</td></tr>
</tbody></table>
The epic, uber threat hunter in me believes that APTSimulator created <span style="font-family: "courier new" , "courier" , monospace;">nslookup</span>, <span style="font-family: "courier new" , "courier" , monospace;">7z</span>, and <span style="font-family: "courier new" , "courier" , monospace;">regedit</span> as processes via cmd.exe. Genius, right? :-)<br />
The <span style="font-family: "courier new" , "courier" , monospace;">e</span> parameter (yes, for edges) is populated with:<br />
<span style="font-family: "courier new" , "courier" , monospace;">e = df.filter(df.action == "processcreate").selectExpr("process_parent_guid as src","process_guid as dst").withColumn("relationship", lit("spawned"))</span><br />
Showing the top three rows of that result set, with <span style="font-family: "courier new" , "courier" , monospace;">e.show(3,truncate=False)</span>, produces the source and destination process IDs as it pertains to the spawning relationship.<br />
Now, to create a graph from the vertices and edges dataframes as defined in the <span style="font-family: "courier new" , "courier" , monospace;">v</span> & <span style="font-family: "courier new" , "courier" , monospace;">e</span> parameters with <span style="font-family: "courier new" , "courier" , monospace;">g = GraphFrame(v, e)</span>. Let's bring it home with a hunt for Process A spawning Process B AND Process B Spawning Process C, the code needed, and the result, are seen from the notebook in <b>Figure 5</b>.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgRlRLFcYaxqa1tvxOgGuLbnAfxHlX6kkGUsfCAJWs7VTfOTw4fqQCUkjbGRtz5kJPx4K-DrSovJ1PLjmq-dJNOrpFNle5T4_25vy9WLSlye2P2ZkskzA2p3aLMc130VQUTPe8WkA/s1600/AB%2526BC.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="328" data-original-width="1600" height="130" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgRlRLFcYaxqa1tvxOgGuLbnAfxHlX6kkGUsfCAJWs7VTfOTw4fqQCUkjbGRtz5kJPx4K-DrSovJ1PLjmq-dJNOrpFNle5T4_25vy9WLSlye2P2ZkskzA2p3aLMc130VQUTPe8WkA/s640/AB%2526BC.PNG" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 5:</b> APTSimulator's happy spawn</td></tr>
</tbody></table>
Oh, yes, APTSimulator fully realized in a nice graph. Great example seen in <span style="font-family: "courier new" , "courier" , monospace;">cmd.exe</span> spawning <span style="font-family: "courier new" , "courier" , monospace;">wscript.exe</span>, which then spawns <span style="font-family: "courier new" , "courier" , monospace;">rundll32.exe</span>. Or <span style="font-family: "courier new" , "courier" , monospace;">cmd.exe</span> spawning <span style="font-family: "courier new" , "courier" , monospace;">powershell.exe</span> and <span style="font-family: "courier new" , "courier" , monospace;">schtasks.exe</span>.<br />
Need confirmation? Florian's <a href="https://github.com/NextronSystems/APTSimulator/blob/1c9048e834f0adabd18c8871d587fda42315575b/test-sets/defense-evasion/js-dropper.bat" target="_blank">CactusTorch JS dropper</a> is detailed in <b>Figure 6</b>, specifically <span style="font-family: "courier new" , "courier" , monospace;">cmd.exe > wscript.exe > rundll32.exe</span>.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhqXzMp1qF0g2L2PmdFUFGFZ79DGqM3WHNVaZDwPX4jyjrI3CICvr7pI52EWRW2K08R_11ISFd6VxkVFIrLuj8EsU7OlPw1MJ1sVNzsTIaJ6enTKOqyU0XwdVjoli_z8n8NnrYGIw/s1600/CactusTorch.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="865" data-original-width="1600" height="215" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhqXzMp1qF0g2L2PmdFUFGFZ79DGqM3WHNVaZDwPX4jyjrI3CICvr7pI52EWRW2K08R_11ISFd6VxkVFIrLuj8EsU7OlPw1MJ1sVNzsTIaJ6enTKOqyU0XwdVjoli_z8n8NnrYGIw/s400/CactusTorch.PNG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 6:</b> APTSimulator source for CactusTorch</td></tr>
</tbody></table>
Still not convinced? How about APTSimulator's <span style="font-family: "courier new" , "courier" , monospace;">schtasks.bat,</span> where APTSimulator kindly loads mimikatz with <span style="font-family: "courier new" , "courier" , monospace;">schtasks.exe</span> for persistence, per <b>Figure 7</b>?<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi4lKCqw8D3ADHbracB_uWAqEYFbV50H2c2ujV1A_HYP4TOptiOJCamOMkNG7trt7cYI3FuswVVq5grVNxBYd2lpMExmGIlRhVTvTAbs2KJDwj3Zs7Stiyg0j76Nt6mbKZNav8WxQ/s1600/schtasks.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="554" data-original-width="1600" height="137" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi4lKCqw8D3ADHbracB_uWAqEYFbV50H2c2ujV1A_HYP4TOptiOJCamOMkNG7trt7cYI3FuswVVq5grVNxBYd2lpMExmGIlRhVTvTAbs2KJDwj3Zs7Stiyg0j76Nt6mbKZNav8WxQ/s400/schtasks.PNG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 7:</b> schtasks.bat</td></tr>
</tbody></table>
I certainly hope that the HELK's graph results matching nicely with APTSimulator source meets with your satisfaction.<br />
The HELK vs APTSimulator ends with a glorious flourish, these two monsters in their field belong in every lab to practice red versus blue, attack and defend, compromise and detect. I haven't been this happy to be a practitioner in the defense against the dark arts in quite awhile. My sincere thanks to Roberto and Florian for their great work on the HELK and APTSimulator. I can't suggest strongly enough how much you'll benefit from taking the time to run through Part 1 and 2 of The HELK vs APTSimulator for yourself. Both tools are well documented on their respective Githubs, go now, get started, profit.<br />
Cheers...until next time.</div>
Russ McReehttp://www.blogger.com/profile/05647342839278416757noreply@blogger.com0tag:blogger.com,1999:blog-20011960.post-10847859862690233762018-02-11T22:56:00.002-08:002018-02-15T09:16:29.507-08:00toolsmith #131 - The HELK vs APTSimulator - Part 1<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhC48UFqeMeLweY8OBA2dJa3HpoO-2oom2hvCz-AW1A3Hf7hQyVVHXsKCvRNYmDxbx4Pi0p7joq9Ju6dETIL3U15LphKdmr-2ysZkWJB9OAmorXy7lb1bIfofyAlfJ03Bf1SHg3oQ/s1600/The_Incredible_Hulk_vs_Doomsday_Ver._2.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="359" data-original-width="640" height="179" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhC48UFqeMeLweY8OBA2dJa3HpoO-2oom2hvCz-AW1A3Hf7hQyVVHXsKCvRNYmDxbx4Pi0p7joq9Ju6dETIL3U15LphKdmr-2ysZkWJB9OAmorXy7lb1bIfofyAlfJ03Bf1SHg3oQ/s320/The_Incredible_Hulk_vs_Doomsday_Ver._2.png" width="320" /></a></div>
Ladies and gentlemen, for our main attraction, I give you...The HELK vs APTSimulator, in a Death Battle! The late, great Randy "Macho Man" Savage said many things in his day, in his own special way, but "Expect the unexpected in the kingdom of madness!" could be our toolsmith theme this month and next. Man, am I having a flashback to my college days, many moons ago. :-) The HELK just brought it on. Yes, I know, HELK is the Hunting ELK stack, got it, but it reminded me of the Hulk, and then, I thought of a Hulkamania showdown with APTSimulator, and Randy Savage's classic, raspy voice popped in my head with "Hulkamania is like a single grain of sand in the Sahara desert that is Macho Madness." And that, dear reader, is a glimpse into exactly three seconds or less in the mind of your scribe, a strange place to be certain. But alas, that's how we came up with this fabulous showcase.<br />
In this corner, from Roberto Rodriguez, <a href="https://twitter.com/cyb3rWard0g" target="_blank">@Cyb3rWard0g</a>, the specter in <a href="https://specterops.io/" target="_blank">SpecterOps</a>, it's...The...HELK! This, my friends, is the s**t, worth every ounce of hype we can muster.<br />
And in the other corner, from Florian Roth, <a href="https://twitter.com/cyb3rops" target="_blank">@cyb3rops</a>, the The Fracas of Frankfurt, we have APTSimulator. All your worst adversary apparitions in one APT mic drop. This...is...Death Battle!<br />
<br />
Now with that out of our system, let's begin. There's a lot of goodness here, so I'm definitely going to do this in two parts so as not undervalue these two offerings.<br />
HELK is incredibly easy to install. Its also well documented, with lots of related reading material, let me propose that you take the tine to to review it all. Pay particular attention to the <a href="https://github.com/Cyb3rWard0g/HELK/wiki" target="_blank">wiki</a>, gain comfort with the architecture, then review <a href="https://github.com/Cyb3rWard0g/HELK/wiki/Installation" target="_blank">installation steps</a>.<br />
On an Ubuntu 16.04 LTS system I ran:<br />
<ul>
<li><span style="font-family: "courier new" , "courier" , monospace;">git clone https://github.com/Cyb3rWard0g/HELK.git</span></li>
<li><span style="font-family: "courier new" , "courier" , monospace;">cd HELK/</span></li>
<li><span style="font-family: "courier new" , "courier" , monospace;">sudo ./helk_install.sh </span></li>
</ul>
Of the three installation options I was presented with, pulling the latest HELK Docker Image from cyb3rward0g dockerhub, building the HELK image from a local Dockerfile, or installing the HELK from a local bash script, I chose the first and went with the latest Docker image. The installation script does a fantastic job of fulfilling dependencies for you, if you haven't installed Docker, the HELK install script does it for you. You can observe the entire install process in <b>Figure 1</b>.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhOnEBNoA5nlcSsTIchQ8MpcryDymACFBsKFlWcbzxwMcn2tKTBuWVBfy-6wYdBkgpjRty2e14Um5Z2GVSkJV4SBbYwDMxRgfGO5HeIoZqZGh9CR-LKg1Jnk-iVQJcArj0dSCtB4A/s1600/DockerInstall.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="807" data-original-width="827" height="390" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhOnEBNoA5nlcSsTIchQ8MpcryDymACFBsKFlWcbzxwMcn2tKTBuWVBfy-6wYdBkgpjRty2e14Um5Z2GVSkJV4SBbYwDMxRgfGO5HeIoZqZGh9CR-LKg1Jnk-iVQJcArj0dSCtB4A/s400/DockerInstall.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 1:</b> HELK Installation</td></tr>
</tbody></table>
You can immediately confirm your clean installation by navigating to your HELK KIBANA URL, in my case http://192.168.248.29.<br />
For my test Windows system I created a Windows 7 x86 virtual machine with Virtualbox. The key to success here is ensuring that you install Winlogbeat on the Windows systems from which you'd like to ship logs to HELK. More important, is ensuring that you run Winlogbeat with the right <a href="https://github.com/Cyb3rWard0g/HELK/blob/master/winlogbeat/winlogbeat.yml" target="_blank"><span style="font-family: "courier new" , "courier" , monospace;">winlogbeat.yml</span></a> file. You'll want to modify and copy this to your target systems. The critical modification is line 123, under Kafka output, where you need to add the IP address for your HELK server in three spots. My modification appeared as <span style="font-family: "courier new" , "courier" , monospace;">hosts: ["192.168.248.29:9092","192.168.248.29:9093","192.168.248.29:9094"]</span>. As noted in the HELK architecture diagram, HELK consumes Winlogbeat event logs via <a href="https://github.com/Cyb3rWard0g/HELK/wiki/Kafka" target="_blank">Kafka</a>.<br />
On your Windows systems, with a properly modified <span style="font-family: "courier new" , "courier" , monospace;">winlogbeat.yml</span>, you'll run:<br />
<ul>
<li><span style="font-family: "courier new" , "courier" , monospace;">./winlogbeat -c winlogbeat.yml -e</span></li>
<li><span style="font-family: "courier new" , "courier" , monospace;">./winlogbeat setup -e</span></li>
</ul>
You'll definitely want to set up Sysmon on your target hosts as well. I prefer to do so with the <a href="https://twitter.com/swiftonsecurity" target="_blank">@SwiftOnSecurity</a> <a href="https://github.com/SwiftOnSecurity/sysmon-config" target="_blank">configuration file</a>. If you're doing so with your initial setup, use <span style="font-family: "courier new" , "courier" , monospace;">sysmon.exe -accepteula -i sysmonconfig-export.xml</span>. If you're modifying an existing configuration, use <span style="font-family: "courier new" , "courier" , monospace;">sysmon.exe -c sysmonconfig-export.xml</span>. This will ensure rich data returns from Sysmon, when using adversary emulation services from APTsimulator, as we will, or experiencing the real deal.<br />
With all set up and working you should see results in your Kibana dashboard as seen in <b>Figure 2</b>.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjB_GAlv2r-rfNtoRjwO6Cc163DRZmxi3nCBcpUboRRJCzgomZHyANVz9CHuGJnB6HRyrzC8y1A66i5LB4fQPZSDoiHQ70sfrCpTP9SEJiYZ6jbiSP0TLyiY4aCF_eUtUq9XHAcpA/s1600/SysmonDBprePwn.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="829" data-original-width="1600" height="205" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjB_GAlv2r-rfNtoRjwO6Cc163DRZmxi3nCBcpUboRRJCzgomZHyANVz9CHuGJnB6HRyrzC8y1A66i5LB4fQPZSDoiHQ70sfrCpTP9SEJiYZ6jbiSP0TLyiY4aCF_eUtUq9XHAcpA/s400/SysmonDBprePwn.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 2:</b> Initial HELK Kibana Sysmon dashboard.</td></tr>
</tbody></table>
Now for the showdown. :-) Florian's <a href="https://github.com/NextronSystems/APTSimulator" target="_blank">APTSimulator</a> does some comprehensive emulation to make your systems appear compromised under the following scenarios:<br />
<ul>
<li>POCs: Endpoint detection agents / compromise assessment tools</li>
<li>Test your security monitoring's detection capabilities</li>
<li>Test your SOCs response on a threat that isn't EICAR or a port scan</li>
<li>Prepare an environment for digital forensics classes </li>
</ul>
This is a truly admirable effort, one I advocate for most heartily as a blue team leader. With particular attention to testing your security monitoring's detection capabilities, if you don't do so regularly and comprehensively, you are, quite simply, incomplete in your practice. If you haven't tested and validated, don't consider it detection, it's just a rule with a prayer. APTSimulator can be observed conducting the likes of:<br />
<ol>
<li>Creating typical attacker working directory C:\TMP...</li>
<li>Activating guest user account</li>
<ol>
<li>Adding the guest user to the local administrators group</li>
</ol>
<li>Placing a svchost.exe (which is actually srvany.exe) into C:\Users\Public</li>
<li>Modifying the hosts file</li>
<ol>
<li>Adding update.microsoft.com mapping to private IP address</li>
</ol>
<li>Using curl to access well-known C2 addresses</li>
<ol>
<li>C2: msupdater.com</li>
</ol>
<li>Dropping a Powershell netcat alternative into the APT dir</li>
<li>Executes nbtscan on the local network</li>
<li>Dropping a modified PsExec into the APT dir</li>
<li>Registering mimikatz in At job</li>
<li>Registering a malicious RUN key</li>
<li>Registering mimikatz in scheduled task</li>
<li>Registering cmd.exe as debugger for sethc.exe</li>
<li>Dropping web shell in new WWW directory</li>
</ol>
A couple of notes here.<br />
Download and install APTSimulator from the <a href="https://github.com/NextronSystems/APTSimulator/releases" target="_blank">Releases</a> section of its GitHub pages.<br />
APTSimulator includes <span style="font-family: "courier new" , "courier" , monospace;">curl.exe</span>, <span style="font-family: "courier new" , "courier" , monospace;">7z.exe</span>, and <span style="font-family: "courier new" , "courier" , monospace;">7z.dll</span> in its <span style="font-family: "courier new" , "courier" , monospace;">helpers</span> directory. Be sure that you drop the correct version of 7 Zip for your system architecture. I'm assuming the default bits are 64bit, I was testing on a 32bit VM.<br />
<br />
Let's do a fast run-through with HELK's Kibana Discover option looking for the above mentioned APTSimulator activities. Starting with a search for TMP in the <span style="font-family: "courier new" , "courier" , monospace;">sysmon-*</span> index yields immediate results and strikes #1, 6, 7, and 8 from our APTSimulator list above, see for yourself in <b>Figure 3</b>.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhHpcPSZhxhXf8YPqBUvV_7481ufizT0Ah1gUZool-GIc-0T3XbwWfQBD2QOmkCfvrtpcCr5Bx6LBAVU4ftVB4_8Ps0IPmfkoA-t_FcQZ3bkD5vWQpnb9xduwM3uxHj79LcTcKR0Q/s1600/DiscoverTMP.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="700" data-original-width="1600" height="171" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhHpcPSZhxhXf8YPqBUvV_7481ufizT0Ah1gUZool-GIc-0T3XbwWfQBD2QOmkCfvrtpcCr5Bx6LBAVU4ftVB4_8Ps0IPmfkoA-t_FcQZ3bkD5vWQpnb9xduwM3uxHj79LcTcKR0Q/s400/DiscoverTMP.PNG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 3: </b>TMP, PS nc, nbtscan, and PsExec in one shot</td></tr>
</tbody></table>
Created TMP, dropped a PowerShell netcat, nbtscanned the local network, and dropped a modified PsExec, check, check, check, and check.<br />
How about enabling the guest user account and adding it to the local administrator's group? <b>Figure 4</b> confirms.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjRo3WyS2C-1THf7rRbBtAJp5sPSwjL3wYIpo9vKtqCZdd_KcjSExRJzDqZD3Rht9vnBmKOH4MyQieApHOCGu2YTiNbwza5jZm_txohtslIAsDEi3I3VsZO7QAYH-KVOxrFFhNGlg/s1600/DiscoverGuest.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="603" data-original-width="1414" height="170" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjRo3WyS2C-1THf7rRbBtAJp5sPSwjL3wYIpo9vKtqCZdd_KcjSExRJzDqZD3Rht9vnBmKOH4MyQieApHOCGu2YTiNbwza5jZm_txohtslIAsDEi3I3VsZO7QAYH-KVOxrFFhNGlg/s400/DiscoverGuest.PNG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 4:</b> Guest enabled and escalated</td></tr>
</tbody></table>
Strike #2 from the list. Something tells me we'll immediately find svchost.exe in C:\Users\Public. Aye, <b>Figure 5</b> makes it so.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi1ab0SG3Ri8GxQzWZjqp3VIC0OkT2jW4mae9gHCmZhyuZ2ffmCwgodS0O7BdoNpt200_nI5iBh45UhDLl0nggoC218dVHPvvqlULLJJ26khUabfwV8OfF4z0-NfFwbzkvsM9N3DQ/s1600/DiscoverSvchost.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="400" data-original-width="1121" height="142" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi1ab0SG3Ri8GxQzWZjqp3VIC0OkT2jW4mae9gHCmZhyuZ2ffmCwgodS0O7BdoNpt200_nI5iBh45UhDLl0nggoC218dVHPvvqlULLJJ26khUabfwV8OfF4z0-NfFwbzkvsM9N3DQ/s400/DiscoverSvchost.PNG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 5:</b> I've got your svchost right here</td></tr>
</tbody></table>
Knock #3 off the to-do, including the <span style="font-family: "courier new" , "courier" , monospace;">process.commandline</span>, <span style="font-family: "courier new" , "courier" , monospace;">process.name</span>, and <span style="font-family: "courier new" , "courier" , monospace;">file.creationtime</span> references. Up next, the At job and scheduled task creation. Indeed, see <b>Figure 6</b>.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjMNvI5Fo-FXhjcHx7VL6Ch-AgIyWiClJt6NS2FX8C3r18Ikl1DtEAYpiNvGGcnI0FUDyHjcuGH-0h5eMkgoZmNW_iVp5ZX4ME6_qUn2IThnh0MWQZl2Yx6c7xf3Jz17YVv0vi-mg/s1600/DiscoverTasks.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="849" data-original-width="1600" height="211" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjMNvI5Fo-FXhjcHx7VL6Ch-AgIyWiClJt6NS2FX8C3r18Ikl1DtEAYpiNvGGcnI0FUDyHjcuGH-0h5eMkgoZmNW_iVp5ZX4ME6_qUn2IThnh0MWQZl2Yx6c7xf3Jz17YVv0vi-mg/s400/DiscoverTasks.PNG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 6.</b> tasks OR schtasks</td></tr>
</tbody></table>
I think you get the point, there weren't any misses here. There are, of course, visualization options. Don't forget about Kibana's Timelion feature. Forensicators and incident responders live and die by timelines, use it to your advantage (Figure 7).<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhxIq5T8BU8RbWpf88mNf7nF0rBi8DeUi_HkWxgdRbrS6T_hdg6OBRhO90-iH8hlfJOtkVHCT2tfbGZHXoz9INx6ejoLip3V5rqrGjyz55yxRuuuLAIPbDV_zkyhinAE4Wf9sFKKg/s1600/Timelion.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="392" data-original-width="864" height="180" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhxIq5T8BU8RbWpf88mNf7nF0rBi8DeUi_HkWxgdRbrS6T_hdg6OBRhO90-iH8hlfJOtkVHCT2tfbGZHXoz9INx6ejoLip3V5rqrGjyz55yxRuuuLAIPbDV_zkyhinAE4Wf9sFKKg/s400/Timelion.PNG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 7:</b> Timelion</td></tr>
</tbody></table>
Finally, for this month, under HELK's Kibana Visualize menu, you'll note 34 visualizations. By default, these are pretty basic, but you quickly add value with sub-buckets. As an example, I selected the Sysmon_UserName visualization. Initially, it yielded a donut graph inclusive of malman (my pwned user), SYSTEM and LOCAL SERVICE. Not good enough to be particularly useful I added a sub-bucket to include process names associated with each user. The resulting graph is more detailed and tells us that of the 242 events in the last four hours associated with the malman user, 32 of those were specific to cmd.exe processes, or 18.6% (<b>Figure 8</b>).<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiRWt8CQdQ0SdAaQvXpyHekFsDA9IapnIiuY7khjE8sfSRvZI-KxM0EuArjJ8uNwTZstY-b6BMDEvSHnswe1rn5iQQhXifDmpthusckk2U0MG2iD349jlEAeYY6sm60ITB-KuPq7Q/s1600/Visualize.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="740" data-original-width="1262" height="232" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiRWt8CQdQ0SdAaQvXpyHekFsDA9IapnIiuY7khjE8sfSRvZI-KxM0EuArjJ8uNwTZstY-b6BMDEvSHnswe1rn5iQQhXifDmpthusckk2U0MG2iD349jlEAeYY6sm60ITB-KuPq7Q/s400/Visualize.PNG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 8:</b> Powerful visualization capabilities</td></tr>
</tbody></table>
This has been such a pleasure this month, I am thrilled with both HELK and APTSimulator. The true principles of blue team and detection quality are innate in these projects. The fact that Roberto consider HELK still in alpha state leads me to believe there is so much more to come. Be sure to dig deeply into APTSimulator's <a href="https://github.com/NextronSystems/APTSimulator#advanced-solutions" target="_blank">Advance Solutions</a> as well, there's more than one way to emulate an adversary.<br />
Next month Part 2 will explore the Network side of the equation via the Network Dashboard and related visualizations, as well as HELK integration with Spark, Graphframes & Jupyter notebooks.<br />
Aw snap, more goodness to come, I can't wait.<br />
Cheers...until next time.Russ McReehttp://www.blogger.com/profile/05647342839278416757noreply@blogger.com2tag:blogger.com,1999:blog-20011960.post-53342448927902956102018-01-01T15:28:00.000-08:002018-01-01T16:48:59.438-08:00toolsmith #130 - OSINT with Buscador<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgRkDOAJNjxpayMKr1fTzyIGocuRLg1CsOab5RijSrpisJklUvGBAkHLCfOhB5aXsE5qyJRszeyZ84D2gJ8Qy5wVAvGEQWCywOCUe0WGi0t4Kpv5kkHSk_ff7wyRYHlS1hrf7eRrg/s1600/logo.PNG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="440" data-original-width="340" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgRkDOAJNjxpayMKr1fTzyIGocuRLg1CsOab5RijSrpisJklUvGBAkHLCfOhB5aXsE5qyJRszeyZ84D2gJ8Qy5wVAvGEQWCywOCUe0WGi0t4Kpv5kkHSk_ff7wyRYHlS1hrf7eRrg/s200/logo.PNG" width="154" /></a></div>
First off, Happy New Year! I hope you have a productive and successful 2018. I thought I'd kick off the new year with another exploration of OSINT. In addition to my work as an information security leader and practitioner at Microsoft, I am privileged to serve in Washington's military as a <a href="http://www.militaryterms.info/acronyms/j.shtml" target="_blank">J-2</a> which means I'm part of the intelligence directorate of a joint staff. Intelligence duties in a guard unit context are commonly focused on situational awareness for mission readiness. Additionally, in my unit we combine part of J-6 (command, control, communications, and computer systems directorate of a joint staff) with J-2, making Cyber Network Operations a J-2/6 function. Open source intelligence (OSINT) gathering is quite useful in developing indicators specific to adversaries as well as identifying targets of opportunity for red team and vulnerability assessments. We've discussed numerous OSINT offerings as part of toolsmiths <a href="https://holisticinfosec.blogspot.com/search?q=osint" target="_blank">past</a>, there's no better time than our 130th edition to discuss an OSINT platform inclusive of previous topics such as <a href="https://holisticinfosec.blogspot.com/2013/05/toolsmith-recon-ng.html" target="_blank">Recon-n</a>g, <a href="https://holisticinfosec.blogspot.com/2014/03/toolsmith-spiderfoot.html" target="_blank">Spiderfoot</a>, <a href="https://holisticinfosec.blogspot.com/2011/02/osint-large-email-address-list-imports.html" target="_blank">Maltego</a>, and <a href="https://holisticinfosec.blogspot.com/2017/08/toolsmith-127-osint-with-datasploit.html" target="_blank">Datasploit</a>. <a href="https://inteltechniques.com/buscador/" target="_blank">Buscador</a> is just such a platform and comes from genuine OSINT experts <a href="https://twitter.com/inteltechniques" target="_blank">Michael Bazzell</a> and David Wescott. Buscador is "a Linux Virtual Machine that is pre-configured for online investigators." <a href="https://twitter.com/inteltechniques" target="_blank">Michael</a> is the author of <a href="https://inteltechniques.com/book1.html" target="_blank">Open Source Intelligence Techniques</a> (5th edition) and <a href="https://inteltechniques.com/book2.html" target="_blank">Hiding from the Internet</a> (3rd edition). I had a quick conversation with him and learned that they will have a new release in January (1.2), which will address many issues and add new features. Additionally, it will also revamp Firefox since the release of version 57. You can download Buscador as an OVA bundle for a variety of virtualization options, or as a ISO for USB boot devices or host operating systems. I had Buscador 1.1 up and running on Hyper-V in a matter of minutes after pulling the VMDK out of the OVA and converting it with <a href="https://cloudbase.it/qemu-img-windows/" target="_blank">QEMU</a>. Buscador 1.1 includes numerous tools, in addition to the above mentioned standard bearers, you can expect the following and others:<br />
<ul>
<li>Creepy</li>
<li>Metagoofil</li>
<li>MediaInfo</li>
<li>ExifTool</li>
<li>EmailHarvester</li>
<li>theHarvester</li>
<li>Wayback Exporter</li>
<li>HTTrack Cloner</li>
<li>Web Snapper</li>
<li>Knock Pages</li>
<li>SubBrute</li>
<li>Twitter Exporter</li>
<li>Tinfoleak </li>
<li>InstaLooter </li>
<li>BleachBit </li>
</ul>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div>
Tools are conveniently offered via the menu bar on the UI's left, or can easily be via <i>Show Applications</i>.<br />
To put Buscador through its paces, using myself as a target of opportunity, I tested a few of the tools I'd not prior utilized. Starting with Creepy, <i>the</i> geolocation OSINT tool, I configured the Twitter plugin, one of the four available (Flickr, Google+, Instagram, Twitter) in Creepy, and searched <span style="font-family: "courier new" , "courier" , monospace;">holisticinfosec</span>, as seen in <b>Figure 1</b>.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi4Ml8ZIq9Z8QGyjtdU0fYiTHZ1PtTcR0eycsQZd3IRTa6Rtk5tjsORqwNz4rUuFIfcdV-LC8QxXpFO93UvY41KWDiE_UBhPyvkeU64Kaw7nJgfqfEsbihMOwt0ts2MByCJV3nWMA/s1600/Creepy3.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="835" data-original-width="1177" height="283" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi4Ml8ZIq9Z8QGyjtdU0fYiTHZ1PtTcR0eycsQZd3IRTa6Rtk5tjsORqwNz4rUuFIfcdV-LC8QxXpFO93UvY41KWDiE_UBhPyvkeU64Kaw7nJgfqfEsbihMOwt0ts2MByCJV3nWMA/s400/Creepy3.PNG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 1:</b> Creepy configuration</td><td class="tr-caption" style="text-align: center;"><br /></td><td class="tr-caption" style="text-align: center;"><br /></td><td class="tr-caption" style="text-align: center;"><br /></td><td class="tr-caption" style="text-align: center;"><br /></td><td class="tr-caption" style="text-align: center;"><br /></td></tr>
</tbody></table>
The results, as seen in <b>Figure 2</b>, include some good details, but no immediate location data.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi74aTp4IZhqurppFpTxAdZU399Wkn5jmfKmnPPF3yf_i6BVOL2rKUx_mc95pezf7yP3dmK4bgPOTkozmbsjg-3qfQEyx8Cgb03QBJ_SCr-ZqY4m5z6vQCKuFeSp8Q80L6Z0cIMvw/s1600/Creepy4.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="623" data-original-width="1130" height="220" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi74aTp4IZhqurppFpTxAdZU399Wkn5jmfKmnPPF3yf_i6BVOL2rKUx_mc95pezf7yP3dmK4bgPOTkozmbsjg-3qfQEyx8Cgb03QBJ_SCr-ZqY4m5z6vQCKuFeSp8Q80L6Z0cIMvw/s400/Creepy4.PNG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 2:</b> Creepy results</td></tr>
</tbody></table>
Had I configured the other plugins or was even a user of Flickr or Google+, better results would have been likely. I have location turned off for my Tweets, but my profile does profile does include Seattle. Creepy is quite good for assessing targets who utilize social media heavily, but if you wish to dig more deeply into Twitter usage, check out <a href="https://tinfoleak.com/" target="_blank">Tinfoleak</a>, which also uses geo information available in Tweets and uploaded images. The report for <span style="font-family: "courier new" , "courier" , monospace;">holisticinfosec</span> is seen in <b>Figure 3</b>.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgavRcCFPMntJWrJzBH1op-XBYzYlGEq0QWkcwouisfIWQ47rOLHyD6WgNeZhyphenhyphen4gc-aofX1G8izFtsx3oKPWPAoCLMqwikFQlHEPSwEMCblS6aNglKHP5-wMbwL6x9dQ7CI0biDZA/s1600/tinfoleak.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="798" data-original-width="946" height="336" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgavRcCFPMntJWrJzBH1op-XBYzYlGEq0QWkcwouisfIWQ47rOLHyD6WgNeZhyphenhyphen4gc-aofX1G8izFtsx3oKPWPAoCLMqwikFQlHEPSwEMCblS6aNglKHP5-wMbwL6x9dQ7CI0biDZA/s400/tinfoleak.PNG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 3:</b> Tinfoleak</td></tr>
</tbody></table>
If you're looking for domain enumeration options, you can start with <a href="https://github.com/guelfoweb/knock" target="_blank">Knock</a>. It's as easy as handing it a domain, I did so with <span style="font-family: "courier new" , "courier" , monospace;">holisticinfosec.org</span> as seen in <b>Figure 4</b>, results are in <b>Figure 5</b>.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh7wX1bWovHqYX8fr2hO0XPsamOvW7K4Jpz9FhQU1U6lUBUpIq-uS2Zs7dniBAWmA6Y76nLORVcc4M118uKBnbAvDD6AzpFcJYorQx6TDTlb_dvyTURtjfCiqBtw8p3F8xLzBz2Vg/s1600/knock1.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="537" data-original-width="457" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh7wX1bWovHqYX8fr2hO0XPsamOvW7K4Jpz9FhQU1U6lUBUpIq-uS2Zs7dniBAWmA6Y76nLORVcc4M118uKBnbAvDD6AzpFcJYorQx6TDTlb_dvyTURtjfCiqBtw8p3F8xLzBz2Vg/s320/knock1.PNG" width="272" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 4:</b> Knock run</td></tr>
</tbody></table>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjwReR9J92Wh_EbR_oIhxJiW0A8WCUSxJ3gBbyAD9eSCAYDMFD9V2wUR-ouA9mI03A7c5qXbElzHN6zwCBMj8VA3mkPJSaTXndIHOaE1jaPwIyB5dsGTtRxlW_FPhk2IeoE7aLtbQ/s1600/knock2.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="161" data-original-width="674" height="76" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjwReR9J92Wh_EbR_oIhxJiW0A8WCUSxJ3gBbyAD9eSCAYDMFD9V2wUR-ouA9mI03A7c5qXbElzHN6zwCBMj8VA3mkPJSaTXndIHOaE1jaPwIyB5dsGTtRxlW_FPhk2IeoE7aLtbQ/s320/knock2.PNG" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 5:</b> Knock results</td></tr>
</tbody></table>
Other classics include <a href="https://www.httrack.com/" target="_blank">HTTrack</a> for web site cloning, and <a href="https://www.sno.phy.queensu.ca/~phil/exiftool/" target="_blank">ExifTool</a> for pulling all available metadata from images. HTTrack worked instantly as expected for <span style="font-family: "courier new" , "courier" , monospace;">holisticinfosec.org</span>. I used <a href="https://github.com/althonos/InstaLooter" target="_blank">Instalooter</a>, "a program that can download any picture or video associated from an Instagram profile, without any API access", to grab sample images, then ran pyExifToolGui against them. As a simple experiment, I ran Instalooter against the <span style="font-family: "courier new" , "courier" , monospace;">infosec.memes</span> Instagram account, followed by pyExifToolGui against all the downloaded images, then exported Exif metadata to HTML. If I were analyzing images for associated hashtags the export capability might be useful for an artifacts list.<br />
Finally, one of my absolute favorites is <a href="https://github.com/laramies/metagoofil" target="_blank">Metagoofil</a>, "an information gathering tool designed for extracting metadata of public documents." I did a quick run against my domain, with the doc retrieval parameter set at 50, then reviewed <i>full.txt</i> results (<b>Figure 6</b>), included in the output directory (home/Metagoofil) along with <i>authors.csv</i>, <i>companies.csv</i>, and <i>modified.csv</i>.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjO8j578Kdpz30-b1wmV-bqNIZ0mAgxckkx5AUrHwuXtYFvQ52uAX010BMNmg2T0Ss_HL29hkABG5lpf48-DQ9iK33_nz510lnDA65deKaF8qNX0uW3T5-QNUCF8SqJnfGGn5NH_A/s1600/Metagoofil.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="751" data-original-width="960" height="250" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjO8j578Kdpz30-b1wmV-bqNIZ0mAgxckkx5AUrHwuXtYFvQ52uAX010BMNmg2T0Ss_HL29hkABG5lpf48-DQ9iK33_nz510lnDA65deKaF8qNX0uW3T5-QNUCF8SqJnfGGn5NH_A/s320/Metagoofil.PNG" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 6:</b> Metagoofil results</td></tr>
</tbody></table>
<br />
Metagoofil is <b>extremely</b> useful for gathering target data, I consider it a red team recon requirement. It's a faster, currently maintained offering that has some shared capabilities with <a href="https://holisticinfosec.blogspot.com/2011/03/more-on-osint-with-foca-26-in-toolsmith.html" target="_blank">Foca</a>. It should also serve as a reminder just how much information is available in public facing documents, consider stripping the metadata before publishing. </div>
<div>
<br />
It's fantastic having all these capabilities ready and functional on one distribution, it keeps the OSINT discipline close at hand for those who need regular performance. I'm really looking forward to the Buscador 1.2 release, and better still, I have it on good authority that there is another book on the horizon from Michael. This is a simple platform with which to explore OSINT, remember to be a good citizen though, there is an awful lot that can be learned via these passive means.<br />
Cheers...until next time.</div>
<div>
</div>
Russ McReehttp://www.blogger.com/profile/05647342839278416757noreply@blogger.com0tag:blogger.com,1999:blog-20011960.post-8533611822582481292017-11-19T16:27:00.000-08:002017-11-23T15:52:29.719-08:00toolsmith #129 - DFIR Redefined: Deeper Functionality for Investigators with R - Part 2<i>You can have data without information, but you cannot have information without data.</i> ~Daniel Keys Moran<br />
<br />
Here we resume our discussion of <i>DFIR Redefined: Deeper Functionality for Investigators with R</i> as begun in <a href="https://holisticinfosec.blogspot.com/2017/10/toolsmith-128-dfir-redefined-deeper.html" target="_blank">Part 1</a>.<br />
First, now that my presentation season has wrapped up, I've posted the related <a href="https://github.com/holisticinfosec/DFIR/blob/master/DFIR-SecureWorld2017.pdf" target="_blank">material</a> on the <a href="https://github.com/holisticinfosec/DFIR" target="_blank">Github</a> for this content. I've specifically posted the most recent version as presented at <a href="https://events.secureworldexpo.com/agenda/seattle-wa-2017/" target="_blank">SecureWorld Seattle</a>, which included <a href="https://www.linkedin.com/in/eric-kapfhammer/" target="_blank">Eric Kapfhammer</a>'s contributions and a bit of his forward thinking for next steps in this approach.<br />
When we left off last month I parted company with you in the middle of an explanation of analysis of emotional valence, or the "the intrinsic attractiveness (positive valence) or averseness (negative valence) of an event, object, or situation", using R and the Twitter API. It's probably worth your time to go back and refresh with the end of <a href="https://holisticinfosec.blogspot.com/2017/10/toolsmith-128-dfir-redefined-deeper.html" target="_blank">Part 1</a>. Our last discussion point was specific to the popularity of negative tweets versus positive tweets with a cluster of emotionally neutral retweets, two positive retweets, and a load of negative retweets. This type of analysis can quickly give us better understanding of an attacker collective's sentiment, particularly where the collective is vocal via social media. Teeing off the popularity of negative versus positive sentiment, we can assess the actual words fueling such sentiment analysis. It doesn't take us much R code to achieve our goal using the <a href="https://www.datacamp.com/community/tutorials/r-tutorial-apply-family#family" target="_blank"><i>apply</i> family of functions</a>. The likes of <span style="font-family: "courier new" , "courier" , monospace;">apply</span>, <span style="font-family: "courier new" , "courier" , monospace;">lapply</span>, and <span style="font-family: "courier new" , "courier" , monospace;">sapply</span> allow you to manipulate slices of data from matrices, arrays, lists and data frames in a repetitive way without having to use loops. We use code here directly from Michael Levy, Social Scientist, and his <a href="http://michaellevy.name/blog/conference-twitter/" target="_blank">Playing with Twitter Data</a> post.<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">polWordTables = </span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> sapply(pol, function(p) {</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> words = c(positiveWords = paste(p[[1]]$pos.words[[1]], collapse = ' '), </span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> negativeWords = paste(p[[1]]$neg.words[[1]], collapse = ' '))</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> gsub('-', '', words) # Get rid of nothing found's "-"</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> }) %>%</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> apply(1, paste, collapse = ' ') %>% </span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> stripWhitespace() %>% </span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> strsplit(' ') %>%</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> sapply(table)</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;">par(mfrow = c(1, 2))</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">invisible(</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> lapply(1:2, function(i) {</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> dotchart(sort(polWordTables[[i]]), cex = .5)</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> mtext(names(polWordTables)[i])</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> }))</span><br />
<br />
The result is a tidy visual representation of exactly what we learned at the end of Part 1, results as noted in <b>Figure 1</b>.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiOWTdYDFnZl6Hfz_ok2OQ11EQEaWyAt9wx4RBKWomS1FNwGsvJm5gZvuR-3CA5MUC0y4n1EgZmyBoH3AzzXfaDXTRqugKqpCTQeiVVsUMvmbIzoZUNm8E9oAcqYG81oxQ-Irpjww/s1600/PositiveWordsNegativeWords.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="862" data-original-width="1600" height="215" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiOWTdYDFnZl6Hfz_ok2OQ11EQEaWyAt9wx4RBKWomS1FNwGsvJm5gZvuR-3CA5MUC0y4n1EgZmyBoH3AzzXfaDXTRqugKqpCTQeiVVsUMvmbIzoZUNm8E9oAcqYG81oxQ-Irpjww/s400/PositiveWordsNegativeWords.PNG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 1:</b> Positive vs negative words</td></tr>
</tbody></table>
Content including words such as killed, dangerous, infected, and attacks are definitely more interesting to readers than words such as good and clean. Sentiment like this could definitely be used to assess potential attacker outcomes and behaviors just prior, or in the midst of an attack, particularly in DDoS scenarios. Couple sentiment analysis with the ability to visualize networks of retweets and mentions, and you could zoom in on potential leaders or organizers. The larger the network node, the more retweets, as seen in <b>Figure 2</b>.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2a-aeyyfNZOa-OLFtY1Nb2TJmIwYs8JCqGa3l9f6nOPP5pKLntcmIFDtbzBiGszxlz0g5e5dmWGlKvAhXNdLe23gMUHVKqvndBuMNPoFAUaFmyY3dqhCMgxPeZEWVqFDnWDStNw/s1600/ReTweetNetwork.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="862" data-original-width="1600" height="215" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2a-aeyyfNZOa-OLFtY1Nb2TJmIwYs8JCqGa3l9f6nOPP5pKLntcmIFDtbzBiGszxlz0g5e5dmWGlKvAhXNdLe23gMUHVKqvndBuMNPoFAUaFmyY3dqhCMgxPeZEWVqFDnWDStNw/s400/ReTweetNetwork.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 2:</b> Who is retweeting who?</td></tr>
</tbody></table>
Remember our initial premise, as described in Part 1, was that attacker groups often use associated hashtags and handles, and the minions that want to be "part of" often retweet and use the hashtag(s). Individual attackers either freely give themselves away, or often become easily identifiable or associated, via Twitter. Note that our dominant retweets are for @joe4security, @HackRead, @defendmalware (not actual attackers, but bloggers talking about attacks, used here for example's sake). <b>Figure 3</b> shows us who is mentioning who.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhaOaaHEDQunX2luO_UD7kwulR6mTYZorWevdHikO3Z6xJ4CVFl3axDvJSYl3bwLBMqt4qlhyt7qH1Qg93l6efLEDAfft9d1g6dSx1nng_okQzkkvQbS_PWeiFt2u9QX0HmssGNag/s1600/MentionNetwork.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="862" data-original-width="1600" height="215" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhaOaaHEDQunX2luO_UD7kwulR6mTYZorWevdHikO3Z6xJ4CVFl3axDvJSYl3bwLBMqt4qlhyt7qH1Qg93l6efLEDAfft9d1g6dSx1nng_okQzkkvQbS_PWeiFt2u9QX0HmssGNag/s400/MentionNetwork.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 3:</b> Who is mentioning who?</td></tr>
</tbody></table>
Note that @defendmalware mentions @HackRead. If these were actual attackers it would not be unreasonable to imagine a possible relationship between Twitter accounts that are actively retweeting and mentioning each other before or during an attack. Now let's assume @HackRead might be a possible suspect and you'd like to learn a bit more about possible additional suspects. In reality @HackRead HQ is in Milan, Italy. Perhaps Milan then might be a location for other attackers. I can feed in Twittter handles from my retweet and mentions network above, query the Twitter API with very specific geocode, and lock it within five miles of the center of Milan.<br />
The results are immediate per Figure 4.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhrbWVwIf78FU8Z3pYu_Dm6V5Pherk_oxS5MMZDa2yYVJJWNDfehz555SGQ2VCvItFLH8O92DXY9jwlYg_qChvdG6ubk7f1wG-uvVKoOvD5eGxXz_QpW7kS9QmOPzpa3F7CDW77rg/s1600/GeoLocateResult2.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="852" data-original-width="1425" height="238" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhrbWVwIf78FU8Z3pYu_Dm6V5Pherk_oxS5MMZDa2yYVJJWNDfehz555SGQ2VCvItFLH8O92DXY9jwlYg_qChvdG6ubk7f1wG-uvVKoOvD5eGxXz_QpW7kS9QmOPzpa3F7CDW77rg/s400/GeoLocateResult2.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 4:</b> GeoLocation code and results</td></tr>
</tbody></table>
Obviously, as these Twitter accounts aren't actual attackers, their retweets aren't actually pertinent to our presumed attack scenario, but they definitely retweeted @computerweekly (seen in retweets and mentions) from within five miles of the center of Milan. If @HackRead were the leader of an organization, and we believed that associates were assumed to be within geographical proximity, geolocation via the Twitter API could be quite useful. Again, these are all used as thematic examples, no actual attacks should be related to any of these accounts in any way.<br />
<br />
<b>Fast Frugal Trees (decision trees) for prioritizing criticality</b><br />
<br />
With the abundance of data, and often subjective or biased analysis, there are occasions where a quick, authoritative decision can be quite beneficial. Fast-and-frugal trees (FFTs) to the rescue. FFTs are simple algorithms that facilitate efficient and accurate decisions based on limited information.<br />
<a href="http://nathanieldphillips.com/2016/08/making-fast-good-decisions-with-the-fftrees-r-package/" target="_blank">Nathaniel D. Phillips</a>, PhD created FFTrees for R to allow anyone to easily create, visualize and evaluate FFTs. Malcolm Gladwell has said that "we are suspicious of rapid cognition. We live in a world that assumes that the quality of a decision is directly related to the time and effort that went into making it.” FFTs, and decision trees at large, counter that premise and aid in the timely, efficient processing of data with the intent of a quick but sound decision. As with so much of information security, there is often a direct correlation with medical, psychological, and social sciences, and the use of FFTs is no different. Often, predictive analysis is conducted with logistic regression, used to "describe data and to explain the relationship between one dependent binary variable and one or more nominal, ordinal, interval or ratio-level independent variables." Would you prefer logistic regression or FFTs?<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjof-LtikVUxtHD6jFphyphenhyphenVghxN2dRfL_Zf9WC63dyAJEf1ii4KW95ugElWxk91ngK_YkzkllKnacKiQM_7FKU-rMZyxhZTNgMHI9jbjjLZRYZBwYLJoL8YVwqaOjRqbW62Z4vciYA/s1600/wtf.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="378" data-original-width="546" height="276" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjof-LtikVUxtHD6jFphyphenhyphenVghxN2dRfL_Zf9WC63dyAJEf1ii4KW95ugElWxk91ngK_YkzkllKnacKiQM_7FKU-rMZyxhZTNgMHI9jbjjLZRYZBwYLJoL8YVwqaOjRqbW62Z4vciYA/s400/wtf.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 5:</b> Thanks, I'll take FFTs</td></tr>
</tbody></table>
Here's a text book information security scenario, often rife with subjectivity and bias. After a breach, and subsequent third party risk assessment that generated a ton of <a href="https://www.first.org/cvss/" target="_blank">CVSS</a> data, make a fast decision about what treatments to apply first. Because everyone loves CVSS.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhLECX3XC6mOF252JfdQ_xt_0_UcyRO4GpVSuQTiwnXIr7uoT224gV-LBArw8QQObuN0qSHFXpYiMtLEVFq64q3cZ8S-PxEUAIggN4voDqaLFdnrSbosRIInSmkHf4Tf_m6y54Ptg/s1600/cvss.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1122" data-original-width="993" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhLECX3XC6mOF252JfdQ_xt_0_UcyRO4GpVSuQTiwnXIr7uoT224gV-LBArw8QQObuN0qSHFXpYiMtLEVFq64q3cZ8S-PxEUAIggN4voDqaLFdnrSbosRIInSmkHf4Tf_m6y54Ptg/s200/cvss.png" width="176" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 6:</b> CVSS meh</td></tr>
</tbody></table>
Nothing like a massive table, scored by base, impact, exploitability, temporal, environmental, modified impact, and overall scores, all assessed by a third party assessor who may not fully understand the complexities or nuances of your environment. Let's say our esteemed assessor has decided that there are 683 total findings, of which 444 are non-critical and 239 are critical. Will FFTrees agree? Nay! First, a wee bit of R code.<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">library("FFTrees")</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">cvss </span><- c:="" coding="" csv="" p="" r="" read.csv="" rees=""><span style="font-family: "courier new" , "courier" , monospace;">cvss.fft </span><- data="cvss)</p" fftrees="" formula="critical"><span style="font-family: "courier new" , "courier" , monospace;">plot(cvss.fft, what = "cues")<br />
plot(cvss.fft,<br />
main = "CVSS FFT",<br />
decision.names = c("Non-Critical", "Critical"))</span><br />
<br />
Guess what, the model landed right on <span style="font-family: "courier new" , "courier" , monospace;">impact</span> and <span style="font-family: "courier new" , "courier" , monospace;">exploitability</span> as the most important inputs, and not just because it's logically so, but because of their position when assessed for where they fall in the area under the curve (AUC), where the specific curve is the receiver operating characteristic (ROC). The ROC is a "graphical plot that illustrates the diagnostic ability of a binary classifier system as its discrimination threshold is varied." As for the AUC, accuracy is measured by the area under the ROC curve where an area of 1 represents a perfect test and an area of .5 represents a worthless test. Simply, the closer to 1, the better. For this model and data, <span style="font-family: "courier new" , "courier" , monospace;">impact</span> and <span style="font-family: "courier new" , "courier" , monospace;">exploitability</span> are the most accurate as seen in <b>Figure 7</b>.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjYFyl66VA9edX5AEUjmgGdBykwlcFC5yHstWd2ZrTokOSvxXPNVFJChIbNxuj3mHuk_V6FF9MdtOE37DcRnic9wawIEPc6XUXTwowKtWkCG2hXZ38ugAkTa1V7qaYUXe-REJExAg/s1600/AUC.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="374" data-original-width="715" height="208" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjYFyl66VA9edX5AEUjmgGdBykwlcFC5yHstWd2ZrTokOSvxXPNVFJChIbNxuj3mHuk_V6FF9MdtOE37DcRnic9wawIEPc6XUXTwowKtWkCG2hXZ38ugAkTa1V7qaYUXe-REJExAg/s400/AUC.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 7:</b> Cue rankings prefer impact and exploitability</td></tr>
</tbody></table>
The fast and frugal tree made its decision where impact and exploitability with scores equal or less than 2 were non-critical and exploitability greater than 2 was labeled critical, as seen in <b>Figure 8</b>.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVIVHEK47mIr6DWN1bnGT3evE_n4chdtte1tpDDgMXEGXOs3JGT96aF0Xw6GDxmyL_gdzJyLZ9V3X3U-fC70AevCvP3et6KVUhNMniJLHZcvkh14xVsapqjLIOeNY_y7DsUQiVAw/s1600/FFT1.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="775" data-original-width="1263" height="245" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVIVHEK47mIr6DWN1bnGT3evE_n4chdtte1tpDDgMXEGXOs3JGT96aF0Xw6GDxmyL_gdzJyLZ9V3X3U-fC70AevCvP3et6KVUhNMniJLHZcvkh14xVsapqjLIOeNY_y7DsUQiVAw/s400/FFT1.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 8:</b> The FFT decides</td></tr>
</tbody></table>
Ah hah! Our FFT sees things differently than our assessor. With a 93% average for performance fitting (this is good), our tree, making decisions on <span style="font-family: "courier new" , "courier" , monospace;">impact</span> and <span style="font-family: "courier new" , "courier" , monospace;">exploitability</span>, decides that there are 444 non-critical findings and 222 critical findings, a 17 point differential from our assessor. Can we all agree that mitigating and remediating critical findings can be an expensive proposition? If you, with just a modicum of data science, can make an authoritative decision that saves you time and money without adversely impacting your security posture, would you count it as a win? Yes, that was rhetorical.<br />
</-></-><br />
<div>
Note that the FFTrees function automatically builds several versions of the same general tree that make different error trade-offs with variations in performance fitting and false positives. This gives you the option to test variables and make potentially even more informed decisions within the construct of one model. Ultimately, fast frugal trees make very fast decisions on 1 to 5 pieces of information and ignore all other information. In other words, "<a href="http://journal.sjdm.org/17/17217/jdm17217.pdf" target="_blank">FFTrees are noncompensatory, once they make a decision based on a few pieces of information, no additional information changes the decision.</a>"</div>
<div>
<br /></div>
<div>
Finally, let's take a look at monitoring user logon anomalies in high volume environments with Time Series Regression (TSR). Much of this work comes courtesy of <a href="https://www.linkedin.com/in/eric-kapfhammer/" target="_blank">Eric Kapfhammer</a>, our lead data scientist on our Microsoft Windows and Devices Group Blue Team. The ideal Windows Event ID for such activity is clearly 4624: an account was successfully logged on. This event is typically one of the top 5 events in terms of volume in most environments, and has multiple type codes including Network, Service, and RemoteInteractive.</div>
<div>
User accounts will begin to show patterns over time, in aggregate, including:</div>
<div>
<ul>
<li>Seasonality: day of week, patch cycles, </li>
<li>Trend: volume of logons increasing/decreasing over time</li>
<li>Noise: randomness</li>
</ul>
</div>
<div>
You could look at 4624 with a <a href="http://www.statisticshowto.com/probability-and-statistics/z-score/" target="_blank">Z-score model</a>, which sets a threshold based on the number of standard deviations away from an average count over a given period of time, but this is a fairly simple model. The higher the value, the greater the degree of “anomalousness”.</div>
<div>
Preferably, via Time Series Regression (TSR), your feature set is more rich:</div>
<div>
<ul>
<li>Statistical method for predicting a future response based on the response history (known as autoregressive dynamics) and the transfer of dynamics from relevant predictors</li>
<li>Understand and predict the behavior of dynamic systems from experimental or observational data</li>
<li>Commonly used for modeling and forecasting of economic, financial and biological systems</li>
</ul>
</div>
<div>
How to spot the anomaly in a sea of logon data?</div>
<div>
<ul>
<li><a href="https://grisha.org/blog/2016/01/29/triple-exponential-smoothing-forecasting/" target="_blank">“Triple Exponential Smoothing (Holt-Winters method) is one of many algorithms used to forecast data points in a series, provided that the series is “seasonal”, i.e. repetitive over some period.”</a></li>
<li>Winters improved on Holts double exponential smoothing by adding seasonality in 1960 and published Forecasting sales by exponentially weighted moving averages </li>
</ul>
</div>
<div>
Let's imagine our user, DARPA-549521, in the SUPERSECURE domain, with 90 days of aggregate 4624 Type 10 events by day.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiEXf-m_N5BvwtBpiS0TzwlBvL52clrWIXLSHbWgFb9WxxfwBt8EBxLw_TynBS8Z1CpsnCpPnyav8bqINj84TFb1cnsWUibe66JktaziuXGBKRmPuMDAn0uSP7Jkc5sRAoSAVDTMg/s1600/TSRdata.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="967" data-original-width="1410" height="219" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiEXf-m_N5BvwtBpiS0TzwlBvL52clrWIXLSHbWgFb9WxxfwBt8EBxLw_TynBS8Z1CpsnCpPnyav8bqINj84TFb1cnsWUibe66JktaziuXGBKRmPuMDAn0uSP7Jkc5sRAoSAVDTMg/s320/TSRdata.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 9:</b> User logon data</td></tr>
</tbody></table>
With 210 line of R, including comments, log read, file output, and graphing we can visualize and alert on DARPA-549521's data as seen in <b>Figure 10</b>. </div>
<div>
<br /></div>
<div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjuhKGzs34pHL7zujA7vsFmLUh1PifqGRN3hskmX_9qUwy-b2nRcj2rBeB89YP6feR3DtU5QGHJPjfsE1NjqmWNMCiMZuKIQE3OEuu2IdXwCyIfuQR70WcnPvrcanx9D-X5l8yKWw/s1600/TSR.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="451" data-original-width="916" height="196" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjuhKGzs34pHL7zujA7vsFmLUh1PifqGRN3hskmX_9qUwy-b2nRcj2rBeB89YP6feR3DtU5QGHJPjfsE1NjqmWNMCiMZuKIQE3OEuu2IdXwCyIfuQR70WcnPvrcanx9D-X5l8yKWw/s400/TSR.PNG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 10:</b> User behavior outside the confidence interval</td></tr>
</tbody></table>
<div class="separator" style="clear: both; text-align: center;">
</div>
We can detect when a user’s account exhibits changes in their seasonality as it relates to a confidence interval established (learned) over time. In this case, on 27 AUG 2017, the user topped her threshold of 19 logons thus triggering an exception. Now imagine using this model to spot anomalous user behavior across all users and you get a good feel for the model's power.<br />
Eric points out that there are, of course, additional options for modeling including:<br />
<ul>
<li><b>Seasonal and Trend Decomposition using Loess (STL)</b></li>
<ul>
<li>Handles any type of seasonality ~ can change over time</li>
<li>Smoothness of the trend-cycle can also be controlled by the user</li>
<li>Robust to outliers</li>
</ul>
<li><b>Classification and Regression Trees (CART)</b></li>
<ul>
<li>Supervised learning approach: teach trees to classify anomaly / non-anomaly</li>
<li>Unsupervised learning approach: focus on top-day hold-out and error check</li>
</ul>
<li><b>Neural Networks</b></li>
<ul>
<li>LSTM / Multiple time series in combination</li>
</ul>
</ul>
</div>
<div>
These are powerful next steps in your capabilities, I want you to be brave, be creative, go forth and add elements of data science and visualization to your practice. R and Python are well supported and broadly used for this mission and can definitely help you detect attackers faster, contain incidents more rapidly, and enhance your in-house detection and remediation mechanisms.<br />
All the code as I can share is <a href="https://github.com/holisticinfosec/DFIR" target="_blank">here</a>; sorry, I can only share the TSR example without the source.<br />
All the best in your endeavors!<br />
Cheers...until next time.</div>
Russ McReehttp://www.blogger.com/profile/05647342839278416757noreply@blogger.com1tag:blogger.com,1999:blog-20011960.post-70352003004719524702017-10-17T21:35:00.000-07:002017-10-17T21:35:02.277-07:00McRee added to ISSA's Honor Roll for Lifetime AchievementHolisticInfoSec's Russ McRee was pleased to be added to ISSA International's Honor Roll this month, a lifetime achievement award recognizing an individual's sustained contributions to the information security community, the advancement of the association and enhancement of the professionalism of the membership.<br />
According to the <a href="http://www.prweb.com/releases/2017/10/prweb14777191.htm" target="_blank">press release</a>:<br />
"Russ McRee has a strong history in the information security as a teacher, practitioner and writer. He is responsible for 107 technical papers published in the ISSA Journal under his Toolsmith byline in 2006-2015. These articles represent a body of knowledge for the hands-on practitioner that is second to none. These titles span an extremely wide range of deep network security topics. Russ has been an invited speaker at the key international computer security venues including DEFCON, Derby Con, BlueHat, Black Hat, SANSFIRE, RSA, and ISSA International."<br />
Russ greatly appreciates this honor and would like to extend congratulations to the ten other <a href="http://www.issa.org/?page=Awards" target="_blank">ISSA 2017 award winners</a>. Sincere gratitude to Briana and Erin McRee, Irvalene Moni, Eric Griswold, Steve Lynch, and Thom Barrie for their extensive support over these many years.Russ McReehttp://www.blogger.com/profile/05647342839278416757noreply@blogger.com0tag:blogger.com,1999:blog-20011960.post-9230325274449298192017-10-17T21:14:00.002-07:002017-11-19T15:58:19.176-08:00toolsmith #128 - DFIR Redefined: Deeper Functionality for Investigators with R - Part 1“To competently perform rectifying security service, two critical incident response elements are necessary: information and organization.” ~ Robert E. Davis<br />
<br />
I've been presenting <i>DFIR Redefined: Deeper Functionality for Investigators with R</i> across the country at various conference venues and thought it would helpful to provide details for readers.<br />
The basic premise?<br />
Incident responders and investigators need all the help they can get.<br />
Let me lay just a few statistics on you, from Secure360.org's The Challenges of Incident Response, Nov 2016. Per their respondents in a survey of security professionals:<br />
<ul>
<li>38% reported an increase in the number of hours devoted to incident response</li>
<li>42% reported an increase in the volume of incident response data collected</li>
<li>39% indicated an increase in the volume of security alerts</li>
</ul>
In short, according to Nathan Burke, “It’s just not mathematically possible for companies to hire a large enough staff to investigate tens of thousands of alerts per month, nor would it make sense.”<br />
The 2017 SANS Incident Response Survey, compiled by Matt Bromiley in June, reminds us that “2016 brought unprecedented events that impacted the cyber security industry, including a myriad of events that raised issues with multiple nation-state attackers, a tumultuous election and numerous government investigations.” Further, "seemingly continuous leaks and data dumps brought new concerns about malware, privacy and government overreach to the surface.”<br />
Finally, the survey shows that IR teams are:<br />
<ul>
<li>Detecting the attackers faster than before, with a drastic improvement in dwell time</li>
<li>Containing incidents more rapidly</li>
<li><b>Relying more on in-house detection and remediation mechanisms</b></li>
</ul>
To that end, what concepts and methods further enable handlers and investigators as they continue to strive for faster detection and containment? Data science and visualization sure can’t hurt. How can we be more creative to achieve “deeper functionality”? I propose a two-part series on Deeper Functionality for Investigators with R with the following DFIR Redefined scenarios:<br />
<ul>
<li>Have you been pwned?</li>
<li>Visualization for malicious Windows Event Id sequences</li>
<li>How do your potential attackers feel, or can you identify an attacker via sentiment analysis?</li>
<li>Fast Frugal Trees (decision trees) for prioritizing criticality</li>
</ul>
R is “100% focused and built for statistical data analysis and visualization” and “makes it remarkably simple to run extensive statistical analysis on your data and then <b>generate informative and appealing visualizations with just a few lines of code</b>.”<br />
<br />
With R you can interface with data via file ingestion, database connection, APIs and benefit from a wide range of packages and strong community investment.<br />
From the Win-Vector Blog, per John Mount “not all R users consider themselves to be expert programmers (many are happy calling themselves analysts). R is often used in collaborative projects where there are varying levels of programming expertise.”<br />
I propose that this represents the vast majority of us, we're not expert programmers, data scientists, or statisticians. More likely, we're security analysts re-using code for our own purposes, be it red team or blue team. With a very few lines of R investigators might be more quickly able to reach conclusions.<br />
All the code described in the post can be found on my GitHub.<br />
<br />
<b>Have you been pwned?</b><br />
<br />
This scenario I covered in an earlier post, I'll refer you to <a href="https://holisticinfosec.blogspot.com/2016/07/toolsmith-release-advisory-steph-lockes.html" target="_blank">Toolsmith Release Advisory: Steph Locke's HIBPwned R package</a>.<br />
<br />
<b>Visualization for malicious Windows Event Id sequences</b><br />
<div>
<br /></div>
<div>
<div>
Windows Events by Event ID present excellent sequenced visualization opportunities. A hypothetical scenario for this visualization might include multiple failed logon attempts (4625) followed by a successful logon (4624), then various malicious sequences. A fantastic reference paper built on these principle is <a href="http://www.thinkmind.org/index.php?view=article&articleid=icimp_2016_2_20_30032" target="_blank">Intrusion Detection Using Indicators of Compromise Based on Best Practices and Windows Event Logs</a>. An additional opportunity for such sequence visualization includes Windows processes by parent/children. One R library particularly well suited to is TraMineR: Trajectory Miner for R. This package is for mining, describing and visualizing sequences of states or events, and more generally discrete sequence data. It's primary aim is the analysis of biographical longitudinal data in the social sciences, such as data describing careers or family trajectories, and a BUNCH of other categorical sequence data. Somehow though, the project page somehow fails to mention malicious Windows Event ID sequences. :-) Consider Figures 1 and 2 as retrieved from above mentioned paper. <b>Figure 1</b> are text sequence descriptions, followed by their related Windows Event IDs in <b>Figure 2</b>.</div>
<div>
<br /></div>
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: left; margin-right: 1em; text-align: left;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhZFuamXpLLKRAV7gFM9-wsXkxwiYp-gl39YO6PvGxWqN-Ib0uynJp-Pij1E0JxX0TJCBSzcNbCAoGVacCtYAWC4DSB-5cgBNTrWdbyOajLMT-rmjvql_ZbugvuPWOmeGFGuwJY_Q/s1600/EventSequence.png" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1486" data-original-width="646" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhZFuamXpLLKRAV7gFM9-wsXkxwiYp-gl39YO6PvGxWqN-Ib0uynJp-Pij1E0JxX0TJCBSzcNbCAoGVacCtYAWC4DSB-5cgBNTrWdbyOajLMT-rmjvql_ZbugvuPWOmeGFGuwJY_Q/s640/EventSequence.png" width="275" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 1</b></td></tr>
</tbody></table>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjd-i-zPQF-dP8lCxRkFrD1VE43hiTOMy_Lenv2_qqvO2N0CgMNHnxUEXMQFkXEzcDg8okOsgnbi4ocF4BJmlkJdpDhtvPJHW0SPgq7Yw0X-Ofr3lZv04qoxHFNnj1_p6A3DU6Glw/s1600/EventIDSequence.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="917" data-original-width="460" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjd-i-zPQF-dP8lCxRkFrD1VE43hiTOMy_Lenv2_qqvO2N0CgMNHnxUEXMQFkXEzcDg8okOsgnbi4ocF4BJmlkJdpDhtvPJHW0SPgq7Yw0X-Ofr3lZv04qoxHFNnj1_p6A3DU6Glw/s640/EventIDSequence.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 2</b></td></tr>
</tbody></table>
Taking related log data, parsing and counting it for visualization with R would look something like <b>Figure 3</b>.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgHJl_NrVECaF6f1wDX0FNXGk4su2eFLIr2g0G_Y8-9fA21oAgD-nojO9JKPniLf1ZAV0IqnCuEctkThnpW4P6b5EKFvJcElPrd5AHWyB0uJcWvNPRjg5SLYMZg8qSFhrO8NmGX_Q/s1600/EventIDdata.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="598" data-original-width="1071" height="178" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgHJl_NrVECaF6f1wDX0FNXGk4su2eFLIr2g0G_Y8-9fA21oAgD-nojO9JKPniLf1ZAV0IqnCuEctkThnpW4P6b5EKFvJcElPrd5AHWyB0uJcWvNPRjg5SLYMZg8qSFhrO8NmGX_Q/s320/EventIDdata.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 3</b></td></tr>
</tbody></table>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div>
How much R code does it take to visualize this data with a beautiful, interactive sunburst visualization? Three lines, not counting white space and comments, as seen in <b>the video below</b>.</div>
</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.blogger.com/video.g?token=AD6v5dwM7lvTZTkB6JgbTCrMsMzS1YceKkq3RRvcMUkUH2XID0A6Creo9UcGbHbfGELDJ-AzB1C8CZ4hWtM' class='b-hbp-video b-uploaded' frameborder='0'></iframe></div>
<div>
<br />
A screen capture of the resulting sunburst also follows as <b>Figure 4</b>.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhcH-Vlk8bYCilddEQ4933IuoSn97jmi6cvPoiPO0EJCO5z6OJ8Tcp637gYC8d5pbJOohfoR2h7SrOlqBPNSkfuWcClTJOhKUv2mcVHMi2sOSZSJ0nyKJZ11GexNYn3koauptkryA/s1600/SunburstViz.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="875" data-original-width="1600" height="218" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhcH-Vlk8bYCilddEQ4933IuoSn97jmi6cvPoiPO0EJCO5z6OJ8Tcp637gYC8d5pbJOohfoR2h7SrOlqBPNSkfuWcClTJOhKUv2mcVHMi2sOSZSJ0nyKJZ11GexNYn3koauptkryA/s400/SunburstViz.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 4</b></td></tr>
</tbody></table>
<br /></div>
<div>
<b><br /></b>
<b>How do your potential attackers feel, or can you identify an attacker via sentiment analysis?</b></div>
<div>
<br /></div>
<div>
<div>
Do certain adversaries or adversarial communities use social media? <b>Yes</b></div>
<div>
As such, can social media serve as an early warning system, if not an actual sensor? <b>Yes</b></div>
<div>
Are certain adversaries, at times, so unaware of OpSec on social media that you can actually locate them or correlate against other geo data? <b>Yes</b></div>
</div>
<div>
Some excellent R code to assess Twitter data with includes Jeff Gentry's twitteR and rtweet to interface with the Twitter API.</div>
<div>
<div>
<ul>
<li>twitteR: provides access to the Twitter API. Most functionality of the API is supported, with a bias towards API calls that are more useful in data analysis as opposed to daily interaction.</li>
<li>Rtweet: R client for interacting with Twitter’s REST and stream API’s.</li>
</ul>
</div>
<div>
The code and concepts here are drawn directly from Michael Levy, PhD UC Davis: Playing With Twitter.</div>
</div>
<div>
Here's the scenario: DDoS attacks from hacktivist or chaos groups.</div>
<div>
Attacker groups often use associated hashtags and handles and the minions that want to be "part of" often retweet and use the hashtag(s). Individual attackers either freely give themselves away, or often become easily identifiable or associated, via Twitter. As such, here's a walk-through of analysis techniques that may help identify or better understand the motives of certain adversaries and adversary groups. I don't use actual adversary handles here, for obvious reasons. I instead used a DDoS news cycle and journalist/bloggers handles as exemplars. For this example I followed the trail of the WireX botnet, comprised mainly of Android mobile devices utilized to launch a high-impact DDoS extortion campaign against multiple organizations in the travel and hospitality sector in August 2017. I started with three related hashtags: </div>
<div>
<ol>
<li>#DDOS </li>
<li>#Android </li>
<li>#WireX</li>
</ol>
</div>
<div>
We start with all related Tweets by day and time of day. The code is succinct and efficient, as noted in Figure 5.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgVYFw-HSuo94rAX-UBfP88_7kFAp5H49dDrvVAQH2C_sixr8OklCfOY4kerGyeKeNZTbik2sgswiWdyDf1-UXfMAdBoPKRtyfwEhR6XZnH5-S446Z-DspMkTSP3fyaz1jWv7vrqw/s1600/TweetsDay%2526TimeCode.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="463" data-original-width="1009" height="146" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgVYFw-HSuo94rAX-UBfP88_7kFAp5H49dDrvVAQH2C_sixr8OklCfOY4kerGyeKeNZTbik2sgswiWdyDf1-UXfMAdBoPKRtyfwEhR6XZnH5-S446Z-DspMkTSP3fyaz1jWv7vrqw/s320/TweetsDay%2526TimeCode.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Figure 5</td></tr>
</tbody></table>
The result is a pair of graphs color coded by tweets and retweets per Figure 6.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_1zRLONV2Kqc2kQfHXrnpvO4NeQRBL0vp4dM5Y8H5Q8lnJeKnoAlOOjDI2dgjev-i1aFNYdlskgkkYrAmC7gqnWrBUjUNoPjV5uVrvOSincKPMkSxl-LGE8jV8Fy6yFxeBR1jMw/s1600/TweetsDay%2526Time.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="862" data-original-width="1600" height="215" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_1zRLONV2Kqc2kQfHXrnpvO4NeQRBL0vp4dM5Y8H5Q8lnJeKnoAlOOjDI2dgjev-i1aFNYdlskgkkYrAmC7gqnWrBUjUNoPjV5uVrvOSincKPMkSxl-LGE8jV8Fy6yFxeBR1jMw/s400/TweetsDay%2526Time.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Figure 6</td></tr>
</tbody></table>
<br />
This gives you an immediate feels for spikes in interest by day as well as time of day, particularly with attention to retweets.</div>
<div>
Want to see what platforms potential adversaries might be tweeting from? No problem, code in Figure 7.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhqscx4d4TwMOdPOYb_yXxoqTP4j-F1MEKwFH8Re3-fHw63P-olPTMfQ6LxFKkqRjk_uoqcsIF8ueef04tNc3m2ENcWOcDfPeyfBRWRcexIqEzG9lDvFrOEt-FbodqWjNpmfEF_YA/s1600/TweetsPostedByPlatformCode.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="202" data-original-width="735" height="87" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhqscx4d4TwMOdPOYb_yXxoqTP4j-F1MEKwFH8Re3-fHw63P-olPTMfQ6LxFKkqRjk_uoqcsIF8ueef04tNc3m2ENcWOcDfPeyfBRWRcexIqEzG9lDvFrOEt-FbodqWjNpmfEF_YA/s320/TweetsPostedByPlatformCode.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 7</b></td></tr>
</tbody></table>
<br />
The result in the scenario ironically indicates that the majority of related tweets using our hashtags of interest are coming from Androids per <b>Figure 8</b>. :-)<br />
<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgiGep8ty3ffTPd7MoBIPjMFudNhELyZDR7Zn8IxlJZwFcxNACh569H0JS9muajDH71iWXEif3OHnRlWIplTWHHIw8P0sxcBWvgcosviNnYw6GoAp5oeIdTwbtaI29NantBC96ZJQ/s1600/TweetsPostedByPlatform.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="862" data-original-width="1600" height="215" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgiGep8ty3ffTPd7MoBIPjMFudNhELyZDR7Zn8IxlJZwFcxNACh569H0JS9muajDH71iWXEif3OHnRlWIplTWHHIw8P0sxcBWvgcosviNnYw6GoAp5oeIdTwbtaI29NantBC96ZJQ/s400/TweetsPostedByPlatform.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 8</b></td></tr>
</tbody></table>
</div>
<div>
Now to the analysis of emotional valence, or the "the intrinsic attractiveness (positive valence) or averseness (negative valence) of an event, object, or situation."<br />
<span style="font-family: "courier new" , "courier" , monospace;">orig$text[which.max(orig$emotionalValence)]</span> tells us that the most positive tweet is "A bunch of Internet tech companies had to work together to clean up #WireX #Android #DDoS #botnet."<br />
<span style="font-family: "courier new" , "courier" , monospace;">orig$text[which.min(orig$emotionalValence)]</span> tells us that "Dangerous #WireX #Android #DDoS #Botnet Killed by #SecurityGiants" is the most negative tweet.<br />
Interesting right? Almost exactly the same message, but very different valence.<br />
How do we measure emotional valence changes over the day? Four lines later...<br />
<span style="font-family: "courier new" , "courier" , monospace;">filter(orig, mday(created) == 29) %>%</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> ggplot(aes(created, emotionalValence)) +</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> geom_point() + </span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> geom_smooth(span = .5)</span></div>
<div>
...and we have <b>Figure 9</b>, which tell us that most tweets about WireX were emotionally neutral on 29 AUG 2017, around 0800 we saw one positive tweet, a more negative tweets overall in the morning.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjP7I9ZjaGoz1dzSiyDBqGyP4FX0y6zqN4oGc5R5t8g4rENorwbs2_rTX0qJyHu-rL_Fals2AZP8ghPsMv7vs2KJnJ2u7Fj4Bv4VVwryj8H1QjsB-ReXNsM0_G8HfhaHzn9JJOdDg/s1600/EmotionalValenceChangesOverTheDay.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="862" data-original-width="1600" height="215" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjP7I9ZjaGoz1dzSiyDBqGyP4FX0y6zqN4oGc5R5t8g4rENorwbs2_rTX0qJyHu-rL_Fals2AZP8ghPsMv7vs2KJnJ2u7Fj4Bv4VVwryj8H1QjsB-ReXNsM0_G8HfhaHzn9JJOdDg/s400/EmotionalValenceChangesOverTheDay.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 9</b></td></tr>
</tbody></table>
Another line of questioning to consider: which tweets are more often retweeted, positive or negative? As you can imagine with information security focused topics, negativity wins the day.<br />
Three lines of R...<br />
<span style="font-family: "courier new" , "courier" , monospace;">ggplot(orig, aes(x = emotionalValence, y = retweetCount)) +</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> geom_point(position = 'jitter') +</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> geom_smooth()</span><br />
...and we learn just how popular negative tweets are in <b>Figure 10</b>.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh39sAHusKtFQrHhUBv25E0hXuMbfCPcwsiLSr-m9C3YaZMG354mAa_iQrHV7HML532dS9oSQfAt3jYQf62LnKfgqlDtuEP8P5jrugarAJqfNnrymk-Q4ZHSs1PJiK-4b12rRSO6w/s1600/EmotionalValencePosOrNegRetweetedMore.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="862" data-original-width="1600" height="215" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh39sAHusKtFQrHhUBv25E0hXuMbfCPcwsiLSr-m9C3YaZMG354mAa_iQrHV7HML532dS9oSQfAt3jYQf62LnKfgqlDtuEP8P5jrugarAJqfNnrymk-Q4ZHSs1PJiK-4b12rRSO6w/s400/EmotionalValencePosOrNegRetweetedMore.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 10</b></td></tr>
</tbody></table>
There are cluster of emotionally neutral retweets, two positive retweets, and a load of negative retweets. This type of analysis can quickly lead to a good feel for the overall sentiment of an attacker collective, particularly one with less opsec and more desire for attention via social media.<br />
In Part 2 of DFIR Redefined: Deeper Functionality for Investigators with R we'll explore this scenario further via sentiment analysis and Twitter data, as well as Fast Frugal Trees (decision trees) for prioritizing criticality.<br />
Let me know if you have any questions on the first part of this series via @holisticinfosec or russ at holisticinfosec dot org.<br />
Cheers...until next time. </div>
Russ McReehttp://www.blogger.com/profile/05647342839278416757noreply@blogger.com3tag:blogger.com,1999:blog-20011960.post-51442995309823708892017-09-10T17:29:00.001-07:002017-09-10T17:41:44.961-07:00Toolsmith Tidbit: Windows Auditing with WINspect<a href="https://github.com/A-mIn3/WINspect" target="_blank">WINSpect</a> recently hit the toolsmith radar screen via Twitter, and the author, <a href="https://www.linkedin.com/in/aminemehdaoui/" target="_blank">Amine Mehdaoui</a>, just posted an update a couple of days ago, so no time like the present to give you a walk-through. WINSpect is a Powershell-based Windows Security Auditing Toolbox. According to Amine's GitHub <a href="https://github.com/A-mIn3/WINspect/blob/master/README.md" target="_blank">README</a>, WINSpect "<i>is part of a larger project for auditing different areas of Windows environments. It focuses on enumerating different parts of a Windows machine aiming to identify security weaknesses and point to components that need further hardening. The main targets for the current version are domain-joined windows machines. However, some of the functions still apply for standalone workstations.</i>"<br />
The current script feature set includes audit checks and enumeration for:<br />
<br />
<ul>
<li>Installed security products</li>
<li>World-exposed local filesystem shares</li>
<li>Domain users and groups with local group membership</li>
<li>Registry autoruns</li>
<li>Local services that are configurable by Authenticated Users group members</li>
<li>Local services for which corresponding binary is writable by Authenticated Users group members</li>
<li>Non-system32 Windows Hosted Services and their associated DLLs</li>
<li>Local services with unquoted path vulnerability</li>
<li>Non-system scheduled tasks</li>
<li>DLL hijackability</li>
<li>User Account Control settings</li>
<li>Unattended installs leftovers</li>
</ul>
<div>
I can see this useful PowerShell script coming in quite handy for assessment using the <a href="https://www.cisecurity.org/controls/" target="_blank">CIS Top 20 Security Controls</a>. I ran it on my domain-joined Windows 10 Surface Book via a privileged PowerShell and liked the results.</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiu7ddgcGFlndw-p_G_f3HRkyRKw4wH4kI6NAmAY51qg58acSYGU-8FnfLUqS0ZnyTNDzym1CezahKpCAsUUO2EwoO0LuWuPCSKPuVUF3njdXpvRZ2iFAPGPSx3LdoSUlTxDynxMw/s1600/WINSpect1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1174" data-original-width="1600" height="468" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiu7ddgcGFlndw-p_G_f3HRkyRKw4wH4kI6NAmAY51qg58acSYGU-8FnfLUqS0ZnyTNDzym1CezahKpCAsUUO2EwoO0LuWuPCSKPuVUF3njdXpvRZ2iFAPGPSx3LdoSUlTxDynxMw/s640/WINSpect1.png" width="640" /></a></div>
<div>
<br /></div>
<div>
The script confirms that it's running with admin rights, checks PowerShell version, then inspects Windows Firewall settings. Looking good on the firewall, and WINSpect tees right off on my Window Defender instance and its configuration as well.</div>
<div>
Not sharing a screenshot of my shares or admin users, sorry, but you'll find them enumerated when you run WINSpect.</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgG7BvdYbJ0Jqk6215BuyPgCPiiMW5y3VrSP5kifZNKj7FASSCs1SAYQMXzwJUVq5zCFSvmeAFF0n2Ac3WV7l42p9L2PFeOatOSGBmZufGT0JjB9bkjpv-LwQ6Bo6gtWOOgYeKEyw/s1600/WINSpect2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="882" data-original-width="1600" height="352" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgG7BvdYbJ0Jqk6215BuyPgCPiiMW5y3VrSP5kifZNKj7FASSCs1SAYQMXzwJUVq5zCFSvmeAFF0n2Ac3WV7l42p9L2PFeOatOSGBmZufGT0JjB9bkjpv-LwQ6Bo6gtWOOgYeKEyw/s640/WINSpect2.png" width="640" /></a></div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
WINSpect then confirmed that UAC was enabled, and that it should notify me only apps try to make changes, then checked my registry for autoruns; no worries on either front, all confirmed as expected.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiUu11MSC1U9XRw2LQTdKz5Fd33c6K71sR4cq_cIeCP_3G-1q5PzWS6XAzcuDy2L2NYxYysuh-8RxPZY02ffDduvsTv8sxJ_xiWTq0BzbGMjuXQ_fkFgnztOyZPX2ZlGsfCpSI5YA/s1600/WINSpect3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1377" data-original-width="1600" height="550" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiUu11MSC1U9XRw2LQTdKz5Fd33c6K71sR4cq_cIeCP_3G-1q5PzWS6XAzcuDy2L2NYxYysuh-8RxPZY02ffDduvsTv8sxJ_xiWTq0BzbGMjuXQ_fkFgnztOyZPX2ZlGsfCpSI5YA/s640/WINSpect3.png" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div>
WINSpect wrapped up with a quick check of configurable services, SMSvcHost is normal as part of .NET, even if I don't like it, but the flowExportService doesn't need to be there at all, I removed that a while ago after being really annoyed with it during testing. No user hosted services, and DLL Safe Search is enable...bonus. Finally, no unattended install leftovers, and all the scheduled tasks are normal for my system. Sweet, pretty good overall, thanks WINSpect. :-)</div>
<div>
<br /></div>
<div>
Give it a try for yourself, and keep an eye out for updates. Amine indicates that Local Security Policy controls, administrative shares configs, loaded DLLs, established/listening connections, and exposed GPO scripts on the to-do list. </div>
<div>
Cheers...until next time.</div>
Russ McReehttp://www.blogger.com/profile/05647342839278416757noreply@blogger.com0tag:blogger.com,1999:blog-20011960.post-88576749271690406202017-08-28T10:59:00.000-07:002017-08-28T11:00:48.259-07:00Toolsmith Release Advisory: Magic Unicorn v2.8<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEggYcd3vLvr65-fiMX7Ukt283rPvBrOxojtSW0Z1M7gbwYYMbK4grCqKWkcWczMyPq_bY9nWr2HmY0JEw7YapwCswARy1h4OUtD02Z7vF7VS_LPBDM5I6RTYxd9OAEcwcjrByoULw/s1600/UNICORN.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="185" data-original-width="319" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEggYcd3vLvr65-fiMX7Ukt283rPvBrOxojtSW0Z1M7gbwYYMbK4grCqKWkcWczMyPq_bY9nWr2HmY0JEw7YapwCswARy1h4OUtD02Z7vF7VS_LPBDM5I6RTYxd9OAEcwcjrByoULw/s1600/UNICORN.PNG" /></a></div>
<a href="https://twitter.com/HackingDave" target="_blank">David Kennedy</a> and the <a href="https://twitter.com/TrustedSec" target="_blank">TrustedSec</a> crew have released <a href="https://github.com/trustedsec/unicorn" target="_blank">Magic Unicorn v2.8</a>.<br />
Magic Unicorn is "a simple tool for using a PowerShell downgrade attack and inject shellcode straight into memory, based on <a href="https://twitter.com/mattifestation" target="_blank">Matthew Graeber</a>'s PowerShell attacks and the PowerShell bypass technique presented by Dave and Josh Kelly at <a href="https://www.youtube.com/watch?v=q5pA49C7QJg" target="_blank">Defcon 18</a>.<br />
<br />
<b><u>Version 2.8:</u></b><br />
<ul>
<li>shortens length and obfuscation of unicorn command</li>
<li>removes <span style="font-family: "courier new" , "courier" , monospace;">direct -ec</span> from PowerShell command</li>
</ul>
<div>
<b><u>Usage:</u></b></div>
<div>
"Usage is simple, just run Magic Unicorn (ensure Metasploit is installed and in the right path) and Magic Unicorn will automatically generate a PowerShell command that you need to simply cut and paste the PowerShell code into a command line window or through a payload delivery system."</div>
<div>
<br /></div>
<div>
<br /></div>
Russ McReehttp://www.blogger.com/profile/05647342839278416757noreply@blogger.com0tag:blogger.com,1999:blog-20011960.post-46789473796162362102017-08-16T07:54:00.001-07:002017-08-16T07:54:35.517-07:00Toolsmith #127: OSINT with DatasploitI was reading an interesting <a href="https://motherboard.vice.com/en_us" target="_blank">Motherboard</a> article, <a href="https://motherboard.vice.com/en_us/article/wjj8ax/legal-hacking-tools-can-be-useful-for-journalists-too" target="_blank">Legal Hacking Tools Can Be Useful for Journalists, Too</a>, that includes reference to one of my all time <a href="http://holisticinfosec.blogspot.com/2011/02/osint-large-email-address-list-imports.html" target="_blank">OSINT</a> favorites, <a href="http://holisticinfosec.org/toolsmith/pdf/december2009.pdf" target="_blank">Maltego</a>. <a href="https://twitter.com/josephfcox" target="_blank">Joseph Cox</a>'s article also mentions Datasploit, a 2016 favorite for fellow tools aficionado, <a href="http://www.toolswatch.org/" target="_blank">Toolswatch.org</a>, see <a href="http://www.toolswatch.org/2017/02/2016-top-security-tools-as-voted-by-toolswatch-org-readers/" target="_blank">2016 Top Security Tools as Voted by ToolsWatch.org Readers</a>. Having not yet explored Datasploit myself, this proved to be a grand case of "no time like the present."<br />
Datasploit is "an #OSINT Framework to perform various recon techniques, aggregate all the raw data, and give data in multiple formats." More specifically, as stated on Datasploit documentation page under <a href="http://datasploit.readthedocs.io/en/latest/#why-datasploit" target="_blank">Why Datasploit</a>, it utilizes various Open Source Intelligence (OSINT) tools and techniques found to be effective, and brings them together to correlate the raw data captured, providing the user relevant information about domains, email address, phone numbers, person data, etc. Datasploit is useful to collect relevant information about target in order to expand your attack and defense surface very quickly.<br />
The feature list includes:<br />
<ul>
<li>Automated OSINT on domain / email / username / phone for relevant information from different sources</li>
<li>Useful for penetration testers, cyber investigators, defensive security professionals, etc.</li>
<li>Correlates and collaborate results, shows them in a consolidated manner</li>
<li>Tries to find out credentials, API keys, tokens, sub-domains, domain history, legacy portals, and more as related to the target</li>
<li>Available as single consolidating tool as well as standalone scripts</li>
<li>Performs Active Scans on collected data</li>
<li>Generates HTML, JSON reports along with text files</li>
</ul>
<b>Resources</b><br />
Github: <a href="https://github.com/datasploit/datasploit">https://github.com/datasploit/datasploit</a><br />
Documentation: <a href="http://datasploit.readthedocs.io/en/latest/">http://datasploit.readthedocs.io/en/latest/</a><br />
YouTube: <a href="https://www.youtube.com/watch?v=A9Inz9U-De4" target="_blank">Quick guide to installation and use</a><br />
<br />
<b>Pointers</b><br />
Second, a few pointers to keep you from losing your mind. This project is very much work in progress, lots of very frustrated users filing bugs and wondering where the support is. The team is doing their best, be patient with them, but read through the Github <a href="https://github.com/datasploit/datasploit/issues" target="_blank">issues </a>to be sure any bugs you run into haven't already been addressed.<br />
1) Datasploit does <i>not</i> error gracefully, it just crashes. This can be the result of unmet dependencies or even a missing API key. Do not despair, take note, I'll talk you through it.<br />
2) I suggest, for ease, and best match to documentation, run Datasploit from an Ubuntu variant. Your best bet is to grab <a href="https://www.kali.org/" target="_blank">Kali</a>, VM or dedicated and load it up there, as I did.<br />
3) My installation guidance and recommendations should hopefully get you running trouble free, follow it explicitly.<br />
4) Acquire as many API keys as possible, see further detail below.<br />
<br />
<b>Installation and preparation</b><br />
From Kali bash prompt, in this order:<br />
<br />
<ol>
<li><span style="font-family: Courier New, Courier, monospace;">git clone https://github.com/datasploit/datasploit /etc/datasploit</span></li>
<li><span style="font-family: Courier New, Courier, monospace;">apt-get install libxml2-dev libxslt-dev python-dev lib32z1-dev zlib1g-dev</span></li>
<li><span style="font-family: Courier New, Courier, monospace;">cd /etc/datasploit</span></li>
<li><span style="font-family: Courier New, Courier, monospace;">pip install -r requirements.txt</span></li>
<li><span style="font-family: Courier New, Courier, monospace;">mv config_sample.py config.py</span></li>
<li>With your preferred editor, open <span style="font-family: Courier New, Courier, monospace;">config.py</span> and add API keys for the following at a minimum, they are, for all intents and purposes required, detailed instructions to acquire each are <a href="http://datasploit.readthedocs.io/en/latest/apiGeneration/" target="_blank">here</a>:</li>
<ol>
<li>Shodan API</li>
<li>Censysio ID and Secret</li>
<li>Clearbit API</li>
<li>Emailhunter API</li>
<li>Fullcontact API</li>
<li>Google Custom Search Engine API key and CX ID</li>
<li>Zoomeye Username and Password</li>
</ol>
</ol>
<div>
If, and only if, you've done all of this correctly, you might end up with a running instance of Datasploit. :-) Seriously, this is some of the glitchiest software I've tussled with in quite a while, but the results paid handsomely. Run <span style="font-family: Courier New, Courier, monospace;">python datasploit.py domain.com</span>, where domain.com is your target. Obviously, I ran <span style="font-family: "Courier New", Courier, monospace;">python datasploit.py holisticinfosec.org </span>to acquire results pertinent to your author. </div>
<div>
Datasploit rapidly pulled results as follows:</div>
<div>
211 domain references from Github:</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjMm_pKrWbpSv71GYe61mWScdLg-7ST3MBM5O0xc6HyuJ9eQK0RSoCl59mOzS5aX4v_VpCLSjRq3_8oLWKu04lAkFzd2sPOoVOzXyXbByqFB5t1vqttHRSYjd7I0omKZzqlDo7UJg/s1600/Github.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="472" data-original-width="1106" height="170" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjMm_pKrWbpSv71GYe61mWScdLg-7ST3MBM5O0xc6HyuJ9eQK0RSoCl59mOzS5aX4v_VpCLSjRq3_8oLWKu04lAkFzd2sPOoVOzXyXbByqFB5t1vqttHRSYjd7I0omKZzqlDo7UJg/s400/Github.PNG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Github results</td></tr>
</tbody></table>
<div>
Luckily, no results from Shodan. :-)</div>
<div>
Four results from Paste(s): </div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi20kLTzV55H0ogKEpmq7fYaKqOJFnD6obY-0u6BtxkVLg15e5QLBJ0QWFAP38dZkufvS9Eu9LXQIuVj4nlgMCKfcKbfRFL2A07N5CuTPrziHP-056HCrQVy4SY6waqX5QcYS56kA/s1600/Pastes.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="471" data-original-width="996" height="188" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi20kLTzV55H0ogKEpmq7fYaKqOJFnD6obY-0u6BtxkVLg15e5QLBJ0QWFAP38dZkufvS9Eu9LXQIuVj4nlgMCKfcKbfRFL2A07N5CuTPrziHP-056HCrQVy4SY6waqX5QcYS56kA/s400/Pastes.PNG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Pastebin and Pastie results</td></tr>
</tbody></table>
<div>
Datasploit pulled russ at holisticinfosec dot org as expected, per email harvesting.</div>
<div>
Accurate HolisticInfoSec host location data from Zoomeye:</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEghjg00qRxv0JvIjLCbT-E0Yb1w_6XeoHbVK2L60-4vCmmbS2Ks9RcEBV_S08REn8Hbyih0pdqHQlwKkj3zKJUA26nycG-2wB7eDRyjIHgzjbSJkjoh5NoECz1d6s4xyUzxZxbFug/s1600/Zoomeye.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="350" data-original-width="1255" height="111" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEghjg00qRxv0JvIjLCbT-E0Yb1w_6XeoHbVK2L60-4vCmmbS2Ks9RcEBV_S08REn8Hbyih0pdqHQlwKkj3zKJUA26nycG-2wB7eDRyjIHgzjbSJkjoh5NoECz1d6s4xyUzxZxbFug/s400/Zoomeye.PNG" width="400" /></a></div>
<div>
<br /></div>
<div>
Details regarding HolisticInfoSec sub-domains and page links:</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjjdT2A-kQfVD5t9Weoev8IGmBPctP1E5Qf1r5XP7feKGxyXHD2kj4ZimmXoADU4TYoW3hJpizjDfQk2NNz_uFD4U7fOjFVqcP1LbWclhXQ7_EuXMWNskAqXk5gzXCjkCnuYT6uyw/s1600/Subdomain%2526Pagelinks.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="633" data-original-width="955" height="265" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjjdT2A-kQfVD5t9Weoev8IGmBPctP1E5Qf1r5XP7feKGxyXHD2kj4ZimmXoADU4TYoW3hJpizjDfQk2NNz_uFD4U7fOjFVqcP1LbWclhXQ7_EuXMWNskAqXk5gzXCjkCnuYT6uyw/s400/Subdomain%2526Pagelinks.PNG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Sub-domains and page links</td></tr>
</tbody></table>
<div>
Finally, a good return on DNS records for holisticinfosec.org and, thankfully, no vulns found via <a href="https://www.punkspider.org/" target="_blank">PunkSpider</a>. </div>
<div>
<br /></div>
<div>
DataSploit can also be integrated into other code and called as individual scripts for unique functions. I did a quick run with python emailOsint.py russ@holisticinfosec.org and the results were impressive:</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh8hH-_rDcEQy1-EaRqAlcZnVwG3RWSEtCikAqmwQSHX84NqCytks1456-MOcxJOgBVqwO7yv8b5l2OjISjwh_MBlyHmnVhyphenhyphen7UFwDwm7VeuQKmximRaHhZr46iHUWlLGTTtSPD6ww/s1600/email.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="581" data-original-width="809" height="286" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh8hH-_rDcEQy1-EaRqAlcZnVwG3RWSEtCikAqmwQSHX84NqCytks1456-MOcxJOgBVqwO7yv8b5l2OjISjwh_MBlyHmnVhyphenhyphen7UFwDwm7VeuQKmximRaHhZr46iHUWlLGTTtSPD6ww/s400/email.PNG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Email OSINT</td></tr>
</tbody></table>
<div>
I love that the first query is of <a href="https://twitter.com/troyhunt" target="_blank">Troy Hunt</a>'s <a href="https://haveibeenpwned.com/" target="_blank">Have I Been Pwned</a>. Not sure if you have been? Better check it out. Reminder here, you'll really want to be sure to have as many API keys as possible or you may find these buggy scripts crashing. You'll definitely find yourself compromising between frustration and the rapid, detailed results. I put this offering squarely in the "shows much promise category" if the devs keep focus on it, assess for quality, and handle errors better.</div>
<div>
Give Datasploit a try for sure.</div>
<div>
Cheers, until next time...</div>
Russ McReehttp://www.blogger.com/profile/05647342839278416757noreply@blogger.com0tag:blogger.com,1999:blog-20011960.post-62574530375667503922017-07-07T15:35:00.000-07:002017-07-08T22:47:21.799-07:00Toolsmith #126: Adversary hunting with SOF-ELK<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhWCqXRnnS2deHMsbmAjoXVExwLF4pbg-DI5WdTMSFseDGKbKq3XMalXkpvDKhm0JKdeLXlvZ1ZclX1_bZYS3VDT0uxvwcyuN2In_zVh2nDIxwFISUiURbAcN5sXEXpASlyu7oR-Q/s1600/logo.PNG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" data-original-height="145" data-original-width="452" height="102" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhWCqXRnnS2deHMsbmAjoXVExwLF4pbg-DI5WdTMSFseDGKbKq3XMalXkpvDKhm0JKdeLXlvZ1ZclX1_bZYS3VDT0uxvwcyuN2In_zVh2nDIxwFISUiURbAcN5sXEXpASlyu7oR-Q/s320/logo.PNG" width="320" /></a></div>
As we celebrate Independence Day, I'm reminded that we honor what was, of course, an armed conflict. Today's realities, when we think about conflict, are quite different than the days of lining troops up across the field from each other, loading muskets, and flinging balls of lead into the fray.<br />
We live in a world of asymmetrical battles, often conflicts that aren't always obvious in purpose and intent, and likely fought on multiple fronts. For one of the best reads on the topic, take the well spent time to read TJ O'Connor's <a href="https://www.sans.org/reading-room/whitepapers/attacking/jester-dynamic-lesson-asymmetric-unmanaged-cyber-warfare-33889" target="_blank">The Jester Dynamic: A Lesson in Asymmetric Unmanaged Cyber Warfare</a>. If you're reading this post, it's highly likely that your front is that of 1s and 0s, either as a blue team defender, or as a red team attacker. I live in this world every day of my life as a blue teamer at Microsoft, and as a joint forces cyber network operator. We are faced, each day, with overwhelming, excessive amounts of data, of varying quality, where the answers to questions are likely hidden, but available to those who can dig deeply enough.<br />
New platforms continue to emerge to help us in this cause. At Microsoft we have a variety of platforms that make the process easier for us, but no less arduous, to dig through the data, and the commercial sector continues to expand its offerings. For those with limited budgets and resources, but a strong drive for discovery, that have been outstanding offerings as well. <a href="http://twitter.com/securityonion" target="_blank">Security Onion</a> has been forefront for years, and is under constant development and improvement in the care of <a href="https://twitter.com/dougburks" target="_blank">Doug Burks</a>.<br />
Another emerging platform, to be discussed here, is <a href="https://github.com/philhagen/sof-elk" target="_blank">SOF-ELK</a>, part of the SANS Forensics community, created by <a href="http://for572.com/course" target="_blank">SANS FOR572, Advanced Network Forensics and Analysis</a> author and instructor <a href="https://twitter.com/PhilHagen" target="_blank">Phil Hagen</a>. Count SOF-ELK in the NFAT family for sure, a strong player in the Network Forensic Analysis Tool category.<br />
SOF-ELK has a great <a href="https://github.com/philhagen/sof-elk/blob/master/VM_README.md" target="_blank">README</a>, don't be that person, read it. It's everything you need to get started, in one place. What!? :-)<br />
Better yet, you can <a href="http://for572.com/sof-elk-vm" target="_blank">download</a> a fully realized VM with almost no configuration requirements, so you can hit the ground running. I ran my SOF-ELK instance with VMWare Workstation 12 Pro and no issues other than needing to <i>temporarily</i> <a href="https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2146361" target="_blank">disable Device Guard and Credential Guard</a> on Windows 10.<br />
SOF-ELK offers some good test data to get you started with right out of the gate, in <span style="font-family: "courier new" , "courier" , monospace;">/home/elk_user/exercise_source_logs</span>, including Syslog from a firewall, router, converted Windows events, a Squid proxy, and a server named muse. You can drop these on your SOF-ELK server in the <span style="font-family: "courier new" , "courier" , monospace;">/logstash/syslog/</span> ingestion point for syslog-formatted data. Additionally, utilize <span style="font-family: "courier new" , "courier" , monospace;">/logstash/nfarch/</span> for archived NetFlow output, <span style="font-family: "courier new" , "courier" , monospace;">/logstash/httpd/</span> for Apache logs, <span style="font-family: "courier new" , "courier" , monospace;">/logstash/passivedns/</span> for logs from the passivedns utility, <span style="font-family: "courier new" , "courier" , monospace;">/logstash/plaso/</span> for log2timeline, and <span style="font-family: "courier new" , "courier" , monospace;">/logstash/bro/</span> for, yeah, you guessed it.<br />
I mixed things up a bit and added my own Apache logs for the month of May to <span style="font-family: "courier new" , "courier" , monospace;">/logstash/httpd/</span>. The <i>muse</i> log set in the exercise offering also included a DNS log (named_log), for grins I threw that in the <span style="font-family: "courier new" , "courier" , monospace;">/logstash/syslog/</span> as well just to see how it would play.<br />
Run down a few data rabbit holes with me, I swear I can linger for hours on end once I latch on to something to chase. We'll begin with a couple of highlights from my Apache logs. The SOF-ELK VM comes with three pre-configured dashboards including Syslog, NetFlow, and HTTPD. You can learn more in the start page for the SOF-ELK UI, my instance is <span style="font-family: "courier new" , "courier" , monospace;">http://192.168.50.110:5601/app/kibana</span>. There are three panels, or blocks, for each dashboard's details, at the bottom of the UI. I drilled through to the HTTPD Log Dashboard for this experiment, and immediately reset the time period for analysis (click the time marker in the upper right hand part of the UI). It defaults to the last 15 minutes, if you're reviewing older data it won't show until you adjust to match your time stamps. My data is from the month of May so I selected an <i>absolute</i> window from the beginning of May to its end. You can also select <i>quick</i> or <i>relative</i> time options, it's great to get comfortable here quickly and early. The resulting opening visualizations for me made me very happy, as seen in <b>Figure 1</b>.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiGSoGghy8o6hoUev4lNS93EpsAjIjtLsiALug7NFfVVwLlLeqmSUOIan8-1diTBVeqFF3IROpRVXteD3qJwpT4_6v1IT38xYBjH7gaJDN4VUM-I2cHE5hgc__OKc5HHWwMWS4Prg/s1600/HTTPD2.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="900" data-original-width="1600" height="222" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiGSoGghy8o6hoUev4lNS93EpsAjIjtLsiALug7NFfVVwLlLeqmSUOIan8-1diTBVeqFF3IROpRVXteD3qJwpT4_6v1IT38xYBjH7gaJDN4VUM-I2cHE5hgc__OKc5HHWwMWS4Prg/s400/HTTPD2.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 1:</b> HTTPD Log Dashboard</td></tr>
</tbody></table>
Nice! An event count summary, source ASNs by count (you can immediately see where I scanned myself from work), a fantastic Access Source map, a records graph by HTTP verbs, and one by response codes.<br />
The beauty of these SOF-ELK dashboards is that they're immediately interactive and allow you to drill right in to interesting data points. The holisticinfosec.org website is intentionally flat and includes no active PHP or dynamic content. As a result, my favorite response code as a web application security tester, the 500 error, is notably missing. But, in both the timeline graphs we note a big traffic spike on 8 MAY 2017, which correlates nicely with my above mention scan from work, as noted in the ASN hit count, and seen here in <b>Figure 2</b>.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiAewI0dhyVI0WsFgboDvwtoR0c1zH67dTxs0ksbg5i2c8ZQzHSPBCg4RhuBZRcPEW9BpsLCvC55QGLkZn3YbnJbUk_-34v1PmRAREwK5YV0LhGo1967sYnmCJwKAveHO__hzURAQ/s1600/HTTPD.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="817" data-original-width="1600" height="203" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiAewI0dhyVI0WsFgboDvwtoR0c1zH67dTxs0ksbg5i2c8ZQzHSPBCg4RhuBZRcPEW9BpsLCvC55QGLkZn3YbnJbUk_-34v1PmRAREwK5YV0LhGo1967sYnmCJwKAveHO__hzURAQ/s400/HTTPD.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 2:</b> Traffic spike from scan</td></tr>
</tbody></table>
This visualizes well but isn't really all that interesting or uncommon, particularly given that I know I personally ran the scan, and scans from the Intarwebs are dime a dozen. What did jump out for me though, as seen back in <b>Figure 1</b>, was the presence of four PUT requests. That's usually a "bad thing" where some @$$h@t is trying to drop something on my server. Let's drill in a bit, shall we? After clicking the graph line with the four PUT requests, I quickly learned that two requests came from 204.12.194.234 AS32097: WholeSale Internet in Kansas City, MO and two came from 119.23.233.9 AS37963: Hangzhou Alibaba Advertising in Hangzhou, China. This is well represented in the HTTPD Access Source panel map (<b>Figure 3</b>).<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjWNfPJckHxZ3q8KlnNrQdBXTEoJylpXB9x1HLI1GcpjoHxaR58HBjuECIMD8tbnWCJq3a-oZi2ae57vgckMn1zkMX3hNcQ-lDfcEDSkfCdBlPZ8u8k8CMu0UrfDzvTjdSbfssiCQ/s1600/AccessMap.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="432" data-original-width="782" height="220" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjWNfPJckHxZ3q8KlnNrQdBXTEoJylpXB9x1HLI1GcpjoHxaR58HBjuECIMD8tbnWCJq3a-oZi2ae57vgckMn1zkMX3hNcQ-lDfcEDSkfCdBlPZ8u8k8CMu0UrfDzvTjdSbfssiCQ/s400/AccessMap.PNG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 3:</b> Access Source</td></tr>
</tbody></table>
The PUT request from each included a txt file attempt, specifically <span style="font-family: "courier new" , "courier" , monospace;">dbhvf99151.txt</span> and <span style="font-family: "courier new" , "courier" , monospace;">htjfx99555.txt</span>, both were rejected, redirected (302), and sent to my landing page (200). <br />
Research on the IPs found that 119.23.233.9 was on the "real time suspected malware list as detected by InterServer's intrusion systems" as seen 22 MAY, and 204.12.194.234 was found twice in the AbuseIPDB, flagged on 18 MAY 2017 for Cknife Webshell Detected. Now we're talking. It's common to attempt a remote file include attack or a PUT, with what is a web shell. I opened up SOF-ELK on that IP address and found eight total hits in my logs, all looking for common PHP opportunities with the likes of GET and POST for <span style="font-family: "courier new" , "courier" , monospace;">/plus/mytag_js.php, </span>noted in PHP injection attack attempts.<br />
SOF-ELK made it incredibly easy to hunt down these details, as seen in <b>Figure 4</b> from the HTTPD Discovery panel.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjri5IS9W5NJBAxdBzyFjtbngJCTIQv-gADPab8WQn5U7ZZydUvuQENEViMlph_RA9S9iStt1SL_QD__AyC8ZyHjQrWHX8gb9Jd-X1YpUwcS8OFK54BivWTZYDbZQsxK-9z6_58tw/s1600/Discovery.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="590" data-original-width="761" height="310" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjri5IS9W5NJBAxdBzyFjtbngJCTIQv-gADPab8WQn5U7ZZydUvuQENEViMlph_RA9S9iStt1SL_QD__AyC8ZyHjQrWHX8gb9Jd-X1YpUwcS8OFK54BivWTZYDbZQsxK-9z6_58tw/s400/Discovery.PNG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 4:</b> Discovery</td></tr>
</tbody></table>
That's a groovy little hunting trip through HTTPD logs, but how about a bit of Syslog? I spotted I likely oddity that could be correlated across a number of the exercise logs, we'll see if the correlation is real. You'll notice tabs at the top of your SOF-ELK UI, we'll use Discover for this experiment. I started from the Syslog Dashboard with my time range set broadly on the last two months. 7606 records presented themselves, sliced neatly by hosts and programs, as seen in <b>Figure 5</b>.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgPTQ_CdPVFdaOC4DQobLzjV2OTzZTlzV5m_XtshNdjX7eDRJP_VXq7OCsJ8uyd6fE1Y08kH4zmtRsE4RtIZ9sZbms-Wh2eLaRw5mH8Islv1T20iJTPbkOPmOeLp7-j1v2EiL6uLg/s1600/Syslog.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="757" data-original-width="1267" height="238" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgPTQ_CdPVFdaOC4DQobLzjV2OTzZTlzV5m_XtshNdjX7eDRJP_VXq7OCsJ8uyd6fE1Y08kH4zmtRsE4RtIZ9sZbms-Wh2eLaRw5mH8Islv1T20iJTPbkOPmOeLp7-j1v2EiL6uLg/s400/Syslog.PNG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 5:</b> Syslog Dashboard</td></tr>
</tbody></table>
Squid proxy logs showed the predominance of host entries (6778 or 57.95% of 11,696 to be specific), so I started there. Don' laugh, but I'll often do keyword queries just to see what comes up, sometimes you land a pointer to a good rabbit hole. Within the body of 6778 proxy events, I searched <i>malware</i>. Two hits came back for GET request via a JS redirector to bleepingcomputer.com for your basic how-to based on "random websites opening in Chrome". Ruh-roh.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgbcglpZa0dt9NIIGrY1FZl0j061THVl1ZneYlgtOwv5gG_M7PS4uOn-Jvfxs_TIII2CK_ti9Spr_GZv3Jd7rjcKkzS_l34XcmWRw-bsD-OQyvHi5E8-SCI7BOjGvSKaBy-U_wr7g/s1600/malware.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="900" data-original-width="1268" height="283" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgbcglpZa0dt9NIIGrY1FZl0j061THVl1ZneYlgtOwv5gG_M7PS4uOn-Jvfxs_TIII2CK_ti9Spr_GZv3Jd7rjcKkzS_l34XcmWRw-bsD-OQyvHi5E8-SCI7BOjGvSKaBy-U_wr7g/s400/malware.PNG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 6:</b> Malware keyword</td></tr>
</tbody></table>
More importantly, we have an IP address to pivot on: 10.3.59.53. A search of that IP across the same 6778 Squid logs yielded 3896 entries specific to this IP, and lots to be curious about:<br />
<ul>
<li>datingukrainewomen.com </li>
<li>anastasiadate.com</li>
<li>YouTube videos for hair loss</li>
<li>crowdscience.com for "random pop-ups driving me nuts"</li>
</ul>
Do I need to build this user profile out for you, or are you with me? Proxy logs tell us so much, and are deeply worthy of your blue team efforts to collect and review.<br />
I jumped over to the <span style="font-family: "courier new" , "courier" , monospace;">named_log</span> from the <i>muse</i> host to see what else might reveal itself. Here's where I jumped to Discover, the Splunk-like query functionality inherent to SOF-ELK (and ELK implemetations). I did reductive query to see what other oddities might surface: <span style="font-family: "courier new" , "courier" , monospace;">10.3.59.53 AND dns_query: (*.co.uk OR *.de OR *.eu OR *.info OR *.cc OR *.online OR *.website)</span>. I used these TLDs based on the premise that bots using Domain Generation Algorithms (DGA) will often use the TLDs. See <a href="https://johannesbader.ch/2016/03/the-dga-of-padcrypt/" target="_blank">The DGA of PadCrypt</a> to learn more, as well as ISC Diary handler <a href="https://isc.sans.edu/handler_list.html#john-bambenek" target="_blank">John Bambanek</a>'s <a href="http://osint.bambenekconsulting.com/manual/padcrypt.txt" target="_blank">OSINT logic</a>. The query results were quite satisfying, 29 hits, including a number of clearly randomly generated domains. Those that were most interesting all included the .cc TLD, so I zoomed in further. Down to five hits with <span style="font-family: "courier new" , "courier" , monospace;">10.3.59.53 AND dns_query: *.cc</span>, as seen in <b>Figure 7</b>.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi0OYSBucvEmpH4gtmK8aCoupZFB_ErDD8wjtKwgbY1VpCX6jeAq7eQNbW2v4LjIepgtoahJ8QnO6kvj0AypHFJhE3buXF8DeK9mkjGH2huK3rDlS92qz_IMRTu63_-oek1t2eSww/s1600/CC.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="758" data-original-width="1600" height="188" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi0OYSBucvEmpH4gtmK8aCoupZFB_ErDD8wjtKwgbY1VpCX6jeAq7eQNbW2v4LjIepgtoahJ8QnO6kvj0AypHFJhE3buXF8DeK9mkjGH2huK3rDlS92qz_IMRTu63_-oek1t2eSww/s400/CC.PNG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 7:</b>. CC TLD hits</td></tr>
</tbody></table>
Oh man, not good. I had a hunch now, and went back to the proxy logs with <span style="font-family: "courier new" , "courier" , monospace;">10.3.59.53 AND squid_request:*.exe</span>. And there you have it, ladies and gentlemen, hunch rewarded (<b>Figure 8</b>).<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhvRgpC6Q-0XwDTXAV9TGLwzS8oZqmK-S-ZQT_bgkvtUjl89zspaXGdTw95poYg1OlbDq9qTxYfxsEsu2YKdtB-GrfwbaZ4j2v81Y9SPszok-ZtktqsfeAE-MRGAHsgccHhQN5Ryw/s1600/EXE.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="430" data-original-width="1572" height="108" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhvRgpC6Q-0XwDTXAV9TGLwzS8oZqmK-S-ZQT_bgkvtUjl89zspaXGdTw95poYg1OlbDq9qTxYfxsEsu2YKdtB-GrfwbaZ4j2v81Y9SPszok-ZtktqsfeAE-MRGAHsgccHhQN5Ryw/s400/EXE.PNG" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 8:</b> taxdocs.exe</td></tr>
</tbody></table>
It <span style="font-family: "courier new" , "courier" , monospace;">taxdocs.exe</span> isn't malware, I'm a monkey's uncle. Unfortunately, I could find no online references to these .cc domains or the .exe sample or URL, but you get the point. Given that it's exercise data, Phil may have generated it to entice to dig deeper.<br />
When we think about the IOC patterns for <a href="https://blogs.technet.microsoft.com/mmpc/2017/06/27/new-ransomware-old-techniques-petya-adds-worm-capabilities/" target="_blank">Petya</a>, a hunt like this is pretty revealing. Petya's "initial infection appears to involve a software supply-chain threat involving the Ukrainian company M.E.Doc, which develops tax accounting software, MEDoc". This is not Petya (as far as I know) specifically but we see pattern similarities for sure, one can learn a great deal about the sheep and the wolves. Be the sheepdog!<br />
Few tools better in the free and open source arsenal to help you train and enhance your inner digital sheepdog than SOF-ELK. "<b>I'm a sheepdog. I live to protect the flock and confront the wolf.</b>" ~ LTC Dave Grossman, from <a href="https://www.amazon.com/Combat-Psychology-Physiology-Deadly-Conflict/dp/0964920549" target="_blank">On Combat</a>.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhs0vofB0iNxAaUGeqUtBi-nT24pZoHMHRmENrafhNuRjCnJsy3rY0BnLzruI3YWsNoOMQgFLvqeBm1K1R4vPnCbV9jBXL_LQOoGhkAQsTXm3Chl3RBhIZaDfDSynXHSjVYYQmyTQ/s1600/sheepdog.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="413" data-original-width="466" height="283" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhs0vofB0iNxAaUGeqUtBi-nT24pZoHMHRmENrafhNuRjCnJsy3rY0BnLzruI3YWsNoOMQgFLvqeBm1K1R4vPnCbV9jBXL_LQOoGhkAQsTXm3Chl3RBhIZaDfDSynXHSjVYYQmyTQ/s320/sheepdog.jpg" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
Believe it or not, there's a ton more you can do with SOF-ELK, consider this a primer and a motivator.<br />
I LOVE SOF-ELK. Phil, well done, thank you. Readers rejoice, this is really one of my favorites for toolsmith, hands down, out of the now 126 unique tools discussed over more than ten years. Download the VM, and get to work herding. :-)<br />
Cheers...until next time.Russ McReehttp://www.blogger.com/profile/05647342839278416757noreply@blogger.com1tag:blogger.com,1999:blog-20011960.post-72443700279518935412017-05-21T23:35:00.001-07:002017-05-22T00:15:35.490-07:00Toolsmith #125: ZAPR - OWASP ZAP API R Interface<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
It is my sincere hope that when I say OWASP Zed Attack Proxy (ZAP), you say "Hell, yeah!" rather than "What's that?". This publication has been a longtime supporter, and so many brilliant contibutors and practitioners have lent to OWASP ZAPs growth, in addition to <a href="https://twitter.com/psiinon" target="_blank">@psiinon</a>'s extraordinary project leadership. OWASP ZAP has been 1st or 2nd in the last four years of <a href="https://twitter.com/toolswatch" target="_blank">@ToolsWatch</a> best tool survey's for a damned good reason. OWASP ZAP usage has been well documented and presented over the years, and the <a href="https://github.com/zaproxy/zaproxy/wiki" target="_blank">wiki</a> gives you tons to consider as you explore OWASP ZAP user scenarios.<br />
One of the more recent scenarios I've sought to explore recently is use of the OWASP ZAP API. The OWASP ZAP API is also well <a href="https://github.com/zaproxy/zaproxy/wiki/ApiDetails" target="_blank">documented</a>, more than enough detail to get you started, but consider a few use case scenarios.<br />
First, there is a functional, clean <a href="https://github.com/zaproxy/zaproxy/wiki/ApiDetailsUI" target="_blank">OWASP ZAP API UI</a>, that gives you a viewer's perspective as you contemplate programmatic opportunities. OWASP ZAP API interaction is URL based, and you can invoke both access views and invoke actions. Explore any component and you'll immediately find related views or actions. Drilling into to core via <span style="font-family: "courier new" , "courier" , monospace;">http://localhost:8067/UI/core/</span> (I run OWASP ZAP on 8067, your install will likely be different), gives me a ton to choose from.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi5O6hTJLq9tC7sbxSD-QIFeQIDndqFGCaBBfrhE9ZauOtu7sft3ZXvqPbhSVYdQhGK8uHuZLmSQYGnOjlvjFyOVWSzh_XNDkA7GOXZ8rlQc1cTONHbBzcJwaCG26psu_D7TACtRg/s1600/core.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="167" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi5O6hTJLq9tC7sbxSD-QIFeQIDndqFGCaBBfrhE9ZauOtu7sft3ZXvqPbhSVYdQhGK8uHuZLmSQYGnOjlvjFyOVWSzh_XNDkA7GOXZ8rlQc1cTONHbBzcJwaCG26psu_D7TACtRg/s400/core.png" width="400" /></a></div>
You'll need your API key in order to build queries. You can find yours via <span style="font-family: "courier new" , "courier" , monospace;">Tools | Options | API | API Key</span>. As an example, drill into <span style="font-family: "courier new" , "courier" , monospace;">numberOfAlerts (baseurl )</span>, which gets the number of alerts, optionally filtering by URL. You'll then be presented with the query builder, where you can enter you key, define specific parameter, and decide your preferred output format including JSON, HTML, and XML.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgLAo_nbxkS2Yaqof9eFe8wEC0z3CCm2J8cPvX_SSSn9M2BEQuyxuRmad9GpEwBEeZjizm018dNjv3euUXIkbi4Bswx_QeVwKuhQ_EVKD9w_-ILYo6YnsSuKiKCdiV2LNyWnlChDQ/s1600/coreBuilder.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="160" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgLAo_nbxkS2Yaqof9eFe8wEC0z3CCm2J8cPvX_SSSn9M2BEQuyxuRmad9GpEwBEeZjizm018dNjv3euUXIkbi4Bswx_QeVwKuhQ_EVKD9w_-ILYo6YnsSuKiKCdiV2LNyWnlChDQ/s200/coreBuilder.png" width="200" /></a></div>
Sure, you'll receive results in your browser, this query will provide answers in HTML tables, but these aren't necessarily optimal for programmatic data consumption and manipulation. That said, you learn the most important part of this lesson, a fully populated OWASP ZAP API GET URL: <span style="font-family: "courier new" , "courier" , monospace;">http://localhost:8067/HTML/core/view/numberOfAlerts/?zapapiformat=HTML&apikey=2v3tebdgojtcq3503kuoq2lq5g&formMethod=GET&baseurl=</span><i>. </i><br />
This request would return <br />
<div style="text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgebUJMTbRWmx644z8CDsj_LIBykFIpenjt0QFNHAluKjvXk_rL-1L-_EodvMfpfcBo55e31nmO-fUj1JUrbKfPEXRBy2_E_F0oD9qxES_Qd9uiY-9wGYpVYQ-ijrtxSQ-HnAJ2CA/s1600/HTMLalerts.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgebUJMTbRWmx644z8CDsj_LIBykFIpenjt0QFNHAluKjvXk_rL-1L-_EodvMfpfcBo55e31nmO-fUj1JUrbKfPEXRBy2_E_F0oD9qxES_Qd9uiY-9wGYpVYQ-ijrtxSQ-HnAJ2CA/s1600/HTMLalerts.png" /></a></div>
<br />
<br />
<br />
<br />
in HTML. Very straightforward and easy to modify per your preferences, but HTML results aren't very machine friendly. Want JSON results instead? Just swap out HTML with JSON in the URL, or just choose JSON in the builder. I'll tell you than I prefer working with JSON when I use the OWASP ZAP API via the likes of R. It's certainly the cleanest, machine-consumable option, though others may argue with me in favor of XML.<br />
Allow me to provide you an example with which you can experiment, one I'll likely continue to develop against as it's kind of cool for active reporting on OWASP ZAP scans in flight or on results when session complete. Note, all my code, crappy as it may be, is available for you on <a href="https://github.com/holisticinfosec/ZAPR" target="_blank">GitHub</a>. I mean to say, this is really v0.1 stuff, so contribute and debug as you see fit. It's also important to note that OWASP ZAP needs to be running, either with an active scanning session, or a stored session you saved earlier. I scanned my website, holisticinfosec.org, and saved the session for regular use as I wrote this. You can even see reference to the saved session by location below. <br />
R users are likely aware of <a href="https://shiny.rstudio.com/" target="_blank">Shiny</a>, a web application framework for R, and its dashboard capabilities. I also discovered that <a href="https://ramnathv.github.io/rCharts/" target="_blank">rCharts</a> are designed to work interactively and beautifully within Shiny.<br />
R includes packages that make parsing from JSON rather straightforward, as I learned from<a href="http://zevross.com/blog/2015/02/12/using-r-to-download-and-parse-json-an-example-using-data-from-an-open-data-portal/" target="_blank"> Zev Ross</a>. RJSONIO makes it as easy as <span style="background-color: purple;">fromJSON</span>("<span style="background-color: blue;">http://localhost:8067/JSON/core/view/alerts/?zapapiformat=JSON&apikey=2v3tebdgojtcq3503kuoq2lq5g&formMethod=GET&baseurl=&start=&count=</span>")<br />
to pull data from the OWASP ZAP API. We use the <span style="background-color: purple;">fromJSON</span> "function and its methods to read content in JSON format and de-serializes it into R objects", where the <span style="background-color: blue;">ZAP API URL</span> is that content.<br />
I further parsed alert data using Zev's grabInfo function and organized the results into a data frame (<span style="font-family: "courier new" , "courier" , monospace;">ZapDataDF</span>). I then further sorted the alert content from <span style="font-family: "courier new" , "courier" , monospace;">ZapDataDF</span> into objects useful for reporting and visualization. Within each alert objects are values such as the risk level, the alert message, the <a href="https://cwe.mitre.org/data/index.html" target="_blank">CWE</a> ID, the <a href="http://projects.webappsec.org/w/page/13246974/Threat%20Classification%20Reference%20Grid" target="_blank">WASC</a> ID, and the <a href="https://github.com/zaproxy/zaproxy/wiki/Setting-up-and-developing-new-plugins" target="_blank">Plugin</a> ID. Defining each of these values into parameter useful to R is completed with the likes of:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi7CikCW_tFnPvbVvGgagcfvmaHyPQlmQG2PRBkR4_I4yXDwCvAr5AEpc78EWX5XIOU_14Sq6bC_Xlvfg_d6abteKSpAS5K0pW5JtCM3KhM_4ilhJYon5sTR1Qhp9HrdopOrrJiGw/s1600/ZapDataDF.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="77" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi7CikCW_tFnPvbVvGgagcfvmaHyPQlmQG2PRBkR4_I4yXDwCvAr5AEpc78EWX5XIOU_14Sq6bC_Xlvfg_d6abteKSpAS5K0pW5JtCM3KhM_4ilhJYon5sTR1Qhp9HrdopOrrJiGw/s400/ZapDataDF.png" width="400" /></a></div>
I then combined all those results into another data frame I called <span style="font-family: "courier new" , "courier" , monospace;">reportDF</span>, the results of which are seen in the figure below.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgX6cdaISwlSZuvHQsvUaQQFkV1uvc6ovPlhyphenhyphen3UcwQ8wX4CVeu8ly-C9Zkd-puP4WgOH_SnqzzcQFjFqB0N74m-JMENCma2015fh_jU2vIIWVQq2GAlOuDP3J00YWVph4A3cH8BhQ/s1600/reportDF.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="110" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgX6cdaISwlSZuvHQsvUaQQFkV1uvc6ovPlhyphenhyphen3UcwQ8wX4CVeu8ly-C9Zkd-puP4WgOH_SnqzzcQFjFqB0N74m-JMENCma2015fh_jU2vIIWVQq2GAlOuDP3J00YWVph4A3cH8BhQ/s320/reportDF.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>reportDF results</b></td></tr>
</tbody></table>
Now we've got some content we can pivot on.<br />
First, let's summarize the findings and present them in their resplendent glory via ZAPR: OWASP ZAP API R Interface.<br />
Code first, truly simple stuff it is: <br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigfO5nGpvYoGyKPTxIUdE3-ZX27j5Esj_ZzGa6ao2iuwbjsoLRO_xQs9YMKIoFRl9LpzBfrYk0KWzVjo_oPbA1rhJueKZnK9bS8S8sSgDh6ntWI7HidX34bH_fepUhk3mjrf6kyA/s1600/summary.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="105" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigfO5nGpvYoGyKPTxIUdE3-ZX27j5Esj_ZzGa6ao2iuwbjsoLRO_xQs9YMKIoFRl9LpzBfrYk0KWzVjo_oPbA1rhJueKZnK9bS8S8sSgDh6ntWI7HidX34bH_fepUhk3mjrf6kyA/s640/summary.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Summary overview API calls</b></td></tr>
</tbody></table>
<br />
You can see that we're simply using RJSONIO's fromJSON to make specific ZAP API call. The results are quite tidy, as seen below.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgjmBV0Gn9XWeWQJKZOsxfJ0CoSerdyu7PkJZa9Y8bS4xfxzLkNO1zYHyyQgioCJV3uhHF2Pt2DczJiUCSX7udrMYr8REHEx5dW9YfOCG1GwjW8gMDZdF9Rf9Fo4fVGoCHtVLk7Ow/s1600/overview.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="270" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgjmBV0Gn9XWeWQJKZOsxfJ0CoSerdyu7PkJZa9Y8bS4xfxzLkNO1zYHyyQgioCJV3uhHF2Pt2DczJiUCSX7udrMYr8REHEx5dW9YfOCG1GwjW8gMDZdF9Rf9Fo4fVGoCHtVLk7Ow/s320/overview.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>ZAPR Overview</b></td></tr>
</tbody></table>
One of my favorite features in Shiny is the <span style="font-family: "courier new" , "courier" , monospace;">renderDataTable</span> function. When utilized in a Shiny dashboard, it makes filtering results a breeze, and thus is utilized as the first real feature in ZAPR. The code is tedious, review or play with it from GitHub, but the results should speak for themselves. I filtered the view by <a href="https://cwe.mitre.org/data/definitions/89.html" target="_blank">CWE ID 89</a>, which in this case is a bit of a false positive, I have a very flat web site, no database, thank you very much. Nonetheless, good to have an example of what would definitely be a high risk finding.<br />
<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh4J6GRh61UK3U5-CF0n9FGjxEgw1NLDU5YRIR-Cm79kmC8mx3fV6iZ0aH_A3GOxHWs11LmX6T1IZB_uN0OFcTlWLPwEZNlTOgEbSq5zv9HPG2ZY0KVife5H4R4a67CgrAilGc9bQ/s1600/alertFilter.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="110" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh4J6GRh61UK3U5-CF0n9FGjxEgw1NLDU5YRIR-Cm79kmC8mx3fV6iZ0aH_A3GOxHWs11LmX6T1IZB_uN0OFcTlWLPwEZNlTOgEbSq5zv9HPG2ZY0KVife5H4R4a67CgrAilGc9bQ/s400/alertFilter.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Alert filtering</b></td></tr>
</tbody></table>
<br />
Alert filtering is nice, I'll add more results capabilities as I develop this further, but visualizations are important too. This is where <a href="https://ramnathv.github.io/rCharts/" target="_blank">rCharts</a> really come to bear in Shiny as they are interactive. I've used the simplest examples, but you'll get the point. First, a few, wee lines of R as seen below.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjBYhWWg-Hti-KdyVaLG-xftGfCRJaoHvABsAhpuoXQK6g4QozavJ6wrEaZuwXSp_eJiy0EaU6kI4_uhVdEOQSb7g6VAE6CN-QK0vUVXattKQmudq5C7qNiM3pY7LrqD2o_bcDM_w/s1600/chartsCode.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="111" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjBYhWWg-Hti-KdyVaLG-xftGfCRJaoHvABsAhpuoXQK6g4QozavJ6wrEaZuwXSp_eJiy0EaU6kI4_uhVdEOQSb7g6VAE6CN-QK0vUVXattKQmudq5C7qNiM3pY7LrqD2o_bcDM_w/s400/chartsCode.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Chart code</b></td></tr>
</tbody></table>
The results are much more satisfying to look at, and allow interactivity. <a href="https://github.com/ramnathv" target="_blank">Ramnath Vaidyanathan</a> has done really nice work here. First, OWASP ZAP alerts pulled via the API are counted by risk in a bar chart.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjsjLKXuo8jxXrg5UfUqqYhSlyaNf3-4ntsu2wJ8Rv1vY8PmcNakIwOfzUqUYh-73JbXmCndxtHYZWe0kesWMy7YyeNmE1keVZJLE5MXaiUFcMLvNAWzAipWtEPQDf2bthd4xB7vw/s1600/bar.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="216" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjsjLKXuo8jxXrg5UfUqqYhSlyaNf3-4ntsu2wJ8Rv1vY8PmcNakIwOfzUqUYh-73JbXmCndxtHYZWe0kesWMy7YyeNmE1keVZJLE5MXaiUFcMLvNAWzAipWtEPQDf2bthd4xB7vw/s400/bar.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Alert counts</b></td></tr>
</tbody></table>
<br />
As I moused over Medium, we can see that there were specifically 17 results from my OWASP ZAP scan of holisticinfosec.org. <br />
Our second visualization are the CWE ID results by count, in an oft disdained but interactive pie chart (yes, I have some work to do on layout).<br />
<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgmk3T28s7vukAiLNiUcgo3voR6LIk7y_dv4WTUHWBZJR32u1Le_ssS-L2mTUkhwc7vCmlzdXrYDfIjkkkJqnfpTQC7f2ve71XiFilD-OR3CgxQFibRtfnT3dEiOKqjUh-gPPZQsQ/s1600/pie.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="155" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgmk3T28s7vukAiLNiUcgo3voR6LIk7y_dv4WTUHWBZJR32u1Le_ssS-L2mTUkhwc7vCmlzdXrYDfIjkkkJqnfpTQC7f2ve71XiFilD-OR3CgxQFibRtfnT3dEiOKqjUh-gPPZQsQ/s400/pie.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>CWE IDs by count</b></td></tr>
</tbody></table>
<br />
As we learned earlier, I only had one CWE ID 89 hit during the session, and the visualization supports what we saw in the data table.<br />
The possibilities are endless to pull data from the OWASP ZAP API and incorporate the results into any number of applications or report scenarios. I have a feeling there is a rich opportunity here with <a href="https://powerbi.microsoft.com/en-us/" target="_blank">PowerBI</a>, which I intend to explore. All the code is <a href="https://github.com/holisticinfosec/ZAPR" target="_blank">here</a>, along with the OWASP ZAP session I refer to, so you can play with it for yourself. You'll need <a href="https://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Project" target="_blank">OWASP ZAP</a>, <a href="https://mran.microsoft.com/open/" target="_blank">R</a>, and <a href="https://www.rstudio.com/" target="_blank">RStudio</a> to make it all work together, let me know if you have questions or suggestions.<br />
Cheers, until next time. Russ McReehttp://www.blogger.com/profile/05647342839278416757noreply@blogger.com0tag:blogger.com,1999:blog-20011960.post-37497523049646452412017-03-26T21:49:00.001-07:002017-03-26T21:49:08.619-07:00Toolsmith #124: Dripcap - Caffeinated Packet Analyzer<a href="https://dripcap.org/" target="_blank">Dripcap</a> is a modern, graphical packet analyzer based on Electron. <br />
<a href="https://electron.atom.io/" target="_blank">Electron</a>, you say? "<i>Electron is a framework for creating native applications with web technologies like JavaScript, HTML, and CSS. It takes care of the hard parts so you can focus on the core of your application.</i>"<br />
We should all be deeply familiar with the venerable <a href="https://www.wireshark.org/" target="_blank">Wireshark</a>, as it has long been the forerunner for packet analysts seeking a graphical interface to their PCAPs. Occasionally though, it's interesting to explore alternatives. I've long loved <a href="http://holisticinfosec.blogspot.com/2011/11/tool-review-networkminer-professional.html" target="_blank">NetworkMiner</a>, and the likes of <a href="https://www.microsoft.com/en-us/download/details.aspx?id=44226" target="_blank">Microsoft Message Analyzer</a> and <a href="http://holisticinfosec.blogspot.com/2011/06/toolsmith-xplico.html" target="_blank">Xplico</a> each have unique benefits.<br />
For basic users comfortabel with Wireshark, you'll likely find Dripcap somewhat rudimentary at this stage, but it does give you opportunities to explore packet captures at fundamental levels and learn without some of the feature crutches more robust tools offer.<br />
However, for JavaScript developers, Dripcap opens up a whole other world of possibilities. Give the <a href="https://docs.dripcap.org/tutorial-create-ntp-dissector-package.html" target="_blank">Create NTP dissector package</a> tutorial a read, you can create, then <a href="https://docs.dripcap.org/publish-package.html" target="_blank">publish</a> and load dissector (and others) packages of your choosing.<br />
<br />
<b>Installation</b> <br />
I built Dripcap from source on Windows as follows, using <a href="https://chocolatey.org/" target="_blank">Chocolatey</a>.<br />
From a administrator PowerShell prompt (ensure Get-ExecutionPolicy is not Restricted), execute the following (restart your admin PS prompt after #2):<br />
<ol>
<li><span style="font-family: "courier new" , "courier" , monospace;">iwr https://chocolatey.org/install.ps1 -UseBasicParsing | iex</span></li>
<li><span style="font-family: "courier new" , "courier" , monospace;">choco install git make jq nodejs</span></li>
<li><span style="font-family: "courier new" , "courier" , monospace;">git clone https://github.com/dripcap/dripcap.git</span></li>
<li><span style="font-family: "courier new" , "courier" , monospace;">cd dripcap</span></li>
<li><span style="font-family: "courier new" , "courier" , monospace;">npm install -g gulp node-gyp babel-cli</span></li>
<li><span style="font-family: "courier new" , "courier" , monospace;">npm install</span></li>
<li><span style="font-family: "courier new" , "courier" , monospace;">gulp</span></li>
</ol>
Step 1 installs Chocolatey, step 2 uses Chocolatey to install tools, step 3 clones Dripcap, steps 5 & 6 install packages, and step 7 builds it all. <br />
Execute <span style="font-family: "courier new" , "courier" , monospace;">dripcap</span>, and you should be up and running.<br />
You can also use npm, part of Node.js' package ecosystem to install Dripcap CLI with <span style="font-family: "Courier New",Courier,monospace;">npm install -g dripcap</span>, or just download dripcap-windows-amd64.exe from Dripcap <a href="https://github.com/dripcap/dripcap/releases" target="_blank">Releases</a>.<br />
<br />
<b>Experiment </b><br />
<br />
I'll walk you through packet carving of sorts with Dripcap. One of Dripcap's strongest features is its filtering capabilities. I used an old PCAP with an Operation Aurora Internet Explorer exploit (<a href="https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2010-0249" target="_blank">CVE-2010-0249</a>) payload for this tool test.<br />
<span style="font-family: "courier new" , "courier" , monospace;">Ctrl+O</span> will <span style="font-family: "courier new" , "courier" , monospace;">Import Pcap File</span> for you.<br />
<br />
Click <span style="font-family: "courier new" , "courier" , monospace;">Developer</span>, then <span style="font-family: "courier new" , "courier" , monospace;">Toggle Log Panel</span> for full logging.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg3km6KkmtCctnAx13U-iXu3GnJ-QftYq66ikasJaJzdLICf0gPwWqFdpNTVfhoFw391MbMSXt3yvgRHqbgfJ9P_bd0DPUiIJNtaQFzgGMOVSj1LKeVMtwsIsB5_qHzcyBU5YRWYQ/s1600/Dripcap.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="142" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg3km6KkmtCctnAx13U-iXu3GnJ-QftYq66ikasJaJzdLICf0gPwWqFdpNTVfhoFw391MbMSXt3yvgRHqbgfJ9P_bd0DPUiIJNtaQFzgGMOVSj1LKeVMtwsIsB5_qHzcyBU5YRWYQ/s320/Dripcap.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 1:</b> Dripcap</td></tr>
</tbody></table>
You'll note four packets with lengths of 1514, as seen in <b>Figure 1</b>. Exploring the first of these packets indicates just what we'd expect: an Ethernet MTU (maximum transmission unit) of 1500 bytes, and a TCP payload of 1460 bytes, leaving 40 bytes for our header (20 byte IP and 20 byte TCP).<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiFX6higeOZbDDC9hlvDT2AqJl-tFQ9TiN4zQmQKBVB1Zbn_W9uiFoCHBd0LLS_slMSxm9O8zucYc5TEhxIaNJ-mx9rNI27FmKdBmW-G7euY5YNTHD_g-e7qCKC4xokAOW_l8L0bw/s1600/payload.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="154" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiFX6higeOZbDDC9hlvDT2AqJl-tFQ9TiN4zQmQKBVB1Zbn_W9uiFoCHBd0LLS_slMSxm9O8zucYc5TEhxIaNJ-mx9rNI27FmKdBmW-G7euY5YNTHD_g-e7qCKC4xokAOW_l8L0bw/s320/payload.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 2:</b> First large packet</td></tr>
</tbody></table>
Hovering your mouse over the TCP details in the UI will highlight all the TCP specific data, but you can take such actions a step further. First, let's filter down to just the large packets with <span style="font-family: "Courier New",Courier,monospace;">tcp.payload.length == 1460</span>.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZjOg0LbhcjYCZ3k5t6_DzrcpdZXQgIfczpg1gGThnHXinwahZ77FO-B0dNd0mkzW9yg002iIZjul-r2c4Qc9RQC4DWFksUvhGd0fkCpzcj0StCQRYg8XC1M5o60xW1mH0ftIngA/s1600/filtered.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="63" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZjOg0LbhcjYCZ3k5t6_DzrcpdZXQgIfczpg1gGThnHXinwahZ77FO-B0dNd0mkzW9yg002iIZjul-r2c4Qc9RQC4DWFksUvhGd0fkCpzcj0StCQRYg8XC1M5o60xW1mH0ftIngA/s320/filtered.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 3:</b> Filtered packets</td></tr>
</tbody></table>
With our view reduced we can do some down and dirty carving pretty easily with Dripcap. In each of the four filtered packets I hovered over Payload 1460 bytes as seen in <b>Figure 4</b>, which highlighted the payload-specific hex. I then used the mouse to capture the highlighted content and, using Dripcap's Edit and Copy, grabbed only that payload-specific hex and pasted it to a text file.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEggQCFNlGUD8NDFHDurtMIrWsuVCcShnUwfth43dhCvXXl6OT61RnGZ3qeyhwmTDTquPk3QOz2SnkEFIGvs0vyLsVzdrhj-D0JFe8zPRQcCYm-foAfGGDar_ZBmXc5Y6dVvVpww0w/s1600/hexPayload.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="258" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEggQCFNlGUD8NDFHDurtMIrWsuVCcShnUwfth43dhCvXXl6OT61RnGZ3qeyhwmTDTquPk3QOz2SnkEFIGvs0vyLsVzdrhj-D0JFe8zPRQcCYm-foAfGGDar_ZBmXc5Y6dVvVpww0w/s320/hexPayload.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 4:</b> Hex payload</td></tr>
</tbody></table>
I did this with each of these four packets and copied content, one hex blob after the other, into my text file, in tight, seamless sequence. I then used Python Tools for Visual Studio to do a quick hexadecimal to ASCII translation as easily as b<span style="font-family: "Courier New",Courier,monospace;">ytearray.fromhex("my hex snippet here").decode()</span>. The result, in <b>Figure 5</b>, shows the resulting JavaScript payload the exploits <a href="https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2010-0249" target="_blank">CVE-2010-0249</a>.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj1tGKv-59Fo1s45cf3Lo77oT164RzCSzUlMFauf-XB3xh9r36TTN6xPSpy-Jqb8KBP67lfpeKwedw-lXpkmhNcKT6AAjHjCjhHXVoLqKUzuYN1EtHGruhyphenhyphenKreiIsbaIBdjwNmRXg/s1600/ASCII.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="66" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj1tGKv-59Fo1s45cf3Lo77oT164RzCSzUlMFauf-XB3xh9r36TTN6xPSpy-Jqb8KBP67lfpeKwedw-lXpkmhNcKT6AAjHjCjhHXVoLqKUzuYN1EtHGruhyphenhyphenKreiIsbaIBdjwNmRXg/s320/ASCII.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 5:</b> ASCII results</td></tr>
</tbody></table>
You can just as easily use online converters as well. I saved the ASCII results to a text file in a directory which I had excluded from my anti-malware protection. After uploading the file to VirusTotal as payload.txt, my expectations were confirmed: 32 of 56 AV providers <a href="https://virustotal.com/en/file/40681c22798a76d86d82300561594c961103d1bd291a68f7cefae43e5b13187b/analysis/1490565035/" target="_blank">detected</a> the file as the likes of Exploit:JS/Elecom.D or, more to the point, Exploit.JS.Aurora.a.<br />
<br />
<b>In closing</b> <br />
Perhaps not the most elegant method, but it worked quickly and easily with Dripcap's filtering and editing functions. I hope to see this tool, and its community, continue to grow. Build dissector packages, create themes, become part of the process, it's always good to see alternatives in available to security practitioners.<br />
Cheers...until next time. Russ McReehttp://www.blogger.com/profile/05647342839278416757noreply@blogger.com0tag:blogger.com,1999:blog-20011960.post-47052958912856105362017-02-19T23:38:00.000-08:002017-02-19T23:46:21.617-08:00Toolsmith Release Advisory: Sysmon v6 for Securitay <b><span style="background-color: yellow;"><a href="https://technet.microsoft.com/en-us/sysinternals/sysmon" target="_blank">Sysmon</a> just keeps getting better.</span></b><br />
I'm thrilled to mention that <a href="https://twitter.com/markrussinovich" target="_blank">@markrussinovich</a> and <a href="https://twitter.com/mxatone" target="_blank">@mxatone</a> have released Sysmon v6.<br />
When I first <a href="http://holisticinfosec.blogspot.com/2015/02/toolsmith-sysmon-20-eventviz.html" target="_blank">discussed</a> Sysmon v2 two years ago it offered users seven event types.<br />
Oh, how it's grown in the last two years, now with 19 events, plus an error event.<br />
From Mark's <a href="https://onedrive.live.com/view.aspx?resid=D026B4699190F1E6!2843&ithint=file%2cpptx&app=PowerPoint&authkey=!AMvCRTKB_V1J5ow" target="_blank">RSA presentation</a> we see the current listing with the three new v6 events highlighted.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiekFAYJA6OHQf37CDy7qK4xO3iBVs4urSIFSUBGS5FNUOzciIGjJMNd_oS_EN3ThlSyum5QGm6a0PGyngvSP-fKBrrejrlkcgw-6KekURgnQR4NkcdREQid40mUsAWv3AaVlUawg/s1600/Events.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="213" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiekFAYJA6OHQf37CDy7qK4xO3iBVs4urSIFSUBGS5FNUOzciIGjJMNd_oS_EN3ThlSyum5QGm6a0PGyngvSP-fKBrrejrlkcgw-6KekURgnQR4NkcdREQid40mUsAWv3AaVlUawg/s400/Events.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Sysmon Events</b></td></tr>
</tbody></table>
<br />
<i>"This release of Sysmon, a background monitor that records activity to
the event log for use in security incident detection and forensics,
introduces an option that displays event schema, adds an event for
Sysmon configuration changes, interprets and displays registry paths in
their common format, and adds named pipe create and connection events."</i><br />
<br />
Mark's presentation includes his basic event recommendations so as to run Sysmon optimally. <br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjaZf0sJ3s3zCBbqov3i3hkbL3SqeixrgBEiBcdoaFnB0-mAY4W4qhq7Dto4q_XO7Nm92dhdbu7wY096-ISs6sOch5n52T9ILClw8cJ5Zhyy7MAg39yT2o5sS2OJBe49dEWZBYVoQ/s1600/Recommendations.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="255" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjaZf0sJ3s3zCBbqov3i3hkbL3SqeixrgBEiBcdoaFnB0-mAY4W4qhq7Dto4q_XO7Nm92dhdbu7wY096-ISs6sOch5n52T9ILClw8cJ5Zhyy7MAg39yT2o5sS2OJBe49dEWZBYVoQ/s400/Recommendations.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Basic Event Recommendations</b></td></tr>
</tbody></table>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgFboJm3SPveN4Va_p_S3qFmuiD8hz_1K41I_kzR9_REQHOOVL8Qc-i5Ll548ii9Mdu1QzuxsvUaN92E2sCgA2K3RSS9ewSMBDRRSQsoT36fMv_9yuzRM4SSclchVw5P4z8oFOaqA/s1600/RecommendationsCont.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="276" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgFboJm3SPveN4Va_p_S3qFmuiD8hz_1K41I_kzR9_REQHOOVL8Qc-i5Ll548ii9Mdu1QzuxsvUaN92E2sCgA2K3RSS9ewSMBDRRSQsoT36fMv_9yuzRM4SSclchVw5P4z8oFOaqA/s400/RecommendationsCont.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Basic Event Recommendations (Cont)</b></td></tr>
</tbody></table>
<br />
I strongly suggest you deploy using these recommendations.<br />
A great way to get started is to use a Sysmon configuration template. Again, as Mark discussed at RSA, consider <a href="https://twitter.com/swiftonsecurity" target="_blank">@SwiftOnSecurity</a>'s <a href="https://github.com/SwiftOnSecurity/sysmon-config" target="_blank">sysmon-config-export.xml</a> via Github. While there are a number of templates on Github, this one has "virtually every line commented and sections are marked with explanations, so it should also function as a tutorial for Sysmon and a guide to critical monitoring areas in Windows systems." Running Sysmon with it is as easy as:<br />
<code>sysmon.exe -accepteula -i sysmonconfig-export.xml</code><br />
<br />
As a quick example of Sysmon capabilities and why <b>you should always run it everywhere</b>, consider the following driver installation scenario. While this is a non-malicious scenario that DFIR practitioners will appreciate, rather than the miscreants, the detection behavior resembles that which would result from kernel-based malware.<br />
I fired up <a href="https://github.com/google/rekall/tree/master/tools/windows/winpmem" target="_blank">WinPMEM</a>, the kernel mode driver for gaining access to physical memory included with <a href="https://github.com/google/rekall" target="_blank">Rekall</a>, as follows:<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiB2RrBzfEDXXFI17NINsv97a3MQ0HQcNzZtUrNj667HQCthC0TCOSte7T-o_jQXezCwBPCKz9BYOtXesqkhUvNubaviCgzz5w6vuyuVDdZ2SrtcVZaPwUn3WJcZAmR5rVSeyvZbA/s1600/WinPmem.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="201" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiB2RrBzfEDXXFI17NINsv97a3MQ0HQcNzZtUrNj667HQCthC0TCOSte7T-o_jQXezCwBPCKz9BYOtXesqkhUvNubaviCgzz5w6vuyuVDdZ2SrtcVZaPwUn3WJcZAmR5rVSeyvZbA/s400/WinPmem.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>WinPMEM</b></td><td class="tr-caption" style="text-align: center;"></td><td class="tr-caption" style="text-align: center;"></td></tr>
</tbody></table>
Upon navigating to Applications and Services Logs/Microsoft/Windows/Sysmon/Operational in the Windows Event Viewer, I retrieved the expected event:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZQ63geoFNIEv0xiBLKT7OuVlYu1ZpcoyScE0-cQx28GAQGqsjIp9nraA8Ib2hbd4xhIBiDzYy3grDLWHKzYOy47NMYvj0F4oBIcwaw2bVvRWN_s_h2E-jFi14c3TfxyXtgq1N0g/s1600/EventID6.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="128" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZQ63geoFNIEv0xiBLKT7OuVlYu1ZpcoyScE0-cQx28GAQGqsjIp9nraA8Ib2hbd4xhIBiDzYy3grDLWHKzYOy47NMYvj0F4oBIcwaw2bVvRWN_s_h2E-jFi14c3TfxyXtgq1N0g/s640/EventID6.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Event ID 6: Driver loaded</b></td></tr>
</tbody></table>
The best way to leave you to scramble off and deploy Sysmon broadly, are with Mark's Best Practices and Tips, again from his RSA presentation.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiYSalz1eHdRNBjNgby2rbEpbhHNPIdVc2F-vROezHbzgrAcPU0fAtn_wOu4f0F8AAbq7CricvaonebglJsnAcTxXkHr4cauyWZ3pddLx4TSW0olMVYMLvHy3JAfOhZ1trdRvuilg/s1600/BestPractices.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="263" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiYSalz1eHdRNBjNgby2rbEpbhHNPIdVc2F-vROezHbzgrAcPU0fAtn_wOu4f0F8AAbq7CricvaonebglJsnAcTxXkHr4cauyWZ3pddLx4TSW0olMVYMLvHy3JAfOhZ1trdRvuilg/s400/BestPractices.png" width="400" /></a></div>
Go forth and deploy!<br />
Cheers...until next time.Russ McReehttp://www.blogger.com/profile/05647342839278416757noreply@blogger.com0tag:blogger.com,1999:blog-20011960.post-41893243772797828222017-02-08T08:45:00.000-08:002017-02-08T08:45:26.274-08:00Aikido & HolisticInfoSec™<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhxWTi3tcbZZ19UP0Bmjxgsn203ZiQ7g8j1gBG1NdV4UN71oJVbT-Ox3s8GS-DmUEgiC_BlOGM8BrSX1jsMN4EQ49Arc2SZDuhma0gJiC-rmi7JGvsRtY87eGBKTLtqNTETAboCBg/s1600/AIKIDO_resize.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhxWTi3tcbZZ19UP0Bmjxgsn203ZiQ7g8j1gBG1NdV4UN71oJVbT-Ox3s8GS-DmUEgiC_BlOGM8BrSX1jsMN4EQ49Arc2SZDuhma0gJiC-rmi7JGvsRtY87eGBKTLtqNTETAboCBg/s200/AIKIDO_resize.jpg" width="85" /></a></div>
This is the 300th post to the HolisticInfoSec™ blog. Sparta, this isn't, but I thought it important to provide you with content in a warrior/philosopher mindset regardless. <br />
Your author is an <a href="https://en.wikipedia.org/wiki/Aikido" target="_blank">Aikido</a> practitioner, albeit a fledgling in practice, with so, so much to learn. While Aikido is often translated as "the way of unifying with life energy" or as "the way of harmonious spirit", I propose that the philosophies and principles inherent to Aikido have significant bearing on the practice of information security.<br />
In addition to spending time in the dojo, there are numerous reference books specific to Aikido from which a student can learn. Among the best is <span data-ved="0ahUKEwjHi4Wc0v3RAhVpJcAKHZ1JB-sQ2koIkQEoAjAS">Adele Westbrook and </span><span data-ved="0ahUKEwjHi4Wc0v3RAhVpJcAKHZ1JB-sQ2koIkQEoAjAS">Oscar Ratti's</span> <a href="https://www.amazon.com/Aikido-Dynamic-Sphere-Illustrated-Introduction/dp/0804832846" target="_blank">Aikido and the Dynamic Sphere</a>. All quotes and references that follow are drawn from this fine publication.<br />
As an advocate for the practice of HolisticInfoSec™ (so much so, I trademarked it) the connectivity to Aikido is practically rhetorical, but allow me to provide you some pointed examples. I've tried to connect each of these in what I believe is an appropriate sequence to further your understanding, and aid you in improving <i>your</i> practice. Simply, one could say each of these can lead to the next.<br />
<br />
<b>The Practice of Aikido</b> <br />
<i>"The very first requisite for defense is to know the enemy."</i><br />
So often in information security, we see reference to the much abused <i>The Art of War</i>, wherein Sun Tzu stated "It is said that if you know your enemies and know yourself, you will not be imperiled in a hundred battles." Aikido embraces this as the first requisite, but so too offers the importance of not underestimating your enemy or opponent. For information security, I liken it to this. If you are uninformed on adversary actor types and profiles, their TTPs (tools, tactics, procedures), as well as current vulnerabilities and exploits, along with more general threat intelligence, then you are already at a disadvantage before you even begin to imagine countering your opponent. <br />
<br />
<i>"A positive defensive strategy is further qualified as being specific, immediate, consistent, and powerful." </i><br />
Upon learning more about your adversary, a rehearsed, exercised strategy for responding to their attack should be considered the second requisite for defense. To achieve this, your efforts must include:<br />
<ul>
<li>a clear definition and inventory of the assets you're protecting </li>
<li>threat modeling of code, services, and infrastructure</li>
<li>an incident response plan and SOP, and regular exercise of the IR plan</li>
<li>robust security monitoring to include collection, aggregation, detection, correlation, and visualization </li>
<li>ideally, a purple team approach that includes testing blue team detection and response capabilities in partnership with a red team. Any red team that follows the "you suck, we rock" approach should be removed from the building and replaced by one who espouses "we exist to identify vulnerabilities and exploits with the goal of helping the organization better mitigate and remediate".</li>
</ul>
As your detection and response capabilities improve with practice and repetition, your meantime to mitigate (MTM) and meantime to remediate (MTR) should begin to shrink, thus lending to the immediacy, consistentcy, and power of your defense. <br />
<br />
<br />
<b>The Process of Defense and Its Factors</b><br />
<i>"EVERY process of defense will consist of three stages: perception, evaluation-decision, and reaction."</i><br />
These should be easy likenesses for you to reconcile.<br />
<b>Perception</b> = detection and awareness<br />
The better and more complete your threat intelligence collection and detection capabilities, the better your situational awareness will be, and as a result your perception of adversary behaviors will improve and become more timely.<br />
<b>Evaluation-decision</b> = triage<br />
It's inevitable...$#!+ happens. Your ability to quickly evaluate adversary actions and be decisive in your response will dictate your level of success as incident responders. Strength at this stage directly impacts the rest of the response process. Incorrect or incomplete evaluation, and the resulting ill-informed decisions, can set back your response process in a manner from which recovery will be very difficult.<br />
<b>Reaction</b> = response<br />
My Aikido sensei, after doing so, likes to remind his students "Don't get hit." :-) The analogy here is to react quickly enough to stay on your feet. Can you move quickly enough to not be hit as hard or as impactfully as your adversary intended? Your reaction and response will determine such outcomes. The connection between kinetic and virtual combat here is profound. Stand still, get hit. Feign or evade, at least avoid some, or all contact. In the digital realm, you're reducing your time to recover with this mindset. <br />
<br />
<b>Dynamic Factors</b><br />
<i>"A defensive aikido strategy begins the moment a
would-be attacker takes a step toward you or turns aggressively in your
direction. His initial motion (movement) in itself contains the factors
you will use to neutralize the action of attack which will spring with
explosive force from that motion of convergence."</i><br />
Continuing on our theme of inevitability, digital adversaries will, beyond the shadow of a doubt, take a step toward you or turn aggressively in your
direction. The question for you will be, do you even know when that has occurred in light of our discussion of requisites above? Aikido is all about using your opponent's energy against them, wherein, for those of us in DFIR, our adversary's movement in itself contains the factors we use to neutralize the action of attack. As we improve our capabilities in our defensive processes (perception, evaluation-decision, and reaction), we should be able to respond in a manner that begins the very moment we identify adversarial behavior, and do so quickly enough that our actions pivot directly on our adversary's initial motion.<br />
As an example, your adversary conducts a targeted, nuanced spear phishing campaign. Your detective means identify all intended victims, you immediately react, and add all intended victims to an enhanced watch list for continuous monitoring. The two victims who engaged the payload are quarantined immediately, and no further adversarial pivoting or escalation is identified. The environment as a whole raised to a state of heightened awareness, and your user-base becomes part of your perception network.<br />
<br />
<i><br /></i>
<i>"It will be immediate or instantaneous when your reaction is so swift that you apply a technique of neutralization while the attack is still developing, and at the higher levels of the practice even before an attack has been fully launched."</i><br />
Your threat intelligence capabilities are robust enough that your active deployment of detections for specific Indicators of Compromise (IOCs) prevented the targeted, nuanced spear phishing campaign from even reaching the intended victims. Your monitoring active lists include known adversary infrastructure such that the moment they launch an attack, you are already aware of its imminence.<br />
You are able to neutralize your opponent before they even launch. This may be unimaginable for some, but it is achievable by certain mature organizations under specific circumstances. <br />
<br />
<b>The Principle of Centralization</b><br />
<i>"Centralization,
therefore, means adopting a new point of reference, a new platform from
which you can exercise a more objective form of control over events and
over yourself."</i><br />
Some organizations decentralize information security, others centralize it with absolute authority. There are arguments for both, and I do not intend to engage that debate. What I ask you to embrace is the "principle of centralization". The analogy is this: large corporations and organizations often have multiple, and even redundant security teams. Even so, their cooperation is key to success.<br />
<ul>
<li>Is information exchanged openly and freely, with silos avoided? </li>
<li>Are teams capable of joint response? </li>
<li>Are there shared resources that all teams can draw from for consistent IOCs and case data?</li>
<li>Are you and your team focused on facts, avoiding FUD, thinking creatively, yet assessing with a critical, objective eye?</li>
</ul>
Even with a logically decentralized security model, organizations can embrace the principle of centralization and achieve an objective form of control over events. The practice of a joint forces focus defines the platform from which teams can and should operate.<br />
<br />
Adversarial conditions, in both the physical realm, and the digital realm in which DFIR practitioners operate, are stressful, challenging, and worrisome. <br />
Morihei Ueshiba, Aikido's founder reminds us that "in extreme situations, the entire universe becomes our foe; at such critical times, unity of mind and technique is essential - do not let your heart waver!" That said, perfection is unlikely, or even impossible, this is a practice you must exercise. Again Ueshiba offers that "failure is the key to success; each mistake teaches us something."<br />
Keep learning, be strong of heart. :-)<br />
Cheers...until next time. Russ McReehttp://www.blogger.com/profile/05647342839278416757noreply@blogger.com0tag:blogger.com,1999:blog-20011960.post-30917299605091403962016-12-31T20:00:00.001-08:002017-01-06T09:54:08.444-08:00The DFIR Hierarchy of Needs & Critical Security ControlsAs you weigh how best to improve your organization's <span class="col-11 text-gray-dark" itemprop="about">digital forensics and incident response (DFIR) capabilities heading into 2017, consider </span><a href="https://twitter.com/MSwannMSFT" target="_blank">Matt Swann</a>'s <span class="col-11 text-gray-dark" itemprop="about"><a href="https://github.com/swannman/ircapabilities" target="_blank">Incident Response Hierarchy of Needs</a></span>. Likely, at some point in your career (or therapy 😉) you've heard reference to <a href="https://en.wikipedia.org/wiki/Maslow%27s_hierarchy_of_needs">Maslow's Hierarchy of Needs</a>. In summary, Maslow's terms, physiological, safety, belongingness & love, esteem, self-actualization, and self-transcendence, describe a pattern that human motivations generally move through, a pattern that is well represented in the form of a pyramid. <br />
Matt has made great use of this model to describe an <span class="col-11 text-gray-dark" itemprop="about"><a href="https://github.com/swannman/ircapabilities" target="_blank">Incident Response Hierarchy of Needs</a>, through which your DFIR methods should move. I argue that his powerful description of capabilities extends to the whole of DFIR rather than response alone. From Matt's Github, "t</span>he Incident Response Hierarchy describes the capabilities that organizations must build to defend
their business assets. Bottom capabilities are prerequisites for
successful execution of the capabilities above them:"<br />
<span class="col-11 text-gray-dark" itemprop="about"></span>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://raw.githubusercontent.com/swannman/ircapabilities/master/hierarchy.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="224" src="https://raw.githubusercontent.com/swannman/ircapabilities/master/hierarchy.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>The Incident Response Hierarchy of Needs</b></td><td class="tr-caption" style="text-align: center;"></td><td class="tr-caption" style="text-align: center;"></td><td class="tr-caption" style="text-align: center;"></td><td class="tr-caption" style="text-align: center;"></td><td class="tr-caption" style="text-align: center;"></td><td class="tr-caption" style="text-align: center;"></td><td class="tr-caption" style="text-align: center;"></td><td class="tr-caption" style="text-align: center;"></td><td class="tr-caption" style="text-align: center;"></td></tr>
</tbody></table>
"The capabilities may also be organized into plateaus or phases that
organizations may experience as they develop these capabilities:"<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://raw.githubusercontent.com/swannman/ircapabilities/master/plateaus.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="221" src="https://raw.githubusercontent.com/swannman/ircapabilities/master/plateaus.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Hierarchy plateaus or phases</b></td></tr>
</tbody></table>
As visualizations, these representations really do speak for themselves, and I applaud Matt's fine work. I would like to propose that a body of references and controls may be of use to you in achieving this hierarchy to its utmost. I also welcome your feedback and contributions regarding how to achieve each of these needs and phases. Feel free to submit controls, tools, and tactics you have or would deploy to be successful in these endeavors; I'll post your submission along with your preferred social media handle.<br />
Aspects of the Center for Internet Security Critical Security Controls Version 6.1 (CIS CSC) can be mapped to each of Matt's hierarchical entities and phases. Below I offer one control and one tool to support each entry. Note that there is a level of subjectivity to these mappings and tooling, but the intent is to help you adopt this thinking and achieve this agenda. Following is an example for each one, starting from the bottom of the pyramid.<br />
<br />
<span style="background-color: #8e7cc3;"><b><span style="background-color: #b4a7d6;"> INVENTORY - Can you name the assets you are defending? </span></b></span><b> </b><br />
<b>Critical Security Control #1: Inventory of Authorized and Unauthorized Devices</b><br />
<b>Family:</b> System<br />
<b>Control:</b> 1.4 <br />
"Maintain an asset inventory of all systems connected to the network and the network devices themselves, recording at least the network addresses, machine name(s), purpose of each system, an asset owner responsible for each device, and the department associated with each device. The inventory should include every system that has an Internet protocol (IP) address on the network, including but not limited to desktops, laptops, servers, network equipment (routers, switches, firewalls, etc.), printers, storage area networks, Voice Over-IP telephones, multi-homed addresses, virtual addresses, etc. The asset inventory created must also include data on whether the device is a portable and/or personal device. Devices such as mobile phones, tablets, laptops, and other portable electronic devices that store or process data must be identified, regardless of whether they are attached to the organization’s network."<b> </b><br />
<b>Tool option: </b><br />
<a href="https://www.spiceworks.com/free-pc-network-inventory-software/" target="_blank">Spiceworks Inventory</a><br />
<br />
<b><span style="background-color: #3d85c6;"> TELEMETRY - Do you have visibility across your assets? </span> </b><br />
<b>Critical Security Control #6: Maintenance, Monitoring, and Analysis of Audit Logs </b><br />
<b>Family:</b> System<br />
<b>Control:</b> 6.6 "Deploy a SIEM (Security Information and Event Management) or log analytic tools for log aggregation and consolidation from multiple machines and for log correlation and analysis. Using the SIEM tool, system administrators and security personnel should devise profiles of common events from given systems so that they can tune detection to focus on unusual activity, avoid false positives, more rapidly identify anomalies, and prevent overwhelming analysts with insignificant alerts."<br />
<b>Tool option: </b><br />
<a href="https://www.alienvault.com/products/ossim" target="_blank">AlienVault OSSIM</a><b> </b><br />
<br />
<span style="background-color: cyan;"><b> DETECTION - Can you detect unauthorized actvity? </b></span><br />
<b>Critical Security Control #8: Malware Defenses </b><br />
<b>Family:</b> System<br />
<b>Control: </b>8.1<br />
"Employ automated tools to continuously monitor workstations, servers, and mobile devices with anti-virus, anti-spyware, personal firewalls, and host-based IPS functionality. All malware detection events should be sent to enterprise anti-malware administration tools and event log servers."<br />
<b>Tool option:</b><br />
<a href="http://ossec.github.io/" target="_blank">OSSEC <b>O</b>pen <b>S</b>ource HIDS <b>SEC</b>urity</a><br />
<br />
<span style="background-color: #d9ead3;"> <b>TRIAGE</b> <b>- Can you accurately classify detection results?</b> </span><br />
<b>Critical Security Control #4: Continuous Vulnerability Assessment and Remediation </b><br />
<b>Family:</b> System<br />
<b>Control: </b>4.3<br />
"Correlate event logs with information from vulnerability scans to fulfill two goals. First, personnel should verify that the activity of the regular vulnerability scanning tools is itself logged. Second, personnel should be able to correlate attack detection events with prior vulnerability scanning results to determine whether the given exploit was used against a target known to be vulnerable."<b><br /></b><b><b>Tool option:</b></b><br />
<a href="http://www.openvas.org/" target="_blank">OpenVAS</a> <b> </b><br />
<br />
<span style="background-color: lime;"><b> THREATS - Who are your adversaries? What are their capabilities? </b></span> <br />
<b>Critical Security Control #19: Incident Response and Management </b><br />
<b>Family: </b>Application <br />
<b>Control: </b>19.7<br />
"Conduct periodic incident scenario sessions for personnel associated with the incident handling team to ensure that they understand current threats and risks, as well as their responsibilities in supporting the incident handling team."<br />
<b>Tool option:</b><br />
<a href="http://resources.infosecinstitute.com/incident-response-and-audit-requirements/" target="_blank">Security Incident Response Testing To Meet Audit Requirements</a><br />
<br />
<span style="background-color: #93c47d;"> <b>BEHAVIORS - Can you detect adversary activity within your environment?</b> </span><br />
<b>Critical Security Control #5: Controlled Use of Administrative Privileges </b><br />
<b>Family:</b> System<br />
<b>Control:</b> 5.1<br />
"Minimize administrative privileges and only use administrative accounts when they are required. Implement focused auditing on the use of administrative privileged functions and monitor for anomalous behavior."<br />
<b>Tool option: </b><br />
<a href="https://www.microsoft.com/en-us/download/details.aspx?id=46899" target="_blank">Local Administrator Password Solution (LAPS)</a><br />
<br />
<span style="background-color: yellow;"><b> HUNT - Can you detect an adversary that is already embedded? </b></span><br />
<b>Critical Security Control #6: Maintenance, Monitoring, and Analysis of Audit Logs</b> <br />
<b>Family:</b> System<br />
<b>Control:</b> 6.4<br />
"Have security personnel and/or system administrators run biweekly reports that identify anomalies in logs. They should then actively review the anomalies, documenting their findings."<br />
<b>Tool option:</b><br />
<a href="https://github.com/google/grr" target="_blank">GRR Rapid Response</a><br />
<br />
<span style="background-color: orange;"><b> TRACK - During an intrusion, can you observe adversary activity in real time? </b></span><br />
<b>Critical Security Control #12: Boundary Defense </b><br />
<b>Family:</b> Network<br />
<b>Control:</b> 12.10 <br />
"To help identify covert channels exfiltrating data through a firewall, configure the built-in firewall session tracking mechanisms included in many commercial firewalls to identify TCP sessions that last an unusually long time for the given organization and firewall device, alerting personnel about the source and destination addresses associated with these long sessions."<br />
<b>Tool option:</b><br />
<a href="https://www.bro.org/index.html" target="_blank">Bro</a> <br />
<br />
<span style="background-color: #e06666;"> <b>ACT - Can you deploy countermeasures to evict and recover?</b> </span><br />
<b>Critical Security Control #20: Penetration Tests and Red Team Exercises </b><br />
<b>Family:</b> Application<br />
<b>Control:</b> 20.3<br />
"Perform periodic Red Team exercises to test organizational readiness to identify and stop attacks or to respond quickly and effectively."<br />
<b>Tool option:</b><br />
<a href="http://holisticinfosec.blogspot.com/2016/01/toolsmith-112-red-vs-blue-powersploit.html" target="_blank">Red vs Blue - PowerSploit vs PowerForensics</a><br />
<br />
<br />
<span style="background-color: red;"> <span style="color: white;"><b>Can you collaborate with trusted parties to disrupt adversary campaigns?</b></span> </span><br />
<b>Critical Security Control #19: Incident Response and Management </b><br />
<b>Family:</b> Application<br />
<b>Control:</b> 19.5<br />
"Assemble and maintain information on third-party contact information to be used to report a security incident (e.g., maintain an e-mail address of security@organization.com or have a web page http://organization.com/security)." <br />
<b>Tool option:</b><br />
<a href="http://www.misp-project.org/" target="_blank">MISP</a><br />
<br />
I've mapped the hierarchy to the controls in CIS CSC 6.1 spreadsheet, again based on my experience and perspective, yours may differ, but consider similar activity.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjlPLtvOnHYpCNhhdYTwunCX_f6bkXZpAE7fUbQrrQbhnH1DXNN9dhVHsCjXNol_4noFGQMLPKh9Ubxyyo5QYw96vPisTnw81Z7H4dDd-KL3vd7D7XFibNYwSROM9qSV5qTfEm5OQ/s1600/matrix.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="150" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjlPLtvOnHYpCNhhdYTwunCX_f6bkXZpAE7fUbQrrQbhnH1DXNN9dhVHsCjXNol_4noFGQMLPKh9Ubxyyo5QYw96vPisTnw81Z7H4dDd-KL3vd7D7XFibNYwSROM9qSV5qTfEm5OQ/s400/matrix.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>CIS CSC with IR Hierarchy mappings</b></td></tr>
</tbody></table>
<br />
<br />
My full mapping of Matt's <span class="col-11 text-gray-dark" itemprop="about"><a href="https://github.com/swannman/ircapabilities" target="_blank">Incident Response Hierarchy of Needs</a> in the </span><br />
CIS CSC 6.1 spreadsheet is available here: <a href="https://holisticinfosec.org/content/CSC-CIS_Critical_Security_Controls&IR_Hierarchy.xlsx">http://bit.ly/CSC-IRH</a><br />
<br />
I truly hope you familiarize yourself with <a href="https://twitter.com/MSwannMSFT" target="_blank">Matt</a>'s <span class="col-11 text-gray-dark" itemprop="about"><a href="https://github.com/swannman/ircapabilities" target="_blank">Incident Response Hierarchy of Needs</a></span> and find ways to implement, validate, and improve your capabilities accordingly. Consider that the controls and tools mentioned here are but a starting point and that you have many other options available to you. I look forward to hearing from you regarding your preferred tactics and tools as well. Kudos to Matt for framing this essential discussion so distinctly.<br />
<table style="width: 144px;"><tbody>
<tr height="20" style="height: 15.0pt;"><td class="xl130" height="20" style="height: 15.0pt; width: 108pt;" width="144"></td><td class="xl130" height="20" style="height: 15.0pt; width: 108pt;" width="144"></td></tr>
</tbody></table>
Russ McReehttp://www.blogger.com/profile/05647342839278416757noreply@blogger.com2tag:blogger.com,1999:blog-20011960.post-28641707512062055652016-12-11T09:54:00.003-08:002016-12-16T11:31:25.123-08:00Toolsmith - GSE Edition: Image Steganography & StegExposeCross-posted on the Internet Storm Center Diary.<br />
<br />
<strong>Updated</strong> with contest winners 14 DEC. Congrats to:<br />
Chrissy <a href="https://twitter.com/SecAssistance">@SecAssistance</a><br />
Owen Yang <a href="https://twitter.com/HomingFromWork">@HomingFromWork</a><br />
Paul Craddy <a href="https://twitter.com/pcraddy">@pcraddy</a><br />
Mason Pokladnik - Fellow STI grad<br />
Elliot Harbin <a href="https://twitter.com/klax0ff">@klax0ff</a><br />
<br />
In the last of a three part (<a href="http://holisticinfosec.blogspot.com/2016/10/toolsmith-gse-edition-snapshotps1.html" target="_blank">Part 1-GCIH</a>, <a href="http://holisticinfosec.blogspot.com/2016/11/toolsmith-gse-edition-scapy-vs-cozyduke.html" target="_blank">Part 2-GCIA</a>) series focused on tools I revisited during my GSE re-certification process, I thought it'd be timely and relevant to give you a bit of a walkthrough re: <span class="st">steganography</span> tools. <a href="http://www.webopedia.com/TERM/S/steganography.html" target="_blank">Steganography</a> "represents the art and science of hiding information by embedding messages within other, seemingly harmless messages."<br />
Stego has garnered quite a bit of attention again lately as party to both exploitation and exfiltration tactics. On 6 DEC 2016, ESET described millions of victims among <a href="http://www.welivesecurity.com/2016/12/06/readers-popular-websites-targeted-stealthy-stegano-exploit-kit-hiding-pixels-malicious-ads/" target="_blank">readers of popular websites who had been targeted by the Stegano exploit kit hiding in pixels of malicious ads</a>.<br />
The <a href="https://blog.sucuri.net/2016/10/magento-credit-card-swiper-exports-image.html" target="_blank">Sucuri blog</a> described credit card swipers in Magento sites on 17 OCT 2016, where attackers used image files as an obfuscation technique to hide stolen details from website owners, in images related to products sold on the victim website.<br />
<br />
The GSE certification includes <a href="https://www.giac.org/certification/security-essentials-gsec" target="_blank">SANS 401 GSEC</a> content, and Day 4 of the GSEC class content includes some time on steganography with the Image Steganography tool. Tools for <span class="st">steganographic </span>creation are readily available, but a bit dated, including <a href="https://imagesteganography.codeplex.com/" target="_blank">Image S</a><span class="st"><a href="https://imagesteganography.codeplex.com/" target="_blank">teganography, </a>last updated in 2011, and <a href="https://sourceforge.net/projects/openstego/" target="_blank">OpenStego, </a></span><span class="st"><span class="st">last updated in 2015</span>. There are other older, command-line tools, but these two are really straightforward GUI-based options. </span><span class="st">Open source or free stego detection tools are unfortunately really dated and harder to find as a whole, unless you're a commercial tool user. StegExpose is one of a few open options that's fairly current (2015) and allows you to </span><span class="st"><span class="repository-meta-content"><span class="mb-2"><span class="col-11 text-gray-dark mb-2" itemprop="about">conduct steganalysis to detect LSB steganography in images. </span></span></span></span><span class="_Tgc">The LSB is the lowest significant bit in the byte value of the image pixel and LSB-based image steganography embeds the hidden payload in the least significant bits of pixel values of an image. </span><br />
Image Steganography uses LSB steganography, making this a perfect opportunity to pit one against the other. <br />
<span class="st">Download </span>Image Steganography from <a href="https://imagesteganography.codeplex.com/downloads/get/251046" target="_blank">Codeplex</a>, then run <span style="font-family: "courier new" , "courier" , monospace;">Image Steganography Setup.exe</span>. Run Image Steganography after installation and select a PNG for your image. You can then type text you'd like to embed, or input data from a file. I chose <span style="font-family: "courier new" , "courier" , monospace;">wtf.png</span> for my image, and <span style="font-family: "courier new" , "courier" , monospace;">rr.ps1</span> as my input file. I chose to write out the resulting stego sample to <span style="font-family: "courier new" , "courier" , monospace;">wtf2.png</span>, as seen in <b>Figure 1</b>.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgvn4TmIOjWKVXUoqLu52uTLqyMc9Z5dnCVCtQ4Ag7H73GpbMg7nwHG9RFov_Mq1eBawryD9Q1Lq5XBGPowpSBgOXwC4W79wI0W9FBZleyzpdeSoF7RadRnKxVmKAM45mT6tOmwQA/s1600/ImageSteganography.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgvn4TmIOjWKVXUoqLu52uTLqyMc9Z5dnCVCtQ4Ag7H73GpbMg7nwHG9RFov_Mq1eBawryD9Q1Lq5XBGPowpSBgOXwC4W79wI0W9FBZleyzpdeSoF7RadRnKxVmKAM45mT6tOmwQA/s320/ImageSteganography.png" width="229" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Figure 1: Image Steganography</td></tr>
</tbody></table>
This process in reverse to decode a message is just as easy. Select the decode radio button, and the UI will switch to decode mode. I dragged the wtf2.png file I'd just created, and opted to write the ouput to the same directory, as seen in <b>Figure 2</b>.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEguO8esgGLRH1UKMcQPZNGxPSbJailHrTL_pSA0xsJYEktyxOigUkTYBxxH1CJ3cpJO4urDg-uSRSJvnezGXWTvm5vuYV2DDkObuljVgfQwjtIgjZHNlTtktlIvq75E7VvTZXa-xw/s1600/ImageSteganographyDecode.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEguO8esgGLRH1UKMcQPZNGxPSbJailHrTL_pSA0xsJYEktyxOigUkTYBxxH1CJ3cpJO4urDg-uSRSJvnezGXWTvm5vuYV2DDkObuljVgfQwjtIgjZHNlTtktlIvq75E7VvTZXa-xw/s320/ImageSteganographyDecode.png" width="229" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 2:</b> wtf.png decoded</td><td class="tr-caption" style="text-align: center;"></td><td class="tr-caption" style="text-align: center;"><br /></td><td class="tr-caption" style="text-align: center;"><br /></td></tr>
</tbody></table>
Pretty simple, and the extracted <span style="font-family: "courier new" , "courier" , monospace;">rr.ps1</span> file was unchanged from the original embedded file.<br />
Now, will <a href="https://github.com/b3dk7/StegExpose" target="_blank">StegExpose</a> detect this file as steganographic? <a href="https://github.com/b3dk7/StegExpose/archive/master.zip" target="_blank">Download</a> StegExpose from Github, unpack <span style="font-family: "courier new" , "courier" , monospace;">master.zip</span>, and navigate to the resulting directory from a command prompt. Run <span style="font-family: "courier new" , "courier" , monospace;">StegExpose.jar</span> against the directory with your steganographic image as follows: <span style="font-family: "courier new" , "courier" , monospace;">java -jar StegExpose.jar c:\tmp\output</span>. Sure enough, steganography confirmed as seen in <b>Figure 3</b>.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiOGdr_btjstVEadmozKl1_fO9LamWEC6nteJxPAVvUBAio124pwpHssVeyZgNfVIVBNNaSlD83g1zA80GUVAafGUlONPGwyXbqQXo781zmjxx_q-r0y0X22cCPQ9O_hHMR1rxhsg/s1600/StegExpose.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="81" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiOGdr_btjstVEadmozKl1_fO9LamWEC6nteJxPAVvUBAio124pwpHssVeyZgNfVIVBNNaSlD83g1zA80GUVAafGUlONPGwyXbqQXo781zmjxx_q-r0y0X22cCPQ9O_hHMR1rxhsg/s320/StegExpose.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 3:</b> StegExpose</td></tr>
</tbody></table>
Not bad, right? Easy operations on both sides of the equation.<br />
<br />
And now for a little <b>contest</b>. Five readers who email me via russ at holisticinfosec dot org and give me the most precise details regarding what I specifically hid in <span style="font-family: "courier new" , "courier" , monospace;">wtf2.png</span> get a shout out here and $5 Starbucks gift cards for a little Christmastime caffeine. <b> </b><br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhNLO8WJB0GYKjAmdniUMGDU83IVrIMx0hqoDPAyJH8neyDrh5YFfSDOm63q3Wq8sJS3LBfxMj8I6iKHDMR619hqQzk5ij1l3wKm4bP59KOpELNgy4JBsxPKwz7ZaLHxbjeCfLFZA/s1600/wtf2.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhNLO8WJB0GYKjAmdniUMGDU83IVrIMx0hqoDPAyJH8neyDrh5YFfSDOm63q3Wq8sJS3LBfxMj8I6iKHDMR619hqQzk5ij1l3wKm4bP59KOpELNgy4JBsxPKwz7ZaLHxbjeCfLFZA/s1600/wtf2.png" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Contest:</b> wtf2.png</td></tr>
</tbody></table>
<b>Note:</b> do not run the actual payload, it will annoy you to no end. If you must run it to decipher it, please do so in a VM. It's not malware, but again, it is annoying. <br />
<br />
Cheers...until next time.Russ McReehttp://www.blogger.com/profile/05647342839278416757noreply@blogger.com0tag:blogger.com,1999:blog-20011960.post-85602770238949260722016-11-27T11:04:00.001-08:002016-11-27T11:22:05.950-08:00Toolsmith - GSE Edition: Scapy vs CozyDukeIn continuation of observations from my <a href="https://www.giac.org/certification/security-expert-gse" target="_blank">GIAC Security Expert</a> re-certification process, I'll focus here on a <a href="https://www.giac.org/certification/certified-intrusion-analyst-gcia" target="_blank">GCIA</a>-centric topic: Scapy. Scapy is essential to the packet analyst skill set on so many levels. For your convenience, the <a href="https://sourceforge.net/projects/packetrix/" target="_blank">Packetrix VM</a> comes preconfigured with Scapy and Snort, so you're ready to go out of the gate if you'd like to follow along for a quick introduction.<br />
<a href="http://www.secdev.org/projects/scapy/" target="_blank">Scapy</a> is "a powerful interactive packet manipulation program. It is able to forge or decode packets of a wide number of protocols, send them on the wire, capture them, match requests and replies, and much more." This includes the ability to handle most tasks such as scanning, tracerouting, probing, unit tests, attacks or network discovery, thus replacing functionality expected from hping, 85% of nmap, arpspoof, tcpdump, and others.<br />
If you'd really like to dig in, grab <a href="https://twitter.com/ViolentPython" target="_blank">TJ O'Connor's</a> <a href="https://www.amazon.com/product-reviews/1597499579" target="_blank">Violent Python: A Cookbook for Hackers, Forensic Analysts, Penetration Testers and Security Engineers</a> (you should already have it), as first discussed here in <a href="http://holisticinfosec.blogspot.com/2013/01/toolsmith-violent-python-book-review.html" target="_blank">January 2013</a>. TJ loves him some Scapy: <a href="https://www.sans.org/reading-room/whitepapers/intrusion/detecting-responding-data-link-layer-attacks-33513" target="_blank">Detecting and Responding to Data Link Layer Attacks</a> is another reference. :-)<br />
You can also familiarize yourself with Scapy's syntax in short order with the <a href="https://blogs.sans.org/pen-testing/files/2016/04/ScapyCheatSheet_v0.2.pdf" target="_blank">SANS Scapy Cheat Sheet</a> as well. <br />
Judy Novak's SANS <a href="https://www.giac.org/certification/certified-intrusion-analyst-gcia" target="_blank">GIAC Certified Intrusion Analyst</a> Day 5 content offers a nice set of walk-throughs using Scapy, and given that it is copyrighted and private material, I won't share them here, but will follow a similar path so you have something to play along with at home. We'll use a real-world APT scenario given recent and unprecedented Russian meddling in American politics. According to <a href="https://www.scmagazine.com/russian-hackers-access-trump-files-in-dnc-hack/article/529426/" target="_blank">SC Magazine</a>, "Russian government hackers apparently broke into the Democratic National Committee (DNC) computer systems" in infiltrations believed to be the work of two different Russian groups, namely Cozy Bear/ CozyDuke/APT 29 and Fancy Bear/Sofacy/APT 28, working separately. As is often the case, ironically and consistently, one the best overviews of CozyDuke behaviors comes via Kaspersky's <a href="https://securelist.com/blog/research/69731/the-cozyduke-apt/" target="_blank">Securelist</a>. This article is cited as the reference in a number of Emerging Threats Snort/Suricata rules for CozyDuke. Among them, <a href="http://doc.emergingthreats.net/bin/view/Main/2020962" target="_blank">2020962 - ET TROJAN CozyDuke APT HTTP Checkin</a>, found in the <span style="font-size: small;"><span style="font-family: "courier new" , "courier" , monospace;">trojan.rules</span></span> file, serves as a fine exemplar.<br />
I took serious liberties with the principles of these rules and oversimplified things significantly with a rule as added to my <span style="font-size: small;"><span style="font-family: "courier new" , "courier" , monospace;">local.rules</span></span> file on my Packetrix VM. I then took a few quick steps with Scapy to ensure that the rule would fire as expected. Of the IOCs derived from the Securelist article, we know a few things that, if built into a PCAP with Scapy, should cause the rule to fire when the PCAP is read via Snort.<br />
<ol>
<li>CozyDuke client to C2 calls were over HTTP</li>
<li>Requests for C2 often included a .php reference, URLs included the likes of <span style="font-family: "courier new" , "courier" , monospace;">/ajax/index.php</span></li>
<li>209.200.83.43 was one of the C2 IPs, can be used as an example destination IP address</li>
</ol>
The resulting simpleton Snort rule appears in <b>Figure 1</b>.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiU_zGPHo-664FzVewzNfpDqLEDig-l7EBB5bUocCVmKUp_yfjRTcDDJjiJEE1D7LP3Uh1P4IC9U5AvEIElNcgPkXY9VNiXfeikbptF-8tCtNbDPrM84aAitRUEHAc56z2A32BJZw/s1600/Figure1.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="145" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiU_zGPHo-664FzVewzNfpDqLEDig-l7EBB5bUocCVmKUp_yfjRTcDDJjiJEE1D7LP3Uh1P4IC9U5AvEIElNcgPkXY9VNiXfeikbptF-8tCtNbDPrM84aAitRUEHAc56z2A32BJZw/s400/Figure1.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 1:</b> Simple rule</td></tr>
</tbody></table>
To quickly craft a PCAP to trigger this rule, at a bash prompt, I ran scapy, followed by<span style="font-size: small;"> </span><span style="font-size: x-small;"><span style="font-family: "courier new" , "courier" , monospace;"><span style="font-size: small;">syn = IP(src="10.0.2.15", dst="209.200.83.43")/TCP(sport=1337, dport=80, flags="S")/"GET /ajax/index.php HTTP/1.1</span>"</span></span>, then wrote the results out with <span style="font-size: x-small;"><span style="font-family: "courier new" , "courier" , monospace;"><span style="font-size: small;">wrpcap("/tmp/CozyDukeC2GET.pcap</span>", syn)</span></span>, as seen in <b>Figure 2</b>.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiQOEKo22tzvRR224zxnYVLPIpUQ3-kThUXo8U1kqX6l3BR9J00z7As7hkik-wDYbbY2wAbqdyUMQpV1OpbtrHz4Feoc2zUesNAYbka0yg9uz8m4wRDp26Y3UwvOBqU44SHgAdm0g/s1600/Figure2.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="41" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiQOEKo22tzvRR224zxnYVLPIpUQ3-kThUXo8U1kqX6l3BR9J00z7As7hkik-wDYbbY2wAbqdyUMQpV1OpbtrHz4Feoc2zUesNAYbka0yg9uz8m4wRDp26Y3UwvOBqU44SHgAdm0g/s400/Figure2.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 2:</b> Simple Scapy</td></tr>
</tbody></table>
Then a quick run of the resulting file through Snort with <span style="font-size: small;"><span style="font-family: "courier new" , "courier" , monospace;">snort -A console -q -K none -r /tmp/CozyDukeC2GET.pcap -c ../etc/snort.conf</span></span>, and we have a hit as seen in <b>Figure 3</b>.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi0D-FYLw0OexYoZskib6NAZeEdxCHpimiu-MaKJo-zXn1SmPqGXHFXJlnKmL3h5EvHpdWARewJPslsBEFP-rYcpBiXGJG6Lt-FwEk0UlZdBb1aSv-mLdtos2T4890i7mn5S56ryg/s1600/Figure3.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="20" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi0D-FYLw0OexYoZskib6NAZeEdxCHpimiu-MaKJo-zXn1SmPqGXHFXJlnKmL3h5EvHpdWARewJPslsBEFP-rYcpBiXGJG6Lt-FwEk0UlZdBb1aSv-mLdtos2T4890i7mn5S56ryg/s400/Figure3.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 3:</b> Simple result</td></tr>
</tbody></table>
<br />
Scapy is ridiculously powerful and is given no justice here, hopefully just enough information to entice you to explore further. With just the principles established here, you can see the likes of options to craft and manipulate with <span style="font-size: small;"><span style="font-family: "courier new" , "courier" , monospace;">ls(TCP)</span></span> and <span style="font-size: small;"><span style="font-family: "courier new" , "courier" , monospace;">ls(IP)</span></span>.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOHqomPsM_LiKEKeuLCX395YQA1Os_KwVfzlUzwTXW8oArw_9vI21hZjaldeIAXsc_A4d9hJ7OQLxRf82Fc0c_0h5NNkHQMvAWqvHTka19dXSedhwuSKrxEC9KGY9VjC5RXN-x_Q/s1600/Figure4.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOHqomPsM_LiKEKeuLCX395YQA1Os_KwVfzlUzwTXW8oArw_9vI21hZjaldeIAXsc_A4d9hJ7OQLxRf82Fc0c_0h5NNkHQMvAWqvHTka19dXSedhwuSKrxEC9KGY9VjC5RXN-x_Q/s400/Figure4.png" width="327" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 4:</b> ls()</td></tr>
</tbody></table>
<br />
If you're studying for the likes of GCIA or just looking to improve your understanding of TCP/IP and <a href="https://www.nostarch.com/nsm" target="_blank">NSM</a>, no better way to do so than with Scapy.<br />
Cheers...until next time.Russ McReehttp://www.blogger.com/profile/05647342839278416757noreply@blogger.com0tag:blogger.com,1999:blog-20011960.post-91316062876810114302016-10-31T14:36:00.004-07:002016-10-31T14:58:24.178-07:00Toolsmith - GSE Edition: snapshot.ps1I just spent a fair bit of time preparing to take the <a href="https://www.giac.org/certification/security-expert-gse" target="_blank">GIAC Security Expert</a> exam as part of the requirement to recertify every four years. I first took the exam in 2012, and I will tell you, for me, one third of the curriculum is a "use it or lose it" scenario. The GSE exam covers <a href="https://www.giac.org/certification/security-essentials-gsec" target="_blank">GSEC</a>, <a href="https://www.giac.org/certification/certified-incident-handler-gcih" target="_blank">GCIH</a>, and <a href="https://www.giac.org/certification/certified-intrusion-analyst-gcia" target="_blank">GCIA</a>. As my daily duties have migrated over the years from analyst to leadership, I had to "relearn" my packet analysis fu. Thank goodness for the Packetrix VM and the SANS 503 exercises workbook, offsets, flags, and fragments, oh my! All went well, mission accomplished, I'm renewed through October 2020 and still <a href="https://www.giac.org/certified-professional/russ-mcree/106487" target="_blank">GSE #52</a>, but spending weeks with my nose in the 18 course books reminded of some of the great tools described therein. As a result, this is the first of a series on some of those tools, their value, and use case scenarios.<br />
I'll begin with <span style="font-family: "courier new" , "courier" , monospace;">snapshot.ps1</span>. It's actually part of the <a href="https://cyber-defense.sans.org/blog/downloads" target="_blank">download package</a> for <a href="https://www.sans.org/course/securing-windows-with-powershell" target="_blank">SEC505: Securing Windows and PowerShell Automation</a>, but is discussed as part of the GCIH curriculum. In essence, <span style="font-family: "courier new" , "courier" , monospace;">snapshot.ps1</span> represents one script to encapsulate activities specific to the <a href="https://www.sans.org/media/score/checklists/ID-Windows.pdf" target="_blank">SANS Intrusion Discovery Cheat Sheet for Windows</a>. <br />
The script comes courtesy of <a href="https://twitter.com/JasonFossen" target="_blank">Jason Fossen</a>, the SEC505 author, and can be found in the Day 5-IPSec folder of the course <a href="https://cyber-defense.sans.org/blog/downloads" target="_blank">download package</a>. The script "dumps a vast amount of configuration data for the sake of auditing and forensics analysis" and allows you to "compare snapshot files created at different times to extract differences."<br />
To use <span style="font-family: "courier new" , "courier" , monospace;">snapshot.ps1 </span>place the script into a directory where it is safe to create a subdirectory as the script creates such a directory named named for the computer, then writes a variety of files containing system configuration data. Run <span style="font-family: "courier new" , "courier" , monospace;">snapshot.ps1</span> with administrative privileges.<br />
The script runs on Windows 7, Server 2008, and newer Windows operating systems (I ran it on Windows 10 Redstone 2) and requires PowerShell 3.0 or later. You also need to have autorunsc.exe and sha256deep.exe in your PATH if you want to dump what programs are configured to startup automatically when your system boots and you login, as well as run SHA256 file hashes.<br />
That said, if you must make the script run faster, and I mean A LOT FASTER, leave file <br />
hashing disabled at the end of the <span style="font-family: "courier new" , "courier" , monospace;">snapshot.ps1</span> for a 90% reduction in run time. <br />
However, Jason points out that this is one of the most useful aspects of the script for identifying adversarial activity. He also points out that <span style="font-family: "courier new" , "courier" , monospace;">snapshot.ps1</span> is a starter script; you can and should add more commands. As an example, referring back to <a href="http://holisticinfosec.blogspot.com/2016/01/toolsmith-112-red-vs-blue-powersploit.html" target="_blank">toolsmith #112: Red vs Blue - PowerSploit vs PowerForensics</a>, after importing PowerForensics, you could add something like <span style="font-family: "courier new" , "courier" , monospace;">Get-ForensicTimeline | Sort-Object -Property Date | Where-Object { $_.Date -ge "12/30/2015" -and $_.Date -le "01/04/2016" } | WriteOut -FileName</span> Timeline which would give you a file system timeline between the 12/30/2015 and 01/04/2016.But wait, there's more! Want to get autoruns without needing autorunsc.exe? Download @p0w3rsh3ll's AutoRuns module, run <span style="font-family: "courier new" , "courier" , monospace;">Import-Module AutoRuns.psm1</span>, then <span style="font-family: "courier new" , "courier" , monospace;">Get-Command -Module AutoRuns</span> to be sure the module is on board, and finally comment out <span style="font-family: "courier new" , "courier" , monospace;">autorunsc.exe -accepteula -a -c | Out-File -FilePath AutoRuns.csv</span> then add <span style="font-family: "courier new" , "courier" , monospace;">Get-PSAutorun | WriteOut -FileName AutoRuns</span>.<br />
It's then as simple as running .\Snapshot.ps1 and watch your computer-named directory populate, 0V3RW4TCH-2016-10-31-9-7 in my case, per <b>Figure 1</b>.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhYjQqacnO7NOn1vij5VDJ_P0_XheDoW0q7MTxN9TqNzYGvzNnaiTx9cm_8neNSbs0pm1qqy3IbmO9kgG0NJpLYZRc-XzijCpcA6_pZjxP5MyvKwSgiUmZ1RqubLzNSAZCeAg-hPw/s1600/snapshot.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="270" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhYjQqacnO7NOn1vij5VDJ_P0_XheDoW0q7MTxN9TqNzYGvzNnaiTx9cm_8neNSbs0pm1qqy3IbmO9kgG0NJpLYZRc-XzijCpcA6_pZjxP5MyvKwSgiUmZ1RqubLzNSAZCeAg-hPw/s320/snapshot.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 1:</b> Snapshot.ps1 run</td></tr>
</tbody></table>
Most result files are written in machine-readable XML, CSV, and TXT, as well as REG files generated by the registry exports via reg.exe.<br />
A great example of a results file, is spawned via <span style="font-family: "Courier New",Courier,monospace;">dir -Path c:\ -Hidden -Recurse -ErrorAction SilentlyContinue | Select-Object FullName,Length,Mode,CreationTime,LastAccessTime,LastWriteTime | Export-Csv -Path FileSystem-Hidden-Files.csv</span>. The resulting CSV is like a journey down evil memory lane, where all the nuggets I've tested in the past leave artifacts. This would be EXACTLY what you would be looking for under real response scenarios, as seen in <b>Figure 2</b>.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><img border="0" height="40" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjWrh3dA8J-u8Br_syhHf5nfBgJiKH-A0cDbm8fC62SGuNRpWeNNcZ3_6i-9EgPdQVM_pPTl84V_CFSNfpbu_sJ4IEqYM0CXs-zJr35cxXMx62hOJb9Dp08LsjwqPebPR0hMoS9_A/s400/HiddenFiles.png" style="margin-left: auto; margin-right: auto;" width="400" /></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Figure 2: Snapshot.ps1 grabs hidden files</td></tr>
</tbody></table>
<div class="separator" style="clear: both; text-align: center;">
</div>
Sure, there are bunches of related DFIR collection scripts, but I really like this one, and plan to tweak it further. Good work from Jason, and just one of many reasons to consider taking SEC505, or pursuing your GSE! <br />
Cheers...until next time.Russ McReehttp://www.blogger.com/profile/05647342839278416757noreply@blogger.com0tag:blogger.com,1999:blog-20011960.post-21861687176008490622016-10-08T14:21:00.000-07:002016-10-08T14:21:14.703-07:00Toolsmith Release Advisory: Malware Information Sharing Platform (MISP) 2.4.527 OCT 2016 saw the release of <a href="http://www.misp-project.org/2016/10/07/MISP-2.4.52-released.html" target="_blank">MISP 2.4.52</a>.<br />
MISP, Malware Information Sharing Platform and Threat Sharing, is free and open source software to aid in sharing of threat and cyber security indicators.<br />
An overview of MISP as derived from the project home <a href="http://www.misp-project.org/index.html" target="_blank">page</a>:<br />
<ul>
<li><b>Automation:</b> Store IOCs in a structured manner, and benefit from correlation, automated exports for IDS, or SIEM, in STIX or OpenIOC and even to other MISPs.</li>
<li><b><a href="https://holisticinfosec.org/df/simplicity" target="_blank">Simplicity</a>:</b> the driving force behind the project. Storing and using information about threats and malware should not be difficult. MISP allows getting the maximum out of your data without unmanageable complexity.</li>
<li><b>Sharing:</b> the key to fast and effective detection of attacks. Often organizations are targeted by the same Threat Actor, in the same or different Campaign. MISP makes it easier to share with and receive from trusted partners and trust-groups. Sharing also enables collaborative analysis, preventing redundant work.</li>
</ul>
The MISP 2.4.52 release includes the following new features:<br />
<ul>
<li><b>Freetext feed import:</b> a flexible scheme to import any feed available on Internet and incorporate them automatically in MISP. The feed imported can create new event or update an existing event. The freetext feed feature permits to preview the import and quickly integrates external sources.</li>
<li>Bro NIDS export added in MISP in addition to Snort and Suricata.</li>
<li>A default role can be set allowing flexible role policy.</li>
<li>Functionality to allow merging of attributes from a different event.</li>
<li>Many updates and improvement in the MISP user-interface including filtering of proposals at index level.</li>
</ul>
Bug fixes and improvements include:<br />
<ul>
<li>XML STIX export has been significantly improved to ensure enhanced compatibility with other platforms.</li>
<li>Bruteforce protection has been fixed.</li>
<li>OpenIOC export via the API is now possible.</li>
<li>Various bugs at the API level were fixed.</li>
</ul>
This is an outstanding project that will be the topic of my next <a href="http://holisticinfosec.blogspot.com/search?q=Toolsmith+In-depth+Analysis&max-results=20&by-date=true" target="_blank">Toolsmith In-depth Analysis</a>. <br />
<br />
Cheers...until next time.Russ McReehttp://www.blogger.com/profile/05647342839278416757noreply@blogger.com0tag:blogger.com,1999:blog-20011960.post-1090101143474923522016-09-16T19:10:00.001-07:002016-09-16T19:17:57.052-07:00Toolsmith In-depth Analysis: motionEyeOS for Security Makers<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhZ3eICFU3LPLdO6C6GkOzX2l3Elwd4VBBqLd1texO0LgP60FeLqT5t3SuPJKBj6HkuZkNb6pyG8sevoMHH3UScoLdyPNl63H1ZSX2Vm5P1doUdoQNuWJNMQWoY9PX7EVVy_0whqw/s1600/celebrating10years.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="170" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhZ3eICFU3LPLdO6C6GkOzX2l3Elwd4VBBqLd1texO0LgP60FeLqT5t3SuPJKBj6HkuZkNb6pyG8sevoMHH3UScoLdyPNl63H1ZSX2Vm5P1doUdoQNuWJNMQWoY9PX7EVVy_0whqw/s200/celebrating10years.png" width="200" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg0jN-Bo0ekoKyKHy5xgxkq6PfQwtDBwnJ3NgFNNaN4HLYe0Xw4Wzfv8eNpdy5f7S56zkHH1HRM9dO_-UK2IYnHAFY6vmCikTP6FrKfE1zGfun_7doi3uFk-kHuTRZqbXtY76-1cg/s1600/title.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg0jN-Bo0ekoKyKHy5xgxkq6PfQwtDBwnJ3NgFNNaN4HLYe0Xw4Wzfv8eNpdy5f7S56zkHH1HRM9dO_-UK2IYnHAFY6vmCikTP6FrKfE1zGfun_7doi3uFk-kHuTRZqbXtY76-1cg/s200/title.png" width="200" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
It's rather hard to believe, unimaginable even, but here we are. This is the 120th consecutive edition of toolsmith; every month for the last ten years, I've been proud to bring you insights and analysis on free and open source security tools. I hope you've enjoyed the journey as much as I have, I've learned a ton and certainly hope you have too. If you want a journey through the past, October 2006 through August 2015 are available on my web site <a href="https://holisticinfosec.org/df/toolsmith" target="_blank">here</a>, in PDF form, and many year's worth have been published here on the blog as well.<br />
I labored a bit on what to write about for this 10th Anniversary Edition and settled on something I have yet to cover, a physical security topic. To that end I opted for a very slick, maker project, using a Raspberry Pi 2, a USB web cam, and <a href="https://github.com/ccrisan/motioneyeos/wiki" target="_blank">motionEyeOS</a>. Per Calin Crisan, the project developer, motionEyeOS is a Linux distribution that turns a single-board computer into a video surveillance system. The OS is based on BuildRoot and uses motion as a backend and motionEye for the frontend.<br />
<ul>
<li><a href="http://buildroot.uclibc.org/" target="_blank">Buildroot</a> "is a simple, efficient and easy-to-use tool to generate embedded Linux systems through cross-compilation."</li>
<li><a href="https://motion-project.github.io/" target="_blank">Motion</a> (wait for it) is a program that monitors the video signal from cameras and is able to detect if a significant part of the picture has changed; in other words, it can detect motion.</li>
<li><a href="https://github.com/ccrisan/motioneye/" target="_blank">motionEye</a> is also Calin's project and is web frontend for the motion daemon.</li>
</ul>
<br />
Installation was insanely easy, I followed Calin's installation <a href="https://github.com/ccrisan/motioneyeos/wiki/Installation" target="_blank">guidelines</a> and used Win32DiskImager to write the image to the SD card. Here's how straightforward it was in summary.<br />
1) <a href="https://github.com/ccrisan/motioneyeos/releases" target="_blank">Download</a> the latest motionEyeOS image. I used build <a href="https://github.com/ccrisan/motioneyeos/releases/tag/20160828" target="_blank">20160828</a> for Raspberry Pi 2.<br />
2) Write the image to SD card, insert the SD into your Pi.<br />
3) Plug a supported web camera in to your Pi, power up the Pi. Give it a couple minutes after first boot per the guidelines: do not disconnect or reboot your board during these first two minutes. The initialization steps:<br />
<ul>
<li>prepare the data partition on the SD card</li>
<li>configure SSH remote access</li>
<li>auto-configure any detected camera devices</li>
</ul>
4) Determine the IP addressed assigned to the Pi, DHCP is default. You can do this with a monitor plugged in the the Pi's HDMI port, via your router's connected devices list, or with a network scan.<br />
For detailed installation instructions, refer to PiMyLifeUp's Build a <a href="https://pimylifeup.com/raspberry-pi-security-camera/" target="_blank">Raspberry Pi Security Camera Network</a>. It refers to a dated, differently named (motionPie) version of motionEyeOS, but provides great detail if you need it. There are a number of YouTube videos too, just search motionEyeOS.<br />
<br />
<a href="https://github.com/ccrisan/motioneyeos/wiki/Configuration" target="_blank">Configuration</a> is also ridiculously simple. Point your browser to the IP address for the Pi, http://192.168.248.20 for me on my wired network, and http://192.168.248.64 once I configured motionEyeOS to use my WiFi dongle.<br />
The first time you login, the password is blank so change that first. In the upper left corner of the UI you'll see a round icon with three lines, that's the setting menu. Click it, change your admin and user (viewer) passwords STAT. Then immediately enable Advanced Settings.<span id="goog_1286886680"></span><span id="goog_1286886681"></span> <br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgGsjWXvh7tD77ZmuhWQxKKcnjqzIkD-MLDF-1Q94i6L9Sz3U2A2-cJXy2eEu-3OsJ_9r8UjTu7GqDQxWKNXQ-Qj_LnLYPtwmuU4oOnWT2Frc5FWyc_9VaGEdPMZI1v03KHbpayBA/s1600/motioneye1+-+Copy.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgGsjWXvh7tD77ZmuhWQxKKcnjqzIkD-MLDF-1Q94i6L9Sz3U2A2-cJXy2eEu-3OsJ_9r8UjTu7GqDQxWKNXQ-Qj_LnLYPtwmuU4oOnWT2Frc5FWyc_9VaGEdPMZI1v03KHbpayBA/s320/motioneye1+-+Copy.png" width="259" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 1:</b> Preferences</td></tr>
</tbody></table>
<br />
You'll definitely want to add a camera, and keep in mind, you can manage multiple cameras with on motionEyeOS devices, and even multiple motionEyeOS systems with one master controller. Check out <a href="https://github.com/ccrisan/motioneyeos/wiki/Usage-Scenarios" target="_blank">Usage Scenarios</a> for more.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgQkJl34-dOR9eagO20sduQsXwsZ5u5g4vh0x-iSGizaIzWy2v8sLkJ3rNX_c6qhM5obUEyW9u_10sBkNBb1l7htW9WWxCPLmr5kj3qnU0e8h-P_8cmTqjpuREwNhmRDpPX_VyDLQ/s1600/motioneye2+-+Copy.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="153" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgQkJl34-dOR9eagO20sduQsXwsZ5u5g4vh0x-iSGizaIzWy2v8sLkJ3rNX_c6qhM5obUEyW9u_10sBkNBb1l7htW9WWxCPLmr5kj3qnU0e8h-P_8cmTqjpuREwNhmRDpPX_VyDLQ/s320/motioneye2+-+Copy.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 2:</b> Add a camera</td></tr>
</tbody></table>
<br />
Once your camera is enabled, you'll see its feed in the UI. Note that there are unique URLs for snapshots, streaming and embedding.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhnDUZuxX0kpKrSE7kzO-wrekyCz4S-psSs3RuOvb1fewu_XvjcWLpcLIN3MF0WHInVJO9R562fNJ4cofgsXoBttwQcwTDfXBQd9iTYGSDftDlRNB0d6zcv-tL-NXVx_LKvLdgfJw/s1600/motioneye9.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="198" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhnDUZuxX0kpKrSE7kzO-wrekyCz4S-psSs3RuOvb1fewu_XvjcWLpcLIN3MF0WHInVJO9R562fNJ4cofgsXoBttwQcwTDfXBQd9iTYGSDftDlRNB0d6zcv-tL-NXVx_LKvLdgfJw/s400/motioneye9.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 3:</b> Active camera and URLs</td></tr>
</tbody></table>
When motion detection has enabled the camera, the video frame in the UI will be wrapped in orange-red. You can also hover over the video frame for additional controls such as full screen and immediate access to stored video. <br />
<br />
There are an absolute plethora of settings options, the most important of which, after camera configuration, is storage. You can write to local storage or a network share, this quickly matters if you choose and always-on scenario versus motion enabled.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhexPox5hyEph5rHD3wPN2AtjW5uQuUX8PU6nh9TLm3sqfuEtJQ3Wf6tEZbD6KX4U2Y5IJsHqFL5RoQB52XCLKWeWHc9O8B5CTRUFUquI3U-D_XyWspAfBWFGCOCRp1vsvP5FHdaw/s1600/motioneye5.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="310" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhexPox5hyEph5rHD3wPN2AtjW5uQuUX8PU6nh9TLm3sqfuEtJQ3Wf6tEZbD6KX4U2Y5IJsHqFL5RoQB52XCLKWeWHc9O8B5CTRUFUquI3U-D_XyWspAfBWFGCOCRp1vsvP5FHdaw/s320/motioneye5.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 4:</b> Configure file storage</td></tr>
</tbody></table>
You can configure text overlay, video streaming, still images, schedules, and more.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh8Go7-cszil57rkZ7fS8pslMeR1ndj5uA3JBw0qV5tm3iNX5uqWqhMYFQ4ct8YVwLd9cTYou69sY0KUkZRwPSeNMRn0o8Fq33PtZxoTYcexH9F-EgwkWNe-ja79Jxl-f2slSVYgg/s1600/motioneye6.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="265" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh8Go7-cszil57rkZ7fS8pslMeR1ndj5uA3JBw0qV5tm3iNX5uqWqhMYFQ4ct8YVwLd9cTYou69sY0KUkZRwPSeNMRn0o8Fq33PtZxoTYcexH9F-EgwkWNe-ja79Jxl-f2slSVYgg/s320/motioneye6.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 5:</b> Options, options, options</td></tr>
</tbody></table>
The most important variable of all us how you want to be notified. <br />
There are configuration options that allow you to run commands so you script up a preferred process or use one already devised.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjNDtclg-lBS6F9_-M9NaGQogHffC0mTrywTyJs-fRerGAdtQYgkfU5IbW5BUZUSIcBhxRXGiEthbSavCDccUWtvGLGxmZcae2ns8DADrCAJjDu5GeLL7CFGo07931KX2Zx3Fs1kg/s1600/motioneye7.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="122" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjNDtclg-lBS6F9_-M9NaGQogHffC0mTrywTyJs-fRerGAdtQYgkfU5IbW5BUZUSIcBhxRXGiEthbSavCDccUWtvGLGxmZcae2ns8DADrCAJjDu5GeLL7CFGo07931KX2Zx3Fs1kg/s320/motioneye7.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 6:</b> Run a command for notification</td></tr>
</tbody></table>
<br />
Best of all, you can make uses of a variety of notification services including email, as well as Pushover, and IFTTT via Web Hooks.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEix24QgtfZPou_nsMN7vEOeL_vwQlnmoybHpmMVErpjGUxhnitxc6MUcc4MR9QD4XXjLY_maVPcg9Q4RY3TuscpPHdJbVc3RSvL4U4EPftLN7Jb1pUsi3UNPenYh7gThyphenhyphenZ9cKFpJw/s1600/motioneye8.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="139" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEix24QgtfZPou_nsMN7vEOeL_vwQlnmoybHpmMVErpjGUxhnitxc6MUcc4MR9QD4XXjLY_maVPcg9Q4RY3TuscpPHdJbVc3RSvL4U4EPftLN7Jb1pUsi3UNPenYh7gThyphenhyphenZ9cKFpJw/s320/motioneye8.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 7:</b> Web Hook notifications</td></tr>
</tbody></table>
There is an <a href="https://www.pi-supply.com/make/adding-push-notifications-motioneyeos-formerly-motionpie/" target="_blank">outstanding article</a> on using Pushover and IFTTT on Pi Supply's Maker Zone. It makes it easy to leverage such services even if you haven't done so before.<br />
The net result, after easy installation, and a little bit of configuration is your on motion-enabled CCTV system that costs very little compared to its commercial counterparts.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjhORBOZLQjoF-ZT2TdAGdUJNoltufiOVSKOgSkfQERKJV5yIOO8A4IyBpxHEHLOcBrg5YN-lNnyGUC1Y1bhu4uTXVft97dPXl0_R6GgnlVEO6vjDDh7VWzqxhyUgTO-aHmXYSoXg/s1600/motiondetected.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="239" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjhORBOZLQjoF-ZT2TdAGdUJNoltufiOVSKOgSkfQERKJV5yIOO8A4IyBpxHEHLOcBrg5YN-lNnyGUC1Y1bhu4uTXVft97dPXl0_R6GgnlVEO6vjDDh7VWzqxhyUgTO-aHmXYSoXg/s320/motiondetected.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><b>Figure 8:</b> Your author entering his office under the watchful eye of Camera1</td></tr>
</tbody></table>
Purists will find image quality a bit lacking perhaps, but with the right camera you can use <a href="https://github.com/ccrisan/motioneyeos/wiki/Fast-Network-Camera" target="_blank">Fast Network Camera</a>. Do be aware of the drawbacks though (lost functionality).<br />
<br />
In closing, I love this project. Kudos to Calin Crisan for this project. Makers and absolute beginners alike can easily create a great motion enabled video/still camera setup, or a network of managed cameras with always on video. The hardware is inexpensive and readily available. If you've not explored Raspberry Pi this is a great way to get started. If you're looking for a totally viable security video monitoring implementation, motionEyeOS and your favorite IoT hardware (the project <a href="https://github.com/ccrisan/motioneyeos/wiki/Supported-Devices" target="_blank">supports other boards</a> too) are a perfect combo. Remember too that there are Raspberry Pi board-specific camera modules available.<br />
<br />
<div class="MediumGrid21">
Ping me via email or Twitter if you have questions (russ
at holisticinfosec dot org or <a href="https://twitter.com/holisticinfosec/" target="_blank">@holisticinfosec</a>).</div>
<div class="MediumGrid21">
Cheers…until next time.</div>
Russ McReehttp://www.blogger.com/profile/05647342839278416757noreply@blogger.com0tag:blogger.com,1999:blog-20011960.post-57085923852282054172016-09-10T12:10:00.000-07:002016-09-10T12:11:38.108-07:00Best toolsmith tool of the last ten yearsAs we celebrate Ten Years of Toolsmith and 120 individual tools covered in detail with the attention they deserve, I thought it'd be revealing to see who comes to the very top of the list for readers/voters.<br />
I've built a poll from the last eight Toolsmith Tools of the Year to help you decide, and it's a hell of a list.<br />
<ul>
<li><a href="https://twitter.com/mandiant" target="_blank">@Mandiant</a>'s <b>Memoryze</b> (2008)</li>
<li><a href="https://twitter.com/paterva" target="_blank">@Paterva's</a> <b>Maltego</b> (2009)</li>
<li><a href="https://twitter.com/sansforensics" target="_blank">@SANSForensics</a>' <b>SIFT</b> (2010)</li>
<li><a href="https://twitter.com/zaproxy" target="_blank">@zaproxy</a> and <a href="https://twitter.com/psiinon" target="_blank">@psiinon</a>'s <b>OWASP Zed Attack Proxy</b> (2011)</li>
<li><a href="https://twitter.com/ModSecurity" target="_blank">@Modsecurity</a> and <a href="https://twitter.com/ivanristic" target="_blank">@IvanRistic</a>'s <b>Modsecurity for IIS</b> (2012)</li>
<li><a href="https://twitter.com/LaNMaSteR53" target="_blank">@LaNMaSteR53</a> <b>Recon-ng</b> (2013)</li>
<li><a href="https://twitter.com/joshsokol" target="_blank">@joshsokol</a>'s <b>Simple Risk</b> (2014)</li>
<li><a href="https://twitter.com/beenuar" target="_blank">@beenuar</a> <b>Hook Analyser</b> (2015) </li>
</ul>
Amazing, right? The best of the best.<br />
<br />
You can vote in the poll to your right, it'll be open for two weeks.Russ McReehttp://www.blogger.com/profile/05647342839278416757noreply@blogger.com0tag:blogger.com,1999:blog-20011960.post-3608829827116543222016-09-04T12:50:00.001-07:002016-09-04T13:20:19.018-07:00Toolsmith Tidbit: Will Ballenthin's Python-evtxAndrew Case (<a href="https://twitter.com/attrc" target="_blank">@attrc</a>) called out Will Ballenthin's (<a href="https://twitter.com/williballenthin" target="_blank">@williballenthin</a>) Python-evtx on Twitter, reminding me that I'm long overdue in mentioning it here as well.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhS8_N5WklyYIzFc9b1KC9XTCuL5g4lgYh5fJ48dYADq53f4YZR3vWvTxR7YGx2s2vQNMcuhqT0JIYYNpKc_7DwtRplFZqqjRtOztBU8Tg7LzB5B4jZ6MHgXd8DkPryVqUVSK5ADg/s1600/Python-evtx.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="63" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhS8_N5WklyYIzFc9b1KC9XTCuL5g4lgYh5fJ48dYADq53f4YZR3vWvTxR7YGx2s2vQNMcuhqT0JIYYNpKc_7DwtRplFZqqjRtOztBU8Tg7LzB5B4jZ6MHgXd8DkPryVqUVSK5ADg/s400/Python-evtx.png" width="400" /></a></div>
Will's Python-evtx description from his <a href="http://www.williballenthin.com/evtx/index.html" target="_blank">website</a> for same follows:<br />
<i>"python-evtx is a pure Python parser for recent Windows Event Log files (those with the file extension “.evtx”). The module provides programmatic access to the File and Chunk headers, record templates, and event entries. For example, you can use python-evtx to review the event logs of Windows 7 systems from a Mac or Linux workstation. The structure definitions and parsing strategies were heavily inspired by the work of Andreas Schuster and his Perl implementation Parse-Evtx."</i><br />
<br />
Assuming you've running Python 2.7, install it via <span style="font-family: "Courier New",Courier,monospace;">pip install python-evtx</span> or download source from Github: <a href="https://github.com/williballenthin/python-evtx">https://github.com/williballenthin/python-evtx</a><i> </i>Russ McReehttp://www.blogger.com/profile/05647342839278416757noreply@blogger.com0tag:blogger.com,1999:blog-20011960.post-66638644655347927762016-09-04T12:50:00.000-07:002016-09-04T12:55:37.020-07:00Toolsmith Release Advisory: Kali Linux 2016.2 ReleaseOn the heels of Black Hat and DEF CON, 31 AUG 2016 brought us the second Kali Rolling ISO release aka <a href="https://www.kali.org/news/kali-linux-20162-release/" target="_blank">Kali 2016.2</a>. This release provides a number of updates for Kali, including:<br />
<ul>
<li>New KDE, MATE, LXDE, e17, and Xfce builds for folks who want a desktop environment other than Gnome.</li>
<li>Kali Linux Weekly ISOs, updated weekly builds of Kali that will be available to download via their mirrors.</li>
<li>Bug Fixes and OS Improvements such as HTTPS support in busybox now allowing the preseed of Kali installations securely over SSL. </li>
</ul>
All details available here: <a href="https://www.kali.org/news/kali-linux-20162-release/">https://www.kali.org/news/kali-linux-20162-release/</a><br />
Thanks to <a href="https://isc.sans.edu/handler_list.html#rob-vandenbrink" target="_blank">Rob Vandenbrink</a> for calling out this release. <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg7bgKrJPTlYbVEy6zevz0iffiZ9TkbT4AIWyj3aWyXBiijjRz3sutGR5hJR6oPCdVuS-ydHuDB3RJVjDRmHF-_P8yl7fTjVB4I3JFLaMkOWpIS-NEdD5kxe1P1p3eWV72eBjC9bA/s1600/kali-rolling-2016-2-release3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="227" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg7bgKrJPTlYbVEy6zevz0iffiZ9TkbT4AIWyj3aWyXBiijjRz3sutGR5hJR6oPCdVuS-ydHuDB3RJVjDRmHF-_P8yl7fTjVB4I3JFLaMkOWpIS-NEdD5kxe1P1p3eWV72eBjC9bA/s640/kali-rolling-2016-2-release3.png" width="640" /></a></div>
Russ McReehttp://www.blogger.com/profile/05647342839278416757noreply@blogger.com0