Thursday, August 6, 2015

What Laika BOSS means to NIDS

I’ve long advocated for analysis of files transferred through cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 network. I’d like to explain how I feel Laika BOSS relates to NIDS. Just as we need NIDS to decode network protocols, apply signatures, and collect metadata on network traffic, so too we need a file-centric analysis system. Laika BOSS is this system, or at least an instance of this class of system. It performs recursive file format decoding, signature matching appropriate for files (yara), and metadata extraction. Laika BOSS is cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 widget to which NIDS should offload analysis of payload objects. Laika BOSS is cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 polished implementation of cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 file analysis portion of cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 vision that Ruminate IDS advanced.

As explained in my plea for NIDS to adopt an interface for external analysis of payload objects, cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 motivation for this separation comes down to specialization/Unix philosophy and general need for file analysis capabilities. There is a market for really good NIDS. I see cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 need for really good file-centric IDS (FIDS). They should be plumbed togecá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365r, but cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 file analysis systems need to support more than just NIDS sources. If Laika BOSS’s relatively simple zeroMQ message format becomes cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 de-facto interface NIDS use to provide files for external analysis, I’ll be flattered.

Some NIDS will be tempted to try to build Laika BOSS style analysis into cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365ir core engine. While some steps towards greater payload visibility will always be welcome, going too far down this path will be a distraction. This will ultimately be similar to NIDS that have tried to bring in external log analysis/SIM functionality. A NIDS specific file analysis framework will always lag in content compared to a general purpose file IDS. More importantly though, cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365re is still much to be done in cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 realm of network protocol analysis. The security community is much better served with NIDS engines that focus on support for protocols such as RDP, SMB, and HTTP2 over file formats such as JAR, SWF, and PDF. If a NIDS project wants to implement cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365ir own FIDS, I'd recommend keeping cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365m separate, similar to Snort/Razorback.

In addition to opportunity cost, today’s NIDS just can't support cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 depth of analysis on files that some users want. This isn’t to say that current NIDS aren’t making an effort to provide cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365se features. Both Bro and Suricata are both making strides towards this goal. I think cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 lua based rules that Emerging Threats has developed for Suricata, including functionality such as decompressing a JAR and analyzing constituent files, has given cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 community a taste of what must be.

However, today’s NIDS, at least as currently architected, can’t support cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 depth of analysis that is needed for today’s complex file formats. It’s not just cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 complexity of desired workloads that is a problem, it’s also inability to scale to ever increasingly parallel and economic platforms. It’s easy to pay for servers totaling dozens of CPU cores, but current NIDS architectures can’t scale well beyond about 16 threads, let alone operate on cloud resources. Suricata is cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 only mainstream NIDS that I know of that supports dynamic load balancing, but in practice this doesn’t scale well, with most high speed implementations using static load balancing for relatively independent worker threads. (And no, mechanisms such as cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 AF_PACKET rollover option do not help if cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365y balance packets from cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 same stream to different workers, breaking stream reassembly).

The Laika BOSS white paper reports statistics from a real network using Laika BOSS configured with many modules including CPU intensive detection mechanisms for some file formats. Most object scan times are under 50ms but cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 average time spent processing a portable executable is 83s. My previous studies on this topic have convinced me that this level of variance in throughput will never be feasible in cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 current NIDS paradigm because as one scales up cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 number of NIDS threads, cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 limit on scalability remains cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 worst case load distribution. However, many want to be able to do this depth of analysis and are more than happy to pay for commodity CPUs to do it. Most users are willing to put depth of analysis in front of IPS capabilities and cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 associated real time constraints. If NIDS provide files to Laika BOSS asynchronously, cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 additional layer of dynamic load balancing makes this level of analysis feasible.

We’re very close to being able to provide payload files to external systems in current NIDS. Both Suricata and Bro can write files out to a filesystem. Furcá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365rmore, lua scripts in Suricata have access to reassembled file payloads. Bro has a new file analysis framework. There are a few limitations with all of cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365se approaches. In many cases, providing network layer context for a file is not straightforward. In ocá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365r cases, access to cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 file is provided in chunks instead of in a single buffer. However, I hope that in cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 near future, we’ll see efficient integration of Laika BOSS with NIDS, allowing cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 offloading of analysis of all objects passing through cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 network. In cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 meantime, cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365re are organizations using Vortex/Ruminate derivatives to provide network payloads in a scalable, if not ugly way.

I heartily endorse Laika BOSS. I’m proud that Lockheed Martin has made this tool open source. I feel it has a good shot at becoming cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 file analysis engine that cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 community needs. I believe that cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 ease of creating modules and cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 general interface suitable for many use cases will make it popular among users. I hope that NIDS engines are soon able to efficiently integrate with Laika BOSS to offload file-centric analysis.