1 / 50

IronBee : O pen source WAF engine with a commercial offering

IronBee : O pen source WAF engine with a commercial offering. Brian Rectanus Director of Engineering, WAF. Who am I?. Past: Commercial WAF developer since 2007 ModSecurity maintainer 2007 – 2010 IDS/IPS Developer (OISF Suricata ) Present: Lead WAF development @ Qualys in Madison

thuong
Download Presentation

IronBee : O pen source WAF engine with a commercial offering

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. IronBee: Open source WAF engine with a commercial offering Brian Rectanus Director of Engineering, WAF

  2. Who am I? Past: • Commercial WAF developer since 2007 • ModSecurity maintainer 2007 – 2010 • IDS/IPS Developer (OISF Suricata) Present: • Lead WAF development @ Qualys in Madison • IronBee architect and developer Lockdown 2013

  3. What am I covering… • Briefly: The what and why of WAF • IronBee, modules and rules • Overview of Qualys commercial WAF (beta) • How we use IronBee • How we have simplified the process • Beta features Lockdown 2013

  4. WAF: What is it? • Web Application Firewall • To many this means: “Block web based attacks.” • But, WAFs are known to be a pain • There must be more Lockdown 2013

  5. WAF: Why does it exist? • View inside your web applications • Log (and potentially block) suspicious activity • Block known and obvious attacks and tools • Limit attack surface • Buy time to fix problems • Feed your developers with more details • Deal with legacy products Lockdown 2013

  6. WAF: What gets in the way? • Different interpretations of HTTP • Document types (HTML, XML, JSON) • Encodings (URL, Base64, entities) • Different Vectors (server, browser, DB, DoS) • Evasion techniques • Application logic (auth, sessions, BI) • Encryption, compression, obfuscation Lockdown 2013

  7. WAF: What can go wrong? • False Positives (oops) • False Negatives (didn't see it) • Performance cannot suffer (too much) • Device failure (site is down) Lockdown 2013

  8. WAF: How can we make it better? • Easier to setup and manage • Separate server/security configs and management • Low False Positives and low tuning costs • Flexible deployments with automated updates • Manage it all centrally • Extensible engine • Solid framework for writing security logic • Integrate with other products • Combine many advanced techniques with correlation • Acceptable performance • Intelligent application of security logic with fast algorithms Lockdown 2013

  9. IronBee : What is it? • Open Source (Apache Software License v2) github.com/ironbee • Framework to inspect, block, modify and log • Extremely flexible • Highly extensible • Tries not to get in your way Lockdown 2013

  10. IronBee: Who is involved? Christopher Alfeld, PhD Mathematics and UW alumni Experimental projects, performance, algorithms, C++ API Sam Baskinger Data structures, Configuration, Lua API Nick Kew, Apache Foundation Server plugins: Apache Trafficserver, Apache httpd, nginx, tserver, … Nick LeRoy Core engine, Testing Brian Rectanus Initial IronBee author, now architect and manager Ivan Ristić Security Research (SSL Labs – ssllabs.com, LibHTP, ModSecurity) Many other supporting players at Qualys – too many to name here. Lockdown 2013

  11. IronBee: What's the basic concept? • Server provides HTTP data • Web server, proxy, IDS, … • Parsers break data into fields/streams • Headers, URI, POST body, cookies, … • Modules/Rules inspect these fields/streams • Sigs, scoring, tracking, learning, correlation, … • Actions performed: • Log, block, modify, track, … Lockdown 2013

  12. IronBee: What's a server? • Provide HTTP data to IronBee • Implement blocking, modification (if possible) • Current: • Apache Trafficserver plugin • Apache Webserver module • Nginx plugin • Tserver (nginx fork) plugin • Clipp (command line with PCAP support) Lockdown 2013

  13. IronBee: What's the engine do? • Notification of events • Core HTTP fields to inspect • Rule execution • Configuration • Logging Very minimalistic, and becoming more so. Lockdown 2013

  14. IronBee: What are modules? • Dynamically loadable shared libraries in C, C++ • Minimal modules in Lua, but reloadable with config • Hook into IronBee events • Extend functionality (C/C++ only), such as: • Parsers, normalizers, operators and actions • Rule languages (and extensions) • Embed scripting languages (Lua) • Enable technologies (libinjection - SQLi detection library) • Correlation (combine sigs, scoring, tracking, learning, …) • Logging • … Lockdown 2013

  15. IronBee: What are rules? • Inspect data and perform actions • Simple signature language • Complex DSL (Lua @ config time) • Full scripting language (Lua @ runtime) • Extendible via modules Lockdown 2013

  16. Module: Simple Rule Language Specify fields, inspect and perform an action: Rule <fields> <op> <meta/actions> Rule REQUEST_HEADERS\ @rx"attack|pattern"\ id:ex/1 rev:1 \ phase:REQUEST_HEADER \ event Lockdown 2013

  17. Module: Simple Rule Language Transformations and meta data: Rule REQUEST_HEADERS.count() \ @gt 15 \ id:ex/2 rev:1 \ phase:REQUEST_HEADER \ severity:75 confidence:80 \ tag:http/limits \ event Lockdown 2013

  18. Module: Simple Rule Language Capture potential CC#s, blocking more than 10: StreamInspect RESPONSE_BODY_STREAM \ @dfa "\d{15,16}" \ id:ex/3 rev:1 \ capture:CC Rule CC.count() \ @gt 10 \ id:ex/4 rev:1 \ phase:RESPONSE_BODY \ event block:immediate Lockdown 2013

  19. Module: Simple Rule Language • These are just signature rules • Simpleand come with limitations • Config file syntax (single line) • Somewhat verbose (requires id/phase) • No real flow control other than phase/file order • Other types of rules eliminate these limits Lockdown 2013

  20. Module: Lua • Embedded scripting language • As a configuration DSL (config time) • As a basic module (core engine runtime) • As a rule (rule engine runtime) Lockdown 2013

  21. Lua: As a DSL DSL is named "waggle" (we like Bee themes here) Rule REQUEST_HEADERS \ @rx"attack|pattern" \ id:ex/1 rev:1 \ phase:REQUEST_HEADER \ event Sig("ex/1w", 1): fields("REQUEST_HEADERS"): op("rx", "attack|pattern"): phase("REQUEST_HEADER"): action("event") Lockdown 2013

  22. Lua: Programmatic Rules Config Lua @ config time means full support for functions, loops, etc. -- Parameterized rule with id/regex local function RequestRegex(id, regex) return Sig("test/lua/" .. id, 1): fields("REQUEST_HEADERS”): op("rx", regex): phase("REQUEST"): actions("event”) end -- Simplify management and readability RequestRegex(1, [[attack|pattern]]) RequestRegex(2, [[attack2|pattern2]]) Lockdown 2013

  23. Lua: Basic Modules Lua executed at runtime to handle core engine events. -- Get the IronBeeModule object. local ibmod= ... -- Define a function to handle an event. local function log_event(ib) ib:logInfo("Handling event=%s”, ib.event_name) return 0 end -- Register to be called with the event. ibmod:request_header_finished_event(log_event) Lockdown 2013

  24. Lua: Rules • Similar to Lua module, but less complex • Lua executed by the rule execution engine • Entire script runs vs. using event callbacks Lockdown 2013

  25. Rules: Scaling to the non-trivial • Simple linear execution with basic rules • Executes a list of rules per phase • All rules are executed • What about 1000s or 100,000s of rules? • Need a way to limit execution • Need a way to specify dependencies/order • Need a way to cache results • Need a higher level of logic and correlation Lockdown 2013

  26. Rules: Made to be extended • Rule injection • Modules can take ownership of rules • Modules can decide if/when rules execute • Currently two modules use this facility • Fast rules module • Predicate rules module Lockdown 2013

  27. Module: Fast Rules • Adds a fast pattern (prequalification) to rules • Rules are executed only if prequalified • All fast rules utilize modified Aho-Corasick • Extensions to utilize fixed width patterns • Speed is independent of number of patterns • Works best with large rulesets • Some limitations Lockdown 2013

  28. Fast Rules: An example Utility suggests fast patterns for existing rules by adding comments to rules # FAST RE: ^(.+),\s*max-age[^,]+,?(.*)$ # FAST Suggest: "fast:max-age[^,]" Rule RESPONSE_HEADERS:Cache-Control \ @rx "^(.+),\s*max-age[^,]+,?(.*)$" … Rule RESPONSE_HEADERS:Cache-Control \ @rx "^(.+),\s*max-age[^,]+,?(.*)$" "fast:max-age[^,]" … Lockdown 2013

  29. Module: Predicate Rules • Uses Lua DSL to produce predicate expressions (and (gt (atoi (field 'Content-Length')) 0) (streq 'GET' (field 'Request-Method')) ) • Complex rules are built from simple rules • Rules form an knowledge graph • Graph optimizations performed at configuration time • Common sub-expression merging & caching • Only required rules execute, and only once • Combines Lua DSL and runtime optimizations • Full Lua support enhances configuration • Graph optimizations enhance runtime Lockdown 2013

  30. Predicate Rules: Named predicates -- Parameterized named predicate local function header(name) return P.Field('REQUEST_HEADERS'):sub(name) end -- Named predicates local range_header_too_long = P.Gt(header('Range'):length(), 1000) local host_header_too_long= P.Gt(header('Host'):length(), 100) -- Combine named predicates into a rule/signature -- NOTE: A "/" operator is overloaded for predicates to P.Or(…) Sig(”ex/p/1", 1): predicate( range_header_too_long / host_header_too_long): phase([[REQUEST_HEADER]]): action([[event]]): message([[Invalid HTTP header: too long.]]) Lockdown 2013

  31. Predicate Rules: Lua DSL in action local sensitive_file_patterns= { unix = [[(?:/etc/passwd|/etc/hosts|/etc/shadow|/bin/id)$]], java = [[(?:WEB-INF/web.xml|/conf/server.xml)$]], apache = [[(?:.htaccess|.htpasswd|.meta|.web)$]] } local function contains_sensitive_files(pattern) local r = P.false for i,v in ipairs({"REQUEST_URI_PATH", "REQUEST_HEADERS", "ARGS"}) r = P.Or(r, P.rx(pattern, P.Field(v):remove_whitespace())) end return r end for name,pattern in pairs(sensitive_file_patterns) do Sig("qrs/LFi/" .. name, "1"): predicate(contains_sensitive_files(pattern)): phase([[REQUEST_HEADER]]): action([[event]]): message("LFi: request for sensitive " .. name .. " files.") end Lockdown 2013

  32. Framework: Automata • Iron Automata (we also like Iron themes here) • Framework and utils for building automata • Splits generation, optimization, execution • Generic execution environment, Eudoxus • Example Automata: Enhanced Aho-Corasick • Caseless matches • Fixed width patterns/sets (char sets, negation • Can be tuned for space vs time through Eudoxus Lockdown 2013

  33. IronAutomata: Aho-Corasick Example1 • Aho-Corasick • Unoptimized • Patterns: • he • she • his • hers Lockdown 2013

  34. IronAutomata: Aho-Corasick Example2 • Aho-Corasick • Speed Optimized • Patterns: • he • she • his • hers Lockdown 2013

  35. IronAutomata: Optimization • Aho-Corasick • Patterns: ~250k English Dictionary • Data: Text of "Pride and Predjudice" novel 10x Lockdown 2013

  36. Module: Eudoxus Executor Execute compiled, eudoxus automata. • Large signature database • Spam keywords • Known attack patterns • Link reputation • Custom, auto generated automata • Based on research • Based on website traffic profiling Lockdown 2013

  37. Utility: Clipp • Command line utility • Testing and rule development • HTTP data via: Raw files, PCAP, protobuf, … • Modify HTTP data via filters • Convert between formats • Highly extendible • Ruby wrapper for unit/regression testing Lockdown 2013

  38. IronBee: Batteries not included • Management is not dictated, so… • No Config Management • No Rule Management • No Log Management • Must do these yourself • You should already be doing this • The point is to stay out of your way • Allow you to use your own management tools Lockdown 2013

  39. Qualys WAF: What will it add? • Managed WAF appliances via cloud • Automated updates • Software • Modules • Rules • Integration with other Qualys products • Web Application Scanning • Asset Management Lockdown 2013

  40. Qualys WAF Beta: What's offered? • Initially Amazon Web Services Platform • EC2 Classic and VPC • Clustering via ELB • Auto-scaling • You decide how much power you need • We are expanding to other platforms Lockdown 2013

  41. Qualys Beta WAF: What's it do? • Manage AWS based WAF Appliances • Generic attack detection • Declarative security (fixup cookies/headers) • Data leakage detection • Reduce attack surface (HTTP limitations) • ACLs (IP and geo) Lockdown 2013

  42. Qualys WAF Beta: What's it look like? • Manage AWS Appliances • Manage events • Generic attack detection • Declarative security • Data leakage detection • Reduce attack surface • Access Control Lockdown 2013

  43. QualysWAF Beta: AppSec Lockdown 2013

  44. Qualys WAF Beta: InfoLeak Lockdown 2013

  45. Qualys WAF Beta: Fixups Lockdown 2013

  46. Qualys WAF Beta: HTTP Lockdown 2013

  47. Qualys WAF Beta: ACLs Lockdown 2013

  48. Qualys WAF: What's coming? • QualysGuard integration • WAS scan result feedback • Shared assets • False positive mitigation • Exception handling • Website and session profiling • Reporting Lockdown 2013

  49. We are Hiring in the Madison! • Product Management • Application Security Researchers • Developers • QA Contact me if you are interested. Lockdown 2013

  50. Thanks! github.com/ironbee qualys.com/waf qualys.com/careers Feel free to contact me for more info. Brian Rectanus brectanus@qualys.com Lockdown 2013

More Related