As I mentioned before once you export firewall logs into human-readable format you can do lots of interesting things - for example script that gives statistics of how many times each Security rule was hit .Be aware that this counts explicit Security rules only - i.e. the ones you see in Security tab of the Smartdashboard. No other rules you usually see in Smartview Tracker are counted - e.g. SmartDefense,Web Filtering etc. Also afterwards I sort it by number of hits to see what rules are used most:
awk -F\; ' {match($0,/rule: +([0-9]+)/,rules);rule_count[rules[1]]++} END {for (rule_number in rule_count) print " Rule number: " rule_number " Hits: " rule_count[rule_number]}' ./fw.log.txt | sort -n -k5 Rule number: Hits: 1197330 <strong> Ignore this line as it counts non-matched lines I dont want to filter with additional conditions and added time processing</strong> Rule number: 2 Hits: 9 Rule number: 5 Hits: 366 Rule number: 11 Hits: 12296 Rule number: 9 Hits: 14457 Rule number: 0 Hits: 17094 Rule number: 1 Hits: 44066 Rule number: 7 Hits: 233643 Rule number: 10 Hits: 366275 Rule number: 6 Hits: 424639 Update 2012 Below is the script to use Rule ID instead of Rule sequential numbers - this way changing rules order will not affect statistics. The script matches also non-security rules - e.g. email session id, that are a bit shorter then Rule ID, but I didn't want to slow down the processing with additional formatting .
awk -F\; ' {match($0,/{([[:print:]]+)}/,rules);rule_count[rules[1]]++} END {for (rule_number in rule_count) print " Rule number: " rule_number " Hits: " rule_count[rule_number]}' ./fw.log.txt | sort -n -k5 Rule number: D199972C-ED3E-4EB4-8B83-813333156D18 Hits: 175 Rule number: 85A905A7-951E-4100-A4BA-E13333151D29 Hits: 219 Rule number: 81333316-E942-4313-BB7D-E1333315802F Hits: 1519 Rule number: 71333215-2DB5-4A3A-95BC-5080AD0F5564 Hits: 2298 Rule number: 11331315-AE52-44E0-A42A-711029B5768E Hits: 3755 Rule number: 01333315-D290-4B05-AFE7-23BF24D889FF Hits: 4116 Rule number: 121FA62F-3885-4328-8090-BF1333315eB1 Hits: 399793 Rule number: FE40E076-BAEB-4979-8E41-5EF1333315e6 Hits: 440101 Rule number: BB3F6772-4D38-4D5A-952A-301333315de8 Hits: 1354341 Running time for a file of 900 Mb with 4.7 million records real 5m50.287s user 4m22.890s sys 0m3.190sFollow me on https://www.linkedin.com/in/yurislobodyanyuk/ not to miss what I publish on Linkedin, Github, blog, and more.
