Table of Contents
- IntroductionStart
nfcapd netflow collector in a daemon mode listening on port 5001 with all extensions enabled and saving received netflow data into the named folder NFS-cisco-rtr. Accept netflow records only coming from the sender with the IP of 13.13.13.137Read and print all records form a single fileDisplay cumulative statistics about all the flows in a records fileRead and print all records from a range of files, starting at nfcapd.202004221040 and up to but not including the current file still being written to nfcapd.current.1609Read all records from a range of files, starting at nfcapd.202209242120 and finishing at nfcapd.202209242150Print sessions where the source or destination IP is 8.8.8.8Print sessions where the destination port is 53, the destination IP is 8.8.8.8, and the protocol is TCPShow top 10 flows sorted by the bits per second statisticsShow all flows sorted by the bits per second statisticsAggregate all flows to/from host 8.8.8.8 based on source IPCalculate statistics for port 443 traffic and sort by bps to see bandwidth abusing hostsSort presented flows by duration, longest at the bottomAnonymize IP addresses in all the flows in the file, overwrite in-placeFind records in the time range of 12:19:00 - 12:20:00 matching the filter of protocol = UDP and port = 53Introduction
Few facts to know before diving into examples:
nfdump packet filter syntax is tcpdump-compatible, and it should come as the last argument on the line.
nfcapd daemon receives Netflow streams and saves them into local files,switching to a new file every 5 minutes (configurable). The naming starts with nfcapd, then dot, and finally date and time stamp.
The newest version for 2022 is 1.7, which is multi-threaded.
There is a GUI web based front end nfsen, which is a separate install.
nfdump reads files from filesystem and outputs to either STDOUT or to binary files (if used with -w option). If it runs out of host memory or free disk space for temporary files, it will crash.
Start nfcapd netflow collector in a daemon mode listening on port 5001 with all extensions enabled and saving received netflow data into the named folder NFS-cisco-rtr. Accept netflow records only coming from the sender with the IP of 13.13.13.137
nfcapd -D -T all -n NFS-cisco-rtr,13.13.13.137,/var/flows/NFS-cisco-rtr -p 5001Read and print all records form a single file
Here the records file is nfcapd.202004221040nfdump prints record in random order, not sorted by any means.
nfdump -r nfcapd.202004221040Display cumulative statistics about all the flows in a records file
nfdump -I -r nfcapd.202004221040Output:
Ident: NFS-cisco-rtrFlows: 378330Flows_tcp: 318586Flows_udp: 54743Flows_icmp: 3864Flows_other: 1137Packets: 11162669Packets_tcp: 8681920Packets_udp: 2163252Packets_icmp: 34346Packets_other: 283151Bytes: 6315310484Bytes_tcp: 5677222352Bytes_udp: 467682299Bytes_icmp: 3717079Bytes_other: 166688754First: 1587551972Last: 1587552299msec_first: 950msec_last: 914Sequence failures: 0Read and print all records from a range of files, starting at nfcapd.202004221040 and up to but not including the current file still being written to nfcapd.current.1609
nfdump -R nfcapd.202004221040Read all records from a range of files, starting at nfcapd.202209242120 and finishing at nfcapd.202209242150
This works if files are in the same directory. If they are not, also specify -M fordirectories list.
nfdump -R nfcapd.202209242120:nfcapd.202209242150Print sessions where the source or destination IP is 8.8.8.8
nfdump -r nfcapd.202004221040 'host 8.8.8.8'Print sessions where the destination port is 53, the destination IP is 8.8.8.8, and the protocol is TCP
nfdump -r nfcapd.202004221040 'host 8.8.8.8 and dst port 53 and proto tcp 'Show top 10 flows sorted by the bits per second statistics
Note: -o extended sets output to include also bps column. -n 10 limits output to top 10 rows (which is default as well). Finally, -O bps tells nfdump to sort the output by bits per second value in descending (default) order.
nfdump -r nfcapd.202004221050 -n 10 -O bps -o extendedOutput:
Date first seen Duration Proto Src IP Addr:Port Dst IP Addr:Port Flags Tos Packets Bytes pps bps Bpp Flows2020-04-22 12:19:58.824 0.004 TCP 44.30.248.239:443 -> 44.244.6.114:54044 ...AP... 16 141 204984 35250 410.0 M 1453 12020-04-22 12:22:37.845 0.004 TCP 175.68.86.47:80 -> 44.244.6.114:53717 ...AP... 128 133 184649 33250 369.3 M 1388 12020-04-22 12:20:37.844 0.004 TCP 175.68.86.47:80 -> 44.244.6.114:53717 ...AP... 128 133 184649 33250 369.3 M 1388 12020-04-22 12:24:37.845 0.004 TCP 175.68.86.47:80 -> 44.244.6.114:53717 ...AP... 128 132 184609 33000 369.2 M 1398 12020-04-22 12:22:59.517 0.008 TCP 44.244.195.12:443 -> 44.244.6.114:58302 ...AP... 16 212 302672 26500 302.7 M 1427 12020-04-22 12:23:15.541 0.036 TCP 44.244.195.12:443 -> 44.244.6.114:58302 ...AP... 16 915 1.3 M 25416 298.9 M 1469 12020-04-22 12:20:03.728 0.004 TCP 50.62.32.42:80 -> 216.88.40.116:52054 ...AP... 40 99 135345 24750 270.7 M 1367 12020-04-22 12:23:08.773 0.012 TCP 44.244.195.12:443 -> 44.244.6.114:58302 ...AP... 16 255 371935 21250 248.0 M 1458 12020-04-22 12:22:58.377 0.004 TCP 50.62.32.25:80 -> 216.88.40.116:52157 ...AP... 0 77 109616 19250 219.2 M 1423 12020-04-22 12:21:25.568 0.028 TCP 158.255.172.17:443 -> 44.244.6.114:55324 ...AP... 16 483 669748 17250 191.4 M 1386 1IP addresses anonymisedSummary: total flows: 492540, total bytes: 7.4 G, total packets: 14.6 M, avg bps: 184.8 M, avg pps: 45237, avg bpp: 510Time window: 2020-04-22 12:19:37 - 2020-04-22 12:24:59Total flows processed: 492540, Blocks skipped: 0, Bytes read: 39403680Sys: 0.216s flows/second: 2277989.2 Wall: 0.404s flows/second: 1219146.3Show all flows sorted by the bits per second statistics
nfdump -r nfcapd.202004221050 -n 0 -O bps -o extendedAggregate all flows to/from host 8.8.8.8 based on source IP
nfdump -r nfcapd.202004221005 -A srcip ' host 8.8.8.8'Date first seen Duration Src IP Addr Packets Bytes bps Bpp Flows2020-04-22 10:05:01.183 241.032 113.166.180.142 122 6938 230 56 1222020-04-22 10:05:00.915 295.020 18.113.121.204 1493 96860 2626 64 622020-04-22 10:05:03.819 289.848 18.113.43.130 750 63000 1738 84 542020-04-22 10:05:02.887 289.828 113.166.180.139 750 63000 1738 84 492020-04-22 10:05:01.455 295.148 113.166.180.138 812 50458 1367 62 8102020-04-22 10:05:03.507 289.852 113.166.180.137 750 63000 1738 84 562020-04-22 10:04:55.799 300.484 89.12.212.116 417 33261 885 79 4172020-04-22 10:05:00.667 289.868 113.166.180.141 750 63000 1738 84 552020-04-22 10:04:56.047 303.116 8.8.8.8 6730 768784 20290 114 18252020-04-22 10:05:01.127 291.740 113.166.88.58 886 70796 1941 79 172Summary: total flows: 3622, total bytes: 1.3 M, total packets: 13460, avg bps: 33731, avg pps: 44, avg bpp: 95Time window: 2020-04-22 10:04:37 - 2020-04-22 10:09:59Total flows processed: 426270, Blocks skipped: 0, Bytes read: 34102112Sys: 0.036s flows/second: 11784203.7 Wall: 0.036s flows/second: 11560177.9Calculate statistics for port 443 traffic and sort by bps to see bandwidth abusing hosts
We can include as many -s as needed, each statistics table will be printed independently. Statistics will be calculated for the flows located in this specific nfcapd. file, to count statistics over longer periods of time see -R & -M
nfdump -r nfcapd.202004220705 -s srcip/bps -s dstip/bps ' port 443'Output:
Top 10 Src IP Addr ordered by bps:Date first seen Duration Proto Src IP Addr Flows(%) Packets(%) Bytes(%) pps bps bpp2020-04-22 12:19:50.336 309.513 any 219.149.22.196 8957( 2.6) 995109(10.7) 1.3 G(21.6) 3215 34.2 M 13272020-04-22 12:19:50.336 309.361 any 219.149.22.201 8167( 2.4) 837173( 9.0) 1.1 G(18.1) 2706 28.6 M 13212020-04-22 12:19:37.828 322.081 any 44.244.6.114 58555(16.9) 2.1 M(22.3) 978.3 M(16.0) 6471 24.3 M 4692020-04-22 12:21:31.120 0.496 any 128.73.82.164 6( 0.0) 958( 0.0) 1.3 M( 0.0) 1931 21.7 M 14062020-04-22 12:19:49.064 310.609 any 244.34.184.28 6849( 2.0) 411384( 4.4) 369.3 M( 6.0) 1324 9.5 M 8972020-04-22 12:23:01.213 2.244 any 148.161.85.162 3( 0.0) 1322( 0.0) 1.9 M( 0.0) 589 6.9 M 14692020-04-22 12:19:49.860 309.909 any 244.34.184.29 4425( 1.3) 270828( 2.9) 250.0 M( 4.1) 873 6.5 M 9222020-04-22 12:19:52.984 306.313 any 219.149.22.228 12205( 3.5) 245171( 2.6) 244.1 M( 4.0) 800 6.4 M 9952020-04-22 12:21:08.360 6.460 any 92.34.211.23 4( 0.0) 3080( 0.0) 4.5 M( 0.1) 476 5.6 M 14722020-04-22 12:19:37.828 321.421 any 219.149.22.229 16208( 4.7) 252091( 2.7) 220.4 M( 3.6) 784 5.5 M 874Top 10 Dst IP Addr ordered by bps:Date first seen Duration Proto Dst IP Addr Flows(%) Packets(%) Bytes(%) pps bps bpp2020-04-22 12:19:37.828 322.081 any 44.244.6.114 57453(16.6) 3.5 M(37.6) 3.1 G(51.2) 10892 77.8 M 8932020-04-22 12:19:43.316 316.633 any 216.88.40.117 56198(16.2) 1.1 M(11.8) 939.0 M(15.4) 3470 23.7 M 8542020-04-22 12:19:48.900 311.033 any 216.88.40.116 56349(16.3) 1.0 M(11.0) 835.6 M(13.7) 3295 21.5 M 8152020-04-22 12:19:54.760 300.929 any 93.161.105.117 833( 0.2) 83367( 0.9) 108.3 M( 1.8) 277 2.9 M 12992020-04-22 12:19:47.736 310.013 any 210.249.165.16 89( 0.0) 79489( 0.9) 92.7 M( 1.5) 256 2.4 M 11652020-04-22 12:19:59.256 298.577 any 70.35.238.51 180( 0.1) 100020( 1.1) 85.8 M( 1.4) 334 2.3 M 8582020-04-22 12:20:07.024 0.004 any 209.213.75.111 2( 0.0) 28( 0.0) 1120( 0.0) 7000 2.2 M 402020-04-22 12:21:16.636 0.004 any 216.88.58.165 2( 0.0) 22( 0.0) 968( 0.0) 5500 1.9 M 442020-04-22 12:19:59.472 299.353 any 207.176.46.233 94( 0.0) 42353( 0.5) 45.7 M( 0.7) 141 1.2 M 10792020-04-22 12:19:50.356 309.537 any 219.149.22.196 9055( 2.6) 274394( 2.9) 47.0 M( 0.8) 886 1.2 M 171IP addresses anonymisedSummary: total flows: 346143, total bytes: 6.1 G, total packets: 9.3 M, avg bps: 151.8 M, avg pps: 28957, avg bpp: 655Time window: 2020-04-22 12:19:37 - 2020-04-22 12:24:59Total flows processed: 492540, Blocks skipped: 0, Bytes read: 39403680Sys: 0.139s flows/second: 3539124.8 Wall: 0.136s flows/second: 3598781.3Sort presented flows by duration, longest at the bottom
nfdump itself has no provision to sort flows by their duration, but we caneasily pipe the output to any Linux sorting tool.Let’s display top 10 flows by duration:
echo 'Date first seen Duration Proto Src IP Addr:Port Dst IP Addr:Port Packets Bytes Flows ' ; \nfdump -r nfcapd.202209281905 | sort -n -k3,3 | tail -10Output:
Date first seen Duration Proto Src IP Addr:Port Dst IP Addr:Port Packets Bytes Flows2022-09-28 18:10:08.820 3360.080 TCP 172.17.12.130:57095 -> 20.199.120.182:443 4 306 12022-09-28 18:10:08.820 3360.080 TCP 20.199.120.182:443 -> 172.17.12.130:57095 2 428 12022-09-28 18:11:48.620 3360.130 TCP 172.17.12.164:49836 -> 20.199.120.151:443 4 304 12022-09-28 18:11:48.620 3360.130 TCP 20.199.120.151:443 -> 172.17.12.164:49836 2 426 12022-09-28 17:06:18.140 7202.630 ICMP 172.17.80.245:0 -> 172.20.0.2:0.8 120 7200 12022-09-28 09:09:36.580 35232.610 PIM 100.100.100.100:0 -> 172.17.46.254:0 0 0 12022-09-27 20:34:35.550 81030.200 ICMP 172.17.80.245:0 -> 87.128.226.58:0.8 1362 81720 12022-09-27 20:34:35.550 81030.200 ICMP 87.128.226.58:0 -> 172.17.80.245:8.8 1362 81720 12022-09-27 15:45:26.850 98610.750 ICMP 172.17.7.12:0 -> 172.17.24.127:8.8 9860 433840 12022-09-27 15:45:26.850 98610.750 ICMP 172.17.24.127:0 -> 172.17.7.12:0.8 9859 433796 1Anonymize IP addresses in all the flows in the file, overwrite in-place
Use bundled with nfdump tool named nfanon. To use it we have to specify a random ASCII of 32 characters or hexadecimal string of 64 characters. The -K option accepts the random key.
To generate random 32 chars:
dd if=/dev/urandom bs=16 count=1 | hexdump -v -e '/1 "%02X "' | tr -d ' ' ; echo1+0 records in1+0 records out16 bytes copied, 0.000491685 s, 32.5 kB/sE9C11DC6F92488E7A13A1F42EF6A9E87nfanon -K E9C11DC6F92488E7A13A1F42EF6A9E87 -r nfcapd.202004220710Find records in the time range of 12:19:00 - 12:20:00 matching the filter of protocol = UDP and port = 53
Using -t option we can limit the time range of the records to look into. nfdump puts 0 for any missing time part, e.g. 12:19 means 12:19:00.
nfdump -r nfcapd.202004220920 -t 2020/04/22.12:19:00-2020/04/22.12:20:00 'port 53 and proto udp'Output:
Date first seen Duration Proto Src IP Addr:Port Dst IP Addr:Port Packets Bytes Flows2020-04-22 12:19:55.564 0.000 UDP 56.30.111.241:53 -> 216.88.40.117:64842 1 636 12020-04-22 12:19:55.564 0.000 UDP 216.88.40.117:64012 -> 158.174.33.78:53 1 81 12020-04-22 12:19:54.852 0.000 UDP 216.88.40.117:63044 -> 70.158.34.8:53 1 80 12020-04-22 12:19:55.712 0.000 UDP 219.154.149.77:53 -> 216.88.40.116:49880 1 89 12020-04-22 12:19:55.716 0.000 UDP 216.88.40.117:65172 -> 246.220.77.233:53 1 82 12020-04-22 12:19:55.152 0.000 UDP 216.88.40.117:63463 -> 177.234.225.103:53 1 79 12020-04-22 12:19:55.364 0.000 UDP 216.88.40.117:63493 -> 51.11.3.16:53 1 73 1IP addresses anonymisedSummary: total flows: 7, total bytes: 1120, total packets: 7, avg bps: 10370, avg pps: 8, avg bpp: 160Time window: 2020-04-22 12:19:37 - 2020-04-22 12:24:59Total flows processed: 492540, Blocks skipped: 0, Bytes read: 39403680Sys: 0.050s flows/second: 9713068.7 Wall: 0.048s flows/second: 10233959.4Follow me on https://www.linkedin.com/in/yurislobodyanyuk/ not to miss what I publish on Linkedin, Github, blog, and more.
