1.
패킷을 고성능으로 캡처하는 부분은 지난 번에 다루었습니다.
단순한 캡처가 목적이 아니기 때문에 한발더 나아가야 합니다. 캡처를 한 후 저장 및 분석을 하여야 원하는 목적인 레이턴시 분석을 할 수 있습니다. 이를 위하여 Corvil 제품을 다시금 살펴보았습니다. 브로커환경에서의 레이턴시 측정과 관련한 구성입니다.
위 그림중 별모양은 TAP장비입니다. Installing the CNE with a Passive TapTAP장비와 관련하여 Corvil은 Passive Tap와 Aggregation Tap 모두를 지원한다고 합니다. Passive Tap이 무엇을 말하는지 궁금해서 확인을 해보니까 아래와 같이 정의하고 있습니다.
A Passive Network TAP has no physical separation between the network ports. When the power to the device is lost, the network link will remain operational without delay. It requires no extra powering. An Active Network TAP does have a physical separation between the network ports, because of the relays and other electronic components inside the device. Extra powering is required by the TAP to fully operate.
일반적으로 레이턴시 측정을 위한 제품들은 Port mirroring,GPAN 혹은 RITE(Raw IP Traffice Export)와 같이 네트워크장비를 통하여 패킷을 수집한 후 이를 다양한 분석하는 기능을 제공합니다. 이런 구성을 취하는 이유는 운영중인 환경에 대한 영향을 최소화하면서 측정을 하기 위함입니다. 문제는 비용입니다.
그래서 고민했던 것이 소프트웨어적인 방식이었습니다. 레이턴시 측정, 패킷 갭처 – pf_ring가 출발점입니다. pf_ring으로 이런저런 시험을 하면서 어떤 구성으로 측정을 할지를 고민했습니다. 가장 좋은 방식은 앞서 Corvil처럼 Port Mirroring이나 TAP등을 이용하여 패킷을 한 곳으로 보내서 측정서버가 패킷을 캡처,저장,가공 및 분석을 하는 것입니다. 이 방식을 포기하면 선택지는 매매서버나 FEP서버에서 패킷을 생성하여 자체로 저장하거나, 측정서버로 보내서 저장하는 방식입니다. pf_ring을 기반으로 해서 분석프로그램을 개발하기 전에 어느 수준에서 가능한지 확인을 하였습니다.
2.
먼저,pf_ring을 만든 ntop은 소프트웨어적으로 위와 같은 구성이 가능한 제품을 공급합니다. n2disk, ntap입니다. n2disk는 패킷을 수집해서 저장까지 하는 역할을 합니다. ntap은 소프트웨어적으로 tap기능을 구현한 제품입니다. ntap_remote와 ntap_collector로 이루어져있습니다. ntap_remote를 제외하면 라이센스가 필요한 제품들입니다. 우선 ubuntu 22.04에서 pf_ring와 ntap을 설치하였습니다. ntap을 어떤 경우에 사용하는지를 설명한 부분입니다.
When shall I use nTap ?
nTap has been designed to collect on a central location traffic coming from remote sites when a port mirror or similar techniques cannot be used. In addition nTap delivers full packets to destination meaning that it can be used to implement cybersecurity solutions or trubleshoot using Wireshark or similar applications. This said, you should not care much about performance as you can limit the number of packets you can nTap-deliver using a filtering expression on the ntap_remote side. If you have instead a lot of traffic (1 Gbit+) to analyze you should consider deploying a network probe such as nProbe instead of nTap and deliver flows rather than packets that are much smaller in size.
ntop은 apt 명령어로 손쉽게 설치하였습니다.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 |
smallake@ubuntu:/etc/ntap$ sudo apt-get install pfring Reading package lists... Done Building dependency tree... Done Reading state information... Done The following package was automatically installed and is no longer required: systemd-hwe-hwdb Use 'sudo apt autoremove' to remove it. The following additional packages will be installed: ndpi Recommended packages: pfring-dkms The following NEW packages will be installed: ndpi pfring 0 upgraded, 2 newly installed, 0 to remove and 161 not upgraded. Need to get 5,913 kB of archives. After this operation, 45.0 MB of additional disk space will be used. Do you want to continue? [Y/n] y Get:1 https://packages.ntop.org/apt-stable/22.04 x64/ ndpi 4.6.0-4112 [1,103 kB] Get:2 https://packages.ntop.org/apt-stable/22.04 x64/ pfring 8.4.0-8112 [4,810 kB] Fetched 5,913 kB in 21s (279 kB/s) Selecting previously unselected package ndpi. (Reading database ... 195321 files and directories currently installed.) Preparing to unpack .../ndpi_4.6.0-4112_amd64.deb ... Unpacking ndpi (4.6.0-4112) ... Selecting previously unselected package pfring. Preparing to unpack .../pfring_8.4.0-8112_amd64.deb ... Unpacking pfring (8.4.0-8112) ... Setting up ndpi (4.6.0-4112) ... Rebuilding ld cache... Setting up pfring (8.4.0-8112) ... Processing triggers for libc-bin (2.35-0ubuntu3.1) ... smallake@ubuntu:/etc/ntap$ pf_ringcfg --list-interfaces This script must be run as root smallake@ubuntu:/etc/ntap$ sudo pf_ringcfg --list-interfaces Name: eno1 Driver: e1000e RSS: Unknown [Supported by ZC] Name: enp2s0f4 Driver: cxgb4 RSS: Unknown [Linux Driver] Name: enp2s0f4d1 Driver: cxgb4 RSS: Unknown [Linux Driver] smallake@ubuntu:/etc/ntap$ sudo pf_ringcfg --configure-driver e1000e [>] Installing PF_RING.ko Hit:1 http://kr.archive.ubuntu.com/ubuntu jammy InRelease Hit:2 http://kr.archive.ubuntu.com/ubuntu jammy-updates InRelease Hit:3 https://packages.ntop.org/apt-stable/22.04 x64/ InRelease Hit:4 http://kr.archive.ubuntu.com/ubuntu jammy-backports InRelease Hit:5 https://packages.ntop.org/apt-stable/22.04 all/ InRelease Hit:6 http://security.ubuntu.com/ubuntu jammy-security InRelease Reading package lists... Done Reading package lists... Done Building dependency tree... Done Reading state information... Done The following package was automatically installed and is no longer required: systemd-hwe-hwdb Use 'sudo apt autoremove' to remove it. The following additional packages will be installed: cpp-12 dctrl-tools dkms gcc-12 libasan8 libgcc-12-dev libtsan2 linux-headers-generic Suggested packages: gcc-12-locales debtags menu gcc-12-multilib gcc-12-doc The following NEW packages will be installed: cpp-12 dctrl-tools dkms gcc-12 libasan8 libgcc-12-dev libtsan2 linux-headers-generic pfring-dkms 0 upgraded, 9 newly installed, 0 to remove and 161 not upgraded. Need to get 145 MB of archives. After this operation, 466 MB of additional disk space will be used. Do you want to continue? [Y/n] y Get:1 http://kr.archive.ubuntu.com/ubuntu jammy-updates/main amd64 cpp-12 amd64 12.1.0-2ubuntu1~22.04 [63.8 MB] Get:2 https://packages.ntop.org/apt-stable/22.04 all/ pfring-dkms 8.4.0.8112 [70.0 kB] Get:3 http://kr.archive.ubuntu.com/ubuntu jammy-updates/main amd64 libasan8 amd64 12.1.0-2ubuntu1~22.04 [2,455 kB] Get:4 http://kr.archive.ubuntu.com/ubuntu jammy-updates/main amd64 libtsan2 amd64 12.1.0-2ubuntu1~22.04 [2,477 kB] Get:5 http://kr.archive.ubuntu.com/ubuntu jammy-updates/main amd64 libgcc-12-dev amd64 12.1.0-2ubuntu1~22.04 [2,618 kB] Get:6 http://kr.archive.ubuntu.com/ubuntu jammy-updates/main amd64 gcc-12 amd64 12.1.0-2ubuntu1~22.04 [73.1 MB] Get:7 http://kr.archive.ubuntu.com/ubuntu jammy/main amd64 dctrl-tools amd64 2.24-3build2 [66.9 kB] Get:8 http://kr.archive.ubuntu.com/ubuntu jammy-updates/main amd64 dkms all 2.8.7-2ubuntu2.1 [70.0 kB] Get:9 http://kr.archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.58.56 [2,282 B] Fetched 145 MB in 22s (6,664 kB/s) Selecting previously unselected package cpp-12. (Reading database ... 195375 files and directories currently installed.) Preparing to unpack .../0-cpp-12_12.1.0-2ubuntu1~22.04_amd64.deb ... Unpacking cpp-12 (12.1.0-2ubuntu1~22.04) ... Selecting previously unselected package libasan8:amd64. Preparing to unpack .../1-libasan8_12.1.0-2ubuntu1~22.04_amd64.deb ... Unpacking libasan8:amd64 (12.1.0-2ubuntu1~22.04) ... Selecting previously unselected package libtsan2:amd64. Preparing to unpack .../2-libtsan2_12.1.0-2ubuntu1~22.04_amd64.deb ... Unpacking libtsan2:amd64 (12.1.0-2ubuntu1~22.04) ... Selecting previously unselected package libgcc-12-dev:amd64. Preparing to unpack .../3-libgcc-12-dev_12.1.0-2ubuntu1~22.04_amd64.deb ... Unpacking libgcc-12-dev:amd64 (12.1.0-2ubuntu1~22.04) ... Selecting previously unselected package gcc-12. Preparing to unpack .../4-gcc-12_12.1.0-2ubuntu1~22.04_amd64.deb ... Unpacking gcc-12 (12.1.0-2ubuntu1~22.04) ... Selecting previously unselected package dctrl-tools. Preparing to unpack .../5-dctrl-tools_2.24-3build2_amd64.deb ... Unpacking dctrl-tools (2.24-3build2) ... Selecting previously unselected package dkms. Preparing to unpack .../6-dkms_2.8.7-2ubuntu2.1_all.deb ... Unpacking dkms (2.8.7-2ubuntu2.1) ... Selecting previously unselected package linux-headers-generic. Preparing to unpack .../7-linux-headers-generic_5.15.0.58.56_amd64.deb ... Unpacking linux-headers-generic (5.15.0.58.56) ... Selecting previously unselected package pfring-dkms. Preparing to unpack .../8-pfring-dkms_8.4.0.8112_all.deb ... Unpacking pfring-dkms (8.4.0.8112) ... Setting up cpp-12 (12.1.0-2ubuntu1~22.04) ... Setting up linux-headers-generic (5.15.0.58.56) ... Setting up libasan8:amd64 (12.1.0-2ubuntu1~22.04) ... Setting up libtsan2:amd64 (12.1.0-2ubuntu1~22.04) ... Setting up dctrl-tools (2.24-3build2) ... Setting up libgcc-12-dev:amd64 (12.1.0-2ubuntu1~22.04) ... Setting up gcc-12 (12.1.0-2ubuntu1~22.04) ... Setting up dkms (2.8.7-2ubuntu2.1) ... Setting up pfring-dkms (8.4.0.8112) ... Loading new pfring-8.4.0.8112 DKMS files... Building for 5.15.0-58-generic Building for architecture x86_64 Building initial module for 5.15.0-58-generic Can't load /var/lib/shim-signed/mok/.rnd into RNG 40D7601D8E7F0000:error:12000079:random number generator:RAND_load_file:Cannot open file:../crypto/rand/randfile.c:106:Filename=/var/lib/shim-signed/mok/.rnd .+...+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*...+...+.+......+...+...........+....+...+..+......+......+.+..+.+......+...+...+........+.+.....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.........+.........+...+.+...........+......+....+...+..+.+..+..........+..+.............+...+...+..+....+...+........+....+...+............+.....+.+..+................+.........+..+.+..+......+.+.....+..................+................+............+..+.......+......+..+.......+......+........+.+...+..+...+............+............................+.....+..........+......+..+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ..............+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.+.......................+....+.....+......................+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.+...........+...+.+..+...+.........+.......+........+....................................+..........+.....+....+...+...+..+.............+.....................+.....+..........+...........+............+...+...+....+........+...+.........+...+....+.....+.............+.....+.+...........+...+.+...........+.+..............+...+......+....+...+...+..............+...+............+..........+......+...........+.........+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ----- EFI variables are not supported on this system /sys/firmware/efi/efivars not found, aborting. Done. pf_ring.ko: Running module version sanity check. - Original module - Installation - Installing to /lib/modules/5.15.0-58-generic/updates/dkms/ depmod...... Created symlink /etc/systemd/system/multi-user.target.wants/pf_ring.service → /etc/systemd/system/pf_ring.service. cluster is a disabled or a static unit not running, not starting it. Processing triggers for man-db (2.10.2-1) ... Processing triggers for libc-bin (2.35-0ubuntu3.1) ... [>] Installing e1000e Zero-Copy driver Continue (y/n)?y Hit:1 http://kr.archive.ubuntu.com/ubuntu jammy InRelease Hit:2 https://packages.ntop.org/apt-stable/22.04 x64/ InRelease Hit:3 https://packages.ntop.org/apt-stable/22.04 all/ InRelease Hit:4 http://kr.archive.ubuntu.com/ubuntu jammy-updates InRelease Hit:5 http://kr.archive.ubuntu.com/ubuntu jammy-backports InRelease Hit:6 http://security.ubuntu.com/ubuntu jammy-security InRelease Reading package lists... Done Reading package lists... Done Building dependency tree... Done Reading state information... Done pfring-dkms is already the newest version (8.4.0.8112). The following package was automatically installed and is no longer required: systemd-hwe-hwdb Use 'sudo apt autoremove' to remove it. The following NEW packages will be installed: e1000e-zc-dkms 0 upgraded, 1 newly installed, 0 to remove and 161 not upgraded. Need to get 249 kB of archives. After this operation, 1,369 kB of additional disk space will be used. Do you want to continue? [Y/n] y Get:1 https://packages.ntop.org/apt-stable/22.04 all/ e1000e-zc-dkms 3.8.7.8112 [249 kB] Fetched 249 kB in 5s (48.6 kB/s) Selecting previously unselected package e1000e-zc-dkms. (Reading database ... 195724 files and directories currently installed.) Preparing to unpack .../e1000e-zc-dkms_3.8.7.8112_all.deb ... Unpacking e1000e-zc-dkms (3.8.7.8112) ... Setting up e1000e-zc-dkms (3.8.7.8112) ... Loading new e1000e-zc-3.8.7.8112 DKMS files... Building for 5.15.0-58-generic Building for architecture x86_64 Building initial module for 5.15.0-58-generic EFI variables are not supported on this system /sys/firmware/efi/efivars not found, aborting. Done. e1000e_zc.ko: Running module version sanity check. - Original module - No original module exists within this kernel - Installation - Installing to /lib/modules/5.15.0-58-generic/updates/dkms/ depmod... [>] Configuring PF_RING [>] Configuring hugepages [>] Detecting interfaces using e1000e eno1 [>] Configuring e1000e driver with 1 RSS queues [>] Restarting PF_RING [>] Configuration completed smallake@ubuntu:/etc/ntap$ |
이제 ntap_remote와 ntap_collector를 이용한 시험을 합니다. 시험환경을 위하여 Rocky linux 8에 ntap을 별도로 설치합니다.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 |
[smallake@xeon ~]$ sudo mv ./ntop.repo /etc/yum.repos.d/ [smallake@xeon ~]$ sudo dnf config-manager --set-enabled powertools [smallake@xeon ~]$ sudo dnf install epel-release Rocky Linux 8 - AppStream 6.9 kB/s | 4.8 kB 00:00 Rocky Linux 8 - AppStream 5.7 MB/s | 9.4 MB 00:01 Rocky Linux 8 - BaseOS 6.6 kB/s | 4.3 kB 00:00 Rocky Linux 8 - BaseOS 3.6 MB/s | 3.8 MB 00:01 Rocky Linux 8 - Extras 6.3 kB/s | 3.1 kB 00:00 Rocky Linux 8 - Extras 22 kB/s | 12 kB 00:00 Rocky Linux 8 - PowerTools 9.4 kB/s | 4.8 kB 00:00 Rocky Linux 8 - PowerTools 3.0 MB/s | 2.6 MB 00:00 Extra Packages for Enterprise Linux 8 - x86_64 147 kB/s | 5.8 kB 00:00 Extra Packages for Enterprise Linux 8 - x86_64 38 kB/s | 13 MB 05:55 ntop packages 13 kB/s | 22 kB 00:01 ntop packages 3.8 kB/s | 5.8 kB 00:01 RPM Fusion for EL 8 - Free - Updates 180 B/s | 3.7 kB 00:20 RPM Fusion for EL 8 - Free - Updates 54 kB/s | 310 kB 00:05 Package epel-release-8-18.el8.noarch is already installed. Dependencies resolved. Nothing to do. Complete! [smallake@xeon ~]$ sudo rpm -ivh http://rpms.remirepo.net/enterprise/remi-release-8.rpm [sudo] password for smallake: Retrieving http://rpms.remirepo.net/enterprise/remi-release-8.rpm warning: /var/tmp/rpm-tmp.VtvPxr: Header V4 RSA/SHA256 Signature, key ID 5f11735a: NOKEY Verifying... ################################# [100%] Preparing... ################################# [100%] Updating / installing... 1:remi-release-8.7-2.el8.remi ################################# [100%] [smallake@xeon ~]$ sudo yum install dnf-plugins-core Remi's Modular repository for Enterprise Linux 8 - x86_64 262 B/s | 833 B 00:03 Remi's Modular repository for Enterprise Linux 8 - x86_64 3.0 MB/s | 3.1 kB 00:00 Importing GPG key 0x5F11735A: Userid : "Remi's RPM repository <remi@remirepo.net>" Fingerprint: 6B38 FEA7 231F 87F5 2B9C A9D8 5550 9759 5F11 735A From : /etc/pki/rpm-gpg/RPM-GPG-KEY-remi.el8 Is this ok [y/N]: y Remi's Modular repository for Enterprise Linux 8 - x86_64 261 kB/s | 1.2 MB 00:04 Safe Remi's RPM repository for Enterprise Linux 8 - x86_64 484 B/s | 833 B 00:01 Safe Remi's RPM repository for Enterprise Linux 8 - x86_64 3.0 MB/s | 3.1 kB 00:00 Importing GPG key 0x5F11735A: Userid : "Remi's RPM repository <remi@remirepo.net>" Fingerprint: 6B38 FEA7 231F 87F5 2B9C A9D8 5550 9759 5F11 735A From : /etc/pki/rpm-gpg/RPM-GPG-KEY-remi.el8 Is this ok [y/N]: y Safe Remi's RPM repository for Enterprise Linux 8 - x86_64 440 kB/s | 2.3 MB 00:05 Last metadata expiration check: 0:00:02 ago on Thu 02 Feb 2023 01:34:30 PM KST. Package dnf-plugins-core-4.0.21-14.1.el8.noarch is already installed. Dependencies resolved. Nothing to do. Complete! [smallake@xeon ~]$ sudo dnf config-manager --set-enabled powertools [smallake@xeon ~]$ sudo dnf config-manager --set-enabled remi [smallake@xeon ~]$ sudo yum clean all 90 files removed [smallake@xeon ~]$ sudo yum update Rocky Linux 8 - AppStream 5.6 MB/s | 9.4 MB 00:01 Rocky Linux 8 - BaseOS 3.4 MB/s | 3.8 MB 00:01 Rocky Linux 8 - Extras 21 kB/s | 12 kB 00:00 Rocky Linux 8 - PowerTools 3.2 MB/s | 2.6 MB 00:00 Extra Packages for Enterprise Linux 8 - x86_64 42 kB/s | 13 MB 05:25 ntop packages 6.9 kB/s | 22 kB 00:03 ntop packages 3.9 kB/s | 5.8 kB 00:01 Remi's RPM repository for Enterprise Linux 8 - x86_64 129 B/s | 833 B 00:06 Remi's RPM repository for Enterprise Linux 8 - x86_64 3.0 MB/s | 3.1 kB 00:00 Importing GPG key 0x5F11735A: Userid : "Remi's RPM repository <remi@remirepo.net>" Fingerprint: 6B38 FEA7 231F 87F5 2B9C A9D8 5550 9759 5F11 735A From : /etc/pki/rpm-gpg/RPM-GPG-KEY-remi.el8 Is this ok [y/N]: y Remi's RPM repository for Enterprise Linux 8 - x86_64 573 kB/s | 4.5 MB 00:08 Remi's Modular repository for Enterprise Linux 8 - x86_64 103 kB/s | 1.2 MB 00:12 Safe Remi's RPM repository for Enterprise Linux 8 - x86_64 216 kB/s | 2.3 MB 00:10 RPM Fusion for EL 8 - Free - Updates 62 kB/s | 310 kB 00:04 Dependencies resolved. ============================================================================================================================================================= Package Architecture Version Repository Size ============================================================================================================================================================= Upgrading: firefox x86_64 102.7.0-1.el8_7 appstream 108 M libXpm x86_64 3.5.12-9.el8_7 appstream 57 k libxml2 x86_64 2.9.7-15.el8_7.1 baseos 695 k python3-libxml2 x86_64 2.9.7-15.el8_7.1 baseos 236 k sudo x86_64 1.8.29-8.el8_7.1 baseos 924 k Transaction Summary ============================================================================================================================================================= Upgrade 5 Packages Total download size: 110 M Is this ok [y/N]: y Downloading Packages: (1/5): libXpm-3.5.12-9.el8_7.x86_64.rpm 298 kB/s | 57 kB 00:00 (2/5): python3-libxml2-2.9.7-15.el8_7.1.x86_64.rpm 2.0 MB/s | 236 kB 00:00 (3/5): libxml2-2.9.7-15.el8_7.1.x86_64.rpm 1.8 MB/s | 695 kB 00:00 (4/5): sudo-1.8.29-8.el8_7.1.x86_64.rpm 4.4 MB/s | 924 kB 00:00 (5/5): firefox-102.7.0-1.el8_7.x86_64.rpm 9.4 MB/s | 108 MB 00:11 ------------------------------------------------------------------------------------------------------------------------------------------------------------- Total 8.8 MB/s | 110 MB 00:12 Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Preparing : 1/1 Upgrading : libxml2-2.9.7-15.el8_7.1.x86_64 1/10 Upgrading : python3-libxml2-2.9.7-15.el8_7.1.x86_64 2/10 Upgrading : sudo-1.8.29-8.el8_7.1.x86_64 3/10 Running scriptlet: sudo-1.8.29-8.el8_7.1.x86_64 3/10 Upgrading : libXpm-3.5.12-9.el8_7.x86_64 4/10 Upgrading : firefox-102.7.0-1.el8_7.x86_64 5/10 Running scriptlet: firefox-102.7.0-1.el8_7.x86_64 5/10 Cleanup : python3-libxml2-2.9.7-15.el8.x86_64 6/10 Cleanup : libxml2-2.9.7-15.el8.x86_64 7/10 Cleanup : sudo-1.8.29-8.el8.x86_64 8/10 Cleanup : libXpm-3.5.12-8.el8.x86_64 9/10 Running scriptlet: firefox-102.6.0-1.el8_7.x86_64 10/10 Cleanup : firefox-102.6.0-1.el8_7.x86_64 10/10 Running scriptlet: firefox-102.6.0-1.el8_7.x86_64 10/10 Running scriptlet: firefox-102.7.0-1.el8_7.x86_64 10/10 Running scriptlet: firefox-102.6.0-1.el8_7.x86_64 10/10 Verifying : firefox-102.7.0-1.el8_7.x86_64 1/10 Verifying : firefox-102.6.0-1.el8_7.x86_64 2/10 Verifying : libXpm-3.5.12-9.el8_7.x86_64 3/10 Verifying : libXpm-3.5.12-8.el8.x86_64 4/10 Verifying : libxml2-2.9.7-15.el8_7.1.x86_64 5/10 Verifying : libxml2-2.9.7-15.el8.x86_64 6/10 Verifying : python3-libxml2-2.9.7-15.el8_7.1.x86_64 7/10 Verifying : python3-libxml2-2.9.7-15.el8.x86_64 8/10 Verifying : sudo-1.8.29-8.el8_7.1.x86_64 9/10 Verifying : sudo-1.8.29-8.el8.x86_64 10/10 Upgraded: firefox-102.7.0-1.el8_7.x86_64 libXpm-3.5.12-9.el8_7.x86_64 libxml2-2.9.7-15.el8_7.1.x86_64 python3-libxml2-2.9.7-15.el8_7.1.x86_64 sudo-1.8.29-8.el8_7.1.x86_64 Complete! [smallake@xeon ~]$ sudo yum install pfring-dkms ntap [sudo] password for smallake: Last metadata expiration check: 0:01:50 ago on Thu 02 Feb 2023 01:42:11 PM KST. Dependencies resolved. ============================================================================================================================================================= Package Architecture Version Repository Size ============================================================================================================================================================= Installing: ntap x86_64 1.1.230202-116 ntop 468 k pfring-dkms noarch 8.5.0.8113-8113 ntop-noarch 79 k Installing dependencies: dkms noarch 3.0.10-1.el8 epel 89 k elfutils-libelf-devel x86_64 0.187-4.el8 baseos 60 k kernel-devel x86_64 4.18.0-425.10.1.el8_7 baseos 22 M ndpi x86_64 4.7.0-4113 ntop 1.2 M numactl x86_64 2.0.12-13.el8 baseos 75 k pfring x86_64 8.5.0-8113 ntop 1.8 M zlib-devel x86_64 1.2.11-21.el8_7 baseos 57 k Transaction Summary ============================================================================================================================================================= Install 9 Packages Total download size: 26 M Installed size: 75 M Is this ok [y/N]: y Downloading Packages: (1/9): numactl-2.0.12-13.el8.x86_64.rpm 1.7 MB/s | 75 kB 00:00 (2/9): elfutils-libelf-devel-0.187-4.el8.x86_64.rpm 1.1 MB/s | 60 kB 00:00 (3/9): zlib-devel-1.2.11-21.el8_7.x86_64.rpm 1.9 MB/s | 57 kB 00:00 (4/9): kernel-devel-4.18.0-425.10.1.el8_7.x86_64.rpm 9.6 MB/s | 22 MB 00:02 (5/9): dkms-3.0.10-1.el8.noarch.rpm 37 kB/s | 89 kB 00:02 (6/9): ntap-1.1.230202-116.x86_64.rpm 232 kB/s | 468 kB 00:02 (7/9): ndpi-4.7.0-4113.x86_64.rpm 291 kB/s | 1.2 MB 00:04 (8/9): pfring-dkms-8.5.0.8113-8113.noarch.rpm 39 kB/s | 79 kB 00:01 (9/9): pfring-8.5.0-8113.x86_64.rpm 463 kB/s | 1.8 MB 00:03 ------------------------------------------------------------------------------------------------------------------------------------------------------------- Total 3.0 MB/s | 26 MB 00:08 ntop packages 4.1 kB/s | 3.0 kB 00:00 Importing GPG key 0x924F7599: Userid : "Luca Deri <deri@ntop.org>" Fingerprint: 8E07 231F 0575 7F56 FECE 3977 3D84 C955 924F 7599 From : https://packages.ntop.org/centos/RPM-GPG-KEY-deri Is this ok [y/N]: y Key imported successfully Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Preparing : 1/1 Installing : zlib-devel-1.2.11-21.el8_7.x86_64 1/9 Installing : elfutils-libelf-devel-0.187-4.el8.x86_64 2/9 Installing : kernel-devel-4.18.0-425.10.1.el8_7.x86_64 3/9 Running scriptlet: kernel-devel-4.18.0-425.10.1.el8_7.x86_64 3/9 Installing : dkms-3.0.10-1.el8.noarch 4/9 Running scriptlet: dkms-3.0.10-1.el8.noarch 4/9 Installing : numactl-2.0.12-13.el8.x86_64 5/9 Running scriptlet: numactl-2.0.12-13.el8.x86_64 5/9 Running scriptlet: ndpi-4.7.0-4113.x86_64 6/9 Installing : ndpi-4.7.0-4113.x86_64 6/9 Running scriptlet: ndpi-4.7.0-4113.x86_64 6/9 Running scriptlet: pfring-8.5.0-8113.x86_64 7/9 Installing : pfring-8.5.0-8113.x86_64 7/9 Running scriptlet: pfring-8.5.0-8113.x86_64 7/9 Running scriptlet: ntap-1.1.230202-116.x86_64 8/9 Creating ntap group Creating ntap user Installing : ntap-1.1.230202-116.x86_64 8/9 Running scriptlet: ntap-1.1.230202-116.x86_64 8/9 Running scriptlet: pfring-dkms-8.5.0.8113-8113.noarch 9/9 Installing : pfring-dkms-8.5.0.8113-8113.noarch 9/9 Running scriptlet: pfring-dkms-8.5.0.8113-8113.noarch 9/9 Loading new pfring-8.5.0.8113 DKMS files... Building for 4.18.0-425.10.1.el8_7.x86_64 Building initial module for 4.18.0-425.10.1.el8_7.x86_64 Done. pf_ring.ko.xz: Running module version sanity check. - Original module - No original module exists within this kernel - Installation - Installing to /lib/modules/4.18.0-425.10.1.el8_7.x86_64/extra/ Adding any weak-modules depmod.... Verifying : elfutils-libelf-devel-0.187-4.el8.x86_64 1/9 Verifying : kernel-devel-4.18.0-425.10.1.el8_7.x86_64 2/9 Verifying : numactl-2.0.12-13.el8.x86_64 3/9 Verifying : zlib-devel-1.2.11-21.el8_7.x86_64 4/9 Verifying : dkms-3.0.10-1.el8.noarch 5/9 Verifying : ndpi-4.7.0-4113.x86_64 6/9 Verifying : ntap-1.1.230202-116.x86_64 7/9 Verifying : pfring-8.5.0-8113.x86_64 8/9 Verifying : pfring-dkms-8.5.0.8113-8113.noarch 9/9 Installed: dkms-3.0.10-1.el8.noarch elfutils-libelf-devel-0.187-4.el8.x86_64 kernel-devel-4.18.0-425.10.1.el8_7.x86_64 ndpi-4.7.0-4113.x86_64 ntap-1.1.230202-116.x86_64 numactl-2.0.12-13.el8.x86_64 pfring-8.5.0-8113.x86_64 pfring-dkms-8.5.0.8113-8113.noarch zlib-devel-1.2.11-21.el8_7.x86_64 Complete! |
이상과 같이 설치한 후 확인을 하였습니다.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
[smallake@xeon ~]$ ntap_collector -h 02/Feb/2023 13:47:33 [ntap_collector.c:53] 02/Feb/2023 13:47:33 [ntap_collector.c:54] nTap collector 1.1.230202 - (C) ntop 2022-23 02/Feb/2023 13:47:33 [ntap_collector.c:55] 02/Feb/2023 13:47:33 [ntap_collector.c:56] Usage: 02/Feb/2023 13:47:33 [ntap_collector.c:57] ntap_collector <config filename> 02/Feb/2023 13:47:33 [ntap_collector.c:58] or 02/Feb/2023 13:47:33 [ntap_collector.c:59] ntap_collector -p <port> -k <key>|-K [-i <interface>] [-o <ip>] [-h] [-V] 02/Feb/2023 13:47:33 [ntap_collector.c:64] 02/Feb/2023 13:47:33 [ntap_collector.c:65] Options: 02/Feb/2023 13:47:33 [ntap_collector.c:67] [--interface|-i] <interface> | Virtual interface where received packets are injected 02/Feb/2023 13:47:33 [ntap_collector.c:69] [--egress|-e] <interface> | Host interface on which received packets are injected 02/Feb/2023 13:47:33 [ntap_collector.c:70] [--port|-p] <port> | Local (tap) collector port 02/Feb/2023 13:47:33 [ntap_collector.c:71] [--vxlan-receiver|-o] <ip> | VXLAN traffic receiver (e.g. OpenSwitch) 02/Feb/2023 13:47:33 [ntap_collector.c:72] [--key|-k] <key> | Encryption key 02/Feb/2023 13:47:33 [ntap_collector.c:73] [--disable-encryption|-K] | Disable data encryption (faster but unsafe) 02/Feb/2023 13:47:33 [ntap_collector.c:74] [--help|-h] | Print the help 02/Feb/2023 13:47:33 [ntap_collector.c:75] [--version|-V] | Print license and version 02/Feb/2023 13:47:33 [ntap_collector.c:76] 02/Feb/2023 13:47:33 [ntap_collector.c:77] Example: 02/Feb/2023 13:47:33 [ntap_collector.c:78] - ntap_collector -p 1234 -k hello -i ntap0 02/Feb/2023 13:47:33 [ntap_collector.c:83] 02/Feb/2023 13:47:33 [ntap_collector.c:41] Shutting down... |
이제 ubuntu서버에서 ntap_remote를 실행하고 rocky linux 서버에서 ntop_collector를 실행합니다. 우선 ntap_remote의 실행화면입니다…
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
smallake@ubuntu:/etc/ntap$ sudo ntap_remote -i eno1 -c 121.121.121.121:5678 -k hello -f eno1 -v 02/Feb/2023 15:53:10 [util.c:116] Default socket send buffer size is 212992 02/Feb/2023 15:53:10 [util.c:132] Send socket buffer size set 425984 02/Feb/2023 15:53:10 [util.c:132] Send socket buffer size set 638976 02/Feb/2023 15:53:10 [util.c:132] Send socket buffer size set 851968 02/Feb/2023 15:53:10 [util.c:132] Send socket buffer size set 1064960 02/Feb/2023 15:53:10 [util.c:132] Send socket buffer size set 1277952 02/Feb/2023 15:53:10 [util.c:132] Send socket buffer size set 1490944 02/Feb/2023 15:53:10 [util.c:132] Send socket buffer size set 1703936 02/Feb/2023 15:53:10 [util.c:132] Send socket buffer size set 1916928 02/Feb/2023 15:53:10 [ntap_remote.c:714] Welcome to nTap remote/virtual tap v.1.0.230201 02/Feb/2023 15:53:10 [ntap_remote.c:524] Capturing traffic from eno1 02/Feb/2023 15:53:10 [ntap_remote.c:542] ERROR: Invalid BPF filter syntax: 'can't parse filter expression: syntax error' 02/Feb/2023 15:53:10 [util.c:201] Dropping privileges... 02/Feb/2023 15:53:10 [util.c:219] User changed to ntap 02/Feb/2023 15:53:10 [ntap_remote.c:567] Running... .... 02/Feb/2023 15:53:11 [ntap_remote.c:702] Packet stats [4 packets sent][0 capture drops] |
그러면 수신쪽 실행화면입니다.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
[smallake@xeon ~]$ sudo ntap_collector -p 5678 -k hello -i ntap0 -v [sudo] password for smallake: 03/Feb/2023 09:29:11 [ntap_collector.c:708] Welcome to collector v.1.1.230202 03/Feb/2023 09:29:11 [ntap_collector.c:711] WARNING: Unable to validate license [Missing license file] 03/Feb/2023 09:29:11 [util.c:116] Default socket receive buffer size is 212992 03/Feb/2023 09:29:11 [util.c:132] Receive socket buffer size set 425984 03/Feb/2023 09:29:11 [util.c:132] Receive socket buffer size set 638976 03/Feb/2023 09:29:11 [util.c:132] Receive socket buffer size set 851968 03/Feb/2023 09:29:11 [util.c:132] Receive socket buffer size set 1064960 03/Feb/2023 09:29:11 [util.c:132] Receive socket buffer size set 1277952 03/Feb/2023 09:29:11 [util.c:132] Receive socket buffer size set 1490944 03/Feb/2023 09:29:11 [util.c:132] Receive socket buffer size set 1703936 03/Feb/2023 09:29:11 [util.c:132] Receive socket buffer size set 1916928 03/Feb/2023 09:29:11 [util.c:201] Dropping privileges... 03/Feb/2023 09:29:11 [util.c:219] User changed to ntap 03/Feb/2023 09:29:11 [ntap_collector.c:556] Ready... 03/Feb/2023 09:29:11 [ntap_collector.c:559] WARNING: *** Running in demo mode *** |
입력값이 없습니다. 무언가 이상합니다. 그래서 문서를 다시 찾아보았습니다.
Collector Injects Received Traffic to Local Interface
In this case you have a local collector instance that receives traffic from one or more remote tap instances and injects traffic to an existing local interface specified with -e. In essence:
With -i you specify a virtual interface created by ntap (and destroject when ntap_collector is terminated) on which collected packets are injected
With -e you specify an existing network interface on which collected packets are injectedExample: – [remote host] ntap_remote -i eth0 -c 1.2.3.4:5678 -k hello – [local host] ntap_collector -p 5678 -k hello -e eth2
도움말을 보니까 이런 부분이 있습니다.
Options:
[–interface|-i]| Virtual interface where received packets are injected
[–egress|-e]| Host interface on which received packets are injected
Virtual interface를 대신하여 Host Interface를 사용하여 실행하였습니다. 결과는 앞서와 동일합니다. Demo Mode라서 그런가 해서 tcpdump로 확인하였습니다.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
[smallake@xeon ~]$ sudo tcpdump -i enp9s0 udp port 5678 -v dropped privs to tcpdump tcpdump: listening on enp9s0, link-type EN10MB (Ethernet), capture size 262144 bytes 09:35:27.822058 IP (tos 0x0, ttl 63, id 21174, offset 0, flags [+], proto UDP (17), length 1500) 121.166.192.199.46071 > xeon.rrac: UDP, bad length 1522 > 1472 09:35:27.822229 IP (tos 0x0, ttl 63, id 21175, offset 0, flags [+], proto UDP (17), length 1500) 121.166.192.199.46071 > xeon.rrac: UDP, bad length 1560 > 1472 09:35:27.822629 IP (tos 0x0, ttl 63, id 21176, offset 0, flags [+], proto UDP (17), length 1500) 121.166.192.199.46071 > xeon.rrac: UDP, bad length 1514 > 1472 09:35:28.832884 IP (tos 0x0, ttl 63, id 21394, offset 0, flags [DF], proto UDP (17), length 606) 121.166.192.199.46071 > xeon.rrac: UDP, length 578 09:35:28.845286 IP (tos 0x0, ttl 63, id 21397, offset 0, flags [+], proto UDP (17), length 1500) 121.166.192.199.46071 > xeon.rrac: UDP, bad length 1556 > 1472 09:35:28.845351 IP (tos 0x0, ttl 63, id 21398, offset 0, flags [+], proto UDP (17), length 1500) 121.166.192.199.46071 > xeon.rrac: UDP, bad length 1668 > 1472 09:35:28.845756 IP (tos 0x0, ttl 63, id 21399, offset 0, flags [+], proto UDP (17), length 1500) 121.166.192.199.46071 > xeon.rrac: UDP, bad length 1496 > 1472 09:35:28.845834 IP (tos 0x0, ttl 63, id 21400, offset 0, flags [+], proto UDP (17), length 1500) 121.166.192.199.46071 > xeon.rrac: UDP, bad length 2060 > 1472 09:35:28.845972 IP (tos 0x0, ttl 63, id 21401, offset 0, flags [+], proto UDP (17), length 1500) 121.166.192.199.46071 > xeon.rrac: UDP, bad length 1970 > 1472 |
데이타가 들어옵니다. 동작중이라는 뜻이네요. 어찌되었든 예상했던 결과를 보지 못하였습니다. ntap_collector는 라이센스가 있어야 합니다. 이를 피하는 방법은 직접 개발하는 방식입니다. 관련한 API입니다.
Developing nTap-based Applications
nTap can be embedded in non-ntop applications by using the nTap SDK that by means of a simple API, allows the creation of custom collector applications. The SDK is part of the nTap package and ince installed it comes with:
/usr/lib/libntap.a (under BSD systems it goes under /usr/local/lib/libntap.a)
/usr/include/libntap.h (under BSD systems it goes under /usr/local/include/libntap.h)
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
#include <stdio.h> #include "libntap.h" /* **************************************** */ void process_pkt(unsigned char *pkt, unsigned int pkt_len) { printf("Received %u bytes packet\n", pkt_len); } /* **************************************** */ int main(int argc, char *argv[]) { void *ntap_info; unsigned int listening_port = 1234; const char *password = "hello"; if(ntaplib_init(password, listening_port, &ntap_info) != 0) { printf("WARNING: Intialization error\n"); return(-1); } else printf("Listening for packets on UDP port %u\n", listening_port); while(ntaplib_poll(ntap_info, process_pkt) > 0) { /* printf("Processed packet"); */ } printf("Terminating ntap library\n"); ntaplib_term(ntap_info); return(0); } |
길을 찾았지만 제가 모르는 것이 있었습니다. ntap과 관련한 부분입니다. virtaul Interface라고 한 부분을 무심히 넘겼습니다. 문서대로 하였지만 머리를 쓰지 않고 하였습니다. 생각해 보면 ntap0라는 Virtual interface에 대한 이해를 하여야 했습니다. ntap이 동작하도록 enp9s0와 연결하여야 했습니다. 즉, Vitural Interface에 대한 공부가 없었습니다.
이 부분을 추가로 정리하려고 합니다.
