IPMI와 KVM-over-LAN

1.
DMA 거래를 위한 서버를 공급할 때 IPMI를 요구하는 경우가 있습니다. 서울IDC는 수도권 트레이더에게 접근성이 용이하지만 부산의 경우는 다릅니다. 거리때문에 조치를 하려면 시간이 필요합니다. 서버용으로 나오는 마더보드는 IPMI기능을 기본으로 탑재하고 있습니다. 다만 Xeon을 사용하기 때문에 속도에서 이익이 거의 없습니다. 데스크탑 마더보드(고사양 게임용)는 IPMI기능을 기본으로 지원하지 않고 특정한 보드만 지원합니다. 지원방식도 PCI방식의 IPMI카드를 설치하는 방식입니다. 이 경우에도 BIOS의 Server Mgnt 기능으로 지원하는 보트와 그렇지 않은 보드로 나뉩니다.

이번 작업을 하여야 하는 보드는 BIOS에서 IPMI를 지원하지 않는 보드입니다. 다만 제조사가 IPMI를 지원한다고 하여 IPMI Expansion Card를 설치하고 부팅하였습니다. 우선 CentOS에서 IPMI를 사용하기 위한 IPMITools를 설치하였습니다. 방식은 두가지입니다. 소스를 직접 컴파일하여 사용할 수도 있고 dnf와 같은 패키지관리기능을 이용하여 설치할 수 있습니다.

먼저 소스컴파일 과정입니다.

[root@smallake ipmitool]# dnf install readline-devel
Last metadata expiration check: 0:01:21 ago on Thu 07 Nov 2024 01:54:09 PM KST.
Dependencies resolved.
=============================================================================================================================================================
 Package                                   Architecture                    Version                                     Repository                       Size
=============================================================================================================================================================
Installing:
 readline-devel                            x86_64                          7.0-10.el8                                  baseos                          204 k
Installing dependencies:
 ncurses-c++-libs                          x86_64                          6.1-9.20180224.el8                          baseos                           58 k
 ncurses-devel                             x86_64                          6.1-9.20180224.el8                          baseos                          528 k

Transaction Summary
=============================================================================================================================================================
Install  3 Packages

Total download size: 790 k
Installed size: 1.4 M
Is this ok [y/N]: y
Downloading Packages:
(1/3): ncurses-c++-libs-6.1-9.20180224.el8.x86_64.rpm                                                                         98 kB/s |  58 kB     00:00    
(2/3): readline-devel-7.0-10.el8.x86_64.rpm                                                                                  274 kB/s | 204 kB     00:00    
(3/3): ncurses-devel-6.1-9.20180224.el8.x86_64.rpm                                                                           579 kB/s | 528 kB     00:00    
-------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                        864 kB/s | 790 kB     00:00     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                                                                     1/1 
  Installing       : ncurses-c++-libs-6.1-9.20180224.el8.x86_64                                                                                          1/3 
  Installing       : ncurses-devel-6.1-9.20180224.el8.x86_64                                                                                             2/3 
  Installing       : readline-devel-7.0-10.el8.x86_64                                                                                                    3/3 
  Running scriptlet: readline-devel-7.0-10.el8.x86_64                                                                                                    3/3 
  Verifying        : ncurses-c++-libs-6.1-9.20180224.el8.x86_64                                                                                          1/3 
  Verifying        : ncurses-devel-6.1-9.20180224.el8.x86_64                                                                                             2/3 
  Verifying        : readline-devel-7.0-10.el8.x86_64                                                                                                    3/3 

Installed:
  ncurses-c++-libs-6.1-9.20180224.el8.x86_64              ncurses-devel-6.1-9.20180224.el8.x86_64              readline-devel-7.0-10.el8.x86_64             

Complete!
[root@smallake ipmitool]# ./configure 
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking target system type... x86_64-pc-linux-gnu
checking for a BSD-compatible install... /bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking whether gcc understands -c and -o together... yes
checking whether make supports the include directive... yes (GNU style)
checking dependency style of gcc... gcc3
checking whether ln -s works... yes
checking whether make sets $(MAKE)... (cached) yes
checking for rpmbuild... rpmbuild
checking for sed... sed
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for stdlib.h... (cached) yes
checking for string.h... (cached) yes
checking sys/ioctl.h usability... yes
checking sys/ioctl.h presence... yes
checking for sys/ioctl.h... yes
checking for sys/stat.h... (cached) yes
checking for unistd.h... (cached) yes
checking paths.h usability... yes
checking paths.h presence... yes
checking for paths.h... yes
checking arpa/inet.h usability... yes
checking arpa/inet.h presence... yes
checking for arpa/inet.h... yes
checking fcntl.h usability... yes
checking fcntl.h presence... yes
checking for fcntl.h... yes
checking netdb.h usability... yes
checking netdb.h presence... yes
checking for netdb.h... yes
checking netinet/in.h usability... yes
checking netinet/in.h presence... yes
checking for netinet/in.h... yes
checking sys/socket.h usability... yes
checking sys/socket.h presence... yes
checking for sys/socket.h... yes
checking sys/byteorder.h usability... no
checking sys/byteorder.h presence... no
checking for sys/byteorder.h... no
checking byteswap.h usability... yes
checking byteswap.h presence... yes
checking for byteswap.h... yes
checking for an ANSI C-conforming const... yes
checking for inline... inline
checking whether byte ordering is bigendian... no
checking sys/select.h usability... yes
checking sys/select.h presence... yes
checking for sys/select.h... yes
checking for sys/socket.h... (cached) yes
checking types of arguments for select... int,fd_set *,struct timeval *
checking for working strtod... yes
checking for alarm... yes
checking for gethostbyname... yes
checking for getaddrinfo... yes
checking for getifaddrs... yes
checking for socket... yes
checking for select... yes
checking for memmove... yes
checking for memset... yes
checking for strchr... yes
checking for strdup... yes
checking for strerror... yes
checking for getpassphrase... no
checking how to print strings... printf
checking for a sed that does not truncate output... sed
checking for fgrep... /bin/grep -F
checking for ld used by gcc... /bin/ld
checking if the linker (/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /bin/nm -B
checking the name lister (/bin/nm -B) interface... BSD nm
checking the maximum length of command line arguments... 1572864
checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @FILE support... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /bin/nm -B output from gcc object... ok
checking for sysroot... no
checking for a working dd... /bin/dd
checking how to truncate binary pipes... /bin/dd bs=4096 count=1
checking for mt... no
checking if : is a manifest tool... no
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC -DPICfiletype:pdf
checking if gcc PIC flag -fPIC -DPIC works... yes
checking if gcc static flag -static works... no
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/bin/ld -m elf_x86_64) supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking for library containing gethostbyname... none required
checking for library containing getaddrinfo... none required
checking for library containing getifaddrs... none required
checking for library containing socket... none required
checking for wget... no
checking for curl... curl
configure: WARNING: ** Download is: curl --location --progress-bar
checking for EVP_aes_128_cbc in -lcrypto... no
checking for EVP_sha256 in -lcrypto... no
checking for MD5_Init in -lcrypto... no
checking for MD2_Init in -lcrypto... no
checking for sd_bus_default in -lsystemd... no
checking for linux/compiler.h... no
checking sys/ioccom.h usability... no
checking sys/ioccom.h presence... no
checking for sys/ioccom.h... no
checking linux/ipmi.h usability... yes
checking linux/ipmi.h presence... yes
checking for linux/ipmi.h... yes
checking for ipmi_open_inband in -lfreeipmi... no
checking for ipmi_ctx_open_inband in -lfreeipmi... no
checking termios.h usability... yes
checking termios.h presence... yes
checking for termios.h... yes
checking for OpenIPMI dual bridge support... no
checking sys/lipmi/lipmi_intf.h usability... no
checking sys/lipmi/lipmi_intf.h presence... no
checking for sys/lipmi/lipmi_intf.h... no
** Unable to build Solaris 9 x86 IPMI interface support!
checking for pkg-config... /bin/pkg-config
checking pkg-config is at least version 0.9.0... yes
checking for READLINE... no
checking for library containing readline... -lreadline
DEFAULT_INTF not found in environment; setting to open
Set IANA PEN dictionary search path to /usr/local/share/misc
Set user's IANA PEN dictionary search path to .local/usr/share/misc
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: creating doc/Makefile
config.status: creating contrib/Makefile
config.status: creating control/Makefile
config.status: creating control/pkginfo
config.status: creating control/prototype
config.status: creating control/ipmitool.spec
config.status: creating lib/Makefile
config.status: creating include/Makefile
config.status: creating include/ipmitool/Makefile
config.status: creating src/Makefile
config.status: creating src/plugins/Makefile
config.status: creating src/plugins/lan/Makefile
config.status: creating src/plugins/lanplus/Makefile
config.status: creating src/plugins/open/Makefile
config.status: creating src/plugins/free/Makefile
config.status: creating src/plugins/imb/Makefile
config.status: creating src/plugins/bmc/Makefile
config.status: creating src/plugins/dbus/Makefile
config.status: creating src/plugins/usb/Makefile
config.status: creating src/plugins/lipmi/Makefile
config.status: creating src/plugins/serial/Makefile
config.status: creating src/plugins/dummy/Makefile
config.status: creating doc/ipmitool.1
config.status: creating doc/ipmievd.8
config.status: creating config.h
config.status: executing depfiles commands
config.status: executing libtool commands

ipmitool 1.8.19.13.gbe11d94

Interfaces (default=open)
  lan     : yes
  lanplus : no
  open    : yes
  free    : no
  imb     : yes
  bmc     : no
  dbus    : no
  usb     : no
  lipmi   : no
  serial  : yes
  dummy   : no

Extra tools
  ipmievd   : yes
  ipmishell : yes

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

[root@smallake ipmitool]# dnf install readline-devel

Last metadata expiration check: 0:01:21 ago on Thu 07 Nov 2024 01:54:09 PM KST.

Dependencies resolved.

=============================================================================================================================================================

Package Architecture Version Repository Size

=============================================================================================================================================================

Installing:

readline-devel x86_64 7.0-10.el8 baseos 204 k

Installing dependencies:

ncurses-c++-libs x86_64 6.1-9.20180224.el8 baseos 58 k

ncurses-devel x86_64 6.1-9.20180224.el8 baseos 528 k

Transaction Summary

=============================================================================================================================================================

Install 3 Packages

Total download size: 790 k

Installed size: 1.4 M

Is this ok [y/N]: y

Downloading Packages:

(1/3): ncurses-c++-libs-6.1-9.20180224.el8.x86_64.rpm 98 kB/s | 58 kB 00:00

(2/3): readline-devel-7.0-10.el8.x86_64.rpm 274 kB/s | 204 kB 00:00

(3/3): ncurses-devel-6.1-9.20180224.el8.x86_64.rpm 579 kB/s | 528 kB 00:00

-------------------------------------------------------------------------------------------------------------------------------------------------------------

Total 864 kB/s | 790 kB 00:00

Running transaction check

Transaction check succeeded.

Running transaction test

Transaction test succeeded.

Running transaction

Preparing : 1/1

Installing : ncurses-c++-libs-6.1-9.20180224.el8.x86_64 1/3

Installing : ncurses-devel-6.1-9.20180224.el8.x86_64 2/3

Installing : readline-devel-7.0-10.el8.x86_64 3/3

Running scriptlet: readline-devel-7.0-10.el8.x86_64 3/3

Verifying : ncurses-c++-libs-6.1-9.20180224.el8.x86_64 1/3

Verifying : ncurses-devel-6.1-9.20180224.el8.x86_64 2/3

Verifying : readline-devel-7.0-10.el8.x86_64 3/3

Installed:

ncurses-c++-libs-6.1-9.20180224.el8.x86_64 ncurses-devel-6.1-9.20180224.el8.x86_64 readline-devel-7.0-10.el8.x86_64

Complete!

[root@smallake ipmitool]# ./configure

checking build system type... x86_64-pc-linux-gnu

checking host system type... x86_64-pc-linux-gnu

checking target system type... x86_64-pc-linux-gnu

checking for a BSD-compatible install... /bin/install -c

checking whether build environment is sane... yes

checking for a thread-safe mkdir -p... /bin/mkdir -p

checking for gawk... gawk

checking whether make sets $(MAKE)... yes

checking whether make supports nested variables... yes

checking for gcc... gcc

checking whether the C compiler works... yes

checking for C compiler default output file name... a.out

checking for suffix of executables...

checking whether we are cross compiling... no

checking for suffix of object files... o

checking whether we are using the GNU C compiler... yes

checking whether gcc accepts -g... yes

checking for gcc option to accept ISO C89... none needed

checking whether gcc understands -c and -o together... yes

checking whether make supports the include directive... yes (GNU style)

checking dependency style of gcc... gcc3

checking whether ln -s works... yes

checking whether make sets $(MAKE)... (cached) yes

checking for rpmbuild... rpmbuild

checking for sed... sed

checking how to run the C preprocessor... gcc -E

checking for grep that handles long lines and -e... /bin/grep

checking for egrep... /bin/grep -E

checking for ANSI C header files... yes

checking for sys/types.h... yes

checking for sys/stat.h... yes

checking for stdlib.h... yes

checking for string.h... yes

checking for memory.h... yes

checking for strings.h... yes

checking for inttypes.h... yes

checking for stdint.h... yes

checking for unistd.h... yes

checking for stdlib.h... (cached) yes

checking for string.h... (cached) yes

checking sys/ioctl.h usability... yes

checking sys/ioctl.h presence... yes

checking for sys/ioctl.h... yes

checking for sys/stat.h... (cached) yes

checking for unistd.h... (cached) yes

checking paths.h usability... yes

checking paths.h presence... yes

checking for paths.h... yes

checking arpa/inet.h usability... yes

checking arpa/inet.h presence... yes

checking for arpa/inet.h... yes

checking fcntl.h usability... yes

checking fcntl.h presence... yes

checking for fcntl.h... yes

checking netdb.h usability... yes

checking netdb.h presence... yes

checking for netdb.h... yes

checking netinet/in.h usability... yes

checking netinet/in.h presence... yes

checking for netinet/in.h... yes

checking sys/socket.h usability... yes

checking sys/socket.h presence... yes

checking for sys/socket.h... yes

checking sys/byteorder.h usability... no

checking sys/byteorder.h presence... no

checking for sys/byteorder.h... no

checking byteswap.h usability... yes

checking byteswap.h presence... yes

checking for byteswap.h... yes

checking for an ANSI C-conforming const... yes

checking for inline... inline

checking whether byte ordering is bigendian... no

checking sys/select.h usability... yes

checking sys/select.h presence... yes

checking for sys/select.h... yes

checking for sys/socket.h... (cached) yes

checking types of arguments for select... int,fd_set *,struct timeval *

checking for working strtod... yes

checking for alarm... yes

checking for gethostbyname... yes

checking for getaddrinfo... yes

checking for getifaddrs... yes

checking for socket... yes

checking for select... yes

checking for memmove... yes

checking for memset... yes

checking for strchr... yes

checking for strdup... yes

checking for strerror... yes

checking for getpassphrase... no

checking how to print strings... printf

checking for a sed that does not truncate output... sed

checking for fgrep... /bin/grep -F

checking for ld used by gcc... /bin/ld

checking if the linker (/bin/ld) is GNU ld... yes

checking for BSD- or MS-compatible name lister (nm)... /bin/nm -B

checking the name lister (/bin/nm -B) interface... BSD nm

checking the maximum length of command line arguments... 1572864

checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop

checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop

checking for /bin/ld option to reload object files... -r

checking for objdump... objdump

checking how to recognize dependent libraries... pass_all

checking for dlltool... no

checking how to associate runtime and link libraries... printf %s\n

checking for ar... ar

checking for archiver @FILE support... @

checking for strip... strip

checking for ranlib... ranlib

checking command to parse /bin/nm -B output from gcc object... ok

checking for sysroot... no

checking for a working dd... /bin/dd

checking how to truncate binary pipes... /bin/dd bs=4096 count=1

checking for mt... no

checking if : is a manifest tool... no

checking for dlfcn.h... yes

checking for objdir... .libs

checking if gcc supports -fno-rtti -fno-exceptions... no

checking for gcc option to produce PIC... -fPIC -DPICfiletype:pdf

checking if gcc PIC flag -fPIC -DPIC works... yes

checking if gcc static flag -static works... no

checking if gcc supports -c -o file.o... yes

checking if gcc supports -c -o file.o... (cached) yes

checking whether the gcc linker (/bin/ld -m elf_x86_64) supports shared libraries... yes

checking whether -lc should be explicitly linked in... no

checking dynamic linker characteristics... GNU/Linux ld.so

checking how to hardcode library paths into programs... immediate

checking whether stripping libraries is possible... yes

checking if libtool supports shared libraries... yes

checking whether to build shared libraries... yes

checking whether to build static libraries... yes

checking for library containing gethostbyname... none required

checking for library containing getaddrinfo... none required

checking for library containing getifaddrs... none required

checking for library containing socket... none required

checking for wget... no

checking for curl... curl

configure: WARNING: ** Download is: curl --location --progress-bar

checking for EVP_aes_128_cbc in -lcrypto... no

checking for EVP_sha256 in -lcrypto... no

checking for MD5_Init in -lcrypto... no

checking for MD2_Init in -lcrypto... no

checking for sd_bus_default in -lsystemd... no

checking for linux/compiler.h... no

checking sys/ioccom.h usability... no

checking sys/ioccom.h presence... no

checking for sys/ioccom.h... no

checking linux/ipmi.h usability... yes

checking linux/ipmi.h presence... yes

checking for linux/ipmi.h... yes

checking for ipmi_open_inband in -lfreeipmi... no

checking for ipmi_ctx_open_inband in -lfreeipmi... no

checking termios.h usability... yes

checking termios.h presence... yes

checking for termios.h... yes

checking for OpenIPMI dual bridge support... no

checking sys/lipmi/lipmi_intf.h usability... no

checking sys/lipmi/lipmi_intf.h presence... no

checking for sys/lipmi/lipmi_intf.h... no

** Unable to build Solaris 9 x86 IPMI interface support!

checking for pkg-config... /bin/pkg-config

checking pkg-config is at least version 0.9.0... yes

checking for READLINE... no

checking for library containing readline... -lreadline

DEFAULT_INTF not found in environment; setting to open

Set IANA PEN dictionary search path to /usr/local/share/misc

Set user's IANA PEN dictionary search path to .local/usr/share/misc

checking that generated files are newer than configure... done

configure: creating ./config.status

config.status: creating Makefile

config.status: creating doc/Makefile

config.status: creating contrib/Makefile

config.status: creating control/Makefile

config.status: creating control/pkginfo

config.status: creating control/prototype

config.status: creating control/ipmitool.spec

config.status: creating lib/Makefile

config.status: creating include/Makefile

config.status: creating include/ipmitool/Makefile

config.status: creating src/Makefile

config.status: creating src/plugins/Makefile

config.status: creating src/plugins/lan/Makefile

config.status: creating src/plugins/lanplus/Makefile

config.status: creating src/plugins/open/Makefile

config.status: creating src/plugins/free/Makefile

config.status: creating src/plugins/imb/Makefile

config.status: creating src/plugins/bmc/Makefile

config.status: creating src/plugins/dbus/Makefile

config.status: creating src/plugins/usb/Makefile

config.status: creating src/plugins/lipmi/Makefile

config.status: creating src/plugins/serial/Makefile

config.status: creating src/plugins/dummy/Makefile

config.status: creating doc/ipmitool.1

config.status: creating doc/ipmievd.8

config.status: creating config.h

config.status: executing depfiles commands

config.status: executing libtool commands

ipmitool 1.8.19.13.gbe11d94

Interfaces (default=open)

lan : yes

lanplus : no

open : yes

free : no

imb : yes

bmc : no

dbus : no

usb : no

lipmi : no

serial : yes

dummy : no

Extra tools

ipmievd : yes

ipmishell : yes

패키지를 설치한 후 부팅을 하고 커널모듈을 확인하였습니다. 아래와 같이 ipmi_si를 확인할 수 있습니다.

[smallake@smallake ~]$ lsmod | grep ipmi
ipmi_si 69632 0
ipmi_devintf 20480 0
ipmi_msghandler 110592 2 ipmi_devintf,ipmi_si

그런데 ipmitool 명령어를 실행하면 오류가 발생합니다. 검색을 해보니까 IPMI와 관련한 서비스를 실행하라고 합니다.

[root@smallake ~]$ sudo systemctl enable ipmi.service
Created symlink /etc/systemd/system/multi-user.target.wants/ipmi.service → /usr/lib/systemd/system/ipmi.service.
[smallake@smallake ~]$ sudo systemctl start ipmi.service
[smallake@smallake ~]$ sudo systemctl status ipmi.service
● ipmi.service - IPMI Driver
   Loaded: loaded (/usr/lib/systemd/system/ipmi.service; enabled; vendor preset: disabled)
   Active: active (exited) since Thu 2024-11-07 14:37:29 KST; 5s ago
  Process: 2075 ExecStart=/usr/libexec/openipmi-helper start (code=exited, status=0/SUCCESS)
 Main PID: 2075 (code=exited, status=0/SUCCESS)

Nov 07 14:37:29 smallake systemd[1]: Starting IPMI Driver...
Nov 07 14:37:29 smallake systemd[1]: Started IPMI Driver.

[root@smallake ~]$ sudo systemctl enable ipmi.service

Created symlink /etc/systemd/system/multi-user.target.wants/ipmi.service → /usr/lib/systemd/system/ipmi.service.

[smallake@smallake ~]$ sudo systemctl start ipmi.service

[smallake@smallake ~]$ sudo systemctl status ipmi.service

● ipmi.service - IPMI Driver

Loaded: loaded (/usr/lib/systemd/system/ipmi.service; enabled; vendor preset: disabled)

Active: active (exited) since Thu 2024-11-07 14:37:29 KST; 5s ago

Process: 2075 ExecStart=/usr/libexec/openipmi-helper start (code=exited, status=0/SUCCESS)

Main PID: 2075 (code=exited, status=0/SUCCESS)

Nov 07 14:37:29 smallake systemd[1]: Starting IPMI Driver...

Nov 07 14:37:29 smallake systemd[1]: Started IPMI Driver.

다시 명령어를 실행하니까 오류가 발생합니다. 그래서 부팅할 때 로그를 확인하였습니다.

[root@smallake ~]$ dmesg | grep ipmi
[    5.627352] ipmi device interface
[    5.656388] ipmi_si: IPMI System Interface driver
[    5.656418] ipmi_si 0000:73:01.0: probing via PCI
[    5.656484] ipmi_si 0000:73:01.0: enabling device (0000 -> 0002)
[    5.656487] ipmi_si 0000:73:01.0: Could not setup I/O space
[    5.656547] ipmi_si 0000:73:01.0: [mem 0x74000000-0x740fffff] regsize 1 spacing 1 irq 0
[    5.656548] ipmi_si: Adding PCI-specified kcs state machine
[    5.656555] ipmi_si: Trying PCI-specified kcs state machine at mem address 0x74000000, slave address 0x0, irq 0
[    5.656578] ipmi_si 0000:73:01.0: Interface detection failed

[root@smallake ~]$ dmesg | grep ipmi

[ 5.627352] ipmi device interface

[ 5.656388] ipmi_si: IPMI System Interface driver

[ 5.656418] ipmi_si 0000:73:01.0: probing via PCI

[ 5.656484] ipmi_si 0000:73:01.0: enabling device (0000 -> 0002)

[ 5.656487] ipmi_si 0000:73:01.0: Could not setup I/O space

[ 5.656547] ipmi_si 0000:73:01.0: [mem 0x74000000-0x740fffff] regsize 1 spacing 1 irq 0

[ 5.656548] ipmi_si: Adding PCI-specified kcs state machine

[ 5.656555] ipmi_si: Trying PCI-specified kcs state machine at mem address 0x74000000, slave address 0x0, irq 0

[ 5.656578] ipmi_si 0000:73:01.0: Interface detection failed

OS가 IPMI Driver를 인식하지 못하는 상황입니다. 이유를 확인하였습니다. BIOS 정보를 확인하는 dmidecode를 실행하였습니다. IPMI카드의 경우 type이 38입니다.

smallake~]$dmidecode | grep -A10 IPMI

을 하면 결과가 없습니다. BIOS정보에서는 IPMI가 없습니다. 반면 OS가 인식한 하드웨어정보를 보면 IPMI가 있습니다.

*-serial
description: IPMI Interface
product: ASPEED Technology, Inc.
vendor: ASPEED Technology, Inc.
physical id: 1
bus info: pci@0000:73:01.0
version: 01
width: 32 bits
clock: 33MHz
capabilities: pm msi kcs cap_list
configuration: driver=ipmi_si latency=0
resources: irq:0 memory:74000000-740fffff memory:74100000-7413ffff

결국 OS는 IPMI가 있어서 관련한 하드웨어에 접근하려고 하는데 BIOS로부터 IPMI정보를 받은 것이 없어서 인식하지 못한 것이 아닐까 추측합니다. 만약 BIOS에서 데이타를 가져오면 아래와 같은 결과이지 않을까 합니다.

Oct 10 21:51:22 hp-microserver kernel: ipmi message handler version 39.2
Oct 10 21:51:22 hp-microserver kernel: IPMI System Interface driver.
Oct 10 21:51:22 hp-microserver kernel: ipmi_si: Adding hardcoded-specified kcs state machine
Oct 10 21:51:22 hp-microserver kernel: ipmi_si: Trying hardcoded-specified kcs state machine at i/o address 0xca8, slave address 0x0, irq 0
Oct 10 21:51:23 hp-microserver kernel: ipmi: Found new BMC (man_id: 0x000001, prod_id: 0x3431, dev_id: 0x20)
Oct 10 21:51:23 hp-microserver kernel: IPMI kcs interface initialized

여기서 핵심은 I/O space(i/o address)정보인 듯 합니다. 구글링을 통해 몇 가지 시도를 해보았지만 IO정보를 확인할 수 없습니다. 아마도 고수는 아실 수 있지만.

2.
지원을 하지 않는가 의구심을 가졌지만 윈도우에서는 가능했다는 말을 듣고 확인을 해보았습니다. 같은 사양의 다른 서버에 윈도우10을 설치해서 작업을 진행하였습니다. 제가 예상한 작업은 윈도우상에서 IPMITool을 이용하여 IPMI설정을 하고 필요한 작업을 하는 것이었습니다. 윈도우에 설치하여야 하는 소프트웨어를 설치한 후 화면을 보았습니다. 가장 먼저 확인한 부분이 ‘네트워크설정’입니다.

다음으로 cmd를 실행하여 ipmitools.exe를 실행하였습니다. 예상과 다른 결과이지만 리눅스와 같은 결과입니다.

네트워크와 관련한 설정을 보면 AMI Ndis adatpor가 보입니다. 아마도 IPMI가 제공하는 랜 어댑터로 보입니다. 이를 이용하여 설정하려고 하지만 설정하더라도 동작하지 않습니다. 어떻게 네트워크를 설정하고 IPMI기능을 이용하였을까 최초 시험자에게 문의하였습니다. 간단하더군요. DHCP를 이용하였습니다.

저도 DHCP 내역을 보면 관련한 주소로 접근해보았습니다. 정상적으로 IPMI를 이용한 기능을 사용할 수 있었습니다.

IPMI와 관련한 Baseboard Management Controller(BMC) Chip을 만드는 Aspeed 제품소개서를 본 적이 있습니다. ARM Processor를 내장하고 있습니다. 이 말은 Intel/AMD 구조와는 완전히 다른 환경을 제공한다는 뜻입니다. 즉, 마더보드와 직접적으로 연결하여 원격관리를 위해 필요한 정보 및 관리를 할 수 있도록 하는 별도의 시스템이라는 뜻입니다. IPMITools와 같은 소프트웨어들은 OS내에서 직접적으로 관리하기 위한 또다른 환경입니다. OS에서 접근하지 못하더라도 IPMI기능을 사용할 수 있는 이유입니다.

BMC와 관련한 국책연구소 자료입니다. 기술을 이해하기 위함입니다.

Download (PDF, 1.44MB)

2 Comments

PotatoGim 12월 23, 2024 at 6:50 오전

안녕하세요
항상 양질의 포스팅 올려주셔서 잘 보고 있습니다!
NanoKVM 이라는 제품이 PCI 슬롯 장착형으로 나왔던데 이런 제품들도 도움이 되실지 궁금하네요

Reply ↓
1. smallake (Post author)12월 24, 2024 at 10:22 오전
  
  감사합니다. 말씀하신 제품을 고려해보겠습니다.
  
  Reply ↓

이 글 공유하기:

2 Comments

Leave a Comment 응답 취소