1.
오래 전에 쓴 IPMI와 Bios 설정에 이어지는 글입니다.
오버클락서버를 요청하는 분들중 IMPI기능을 필요로 하는 분들이 계십니다. 일반 데스크탑 마더보드이지만 IPMI카드를 추가할 수 있는 모델이 있기때문입니다. IPMI와 관련한 작업은 BIOS의 관련한 메뉴에서 IP주소를 설정하고 driver유형을 Linux로 선택하는 일정도만 하였습니다. 오랜 동안 잊고지내다가 고객의 요구로 ipmitool를 설치하여야 하는 일이 생겼습니다. 예전에 성공하지 못했던 기억이 떠올랐습니다.
이전에 실패했던 명령입니다.
sudo modprobe ipmi_devintf
sudo modprobe ipmi_si
다시 실패할 수 없어서 꼼꼼히 문서를 확인하였습니다. Gemini나 ChatGPT는 아주 일반적인 답변을 하기 때문에 직접 조사하는 길을 택하였습니다.
2.
우선 Asus가 제공하는 참고자료를 확인하였습니다.
여기에서 Remote Management LAN Card IPMI expansion card User Guide를 찾아서 살폈습니다. 예전에 살폈는데 혹 놓친 부분이 있나 확인을 하였습니다. 역시나 길은 제조사의 메뉴얼에 있었습니다.
앞서 AI가 말한 방법과 하나가 다릅니다.
sudo modprobe ipmi_si
sudo modprobe ipmi_msghandler
The Linux IPMI Driver와 관련한 리눅스 커널문서중 message handler입니다.
The Upper Layer Interface (Message Handler)
The upper layer of the interface provides the users with a consistent view of the IPMI interfaces. It allows multiple SMI interfaces to be addressed (because some boards actually have multiple BMCs on them) and the user should not have to care what type of SMI is below them.
Watching For Interfaces
When your code comes up, the IPMI driver may or may not have detected if IPMI devices exist. So you might have to defer your setup until the device is detected, or you might be able to do it immediately. To handle this, and to allow for discovery, you register an SMI watcher with
ipmi_smi_watcher_register()to iterate over interfaces and tell you when they come and go.Creating the User
To use the message handler, you must first create a user using ipmi_create_user. The interface number specifies which SMI you want to connect to, and you must supply callback functions to be called when data comes in. This also allows to you pass in a piece of data, the handler_data, that will be passed back to you on all calls.
Once you are done, call
ipmi_destroy_user()to get rid of the user.From userland, opening the device automatically creates a user, and closing the device automatically destroys the user.
Messaging
To send a message from kernel-land, the
ipmi_request_settime()call does pretty much all message handling. Most of the parameter are self-explanatory. However, it takes a “msgid” parameter. This is NOT the sequence number of messages. It is simply a long value that is passed back when the response for the message is returned. You may use it for anything you like.Responses come back in the function pointed to by the ipmi_recv_hndl field of the “handler” that you passed in to
ipmi_create_user(). Remember to look at the receive type, too.From userland, you fill out an ipmi_req_t structure and use the IPMICTL_SEND_COMMAND ioctl. For incoming stuff, you can use select() or poll() to wait for messages to come in. However, you cannot use read() to get them, you must call the IPMICTL_RECEIVE_MSG with the ipmi_recv_t structure to actually get the message. Remember that you must supply a pointer to a block of data in the msg.data field, and you must fill in the msg.data_len field with the size of the data. This gives the receiver a place to actually put the message.
If the message cannot fit into the data you provide, you will get an EMSGSIZE error and the driver will leave the data in the receive queue. If you want to get it and have it truncate the message, use the IPMICTL_RECEIVE_MSG_TRUNC ioctl.
When you send a command (which is defined by the lowest-order bit of the netfn per the IPMI spec) on the IPMB bus, the driver will automatically assign the sequence number to the command and save the command. If the response is not received in the IPMI-specified 5 seconds, it will generate a response automatically saying the command timed out. If an unsolicited response comes in (if it was after 5 seconds, for instance), that response will be ignored.
In kernelland, after you receive a message and are done with it, you MUST call
ipmi_free_recv_msg()on it, or you will leak messages. Note that you should NEVER mess with the “done” field of a message, that is required to properly clean up the message.Note that when sending, there is an
ipmi_request_supply_msgs()call that lets you supply the smi and receive message. This is useful for pieces of code that need to work even if the system is out of buffers (the watchdog timer uses this, for instance). You supply your own buffer and own free routines. This is not recommended for normal use, though, since it is tricky to manage your own buffers.Events and Incoming Commands
The driver takes care of polling for IPMI events and receiving commands (commands are messages that are not responses, they are commands that other things on the IPMB bus have sent you). To receive these, you must register for them, they will not automatically be sent to you.
To receive events, you must call
ipmi_set_gets_events()and set the “val” to non-zero. Any events that have been received by the driver since startup will immediately be delivered to the first user that registers for events. After that, if multiple users are registered for events, they will all receive all events that come in.For receiving commands, you have to individually register commands you want to receive. Call
ipmi_register_for_cmd()and supply the netfn and command name for each command you want to receive. You also specify a bitmask of the channels you want to receive the command from (or use IPMI_CHAN_ALL for all channels if you don’t care). Only one user may be registered for each netfn/cmd/channel, but different users may register for different commands, or the same command if the channel bitmasks do not overlap.To respond to a received command, set the response bit in the returned netfn, use the address from the received message, and use the same msgid that you got in the received message.
From userland, equivalent IOCTLs are provided to do these functions.
위 명령을 실행하니까 모든 것이 정상적으로 돌아갑니다. IPMI 드라이버를 이해했으면 가장 기본적인 절차였을텐데..
3.
대부분 고객들은 Rockylinux 혹은 Centos를 사용하지만 특별한 경우 Ubuntu를 요청하는 경우가 있습니다. 22.04 혹은 24.04를 주로 설치합니다. OS를 설치하고 커널 튜닝까지 마무리한 후 yaml 파일을 이용하여 네트워크 설정을 합니다. 설정을 하고 난 후 확인을 하려고 부팅을 합니다. 이 때 이상한 경험을 하였습니다. 모든 설정값이 사라지고 초기화하는 현상이 보였습니다.무슨 원인이고 어떻게 대응해야 할지 확인하였습니다.
먼저 원인입니다. Link 연결이 없는 경우 cloud-init가 부팅 시 /etc/netplan/50-cloud-init.yaml 파일을 만들어 기존 수동 설정을 덮어버립니다.
이에 대응하는 방법입니다.
첫째는 cloud-init를 비활성화하는 방법입니다.
Ubuntu 22.04에서 재부팅 시 Netplan 네트워크 설정이 사라지는 문제 해결하기
둘째는 yaml파일에 옵션을 추가하는 방법입니다. Optional 값을 사용하는 방법입니다.
An optional device is not required for booting. Normally, networkd will wait some time for device to become configured before proceeding with booting. However, if a device is marked as optional, networkd will not wait for it. This is only supported by networkd, and the default is false.
아래와 같은 식으로 변경하였습니다.
network:
version: 2
renderer: networkd
ethernets:
eno2:
dhcp4: no
optional: true
eno3:
dhcp4: no
optional: true
enp5s0f0:
dhcp4: no
optional: true
enp5s0f1:
dhcp4: no
optional: true
이렇게 할 경우 부팅할 때 케이블 연결이 되었는지 아닌지, DHCP로부터 IP주소를 받는지, 아닌지를 확인하지 않고 바로 IP설정을 마무리합니다. 시험을 해보니까 잘 동작하네요.
