1.
월스트리트 회사들은 오래 전부터 AI에 관심을 가져왔습니다. ChatGPT와 같은 기술은 금융업무 전반에서 영향력을 넓혀가지만 트레이딩과 관련한 업무는 Reinforcement Learning(강화학습)기반으로 이루어집니다. 이와 관련하여 가장 유명한 시스템이 JPMorgan이 Deep Hedging입니다.
JP Morgan의 Deep Hedging
JP Morgan의 Deep Hedging 둘째
Deep Hedging – Learning to Trade
Deep Hedging외 다른 시스템들이 있을 듯 하지만 글로 나타난 시스템은 없었습니다. Evident Brief를 읽다가 2020년에 가동한 Aiden을 알았습니다. 아래는 Aiden과 관련한 자료들입니다.
우선 Aiden은 캐나다 RBC Capitals이 운영하고 있습니다. 이론적인 기반은 RBC Borealis라는 회사가 맡았습니다. RBC Borealis가 Aiden과 관계하여 발표한 논문이 Improving Reinforcement Learning with Human Input입니다. 논문을 작성한 Mattew E. Taylor는 2018년 Borealis AI에서 근무하다고 현재는 대학교수를 재직중인 듯 합니다. RL과 관련한 연구를 계속하고 있습니다.
2.
2022년에 발간한 Applications of Machine Learning in Wealth Management를 보면 자산운용부문에서 기계학습기술을 어떻게 적용할 수 있는지 자세히 소개합니다.
위 논문에 소개한 Aiden입니다.
Many artificial intelligence-based electronic trading platforms,such as Aiden developed by RBC Capital Markets, use the computational power of deep reinforcement learning (RL) to improve trading results. The black box of those real-time trading systems remains mysterious to the public, but academic researchers have also started employing deep learning in high frequency trading. Nevmyvaka et al. (2006) is an early large-scale application of RL to optimize trading execution in the financial markets, which experimentally demonstrates that RL approaches are well-suited for optimized execution. Deep Q-learning networks also have been applied to optimal trade
execution (Ning et al. 2018; Théate and Ernst 2021). Imitative learning deep RL techniques, which are taught by an intelligent trading agent, can help in quantitative trading with balance between exploration and exploitation (Liu, Liu et al. 2020). Kolm et al. (2021) forecast high-frequency returns by training off-the-shelf artificial neural networks on order flow at the most granular level.
Aiden – Reinforcement learning for order execution이 Aiden을 자세히 소개합니다. 논문으로 발표할 수도 있지만 HTML형태로 자세한 내용을 소개하고 있습니다. 시장가주문,지정가주문,시장충격, 알고리즘(VWAP, Arrival)과 같은 익숙한 단어가 등장합니다. 주문집행시스템이니까 당연합니다만.
Aiden setup
In this section we describe the main features of the Aiden reinforcement learning setup: the action space, the state and the reward functions. In the subsequent section we discuss the reinforcement learning algorithm itself.Action
In practice Aiden does not directly select the details of the order that is provided to Aiden, but instead chooses between different high-level actions at each time step that correspond to different levels of aggressiveness as Aiden begins to liquidate the parent order using child orders. These range from crossing the spread (and so immediately executing some of the order) at one end of the spectrum to doing nothing / removing existing orders at the other. These actions form the input to a system that translates them into concrete limit orders.State
Aiden’s state is currently composed of several hundred market features and self-aware features. The market features comprise of hand-crafted functions that compute quantities of interest from the market micro-structure data. Examples might include measurements of the liquidity, recent price changes, or whether there is an imbalance between the bid and ask volumes. The self-aware features relate to the history of previous actions that Aiden has taken. For example, they might include measurements of how aggressive Aiden has been in recent time steps, and how many shares Aiden still has to execute.Rewards
The rewards are chosen so that Aiden optimizes around a core trading objective, such as a benchmark. One such commonly used benchmark to measure performance is the volume-weighted average price (VWAP) of the market for the asset over the whole period. As the name suggests, this is the average price of all transactions in the limit order book, weighted by volume. Consequently, rewards are designed based on the difference between this market VWAP and the actual prices Aiden achieved. Of course, Aiden will not know the market VWAP price until the end of the period and so as is typical in reinforcement learning, the feedback is delayed.
참고로 Aiden은 특허등록을 한 시스템입니다.
Trade platform with reinforcement learning
3.
마지막으로 RBC Borealis가 기계학습을 학습하고자 하는 분들을 위한 안내서를 올렸습니다. 참고하세요.
Top 20 Tutorials for Machine Learning (ML)
한국 명동과 월스트리트를 보면 결정적인 차이가 기술적, 이론적 토대의 차이가 확연합니다. R&D에 대한 투자, 사람에 대한 투자가 차이를 만드는 듯 합니다.
한국 금융회사의 오랜 관행. 기술과 사람을 빌려서 시스템을 개발하고 이익만을 취하는 식의 전략. 장기적으로 보면 경쟁력으로 이어지지 않습니다. 명동 혹은 여의도에서 있는 금융회사 관계자가 AI와 관련한 논문을 발표하는 날을 기대해봅니다.
