By Jun Chen Edward and P K Tsang
Preface
Our book is an attempt to push forward in the field of financial analysis, using new ways to engage with financial data, under our chosen method of Directional Change, and harnessing some of the cutting-edge tools of machine learning and the related algorithmic trading.
One of the questions we had to frame was, ‘what is time?’. Einstein suggested that ‘Time has no independent existence apart from the order of events by which we measure it’ (The Universe & Dr Einstein by Lincoln Barnett). If no one buys and sells in the market, or the price never changes, whether one takes a daily, hourly or minute approach, time series as a concept does not matter. Time series is only useful if it records price changes. And if that is the case, then why don’t we simply record only significant price changes in the market? That is the basic concept behind Directional Change – that – only significant price movements are recorded – and that is the framework that this work is built on.
The idea of Directional Change was known as ‘zigzag’ in technical analysis, because of the pattern it made. Thus, this book does cover aspects of technical analysis. But the approach taken in this book does not make assumptions about trading behaviour in the market. It is solely a data-driven approach: it lets the data tell us what is happening in the market and also indicates the importance of external events to market operation, for the market does not exist in isolation.
The authors have been privileged to have learned about the concept of Directional Change through the pioneering work of Richard Olsen. In high-frequency finance and in Directional Change, Olsen is decades ahead of his time. We are grateful to him for writing for this book such a fascinating Foreword, explaining some of the complexities of Directional Change that have fascinated him. It was Richard Olsen who recognized Mandelbrot’s fractal characteristics of directional changes: that some properties remain constant when the market is observed under different scales. Although the concept of zigzag existed in technical analysis, it has never before been studied as a scientific, data-centric concept, as Olsen and we have attempted to do.
‘Knowledge is power’, said Elizabethan philosopher Sir Francis Bacon. Data scientists believe that knowledge can be acquired through information, where information can be extracted from data. Directional Change provides an alternative view to financial data, and, as Richard Olsen has pointed out in his Foreword, it provides an alternative way to extract information from it. The most significant finding was probably the power laws discovered by Olsen and his team, which could not have been observed under time series.
This book is also about machine learning and algorithmic trading. Machine learning attempts to find regularities in the market. Such regularities would only help trading if they persisted in the market. However, sometimes the traders collectively behave differently; this is known as regime change in the market. When that happens, traders must be able to react accordingly; for example, by closing their positions. For example, data suggests that the regime changed in the currency market around the time when the Brexit referendum took place in the UK. This book proposes an effective direction towards monitoring regime changes in the market.
The authors have also benefited from interaction with many collaborators; in particular, Alex Dupuis, Raju Venkata Chinthalapati, Antoaneta Serguieva, Wing Lon Ng, Steve Phelps, John O’Hara, David Norman, Michael Kampouridis, Yi Cao, Shaimaa Masry, Monira Aloud, Ao Han, Amer Bakhach, Hamid Jalalian, Tao Ran, Chen Chen, Ma Shuai, Alan Ye, Gao Jing, Shengnan Li, and Shicheng Hu. In fact, all the students that we have taught have helped to sharpen our concept on this topic. We are also grateful to Michael Lung, Fay Somerville and Huiling Liu who brought this team together. We would like to express our special thanks to Sara Colquhoun, who started as our English coach but, through dedication and studiousness, has since made herself an expert in Directional Change.
Our work owes much to our colleagues in this field, and we very much hope we have added to the field of research as well.
Introduction
This book is about data analysis in finance. What useful information could one extract from data? To allow us to go into great depth in this book, we focus on information about regime changes, which means changes in the collective behaviour of the traders in the market. Being able to recognize regime changes in the market is important for traders and regulators. This book starts by asking “what are the data telling us about the market”. Then it explains how the information extracted from the data could help us monitor the market, to see whether it has entered a different regime. Then, as a proof of concept, it explains how a trader could benefit from such information. We shall explain that both knowledge representation and machine learning (two important branches of Artificial Intelligence (AI)) play important roles in information extraction. How one represents knowledge determines how one could reason about it. Instead of using time series to summarise price changes in the market, this book takes the Directional Change (DC) approach. In time series, one samples a transaction price at fixed intervals, for example, daily closing prices. Directional change is a data-driven approach. It lets data tell us when to sample a transaction price. This will be explained in details in Chapter 2. By looking at price changes from a different angle, we are able to extract new information from data. Such new information complements what we observe under time series. We shall show that being able to see with two eyes (time series and directional change) is better than seeing with one (time series alone).