Reinforcement Learning Neural Network trading Bitcoin. Visualization is roughly around July 2019.

This video is inspired by the countless videos of reinforcement learning / genetic algorithm neural networks playing snake or driving cars in video games. This model operates in a fairly similar way, but financial machine learning is an entirely different animal. Financial data has a very low signal to noise ratio. Because of this, models must have low complexity to avoid over fitting.

The synapses are gray scale colored by the value going into neuron. White at 1, black at -1.

The output of the network is the position taken for the next bar. It can range from -1 to 1. 1 meaning full long position, and -1 meaning a full short position. The cumulative log return is calculated by multiplying the model output by the next log return and summing the product.

The model has 4 inputs, the first 3 are the closing price minus moving averages of periods 6, 24, and 72. The difference is normalized to -1 and 1 by dividing the raw difference by the average true range (ATR) multiplied by the square root of the look back period. Then compressing the result with the normal cumulative distribution function. The forth input is the models output from the prior bar.

The neural net uses the hyperbolic tangent as the activation function. The output is also compressed with the hyperbolic tangent. There are no bias weights in the network. Mainly to reduce the number of parameters and I found little improvement when using bias.

It was trained using a genetic algorithm and artificial bee colony (I found the ABC to be effective at refining what the genetic search found.) It was trained to optimize the Martin Ratio throughout 2018.

The model has decent performance given the simple input features. There is some mild non linearity in the relation of two CMMAs to the future price change, hence the neural net. This model is a little overkill to capture it. I may go over that relation in a video some day.

Music by Karl Casey @ White Bat Audio "Astronomy"

Patreon:   / neurotrader  

This video and the content of this channel is presented for educational/entertainment purposes only. I am not responsible for any losses incurred by any information presented on this channel.