Why do we shift() to get future returns / fill positions?


I’ve bee going through the sample code and the one thing I can’t intuitively understand is the nature of the shift() function on the DataFrame when we are calculating gross returns or filling positions. I looked up what shift does and it looks like the equivalent of zero-padding in a convolutional neural network, just curious why its needed to fill positions and calculate returns specifically though. What if you didn’t shift()?


shift() moves the values down a row and is simply about properly aligning your DataFrames according to what you’re trying to calculate. A simple example: if you want to know the size of an opening gap, you want opens - closes.shift(), which means the current period’s open minus the prior period’s close. In contrast, opens - closes is the current period’s open minus the current period’s close.

Whether to shift depends entirely on what you’re trying to model. In the case of modeling fills, if you generate signals based on a closing price and enter the following day, you probably want to shift your signals to get your positions. If you enter the same session as the signal, you probably don’t want to shift.

Try making a toy DataFrame of signals and derived positions and returns and see how it all lines up. Do it for a strategy that enters the next day and then for one that enters the same day.


Awesome, great explanation. Thank you Brian!