Is there a way to get Pandas ewm to function on fixed windows?

Question

I am trying to use Pandas ewm function to calculating exponentially weighted moving averages. However i've noticed that information seems to carry through your entire time series. What this means is that every data point's MA is dependant on a different number of previous data points. Therefore the ewm function at every data point is mathematically different.

I think some here had a similar question

Does Pandas calculate ewm wrong?

But i did try their method, and i am not getting functionality i want.

 def EMA(arr, window):
     sma = arr.rolling(window=window, min_periods=window).mean()[:window]
     rest = arr[window:]
     return pd.concat([sma, rest]).ewm(com=window, adjust=False).mean()


 a = pd.DataFrame([x for x in range(100)])
 print(list(EMA(a, 10)[0])[-1])
 print(list(EMA(a[50:], 10)[0])[-1])

In this example, i have an array of 1 through 100. I calculate moving averages on this array, and array of 50-100. The last moving average should be the same, since i am using only a window of 10. But when i run this code i get two different values, indicating that ewm is indeed dependent on the entire series.

Quang Hoang · Accepted Answer · 2019-09-19T16:12:51.030

2

IIUC, you are asking for ewm in a rolling window, which means, every 10 rows return a single number. If that is the case, then we can use a stride trick:

Edit: update function works on series only

def EMA(arr, window=10, alpha=0.5):
    ret = pd.Series(index=arr.index, name=arr.name)

    arr=np.array(arr)
    l = len(arr)
    stride = arr.strides[0]

    ret.iloc[window-1:] = (pd.DataFrame(np.lib.stride_tricks.as_strided(arr, 
                                                                       (l-window+1,window), 
                                                                       (stride,stride)))
                          .T.ewm(alpha)
                          .mean()
                          .iloc[-1]
                          .values
                           )
    return ret

Test:

a = pd.Series([x for x in range(100)])

EMA(a).tail(2)
# 98    97.500169
# 99    98.500169
# Name: 9, dtype: float64

EMA(a[:50]).tail(2)
# 98    97.500169
# 99    98.500169
# Name: 9, dtype: float64

EMA(a, 2).tail(2)
98    97.75
99    98.75
dtype: float64

Test on random data:

a = pd.Series(np.random.uniform(0,1,10000))
fig, ax = plt.subplots(figsize=(12,6))
a.plot(ax=ax)
EMA(a,alpha=0.99, window=2).plot(ax=ax)
EMA(a,alpha=0.99, window=1500).plot(ax=ax)

plt.show()

Output: we can see that the larger window (green) is less volatile than the smaller window (orange).

edited Sep 19 '19 at 16:12

answered Sep 19 '19 at 14:13

Quang Hoang

117,517
10
34
52

I'm actually not quite sure if this is what i'm looking for. Adjusting the window looks to just offset the data, it doesn't increase the amount of data points that a single moving average takes into account. – learningthemachine Sep 19 '19 at 15:03
Are you sure? `EMA(a,window=2).tail(10)` gives: `98 98.75` and `90 90.75` – Quang Hoang Sep 19 '19 at 15:05
Adjusting the window should smooth the data but it looks to just offset it. If i do a shift on the data, i see that it lines up. – learningthemachine Sep 19 '19 at 15:29
Your data is linear, moving average over a fixed window would just shift it. Try the code on `a = pd.DataFrame(np.random.uniform(0,3,100))`. – Quang Hoang Sep 19 '19 at 15:40
@learningthemachine see update. Note that the function now works for series only. – Quang Hoang Sep 19 '19 at 15:56
I think the volatility is only being reduced by the alpha coefficient. If you just change the window size i get the same results. – learningthemachine Sep 19 '19 at 16:11
Actually, the included picture uses `alpha=0.99` for both windows. Updated. – Quang Hoang Sep 19 '19 at 16:12
That does indeed look to be working as i wanted it to at first glance. I'll have to do some more testing but it should be good. Thanks! – learningthemachine Sep 19 '19 at 16:27

Is there a way to get Pandas ewm to function on fixed windows?

1 Answers1

Linked