Gaussian Elimination should be plenty fast, so perhaps the issue is how you are implementing it. We want to solve

$$\begin{bmatrix}
-1 & 0 & 0 &\cdots & 0 & a_0\\
1 & -1 & 0 &\cdots & 0 & a_1\\
0 & 1 & -1 &\cdots & 0 & a_2\\
\vdots & \vdots & \vdots &\ddots & \vdots & \vdots\\
0 & 0 & 0 &\cdots & -1 & a_{n-1}\\
0 & 0 & 0 &\cdots & 1 & a_n-1\\
\end{bmatrix}\begin{bmatrix}x_0 \\ x_1 \\ x_2 \\ \vdots \\ x_{n-1} \\ x_n\end{bmatrix} =\begin{bmatrix}b_0 \\ b_1 \\ b_2 \\ \vdots \\ b_{n-1} \\ b_n\end{bmatrix}$$

By applying $R_{i+1} \leftarrow R_{i+1} + R_i$ for $i=0, 1, \ \ldots \ , n-1$ we have

$$\begin{bmatrix}
-1 & 0 & 0 &\cdots & 0 & \alpha_0\\
0 & -1 & 0 &\cdots & 0 & \alpha_1\\
0 & 0 & -1 &\cdots & 0 & \alpha_2\\
\vdots & \vdots & \vdots &\ddots & \vdots & \vdots\\
0 & 0 & 0 &\cdots & -1 & \alpha_{n-1}\\
0 & 0 & 0 &\cdots & 0 & \alpha_n-1\\
\end{bmatrix}\begin{bmatrix}x_0 \\ x_1 \\ x_2 \\ \vdots \\ x_{n-1} \\ x_n\end{bmatrix} =\begin{bmatrix}\beta_0 \\ \beta_1 \\ \beta_2 \\ \vdots \\ \beta_{n-1} \\ \beta_n\end{bmatrix}$$

where $\alpha_k = \sum_{i=0}^k a_i$ and $\beta_k = \sum_{i=0}^k b_i.$ Make sure to compute the $\alpha_i, \beta_i$ recursively so that you only use $2n$ additions (instead of computing each naively, which would use about $n^2$ additions).

Now we do back substitution. First we have $x_n = \beta_n (\alpha_n-1)^{-1}$ which costs $1$ division. Then for $i=0, 1, 2, \ \ldots, n-1$ we have $x_i = \alpha_i x_n - \beta_i,$ which takes $n$ multiplications and $n$ subtractions to compute.