$(a+b)^2|2a^3 +6a^2b+1$ iff $k(a^2+2ab+b^2) = 2a^3 +6a^2b+1$ or $-2a^3+a^2(k+6b)+a(2kb)+(kb^2+1)=0$. This is a a cubic equation so one could find $(a,b)$ by solving for $a$, but then one would still have to iterate through the $k$ and $b's$ to find an integer value of $a$ and $b$. This is also an elliptic curve, specifically $y^2 = -2a^3+a^2(k+6b)+a(2kb)+(kb^2+1)$. One could reduce this into Weiertrauss form using the cubic reduction formula to obtain something like $y^2 = a^3 + pa + q$. Anyway, elliptic curves have either finitely many or infinitely many solutions. We are searching for integer values of $a,b$ so we can apply what we know about solving for rational points on elliptic curves. If we are looking for rational points, then the rational points $E(\mathbb{Q})$ form an abelian group and is finitely generated, i.e. there is a finite number of rational points that one can use to find the rest of the points.

Solving your problem is the same as finding rational points on an Elliptic Curve and there are lots of tricks to do that, many of which can be found online. As far as I know, there is no formula for solving elliptic curves, just methods to find solutions. However, your question is even more complicated than that, because two of values in your elliptic curve are undetermined.

This link provides some examples of how to solve Elliptic Curves if your $k,b$ were determined. However, even with them determined, one would still use an algorithm to find the points. http://www.math.brown.edu/~jhs/Presentations/WyomingEllipticCurve.pdf