After switching to react-router 4.0 I dumbly wasted way too much time trying to figure out how to get and set the location.search string to a regular query object.
Even after finally realizing they're not handling query params anymore (Parse Query Parameters in React Router v4) and turning to URLSearchParams I then couldn't remember how to easily extract an object from an iterator. Answer below. Hope it helps someone.