I have a large file being uploaded on S3, and for each line in the file I need to make a long running rest API call. I'm trying to figure out the best way to break up the work. My current flow idea is
Lambda (break up file by line) -> SNS (notification per line) -> Lambda (separate per line/notification)
This seems like it is a common use case, but I can't find many references to it, am I missing something? Is there a better option to break up my work and get it done in a reasonable amount of time?