Eliminate an Element from an AWS CLI Response

Question

NOTE: I'm actually attempting this in Python and the Boto3 library, but I'm going to use the AWS CLI to explain what I'm trying to do.

I'm making a list-object-versions call to my bucket in AWS...

aws s3api list-object-versions --bucket averyuniquebucketname --delimiter "/"

The response is like the following...

{
    "DeleteMarkers": [

       ...snip...

    ],
    "CommonPrefixes": [

       ...snip...

    ],
    "Versions": [

       ...snip...

    ]
}

WHAT I would like is to tell AWS not to return any results with DeleteMarkers.

Any ideas as to how I can use the --query and JMESPath to eliminate the DeleteMarkers but preserve CommonPrefixes & Versions in the response?

What is your actual motivation for wanting to eliminate the delete markers from the response? It's a pointless exercise, I suspect, because S3 is still going to send them... but would like to understand your underlying motivation or whether perhaps there is a larger issue that you believe this would be the solution for. — Michael - sqlbot, Dec 15 '18 at 00:18
You can use `--query` to specifically request which fields _should_ be shown as output. (S3 still sends the `DeleteMarkers`, but the CLI excludes them from the result set.) Thus, if you uses boto3, your application would still receive the `DeleteMarkers`. But, your app can just ignore them. — John Rotenstein, Dec 15 '18 at 03:44
When I’m iterating through the object versions some of the responses only include a 1000 deleted versions. I was hoping there would a way to prevent DeleteMarkers from ever being sent saving time and bandwidth. — grbonk, Dec 15 '18 at 18:26

score 0 · Answer 1 · answered Dec 15 '18 at 05:29

As Michael - sqlbot, points out, aws cli --query is something that runs on the client side, so it won't save you anything to 'remove' DeleteMarkers. Nevertheless, you could use --query '{CommonPrefixes: CommonPrefixes, Versions: Versions} from the CLI... but in Python/boto3 it makes much more sense to just look at what you're interested in, or if really keen do a Python del or construct a new dictionary (there's definitely no reason to use jmespath).

Eliminate an Element from an AWS CLI Response

1 Answers1