100 result limit from Splunk API call

Question

I should be getting around 1000 results, but it this is only returning 100 of them into the output file.

g = requests.get(base_url + '/services/search/jobs/' + sid + '/results/',

headers = {'Authorization':('Splunk %s' %session_key)},data={'output_mode': 'json', 'count':'0'},  verify = False)

data = g.json()

names = [item['name'] for item in data['results']]

with open ('sOutput.csv', mode='w') as csv_file:
        csv_writer = csv.writer(csv_file,  delimiter='\n', quotechar='"', quoting=csv.QUOTE_MINIMAL)
        csv_writer.writerow(names)

score 1 · Accepted Answer · answered Aug 06 '20 at 22:01

In the HTTP spec, GET requests don't contain a request body (relevant Stack Overflow thread), which is what you're define with the data parameter of the requests.get() method (relevant Real Python blog post explaining this). Splunk's API documentation also specifically refers to these options as "parameters for [...] GET methods".

Use request.get()'s params argument instead to correctly pass these options along to the server:

g = requests.get(base_url + '/services/search/jobs/' + sid + '/results/',

headers = {'Authorization':('Splunk %s' %session_key)},params={'output_mode': 'json', 'count':'0'},  verify = False)

data = g.json()

names = [item['name'] for item in data['results']]

with open ('sOutput.csv', mode='w') as csv_file:
        csv_writer = csv.writer(csv_file,  delimiter='\n', quotechar='"', quoting=csv.QUOTE_MINIMAL)
        csv_writer.writerow(names)

As an aside, you may find it useful to familiarize yourself with the Requests documentation, more specifically with the Passing Parameters in URLs section.

100 result limit from Splunk API call

1 Answers1