Add a fix argument in map - Python

Question

I am trying to iterate on a method and passing others arguments that the method needs. To do so, I think my best shot is to use map() method.

More precisely, I need performs several dns requests (A / AAAA / NS / MX) on several FQDN (Fully Qualified Domain Name, ie: www.mywebsite.com).

As requests can wait few seconds waiting for an answer, I am using threads to compensate.

Here is my actual code :

with ThreadPoolExecutor(max_workers=4) as executor:
        for fqdn in fqdn_list:
            future_01 = executor.submit(get_dns_query, fqdn, 'A')
            future_02 = executor.submit(get_dns_query, fqdn, 'AAAA')

            ip = future_01.result()
            ipv6 = future_02.result()

Now what I think is best:

with ThreadPoolExecutor(max_workers=4) as executor:
        ip = executor.map(get_dns_query, fqdn, 'A')
        ipv6 = executor.submit(get_dns_query, fqdn, 'AAAA')

get_dns_query will perform a dns request and requires FQDN and RDTYPE (A / AAAA / NS / MX)

I want map() to iterate on FQDN and I also want to be able to pass RDTYPE as a normal argument.

So far, I understood that it is not the purpose of map. But solutions seems to exist: Functools or Itertools solutions

Unfortunately, I am not able to understand them and if they are applicable to my case.

Any ideas?

DaviLevi · Answer 1 · 2020-05-07T13:04:57.507

0

I think this solution could work for you :

import functools


fqdn_list = ['www.google.com', 'www.facebook.com']

with ThreadPoolExecutor(max_workers=4) as executor:
        ips = executor.map(functools.partial(get_dns_query, rdtype_freezed_arg='A'), fqdn_list)
        ipsv6 = executor.map(functools.partial(get_dns_query, rdtype_freezed_arg='AAAA'), fqdn_list)

## SOME CODE

for ip in ips:
    print(ip)


for ipv6 in ipv6s:
    print(ipv6)

What i did is : 'freezing' part of get_dns_query function that i imagine take 2 arguments

def get_dns_query(rdtype, fqdn):
# lines of code

mapping it along a list of dynamic args 'fqdn_list' So it will behave like this :

get_dns_query(rdtype_freezed_arg='A', fqdn_list[0])
get_dns_query(rdtype_freezed_arg='A', fqdn_list[1])
# and so on ...

then i collect the results (n results, defined by fqdn_list.size()) on generator ips and ipv6s

and finally i iterate over them.

Observation : I didn't test it.

EDIT : I used 'rdtype_freezed_arg' as the arg name for didactical purpose, you should use the name of the arg used in the function declaration.

edited May 07 '20 at 13:04

answered May 07 '20 at 12:42

DaviLevi

38
1
6

@steackfrite My answer helped you? If possible, give me a feedback in the comments please. – DaviLevi May 07 '20 at 14:09
Thanks man, that's exactly what I want. The result send by executor.map() is: generator object Executor.map..result_iterator at 0x7f9286451ad0>. I should be able to iterate on it with a for loop? (As you pointed in your response). – steackfrite May 07 '20 at 14:25
I think you should be able to do that with the syntax for generator_item in generator_object: # some code here Could you please test it and report me if it's correct? In that case, please rate my answer as useful/solver. Thanks man – DaviLevi May 07 '20 at 14:28
It worked well! Thanks again (rating should come after moderation, seems like new users actions are oversee for a time) – steackfrite May 07 '20 at 15:19

Add a fix argument in map - Python

1 Answers1