How such code executes in parallel despite GIL?

Question

It's all about Python2.7

I have next question:

from futures import ThreadPoolExecutor

def test():

    while True:

        i = 4


executor = ThreadPoolExecutor(max_workers=2)

executor.submit(test)

executor.submit(test)

So when I run such code on dualcore machine, I get both of cores get busy Why does it happens, shouldn't they execute using just half of resourses, because of GIL? I think they use some system calls and thats why executes in parallel, but I can't understand which part of code executes on system core.

here is disassabmled:

dis.dis(main)
Disassembly of test:
2         0 SETUP_LOOP              16 (to 19)
    >>    3 LOAD_GLOBAL              0 (True)
          6 POP_JUMP_IF_FALSE       18

3         9 LOAD_CONST               1 (4)
         12 STORE_FAST               0 (i)
         15 JUMP_ABSOLUTE            3
    >>   18 POP_BLOCK           
    >>   19 LOAD_CONST               0 (None)
         22 RETURN_VALUE

Also should be noted, that when I run ths code, I can't stop program by typing Ctr+C.

I think its happend because when I have 2 cores, and 3 threads (1 main and 2 threadpool) After typing Ctr+C interpreter execute check after each tick and though ThreadPool threads are cpu bound and have low priority main thread can't accuire GIL because on another core on of ThreadPool thread catch GIL faster

But in my understanding, sometime main thread should aquire GIL and stop program, but it not happend during 5 min.

Whats going wrong?

because GIL works on I/O, insert print() / file / network I/O. p.s. http://stackoverflow.com/questions/29270818/why-is-a-python-i-o-bound-task-not-blocked-by-the-gil — strangeqargo, May 23 '16 at 18:45
Unable to reproduce. The script uses only one core both on linux-py2.7.6 and on win7-py2.7.11. Ctrl-C works for me on both systems too. — robyschek, May 23 '16 at 19:12
@strangeqargo, but I do not see here any I/O etc. operations. What do you mean? — Igor, May 23 '16 at 19:48
@robyschek, it's very interesting. I use linux Python2.7.6 futures 3.0.5 First try to exit from Python interpreter by typing Ctr+D can you do this? What version futures do you use? How many cores do you have? If you have more than 2 cores, disable all except 2 and try again? I'm looking forward to hearing from you — Igor, May 23 '16 at 19:52
@Igor I mean, it executes in parallel without introducing GIL because it does not have any input/output related issues — strangeqargo, May 23 '16 at 19:58
@strangeqargo, I think you miss something: GIL is aquired to execute python code and, when task perform I/O GIL, is relased — Igor, May 23 '16 at 20:01
I check it on python 3.5.1 linux and it use exeactly 100% cpu (1 core). And it is interruptable. @robyschek can you post stacktrace after Ctr+C? — Igor, May 23 '16 at 20:13
I ran the script in console like this: `python the_script.py`. Now I've tryed to run its lines interactively with the following result: CPU usage: 1 core, Ctrl-C, Ctrl-D - not working. I use the same version of `futures`, I run linux on 2 cores and python process shows cpu usage of about 60%. — robyschek, May 23 '16 at 20:17
@Igor you know that default implementation of python 2.7 IS cpython? If you're using other version (PyPy, or IronPython) you should mention it. If you're using default system python/downloaded it from python website, you're using cpython. Also, see this http://stackoverflow.com/a/37373471/5006740 about processor load — strangeqargo, May 23 '16 at 20:23
@Igor It prints [two stacktraces](https://gist.github.com/anonymous/c12c7aeaafbab26b301bddcaa736d788) — robyschek, May 23 '16 at 20:28
@robyschek, thanks for traces. Was I clearly understood you: if you run as `python the_script.py` then you can stop execution by Ctr+C. But if you run interactively, then you can't exit from interpretator by typing Ctr+D? 60% of cpu you mean 60% of one core or 120%? — Igor, May 23 '16 at 20:46
@strangeqargo, Thanks. I use default python. may be I miss something. I'll read about it. — Igor, May 23 '16 at 20:51
@Igor, yea, exactly. When I said 60% cpu, I meant 60% of two cores. `top` command shows me 120% 100% eats python limited by GIL and 20% ,I guess, the task switching,. In absence of GIL it'd be 200%. Py3 shows even less - 100%. — robyschek, May 23 '16 at 20:52
Ok, I got it. I do not think that this 20% of CPU is about task swithching, because on python3 I get exactly100% in top (only one core) load, and task switching executes when python threads are sleeping. — Igor, May 23 '16 at 20:57
@Igor, I meant context switching, and it easily able to eat 20% as it tryes by default to switch every [100 ticks](https://docs.python.org/2/library/sys.html#sys.setcheckinterval), (a tick is roughly python VM's instructions). When I insert `sys.setcheckinterval(1000000)` in the script the cpu usage drops from 120% to 100%. — robyschek, May 23 '16 at 21:21

How such code executes in parallel despite GIL?

0 Answers0