Concurrent read/write to shared variables in OpenMp

Question

I have basically three questions on OpenMp.

Q1. Does OpenMp provide mutual exclusion to shared variables? Consider the following simple matrix multiplication code with three nested loops, parallelised using OpenMp in C++. Here A, B, and C are dynamically space allocated double** type of variables. Thread count is appropriately assigned a value.

#pragma omp parallel
{
 int tid = omp_get_thread_num();
 int fraction = (n/threadCount);
 int start = tid * fraction;
 int end = (tid+1) * fraction;

    for (int start = 0; i < end; i++)
    {
        for (int j = 0; j < N; j++)
        {
            C[i][j] = 0;

            for (int k = 0; k < N; k++)
                C[i][j] += A[i][k] * B[k][j];
        }
     }
}

The thing here is that mutual exclusion for reading from A and B and writing to C is unnecessary. But if extra overhead is incurred due to mutex on A, B, and C, it is favourable to relieve A, B, and C of mutex. How can it be achieved?

Q2. Consider introducing two private variables tempA and tempB into the above code as follows.

double **tempA, **tempB;

#pragma omp parallel private(tempA, tempB)
{
 int tid = omp_get_thread_num();
 int fraction = (n/threadCount);
 int start = tid * fraction;
 int end = (tid+1) * fraction;
 tempA = A;
 tempB = B;

    for (int start = 0; i < end; i++)
    {
        for (int j = 0; j < N; j++)
        {
            C[i][j] = 0;

            for (int k = 0; k < N; k++)
                C[i][j] += tempA[i][k] * tempB[k][j];
        }
     }
}

Would this strategy relieve A and B from mutex in calculations? I mean, although the same locations (referred by A and tempA, and B and tempB) are accessed by all threads, they refer them through different local variables.

Q3. Also, I would like to know about the difference in declaring the variables tempA and tempB inside the parallel code segment, instead of declaring them outside. Of course, then we won't need that private clause in the directive. Is there any other significant difference.

(1) Why do you do worksharing by hand instead of using `#pragma omp for`? (2) To synchronize reads and writes to shared variables there is [`#pragma omp atomic`](https://www.google.com/search?q=pragma+omp+atomic). (3) Don't reinvent the wheel, use [Blaze](https://bitbucket.org/blaze-lib/blaze). This linear algebra library implements OpenMP parallelized and SIMD vectorized matrix operations. It can't get any faster than this. — Henri Menke, Jun 22 '17 at 06:10

k.v. · Accepted Answer · 2017-06-22T07:08:59.730

2

By default no synchronization mechanisms are provided. But OpenMP provides a possibility to explicitly use such mechanisms. Use #pragma omp atomic, #pragma omp atomic read, #pragma omp atomic write for such purposes. Another option to use critical section: #pragma omp critical - more generic and powerful option, but not always required.
Accessing same memory location through different variables does not change anything regarding concurrent access. You should use atomics to provide guarantee.
If declaring variable inside pragma omp parallel - they will be private for a thread. See this and this posts for more information.

Also if you are using C++11, you can use std::atomic variables.

edited Jun 22 '17 at 07:08

answered Jun 22 '17 at 06:14

k.v.

1,143
8
11

1.What I want to know is whether in the sample code I posted, the reads from A, B and writes to C, are provided with mutex by default. If yes is there any way to relieve it from mutex? – Melanka Jun 22 '17 at 06:57
@Melanka no, mutex is *not* provided by default. You should explicitly provide them. It would be unreasonable from the performance point of view to provide synchronization mechanisms by default. – k.v. Jun 22 '17 at 06:59
3. Yes it would be private. But is there any difference in making it private by declaring inside when compared to declaring it outside and then adding private(varName) in the directive? – Melanka Jun 22 '17 at 06:59
@Melanka 3. - [this](https://stackoverflow.com/a/30026383/2466572) and [this](https://stackoverflow.com/a/11002550/2466572) posts probably are what are you looking for – k.v. Jun 22 '17 at 07:04
Thank you for the above links to previous posts. Yes that was what I was looking for exactly. Got it. – Melanka Jun 22 '17 at 07:12
@Melanka you are welcome. I've update an answer according to your clarifications. – k.v. Jun 22 '17 at 07:13
1

Take care with C11/C++11 atomics. If I remember correctly, mixing them with OpenMP's was not supported by some compilers. [Related question](https://stackoverflow.com/questions/41309299/mixing-c11-atomics-and-openmp). – Jorge Bellon Jun 22 '17 at 07:58

Concurrent read/write to shared variables in OpenMp

1 Answers1