Python: "long int too large to convert to float" when checking (really big) prime numbers

Question

I have tried to implement the first is_prime-function from this answer: https://stackoverflow.com/a/17298131/6208280

# for large numbers, xrange will throw an error.
# OverflowError: Python int too large to convert to C long
# to get over this:

def mrange(start, stop, step):
    while start < stop:
        yield start
        start += step

# benchmarked on an old single-core system with 2GB RAM.

from math import sqrt

def is_prime(num):
    if num == 2:
        return True
    if (num < 2) or (num % 2 == 0):
        return False
    return all(num % i for i in mrange(3, int(sqrt(num)) + 1, 2))

Yet I have a little probleme while testing high numbers.

For really high numbers, I get an overflow error: long int too large to convert to float

I checked the max for float:

sys.float_info.max
1.7976931348623157e+308

for is_prime(10**308) everything works fine... but for example with is_prime(10**309) there will be this overflow error (because of the float max?).

Does that mean 1.7976931348623157e+308 is the limit for this kind of is_prime()-function or is there any solution to check higher numbers with the is_prime()-function?

In my mind such solutions as "use decimal" will not really solve the problem, because of the lack of precision a prime number checking function will need?

It means that `(sys.float_info.max - ϵ) ** 2` is the limit, as you're calling `sqrt(num)`. There are "integer square root algorithms" that side step any use of floats, maybe like [this one](https://stackoverflow.com/a/15391420/736937) — jedwards, May 01 '18 at 20:02
Have you thought about how many operations would be required to check that a 308-digit number is prime, and how long that would take? Your function is already going to be way too slow for 20-digit numbers, let alone 300-digit ones. — Mark Dickinson, May 01 '18 at 20:17

abarnert · Answer 1 · 2018-05-01T20:18:21.443

If the only reason you need floats is to be able to do int(sqrt(num)), you can find a reasonably efficient intsqrt function and use that instead. See this question, and the blog post linked from the accepted answer, for some ideas.

Or you can just change your code so that it doesn't need to sqrt at all. For example, instead of a range that ends with int(sqrt(num))+1, you can use a takewhile that tests lambda i: i*i <= num. Or, since you've already got that mrange function:

def sqrmrange(start, sqrstop):
    while start*start < sqrstop:
        yield start
        start += step

Or, if you don't need it to be a one-liner, you can probably write something less abstract (and maybe more efficient?) than either of those. (But really, any intsqrt could well be faster than the best possible **2 test, because you only have to do the intsqrt once, at the start of the loop, rather than every time through the loop.)

Or, if you really want to keep things structured this way, you can just use decimal. You say:

In my mind such solutions as "use decimal" will not really solve the problem, because of the lack of precision a prime number checking function will need?

But that's not right. Using float means you're inherently limited to 52 bits of precision, which if a problem long before you get to overflowing. Using decimal means you get as many digits of precision as you ask for—and even the default 30 digits is already much more than 52 bits. For example:

>>> float(10**20) == float(10**20)+1
True
>>> Decimal(10**20) == Decimal(10**20)+1
False

(In fact, since you're constructing that Decimal from a huge int, it's going to automatically expand to track as many digits as are needed for the int… but you do still need to set the precision before calling sqrt on it.)

It can be complicated to programmatically calculate and set the desired precision for an operation, but in this case, it's pretty simple. The bigger problem is that really big Decimals are a lot slower than really big ints.

Python: "long int too large to convert to float" when checking (really big) prime numbers

1 Answers1