Large examples of strong pseudoprimes to several bases

A strong pseudoprime to base a is a composite number that passes the strong probable prime test (i.e. the Miller-Rabin test) in the base a. There are indications that strong pseudoprimes are rare. According to [6], there are only 4842 numbers below 25 \cdot 10^9 that are strong pseudoprimes to base 2. Strong pseudoprimes to several bases are even rarer. According to [6], there are only 13 numbers below 25 \cdot 10^9 that are strong pseudoprimes to bases 2, 3 and 5. The paper [4] tabulates the numbers below 10^{12} that are strong pseudoprimes to bases 2, 3 and 5. That listing contains only 101 numbers. These examples show that strong pseudoprimes to several bases may be rare but they do exist. In this post we discuss two rather striking examples of large numbers that are strong pseudoprimes to the first 11 prime bases for the first one and to the first 46 prime bases for the second one. One important lesson that can be drawn from these examples is that the implementation of the strong probable prime test must be randomized.

___________________________________________________________________

The strong probable prime test

Given an odd number n whose “prime versus composite” status is not known, set n-1=2^k \cdot Q where k \ge 1 and Q is odd. Then calculate the following sequence of k+1 numbers:

    \displaystyle a^Q, \ a^{2Q}, \ a^{2^2 Q}, \ \cdots, \ a^{2^{k-1} Q}, \ a^{2^{k} Q} \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (1)

where a is some integer relatively prime to n. The first term a^Q can be efficiently calculated using the fast powering algorithm. Each subsequent term is the square of the preceding term. Each term is reduced modulo n.

If a^Q \equiv 1 \ (\text{mod} \ n) (i.e. the first term in sequence (1) is a 1) or a^{2^j \cdot Q} \equiv -1 \ (\text{mod} \ n) for some j=0,1,2,\cdots,k-1 (i.e. the term preceding the first 1 in the sequence is a -1), then n is said to be a strong probable prime to the base a. A strong probable prime to base a could be a prime or could be composite. If the latter, it is said to be a strong pseudoprime to base a. In fact, most strong probable primes to one base are prime.

The strong probable prime test consists of checking whether n is a strong probable prime to several bases. If n is not a strong probable prime to one of the bases, then n is composite for sure. If n is a strong probable prime to all of the bases being used, then n is likely a prime number in that the probability that it is composite is at most 4^{-u} if u is the number of bases.

The last probability of 4^{-u} is what makes the 337-digit number N defined below striking. Here we have a number that is a strong probable prime to 46 bases. What could go wrong in declaring it a prime number? The problem is that using a pre-determined set of bases is not a randomized implementation of the strong probable prime test.

___________________________________________________________________

Examples

The following is a 46-digit found in [3] that is a strong pseudoprime to the first 11 prime bases (the prime numbers from 2 to 31).

    M=
    1195068768795265792518361315725116351898245581

The following is a 337-digit number found in [3] that is a strong pseudoprime to all 46 prime bases up to 200 (the prime numbers from 2 to 199).

    N=
    80383745745363949125707961434194210813883768828755
    81458374889175222974273765333652186502336163960045
    45791504202360320876656996676098728404396540823292
    87387918508691668573282677617710293896977394701670
    82304286871099974399765441448453411558724506334092
    79022275296229414984230688168540432645753401832978
    6111298960644845216191652872597534901

The following sets up the calculation for the Miller-Rabin test (strong probable prime test).

    M-1=2^2 \cdot Q

    N-1=2^2 \cdot R

where Q and R are odd and

    Q=
    298767192198816448129590328931279087974561395

    R=
    20095936436340987281426990358548552703470942207188
    95364593722293805743568441333413046625584040990011
    36447876050590080219164249169024682101099135205823
    21846979627172917143320669404427573474244348675417
    70576071717774993599941360362113352889681126583523
    19755568824057353746057672042135108161438350458244
    6527824740161211304047913218149383725

___________________________________________________________________

Random bases

Though the second number N is very striking, the author of [3] has an even larger example in [2], a 397-digit Carmichael number that is a strong pseudoprime to all the 62 prime bases under 300! One lesson from these examples is that the implementation of the strong probable prime test should be randomized, or at least should include some randomly chosen bases in the testing. Any algorithm that implements the strong probable prime test in any “fixed” way (say, only checking the prime bases up to a certain limit) may incorrectly identify these rare numbers as prime.

Let’s apply the strong probable prime test on the above numbers N and M using some random bases. Consider the following randomly chosen bases b and c where 1<b<M and 1<c<N.

    b=
    932423153687800383671087185189848318498417236

    c=
    23876349986768815408041169070899917334655923628776
    58344592618224528502905948639172375368742187714892
    08287654410018942444630244906406410549094447554821
    42803219639200974486541191341820595453041950723891
    75748815383568979859763861820607576949539863746244
    98291058421101888215044176056586791235119485393994
    789287924346696785645273545040760136

___________________________________________________________________

The calculation using random bases

Here’s the calculation for the 46-digit number M.

    b^Q \equiv t_1 \ (\text{mod} \ M)

    b^{2 \cdot Q} \equiv t_2 \ (\text{mod} \ M)

    b^{M-1}=b^{4 \cdot Q} \equiv t_3 \ (\text{mod} \ M)

where t_1, t_2 and t_3 are:

    t_1=
    1042866890841899880275968177787239559549445173

    t_2=
    826876320842405260107866407865475914456579581

    t_3=
    1195068768795265792518263537659322626328455738

Note that the last number t_3 is not a 1. So the number M is not a strong probable prime to base b. This means that it is composite.

Here’s the calculation for the 337-digit number N.

    c^R \equiv g_1 \ (\text{mod} \ N)

    c^{2 \cdot R} \equiv g_2 \ (\text{mod} \ N)

    c^{N-1}=c^{4 \cdot R} \equiv g_3 \ (\text{mod} \ N)

where g_1, g_2 and g_3 are:

    g_1=
    43050290968654965761881145696359381339174664947842
    93659429396009893693594328847691223585119425166890
    43134041173054778367051375333950357876719375530986
    40705386242996844394887879855798166233504226845778
    76290707027478869178569806270616567220414388766208
    75314254126730991658967210391794715621886266557484
    525788655243561737981785859480518172

    g_2=
    69330060289060873891683879069908453474713549543545
    79381982871766126161928753307995063953670075390874
    26152164526384520359363221849834543643026694082818
    11219470237408138833421506246436132564652734265549
    18153992550152009526926092009346342470917684117296
    93655167027805943247124949639747970357553704408257
    9853042920364675222045446207932076084

    g_3=
    80383745745363949125707961434194210813883768828755
    81458374889175222974273765333652186502336163960045
    45791504202360320876656996676098728404396540823292
    87387918508691668493091034289810372813316104284761
    45244183107779151749589951324358651494383103941607
    35777636976785468267798055151468799251924355070195
    1772723904683953622590747688533861698

Interestingly, the last number g_3 agrees with the modulus N from the first digit (the largest) to digit 167. It is clear that g_3 is clearly not a 1. So the 337-digit number N is not a strong probable prime to base c, meaning it is composite. Even though the number M is a strong pseudoprime to all of the first 46 prime bases, it is easy to tell that it is composite by using just one random base. This calculation demonstrates that it is not likely to make the mistake of identifying large pseudoprimes as prime if randomized bases are used in the strong probable prime test.

___________________________________________________________________

Some verification

We also verify that the 46-digit M is a strong pseudoprime to the first 11 prime bases.

    \left[\begin{array}{rrrrrrr}      a & \text{ } & a^Q \ (\text{mod} \ M) & \text{ }  & a^{2Q} \ (\text{mod} \ M) & \text{ } & a^{4Q} \ (\text{mod} \ M) \\           \text{ } & \text{ } & \text{ }  \\            2 & \text{ } & * & \text{ } & -1 & \text{ } & 1  \\      3 & \text{ } & 1 & \text{ } & 1 & \text{ } & 1  \\      5 & \text{ } & -1 & \text{ } & 1 & \text{ } & 1  \\      7 & \text{ } & * & \text{ } & -1 & \text{ } & 1  \\      11 & \text{ } & * & \text{ } & -1 & \text{ } & 1  \\        13 & \text{ } & * & \text{ } & -1 & \text{ } & 1  \\      17 & \text{ } & * & \text{ } & -1 & \text{ } & 1  \\      19 & \text{ } & * & \text{ } & -1 & \text{ } & 1  \\      23 & \text{ } & * & \text{ } & -1 & \text{ } & 1  \\      29 & \text{ } & * & \text{ } & -1 & \text{ } & 1  \\        31 & \text{ } & * & \text{ } & -1 & \text{ } & 1             \end{array}\right]

The asterisks in the above table mean that those cells have numbers that are \not \equiv \pm 1 modulo M. Clearly the table shows that the number M passes the strong probable prime test for these 11 bases. We also verify that M is not a strong probable prime to base 37, the next prime base.

    37^Q \equiv w_1 \ (\text{mod} \ M)

    37^{2 \cdot Q} \equiv w_2 \ (\text{mod} \ M)

    37^{M-1}=b^{4 \cdot Q} \equiv w_3 \ (\text{mod} \ M)

where

    w_1=
    977597583337476418144488003654858986215112009

    w_2=
    368192447952860532410592685925434163011455842

    w_3=
    1195068768795265792518263537659322626328455738

Note that the last number w_3 agrees with the modulus M for the first 22 digits and they differ in the subsequent digits. It is clear that w_3 is not 1. So the strong pseudoprimality of M stops at the base 37.

We do not verify the number N for all the 46 prime bases. We only show partial verification. The following calculation shows the pseudoprimality of N to bases 197 and 199, and that the strong pseudoprimality of N fails at 211, the next prime base.

    197^R \equiv * \ (\text{mod} \ N)

    197^{2 \cdot R} \equiv -1 \ (\text{mod} \ N)

    197^{N-1}=197^{4 \cdot R} \equiv 1 \ (\text{mod} \ N)

    __________________

    199^R \equiv * \ (\text{mod} \ N)

    199^{2 \cdot R} \equiv -1 \ (\text{mod} \ N)

    199^{N-1}=199^{4 \cdot R} \equiv 1 \ (\text{mod} \ N)

    __________________

    211^R \equiv v_1 \ (\text{mod} \ N)

    211^{2 \cdot R} \equiv v_2 \ (\text{mod} \ N)

    211^{N-1}=211^{4 \cdot R} \equiv 1 \ (\text{mod} \ N)

where

    v_1=
    48799236892584399744277334997653638759429800759183
    35229821626086043683022014304157526246067279138485
    99367014952239377476103536546259094903793421522217
    99291447356172114871135567262925519534270746139465
    22832800077663455346130103616087329184090071367607
    57478119260722231575606816699461642864577323331271
    2974554583992816678859279700007174348

    v_2=
    80191643327899921083661290416909370601037633208226
    50175490124094760064341402392485432446383194439467
    16432633017071633393829046762783433857505596089159
    3600905184063673203

Note that v_2 is clearly not congruent to -1. Thus the number N is not a strong pseudoprime to base 211 (though it is a pseudoprime to base 211). Clearly, the above calculation indicates that the number N is a strong pseudoprime to bases 197 and 199.

___________________________________________________________________

Remark

Because strong pseudoprimes are rare (especially those that are strong pseudoprimes to several bases), they can be used as primality tests. One idea is to use the least pseudoprimes to the first n prime bases [5]. This method is discussed in this previous post.

Another idea is to use the listing of strong pseudoprimes to several bases that are below a bound. Any number below the bound that is strong probable primes to these bases and that is not on the list must be a prime. For example, Table 1 of [4] lists 101 strong pseudoprimes to bases 2, 3 and 5 that are below 10^{12}. This test is fast and easy to use; it requires only three modular exponentiations to determine the primality of numbers less than 10^{12}.

The numbers M and N are not Carmichael numbers since b^{M-1} \not \equiv 1 \ (\text{mod} \ M) and c^{N-1} \not \equiv 1 \ (\text{mod} \ N), making the random numbers b and c Fermat witnesses for these two numbers respectively. Another way to see that they are not Carmichael is that each of the two number M and N is also a product of two distinct primes according to [3].

An even more striking result than the examples of M and N is that it follows from a theorem in [1] that for any finite set of bases, there exist infinitely many Carmichael numbers which are strong pseudoprimes to all the bases in the given finite set. This is discussed in the next post.

___________________________________________________________________

Reference

  1. Alford W., Granville A., and Pomerance C., On the difficulty
    of finding reliable witnesses
    , L. Adleman and M.-D. Huang, editors, Algorithmic Number Theory: Proc. ANTS-I, Ithaca, NY, volume 877 of
    Lecture Notes in Computer Science, pages 1–16. Springer–Verlag, 1994.
  2. Arnault F., Constructing Carmichael numbers which are strong pseudoprimes to several bases, J. Symbolic Computation, 20, 151-161, 1995.
  3. Arnault F., Rabin-Miller primality test: composite numbers that pass it, Math. Comp., Volume 64, No. 209, 355-361, 1995.
  4. Jaeschke G., On strong pseudoprimes to several bases, Math. Comp., Volume 61, No. 204, 915-926, 1993.
  5. Jiang Yupeng, Deng Yingpu, Strong pseudoprimes to the first 9 prime bases, arXiv:1207.0063v1 [math.NT], June 30, 2012.
  6. Pomerance C., Selfridge J. L., Wagstaff, S. S., The pseudoprimes to 25 \cdot 10^9, Math. Comp., Volume 35, No. 151, 1003-1026, 1980.

___________________________________________________________________
\copyright \ \ 2014 \ \text{Dan Ma}

Looking for a good witness

For some primality tests, the approach of proving a number as composite is to produce an auxiliary number that shows that a property of prime numbers is violated. Such an auxiliary number is called a “witness” for compositeness. When using such a primality test on a composite number, for the test to be correct, at least one “witness” must be found. In this post, we compare the Fermat primality test and the Miller-Rabin test from the standpoint of producing “witnesses”. In this regard, the Fermat test works correctly most of the time. But there is a rare but infinite class of composite numbers for which no Fermat test witness can be found. On the other hand, using the Miller-Rabin test on a composite number, at least one witness can be found (in fact, there are lots of them).

___________________________________________________________________

Fermat witnesses

According to Fermat’s little theorem, if n is a prime number, then the following congruence holds for all numbers a that are coprime to n.

    \displaystyle a^{n-1} \equiv 1 \ (\text{mod} \ n) \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (1)

If there exists a number a such that a^{n-1} \not \equiv 1 \ (\text{mod} \ n), then n is proved to be composite, in which case the number a is said to be a Fermat witness for the compositeness of n. If the congruence (1) holds for several values of a, then n is a likely a prime.

Suppose n is a large odd number whose “prime versus composite” status is not known. If the congruence (1) holds for some positive integer a, then n is said to be a probable prime to base a. A probable prime to the base a could be prime or could be composite. If the latter, the number n is said to be a pseudoprime to base a.

This is how the Fermat test (or probable prime test) works. Given that n is a large odd number whose “prime versus composite” status is not known, if n is not a probable prime to one base a, then n is proven to be composite and a is a Fermat witness for n. On the other hand, if n is a probable prime to several randomly selected bases, then n is a probable prime.

Suppose that you apply the Fermat test on the number n and find that n is a probable prime to several bases. There are two possibilities. Either n is prime or is a pseudoprime to the several bases that are being used. Pseudoprimess to one base are rare. Pseudoprimes to several bases at once are even rarer. For example, according to [3], there are only 21853 pseudoprimes to the base 2 that are less than 25 \cdot 10^9. The number of primes below 25 \cdot 10^9 is roughly 10^9. So most probable primes to base 2 are primes. Of the 21853 pseudoprimes to the base 2 that are below 25 \cdot 10^9, 4709 of them are pseudoprimes to the bases 2 and 3, 2522 of them are pseudoprimes to the bases 2, 3 and 5, and 1770 of them are pseudoprimes to the bases 2, 3, 5 and 7. Thus there is indication that the numbers that are pseudoprimes to several bases are very rare. Therefore, the number n being a simultaneous pseudoprime to several bases is very strong evidence that n is prime. But this is not the end of the story for the Fermat test.

The Fermat test can detect the compositeness of most composite numbers. These are the numbers n that are probable prime to one base but are not probable prime to another base. For example, 341 is a probable prime to base 2 since 2^{340} \equiv 1 \ (\text{mod} \ 341). On the other hand, 3^{340} \equiv 56 \ (\text{mod} \ 341). Thus 341 is not a probable prime to base 3 (i.e. 3 is a Fermat witness for the compositeness of 341). For integers like 341, they may be probable primes to one base, but they are not probable primes to another base. If there exists at least one Fermat witness for a composite number, there is a good chance that the Fermat test will detect the compositeness.

A odd composite number n is said to be a Carmichael number if the congruence (1) holds for all numbers a coprime to n. The smallest Carmichael number is 561. There is no danger of mistaking a small Carmichael number such as 561 as prime because factoring is possible. Carmichael numbers are rare but there are infinitely of them. So there are large Carmichael numbers (e.g. with hundreds of digits or thousands of digits). The Fermat test fails completely with these large Carmichael numbers. In other words, no matter how many bases are used, it is impossible to find a number a that will witness the compositeness of a Carmichael number. Unlike the number 341, the Fermat test will fail to detect the compositeness of a Carmichael number.

___________________________________________________________________

Looking for a better type of witness

Let n be an odd number with n-1=2^k \cdot Q where k \ge 1 and Q is odd. Then compute the following sequence of k+1 numbers modulo n:

    \displaystyle a^Q, \ a^{2Q}, \ a^{2^2 Q}, \ \cdots, \ a^{2^{k-1} Q}, \ a^{2^{k} Q} \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (2)

where a is some number coprime to n. The first term in the sequence can be calculated by using the fast powering algorithm. Each subsequent term is the square of the preceding one. Of course, the last term is the same as a^{n-1}, the calculation for (1). Then if this calculation is done on a prime number n, the last term in (2) is always a 1. But more can be said about the sequence (2).

The better witnesses and primality test we discuss here are based on this theorem about prime numbers.

    If n is a prime number, then either one of the following conditions holds:

    • \displaystyle a^{Q} \equiv 1 \ (\text{mod} \ n) \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (2a)
    • \displaystyle a^{2^j \cdot Q} \equiv -1 \ (\text{mod} \ n) for some j=0,1,2,\cdots, k-1 \ \ \ \ \ \ \ \ (2b)

    \text{ }

    for each number a that is coprime to n.

The theorem just mentioned is the basis of the Miller-Rabin test. If there exists some a such that both (2a) and (2b) are not true, then the number n is proved to be composite, in which case the number a is said to be a witness for the compositeness of n. So a witness from the Fermat test standpoint is called Fermat witness. The term witness here refers to the calculation of (2). If the conditions (2a) or (2b) holds for several values of a, then n is likely a prime number.

Suppose n is a large odd number whose “prime versus composite” status is not known. If the conditions (2a) or (2b) holds for some positive integer a, then n is said to be a strong probable prime to base a. A strong probable prime to the base a could be prime or could be composite. If the latter, the number n is said to be a strong pseudoprime to base a.

This is how the Miller-Rabin test (or strong probable prime test) works. Given that n is a large odd number whose “prime versus composite” status is not known, if n is not a strong probable prime to one base a, then n is proved to be composite and the number a is a witness for a. If n is a strong probable prime to several randomly chosen bases, then the probability that n is composite is smaller than 4^{-k} (if k bases are used). In other words, the error probability can be made arbitrarily small by using more random bases in the calculation for sequence (2).

The last point about the small error probability is the most important advantage of the Miller-Rabin test over the Fermat test. Let’s look at a small example first. Recall that 561 is the smallest Carmichael number. Note that 560=2^4 \cdot 35. The following is the calculation for (2).

    2^{35} \equiv 263 \ (\text{mod} \ 561)

    2^{2 \cdot 35} \equiv 166 \ (\text{mod} \ 561)

    2^{4 \cdot 35} \equiv 67 \ (\text{mod} \ 561)

    2^{8 \cdot 35} \equiv 1 \ (\text{mod} \ 561)

    2^{16 \cdot 35} \equiv 1 \ (\text{mod} \ 561)

The number 561 is a probable prime to base 2 (from a Fermat perspective). Notice that the condition (2a) is not met. The condition (2b) is also not met since the term preceding the first 1 is 67, which is not congruent to -1. So the number 2 is a witness for the compositeness of 561. The Fermat test will have virtually no chance of detecting the compositeness of 561. Yet the strong probable prime test takes only one calculation for 561. In this case at least, it is pretty easy to find a witness.

___________________________________________________________________

At least one witness can be found

One characteristic of the strong probable prime test is that there are lots of witnesses when testing composite numbers. In fact, if n is a composite number, over 75% of the possible choices of bases a are witnesses for n. We prove a simpler theorem that if n is composite, at least one witness for the compositeness of n can be found. Even this much weaker theorem is a big improvement over the Fermat test.

Theorem 1
Let n be a composite positive odd number. Then there a number a that is coprime to n such that n is not a strong pseudoprime to base a.

Proof of Theorem 1
Let n=u \cdot v such that u,v are odd and are coprime. Consider the following two congruence equations.

    x \equiv 1 \ (\text{mod} \ u)

    x \equiv -1 \ (\text{mod} \ v)

By the Chinese remainder theorem (CRT), there is an a such that

    a \equiv 1 \ (\text{mod} \ u)

    a \equiv -1 \ (\text{mod} \ v)

We can assume that 1<a<n. If not, reduce a modulo n. Then a \not \equiv 1 \ (\text{mod} \ n). If a \equiv 1 \ (\text{mod} \ n), then a=1+n \cdot t=1+u \cdot v \cdot t for some t. This would mean that a \equiv 1 \ (\text{mod} \ v), contradicting a \equiv -1 \ (\text{mod} \ v). Similarly a \not \equiv -1 \ (\text{mod} \ n). So we have a \not \equiv \pm 1 \ (\text{mod} \ n). On the other hand, we have

    a^2 \equiv 1 \ (\text{mod} \ u)

    a^2 \equiv 1 \ (\text{mod} \ v)

By CRT, a^2 \equiv 1 \ (\text{mod} \ n). Let n-1=2^k \cdot Q where k \ge 1 and Q odd. Let Q=2w+1 for some integer w. The following calculates a^Q and a^{2 \cdot Q}.

    a^Q=a^{2w} \cdot a = (a^2)^w \cdot a \equiv 1^w \cdot a \equiv a \not \equiv \pm 1 \ (\text{mod} \ n)

    a^{2 \cdot Q} \equiv (a^2)^Q \equiv (1)^Q \equiv 1 \ (\text{mod} \ n)

The first term in (2) that is a 1 is a^{2 \cdot Q}. The preceding term is not a -1. This shows that the composite number n is not a strong pseudoprime to base a. Thus for any composite number n, there always exists a witness for the compositeness of n. \blacksquare

The proof of Theorem 1 is essentially that if n is composite, there is a square root other than \pm 1. This nontrivial square root is a witness to the compositeness of n.

___________________________________________________________________

The least witness

If n is a composite number, there exists a witness for n according to Theorem 1. Of all the witnesses for a given composite number n, there must be the smallest one. Let w(n) be the least witness for the compositeness for the composite number n. The number w(n) is an interesting one.

In general, the larger w(n) is, the harder to find the number n. This stems from the fact that strong pseudoprimes are very rare. Most composite numbers are not strong pseudoprimes to base 2. For these composite numbers, w(n)=2. For the strong pseudoprimes to base 2, w(n)>2. For w(n)>3, the composite numbers must be strong pseudoprimes to base 2 and base 3. For w(n)>5, the composite numbers must be strong pseudoprimes to the prime bases 2, 3 and 5. There are only 13 such numbers below 25 \cdot 10^9. Thus it is not easy to find composite numbers that have large least witnesses.

Two more comments to make. One is that knowing the least integer n such that w(n) is greater than the first k prime numbers can be as a deterministic primality test [2]. This is discussed in the next post. The other is that even though a large w(n) means that the composite numbers n are rare (under a specific bound). But there are infinitely many of them (over the entire number line). It follows from a theorem in [1] that for any finite set of integers, there are infinitely many Carmichael numbers whose w(n) is not found in the finite set.

___________________________________________________________________

Reference

  1. Alford W., Granville A., and Pomerance C., On the difficulty
    of finding reliable witnesses
    , L. Adleman and M.-D. Huang, editors, Algorithmic Number Theory: Proc. ANTS-I, Ithaca, NY, volume 877 of
    Lecture Notes in Computer Science, pages 1–16. Springer–Verlag, 1994.
  2. Jiang Yupeng, Deng Yingpu, Strong pseudoprimes to the first 9 prime bases, arXiv:1207.0063v1 [math.NT], June 30, 2012.
  3. Pomerance C., Selfridge J. L., Wagstaff, S. S., The pseudoprimes to 25 \cdot 10^9, Math. Comp., Volume 35, 1003-1026, 1980.

___________________________________________________________________
\copyright \ \ 2014 \ \text{Dan Ma}

The Fermat primality test

Fermat’s little theorem describes a property that is common to all prime numbers. This property can be used as a way to detect the “prime or composite” status of an integer. Primality testing using Fermat’s little theorem is called the Fermat primality test. In this post, we explain how to use this test and to discuss some issues surrounding the Fermat test.

___________________________________________________________________

Describing the test

The Fermat primality test, as mentioned above, is based on Fermat’s little theorem. The following is the statement of the theorem.

    Fermat’s little theorem
    If n is a prime number and if a is an integer that is relatively prime to n, then the following congruence relationship holds:

      a^{n-1} \equiv 1 (\text{mod} \ n) \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (1)

The above theorem indicates that all prime numbers possess a certain property. Therefore if a given positive integer does not possess this property, we know for certain that this integer is not prime. Suppose that the primality of an integer n is not known. If we can find an integer a that is relatively prime to n such that a^{n-1} \not \equiv 1 \ (\text{mod} \ n), then we have conclusive proof that n is composite. Such a number a is said to be a Fermat witness for (the compositeness of) n.

The Fermat test is closedly linked to the notations of probable primes and pseudoprimes. If the congruence relation (1) is true for n and a, then n is said to be a probable prime to base a. Furthermore, if n happens to be a composite number, then n is said to be a pseudoprime to base a. Pseudoprime prime is a composite number that possesses the prime-like property as indicated by (1) for one base a.

The Fermat primality test from a compositeness perspective is about looking for Fermat witnesses. If a Fermat witness is found, the number being tested is proved to be composite. On the other hand, the Fermat primality test, from a primality perspective, consists of checking the congruence relation (1) for several bases that are randomly selected. If the number n is found to be a probable prime to all the randomly chosen bases, then n is likely a prime number.

If the number n is in reality a prime number, then the Fermat test will always give the correct result (as a result of Fermat’s little theorem). If the number n is in reality a composite number, the Fermat test can make the mistake of identifying the composite number n as prime (i.e. identifying a pseudoprime as a prime). For most composite numbers this error probability can be made arbitrarily small (by testing a large number of bases a). But there are rare composite numbers that evade the Fermat test. Such composite numbers are called Carmichael numbers. No matter how many bases you test on a Carmichael number, the Fermat test will always output Probably Prime. Carmichael numbers may be rare but there are infinitely many of them over the entire number line. More about Carmichael numbers below.

The following describes the steps of the Fermat primality test.

    Fermat primality test
    The test is to determine whether a large positive integer n is prime or composite. The test will output one of two results: n is Composite or n is Probably Prime.

  • Step 1. Choose a random integer a \in \left\{2,3,\cdots,n-1 \right\}.
  • Step 2. Compute \text{GCD}(a,n). If it is greater than 1, then stop and output n is Composite. Otherwise go to the next step.
  • Step 3. Compute a^{n-1} \ (\text{mod} \ n).
    • If a^{n-1} \not \equiv 1 \ (\text{mod} \ n), then stop and output n is Composite.
    • If a^{n-1} \equiv 1 \ (\text{mod} \ n), then n may be a prime number. Do one of the following:
      • Return to Step 1 and repeat the process with a new a.
      • Output n is Probably Prime and stop.

    \text{ }

The exponentiation in Step 3 can be done by the fast powering algorithm. This involves a series of squarings and multiplications. Even for numbers that have hundreds of digits, the fast powering algorithm is efficient.

One comment about Step 2 in the algorithm. Step 2 could be called the GCD test for primality. If you can find an integer a such that 1<a<n and such that \text{GCD}(a,n) \ne 1, then the integer n is certainly composite. Such a number a is called a GCD witness for the compositeness of n. So the Fermat test as described above combines the GCD test and the Fermat test. We can use the Euclidean algorithm to find the GCD. If we happen to stumble upon a GCD witness, then we can try another n for a candidate of a prime number. For most composite numbers, it is not likely to stumble upon a GCD witness. Thus when using the Fermat test, it is likely that Step 3 in the algorithm is used.

An example of Fermat primality testing is the post called A primality testing exercise from RSA-100.

____________________________________________________________________________

More about the test

When using the Fermat test, what is the probability of the test giving the correct result? Or what is the probability of making an error? Because the Fermat test is not a true probabilistic primality test, the answers to these questions are conditional. In one scenario which covers most of the cases, the test works like an efficient probabilistic test. In another scenario which occurs very rarely, the Fermat test fails miserably.

As with most diagnostic tests, the Fermat test can make two types of mistakes – false positives or false negatives. For primality testing discussed in this post, we define a positive result as the outcome that says the number being tested is a prime number and a negative result as the outcome that says the number being tested is a composite number. Thus a false positive is identifying a composite number as a prime number and a false negative is identifying a prime number as a composite number.

For the Fermat test, there is no false negative. If n is a prime number in reality, the statement of Fermat’s little theorem does not allow the possibility that n be declared a composite number. Thus if the Fermat test gives a negative result, it would be a true negative. In other words, finding a Fermat witness for n is an irrefutable proof that n is composite.

However, there can be false positives for the Fermat test. This is where things can get a little tricky. A composite number n is said to be a Carmichael number if the above congruence relationship (1) holds for all bases a relatively prime to n. In other words, n is a Carmichael number if a^{n-1} \equiv 1 (\text{mod} \ n) for all a that are relatively prime to n. Saying it in another way, n is a Carmichael number if there exists no Fermat witness for n.

The smallest Carmichael number is 561. Carmichael numbers are rare but there are infinitely many of them. The existence of such numbers poses a challenge for the Fermat test. If you apply the Fermat test on a Carmichael number, the outcome will always be Probably Prime. So the Fermat test will always give a false positive when it is applied on a Carmichael number. To put it in another way, with respect to Carmichael numbers, the error probability of the Fermat test is virtually 100%!

So should a primality tester do? To keep things in perspective, Carmichael numbers are rare (see this post). If the primality testing is done on randomly chosen numbers, choosing a Carmichael number is not likely. So the Fermat test will often give the correct results. For those who are bothered by the nagging fear of working with Carmichael numbers, they can always switch to a Carmichael neutral test such as the Miller-Rabin test.

___________________________________________________________________

One bright spot about the Fermat test

There is one bright spot about the Fermat test. When applying the Fermat test on numbers that are not Carmichael numbers, the error probability can be made arbitrarily small. In this sense the Fermat test works like a true probabilistic primality test. Consider the following theorem.

    Theorem 1
    Let n be a composite integer such that it is not a pseudoprime to at least one base (i.e. n has a Fermat witness). In other words, n is not a Carmichael number. Then n is not a pseudoprime to at least half of the bases a (1<a<n) that are relatively prime to n. In other words, n is a pseudoprime to at most half of the bases a (1<a<n) that are relatively prime to n.

Theorem 1 means that the Fermat test can be very accurate on composite numbers that are not Carmichael numbers. As long as there is one base to which the composite number is not a pseudoprime (i.e. as long as there is a Fermat witness for the composite number in question), there will be enough of such bases (at least 50% of the possible bases). As a result, it is likely that the Fermat test will find a witness, especially if the tester is willing to use enough bases to test and if the bases are randomly chosen. When a base is randomly chosen, there is at least a 50% chance that the number n is not a pseudoprime to that base (i.e. the Fermat test will detect the compositeness) or putting it in another way, there is at most a 50% chance that the Fermat test will not detect the compositeness of the composite number n. So if k values of a are randomly selected, there is at most 0.5^k probability that the Fermat test will not detect the compositeness of the composite number n (i.e. making a mistake). So the probability of a false positive is at most 0.5^k. For a large enough k, this probability is practically zero.

Proof of Theorem 1
A base to which n is a pseudoprime or not a pseudoprime should be a number in the interval 1<a<n that is relatively prime to n. If n is a pseudoprime to base a, then a raised to some power is congruent to 1 modulo n. For this to happen, a must be relatively prime to the modulus n. For this reason, when we consider a base, it must be a number that is relatively prime to the composite integer n (see the post on Euler’s phi function).

Let a be a base to which n is not a pseudoprime. We make the following claim.

    Claim
    If b is a number such that 1<b<n and such that n is a pseudoprime to base b, then n is not a pseudoprime to base a \cdot b.

Since both integers a and b are assumed to be relatively prime to n, the product a \cdot b is also relatively prime to n (see Lemma 4 in this post). Now consider the congruence (ab)^{n-1} \ (\text{mod} \ n), which is derived as follows:

    (ab)^{n-1} \equiv a^{n-1} \cdot b^{n-1} \equiv a^{n-1} \not \equiv 1 \ (\text{mod} \ n)

In the above derivation, we use the fact that n is not a pseudoprime to base a and n is a pseudoprime to base b. The above derivation shows that n is not a pseudoprime to base ab.

If n is not a pseudoprime to all bases in 1<w<n, then we are done. So assume that n is a pseudoprime to at least one base. Let b_1,b_2,\cdots,b_k enumerate all bases to which n is a pseudoprime. We assume that the b_j are all distinct. So b_i \not \equiv b_j \ (\text{mod} \ n) for all i \ne j. By the above claim, the composite number n is not a pseudoprime to all the following k numbers:

    a \cdot b_1, \ a \cdot b_2, \cdots, \ a \cdot b_k

It is also clear that a \cdot b_i \not \equiv a \cdot b_j \ (\text{mod} \ n) for i \ne j. What we have just shown is that there are at least as many bases to which n is not a pseudoprime as there are bases to which n is a pseudoprime. This means that n is not a pseudoprime to at least 50% of the bases that are relatively prime to n. In other words, as long as there exists one Fermat witness for n, at least 50% of the bases are Fermat witnesses for n. It then follows that n is a pseudoprime to no more than 50% of the bases relatively prime to n. \blacksquare

There is another way to state Theorem 1. Recall that Euler’s phi function \phi(n) is defined to be the number of integers a in the interval 1<a<n that are relatively prime to n. With this in mind, Theorem 1 can be restated as the following:

    Corollary 2
    Let n be a composite integer such that it is not a pseudoprime to at least one base. Then n is not a pseudoprime to at least \displaystyle \frac{\phi(n)}{2} many bases in the interval 1<a<n.

___________________________________________________________________

Concluding remarks

Of course, Theorem 1 works only for the composite numbers that are not pseudoprime to at least one base (i.e. they are not Carmichael numbers). When you test the compositeness of a number, you do not know in advance if it is Carmichael or not. On the other hand, if the testing is done on randomly chosen numbers, it is not likely to randomly stumble upon Carmichael numbers. The Fermat test works well for the most part and often give the correct results. If one is concerned about the rare chance of a false positive in the form of a Carmichael number, then the Miller-Rabin test will be a good alternative.

___________________________________________________________________
\copyright \ \ 2014 - 2015 \ \text{Dan Ma} (Revised march 29, 2015)