Re: DJB's students release 44 *nix software vulnerability advisories

[Crispin Cowan <crispin@xxxxxxxxxxx>]

Thor Larholm wrote:

> This small group of students highlights how individuals outside the
> security industry without special security prerequisites can still
> manage to outperform the average Bugtraq poster in sheer quantity of
> discoveries.
That might be just a tad overstated.

The slashdot article 
http://it.slashdot.org/article.pl?sid=04/12/15/2113202 was submitted by 
one of these students. The student said that he spent 300 hours on the 
project. The class had 25 students, so if we assume that is typical, 
that is 7500 man-hours to find 44 vulnerabilities, or 170 hours per bug.

I don't believe that this "outperforms" the typical bugtraq poster. More 
likely, it shows that when you are a professor, you can mandate a lot of 
work if you want to :)

> This adequately validates the typical estimate of between 5
> and 15 errors in every thousand lines of code.
>  
How so? The assignment was to find bugs in "UNIX" code, which arguably 
is at least 10,000,000 lines of code for a typical UNIX desktop, which 
should have over 50,000 bugs. That the class could find approx. 50 of 
them does not come close to validating a rate that predicts 50,000.

None of which is to denigrate the fine work that DJB and his class have 
done. I just don't think it validates the claims that Thor says it does.

Crispin

--
Crispin Cowan, Ph.D.  http://immunix.com/~crispin/
CTO, Immunix          http://immunix.com