Putnam exam link. Note that it happens every December.
The AI must score in the top 100 for a particular year.
By "top 100" I mean that its score must be >= the score of the 100th place scorer. (If 100th place is a tie I'll use the tying score).
If we know the details of the training data, then the training data must all have been released prior to the release of the Putnam questions for that year. i.e. if ModelNet is run on the 2026 Putnam, it must be trained on data from before the date of the 2026 Putnam exam.
The AI does not have to be trained before the relevant exam, as long as the data predates the exam.
The scoring for the AI's exam must either be done by the actual Putnam scorers, mathematicians who have been Putnam scorers, or mathematicians who are actively involved in competitive mathematics in some way. (i.e. a professor who runs a university's competitive team counts, a software engineer who did well in the Putnam 5 years ago does not).
I may accept scoring that isn't blinded, but I reserve the right to ignore any scoring that's vaguely suspect/biased/etc.
1,000
3.00https://x.com/cool_cocohearts/status/1865910680035205585
o1 gets ~40 points on this year's putnam, probably top 50-100. Another estimate puts it at 60 points.
@jack I think there's a good chance this will count, but resolution may take a while because I want the scores from this year and also the opinions of more informal graders (or at least one formal grader)
@jack as a former math olympian i would add that rigorous grading of this response will not be as high as 30-40 points (likely 10-15). The jump from “educated guess” to “a complete proof” is the most difficult part in math proofs and usually “educated guess” receives 0/1/2 points out of 10.
That being said, other prompts or o1-pro may perform better.
@mathvc Yeah most of my uncertainty here is whether this twitter is actually doing the putnam "you can't get 4, 5 or 6 points" or even thinking about partial credit at all.