He was born in Philadelphia, Pennsylvania to John Lyon and Ida Frances (Everett) Vandiver. He did not complete a formal education, choosing instead to leave school at an early age to work for his father's firm, although he did attend some graduate classes at the University of Pennsylvania in 1904–5.
From 1917 to 1919 he was a member of the United States Naval Reserve, and in 1919 became an instructor of mathematics at Cornell University, where he taught for five years before becoming an associate professor of pure mathematics at the University of Texas in 1924. He was made a full professor the following year, and named distinguished professor of applied mathematics and astronomy in 1947. He remained at Texas until his retirement in 1966.
Vandiver won the Frank Nelson Cole Prize of the American Mathematical Society for his paper on Fermat's Last Theorem in 1931. In 1952 he used a computer to study it, proving the result for all primes less than 2000.
For the academic year 1927–1928 Vandiver received a Guggenheim Fellowship. In 1934 he was elected to the National Academy of Sciences. In 1945 the U. of Pennsylvania gave him an honorary doctoral degree.