Ernest Gary Gygax was born in Chicago, Illinois, to Ernest Edward Gygax, a violinist and insurance salesman, and Almina Emelie 'Alma' Gygax (née Burwick).

The Gygax family moved to Lake Geneva, Wisconsin, where Gary spent most of his childhood. Lake Geneva became a significant backdrop for his life and later gaming ventures.

Gygax developed an early passion for science fiction and fantasy literature. He was heavily influenced by authors such as Robert E. Howard (Conan), Edgar Rice Burroughs (Tarzan, Mars), and J.R.R. Tolkien (The Hobbit, The Lord of the Rings). These works would later inform the themes and settings of Dungeons & Dragons.

Gygax participated in wargaming and miniature figure collecting from a young age. He developed a love for historical simulations and strategic gameplay, which laid the foundation for his future game design endeavors.

Gary Gygax married Mary Jo Hyatte. They would have five children together. This marriage was a significant part of his early adult life and family foundation.