A native of Wichita Falls, James Riddle was a member of the undefeated 1948 freshman football team. He was a three year letterman in baseball as a shortstop/second baseman and was elected co-captian in 1951. He earned two letters in football as a quarterback and was also elected co-captian in 1952. In boxing, he won the Abilene Regional Golden Gloves Novice Lightweight Championship in 1949. In 1951, he was in the finals of the Open Class Middleweight Dicision of the Abilene Regional Golden Gloves. In 1953, he represented Hardin-Simmons as the American College Student Leader. After graduation, he played quarterback for the U.S. Army in Killeen from 1954-1955. He then became the backfield coach at Midwestern University.