The Erd\H os-Anning Theorem states that an integer distance set in the Euclidean plane is either finite or must have all of its points on a single line. Erd\H os found an upper bound of $4(\delta +1)^2$ points for a non-collinear distance set with a diameter of $\delta$. We prove a finite field version of this result for dimensions two and three, showing that if $E \subset \Bbb{F}_q^d, q=p^2$, where $p$ is an odd prime and the distance set of $E$ is $\Bbb{F}_p$, then the size of $E$ is at most $p^d$.