Welcome to my world, Pat.
Numbers are indeed magickal... They provide a unique way of explaining inherently complex relationships between other things. If you think about it, even counting is a magickal act... it forms the basis of interval, frequency and rhythm!
And thinking about prime numbers (and the relationship between them)... 1, 3, 5... this is also the basis for musical interval... the root (1), 3rd and the 5th form a basic chord.
That same chord can be expressed as a mathematical term in geometry, being a line through the interior of a circle, who's end points lie upon the circumference. The longest chord for any circle is the diameter.
Prime numbers are also very important in cryptology, in fact, factoring prime numbers is one of the bases of modern encryption methodologies.
And while you're off down your rabbit hole, consider the golden ratio or Phi (1.6180339887...) In mathematics, two quantities are in the golden ratio if their ratio is the same as the ratio of their sum to the larger of the two quantities. i.e. Where a>b>0, (a+b)/a = a/b or (3+2)/3 = 3/2 = 1.6180339887...
The golden mean or ratio has a profound affect on our lives and even defines what we consider to be beautiful... not just physically, but also in terms of musical interval.
Oh, and have you ever thought about what thoughts are made of?
Go ponder that for a while