Hmm as you know I think anecdotal evidence is often worth less than no evidence at all, but this is interesting.
I ate huge amounts of dairy while I had my baby teeth, and had huge amounts of cavities. I think about midway through high school I stopped drinking milk altogether (i lost my baby teeth late) and never started drinking it again.
Recently I went to the dentist for the first time in over a decade, expecting the worst and all I needed was one very minor filling.