I eschew milk and cheese entirely.  That's an easy decision.

But what about yogurt?  Should that also be avoided, or it is somehow different i.e. not bad like milk is?  I can tell you empirically I don't feel crappy and look crappy after eating yogurt like I do after drinking milk

The only difference between milk and yogurt is that the lactose in milk has been converted to lactic acid in yogurt. This makes it more tart, prevents spoilage and digestible for the large parts of the adult world that can not digest lactose -only babies were supposed to drink milk so babies all over the world can.
So yogurt has the same properties as milk, just spoils less and is more digestible. All other properties, as far as I can tell, apply to both.   

Try non dairy yogurt like coconut milk yogurt (my fav); you can also make your own non dairy yogurt. there are many recipes available on the web. A simple google search will get your there. So Delicious makes a coconut milk yogurt; I haven't checked out the ingredients so I don't know if it's paleo, but it's worth checking out.

Or just stick to not having yogurt, since that seems to be working well.

This. I used to make my own kefir and drink at least 1/2 liter (0.13 gallon ? ) a day. I feel better without it, in full honesty.