Or am I the one who doesn't understand the paleo diet?

You are right and he is wrong.

Paleo does not include potato or dairy.

Cream and butter are dairy, and thus clearly nonpaleo.  However, some feel that small amounts are less bad than other forms of dairy because they are still animal fats and don't include the sugars and potentially allergenic proteins that milk does.  I don't think anyone would claim they are strictly paleo, but some consider them among the less harmful cheats.

Potatoes are questionable in a couple of respects.  First, they are a tuber - an underground plant reproductive organ - and there's an argument that unlike fruit, plants have evolutionary reasons to make such organs strongly deleterious to animals when eaten.  The evidence that humans ate any tubers during the paleolithic is also limited and equivocal.  Second, they are a new world plant, and thus from a group of plants that would not have been available to paleolithic humans; some feel they are not sufficiently closely related to plants we did eat during the paleolithic to be considered paleo.  However, some people do consider potatoes to be paleo, since neither of these arguments is black and white.

Think about it... you're not considering a potatoe a cheat meal :d   a lot better than pizza and beer... lol.

Personally I would definitely consider potatoes to be a cheat meal.  It's pretty much pure carbohydrate, like a candy bar or a slice of pizza without the healthy toppings.

I agree. Especially if you don't eat the potato peel, they are just carbs and super high glycemic index.

And the peel is where the poison is isn't it?

potato skins are the part where the vitamins are for the most part.

I may regret potatoes.... although they were usually an excuse for eating butter

In Cordain's study of hunter gatherers living today he did find multiple eating tubers. Here is the link to his research paper:

http://www.thepaleodiet.com/articles/AJCN%20PDF.pdf

Now, potatoes... I would rather go for sweet potato, taro, etc.

If you feel better without them, why eat them? Is a personal thing, you would have to try it out for yourself.