My Ph.D. is in theoretical physics, so naturally, I have taken a fair number of “rigorous” math courses (theorem, proof, theorem, proof, …on and on, like an Iowa corn field). I think it’s important, although not easy, for a theoretical physicist not to be intimidated by rigorous math.
One thing that I’ve noticed from reading their books is that mathematicians have a fetish for inequalities. Their proofs often follow easily from an inequality, but only God, and mathematicians, know where that inequality came from. Sure, mathematicians will give you a proof of an inequality, but the proof is often as mysterious as the inequality itself. Providing motivation for what they are saying is not the strong suit of mathematicians.
Then one day I realized, hey, physicists have a fetish for approximations. And although mathematicians wrinkle their noses every time they see a physicist do an approximation, and physicists wrinkle their noses every time they see a mathematician write down yet another inequality, they are essentially doing the same thing.
As a trivial example, take the Cauchy-Schwarz inequality for two vectors. All it’s saying is that when the two vectors are nearly parallel, you can approximate their dot product by the product of their lengths. Mathematicians, you don’t fool me, you love approximations too. Come out of the closet!
It’s true, inequalities are more powerful than approximations; they’re like approximations on steroids. But I bet most inequalities were initially conceived as humble approximations, and then someone came along and said, What Ho!, we can promote this into an inequality, and get a math paper out of it. Goes the other way too. Often, when we are doing a back of the envelope estimate, we demote an inequality into an approximation.