My friend had bariatric (weight loss surgery) and she said at one of her early post-surgery visits the doctor felt her fat and said since it was so soft and loose, she was losing weight. When the fat is hard and solid, you're gaining. I guess it's like air being let out of a balloon.
But you don't have to lose fat to gain muscle. They aren't mutually exclusive. It's just that muscle weighs more than fat and burns more calories, so the more muscle you build, the higher your metabolism is, which makes it easier to lose weight.