I was extremely lucky that this never happened to me because I have a delicate nose (remind me to tell you about the time my daughter swallowed a small object and I spent a week picking through poopy diapers- thanks Sweetie!). But I had a friend who said that when she pumped breastmilk everything seemed normal, and that when her daughter nursed, she had normal smelling and tasting breastmilk. However, every time she stored the milk overnight in the fridge or put it in the freezer for use later, it smelled "funky" when she thawed it. Sometimes her daughter would drink it and sometimes she wouldn't. And come on, how disheartening is it when your kid refuses the milk you pumped for them?
What's going on?
Fear not, although it doesn't happen to everyone, it's also not completely unusual. A protein found in milk, lipoprotein lipase, starts digesting fat in the milk. The milk fat is hydrolyzed; it's broken down. There was a study from 2009 about this breakdown and the effect this breakdown may have on the sensory experience that babies have while drinking the milk.
It was found that "the majority of the stored human milk samples exhibited a specific metallic-fishy odour-note that, in several cases, was described as very intense. Apart from that, some samples also exhibited a characteristic rancid-sweaty odour impression, comparable to that of rancid cow milk" (Spitzer, 242). I've also heard it referred to as smelling "soapy". Maybe this is you? What I found interesting from this study was that although the tasters found the stored milk taste to be unpleasant, the babies themselves generally didn't respond negatively to the stored milk- good news! They weren't sure why this was.
I couldn't find good data on this, but from friends and searching the web, it may be possible to decrease this taste by warming the milk up. I'd be careful with this though, because getting breastmilk to a temperature that's too high could degrade the immunological factors.
TL;DR It's probably not you and most babies don't mind!