When you send a letter via snail mail through the post office, you usually write your return address on the envelope so the recipient can identify the sender, and so that the post office can return the mail to the you in the event of a problem. But nothing prevents you from writing a different return address than your own; in fact, someone else could send a letter and put your return address on the envelope. Email works the same way, email messages have envelope information too. When a server sends an email message, it specifies the sender in its envelope information, but this sender field can be forged. If there is a problem with delivery and someone forged your address on the message, then the message will be returned to you, even if you weren't the one who actually sent the message.
If you've received a reply to a message that wasn't sent from your address, then the message was probably spoofed, forging your address as the sender or the original sender of the email used your address as a reply-to address so that responses would be sent to you.
Spammers will often use these techniques as a way of bypassing your spam filters and tricking you into reading their junk mail.