Elves aren’t born. Not as elves, but as humans. They turn into elves when a true elf puts a spore in them. Spores are organs that grow in the back of their throats, much like tonsils. When they get too large, say grape or walnut sized, the true elf needs to spit them out.
The spores have legs and for about a week they can move up to half a mile away before they die. They crawl down the throats of unsuspecting humans and implant themselves in the stomach. Over the course of the next eighteen months, an elf will grow in their torso and abdomen. This elf will copy the host’s features, memories and personality. The host will start to lose those qualities, grow sick, lose appetite and hair. When the elf copy is ready, it will literally molt the human body off itself.
The resulting elf is not the same person (or spirit) but a copy of its memories and personality (depending of course on the metaphysics you use for your campaign). They are shorter because they can’t grow to full height (always wondered why, didn’t you?).
Elves do live forever, barring accident or misadvanture, but they can’y hold more memories than a human. After a few hundred years, an elf will probably not remember that she was ever human. The fragments of that life that remain will eventually fade and might be interpreted as dreams. Elves as a whole are ignorant of their origins.
This is why they are so keen on composing songs and writing books, to preserve what they can. Their rich culture is an amalgam of millennia worth of human culture, as remembered and recorded by elves and then read and reinterpreted by elves who don’t remember it anymore. The errors in interpretation trend toward the prevailing elven culture of the region and over time, it all skews into a very idiosyncratic synthesized culture that could easily be mistaken for something borne of its own.
What about these true elves that spread spores? They are the real elves. The ones we know are really sort of half. The DM should decide how they should look. They could resemble the gray aliens, or the ur-elf to out-elf all elves, or they could be some sort of yuggoth in the bottom of a well. Best not to reveal them if you don’t have to.
You can also play with this for other fairie species in addition to or in place of elves. Dryads or nymphs could come from the fruit of a certain tree, or sprites and pixies could be spore-infected halflings or gnomes.
With apologies to vampires, Alien, Chtorr and any other sci-fi or fantasy creature or trope this closely resembles.