I think that it's easier to consider field recording as one way to earn income, but not to survive on alone. Many who seem to do it for a living still work in radio, editing, sound design, engineering, or any number of other fields (sometimes not even related to audio). I'm of the opinion that while specialization suits some, it's not for everyone, and those who can switch gears between different aspects of audio are more flexibly employable...especially early in their careers.
I think the way to set yourself apart in field recording is to take the time, effort, and energy to either record things people haven't yet heard, haven't recorded well or deeply, haven't recorded enough of, or haven't recorded in certain techniques or contexts. This is not easy, but that's how you'll get noticed above the rest. Consider that a lifetime goal, not something to get into right out of the gate.
How you use the field recordings may vary, too. Some field recordists are fine artists, sound designers, scientific researchers, musicians, et multiple cetera. This is perhaps the biggest variance in potential paycheck and how crowded the field is, competition-wise.
If approached with zeal and enthusiasm, hobbies and side projects can absolutely lead to personal enrichment and career changes. It's just important to do it for the love of doing it, and set your expectations around earning potential accordingly. The intensity of what you put in usually pays you back in some way, just be open minded what form that payback takes.