I understand that setting your audio interface to 96Khz or 192Khz can change the sound of a software synthesizer. So would it be best to record the synthesizer at 96Khz or 192Khz?
Neither is best. You do what creates the best sound according to the ideals you want to achieve. 44.1 kHz or 48kHz are perfectly usable as recording sample rates, going higher will only make minor changes, not necessarily better, but slightly different. To some ears these changes are important, to other ears not so. So the recommendation is to try it and use your own ears and listen. My bet is that you will end up at 44.1kHz unless you are into video where 48kHz is used.