Sample rate is controlled by electronic circuits. They are not perfect and will vary from one box to another and may vary depending on temperature. This means that even if they "nominally" run on the same sample rate, there will be a difference. Hence getting out of sync. So, actually the lower temperature when raining might have an effect.
The electronic circuits contains a crystal which might have a frequency stability of +/- 50 ppm (parts per million) if you are lucky. Really expensive crystal oscillators places the crystal in an oven to keep the temperature constant. Or might sync to GPS satellite signals (that are extremely accurate). You will not find those expensive solutions in audio recorders (well, there might be somewhere).
50 ppm means 2.4 samples per second at 48kHz or about 3ms per minute which is about what you can perceive. Of course if one is slow and one is fast the difference might be double that.