A childhood belief about weather

I used to believe, sun moves when nobody is looking. And the sun psyches the observer into thinking something and not notice his movements (I guess my grand mother told me this as she couldn't answer my many questions).
On many evenings, I used to sit in the balcony determined to catch a setting sun but in the process invariably used to be distracted by some thought or the other and the next time I look the sun is already a foot below from where he was when I started. As I woudn't notice I spent so much time thinking something else, i used to think he moved all that distance at my momentary distraction.
After many trials, I gave up thinking it's a human limitation and sun always manages to move when nobody is looking. Similar to the deceiving rains. I knew when it rains it doesn't rain everywhere in the world. So, i used to think, there has to be a place where the rain ends and because nobody i knew had seen such a thing, this always happens when nobody is around.