I was confused, why would the reboot take so much time? Was Why did it always coma back at 06:00?
I got around to have a look now, while it was happening. I SSH'd into the Pi and checked the uptime at around 04:00, when it should have been roughly 45 minutes. It said "03:02:00 uptime 2 min". Uhhh. I checked again. Now the system time was correct, and shortly after that, it rebooted again.