Lynard (rawk), it seems to be a common problem, but Google says nobody really knows what causes it. One of the closest things has been
blocking auth servers (dev1ance.net/forum), but this was for 1.11, and can/will break 1.32, and networking fixes have already been applied in old revisions.
You may wish to check your network settings, firewall, malware, etc., but all this seems to be a shotgun approach, because it's something really hard to troubleshoot, let alone as a third party. (I suspect that a router breaks packets somewhere, maybe when it's not supposed to, or maybe because that's the "right" thing to do, but I really don't know.)
Of course, it could also be something really obvious, and nobody's replied yet.
Hope you can figure something out. If you need some basic / general information on networking,
http://portforward.com/ has some stuff.
Good luck.