Problem with Remote Desktop Connection

Recently, we got telus internet at our workplace. And I have telus at home. Every time I try connecting using remote desktop connection, it works fine. But after I disconnect, and try to connect again within the next 1-2 minutes, it says it can't connect. After about 1-2 minutes, it connects again fine. And it just goes on like that.

Before when we had a different company for internet, this problem did not happen. Anyone know the problem? Do you need more info from me?

Thanks.