The biggest latency in the internet's infrastructure is electronic routers and switches. Slowed down by the constant conversion and reconversion of information from its optical (light) form to electronic data used by switches and routers, network communication runs far slower than it could, even with existing fibre optic cabling. But with improved equipment that tackles this latency, we could have an internet that's about 100 times faster.
New "optical" switches and routers are on the drawing board that will provide a solution to the latency problems, using buckyballs glued together by a custom polymer.