By the time 6GB isn't enough you should be replacing the card. I would say about 3 years or so, because I expect new consoles to be in full swing by then.
That is typically when you see a major uptick in VRAM demand, about a year after new consoles arrive. This is simply because the majority of games released are then designed for those new consoles in mind. As soon as they get a generational spec upgrade, all the game engines and PC performance demands increase over a relatively short period.
This is also likely to apply for CPU specs as well, if new consoles deliver on the rumoured Zen 8 core upgrade. You could quickly find a lot of games needing at least a very fast 6 core CPU, or a slower 8 core one just to get good performance.
The game that springs to mind after the PS4/XB1 generation for me personally was Watchdogs out in May 2014. Clearly a game designed for new consoles that had 3-4GB of VRAM available and ported (badly) to PC. That poor porting meant the VRAM demands of the game were very high, requiring at least 3GB of VRAM to make it run smoothly on anything better than medium settings.
That ruled out a massive swathe of video cards from 2011-2013 that would otherwise have been fast enough, but lacked memory to avoid stutter. The GTX670 2GB for example was only two years old and still plenty fast for nearly everything, but crumbled with Watchdogs due to lack of memory.