And when England were awarded an obvious penalty when Carlos Sanchez hauled down the magnificent Harry Kane, there was a three-minute delay for Colombia's protests before the striker scored from the spot.
Harry Kane gave England the lead with a penalty kick in the 57th minute.
Red Square, in the centre of the capital, was awash with bright yellow and red and white as Colombian and England fans posed for pictures next to St Basil's Cathedral.
Jordan Henderson saw his effort saved by David Ospina, but after Mateus Uribe hit the bar, Kieran Trippier found the top corner, Jordan Pickford saved from Carlos Bacca and then Eric Dier stepped up to slide home the winning kick.
"To get knocked down at the end like we did with that goal in extra time, you know, it's hard to come back from that".
England substitute Dier missed the best chance of the extra period when he headed over but he made up for it in emphatic style before being mobbed by his relieved team mates.
"We've not got a good record against them, I think we've always underestimated them.
But we did", Dier said.
That came 22 years ago, against Spain at Euro 96, but it has been otherwise been a tale of woe, with current manager Gareth Southgate famously missing the crucial kick in a defeat against Germany later in the same tournament.
Southgate has admitted it would never get off his back but leading England to their first ever shootout success at a World Cup is some redemption at least. We still know we have extra time; we still have a penalty shootout to win this game.
Kane took his tally at the 2018 World Cup to six goals, which puts the England forward two clear of Belgian frontman Romelu Lukaku in the race for the Golden Boot. Now back to full form and fitness, the Colombian skipper is a very risky player and one that certainly Gareth Southgate must have spoken about at length in his team meetings. Now coach Gareth Southgate will need guide his young players against an experienced Colombia squad capable of dominating both sides of the ball. It's a big night for England.
Colombia captain Radamel Falcao also accused referee Mark Geiger of bias towards England, calling the American's performance "shameful".
That mentality and the preparation was evident in the way England dealt with the challenge of the shootout. "That has not been good for us, and hurt us a lot", said Colombian coach Jose Pekerman at a post-match press conference. The first match I ever watched was the 66 final but I was eight and can not remember much about it.
The have shown those qualities in this tournament but on Tuesday, under pressure and facing unsafe players of the ability of Radamel Falcao and Juan Cuardado they also showed they have all the defensive basics in place.