"The Royal Navy are now in attendance and will safely remove and dispose of the ordnance over the next few hours," London's Metropolitan Police said in a statement.
The involvement of the armed forces came hours after police temporarily evacuated Westminster underground station, which serves the Houses of Parliament.
The rush hour shutdown also saw the closure at around 5:15 pm of Westminster Bridge, in the shadow of parliament's Big Ben clock, and Waterloo Bridge, both of which have since reopened.
"Further temporary road closures are expected to be necessary throughout the night as the ordnance is dealt with. The majority of these will be in central London," police said.
The ordnance was spotted after coming to the surface of the water in the tidal flow of the Thames, a police spokesman told AFP.
It was found in the river by Victoria Embankment, close to Horse Guards Avenue where the Old War Office Building is located, which was used by the British government during World War II and was bombed.
More than 12,000 metric tons of bombs were dropped on the British capital during the war, according to London's Imperial War Museum.
