The four-day Festival was staged as planned, with additional hygiene measures in place due to the coronavirus pandemic, and concluded on March 13 - with Prime Minister Boris Johnson addressing the UK 10 days later to impose the current lockdown.
Appearing on Good Morning Britain, Dowden was accused by presenter Piers Morgan of "actively encouraging" people to attend the likes of the Cheltenham and the Liverpool versus Atletico Madrid football match after the Government decided not to ban major events until late March.
Mr Dowden said: "The scientific evidence we were being given was that, at a mass gathering, the threat at a mass gathering relates to the people who immediately surround you - the people in front of you and behind you.
"The risk at mass gatherings was no greater or less than it would have been in pubs or restaurants, and the advice at that point was that we did not need to ban mass gatherings."
Asked whether the advice was wrong, the Cabinet minister replied: "As the situation developed, the scientific advice changed and we changed our guidance off the back of it.
"But mass gatherings are not different to any of those other events I described and at the appropriate moment we took the decision to close pubs, to close restaurants."
The decision to go ahead with the Festival subsequently attracted criticism, but speaking on Racing TV on Sunday, British Horseracing Authority chief executive Nick Rust underlined the meeting had taken place in conjunction with Government advice at the time.
He said: "There's a lot of comment around and blame around, but you have to put it into context. The decision to go ahead with Cheltenham was taken with Government advice and Government scientific and medical advice.
"Many other activities took place that week, we had a Premiership football weekend beforehand, a Six Nations rugby match on the Sunday, Crufts indoors, millions of people were going down the tube in London.
"The advice was keep going. Then things changed that week. Of course I guess we should be a little bit worried, not necessarily because of the decisions taken around that but because of the perception of it.
"Winding back to the Tuesday of that week, I think if we'd cancelled racing against Government advice at that time for Cheltenham, I just don't think it would have been the right decision and it would have been widely criticised as being somewhat alarmist.
"But the mood did change that week, Rishi Sunak came out with his budget on the Wednesday and there was a more worried update from Boris Johnson on the Thursday.
"Then quickly we got into the next week on the Monday and things had changed very rapidly straight into limitations on gatherings, drains on public services and a much different mood and scientific message."
Fast Results
Scores
Racecards 
