Last month, Gonzales dropped his re-election bid after admitting to having an affair with a staff member. A House ethics panel is investigating his conduct.
US media reported four women have accused Swalwell of misconduct ranging from harassment to sexual assault, including an alleged incident in New York that has prompted an investigation by Manhattan district attorney’s office. The BBC has not independently verified the unnamed accuser’s account or identity.
Swalwell – who was elected in 2012 to a district near San Francisco – vowed in a statement on Friday to defend himself.
“For nearly 20 years, I have served the public – as a prosecutor and a congressman, and have always protected women,” the married father-of-three added. “I will defend myself with the facts and where necessary bring legal action.”
Within hours of the accusations surfacing, Swalwell’s California gubernatorial campaign had lost the backing of prominent supporters, including House Democratic Leader Hakeem Jeffries.
Now some of his colleagues are going further in calling for him to step down from Congress altogether.
Jared Huffman, a California Democrat, said if Swalwell and Gonzales did not resign from Congress, he would support their expulsion.
Democrat Pramila Jayapal of Washington said she would also vote to expel.
She said it was “important for staffers across the Capitol to see that their bosses don’t get to do this to them”.
Without explicitly committing to back an expulsion motion, Ro Khanna, a California Democrat, condemned Swalwell’s alleged conduct.
“There needs to be consequences to that,” Khanna said. “And I have said not only does he need to step aside, there needs to a House ethics investigation and a law enforcement investigation.” Khanna said Gonzales also needed to go.
The calls for Swalwell and Gonzales to stand down were bipartisan.
Mike Lawler, a Republican from New York, said: “Congress must hold itself to the highest ethical standard, regardless of party.”
Byron Donalds, a Florida Republican, told NBC News: “That vote comes to the floor, I will be voting yes on both measures… As far as I am concerned, both gentlemen need to go home.”
Anna Paulina Luna, a Florida Republican, said on Saturday that she would file a motion to expel Swalwell.
The BBC has contacted representatives for Swalwell and Gonzales for comment.
An expulsion measure from the US House is extremely rare and requires support from two-thirds of lawmakers present and voting at the time it is called. Only six lawmakers have been expelled from the US House in its 237 year history.
The Swalwell allegations surface at a crucial moment in the wide-open Democratic primary race to lead the nation’s most populous state – which has elected a Democrat to the role for the past 20 years – just weeks before voters receive postal ballots.