Ken Fisher and Fisher Investments has now lost more than $2 billion in clientele — with Fidelity’s withdrawal of $500 million Monday and The New Hampshire Retirement System pulling $239 million Tuesday — in the aftermath of the CIO’s inappropriate comments at a conference earlier this month.

Fidelity Investments announced it would be ending its relationship with Fisher, whose firm was in charge of managing the $500 million in Fidelity’s Strategic Advisers Small-Mid Cap Fund.

“Fisher Investments does not provide investment advisory services for any portion of the assets of Strategic Advisers Small-Mid Cap Fund,” Fidelity spokesperson Vin Loporchio said Monday, according to CNBC. “Assets previously managed by Fisher Investments have been reallocated within the fund.”

Fidelity was the fifth major firm to cut ties with the money manager after the Iowa Public Employees Retirement System announced it would also relieve Fisher of its $386 million in pensions Friday. This followed Boston, Philadelphia and Michigan, who had already removed funds last week. The totals of the five major clients is almost $1.8 billion.

Later Monday, the New Hampshire Retirement System voted unanimously to end its relationship with firm, bringing its total funds divested to more than $2 billion.

The firm had $112 billion in managed assets as of Sept. 30. But some analysts think the response could lead to a domino effect of others fleeing the firm.

As it stands, Fisher Investments’ main clientele is individuals who are worth a combined $59.5 billion.

The backlash started after Alex Chalekian, CEO of Lake Avenue Financial,  posted a video on Twitter on Oct. 9 calling out Fisher for statements made during the Tiburon conference. He said the money manager’s lewd remarks compared gaining a client’s trust to soliciting girls at a bar and “getting in a girl’s pants,” and he also made references to genitalia and even the scorned financier Jeffrey Epstein.

Organizers from the conference, in tandem with organizers from the Evidence-Based Investing conference where Fisher made similar comments in 2018, have barred Fisher from speaking at the events in the future.