The resignation of John Setka's deputy over "irreconcilable differences" with the embattled union leader may have led to the end of a key Labor faction, according to union and Labor sources.

Construction, Forestry, Maritime, Mining and Energy Union Victorian assistant secretary Shaun Reardon quit on Thursday for reasons relating to his "personal values" after Mr Setka defied calls by a dozen unions to stand down over charges of harassing a woman.