UMkhonto weSizwe (MK) leader and former president Jacob Zuma said the ANC was not consistently applying its exit rules.
Mr Zuma was speaking at an MK party rally in KwaMashu, north of Durban.
The ANC's exit rules require members charged with corruption or other serious crimes to refrain from party and government activities.
Mr Zuma announced in December last year that he would support the MK party in this month's general election.
Mr Zuma says the current ANC leadership has lost the values on which the party was founded.
“Then some leaders started acting in a certain way because they thought they were the owners of the organization and not us, the members of the organization. They were creating all kinds of rules that they didn't know what to do with. They didn't consult anybody and this is what we're doing now. You don't know when you hear there's something new like this. Step aside, I don't see why you have to step aside. They tell people to step aside, but they don't step aside.”
Mr Zuma, on the other hand, remains a member of the ANC party but insists he will vote for the MK party.
“Some of you are wondering why he's still in the ANC, so I want you to know this. I'm not actually staying there, I'm just maintaining my membership. I'm not going to vote for it, I'm going to vote here. And if we win, I'm going to come knocking and say take back the ANC.”