After facing a backlash over its decision to verify some of its extremist users, Twitter temporarily stopped its verification process last week. Now, just hours after announcing a review of the program, the site has started removing the verified check marks from white supremacist accounts.

Some people view Twitter’s verification badge as an endorsement by the company, but it claims the feature is simply there to confirm a person is who they say they are, and not one of Twitter’s many impostor accounts. This doesn’t explain, however, why it chose to remove former Breitbart tech editor and controversial figure Milo Yiannopoulus’ badge in January 2016 (he was permanently banned a few months later).

Twitter now admits that verifying accounts can give the wrong impression, and that the situation worsened when it allowed anyone to apply for verified status last year. “Verification has long been perceived as an endorsement,” the company tweeted. “We gave verified accounts visual prominence on the service which deepened this perception. We should have addressed this earlier but did not prioritize the work as we should have.”

“This perception became worse when we opened up verification for public submissions and verified people who we in no way endorse.”

Accounts will now lose their verification checkmarks for behavior “on and off Twitter,” which includes promoting hate and violence; threatening people on the basis of race, ethnicity, national origin, sexual orientation, gender, gender identity, religious affiliation, age, disability or disease; supporting hate groups; harassing others and violent behavior.

Two of the first Twitter users to lose their verification status were white supremacist Richard Spencer and Charlottesville ‘Unite the Right’ protest organizer Jason Kessler. It was Kessler’s verification last week that led to Twitter halting the process and introducing the new rules. He claims the site “changed their verification policy just to be able to censor me.” Other users to lose their status include Luara Loomer and Tommy Robinson, founder of the UK’s far-right English Defence League.

Some Twitter accounts have now been completely banned. Tim Gionet, better known by his username BakedAlaska, was removed from the platform. A spokesperson said the ban was related to “repeated and/or or non-consensual slurs, epithets, racist and sexist tropes, or other content that degrades someone.”