People who "like anime" also tend to be really into it. People who have casually seen an anime or two but who don't really think to bring it up are fine. People who "can't shut up" about their interests get marked as acceptable bullying targets.

While that does tie in with like neurodivergence and special interests, there's also a culturally "acceptable" threshold for what interests are reasonable versus abnormal and what's the proper amount of investment to place in those. For example, people who live, breathe, and love their jobs are also considered freaks, even though it's generally acceptable to like your job. I think having strong investment in stuff is a vulnerability that bullies like to prey on, because obviously if you care passionately about something, it's easy to exploit.