I discovered a passion for the unique and fascinating characters and worlds, and uniquely Japanese culture found in anime a few years ago around the time I turned 30. Recently I like keep up with seasonal shows and whatever is going on in the community as much as possible.
I value the strength and range of emotions a show makes me feel from it's character interactions and atmosphere above all else. Emotions as in not just "feels" but amazement, wonder, suspense, arousal, cute response (can you believe there is no word for the emotion of finding something cute?) etc. This is as opposed to judging a show by it's technical prowess in terms of direction, art and animation quality. Those factors are pointless if they don't work together with other aspects of the show in a way that actually reaches me emotionally.
I have tried to use that as basis for how I rate shows. I take objective quality in to account in a small way, I believe there is a limit to how well someone can judge objective quality. I also try to take my taste in to account in small way. Nothing I don't enjoy can get more than a 7, but even if something really wasn't for me, I will bump it up if I can tell there is quality there because I think quality deserves to be rewarded even when it doesn't mesh with my personal taste.
While I don't believe popularity makes a show good, as highly popular shows tend to be formulaic, overly tropey, and paced to appeal to people with short attention spans; I also don't believe that objective quality can only be judged by a small elite group of people who are able to see shows in some special way. The latter attitude tends to lead to a preference for shows that avoid tropes for the sake of it (tropes can be done well), pretentiousness and genre snobbery. So, when thinking about objective quality, I like to find a middle ground between the populist and elitist approach.
10 & 9 (Masterpiece)
The basis of my rating system. I use a top down approach starting from these shows. Everything is compared to the tier above. My 10s are my absolute favourites that have left a lasting emotional impact and never or very rarely bored me. I consider my 9s masterpieces like my 10s. The difference is completly subjective in that I have the strongest possible personal connection to my 10s. I simply couldn't imagine giving anything but the highest rating to these shows and feel they deserve their own tier.
8 (Flawed Masterpiece)
While no anime is flawless, my 9s and 10s will tend to have flaws which actually work within the story and don't hold them back. But my 8s will start to have either one large or several small issues which are less forgivable and actually affect my enjoyment of the show. They still ellicited decent emotion consistently or moderately strong emotion at certain points. This is the highest tier where I consider objectively quality slightly which might raise a high 7 in to this tier sometimes.
The first tier where there are likely to be segments where I was genuinely bored. Unlike my 8s, these anime never reach the height of emotion and immersion that are present in my 9s and 10s. These are however still good - even very good - shows, for me, and I remember most fondly. This is the highest tier I will give to a show that I would say didn't acually enjoy as it wasn't to my tastes, but I could discern objectively high quality in some aspects.
6, 5 & 4 (Average)
Like my 9s and 10s, 6 through 4 are linked. 5s are average. Uninspired. Typical. When I actually finish these shows it's because they entertained me just enough to keep me interested and I was hoping they'd improve at some point but just never got to a point where I felt genuine affinity for them. Not bad, but not good. The 6s are much the same, only there was one or two qualities which raised them above average, and my 4s went the other direction.
3 & Below (Bad)
I will very rarely watch enough of a show I find to be less than average enough to feel comfortable rating it. And when I do rate shows below average I like to be really sure it deserves the low score. These are anime I came away from actually disliking and whether I give 3, 2 or 1 depends on how much negative feeling I have towards it. It is likely that every imaginable factor has combined for me to be so spiteful to these shows, including:
They are not to my taste.
Have no redeeming technical aspects such as animation quality.
Characters are a varying combination of bland, formulaic, lacking depth, unlikable when they're made out to be likable, unfunny when they're meant to be funny etc.
Elements are introduced which actively hurt the ability to tell the story.
Production issues such as being unfinished or rushed.
Plus loads of other stuff. To be honest it has to be a real dumpster fire for me to rate so low and most of the time I love anime too much to hate on stuff.
This is the system I find easiest to use when rating shows. It isn't perfect. You might be able to tell from how I describe it that if I rated on a scale of 100, the average of each tier would look like this: