I rate based on: (Most important - Least important)
Enjoyment - Characters - Story - Art - Music
That isn't to say that any of these elements aren't important, just that I prefer anime that I find fun or well written to have more meaning to me than anime that is technically impressive.
I tend to like anime with complex characters, interesting themes or ideas, and good tone control. Usually my favorites fall under "Psychological" or "Seinen". I'll watch pretty much anything, though.
I try to only rate anime I've seen at least half of, and have a good idea what the rest will be like. I also don't usually read manga, because something about being almost animated but not quite bothers me a little. I usually only read manga that hasn't been adapted / has only been partially adapted. I also play some visual novels, but I consider it more of a guilty pleasure. Most of the visual novels I play are trashy romance / otome / yaoi games.
In true weeb fashion, I'm also learning Japanese. I know a little, but am nowhere near fluency. (N4-N5 Level)