| Everyone is saying that MAL rating is bad and they never depend on it. But no one said how to improve it and they keep ignoring it. Personally I don't think it's that bad and only a few shows that I've watched are underestimated. Here's how the current MAL Rating System works: How are top anime/manga scores calculated? Only scores where a user has completed at least 1/5 of the anime/manga are calculated. Example: If you watched a 26 episode series, this means you would had to have watched at least 5 episodes (26/5.2)=5. We're using 5.2 instead of 5 so we get a whole number for "most" series. Formula Weighted Rank (WR) = (v / (v + m)) * S + (m / (v + m)) * C S = Average score for the Anime (mean). v = Number of votes for the Anime = (Number of people scoring the Anime). m = Minimum votes/scores required to get a calculated score (currently 50 scores required). C = The mean score across the entire Anime DB. Top Upcoming and Most Popular ranking calculation The Top Upcoming and Most Popular titles are calculated differently to above, and are instead judged by popularity. This popularity is measured according to the number of users who have the title in their list. The more users that have the title shown in their Anime or Manga list, the higher it will be ranked. So how do we improve it? |
 |
| How about the rating system stays as is? The system itself isn't broken. The main (and pretty much the only) problem I see people complain about with the scores is upvote/downvote bots fudging the scores. The MAL Dev team is working on a solution to deal with these though. You can read more HERE. Once that solution rolls out, I'm pretty sure that people will actually consider scores more. The only other complaint I heard was about "airing" and just "finished airing" shows having rapidly changing scores, but that's expected as more user input their scores. Scores usually stabilize after like 2-5 weeks after the last episode airs though. |
 Just some guy who likes anime |
| I appreciate the effort you put into this post but it's futile. MAL ratings are terrible and most of them are done by people who have only seen a dozen anime. They're never going to be accurate. So the problem isn't the system itself but the people who use it (including me). |
| Including user's average in the formula is the easiest method. Because people gives scores differently, some people use fullscale 1-10 honestly, while some other people only use 7-10 scale. And also MAL should remove "definition" of scores (7 as good or 3 as very bad), because it stopped me from using fullscale because I don't know how could a 1 or 2 are worse than a "very bad" show. |
"The Slave is the have-not, the oppressed one with nothing to spare. But because the Slave is in that despairing situation, having nothing, it can kill the Emperor !" |
| I don't think MAL rating formula that bad. You can't only watch 3 seconds of a 1-hour video on youtube and be counted towards the view count. You can't only read two chapters of a 500-chapter manga and think your rating matters. You can't watch only 3 episodes of a 400-episode anime and think you've grasped the quality of the whole series. I also don't think there is a problem with Top Upcoming and Most Popular series. Popularity has nothing to do with good or bad, it just matters whether you have heard it or not. So if someone puts the title in their list it means they've heard of it. As the saying goes "there is no bad publicity" lol I think it's better to suggest a way to prevent multiple account trolling/rating |
  . |
| MAL rating isn't bad, though I would add option to see, how is the show doing in spectre of art, sound etc. Ratings are based on subjective opinion and feelings of individual towards an anime. Though some complains with system, in the end ratings on MAL means jack shit. Anime rates only as high as you see it, no numbers change the fact you feel. Ratings on MAL moreless only shows, how many people feel in similar way you do. No rating system ever's gonna be perfect, we rate based on what we like, love, dislike and hate ( all subjective feelings), so there's no real need to improve this system, except adding the option I stated above, that would be nice. |
| First we need to ask what the purpose of the MAL rating system is. If the purpose is to give an idea of how enjoyable a show will be to an imaginary person who is some kind of mix of every taste, then it's working fine. My problem with that is it really doesn't benefit all that many people. Shows which everyone likes are more likely to rise to the top than shows which some people absolutely love and some people hate. So what you end up with is a list of the most milquetoast, broad-appeal shows which you aren't likely to dislike, but aren't likely to be the very best best show for the individual. I would prefer a system which indicates how likely a certain type of fan with a certain taste is going to enjoy a show. I have two ideas for achieving that purpose: 1. Have everyone rate genres by how much they enjoy them and give greater weight to scores given by people who are fans of the genre. Or give seperate scores, like there would be a score for SOL fans and a score for Battle Shounen fans. The problem with that is that people could lie first of all. But second, and more of an issue, is that the genres given to shows aren't descriptive enough of the actual shows. MAL would be better off employing a tagging system where there are tons of tags such as "all girl cast", "male hero", "big breasts", "tsundere love interest", even spoiler tags which we could hide or show like "happy ending", "consumated romance" and many many more. There are tons of tags people could think of and if the method of creating and applying those tags to shows was democratized then everyone could contribute to give a selection of tags which describe the show well. Also, it wouldn't be a case of either having the tag or not, but the more people vote for the tag, the more that tag applies to the show. Now we have a much better system of classifying shows. One could select tags they like. Maybe have like a favourite tags system or a rate the tags out of five for how much they like them. With that info, you could give greater weight to someone's score when the show fits their preferred tags and voila! You take out some of the issues of taste. 2. My other idea would take the tag system above, but instead of having people choose preferred tags or whatever, there would be an algorithm which figured out peoples preffered tags based on their scores and then give a user-adjusted score which would be better tailored for them. There would still be an overall average score, but preferably it would be mostly ignored. You could also have a general "tag preference-adjusted score" which indicates how highly a show is rated by people who tend to like the tags. If a show is rated low by people who generally like the major tags, then it's more likely to be genuinely bad and just not to peoples tastes. The opposite is true also. |
| “In individuals, insanity is rare; but in groups, parties, nations and epochs, it is the rule.” -Friedrich Nietzsche Aggregate scoring is bad for the anime fandom |
| If you want something you can depend on I'd say you should weight scores based on affinity (60% affinity = 0.6 weight, 20% = 0.2, 0% and below weighted at 0). It could show up next to the actual rating as a "personalized rating" so that it could be something you could rely on for recommendations instead of some arbitrary score of what people in general have given it. |
  |
Ny3 said: Yea it's crazy how the average user's average rating here is like 7.5. Like it literally says (average) is 5. If your average is not around 5 then you went wrong somewhere. Not really ; selection bias is a thing. If someone only/mostly watches anime they're likely to enjoy, it's not so surprising for their average to be higher. 5-ish user averages are usually the result of people who try watching (and rate) even anime they have a low probability of enjoying. (That's my case.) But it's completely unrealistic to expect a majority of MAL users to do that. Thanks to selection bias, most anime are watched mostly by people who enjoy them. That's why most "middle-of-the-road" anime get 7-ish average ratings (especially combined with the 1/5 rule), and you need to be quite remarkable to deviate from that either way. (It works a bit differently for not-watched-by-many anime, shorts and hentai, because the selection bias has other factors in play. But it's still a good rule of thumb.) |
Ny3 said: By definition of the average, or rather the median, half of what you watch must be below it. No, because you're dealing with two different sets here : the overall production of media, and the subset of anime that someone watches. Since you can't rate shows you don't watch (and that's a good thing !) nor those that don't conform to MAL definitions of anime, there's no reason for those two to overlap in any consistent way. And then there are other statistical biases at play, most obviously sampling bias. A MAL user who has only watched three anime, and enjoyed all of them, will obviously never get a 5-ish average. (It's technically also true the other way around, although I doubt many people hating on the three anime they did watch would even bother registering to MAL, outside of the fake duplicate accounts.) It's entirely possible to force one's own MAL average to be 5-ish because you think it's neater. But trying to coerce the millions of MAL users into conforming to your vision is a fool's errand. And it's not like one user's average rating is representative of their actual weight on anime scores (as dropped series rarely count), which is the topic actually under discussion in this thread. |
Jhiday said: I personally think people who have super high averages are far too pessimistic. "No matter what I watch, even if I absolutely hated it, I'll still give it at least a 6 since 5 is average and the average anime must be worse than anything else I've ever seen." This is a medium we all like, I don't think people should be so harsh on all the anime they've never watched. MAL's average score is 6.8/10.Ny3 said: By definition of the average, or rather the median, half of what you watch must be below it. No, because you're dealing with two different sets here : the overall production of media, and the subset of anime that someone watches. Since you can't rate shows you don't watch (and that's a good thing !) nor those that don't conform to MAL definitions of anime, there's no reason for those two to overlap in any consistent way. And then there are other statistical biases at play, most obviously sampling bias. A MAL user who has only watched three anime, and enjoyed all of them, will obviously never get a 5-ish average. (It's technically also true the other way around, although I doubt many people hating on the three anime they did watch would even bother registering to MAL, outside of the fake duplicate accounts.) It's entirely possible to force one's own MAL average to be 5-ish because you think it's neater. But trying to coerce the millions of MAL users into conforming to your vision is a fool's errand. And it's not like one user's average rating is representative of their actual weight on anime scores (as dropped series rarely count), which is the topic actually under discussion in this thread. |
|
| MAL system is different from democracy, that's a blatant fact. However, we can take into consideration about some quite conspicuous similarities between the two. That is, one problem with both system is ignorance of the voters. Of course, ignorance in terms of voting for anime is indisputably different from electing or voting for policies - it doesn't involve facts, knowledge and objectivity. Ignorance in MAL, however, stems from the fact that people are more than willing to spare their time creating fake accounts, alts and all that kinda stuffs just to boost their beloved shows and lower their despised shows. Also, ignorance derives from the attitude, which by that, I'm referring to conservatism, lack of openness, ambiguous hatred and braggadocio - in other words, they might not listen to criticisms of their favorite shows, likewise, blindly praise it, or scorn the shows they hate without thinking of others' opinions. All of these have led to scores that are unjust and without much explanation, whereas it should've been more contrived and thoughtful. So, the system of MAL ratings, I believe, isn't bad by any means, or at least I haven't found it to be bad. The improvement needs to start within the users themselves, which is technically impossible. There's no way to improve MAL rating system. |
| . . . |
| This thread is useless. The scores are only good for pushing your favorite show or bashing the shows you dislike. You could even put in just a 1/10 and use an episode count higher than 1/5 even if you dropped it early. Totally valid and allowed unless you don't make more accounts for that. Also the early drops and scores from people that drop with less than 1/5 could be interesting - if a show is that bad that it gets dropped a lot early ... then the score is biased by people that like the show. (We have the similar problem already for sequels where only people that liked and completely watched the previous stuff continue to watch.) And the people saying you'd have to score 5 on average for your personal list are funny. I'd rather play smoe games (MMO) than watching bad stuff - I only select the good things. 7-8 as mean score is totally normal if >= is "watchable" and you watch only that stuff. Then the distribution is betweena 5 and 10 mostly ... with mean at 7-8. Also the system againgst bots doesn't seem to work. Kimetsu no Yaiba still not normalized towards 8.5 or something around there ... and still at 8.95. (Must have been pushed a lot.) System that lets people select favorite genre and then couting their votes more (higher weight) also would be useless. The fans would just "push" their favorite stuff in their selected genre wile for other people it would be a lot harder to find the "true gems" that are good enough to entertain everyone (even the ones that are not hardcore fanbois of that specific genre). The only thing needed is a system to prevent anime with a lot amount of people that scored them to get "pushed" too easily. Afaik current formula already tries to prevent this by usng a mean score from all of the anime here on MAL. Could be "stronger" though ... I'd like to have something like the mean number of people that scored anime over the whole database instead of the fixed 50 there in the formula quoted in the OP post. (For example if only 5 anime were in the database scored by 100,200,300,400,500 people the 50 would be 300 instead.) Edit: Would need a different formula. Imo at 0 votes the mean score over all the anime in the DB should show and the weight of the scores for the anime is 0 (since there aren't any scores yet). At 300 in that example only the scores given to that anime should count. Between 0 and 300 the weight of the mean score of all anime in the DB shoudl go down where it should be x+y=1 with x the weight of the scores given to an anime. y the weight of the mean score over all the anime in the DB. x decreasing quick at the beginning and slower later (not linear). |
LuthandoriusNov 2, 2019 4:11 AM
| The biggest issue is the massive amount of bots I don't think the rating system is bad although I was bored so I did make a thread a few months back ranking MAL anime by Favorites percentage: https://myanimelist.net/forum/?topicid=1786779&show=0#msg57792810 Also there are some potentially cool statics that they could try adding like score distribution based on age group. It doesn't need to be updated every day, maybe once every 3 months. Vivator said: The MAL Dev team is working on a solution to deal with these though. You can read more HERE. How many more years is that gonna take? And how long before the bot creators adapt? Jim_Heart said: Including user's average in the formula is the easiest method. Because people gives scores differently, some people use fullscale 1-10 honestly, while some other people only use 7-10 scale. I can't imagine the extra calculations it would take to implement this. Doesn't seem feasible. Ny3 said: Yea it's crazy how the average user's average rating here is like 7.5. Like it literally says (average) is 5. If your average is not around 5 then you went wrong somewhere. No your statement is what's wrong. People filter shows based on the overall rating, genres, trailers and the general consensus to watch what they'll probably like. Unless someone has watched a whole lot of anime, is content with watching mediocre shows or they're terrible at filtering shows there's no reason their average rating should be 5. You're also not taking into consideration sequels, people aren't gonna watch sequels of shows they hate but they will for what they like. Not to mention OVAs and movies as well. A percentile system would be interesting though. |
<Insert clever quote> |
-InfiniteLoop- said: Jim_Heart said: Including user's average in the formula is the easiest method. Because people gives scores differently, some people use fullscale 1-10 honestly, while some other people only use 7-10 scale. I can't imagine the extra calculations it would take to implement this. Doesn't seem feasible. Why ? It can be done with addition of one more factor to the formula above. |
"The Slave is the have-not, the oppressed one with nothing to spare. But because the Slave is in that despairing situation, having nothing, it can kill the Emperor !" |
Jim_Heart said: -InfiniteLoop- said: I can't imagine the extra calculations it would take to implement this. Doesn't seem feasible. Why ? It can be done with addition of one more factor to the formula above. Well it depends on how MAL's database is structured I suppose but I'm sure MAL uses some tricks right now to optimize calculate the average score of an anime. Probably has all the scores of a particular anime in one table and people probably don't change their scores for an anime that often. However with average score the value fluctuates every day and the average score of a user is probably stored in a different table. This is because not every user watches every anime. It'll take time to just retrieve the average score of each individual user of that anime. Now do this for every single anime and you've got yourself a problem. Make an update to an anime score is easy because you probably just have to update that anime table. But when a users average score changes all anime that user has added will get affected which would be redundant so we have to isolate the data (by keeping in a separate table) and retrieve the data during calculation time, we have to query for each user. That'll add a lot of extra time to the calculation. In addition to that, performing a simple mean calculation is less taxing than some weighted formula. Also average scores contain decimal values as well unlike individual scores. That's my thoughts at least (explanation is kinda messy). I could be wrong but to me it does look like it'll increase the calculation time by a lot. |
<Insert clever quote> |
| @-InfiniteLoop- I understand but I think that is not really a problem, looking at the formula there is factor C there, probably it is more taxing than a weighted user average because it comprises entire databases. They can just update the "weighted user average" everytime the user update their list, and just use that when they update the score in top anime ranking. Anyway MAL is the most popular anime related site in the world right ? @Jese23 How would this work for someone new who only watched a few S tier shows and has high average of 8+? It just means that his weighted value is not as high as people with average 6 if both of them rated anime A with 9. |
Jim_HeartNov 3, 2019 5:33 PM
"The Slave is the have-not, the oppressed one with nothing to spare. But because the Slave is in that despairing situation, having nothing, it can kill the Emperor !" |
| The system is fine. It's users who bring inconsistency by not using the full scale or giving extremely biased scores. You can't 'fix' that by tampering with the formula. You can't fix people. But the average score is still a somewhat reliable piece of information once you know how to interpret the score depending on stuff like genre, obscureness and age. It's just that the same score for an artsy short from the 80s doesn't mean the same thing as it would for a mainstream seasonal of the 2010s. |
| I probably regret this post by now. |
| I think the simplest way to limit the amount of anime that the users can rate with each score. That would eradcate the top and bottom heavy accounts, and the users would have to actually have to think about what rating should they give to an entry instead of dropping 7+ on everything, because it was a decent enough background noise. |
  |
| I find people to be the problem behind horrible ways of scoring. |
  “The most shameless thing in the world is political power that can be inherited regardless of ability or talent!” |
More topics from this board
» Add the option to change profile favorites picturesk1rb - Oct 21, 2022 |
19 |
by k1rb
»»
4 hours ago |
|
Poll: » Add list setting to make notes private (on public lists)S_h_a_r_k_93 - Nov 12, 2022 |
25 |
by anonymate
»»
Apr 24, 9:57 PM |
|
» Add number of episodes and number of members in the advanced search.Yacine2104 - Jan 10 |
8 |
by Alexioos95
»»
Apr 24, 12:26 PM |
|
» Local Language districtskuroneko99 - Apr 22 |
5 |
by Luchipher-Zen
»»
Apr 23, 1:02 PM |
|
Poll: » Change picture of favorite character ( 1 2 )gehoti2822 - Nov 12, 2022 |
60 |
by AgravityBoy
»»
Apr 23, 9:09 AM |