Rework the database algorithm.

These are different cases tho.

Weighted scores are used instead of the average ones, to avoid having an obscure entry with a single user who rated it 10/10 skyrocketed to #1.
Pingu in the City and Ishuzoku Reviewers were bombarded via bots and vote brigading/raids, and were countered with the illegitimate account detection algorithm.

To answer the question of whether the algorithm is "fair", we need to look at what the algorithm actually does, and MAL has helpfully provided a full explanation:

https://myanimelist.net/info.php?go=topanime


To quickly explain the formula: The weighted score is composed of two components: The average score of the anime and the mean score of all anime in the database. These are weighted by how many users have scored the anime. The more users have scored a show, the larger the weighting for the anime's own score is, and the less it is "corrected" by the database score.

There are some intransparent parts; we don't know how illegitimate accounts are filtered, but if we knew they'd just avoid the filter. We don't have an easy way of extracting all scores used for calculation as some are from private lists, and some in the statistics section of a show may not be counted due to being illegitimate or not included because of the 1/5 rule. We also don't have a solid number of the database's mean score as far as I know.

But now to put some numbers to this formula:
The minimum number used to be 50, but my understanding is that it has been raised to something in the range of 100.
The mean score of all titles is in the range of 6.3-6.5 according to numbers I've seen thrown around. I'll use 6.4.
So let's calculate a score or two with these approximate numbers.

Anime X has 1000 users and an average score between them of 7.00. Its weighted MAL score is thus:
(1000/(1000+100)) * 7.00 + (100/(1000+100)) * 6.40 = 6.3636... + 0.58181... = 6.94545... = 6.95

Anime Y has 400 users and an average score of 5.50. Its weighted MAL score is thus:
(400/(400+100)) * 5.50 + (100/(400+100)) * 6.40 = 4.4 + 1.28 = 5.68

Anime Z has 1,000,000 users and an average score of 7.91. Its weighted MAL score is thus:
(1000000/(1000000+100)) * 7.91 + (100/(1000000+100)) * 6.40 = 7,90920907... + 0,00063993600... = 7,909849015... = 7,91

So the MAL formula does pull shows up or down towards the average. But as you can see the effect is relatively minor, to the point of imperceptibility on the very big shows; this is all intentional of course.

Is this unfair? No, it simply accounts for the small sample size of scores and basically prevents a show from being pushed super high or low with very few votes, as that would not be representative of the community opinion (if such a thing even exists for a show that 300 people have watched).

I think that the main factor behind obscure shows generally being rated lower is what Alexioos has mentioned: Different people score differently, and thus not all scores are created equal, as such a thing would be impossible given how people watch anime and what anime they watch (it could only work if everyone scored every show, or the opposite extreme, only the shows that everyone has scored get rated).
Another factor is that many shows are niche for a reason. They often do not have a wide appeal, so you shouldn't expect them to get big scores from everyone who watches them, even if they're great shows in your eyes.

I do understand this, but they're not always niche for a good reason. An untranslated title, for example, can't be watched even by those users who are likely to rate it highly.

And what's the score distribution ?

The algorithm itself is fair. It's even a weighted based one, made to be fairer than the average one, which is exactly why it's used in almost every site proposing a ranking. The problem comes from the mentality of the people ratings entries.

You'll see more 10/10 scores on popular entries that you'll do on obscure one, because those who goes deep to find these obscures entries won't rate the same way as someone who's only looking at the mainstream entries. Trying to do something about that will straight up be manipulation, and be inequitable.

True.

We would need to compare the factual stats between an average and a weighted system, which is not truthfully possible on MAL due to the stats given to us containing the illegitimate accounts. Logically speaking, the weighted one should be more fair tho, which explain why it's used everywhere.



I'm not really familiar with vndb and their system, but looking at the site, the most popular entry is Saya no Uta with 17067 votes... with only this much votes, the threshold of 100 is 0.5% at the strict maximum, which is absolutely tremendous. (In comparison, it is 0.0035% on MAL, also at maximum.) That means the shift in score due to the weighted system will also be at that level...

Applying the system to only unpopular entries seems very weird tho... but as i'm not very good at maths or knowledgeable about such datas, and that i don't know their exact calculation, i'll just let someone else talk about that.



The weighted system is not done to counter trolls and bots tho. It's really just because a score made of the ratings from thousand of peoples is more trustful than one being made from only a single rating (or in the case of MAL, less than 100).

As pointed out by Darkerplayer above, the weighted system has almost no impact on very popular entries, and with the actual distributions of ratings on the less popular ones, these ones lose only a few 0.01 points. Only a handful entries gets degraded through the system, and as i've seen it, it is always a newly released entry of a very popular franchise (especially Movies, as they are, most of the time, not available quickly).

Edit : Even with illegitimate accounts counted, you can try to make a comparison between an average and weighted score of a random entry tho. That may show you the very little bit of difference it does on most entry.

But MAL does not even score anime with fewer than 100 legitimate votes. I am in favor of maintaining this restriction. I only wish to modify the formula for those entries that do have scores. I trust a system that displays the genuine opinions of 100 users over a system that handicaps them. For anime with few votes, the scores currently displayed are no different from lies.


That means there is no drawback to implementing my suggestion, and that a weighted formula is wholly unnecessary.

I don't feel the current system is helpful. It is difficult to tell good obscure anime apart from bad obscure anime when they all have the same low scores.