Anime scores - amount of votes

Feb 24, 2021 2:20 PM

Offline

Apr 2017

According to first screenshot of original post - https://myanimelist.net/forum/?topicid=1824703 - amount of votes include "troll votes". However it still doesnt match troll votes from calculated total of score stats (from the screenshot on the post 1 004 005 compared to 1 057 797). I did more calculations over some time and something is still not right. Looking at anime at the top with large number of votes - Fullmetal Alchemist: Brotherhood.
Current displayed number of votes is 1,372,393
Current calculated number of votes is 1,445,438
That's ~5% votes difference btw, so rather significant.
Data i took more than 2 weeks ago:
Displayed number of votes was 1,343,154
Calculated number of votes was 1,420,176
That means displayed number of votes right now is still a lot smaller than calculated number of votes more than 2 weeks ago, even after it was updated.
I thought maybe that 5% difference is number of troll votes not included in weighted score, but after some calculations it seems highly unlikely. Is there some other explanation why there's such a huge difference in votes?

Feb 24, 2021 2:26 PM

hacker09

Offline

Dec 2019

3353

I did that math a couple of times too to make this
https://greasyfork.org/en/scripts/413661-show-the-average-score-for-animes-mangas-instead-of-n-a

My script includes trolls voters and also weights their scores...

The only difference is that MAL hides and doesn't weight the trolls.

Click here to see My Tampermonkey Scripts For MAL

If you like my work, please consider supporting it!
https://www.patreon.com/hacker09

Feb 25, 2021 3:26 AM

Frq

Offline

Apr 2017

This script only works for N/A anime/manga so not what i was looking for.

Feb 25, 2021 10:42 AM

hacker09

Offline

Dec 2019

3353

I know that you are not looking for that.

But you can check the codes if you know how to code and check the math I did...

Click here to see My Tampermonkey Scripts For MAL

If you like my work, please consider supporting it!
https://www.patreon.com/hacker09

Feb 25, 2021 1:43 PM

Roevhaal

Tail On!

Offline

Aug 2018

2219

20% rule? votes from people who haven't watched atleast 20% of the episodes don't count.

Feb 25, 2021 2:49 PM

Frq

Offline

Apr 2017

Based on the explanation of weighted score in here https://myanimelist.net/info.php?go=topanime number of votes including 20% rule is not displayed in public and only used to calculate the weighted score. I'd love to see staff member confirm that though

Feb 25, 2021 2:58 PM

Frq

Offline

Apr 2017

hacker09 said:
I know that you are not looking for that.

But you can check the codes if you know how to code and check the math I did...

I don't know how to code, looking at it right now but can't figure out the formula. If you could show me how to calculate it on any anime/manga example that would be very helpful.

Feb 25, 2021 3:05 PM

hacker09

Offline

Dec 2019

3353

https://myanimelist.net/forum/?topicid=1874574

https://myanimelist.net/forum/?topicid=1876499

If you are that much interested in it you must carefully read everything written on these 2 topics...

Click here to see My Tampermonkey Scripts For MAL

If you like my work, please consider supporting it!
https://www.patreon.com/hacker09

Feb 25, 2021 4:18 PM

Frq

Offline

Apr 2017

Yeah, that 1st topic is exactly the stuff i was thinking about. I thought the displayed number of votes is the number of votes without "troll votes" and calculated number of votes from 1-10 scores include illegitimate votes. I assume massive majority of troll votes are 1 and 10 (maybe some 2 and 9 to try and fool the system).
I was trying to compare top 30 anime and manga on the list by checking individualy all illegimate votes and assuming they're all 10 and 1 checking the ratio between those scores to match weighted score after subtracting them. After that i would calculate each anime score again in 2 methods to find troll votes that i believe still made it through the system - assuming the ratio of 10 and 1 scores of caught troll votes i would remove larger part of votes until number of 1 scores is 0,1% of total votes - most common number of 2 and 3 scores for top animes, and for the second method - until number of 1 scores is 0,5% of total votes - most common for less popular but still highly voted positions (also more common in manga, it seems brigading is less of an issue there).
However i couldn't find the ratio for 1st anime i checked - Fullmetal Alchemist: Brotherhood - the calculations couldn't match displayed weighted score.
That's why i started this topic, because i thought it's strange that displayed and calculated number of votes doesn't add up, even though according to this topic https://myanimelist.net/forum/?topicid=1824703 there's no weighted score calculations displayed to the public - so that shouldn't include the 20% rule as well. Judging from the fact that 2 top anime dropped quite a lot about 3 weeks ago in the window of 2 days - without even dropping any episode then - i think that fake votes are still a significan't problem.

Feb 25, 2021 5:17 PM

#10

hacker09

Offline

Dec 2019

3353

MAL didn't tell anyone how their system works.
So don't matter the math you use, the program/code/script you use/make or the percentage you think it's or not safe to ignore, it's always going to be wrong and not correct...

Apparently the secret of mal is that they simply ignore all scores given by some specific accounts that they think are trolls, what accounts are these? What is the criteria to detect these accounts? Nobody knows, so it's not correct to just assume that percentage X are always trolls... The math you do will always be wrong and not 100% accurate.

Click here to see My Tampermonkey Scripts For MAL

If you like my work, please consider supporting it!
https://www.patreon.com/hacker09

Feb 25, 2021 6:02 PM

#11

Frq

Offline

Apr 2017

Oh, i know that my calculations wouldn't be 100% accurate. I think that current system is somehow flawed and by checking different outcomes from my assumptions i could shed some light on how big of a problem it is. I was planning about writing small article on reddit about it to make this subject more popular and maybe someone would find a solution or mal staff would notice it.

Last answer from lead admin about this problem - https://myanimelist.net/forum/?topicid=1824703&show=1200#msg61923647 shows they're not going to work on it until they deal with favorite character botting. I don't know how much of an issue is that, but in my opinion accurate scoring of a series is more important, considering it correlate with number of sales and giving creators more or less money they possibly would deserve.

Feb 25, 2021 7:59 PM

#12

hacker09

Offline

Dec 2019

3353

That's true and interesting, but a lot of people have tried to do this before without success really, mal never really cared about them or their opinions

Click here to see My Tampermonkey Scripts For MAL

If you like my work, please consider supporting it!
https://www.patreon.com/hacker09

Feb 26, 2021 1:06 AM

#13

Jhiday

Offline

Apr 2010

2498

Frq said:
Is there some other explanation why there's such a huge difference in votes?

The 20% rule does seem to be enough of an explanation.

There are currently about 300k members who have marked FMA:B as started but not completed (Watching, On-Hold or Dropped). It's not unreasonable for about 75k of them to have already rated the anime, while marking it with less than 13 episodes watched. That number seems fairly consistent (and even dropping slightly) between your two snapshots, so I don't see any issue there. All those numbers feel perfectly normal for a hugely popular and widely available anime that takes a while to watch through.

Yes, the fact that the total number of scores (which excludes those not meeting the 20% threshold) does not match the individual score stats (which do include them) makes it pretty much impossible to double-check score calculations and see how much troll votes make an impact. This was already the case for years even before troll votes started getting automatically silently ignored (i.e. both sets of numbers include them despite them not counting for score calculations).

Since MAL staff have made it clear that they don't want troll scorers to be able to gauge whether they're "counting", it's not too surprising they've made it basically impossible for regular users to do it either (and have zero plans to change that). You just have to trust that the system works.
(And given how little the FMA:B score has actually moved over the last few months, it does seem to mostly be the case.)

[AWC '16] - [AWC '17] - [AWC '18] - [AWC '19] - [AWC '20] - [AWC '21]

Feb 26, 2021 9:51 AM

#14

Frq

Offline

Apr 2017

Jhiday said:
The 20% rule does seem to be enough of an explanation.

I checked the math on 3 different series. I took all individual votes and counted the one with episodes watched less than 20%. I checked positions with total votes (including not scored) less than 1000 because i'm too dumb to write a script for it so all was done manually.
Results:
Ougon Senshi Gold Lightan
Displayed number of votes = 121
Calculated number of votes = 158
Difference = 37
Difference from individual votes = 39

Time bokan Series: Gyakuten Ippatsuman
Displayed number of votes = 150
Calculated number of votes = 170
Difference = 20
Difference from individual votes = 21

Mini Van 2nd Season
Displayed number of votes = 113
Calculated number of votes = 143
Difference = 30
Difference from individual votes = 31, i should note that 2 people voted more than 100% of episodes, i assume those should be removed too, in that case difference is 33

Results are quite close, but never 100%. None of the votes with less than 20% watched were added recently, so it's not a delay in voting. Not sure what to think about it.

See All Discussions

» How do you get anime list data into XLS format for MS Excel? MaelwysSC113 - May 13, 2016	3	by davidtung »» 44 minutes ago
» Change password IAMQT - Dec 24, 2014	12	by traed »» 1 hour ago
Sticky: » Inactive Username Request Thread ( 1 2 3 4 5 ... Last Page ) Kineta - Sep 21, 2015	3399	by Sethispr »» 5 hours ago
» Problems when rewatching an anime Veigar4 - 7 hours ago	2	by goodtasty »» 6 hours ago
» 2FA not works Dhanamerdeka - May 5	4	by traed »» 9 hours ago

Anime scores - amount of votes

More topics from this board

» How do you get anime list data into XLS format for MS Excel?

» Change password

Sticky: » Inactive Username Request Thread ( 1 2 3 4 5 ... Last Page )

» Problems when rewatching an anime

» 2FA not works

MoreTop Anime

MoreTop Airing Anime

MoreMost Popular Characters