| I personally enjoy reviewing anime, its something that I found interest in doing and in general I enjoy, however lately I am coming to terms with the fact that the reviewing process is entirely broken and nobody should take the reviews on this site seriously. This is coming from someone who has written reviews, not as many as the prominent reviewers, but I have written a small number of them. I will explain why the reviewing process in this post, if you don’t like reading I have a TLDR at the end. I will also have a list of proposed solutions, not that I expect the overworked volunteers on this site to actually implement them. Reviewing is totally broken on MAL for the following reasons, dishonest scoring for popularity, frontloading reviews, toxic review community treating it like a competition, bots, a shortage of moderation. Above all things, a review should at least be a reviewers honest view on what they review, I have personally given majority of things I review 7s or 8s as that’s what I rate most anime, 7s or 8s. However I have noticed that most reviewers choose extremes for reviewing based on the audience of anime, if the anime is unpopular in the community or has a general hate boner towards it, then reviewers will choose a 1 or a 2 even if on their own list they gave the show a 4-5. If a show has a cult following of dedicated shillers, a cynical reviewer will inflate their review score and give the show a 10 even if they clearly have it as an 8 in their personal list, this is to guarantee the herd of shillers note their review as “helpful”. This is starting to become a problem because the most “helpful” and supposedly serious reviews are actually just carefully crafted to be the most popular, its like steam reviews except the shillers don’t even get awards or steam shop points out of it, which makes me wonder why bother expressing opinions you don’t really hold. Another issue with the reviewing process is front loading reviews, this refers to users posting a review a nanosecond after the anime finishes airing in Japan, I will admit straight up that I am guilty of doing this myself. This becomes an issue as its very clear that only the frontloaded reviews actually get any traction on the site, yet most of the front loaded reviews don’t really take into account the final episode of a show, this is because even if you have seen the final episode, you likely are not going to add another 200 or so words to your review to address how the final impacted your view on the show. This is because you can’t really type fast enough to adjust your review before you post it after the show finished airing, chances are if you are not a native speaker of Japanese as well such as myself, the final episode you watched was RAW and you only half understood what happened. As Toxicity in the community is a big topic, I will dedicate several paragraphs to it. Toxicity in the community is noteworthy as well, many reviewers will actively sabotage the efforts of other reviewers if it means their review will be placed higher. For example if somebody submits a review but does it a second before the final episode thread is made, they will get reported and their review will likely be removed in a week or so, this is despite the fact the episode was likely fully aired. The reports are not coming from normal users, normal users are not sitting around review sections of popular shows trying to snitch on other reviewers, its obviously reviewers themselves who are reporting each other. Reviewers will also nitpick through your review and try to find a single mention of a spoiler that was not indicated and report you, this is why I have had to resort to putting a spoiler preface on every review I post even if the “spoilers” are basically non existent. Witty commentary on the direction of a series, discussion of any side topic outside of the anime being reviewed even if relevant, and other additions to a review are heavily reported by other reviewers to get the review removed. This is leading to a lot of boilerplate reviews being posted which lack any creativity or depth as the anime needs to be reviewed in a complete vacuum lest some overzealous clout seeking reviewer reports you. In addition, many reviewers are clearly building their own little cults and adding 100s of friends who exist as their little personal “find helpful” click army. I will not name names as its against forum rules to slander other users, but if you are in the review community in any sense, you know exactly who the users I am talking about are. Finally I would like to address that there just are not enough mods, there are clearly bots phishing for info in the reviews of many long running series, there are clearly abuses of the review system, and there is clearly a lot of toxicity surrounding the current reviewing process. Proposed Solutions: Well if you read through my lengthy bitching about the reviewing process on here, you will have noted I promised to provide some potential solutions, so here they are. 1. Adopt review decay over time Essentially, the top 4 reviews will not be weighted just by how many likes they have but by how recent they are as well. If a 5 year old review has 120 helpfuls and a brand new review has 100 helpfuls, the brand new review should be shown in the top 4 section instead. This will probably end the stupid clout chasing behavior I witness in the review community permanently, it will also provide reviews from demographics who are closer to the average age of the site in the long term, as new reviewers will come from a pool of new users most likely. 2. Goddammit get some more mods I am sure there are plenty of people who are willing to provide free labor for some forum badge and colorful name, just hire some more mods, it would sort out a lot of the problems. 3. Reviewing guidelines could use some discretion right now its being weaponized by users to sabotage each other, I get that review mods cant have a full on jury decision on every single review since its unrealistic, but try to add discretion and common sense into the guidelines. A review that goes off on tangents but still connects these tangent points to the main review shouldn’t be taken down if its relatively helpful. Spoilers shouldn’t be auto removed if they are super minor ones, for example noting that Asta is a magic knight or talking about terms only fans of a series would get like the sisters noise project in Railgun. Spoilers should also be connected to timelines, spoilers for the season 2 of a series don’t actually matter in a season 3 review, I don’t know why reviews which spoil season 2 get a season 3 review removed, its common sense that one shouldn’t read a season 3 review without watching a season 2 of a show. 4. Perhaps make it so that reviews for a new show are closed during the last 2 episodes and only are reopened 1 day after the final episode is aired. This would prevent frontloading reviews for first place. I don’t actually expect anything to come of this, I just wanted to express what I thought about the current reviewing process. TLDR, reviewers are dishonest about their ratings, they are chasing clout and rushing their reviews, the community is rather toxic in general. PS Mods if this is the wrong part of the forum to post this in, please migrate the thread to casual discussion or suggestions, whichever is more relevant. |
ACasualViewerJan 19, 2021 10:53 AM
 |
| I completely agree. Reviews are perhaps the feature in most need of a major overhaul. Personally, I'd want the default sort not to be Most Helpful or Most Recent, but to be completely random. That way it wouldn't matter when it's posted or how many people agree with it, everyone who makes a review would have as much a chance of theirs being read as everyone else's. |
| Well I for one already loved Lain. |
| I only go out of my way to read reviews if they're written by you. I never give much importance to reviews since I have watched enough anime to know if I'm gonna like something or not even before I start watching it. What I mean, is that I don't need to read a good review of something I like or a bad review of something I dislike because I don't need a strangers validation to like what I like or dislike what I dislike. |
 cunnysseur |
Kurgo said: My inclination is that you're taking this matter far more seriously than it has any right to be. Well I had some free time today and I don't really mind spending 15 min to write my thoughts on something, probably how I got into the useless hobby of writing reviews on stuff be it on Amazon / Steam / Reddit / MAL / Movie aggregation websites Kurgo said: 1) I suppose I see how it could be interesting (I've watched a fair few anime that have aged horribly even though they had pretty high scores) but, realistically speaking, not doable. You can't exactly (well, I'm assuming you can't, I don't know shit about how MAL is run/works) just add something like this at the blink of an eye. It'd probably involve a lot of work and I don't exactly see people clamoring from the rooftops for it to happen. Well this is a solution already present on many large websites, Steam being one of the most well known, if you look at any old game the top reviews are weighted by both recency and helpful score. Its the same with google reviews for restaurants and such. Kurgo said: 3) I mean, 99% of reviews I've read on this site HAD spoilers, from minor ones to pretty major ones (which were clearly indicated so no big deal to me). I don't get why all these reviews you mention are being taken down for small things that no one would really see as problematic (if you read a review for a s3 that spoils s2 and you complain, you're the idiot, there's no two ways about it) but again, it seems more of a problem of these braindead cliques and there's not much to be done about that. Mods being more logical on such matters would be welcome but I have no idea how the whole "review a review" system works myself. Having spoilers isn't against the rules if you note that it clearly has spoilers however it messes up the formatting for your review and makes it harder to write a good hook. Its what constitutes spoilers that really bothers me, since I have seen actual reviews taken down for the s3 review spoiling s2. As for your other points I don't disagree with them, adding a delay until the show is finished before posting a review would be great, but I expect the site to want preliminary reviewers which is why I went with my solution proposal. I also understand there aren't that many people who would respond positively to modding the reviews if they were approached directly, but at least having a recruitment run would be a good idea. Sure some current reviewers might finesse their way onto the team, but decent internal controls prevent abuse, maybe have 2 mods look at every report for example. |
|
| I always find the most negative reviews to be the most honest. the top voted/featured ones are always claiming the show is a masterpiece without flaws and the director is a god walking on earth. |
| 馬鹿げた倫理 全部ガラクタで |
| It's a flawed system, but it's the system we have, so I'm gonna use it. I no longer have the time to just watch anime on a whim, when I get to watch anime it's gotta count, and the reviews can help with that. I understand I'm not going to have the same opinions as every top reviewer, I understand the most 'helpful' review doesn't always mean it's the most reliable, however as flawed as that is I don't really have much outside of that. I've ran into too many good anime with generic or just awful synopsis, and you have decent sounding anime synopsis for shit like Gibiate. The system can be improved, but I'm not waiting for that impossibility to start using the reviews. Even with all these issues, using the reviews to help me decide whether to watch an anime still works to a certain degree, which is sure as hell much better than me just guessing. I'm not exactly taking their points very seriously, I'm only using the most base information from all these top reviews to try and decide whether I should give something a watch. |
 |
| "1. Adopt review decay over time Essentially, the top 4 reviews will not be weighted just by how many likes they have but by how recent they are as well. If a 5 year old review has 120 helpfuls and a brand new review has 100 helpfuls, the brand new review should be shown in the top 4 section instead. This will probably end the stupid clout chasing behavior I witness in the review community permanently, it will also provide reviews from demographics who are closer to the average age of the site in the long term, as new reviewers will come from a pool of new users most likely." I've seen some reviews with less "Helpful" mentions than others on the top4 shown that were placed higher so I think that there is already something like that put in place. |
 |
| Reviews are pretty ehh and that's the best it can really be. I mean for any single anime/manga, if you haven't seen/read it, it's kind of hard to tell whether the review is worth it's salt. It's the same way on any other place where people review things, from RYM to letterboxd, so I don't see the reviews changing anytime soon. |
| Written reviews of anime in general I find somewhat pointless. It's not like a review of a physical product, software, or a video game, where reviews can actually provide useful information and insight. In anime reviews, the only thing in the review that I can't figure out elsewhere, is your opinion, which interestingly enough is the thing I care the least about. Basically, anime is far too subjective, and the concept of a "review" is not an opinion piece. Movie reviews suffer the same, though that might even be worse when you have "critics". When deciding from literally hundreds of anime to watch next, I've often looked at reviews to help me determine what is best to watch. What I've found is negative reviews have pushed me away from watching a show I eventually enjoyed, and positive reviews led me in to watching a show that was actually terrible. So I've tried to actively avoid this trap recently. I usually believe this is because people rate a review as helpful as a form of vindication. It may be having four positive reviews on a show generally perceived as bad, or four negative reviews on a show generally perceived as good. People are contrarian. Rant about useless reviews out of the way, just get rid of the most helpful system. It clearly doesn't work. There are some god awful terrible written reviews that are "the most helpful", the only way they could've gotten there is by being first or people again voting for it as a form of vindication. In a perfect world preliminary reviews wouldn't exist. The problem is that this isn't a perfect world, and you can justifiably review One Piece after watching 800 episodes. How MAL could functionally differentiate between that and reviewing a one cour seasonal after one episode I'm not sure. At the least, you shouldn't be able to review a completed series without finishing it. I can't believe there are reviews that are "the most helpful" with 2 out of 24 episodes completed. There's a reason reviews for Amazon verify you actually bought the product. |
  |
| I almost never read reviews to begin with, especially not before I've seen an anime. |
     |
| I only read reviews to see what other people think about the stuff I watch, it was never a deciding factor for me to start watching a series or anything. |
| Who are you and why do you show your hostility towards a complete stranger whom you've not once spoken with before. Are you seriously asking to get blocked? Well, if that's what your intent is; to tempt me into throwing hands with someone as lowly and insignificant as you, then i may grant your wish provided you articulate yourself a bit better when trying to spite a person of my wavelength. |
| All your complains have been voiced before by multiple people already, myself included. It ain't changing :^) You're talking to a brick wall here, buddy. Alas, a brick wall on the wrong forum section. |
BiDiGiN said: "1. Adopt review decay over time Essentially, the top 4 reviews will not be weighted just by how many likes they have but by how recent they are as well. If a 5 year old review has 120 helpfuls and a brand new review has 100 helpfuls, the brand new review should be shown in the top 4 section instead. This will probably end the stupid clout chasing behavior I witness in the review community permanently, it will also provide reviews from demographics who are closer to the average age of the site in the long term, as new reviewers will come from a pool of new users most likely." I've seen some reviews with less "Helpful" mentions than others on the top4 shown that were placed higher so I think that there is already something like that put in place. Eh, no. You probably saw the preliminary reviews which can end up having higher helpful votes than the final reviews but it still will not appear in the front page of the anime because the front page prioritizes final reviews over prelim reviews no matter the vote count. _FRB_ said: Written reviews of anime in general I find somewhat pointless. It's not like a review of a physical product, software, or a video game, where reviews can actually provide useful information and insight. In anime reviews, the only thing in the review that I can't figure out elsewhere, is your opinion, which interestingly enough is the thing I care the least about. In a perfect world preliminary reviews wouldn't exist. The problem is that this isn't a perfect world, and you can justifiably review One Piece after watching 800 episodes. How MAL could functionally differentiate between that and reviewing a one cour seasonal after one episode I'm not sure. Point 1.) Anime reviews, like long-piece movie or series reviews in Youtube and elsewhere can introduce new appreciation of narrative tricks and framing, among other things when done right. The caveat is when done right. Point 2.) Preliminary reviews (the anituber version is called first impressions) do need to exist. It helps drive seasonal conversations and more importantly, a surge of positive acknowledgement is one of the ways an underwatched series float into the seasonal conversation and gain traction. Watching everything is impossible so by trusting in reviewers with the same mindset, favorites, and values as them, people can eliminate the weeds of the seasonals and zero in on those with potential. Point 3.) Your Amazon comparison is useless. Yes, you review Amazon products after purchasing so you can actually evaluate the product. You consume, you review. Now, what do you call consuming 4 episodes of a 12 episode show? Its still called consuming. In 4 episodes, you get a taste of a product enough to form an informed opinion about it. And when 12 episodes are done, the system automatically hides your initial review to make way for the reviews that took into account the whole cour. |
 You gave up your freedom of speech when you clicked Agree to the User Agreement This is not a public platform. My gaze is the measure of all things: I stopped considering "anime" a helpful tag Recommended Essays Exploring Actually Excellent World-Building |
| @Eanki Literaturenerd's review of "Ashita no Joe 2" with 19 helpful mentions appears as the 3rd review over Warriors_of_God's review which has 44 mentions. Chaf's review of "The World is Mine" with 9 helpful mentions appears as the 1st review over D3va's review which has 10 mentions. Those are the 2 examples that first come to mind but I am sure to have seen more. Both aren't preliminary and are from last year. |
|
| The main problem with reviews is that most of them don't actually evaluate anything. |
| Reviews are made with subjectivity doesnt matter if they say somethint about the plot or characters, people like just beeing supportive somehow, thats maybe the main issue lol. Thats why you can upvote a review if its worth enough, people just like to give support dont matter if isnt reviewing about the sound characters whatever, its like "yes i like this anime, dont worry give it a try" i dont see a problem becuz upvotes exist...bfor that reason .... |
 |
| If someone finds a negative review useful still gonna be okey , there is always someone couldn't like a piece of the show so can see the negative reviews okey, thats why upvotes existed.. I only see bad, army of reviewers upvoting their own reviews thats ridiculous lmao... What the heck |
Lauty_coolJan 19, 2021 9:31 PM
| |
| Negative review of shows you don't like are fun to read. And I rely on reviews for some not so popular series or whose summary don't tell you much |
 |
| You get what you pay for. There is no incentive for good reviews, and no real editorial process, no minimum standard other than word count. So it's the creative writing 101 class test bed and ego stroking of opinion leaders you see. Writing good reviews is journalism and requires profound knowledge about making anime and telling stories. Most people on MAL confuse reviews with "my opinion is" blurbs. Don't expect miracles. The only realistic way to improve the ranking would be an AI of some kind. The other would be to pay writers and have editors classify random contributions into quality classes - i.e. throw money at it. |
 |
Eanki said: _FRB_ said: Written reviews of anime in general I find somewhat pointless. It's not like a review of a physical product, software, or a video game, where reviews can actually provide useful information and insight. In anime reviews, the only thing in the review that I can't figure out elsewhere, is your opinion, which interestingly enough is the thing I care the least about. In a perfect world preliminary reviews wouldn't exist. The problem is that this isn't a perfect world, and you can justifiably review One Piece after watching 800 episodes. How MAL could functionally differentiate between that and reviewing a one cour seasonal after one episode I'm not sure. Point 1.) Anime reviews, like long-piece movie or series reviews in Youtube and elsewhere can introduce new appreciation of narrative tricks and framing, among other things when done right. The caveat is when done right. Point 2.) Preliminary reviews (the anituber version is called first impressions) do need to exist. It helps drive seasonal conversations and more importantly, a surge of positive acknowledgement is one of the ways an underwatched series float into the seasonal conversation and gain traction. Watching everything is impossible so by trusting in reviewers with the same mindset, favorites, and values as them, people can eliminate the weeds of the seasonals and zero in on those with potential. Point 3.) Your Amazon comparison is useless. Yes, you review Amazon products after purchasing so you can actually evaluate the product. You consume, you review. Now, what do you call consuming 4 episodes of a 12 episode show? Its still called consuming. In 4 episodes, you get a taste of a product enough to form an informed opinion about it. And when 12 episodes are done, the system automatically hides your initial review to make way for the reviews that took into account the whole cour. First, you'll notice I took specific care to say "written" anime reviews, like in the form we have on MAL. Why you would bring up Youtube I don't know. But no, a review can't do those things, because similarly to how I don't believe a review should be an opinion piece, it also shouldn't be an analysis. Both of those things have their place, but its not in the reviews section on a MAL page. The point still stands that the only extra information that a review offers me is an opinion. Before watching, I don't read reviews for analysis, because I'm trying to experience those narrative tricks and framing for myself. I don't read reviews after watching, because I'm not looking for the vindication. What else is on offer then besides someone's opinion that I don't care about? MAL reviews also don't drive seasonal hype. There are impressions, discussions, videos, etc etc. All those things drive hype. And then there are reviews. They aren't in the same playing field. You don't need garbage preliminary reviews to get any of that. It would be delusional to think the rush to get a preliminary review out is creating hype for anyone except those actually making the review. Other than the point I already made in the first post of anime reviews being fundamentally useless compared to product reviews, the Amazon comparison is perfectly valid. You don't try a demo of a product and review it. You buy it. In the same vein, "you consume, you review" is not accurate. You don't review a 90 minute movie after watching 30 minutes of it. Or shouldn't be taken seriously if you do. The only remaining problem is as I mentioned: there are things you can justifiably review before the end, but 4 out of 12 episodes isn't among them. Finally, the system doesn't put preliminary reviews below completed reviews. I legitimately don't know how you came to this conclusion. I can look at the review page of just about any finished show, sort by most helpful (either weighted or all time, both work) and there preliminary reviews above completed reviews. |
|
| I don't agree with review decay, some old reviews may be good so why punish them for being old. But I agree with your rants, just take a look at Darling in the FranXX reviews, almost all gave it a 1/10, spoilers and fake information about the plot and characters and they are still there with hundreds of "helpfuls". I think we could implement something like, only users who have the anime in PTW or don't have the anime in their list could rate it "helpful", what's the reason to press the "helpful" on a review on urusei yatsura rated 9/10 if you already watched the anime and gave it a 10/10? (for example) |
    |
_FRB_ said: Finally, the system doesn't put preliminary reviews below completed reviews. I legitimately don't know how you came to this conclusion. I can look at the review page of just about any finished show, sort by most helpful (either weighted or all time, both work) and there preliminary reviews above completed reviews. You seem set in your ways. Let us just tackle the last one. Go to the front page of a finished anime. The first four reviews on the front page, aka in the business lingo the reviews that matter, are all reviews made by people who completed the cour and made the review. Nobody clicks on the review section. You click on the anime, you scroll down the synopsis, you see 4 reviews, voila! Those are the only reviews. As long as there are at least 4 complete reviews, any prelims will always be pushed to the back, to the second page, the page nobody bothers with. |
You gave up your freedom of speech when you clicked Agree to the User Agreement This is not a public platform. My gaze is the measure of all things: I stopped considering "anime" a helpful tag Recommended Essays Exploring Actually Excellent World-Building |
| too long didn't read, but yes, i don't care about reviews on mal. |
| I don't care, people here use numbers to measure art Story x/10 Characters x/10 Animation x/10 Enjoyment x/10 |
| If you're a fanboy, please don't waste my time. Watch more movies, please. Perhaps, this is hell. |
Eanki said: _FRB_ said: Finally, the system doesn't put preliminary reviews below completed reviews. I legitimately don't know how you came to this conclusion. I can look at the review page of just about any finished show, sort by most helpful (either weighted or all time, both work) and there preliminary reviews above completed reviews. You seem set in your ways. Let us just tackle the last one. Go to the front page of a finished anime. The first four reviews on the front page, aka in the business lingo the reviews that matter, are all reviews made by people who completed the cour and made the review. Nobody clicks on the review section. You click on the anime, you scroll down the synopsis, you see 4 reviews, voila! Those are the only reviews. As long as there are at least 4 complete reviews, any prelims will always be pushed to the back, to the second page, the page nobody bothers with. I won't disagree that the top 4 are obviously the most important, otherwise people wouldn't be gunning for it which is part of what makes reviews so terrible. The insistence that those are the reviews that matter is part of the problem. But I will disagree that no one ever looks at the review page. If that were true, no review outside of that top 4 would having any significant number of helpful ratings to show for its trouble. But they do. We know that the top 4 are not usually changing, so its not as though they get those votes from being visible on the main anime page. The only other exposure they get is if you are following what that person reviews, which I assume is not the case for many reviewers who are not in the top 4 often. Nonetheless, that's still leading you to a review page in the end. All in all, preliminaries being one extra click away doesn't make them less stupid. |
|
| I use reviews when deciding what to watch next from my ptw list, many of them are older shows, so it's useful to have someone's review to get an idea of what to expect. I don't tend to read reviews for airing/just finished shows due to the many problems you outlined, the reviews tend to be either very pro or very con, never in between. I like that you proposed solutions. Well written piece. |
 |
| Fuck reviews Fuck reviewers Fuck the people who read them and FUCK the people who decide to watch or not a show based on reviews |
 OBAMOS |
| Mods don't even ban bot accounts with troll reviews. Some time ago I've reported an empty troll account with a troll AOT review. The mod answered me that "it was taken care of" and that I shouldn't worry because they have an antibot system now blah blah. And guess what? That account wasn't banned. They took a report, they answered, but for some reason they didn't bother to manually ban/delete the account. And at the same time I get a ban warning just for calling fujoshi a cancer. After that I've lost all trust towards mods and site's boasted antibot features. Yeah they deleted that review, but a few minutes later the same account posted the same review again. And again. They kept deleting it and the bot kept reposting. That all is just a big massive BRUH. |
Nemo_NiemandJan 20, 2021 2:11 AM
 |
| Another thing to consider is that "Helpful" doesn't even work as intended, and never will. I maintain it should refer to reviews with good prose, research, and that make cogent commentary on the work itself. The foolish average user, though, deems anything that matches their own score as "Helpful." Everything eventually skews to the popular opinion, rather than the impartial analysis it should be. |
| Well I for one already loved Lain. |
ACasualViewer said: At the end of the day people want more "views". I don't see anything wrong with that. Also I "believe" the majority of people who click on the "helpful" button thing, just see the score and don't actually read the review and that is why you see a lot of 1/10 and 10/10 thereAbove all things, a review should at least be a reviewers honest view on what they review, I have personally given majority of things I review 7s or 8s as that’s what I rate most anime, 7s or 8s. However I have noticed that most reviewers choose extremes for reviewing based on the audience of anime, if the anime is unpopular in the community or has a general hate boner towards it, then reviewers will choose a 1 or a 2 even if on their own list they gave the show a 4-5. If a show has a cult following of dedicated shillers, a cynical reviewer will inflate their review score and give the show a 10 even if they clearly have it as an 8 in their personal list, this is to guarantee the herd of shillers note their review as “helpful”. This is starting to become a problem because the most “helpful” and supposedly serious reviews are actually just carefully crafted to be the most popular, its like steam reviews except the shillers don’t even get awards or steam shop points out of it, which makes me wonder why bother expressing opinions you don’t really hold. ACasualViewer said: OK but you're solution is...Another issue with the reviewing process is front loading reviews, this refers to users posting a review a nanosecond after the anime finishes airing in Japan, I will admit straight up that I am guilty of doing this myself. This becomes an issue as its very clear that only the frontloaded reviews actually get any traction on the site, yet most of the front loaded reviews don’t really take into account the final episode of a show, this is because even if you have seen the final episode, you likely are not going to add another 200 or so words to your review to address how the final impacted your view on the show. This is because you can’t really type fast enough to adjust your review before you post it after the show finished airing, chances are if you are not a native speaker of Japanese as well such as myself, the final episode you watched was RAW and you only half understood what happened. ACasualViewer said: For the first one it's already like that. I don't know the exact formula myself and I searched for it and couldn't found anything better than this:1. Adopt review decay over time Essentially, the top 4 reviews will not be weighted just by how many likes they have but by how recent they are as well. If a 5 year old review has 120 helpfuls and a brand new review has 100 helpfuls, the brand new review should be shown in the top 4 section instead. This will probably end the stupid clout chasing behavior I witness in the review community permanently, it will also provide reviews from demographics who are closer to the average age of the site in the long term, as new reviewers will come from a pool of new users most likely. 4. Perhaps make it so that reviews for a new show are closed during the last 2 episodes and only are reopened 1 day after the final episode is aired. This would prevent frontloading reviews for first place. Kineta said: so idk it's just MAL's average thing but I've seen several times that the most/more helpful review isn't on top(s) and instead a fairly new review is on top because of the weighted system. But you know why always the first reviews are on top? Because almost nobody bothers to read/find helpful any reviews other than the top 4 that are on the main page so the first ones getting more and more attention simply because of thatJosh said: Total Helpful Votes / Total Time Uploaded. So, # of votes per day, essentially. This is a rough calculation, so we'd like to do better in the future by actually counting helpful votes given over the last X days.Do you mind if I ask what the weighting formula is? For the second solution someone already suggested that before and as MAL doesn't change anything he just left MAL and migrated to AniList but he still write reviews also here because no one cares about reviews/forums on AL lol ACasualViewer said: This is the part I can't comment because I'm not a reviewer myself but the only advice I can give you is "talk to mods". The mods here actually respond to you for your problemsToxicity in the community is noteworthy as well, many reviewers will actively sabotage the efforts of other reviewers if it means their review will be placed higher. For example if somebody submits a review but does it a second before the final episode thread is made, they will get reported and their review will likely be removed in a week or so, this is despite the fact the episode was likely fully aired. The reports are not coming from normal users, normal users are not sitting around review sections of popular shows trying to snitch on other reviewers, its obviously reviewers themselves who are reporting each other. Reviewers will also nitpick through your review and try to find a single mention of a spoiler that was not indicated and report you, this is why I have had to resort to putting a spoiler preface on every review I post even if the “spoilers” are basically non existent. Witty commentary on the direction of a series, discussion of any side topic outside of the anime being reviewed even if relevant, and other additions to a review are heavily reported by other reviewers to get the review removed. This is leading to a lot of boilerplate reviews being posted which lack any creativity or depth as the anime needs to be reviewed in a complete vacuum lest some overzealous clout seeking reviewer reports you. ACasualViewer said: First, what's wrong with that lol? It's not like they became popular immediately. They wrote reviews for several seasons on many popular shows and then they became popular. And as far as I know you, you're an admin of a 2500+ discord server (and yes I know that Discord is actually full of ghost members because I'm one of them myself) and you can make yourself popular just by that server; If you think it's stupid to put your review on for example #announcements channel (and it is stupid) you can make a new channel that promote anyone's creations like what the MAL's discord have and put your review there.In addition, many reviewers are clearly building their own little cults and adding 100s of friends who exist as their little personal “find helpful” click army. I will not name names as its against forum rules to slander other users, but if you are in the review community in any sense, you know exactly who the users I am talking about are. And second, other than the 2 popular reviewers who did this you mentioned, I don't see anyone who is popular because of this "technique" in the Most Helpful Reviewers's page. Stark700 literally closed his FR, closed the comments and idk apparently you can't even DM him, it's off-topic but I can't understand what "hurt" him in the past that made him like this on the site but whatever; And he has his cult, his reviews are always in top 4. And Veronin don't accept FR and yet I dare to say that he has the largest "cult" among the reviewers on this site. Yes, having 100s of friends make you somewhat popular but it's not the best method you could use for making yourself popular. If you want to get popular just write reviews for every popular show for several seasons as quickly as possible after their final episode and you get popular at the worst after 2 years ACasualViewer said: I'm an outsider but I don't think this is a problem here. If you think something's wrong with the helpfuls you can actually report the review yourself and if that doesn't work you can make a post or topic or idk about it just like what CBF did here. And after that again you can just talk directly to mods about itFinally I would like to address that there just are not enough mods, there are clearly bots phishing for info in the reviews of many long running series, there are clearly abuses of the review system, and there is clearly a lot of toxicity surrounding the current reviewing process. 2. Goddammit get some more mods I am sure there are plenty of people who are willing to provide free labor for some forum badge and colorful name, just hire some more mods, it would sort out a lot of the problems. |
mhkrJan 20, 2021 3:00 AM
| I highly doubt you spend all your time going through review scores and comparing them to the users' lists with a sample size big enough that matters. Most likely you found a bunch of users that don't have any differences, and subconsciously ignored them before you found the examples that does. And in fact, it doesn't matter. Who cares if someone gives a review score a 1 instead of a 4, or a 10 instead of an 8? Having an exaggerated score is indication that the system is broken? That's like Trump saying the election was stolen because Georgia officials found 2 ballots filled out wrong. In other words, you're being a bit of a drama queen. |
| My subjective reviews: katsureview.wordpress.com THE CHAT CLUB. |
ACasualViewer said: I personally enjoy reviewing anime, its something that I found interest in doing and in general I enjoy, however lately I am coming to terms with the fact that the reviewing process is entirely broken and nobody should take the reviews on this site seriously. This is coming from someone who has written reviews, not as many as the prominent reviewers, but I have written a small number of them. I will explain why the reviewing process in this post, if you don’t like reading I have a TLDR at the end. I will also have a list of proposed solutions, not that I expect the overworked volunteers on this site to actually implement them. ... Hi ACasualViewer, On behalf of the review moderators team, I'd like to first of all thank you for the thoughtful post. We really appreciate your obvious passion for the review system. You raised many good points, and in fact a lot of them are issues that we often discuss amongst ourselves in the team. While some of your points have already been addressed by the current system, there are others where a good solution is not obvious so we are still thinking about them, and there also ones where we would like to address, but due to the limited developer resources and the huge backlog of changes still awaiting implementation, we are not sure when the changes can be realised. Let me go through your proposed solutions one by one: 1. Review decay over time This is actually already built into the system. "If a 5 year old review has 120 helpfuls and a brand new review has 100 helpfuls, the brand new review should be shown in the top 4 section instead" - I'm pretty sure this is the case already. Currently, you’re not seeing this not because that brand new popular review can't get into the top 4; it's that brand new reviews rarely garner that many votes, which is a much deeper underlying problem. This hasn't always been the case. I started posting reviews here over 10 years ago, and I can tell you from personal experience (and you can also see this on the review pages of old anime) that it used to be possible to write a review months or even years after a show had aired and still have it climb to the top of the pile eventually. The landscape has changed a lot since then, and I think one of the main problems is that these days, unless you have your own fanbase/circle, it's hard to get eyeballs on your reviews if you post them more than a day or two after the anime airs. There's so much content output these days - both in terms of anime and reviews - that people are quick to move onto the next new thing. I'm not sure there's much we can do to reverse this problem (which, incidentally, extends way beyond just anime fandom). After all, you can't really force people to read reviews they're not interested in - and anything we can do will probably require dev resources. 2. Getting more mods We do rounds of recruitment every year or two, but it's not that easy to find good candidates. Though there's no shortage of applications, obviously not everyone is suited to be a mod, and even ones who eventually get selected and become mods often burn out pretty quickly. 3. Moderating with some discretion This is another thing that exists already. To quote from the review guidelines themselves "Please note these have been written as general guidelines only. Not meeting one or two of the following points may not automatically result in the removal of the review. Administrators will consider reviews with discretion, and reserve the right to alter the following rules as they see fit to maintain a standard that does not devalue the feature's purpose." Regarding spoilers, the rule has lightened over time and these days reviews containing minor spoilers are generally edited to put in a tag at the top, with only the most serious cases of spoilers-containing reviews being taken down. If you believe you have been a victim of over moderation, you can communicate that directly with the mods or use the feedback mechanism provided by the report button. 4. Closing the reviews for new shows for a while after the show has finished airing. Any changes to the system will need dev resources to implement. In addition, there are design and technical difficulties around restricting posts to the review section (See this thread for further details: https://myanimelist.net/forum/?topicid=606383). At the moment, we manually moderate the reviews that are frontrunning the airing of the final episodes. But like you pointed out, we currently have a situation where everyone is racing to post after the final episode has aired, and for sure this is having a negative effect on the quality of the reviews on series as a whole. It's a concern we share and we are having ongoing discussions about what we can do about this. You *may* see changes to the rules in the future if we can agree on a workable solution. Enforcing a longer no-review time through manual moderation is one solution, but depending on how long we make this period, it can also hugely increase the workload of the mods as there'll be plenty of people posting soon after the show has aired and it will no doubt also lead to a lot more complaints about the heavy handed rules, complaints that mods would then need to spend more time to deal with. I hope this has given you some insight into our thinking behind the scenes. There are rarely easy solutions, otherwise we'd have implemented them already. All we can do is try to improve and evolve the system bit by bit. But rest assured that these are problems that we are very much aware of and they have not been overlooked. |
| Take off every SIG!! |
kekekeKaj said: Hi ACasualViewer, On behalf of the review moderators team, I'd like to first of all thank you for the thoughtful post. We really appreciate your obvious passion for the review system. You raised many good points, and in fact a lot of them are issues that we often discuss amongst ourselves in the team. While some of your points have already been addressed by the current system, there are others where a good solution is not obvious so we are still thinking about them, and there also ones where we would like to address, but due to the limited developer resources and the huge backlog of changes still awaiting implementation, we are not sure when the changes can be realised. Hello KekekeKaj, I really appreciate you getting back to me, I had not actually expected to get any feedback so thanks. Its also good to know that the review team is actively looking into solving some of these issues or are at least providing reasons as to why some of the issues are not realistic to resolve. kekekeKaj said: 1. Review decay over time This is actually already built into the system. "If a 5 year old review has 120 helpfuls and a brand new review has 100 helpfuls, the brand new review should be shown in the top 4 section instead" - I'm pretty sure this is the case already. Currently, you’re not seeing this not because that brand new popular review can't get into the top 4; it's that brand new reviews rarely garner that many votes, which is a much deeper underlying problem. This hasn't always been the case. I started posting reviews here over 10 years ago, and I can tell you from personal experience (and you can also see this on the review pages of old anime) that it used to be possible to write a review months or even years after a show had aired and still have it climb to the top of the pile eventually. The landscape has changed a lot since then, and I think one of the main problems is that these days, unless you have your own fanbase/circle, it's hard to get eyeballs on your reviews if you post them more than a day or two after the anime airs. There's so much content output these days - both in terms of anime and reviews - that people are quick to move onto the next new thing. I'm not sure there's much we can do to reverse this problem (which, incidentally, extends way beyond just anime fandom). After all, you can't really force people to read reviews they're not interested in - and anything we can do will probably require dev resources. Well this is actually a surprise, due to not actually seeing it happen myself I never thought MAL actually used decay ranking. I do see your point as others have raised in this thread, as very few of the 10000 plus anime categorized on this site actually have any staying power, chances are that most people are not posting reviews for seasonal shows well after they are finished airing which in turn leads to a stagnant top 4 even with decay. I considered recommending that the decay calculation be adjusted so its sped up but I assume with only one real developer, at least based on the thread you linked, this is not a top priority. I was wondering how dev intensive it might be to migrate the sort button to the top 4 reviews as well? So that people don't have to dive into the full set of reviews and can quick sort with their preferred category involving the top 4. Such as top 4 newest, top 4 reviews from people with similar taste, top 4 helpful, top 4 helpful age adjusted and such. I never really code anything outside of scripts in Excel so I don't actually know if this is a challenging thing to add. kekekeKaj said: 2. Getting more mods We do rounds of recruitment every year or two, but it's not that easy to find good candidates. Though there's no shortage of applications, obviously not everyone is suited to be a mod, and even ones who eventually get selected and become mods often burn out pretty quickly. 3. Moderating with some discretion This is another thing that exists already. To quote from the review guidelines themselves "Please note these have been written as general guidelines only. Not meeting one or two of the following points may not automatically result in the removal of the review. Administrators will consider reviews with discretion, and reserve the right to alter the following rules as they see fit to maintain a standard that does not devalue the feature's purpose." Regarding spoilers, the rule has lightened over time and these days reviews containing minor spoilers are generally edited to put in a tag at the top, with only the most serious cases of spoilers-containing reviews being taken down. If you believe you have been a victim of over moderation, you can communicate that directly with the mods or use the feedback mechanism provided by the report button. I will skip point 2 as I have seen it mentioned many times by other mods, and I definitely do understand that there will be a limited pool of qualified talent who are also willing to work for free, that's just how it is. For point 3, I know that there is an appeals process and things can and have been resolved through it. But there are some issues that I want to address. The over moderation through the use of coordinated reports still achieves its effects even if the review is unremoved. This is because the time the review is removed leads it to becoming irrelevant and the prior helpfuls are not restored. Now this isn't wouldn't be that big of a deal if the review could at least keep its original helpful score, but currently targeting reviews by other reviewers is a viable strategy to get in the top 4 and the appeals process doesn't change the positive outcome the bad actor achieved. There should be some sort of rectification mechanism, maybe previously removed but reinstated reviews keep helpful scores or are given an anti decay effect for a while? I recognize these are probably not viable once again with limited dev resources. kekekeKaj said: 4. Closing the reviews for new shows for a while after the show has finished airing. Any changes to the system will need dev resources to implement. In addition, there are design and technical difficulties around restricting posts to the review section (See this thread for further details: https://myanimelist.net/forum/?topicid=606383). At the moment, we manually moderate the reviews that are frontrunning the airing of the final episodes. But like you pointed out, we currently have a situation where everyone is racing to post after the final episode has aired, and for sure this is having a negative effect on the quality of the reviews on series as a whole. It's a concern we share and we are having ongoing discussions about what we can do about this. You *may* see changes to the rules in the future if we can agree on a workable solution. Enforcing a longer no-review time through manual moderation is one solution, but depending on how long we make this period, it can also hugely increase the workload of the mods as there'll be plenty of people posting soon after the show has aired and it will no doubt also lead to a lot more complaints about the heavy handed rules, complaints that mods would then need to spend more time to deal with. I hope this has given you some insight into our thinking behind the scenes. There are rarely easy solutions, otherwise we'd have implemented them already. All we can do is try to improve and evolve the system bit by bit. But rest assured that these are problems that we are very much aware of and they have not been overlooked. I never read that thread before, thanks for sharing it, I now know that the site is a bit locked for what can be applied to the reviews section. I do hope that some of this can be fixed over time, especially since these design and technical issues appear to be present from all the way back in 2013. Finally thank you for the well thought out response, it definitely has provided me some insight and although I don't really feel more secure about the review process due to the aforementioned issues, at least I know now that its on the review team's radar. |
ACasualViewerJan 24, 2021 5:34 PM
|
freizel said: What the fuck? There's a little check box in the top right corner labeled "Preliminary". Uncheck that box and leave other people alone. Who cares what you want? That's like saying "I hate Indian movies so Bollywood shouldn't exist."For me the only change" i really wish this site aplied is that for airing animes DONT ALLOW anyone To post a review IF THE ANIME IS NOT 3 EPISODES YET I seen this site full of weirdos posting a review for 1 episode anime ... Dude no, thats only an oppinion, write in a forum that is why is THERE.. not a review, What do you have to write about ? Is not even 3% complete .. many animes make a huge plot twist on episode 3, that why the "3 episode rule exist" And people have that mob mentality, if they see a 7 they start to score 7s or lower since their taste is better thereford they dont rate 7s This site is full of edgy kids Who insult you personally if you dont agree with them ... I DONT want a new anime with 10 reviews for ONE EPISODE ... Wait until 3 Not to mention MAL already deletes reviews under 3 episodes anyways. |
| My subjective reviews: katsureview.wordpress.com THE CHAT CLUB. |
| Way ahead of you, never cared about reviews in the 8 years I've been on this website. Sadly, the script I was using to hide the review section broke and I have to see them again when I go to an anime's page. |
on | I put a note in my calendar to check this thread again one day, I am actually surprised that about half of my concerns were dealt with, that's a lot of progress for a single year. |
|
| I agree with a lot of this. Personally, I don't think a review decay is the best option. I would be surprised if a new review reached 20 likes. 100 likes is unheard of. A simple solution I like is showing the best reviews from the past year rather than overall. Steam does that for game reviews (except that it's only from the past 30 days). This system means that all reviews have an effective lifespan of a year, allowing new reviews to get a chance at the spotlight. |
 |
More topics from this board
» Any openings from anime you don't watch or dislike that you like?Fukoku - Yesterday |
25 |
by nirererin
»»
22 seconds ago |
|
» Which anime are you taking your sweet time to complete?Rally- - Sep 27 |
43 |
by KyushuYokai
»»
1 minute ago |
|
» College Life animeNilyaaa - Sep 25 |
19 |
by valico
»»
5 minutes ago |
|
» 🖊️ Hall of Sensei: Who Teaches Best! ( 1 2 3 4 5 )nirererin - Sep 23 |
219 |
by ZeroMajor12
»»
6 minutes ago |
|
» Do you have a comfort character?swirlydragon - 25 minutes ago |
3 |
by nyugvo6
»»
6 minutes ago |