Forum Settings
Forums
Must be a Club Member to Reply 
#1
Sep 19, 2013 4:45 AM
Offline
Joined: Jun 2010
Posts: 3
Sorry if I should've asked this elsewhere, but I wasn't sure where it should go.
Is there a way to request a recalculation of stats and stuff? I can't seem to find an option anywhere, and I only really fixed my mangalist after it was initially done.
 
#2
Sep 19, 2013 5:59 AM

Offline
Joined: Jul 2009
Posts: 393
When you visit your profile, it gets added to the queue and will be refreshed if your data is older than 24 hours. Since the queue is a bit long (2000+ users), this could take a while.

rr- and I can refresh your profile manually, you can view it now: http://mal.oko.im/rakiru
 
#3
Sep 19, 2013 7:07 PM
Offline
Joined: Jun 2010
Posts: 3
Ah right, I see. Is there any way to see where you are in the queue?

Thanks for refreshing it, and thanks for the great site.
 
#4
Sep 22, 2013 7:08 PM

Offline
Joined: Mar 2013
Posts: 60
I'm also wondering what rakiru just posted about how to know where you are in the update queue.

Love the site!
 
#5
Sep 30, 2013 4:17 AM
Offline
Joined: Sep 2012
Posts: 3
Good to see I am not the only one around with these concerns. I understand you don't have enough resources to enable realtime update of stats. A couple of suggestions:

- Some way of indicating the position in the update queue and a rough ETA would be good feedback for the users and prevent them from refreshing constantly.

- Also, could you please let us know if each time a user refreshes the page, is the position in the queue retained or is the user added to the back of the queue? For some reason latency in my profile updates runs into days
 
#6
Sep 30, 2013 4:40 AM

Offline
Joined: Nov 2010
Posts: 87
jangofetts said:
Good to see I am not the only one around with these concerns. I understand you don't have enough resources to enable realtime update of stats. A couple of suggestions:

- Some way of indicating the position in the update queue and a rough ETA would be good feedback for the users and prevent them from refreshing constantly.

- Also, could you please let us know if each time a user refreshes the page, is the position in the queue retained or is the user added to the back of the queue? For some reason latency in my profile updates runs into days


Your position stays the same whenever you refresh the page, otherwise it would be totally senseless. Waiting time is increasing due to big queue sizes and long communication time with MAL - we have no control over these and spamming MAL with too many parallel requests would be considered harmful. You can see queue size here: http://mal.oko.im/s/globals
 
#7
Oct 3, 2013 7:18 PM

Offline
Joined: Mar 2012
Posts: 18
My issue is that I never keep my computer on for 15 hours at a time.. Since the queue started existing; I haven't had a working mal graph.
<img src="http://i1207.photobucket.com/albums/bb480/Yumekichi11/Picture%2018/10643684.gif" />
 
#8
Oct 4, 2013 12:46 PM

Offline
Joined: Dec 2012
Posts: 215
I use MALgraph every day, and yet it only gets updated every 2-7 days. Bring back however it used to get refreshed, this doesn't seem to work too well.
Never forget! Don't believe in you who believes in me, don't believe in me who believes in you, believe in you...who believes in yourself!
 
#9
Oct 4, 2013 1:22 PM

Offline
Joined: Jul 2009
Posts: 393
No. This isn't so easy when you have a database that is tens of millions rows long.

The system you want to "bring back" was very slow, unstable and caused server crashes. Remember "500 Internal Server Error" messages that wouldn't disappear for two days straight?

With a queue system we can control the server load, while making sure the previous data is still available for our users. What's better: seeing stats generated three days ago, still perfectly available, or seeing a blank page you can do nothing with?

rr- is trying to find a way to make it faster; he's split the database to 64 parts that will update a lot quicker. Please give us some more time to get it working.
Modified by fri, Oct 4, 2013 1:28 PM
 
Oct 4, 2013 1:57 PM

Offline
Joined: Dec 2012
Posts: 215
fri said:
No. This isn't so easy when you have a database that is tens of millions rows long.

The system you want to "bring back" was very slow, unstable and caused server crashes. Remember "500 Internal Server Error" messages that wouldn't disappear for two days straight?

With a queue system we can control the server load, while making sure the previous data is still available for our users. What's better: seeing stats generated three days ago, still perfectly available, or seeing a blank page you can do nothing with?

rr- is trying to find a way to make it faster; he's split the database to 64 parts that will update a lot quicker. Please give us some more time to get it working.


Thanks for explaining, I didn't have many problems with the old system but if it wasn't working properly then the new one's probably better.
Never forget! Don't believe in you who believes in me, don't believe in me who believes in you, believe in you...who believes in yourself!
 
Oct 4, 2013 11:39 PM

Offline
Joined: Dec 2012
Posts: 309
Oh thanks, I was about to ask this myself lol.
"When life gives you lemons, make orange juice, then sit back and watch the world wonder how you did it."
 
Oct 18, 2013 8:34 AM

Offline
Joined: Mar 2013
Posts: 60
Eagle1337 said:
My issue is that I never keep my computer on for 15 hours at a time.. Since the queue started existing; I haven't had a working mal graph.


Do you have to keep your computer on for 15 hours? My understanding is that once you're in the queue, you're in the queue. I seem to remember closing the tab and opening it later to find myself--logically--having moved up in line. I certainly haven't kept my window open the entire time while waiting for things to update.

I guess my main thing is that you don't NEED everything to update instantaneously. It's not like all your data will get deleted if it takes more than a few hours to get through the queue.
 
Oct 18, 2013 11:46 AM

Offline
Joined: Jul 2009
Posts: 393
You don't need to have your browser open all the time. Everything is computed on our server, you're only reading it. The only thing that's only stored on your end is hidden titles list, and it doesn't get erased even if you turn your computer off.
 
Oct 20, 2013 10:50 AM

Offline
Joined: May 2013
Posts: 462
CamelliaBlossom said:
Eagle1337 said:
My issue is that I never keep my computer on for 15 hours at a time.. Since the queue started existing; I haven't had a working mal graph.


Do you have to keep your computer on for 15 hours? My understanding is that once you're in the queue, you're in the queue. I seem to remember closing the tab and opening it later to find myself--logically--having moved up in line. I certainly haven't kept my window open the entire time while waiting for things to update.

I guess my main thing is that you don't NEED everything to update instantaneously. It's not like all your data will get deleted if it takes more than a few hours to get through the queue.


Doesn't seem to be the case for me. After entering the queue and had to exit unwillingly. The next day I tried to reopen malgraph it just re-queues my profile. I thought I had to keep the page open for 5+ hours straight to allow my profile to be calculated and thought this was BS. Any solution welcome.
 
Oct 20, 2013 6:03 PM

Offline
Joined: Apr 2013
Posts: 307
Crassa said:
CamelliaBlossom said:
Eagle1337 said:
My issue is that I never keep my computer on for 15 hours at a time.. Since the queue started existing; I haven't had a working mal graph.


Do you have to keep your computer on for 15 hours? My understanding is that once you're in the queue, you're in the queue. I seem to remember closing the tab and opening it later to find myself--logically--having moved up in line. I certainly haven't kept my window open the entire time while waiting for things to update.

I guess my main thing is that you don't NEED everything to update instantaneously. It's not like all your data will get deleted if it takes more than a few hours to get through the queue.


Doesn't seem to be the case for me. After entering the queue and had to exit unwillingly. The next day I tried to reopen malgraph it just re-queues my profile. I thought I had to keep the page open for 5+ hours straight to allow my profile to be calculated and thought this was BS. Any solution welcome.


Due to the way webpages in general work, it doesn't really matter how long you keep the page open. If it re-queued you, either the first queue didn't work or there's something wrong with the queuing code.
 
Oct 21, 2013 3:33 AM

Offline
Joined: Jul 2009
Posts: 393
Crassa said:
I thought I had to keep the page open for 5+ hours straight to allow my profile to be calculated and thought this was BS. Any solution welcome.
Of course it would be BS, but that's not how the majority of webpages, including MALgraph, work.

Nothing is happening on your computer. You're only viewing the result that MALgraph calculated. You can enter your nick, turn off your computer, go to the Moon for a year, turn your computer back on and it will be the same as if you were refreshing the page once per second. Really, you don't have to keep your browser window open.

Regarding the queue: If you're in the queue already, you cannot be deleted from it or added to it again unless you're at the very front of it.

If you are in the queue, nothing is done unless you're first.
If you are not in the queue, you're added at the end.

Only the first nickname is processed, and nothing is done until it's finished. It can finish in a few ways:
1. User doesn't exist. (all is good)
2. User has been processed less than 24 hours ago. (all is good)
3. Data is downloaded and processed. (all is good)
4. Data is downloaded, but the download was interrupted or delayed and we only got a part of it. The script doesn't know that it's incomplete and processes it normally. (this is what's been happening nowadays)
5. Data can't be downloaded at all. (when MAL is down)

When the script recognizes something's wrong (e.g. MAL returns error 403), your nickname is put at the end of the queue automatically.

Due to extreme lags when downloading data from MAL, the queue stays long (over 7000 right now), but it's moving, processing, and enqueuing all the time. We can't do anything about these delays; Kineta said she'll contact Xinil to help solve this problem.

In any case, your computer has nothing to do with server-side calculations. Just like your window - even if you close it and shut off all light coming in with curtains, the world outside still exists.
Modified by fri, Oct 21, 2013 3:41 AM
 
Nov 2, 2013 1:58 AM

Offline
Joined: May 2009
Posts: 132
I have a question. My profile wasn't updated during last week, so I looked more in it. At about 1:30 am UTC+1 my profile was around 2k in queue. After I woke up at 10 am profile wasn't updated and posistion in queue over 9000.

Was some restarts on server done or you're working to optimize processing which why it isn't working right now or my profile on mal seems to be broken for your script?
 
Nov 2, 2013 2:44 AM

Offline
Joined: Jul 2009
Posts: 393
Recent changes in MAL API made it impossible to get any data unless you're whitelisted. I contacted Xinil and sent him our identifier, so he can forward it to Crave and put us on whitelist. We let the queue run in case Xinil whitelists us without sending us a confirmation.

We didn't make any significant changes in recent days - all problems arise from MAL's end and its inability to send us data.
 
Nov 2, 2013 4:47 AM
Offline
Joined: Jul 2012
Posts: 75
Hey my MALgraph doesnt update, it was last updated 7 days ago but i have watched my queue go down to 0 two times once today i once yesterday but it just starts over at queue 9000+ why is that happening?
 
Nov 2, 2013 5:03 AM

Offline
Joined: May 2009
Posts: 132
Thanks for explanation :)
 
Nov 2, 2013 9:04 AM

Offline
Joined: Jan 2010
Posts: 468
puzzlemaster said:
Hey my MALgraph doesnt update, it was last updated 7 days ago but i have watched my queue go down to 0 two times once today i once yesterday but it just starts over at queue 9000+ why is that happening?


Same here too. I wonder why?
Ah...hope Xinil work done faster. 0w0

 
Nov 3, 2013 3:55 AM

Offline
Joined: Mar 2013
Posts: 240
AoiMizu said:
puzzlemaster said:
Hey my MALgraph doesnt update, it was last updated 7 days ago but i have watched my queue go down to 0 two times once today i once yesterday but it just starts over at queue 9000+ why is that happening?


Same here too. I wonder why?
Ah...hope Xinil work done faster. 0w0


Actually, I wanted to ask exactly about this. My MALgraph has gone down to 100 and now it's back to 8000 and hasn't been updated for 8 days. Not that I'm in a hurry to see it updated. I was just curious about how the MALgraph updates and the queue line work so that I can understand why it does this.
 
Nov 3, 2013 5:05 AM

Offline
Joined: Jul 2009
Posts: 393
I already explained that in detail on the previous page. Actually, every other post here is a question that has been answered a few posts earlier, so here's a handy guide for future travelers:

This whole queue stuff sucks, bring the old one back!
No, that's a really bad idea. The old system crashed the server, because it tried to serve everyone at the same time. Processing thousands of users daily isn't possible unless you have a very powerful machine, or put things in queue and deal with them one at a time.
More: http://myanimelist.net/forum/?topicid=664847&show=0#msg25521223

Do I have to keep my computer on to get my stats?
No. I don't even know where did this idea come from. Everything is computed on our server, you're only reading the results of MALgraph's calculation. Your computer has nothing to do with server-side operations.
More: http://myanimelist.net/forum/?topicid=664847&show=0#msg25878889

How does the queue work?
When you enter your name, and it isn't in the queue already, you're put at the end of it. The queue moves at its own speed and processes the user who is currently at the front. You know, like a queue in a supermarket. After it does its work without errors, you're removed from the queue (leave supermarket).
More: http://myanimelist.net/forum/?topicid=664847&show=0#msg25878889

Why am I being put at the queue's end again and again, while my stats don't update?
When this happens, that's a sign that your data was not correctly downloaded and the engine detected the anomaly, so it enqueued you again. Normally this shouldn't ever happen, but recent problems with MAL cause this problem to occur every time. We contacted people who can help us over two weeks ago, and all we can do is wait for a response. Other apps that use MAL API (e.g. Taiga) have the same problem, and Taiga's developer hasn't received a response either.
More: http://myanimelist.net/forum/?topicid=664847&show=0#msg26119387

My stats haven't updated for ages! Why?
Read the section above. We can't get data from MAL, so the cached version is kept, and you still have access to your old stats.

My profile shows incomplete data! It shows 100 anime completed while I watched over 500!
This means that MAL cut off the connection while MALgraph was downloading data. It managed to download a part of your list, and doesn't even know that there should be more. If this happens, post in the club's shoutbox, and I'll try to fix it.

---

I don't like blaming others, but I also don't like what's happening to this website. It's very unreliable and you have to work around its stupid limitations (private lists not being private, so we have to make additional checks), or even omitted data (no way to get rewatch count). Xinil told developers to send him a message with API identifier for whitelisting, and nothing's changed since then.

If you've been wondering whether to make a MAL app, save yourself the drag and try somewhere else. Try a site that's more active and developer-friendly. I thought something would happen after they created an official club for API suggestions, but guess how many of the suggested things were added? Hint: the number is an oval.
Modified by fri, Nov 3, 2013 8:58 AM
 
Dec 21, 2013 11:05 PM

Offline
Joined: Jul 2008
Posts: 92
After reading this I know why my profile has not been updated since "ages".
Seems like we all have too wait.
 
Jan 17, 2014 11:52 AM
Offline
Joined: May 2011
Posts: 277
I Kind of have an Idea, but that might not be possible. I'll say it anyway.

Sometimes i check my malgraph even if i haven't updated my list at all, just to check old stats. I think a lot of other people do this too, but when they do, we still get added to the queue and slow down the ones that need a fast update. Is it possible to add a "Request Update" button that puts you in the queue instead of doing it automatically? If it's theorically possible i think it could be a nice way to slim the queue.

I'm Italian, so forgive me for grammar errors.

 
Mar 23, 2014 7:49 AM

Offline
Joined: Aug 2011
Posts: 214
-Crys said:
I Kind of have an Idea, but that might not be possible. I'll say it anyway.

Sometimes i check my malgraph even if i haven't updated my list at all, just to check old stats. I think a lot of other people do this too, but when they do, we still get added to the queue and slow down the ones that need a fast update. Is it possible to add a "Request Update" button that puts you in the queue instead of doing it automatically? If it's theorically possible i think it could be a nice way to slim the queue.


that would make a lot of sense
i dont think people are so desperate that they need to see the change after just one week
i hadnt visited this in over 130 days and really wanna check the difference, but im still around 2800 in queue after a good few hours of waiting...
 
Apr 2, 2014 7:46 PM
Offline
Joined: Jan 2008
Posts: 162
I think the "request update" button is a good idea, but only if you haven't moved that far in the queue for a long time. Mine's been going slower than usual (I'm at 1.3 days ago, #3724 as of this post) but I don't mind, even though I'm kinda addicted to seeing every little change in my graphs/stats (heh).

Hmm...if your position hasn't changed in a long time (within a few days) or if it's been a ridiculously long time (like a three months or even longer) the option would become available. It depends on the server load if it would actually work though. A high server load/lots of people queued up would be the cause for such problems in the first place, so a bunch of people hammering on the button might cause a catastrophe.

I've put in random people into MALgraph and seen "last updated" stats of over a YEAR. It might just be because they stopped updating their anime list, but that's definitely the situation in which I'd want such a feature so I wouldn't have to pester the admins about it.
 
Apr 12, 2014 5:34 PM

Offline
Joined: Aug 2013
Posts: 52
ZodiacBeast said:
I've put in random people into MALgraph and seen "last updated" stats of over a YEAR. It might just be because they stopped updating their anime list, but that's definitely the situation in which I'd want such a feature so I wouldn't have to pester the admins about it.


You will only get added to the queue again if you try to view your MALgraph again. So the people who have not been updated in over 10 months, obviously haven't visited their MALgraph in that time.
You visiting their MALgraph will add them to the back of the queue.
 
Apr 14, 2014 6:54 AM

Offline
Joined: Jun 2011
Posts: 3
Does MALgraph still update the same way today?
I was at about 3500 before going to bed and turning my computer off. When I came back after about 13 hours I was at 3100. Now if I refresh every couple of minutes, the number steadily decreases by about 8 per minute.
This doesn't really work with the prior explanation of the update queue.
 
Apr 14, 2014 8:09 PM
Offline
Joined: Jun 2013
Posts: 58
Dorry said:
Does MALgraph still update the same way today?
I was at about 3500 before going to bed and turning my computer off. When I came back after about 13 hours I was at 3100. Now if I refresh every couple of minutes, the number steadily decreases by about 8 per minute.
This doesn't really work with the prior explanation of the update queue.

I noticed my list doesn't update either. I get to the front of the queue, and then I'm right back at the end again with no update. Probably left over problems from the technical difficulties the other day. Not a big deal right now since my last update isn't even a week old.
 
Apr 16, 2014 8:20 PM

Offline
Joined: Aug 2013
Posts: 52
Dorry said:
Does MALgraph still update the same way today?
I was at about 3500 before going to bed and turning my computer off. When I came back after about 13 hours I was at 3100. Now if I refresh every couple of minutes, the number steadily decreases by about 8 per minute.
This doesn't really work with the prior explanation of the update queue.

As PixelPenguin said, It hasn't been updating correctly for the last few days, I believe it is related to them changing servers and doing maintenance 5 days ago.
 
May 24, 2014 12:31 PM
猫娘

Offline
Joined: Jan 2012
Posts: 518
It's been a month since my MALgraph stats updated, but after seeing how many users there are now... I understand the situation heaps better (haha)
 
Top