Forum Settings
Forums
New
Feb 3, 2014 8:04 PM
#1
Offline
Feb 2014
5
I am trying to access MyAnimeList.net from my Java program with the following code:

sURL = "http://www.myanimelist.net";

String result = "";
try {
URL url = new URL(sURL);
URLConnection urlc = url.openConnection();
BufferedReader buffer = new BufferedReader(new InputStreamReader(urlc.getInputStream()));
StringBuilder sb = new StringBuilder();
String str;
while ((str = buffer.readLine()) != null) {
sb.append(str);
}
result = sb.toString();

} catch (Exception e) {
e.printStackTrace();
}

Which I have seen other unofficial API's use, however what I get returned is this:

<html><head><META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW"></head><iframe src="/_Incapsula_Resource?CWUDNSAI=9&incident_id=163000490103874927-308234194933449498&edet=12&cinfo=464f095fc753818104000000" frameborder=0 width="100%" height="100%" marginheight="0px" marginwidth="0px">Request unsuccessful. Incapsula incident ID: 163000490103874927-308234194933449498</iframe></html>

I don't know if it's something I'm doing wrong, or if some update to the website isn't letting 3rd party clients from accessing it, or if I'm not allowed to view the website in this manner.

If I'm not allowed / myanimelist doesn't like 3rd party clients, then please tell me and I will cease creation of my program immediately
Feb 3, 2014 9:25 PM
#2

Offline
Mar 2013
5831
It's Incapsula, it would seem. Alongside with it being encrypted, seeing a mess in the code is expected.
Basically, the so-called "DDoS and other attacking preventing system" has triggered for you. Don't worry, it's not your fault. Has happened to many, and unless things take action for the better, it will continue to do so.

If you've only obtained a page block, reload the page and you'll be able to access the site again. If you've been automatically banned by it, however, wait a little bit. You should be unbanned in less than a day.

Edit: Check Luna's reply for the correct info.
SubbedFeb 4, 2014 6:37 AM
Feb 3, 2014 11:58 PM
#3
Offline
Feb 2014
5
Well I hope they don't extend bans for multiple attempts... I've been trying for a number of days now... Probably 50+ attempts... Is it possible to be white listed? Or know when I can access it agsin? The weird part I find is that I can access the website through chrome just fine. I tried my phone, and other laptops through chrome or other browsers and they work fine. Just the code I have is giving me the incapsula
Feb 4, 2014 12:56 AM
#4

Offline
Mar 2013
5831
As far as I know, the bans do not extend in length through time. The ban's duration isn't known, but it lasts a few hours max.
You cannot be whitelisted as for the moment, but such an idea has already been forwarded to the higher ranks.

Edit: Check Luna's reply for the correct info.
SubbedFeb 4, 2014 6:37 AM
Feb 4, 2014 2:14 AM
#5

Offline
Oct 2009
7752
rweqx said:
The weird part I find is that I can access the website through chrome just fine. I tried my phone, and other laptops through chrome or other browsers and they work fine. Just the code I have is giving me the incapsula

It's not weird, it's that way for everyone. It only blocks specific clients (usually browsers) that perform suspicious actions (in the Incapsula's point of view), e.g. if you are blocked with firefox, you can access site with chrome; if you also manage to get blocked with chrome, you can use IE, and so on. The blocks expire eventually (repeated attempts extend it greatly though) or may be bypassed by changing your IP
Feb 4, 2014 3:28 AM
#6
★★★★★

Offline
Sep 2008
19310
There is some misinformation in this thread. What OP tries is to access the site with a program, not with a browser. He's not getting the Incapsula error messages that users sometimes get in their browsers, his problem is different. Doing anything with the browser like reloading the page won't help him. Waiting until he gets "unbanned" won't help him.

@rweqx:

Due to issues with DDoS attacks last year, a few changes how to handle requests from 3rd parties have been made. The default is that all 3rd party clients are blocked from accessing the site or API. To get your programs/scripts/apps working properly, you need to get whitelisted by Crave. For more information see this thread. There you can also find the form that you have to fill out in order to get whitelisted. It seems it is currently closed, but I've seen it closed before and then open again a few days later, so I'm assuming it's only temporarily closed because of many requests.
Feb 4, 2014 5:42 PM
#7
Offline
Feb 2014
5
@Luna_

How does it whitelist? By IP Address, or something is added to your program such as a certificate or ???

Also the form currently says: "Sorry, but this form is no longer accepting submissions.", so I guess I'll have to wait a few days...

More topics from this board

» (BUG) Can't edit details on anime/manga entries + can't log in elsewhere but can stay logged in on current session (400 Bad Request)

abystoma2 - Dec 15

5 by abystoma2 »»
11 hours ago

» Plan to Drop (watched a sneak peak)

Daichi09 - Dec 15

3 by Timcampy »»
Dec 16, 4:00 PM

» How can I appeal a permanent ban?

Danila_Naumov - Dec 15

0 by Danila_Naumov »»
Dec 15, 3:49 PM

Sticky: » Club President Request Thread ( 1 2 3 4 5 ... Last Page )

Kineta - Aug 7, 2010

787 by elkensteyin »»
Dec 14, 5:09 PM

» Can’t get into the discord server and can’t ask for help there

RoyalyGaming - Dec 13

4 by RoyalyGaming »»
Dec 14, 4:26 PM
It’s time to ditch the text file.
Keep track of your anime easily by creating your own list.
Sign Up Login