Forum Settings
Forums
Must be a Club Member to Reply 
Apr 5, 2010 9:57 AM

Offline
Joined: Apr 2009
Posts: 100
Unicode in titles that works fine on the site is returned as garbage.

http://myanimelist.net/api/anime/search.xml?q=maria+holic

Looks like UTF-8 interpreted byte for byte as latin1 and then entity-encoded. MAL is a treasure trove of this kind of bug ;)
 
Apr 14, 2010 10:05 PM

Offline
Joined: Aug 2007
Posts: 172
Good thing I checked this group... Explains why I wasted the last 2 hours of my life.

XML does NOT ALLOW ampersands. The search API returns results with ampersands due to special characters. This is the cause of a lot of trouble for some people using it I bet. As soon as you strip the ampersands (I do with regular expressions) everything is fine.

However, this also means you don't get to make use of any special characters using this method, unless you add them back in when you are done parsing your XML.
Modified by brass2themax, Apr 16, 2010 3:45 PM
 
Sep 21, 2010 10:52 AM

Offline
Joined: Nov 2007
Posts: 1
Any updates on the Search API, Xinil? The idea of having synonyms and such in child tags is a very good one and very much in the spirit of XML.

The fact that the returned XML needs to be ran through a tidy program is slightly annoying for me, given how I'm trying to do this on Android and outside packages can't be imported as easily. Any easy ways for you to convert ALL ampersands to & before returning the XML? My app is currently breaking under things like Å .

I've also noticed that the API search and the web site search show slightly different results.

Other than these points, good job on the API.
 
Oct 10, 2010 8:25 AM

Offline
Joined: Mar 2009
Posts: 7
For everybody who has problems with HTML entities in XML (yes, these are illegal, period).
Just force the encoding to "latin1" and use DTD for XHTML-transitional-latin1 to parse HTML entities into valid chars, then recode into UTF-8

The bigger problem is incompatibility between API search and Web site search, for example:

Modified by psorcerer, Oct 10, 2010 8:40 AM
 
Apr 28, 2012 6:43 PM

Offline
Joined: Feb 2009
Posts: 83
Sorry for the necro-bump, but I think I have a fix for the encoding issue. It's quite simple and I've tested it and I have proof it works.

psorcerer said:
For everybody who has problems with HTML entities in XML (yes, these are illegal, period).
Just force the encoding to "latin1" and use DTD for XHTML-transitional-latin1 to parse HTML entities into valid chars, then recode into UTF-8

That is only a band-aid solution to a larger problem. While it works, it doesn't kill the bug.

Now, I've talked with a friend about this problem and we've come to a conclusion about whats causing this and what will fix it.

Anyway, we believe that the API is calling the PHP function "htmlentities($string)" on the data before they're placed into the xml. Now, PHP's documentation says that it defaults to UTF-8 encoding, but it doesn't, it reverts to iso-8859-1 (latin1) instead. That will produce stuff like this:
�女人形
as well all know.

Now the fix for this is actually quite simple. It only involves changing the htmlentities call to add 2 more parameters to it. Transforming it from:
htmlentties($string)
to
htmlentities($string, ENT_COMPAT, "UTF-8")

Let's compare that one change and what it will output:
Currently:
htmlentities($string) --> �女人形
Proposed Change:
htmlentities($string, ENT_COMPAT, "UTF-8") --> 少女人形

If you don't believe that, here is a link to show you that the proposed change works, including script source just as extra proof it does work. Make sure your browser encoding is set to UTF-8, if it isn't change it to that then restart it, and you may need to also set a unicode font in case the text doesn't show up right after making sure your encoding is set to UTF-8.

That change, and handling the response stream as a UTF-8 stream, will fix all the response problems anyone should have. After that, it's just replacing the remaining html entities like ', &quote; and the like in your program.

Edit: I was looking through the PHP doc again and it mentioned htmlspecialchars() working essentially the same way. You'll be able to use this fix I mentioned there as well since the first three parameters of htmlspecialchars() and htmlentities() are the same.
Modified by Diablofan, Apr 29, 2012 10:46 AM


 
Jul 4, 2012 9:39 AM

Offline
Joined: Sep 2010
Posts: 2
Hi Guys. I am trying to create a MAL app and it's my first time using the unofficial app. See my code

var get_db = function (url,form_data,op) {
$.get(
''http://mal-api.com/anime/search',
{format: 'xml', q:'haruhi'},
function(data) {
$('#result').text(data);
}
);
}

If I remove the format and default it will give me an error:

Origin http://localhost is not allowed by Access-Control-Allow-Origin

If I specify a format it doesn't show the error. But I can't verify if there's data in it. I maybe doing this wrong, so I would like to ask some guidance on this.

Oh. I am using JQUERY in case you didn't know.

Thanks.
 
Jul 20, 2012 9:53 PM

Offline
Joined: Feb 2009
Posts: 83
Solidad said:
Hi Guys. I am trying to create a MAL app and it's my first time using the unofficial app. See my code

var get_db = function (url,form_data,op) {
$.get(
''http://mal-api.com/anime/search',
{format: 'xml', q:'haruhi'},
function(data) {
$('#result').text(data);
}
);
}

If I remove the format and default it will give me an error:

Origin http://localhost is not allowed by Access-Control-Allow-Origin

If I specify a format it doesn't show the error. But I can't verify if there's data in it. I maybe doing this wrong, so I would like to ask some guidance on this.

Oh. I am using JQUERY in case you didn't know.

Thanks.
You'll want to talk to the person in charge of the unofficial MAL API (club link: http://myanimelist.net/clubs.php?cid=14973 ), as that is the API you're calling from the URL in your post there.

If you want to use the official API, you can read the documentation for it here: http://myanimelist.net/modules.php?go=api


 
Mar 23, 2013 7:49 AM

Offline
Joined: Sep 2011
Posts: 20
While writing on a Plex Media Server Agent I noticed that you only can search for the Name. It would be nice to search for the ID too so you can get just a single result.
 
Jan 3, 2014 3:05 AM

Offline
Joined: Oct 2013
Posts: 1
when you log in to mal website and see any particular anime page you see external links far down the page but
the mal search api doesn't provide external links
is there any way to get external links like "official site, animedb and wikipedia" from a mal
sorry for my bad English
 
Mar 24, 2014 4:43 AM

Offline
Joined: Nov 2007
Posts: 681
When the search method fails, it returns 204 No Content status code and an empty message-body as expected. However, the same response includes either "Content-Length: 10" or "Transfer-Encoding: chunked" header. This behavior is questionable at best.
 
Jun 2, 2014 7:00 AM

Offline
Joined: Mar 2014
Posts: 4
http://myanimelist.net/api/anime/search.xml?q=naruto

It seems the img tag isn't being closed in the xml file could you fix this.
 
Jun 8, 2014 5:46 PM

Offline
Joined: Oct 2012
Posts: 2
When I search "Space Dandy" on the page I get the expected results, but over the api I only get results when I search "SpaceâDandy".

EDIT: The broken unicode is that star symbol here: http://myanimelist.net/anime/20057/Space%E2%98%86Dandy

No results: http://myanimelist.net/api/anime/search.xml?q=Space+Dandy
No results: http://myanimelist.net/api/anime/search.xml?q=Space%20dandy
Ok result: http://myanimelist.net/api/anime/search.xml?q=SpaceâDandy
More than I want, but ok: http://myanimelist.net/api/anime/search.xml?q=Space

Correct result on html page: http://myanimelist.net/anime.php?q=space%20dandy

Unrelated:
The api uses '+' as a space character (which is non-standard URL-encoding) while the html page uses "%20". This should be explicitly mentioned on the documentation, or rather unified.

Different problem:
Like erengy, I would like an unified error response.
Sometimes I get "No results" or something with 'Incapsula incident'.
For example when I punch this url into curl on terminal, with correct credentials and userAgent Incapsula: curl ...auth-here... http://myanimelist.net/api/anime/search.xml?q=Space dandy
Okay: curl ...auth-here... http://myanimelist.net/api/anime/search.xml?q=Space+dandy

Still hoping for updates for the API.
Modified by Nehmu, Jun 8, 2014 5:51 PM
 
Aug 27, 2015 10:39 PM

Offline
Joined: Mar 2012
Posts: 103
Yeah, I know this is old, but I do want to correct something here:

Nehmu said:

The api uses '+' as a space character (which is non-standard URL-encoding) while the html page uses "%20". This should be explicitly mentioned on the documentation, or rather unified.


That's not fully correct. For spaces in paths you must use %20. For items in the query string, you can use + to represent a space. This is usually when sending data as "application/x-www-form-urlencoded". You may use %20, but it's not required. Because of the special use of the +, you must percent-escape it when using it literally. It's old usage, but it is valid.

More on URL encoding is over at Wikipedia
Developer, sysadmin, and anime addict.
Have an Android smartphone? Try Atarashii!