ZeroCrystal's Blog

Jul 3, 2020 8:05 AM
Last revision: December 24, 2020

Introduction
The new MAL API makes use of the OAuth 2.0 protocol to authenticate and authorise all users. The official documentation can be found here, but, for the sake of simplicity, I'll summarise it in this blog post.

Let's get started!




Step 0 - Register your application
API access is restricted to the owners of a MAL Client ID. You can create your application by clicking here or opening Account SettingsAPICreate ID.



Here you will be able to get your personal Client ID and, optionally, a Client Secret. The Client Secret must be kept somewhere safe! Anyone who can access it may issue privileged commands on behalf of your application.

The application form contains several mandatory fields. I'll focus on some of them:
  • App Type ⁓ You have four choices: "Web", "Android", "iOS", and "Other", but they can be split into two functional categories: private API clients ("Web"), and public API clients ("Android", "iOS", and "Other").

    A private API client is meant as a piece of software which can safely store a Client Secret in a place unreachable by a regular user. Ordinarily, this condition can only be satisfied by setting up a web server, hence the "Web" app type.

    In contrast, public API clients cannot store a secret due to their architecture, so they don't receive one. Typically, this is the case of applications running their entire codebase on a user's device, like mobile or native apps, or browser's userscripts.

    Both public and private clients can interact with the API in the exact same way, the only difference is how a user is authenticated. The App Type of an application cannot be changed in the future.
  • App Redirect URL ⁓ This parameter will be explained later. For now, you can set it to http://localhost/oauth. You will be able to change it every time you want.
  • Homepage URL ⁓ The URL of a webpage which is linked to your API application. If you don't have one, you can set it to your MAL's profile URL. This can be changed in the future.
  • Name / Company Name ⁓ You can set this field in the same way as App Name.
Note that these suggestions are valid only for non-commercial applications. If you're running a business or making a profit thanks to the API, then you must fill each field properly.




Step 1 - Generate a new Code Verifier and Code Challenge
The OAuth workflow is susceptible to the authorisation code interception attack. The PKCE protocol has been designed to mitigate this threat.

Before you can authenticate a user, your client needs to generate a Code Verifier and a Code Challenge. A Code Verifier is a high-entropy, cryptographic, random string containing only the characters [A-Z] / [a-z] / [0-9] / "-" / "." / "_" / "~". The length of the string must be between 43 and 128 characters (personally, I would recommend using 128 characters).

MAL only allows the plain transformation for the Code Challenge. In other words, it means that you have to set the Code Challenge equal to the Code Verifier. Simple.

This is a small example of a PKCE generator written in Python (open in GitLab):




Step 2 - User authorisation
We can now build the URL which can be sent to a user to generate an Access Token for our API client. Here you can find all the parameters you can append to the URL:
  • Base URL: https://myanimelist.net/v1/oauth2/authorize (GET request)
  • Parameter response_type: must be set to "code". (REQUIRED)
  • Parameter client_id: your Client ID. (REQUIRED)
  • Parameter code_challenge: the Code Challenge generated during the previous step. (REQUIRED)
  • Parameter state: a string which can be used to maintain state between the request and callback. It is later returned by the MAL servers to the API client. (RECOMMENDED)
  • Parameter redirect_uri: the URL to which the user must be redirected after the authentication. (OPTIONAL)
  • Parameter code_challenge_method: defaults to "plain". No other option is currently available. (OPTIONAL)

Here's a possible URL:
https://myanimelist.net/v1/oauth2/authorize?response_type=code&client_id=1a2b3c...&code_challenge=A1B2C3...&state=RequestID42

Clicking on the previous link would redirect the user to an authorisation page like this:





Step 3 - User redirection
After pressing on "Allow", the user is redirected to the provided (or default) URL registred to your application. Two parameters are automatically appended to the query string:
  • Parameter code: Authorisation Code needed to obtain an Access Token for the user.
  • Parameter state: the same state parameter sent during the previous step.

For instance, the user may land on the page pointed by the following URL:
https://mywebsite.net/callback?code=a1B2c3...&state=RequestID42




Step 4 - Get the user's access token
At this point, our API client received an Autorisation Code and an optional state value. We're just one step away from obtaining the user's Access Token.

We should now prepare a final HTTP request shaped like this:
  • Base URL: https://myanimelist.net/v1/oauth2/token (POST request)
  • Structure: form URL encoded.
  • Parameter client_id: your Client ID. (REQUIRED)
  • Parameter client_secret: your Client Secret. (REQUIRED if your application has a Client Secret; OMIT in all other cases (i.e. you selected "Android", "iOS", or "Other" as App Type))
  • Parameter code: the user's Authorisation Code received during the previous step. (REQUIRED)
  • Parameter code_verifier: the Code Verifier generated in Step 2. (REQUIRED)
  • Parameter grant_type: must be set to "authorization_code". (REQUIRED)

In response, we will get a JSON object containing four properties:
  • token_type (string): always set to "Bearer".
  • expires_in (number): expiration time of the Access Token (expressed in seconds).
  • access_token (string): the user's Access Token.
  • refresh_token (string): the user's Refresh Token (explained in the next step).

Possible JSON response:
{
    "token_type":    "Bearer",
    "expires_in":    2678400,
    "access_token":  "a1b2c3d4e5...",
    "refresh_token": "z9y8x7w6u5..."
}

IMPORTANT: currently, the lifetime of the Access Token and Refresh Token is the same (31 days). This contradicts the documentation, stating that the Access Token expires 1 hour after its issuing. This behaviour will be fixed in the future, so always refer to the "expires_in" parameter to know the lifetime of the Access Token. The Refresh Token will still be valid for 31 days since its issuing (source).


That's all! We can now call any API function on behalf of the user who performed the authorisation procedure. You can open the API documentation by clicking here.

All API calls use the Bearer HTTP authentication scheme. To implement it, you only need to add an additional "Authorization" header to all your HTTP requests. The header's value must be set to "Bearer USER_ACCESS_TOKEN", where USER_ACCESS_TOKEN is the Access Token obtained a few lines above.

Example request:
GET https://api.myanimelist.net/v2/users/@me
Authorization: Bearer a1b2c3d4e5...

Response:
HTTP 200 OK
Body:
{
    "id":   5292566,
    "name": "ZeroCrystal",
    ...
}




Step 5 - Refresh the Access Token
As you may have noticed, during the authorisation step we didn't receive just an Access Token, but also a Refresh Token. In the OAuth protocol, the Access Token generally has a short lifetime. According to MAL's docs, it should expire after an hour, but the actual time limit is longer. This will be surely fixed in the future, so make sure to read the documentation.

Either way, once an Access Token expires, it cannot be used anymore to make API calls. If we try to use it again, we would receive an HTTP response with status code 401 (Unauthorized).

In this case, we don't have to force the user to perform the OAuth authentication procedure once again, we just have to refresh the Access Token. This can be accomplished thanks to the Refresh Token which we received together with the Access Token.

To get a fresh token we need to send the following request:
  • Base URL: https://myanimelist.net/v1/oauth2/token (POST request)
  • Structure: form URL encoded.
  • Parameter client_id: your Client ID. (REQUIRED)
  • Parameter client_secret: your Client Secret. (REQUIRED if your application has a Client Secret; OMIT in all other cases (i.e. you selected "Android", "iOS", or "Other" as App Type))
  • Parameter grant_type: must be set to "refresh_token". (REQUIRED)
  • Parameter refresh_token: the user's Refresh Token. (REQUIRED)

In response, we will get a JSON object containing the same four properties as when we received the Access Token for the first time. The fresh tokens provided by this response should overwrite any previously saved data for the involved user.




F.A.Q.
> Can you show me a simple example on how to implement all of this?


> What's the difference between "query string", "form URL encoded", and "JSON"? Which format should I use?


> Why do I have to use OAuth to authenticate a user? Wasn't it simpler with the previous authentication scheme?


> Do I need a back-end infrastructure to run the API?


> Can I store the Access Token inside the user's device?


> I'm working on a mobile or native app, and I don't own a domain or web server. How should I handle the authorisation flow?
Posted by ZeroCrystal | Jul 3, 2020 8:05 AM | 31 comments
ZeroCrystal | Aug 5, 7:29 AM
(@Plippi) Hi! You're getting an error because you're encoding the request's body in JSON, while it must be Form-URL-Encoded.

Just change the encoding and it should work.
 
Plippi | Aug 1, 8:26 PM
I seem to also be getting the 'unsupported_grant_type' error. I have the contents of the request in the body but the issue still occurs. I'm using JS node-fetch here, and omitting credentials. Any help would be appreciated, thanks!! Also (@ZeroCrystal) this post was very helpful, thanks a ton!!

const body = {
title: 'MAL API',
body: 'mal api postreq',
grant_type: 'authorization_code',
client_id: '',
client_secret: '',
code_verifier: '',
code: '' };

await fetch('https://myanimelist.net/v1/oauth2/token', {
method: 'POST',
body: JSON.stringify(body),
headers: { 'Content-Type': 'application/json' },
}).then(res => res.json())
.then(json => console.log(json))
.catch(err => console.log(err));
 
ZeroCrystal | Jul 31, 9:46 AM
(@Robstersgaming) You can send a message to Xinil, or you can describe it to me and I can file the issue for you (but in both cases expect a long time before a solution is deployed).
 
Robstersgaming | Jul 31, 8:14 AM
Do you know how to get in contact with the mal API team? Currently I am having an issue where certain shows aren't being pulled from the API.
 
ZeroCrystal | Jul 3, 6:35 AM
(@DiamondTMZ) You can access the re-watch status of a series. You just need to add the "my_list_status" option to the "fields" query parameter. This works for both the "Get user anime list" and "Get anime details" API endpoints. E.g.:

https://api.myanimelist.net/v2/users/@me/animelist?fields=my_list_status

If "my_list_status[is_rewatching]" is set to "true", then you're currently re-watching that series.
 
DiamondTMZ | Jul 2, 7:17 AM
I'm pretty new to APIs and was going to use this for a school project, I looked through the documentation and there didn't seem to be any mention of rewatched series, does this mean you can't filter out series that the user has rewatched or interact with the rewatch feature in any way?
 
ZeroCrystal | Feb 7, 12:24 AM
(@Robstersgaming) There's not a function to increment by 1 the number of watched episodes of a series, but you can still optimise the amount of requests you make.

  1. Retrieve the user's watching list and cache it locally. You can do it by using the "Get user anime list" endpoint, passing "status=watching" and "fields=num_episodes,my_list_status" as part of the query string. The returned anime list contains both the number of existing episodes and how many of them were already watched by the user.
  2. When the user wants to update a series, your application should first check in the cached version of the anime list how many episodes were already watched, increment it, and send the appropriate PATCH request.

In this way, you can send a single request for each update, assuming that all entries are listed in the watching list.
 
Robstersgaming | Feb 6, 4:41 PM
I don't know if you're able to help me but don't know where else to ask lol. If I wanted to make a request to increase the amount of episodes a user has watched by one is there a way I can do that in a single request. Currently I have it so I make a GetAnimeDetails request get the number of watched episodes add one to it then make the UpdateAnimeList request with the new number. However I want to both reduce the requests I am making and this has issues with handling multiple requests as the response from mal can be delayed. Is there a better way you know of to handle this?
 
ZeroCrystal | Jan 8, 2:33 AM
(@Robstersgaming) Unfortunately, there's no concrete information available. I only know that each API client has a “very generous” amount of requests, but nothing more. Personally, I never reached the limit, so I have no idea to which value it is set.

Either way, the rate limit should only apply to API calls. You can keep issuing or refreshing Access Tokens without thinking too much about it.
 
Robstersgaming | Jan 7, 5:45 PM
Do we have concrete numbers on the rate limiting for mal? Does the rate limiting only count towards the number of authorization requests or is it also the amount of times you're pulling watchlists, etc.?
 
ZeroCrystal | Jan 4, 3:01 AM
(@Robstersgaming) Yes, it's standard procedure. The Client Secret is the only element that mustn't be disclosed.
 
Robstersgaming | Jan 3, 10:29 PM
Is it okay for us to expose the authorization code, client id and code_challenge to the user because the client secret is held securely in our backend?
 
LukyGamer | Dec 29, 2020 3:08 AM
If you got the problem "The authorization grant type is not supported by the authorization server." Probably you are having some problems in your code. I will leave you here the code I used you Just need to replace the fields in the structure and that should work.

POST https://myanimelist.net/v1/oauth2/token HTTP/1.1
Content-Type: application/x-www-form-urlencoded

client_id=
&client_secret=
&grant_type=
&code=
&code_verifier=

Just complete the fields with your own information, (double or simple quotes non needed)

Sorry for Any Mistake I hope that could help <3. Btw thanks to ZeroCrystal por this post
 
ZeroCrystal | Dec 21, 2020 8:04 AM
(@joel360) I guess that's because there's no application listening on localhost for incoming HTTP requests, which is normal if you're just playing around with the API.

What matters is the "code" parameter inside the URL. You can simply copy it manually and use it in Step 4. In a real application, you would need a piece of software that does it automatically, and you'd probably need to change the Redirect URL to something more useful.
 
joel360 | Dec 21, 2020 7:15 AM
why would in step 2, after I press the allow, button, it shows localhost refuse to connect due to setting the App Redirect URL? What should I do?
 
almeidaalajoel | Dec 7, 2020 1:41 PM
Dude, I thought this was their official guide lmao. You're a beast, thanks for this. Great work.
 
ZeroCrystal | Nov 13, 2020 4:05 AM
(@TakemotoUruka) Your problem is that you've passed the parameters as part of the query string (i.e. inside the URL) instead of placing them in the body of the request.

The POST request in Step 4 requires that the data must be form URL encoded. The specific way to achieve this depends on the programming language and library you're using.
 
TakemotoUruka | Nov 13, 2020 1:47 AM
I am stuck at Step 4 can anyone please help me
This is url ' https://myanimelist.net/v1/oauth2/token?client_id=8......&client_secret=e.......&code=d.....&code_verifier=N.....&grant_type=authorization_code '

After doing post request it is giving me this error :-

b'{"error":"unsupported_grant_type","message":"The authorization grant type is not supported by the authorization server.","hint":"Check the `grant_type` parameter"}'
 
Quriz | Oct 22, 2020 10:08 AM
Really helpful
 
ZeroCrystal | Sep 1, 2020 10:10 AM
(@Kayuway) There's not an explicit way to do that. If you're working on something like a web crawler, then you can manually issue an Access Token linked to your account and feed it to your programme.

However, if you also need to access users' data, then you need to redirect them to the OAuth authorisation flow and store their Access Tokens somewhere (either on your server or on the users' browser).
 
Kayuway | Sep 1, 2020 7:54 AM
Hi! I couldn't figure out the flow from the docs, thanks for writing this!

Do you know if there's an intended way of authorizing without the redirection? For example, for backend apps that might connect to the API.
 
ZeroCrystal | Jul 19, 2020 2:17 AM
(@ivanovishado) The dev team said they're using this first month to determine the rate limit. At the moment, they say that the API have a «very generous» rate limit.
 
ivanovishado | Jul 18, 2020 9:25 PM
Thanks for the guide!

Do you know what the monthly rate limit is?
 
ZeroCrystal | Jul 8, 2020 2:51 AM
(@x_Acnologia) I'm glad to hear that!
 
x_Acnologia | Jul 7, 2020 4:37 PM
Thank you, this helped me a lot as a complete beginner working with APIs and Oauth.
 
ZeroCrystal | Jul 7, 2020 3:34 PM
(@Rafaeltab) I'm still gathering information about the security implications of apps without a client_secret, so I'll let you know if I discover something interesting. ;-)
 
Rafaeltab | Jul 7, 2020 3:30 PM
I don't think that'll stop anyone trying to do harm. However, there is a very easy solution anyways and that is just choosing web instead of ios so you get a client_secret and using that. Thanks for all the help anyways, have a nice night
 
ZeroCrystal | Jul 7, 2020 3:17 PM
(@Rafaeltab) The client_id is not the only element evaluated during the authorisation procedure. The redirect URL is also fundamental.

It is recommended for mobile and native apps to register a custom URI scheme to handle redirections. For example, you may associate your app with the URI "myapp://auth", and then using it as the redirection URL for your OAuth application. In this way, your app is the only one that can catch the Authorisation Code and issue Access Tokens.
 
Rafaeltab | Jul 7, 2020 12:43 PM
I am making an ios app so I have no client_secret. But does that mean any user can just grab that client_id, get their own code and make a thousand requests on my behalf which would end up with my app getting either removed, banned or blocked?
 
ZeroCrystal | Jul 7, 2020 12:39 PM
(@Rafaeltab) Yes, it does. The client_id is not a secret: it simply tells the API server which application started the OAuth process. What you should protect is the client_secret (if you're developing a web app).
 
Rafaeltab | Jul 7, 2020 12:17 PM
Wouldn't clicking on the link in step 2 allow the user to see the client_id?
 
ZeroCrystal | Jul 7, 2020 9:27 AM
Forum topic: https://myanimelist.net/forum/?topicid=1850649
 
  • Friend Requests
View All