Wahoo, It's-A-Me, Critical and Commerical Failure
Let's face it: movies based off video games aren't very good. Who knows, maybe Duncan Jones can turn that around with Warcraft, hopefully the It's Always Sunny in Philadelphia guys can knock Minecraft out of the park. As of this moment, though, the highest rated video game-based movie (not counting movies about games that aren't direct adaptations like Tron or Wreck-It Ralph) on Rotten Tomatoes is the Angry Birds movie with a rating of 45%, still in "rotten" territory. At the genre's worst, you get the works of Uwe Boll (House of the Dead, Alone in the Dark, multiple Bloodrayne movies, and his "parody of 9/11" Postal). In the world of anime, you can find many successful series based on visual novels like Clannad and Steins;Gate, but with adaptations of more traditional gameplay-based games the success rate lowers.
Nintendo's Reluctance
For a company with as many instant money-generating franchises under its belt as Nintendo, they've been reluctant to license their properties out for media adaptations. The one obvious exception, the constant stream of Pokemon series and movies, is an unusual case because licensing rights are managed not by Nintendo directly but by the second-party studio The Pokemon Company. There was a Super Mario Brothers anime movie in 1986 that has never been released on DVD - even in Japan - and the infamous 1993 live-action Super Mario Brothers disaster that explains why Nintendo has avoided movies since.
The first non-Pokemon Nintendo movie to be released after Super Mario Brothers was, confusingly, an Animal Crossing movie. Yeah, Animal Crossing is a well designed game, but who wants to watch a movie with no plot beyond "pay debts to shady raccoon"? And WHAT SENSE DOES A CHARACTER NAMED MR. RESETTI MAKE IN A MOVIE WHEN MOVIE THEATERS HAVE NO RESET BUTTON!?!
Recent financial hardships, however, have brought about a change of heart at The Big N. With the Wii U's failure as a console (a rarity for Nintendo), the company's been compensating by doing things they hadn't done before: mobile games, a deal with Universal theme parks, and now they're working on 3D animated movies. Details are scarce, but it sounds like they want to keep as much of the production in-house as possible, likely influenced by Marvel's decision to control production of their Cinematic Universe.
How Would This Even Work?
Can Nintendo actually pull off Marvel's trick? Adapting comics to film is a different process than adapting games: going from comics to film you're adding elements of sound and movement, but going from game to film you're subtracting a major element of interactivity. Nintendo's library of games poses a particular challenge in that a lot of them are designed with fun mechanics in mind over complex characters or coherent narrative, so movie adaptations will have to either invent those story elements or suffer from a lack of them.
Smash Bros seems like the obvious Avengers parallel in crossover potential, but when even the "Story Mode" of Smash Bros barely bothers to have an actual story, how do you adapt that without losing the anarchic spirit of fun that makes the game beloved in the first place? For the wackier Nintendo games, my suggested narrative model would be Adventure Time. The popular cartoon series has a sense of absurdity that owes a lot to Nintendo's games, but carries all the action and weirdness along with compelling, well-developed characters. Natasha Allegri (Adventure Time artist and Bee and Puppycat creator) is friends with Ko Takeuchi, who's done art for various Nintendo games. Take advantage of that connection and hire her for a Mario movie.
The biggest decision to be made for a Mario movie: what is Mario's personality? Even in the more story-driven Mario RPG games, Mario is still very much a cipher compared to his more colorful friends and enemies. Honestly, the only character in that universe less-developed is Waluigi (on that note, Nintendo, MAKE WALUIGI A MAJOR CHARACTER IN THE MOVIE, THE MEMES WILL BE AMAZING!). Similar issues will also have to be dealt with for the Legend of Zelda movie. The Zelda games' rich mythology and the appeal of heroic fantasy lend themselves to movie treatment, and the idea of a Ghibli/Hosoda-level production would make any gamer or otaku salivate, but the big question is what to do about Link, a protagonist explicitly designed to serve as a self-insert for the player? Mario at least has a familiar voice. What is it going to feel like hearing the notably silent Link speak?
EXCUSE ME PRINCESS
Hopefully this isn't used as a reference
Maybe Metroid?
There's one movie that should be easy for Nintendo to make work: Metroid. The series is already indebted to the Alien movies, so an off-brand Alien riff that happens to be better than any of the actual Alien movies since 1986 (not too difficult) seems like a sure bet for a hit. Large scale action, explorations of alien worlds that could only be portrayed in animation, just enough scares to fit within the PG-13 rating, a kick-ass female lead, it's pretty much a ready-made blockbuster so long as Nintendo does one thing: DON'T ADAPT OTHER M!