I've only ever known Tales From Earthsea as Studio Ghibli's black sheep. But I don't think this movie deserves all of the hate it's gotten over the years. Ghibli movies are notorious for their art direction and soundtracks, so I'm going to just focus on the plot and how it relates to the books it's based on.
This movie gets a lot of it's fame from being Ghibli's worst movie, even Hayao Miyazaki has gone on record to publicly criticize it. The downfall of this movie, among other other things, had even caused the director and Miyazaki's son, Goro Miyazaki, to stop perusing animation for a time. But Goro Miyazaki had a very dense and complex narrative to work with.
If you didn't know, this movie is based off of the book series by Ursula LeGuin, who is an accomplished writer that has helped shape the modern fantasy and sci-fi genres. That being said, her works are nearly impossible to translate to film!
(A little bit unrelated, but if you really want my two cents on this, I think Ursula LeGuin's books are okay. Her books aren't bad, they're not great. Just some solid okay books)
The two most common things I've heard about this movie is that it is confusing and boring. While Goro was an inexperienced director, I think some of the fault also lies with Ursula LeGuin's writing. Ursula's plots have pacing issues and are littered with chapters upon chapters where seemingly no action takes place. Rather some of the biggest fights are the characters fighting with themselves (ie, Arron having a good an bad side to himself to the point where his biggest fear is himself). Ursula's stories do have magic battles and sword fights, but I feel like this heavily introspective storyline would be difficult for even some of the more experienced directors.
An okay book series can only be expected to make an okay movie. But it is unfortunate that this was Goro's first directing position at Ghibli. I believe that Hayao Miyazaki was originally going to direct this film, so the big question for me is what would have changed under the direction of Miyazaki senior? Would this movie still have been considered one of Ghibli's worst?