The characters, the world, the concept, the art and the animation are all pluses. The story has its ups and downs, and fan-service was used in a way that actually took away from some parts of the show. Fan-service in and of itself is fine, but when boob physics need to be accounted for in the winning of limbo, unless the show is meant to be funny, perverted or both, it kind of takes the viewer out of the story.
I will say, I would be giving it a 6/10, but the last 4 episodes earned another point out of me. Episode 11 in particular did use a cliche and stupid concept for an episode, but it was pulled it off impressively well. Episode 11 in particular I would give a 9/10.
I'd say this one is worth watching, I wouldn't go too far out of my way to watch it or watch it over many others, but I don't regret watching it in any way. So if you're looking for a fun anime to watch while and don't have too terribly high standards at the moment, then sure, Musaigen no Phantom World is right for you.
