The hardest part of level design in Mario Maker is playtesting. You can't do any sort of limited release and you definitely can't see how everyone plays it even once it does release. All you can do is play it yourself extensively and hope for the best. This is what I've been doing. Designing levels while occasionally testing each part and trying them in as many different ways I can think of, making minor tweaks along the way. I've done this because I believe that making as many different approaches as viable as I can manage is the right approach. I do this and see clear rates in the 10%-30% range and feel good. I’d never seen my levels played, but the average person took a few deaths, clears it and is likely to star it. Seemed a good enough impression to me.

Yesterday however that changed for me. Yesterday, for the first time, I did actually get the opportunity to watch people play my latest level. Beneath The Runway was one of seven courses in the second week of Mario Maker Mondays, a blind race of Super Mario Maker levels which dozens of runners and players participated in. Exciting! I would actually see players (albeit skilled players) take on one of my stages! Unbelievably cool!

The first two levels go quickly. My level comes up. I excitedly IM my buddy watching the race. The commentator sells the idea of this rundown abandoned runway as runners run down conveyors speeding along its length. At the end coins rise into the sky where becaped runners will eventually fly into the sky and soar to the exit. First though they must venture into the unseen area beneath where Boos, Thwomps and lines of yellow blocks breakable through spin jumping lie. Things go about as well as I wanted for the runners. There's a few deaths here and there for the four on stream but they get through the level in short order all the same. My friend complements my stage and I feel good. It showed really well.

The race goes on and racers start to finish around 22 minutes in. They switch out runners further and further behind in levels 7 and 6 until I tune out around 50 minutes in. At least, I tune out until I'm told that my level was back on stream a few minutes later. I didn't see how that was possible. I pulled back up the stream to witness a struggle.

I like to believe my levels are fair. I’ve been conscious that there’s a lot of things Super Mario Maker does not teach. There are people who’ve never played Super Mario World. Players who don’t know how to spin jump, don’t know spin jumping breaks blocks, don't know it allows you to bop on Thwomps, don't know how to fly. I tried to account for this as much as I could. I introduced spin jumping to break yellow blocks by trapping Mario in a room with a breakable floor, a mushroom and the letter R. I showed that Thwomps can be spin jumped on by making the next time you break blocks you landed on a Thwomp. But flight? Couldn’t do it. Didn’t see a way to test player’s understanding of that before the actual flight.

You can imagine my joy as Mario Galaxy runner allyoucaneat1620, who’s never played Super Mario World, cleared the last actual jump in the level (tricky to do if you don’t hold Y in the air turns out) and you can imagine my horror as allyoucaneat1620 went out onto the runway, jumped up and slowly floated to down to his death. What was meant to be the ultimate payoff backfired. I didn’t like it any better when it happened again a few minutes later.

I panicked and did what you shouldn’t do during an actual playtest. I went to him stream and explained that he had to hold down Y to fly. You really shouldn’t do this because you’re not always going to be there to explain to everyone who struggles in this way. I wasn't exactly alone in this but still. Soon the main stream ended and hundreds of viewers flocked to cheer him on. His Twitch chat became a blur of “I believe you can fly!” Once he got back to the end he took a few quick practice flights over the runway itself and, after confirmed he could in fact fly, he soared off towards the exit finally finishing my level. Course cleared.

Whether or not you see them players like allyoucaneat1620 exist and that’s what playtesting can highlight. It reminds you that not everyone plays in the same way, even if you honestly believed you were accounting well enough for that. In all my tests I never tried doing runs where I let go of Y because that would honestly have never occurred to me. Additionally it had never occurred to me that the space over the runway could be used to practice cape flight for those who are inexperienced. If I had seen allyoucaneat1620 play my level before it went live maybe I could’ve put some extra coins above the runway to encourage that sort of bonus flight and give him people who struggle with flying the idea to experiment sooner. Those are things you don’t see without playtesting.

Currently Beneath the Runway sits at 14% completion rate which is well within my normal range, but I feel uneasy about it. It turns out looking at a statistic is one thing and seeing something struggle is quite another. The difference between suspecting there might be issues and seeing and believing. That's the difference playtesting makes.

Nice piece there! There are a whole suite of features I think would benefit Mario Maker, tools for playtesting and fine tuning released levels being one of them. Interesting to hear your personal experience with it!

Great article, and great level duder! Took me a while, but I got through it eventually! :)

Great article and super cool to see the game be taken so seriosly by - at least some - of the player base. Will definetly check your stages out once I get around to buying the game :)

Interesting read! I'll have to look up your course later on.

I'm currently designing my first course in Super Mario Maker. It is in the World tile set and I have been play testing it throughout. The idea is Mario is escaping from a castle and the enemies are out to get you since you escaped from your cell. You have to keep moving and find the exit. If you stay in an area too long enemies start coming out of pipes, boos will surround you, or traps will make the level more difficult. You aren't supposed to fight everything off. You're supposed to keep moving and head for the exit.

For me, I would rather have courses be fun than difficult. (No Giant Bomb community levels for me, haha) So I've been constantly play testing my course and if I keep dying on something, I would add in another mushroom or just take or change whatever is killing me. I'm having fun playing through it and it is currently "done", but I'm worried it may be a little too difficult. It is fairly easy to clear, but I also know exactly what to do. I know where you have to go. I know you have to spin jump through these blocks. I know how to use a cape and use it to clear this area. So yeah, I guess my course has a similar problem.

If you don't know those two specific mechanics of Super Mario World, then you will just hit a roadblock and the course is impossible. I don't really think like there's a lot I can do about this though. It's weird to say, but it's not really my job to tell somebody how to fly. From a game design perspective, that statement is kind of nonsense, but at the same time it's a custom level to a 20+ year old game and any tutorials like that should probably be somewhere in Super Mario Maker itself.

I watched a little of allyoucaneat1620's initial time with my level which I didn't see the first time and have some thoughts. The biggest take aways discussed in the original post focused on his biggest stumbling block but that doesn't mean the little difficulties at the start aren't worth considering. All these thoughts will be centered around the following screen of content.

Very simple set up here. Basically you're to jump up the platforms on the right, go around through the one way door, grab the mushroom in the block and, finally, hit RT to spin jump and break your way down before returning to the left and jumping back up and out. A very simple and effective little piece I believe. The main goal of the level is finding a Cape so making a small puzzle around the mushroom makes a lot of sense. Additionally this serves as an effective test/demonstration of how Big Mario can break blocks by spin jumping.

Here are a few notes about this segment from watching it.

• He initially tries to run and jump his way in. This is fine, he realizes he can't soon enough. Nothing really to be done about this.
• He then tries luring ghosts out to the left. I don't see why initially, I figure the path is clear enough. However, upon reflection, my guess would be that he did not know that bridges are jump-throughable. He likely thought he had to lure ghosts out and do a tricky jump around the outside. Maybe I ought to have included a gap in the bridges to help him realize to just jump up them? Maybe not.
• He spin jumps a few times while trying to do the above two solutions. I suppose the R can be confusing before you get to the locked room.
• Once he gets huge he stomps down successfully quickly. Score one for the R.
• One huge he messes up the jump by jumping too soon and hitting his head on the ceiling below the R. I detest bonking heads on ceilings, I'd have make slight tweaks to prevent that probably if it'd have happened to me. Either by moving this whole sequence right one or by raising that ceiling up by one. Hard to say. I'd currently favor moving the ceiling up one but at the time I might've favored lengthening the bridge by one. Originally it was actually longer before I shortened it to try to make the level a bit faster.

Those are a few minor thoughts about a small piece of the level that really didn't give allyoucaneat1620 too much trouble but is still worth thinking about.

By the way every star counts towards me getting major props and chance at 100 dollars down the line! Right now I'm in third, but 2 stars would put me in first! It's neck and neck! Please star me!

So last few days I've found out there's actually a pretty reliable way for nearly anyone to get the same sort of experience I got. It's pretty straight forward! Simply go to Twitch, look for Super Mario Maker streams, choose a small one with a few viewers that looks like it accepts requests, request your stage, wait a bit possibly while doing something else, and voila. Your stage is being played before your eyes. It's really doable! Also a great way to keep stars coming your way if your stage is good. The streams don't seem to convert into viewers playing your stages like it might for streams with hundreds of viewers, but the streamer alone playing your stage is enough when every star counts I think.

Go ahead and give it a shot! It's pretty great!