"The fourth wall" is an expression stemming from the world of theater. In most modern theater design, a room will consist of three physical walls, as well as a an imaginary fourth that serves to separate the world of the characters from that of the audience.
In fiction, "breaking the fourth wall" often means having a character become aware of their fictional nature. This can range from a character advising the player to "Press X" or "Press A" in a tutorial, (referring to a controller button that does not exist in the game) all the way to Psycho Mantis reading the player's memory card and mentioning the other games they've been playing.
However, the most conventional violation of the fourth wall is when a character (or the game itself) openly acknowledges they are in a video game, or directly refers to the player.
Breaking the 4th wall in games is often used for a comical effect or to re-enforce the fictional natures of the game. Games that break the wall in this fashion include:
Tutorials that introduce new games to players are the most common way of breaking the 4th wall, often instructing players to press buttons on their control pad either directly to the player or as the character being played. For example, within the latest Pokémon games that introduced running shoes, the instructions read “Press B and blaze new trails of adventure!”. Obviously there is no B button for the character to press, so the instruction is directed at the player, thus breaking the 4th wall.
Paper Mario also makes references to the players control, something that Mario doesn’t understand but is promptly told that a being "watching from another dimension" will.
The Metal Gear franchise contains some of the more popular examples of breaking the 4th wall within a video game story event. During Metal Gear Solid 2: Sons of Liberty, the protagonist Raiden is contacted via codec by his boss, the Colonel. Raiden is falsely told that "This mission is a failure" and is immediately ordered to "Turn the game console off right now!". When the protagonist questions his orders, he is told "Don't worry, it's a game! It's a game just like usual" before the Colonel involuntarily reveals him self as an AI program which has started to crash. The story event suddenly climaxes when the game's familiar "Game-over Music" is played and a (purposefully) misspelled “Fission Mailed” screen appears and the game cuts to black. This unique plot device is effectively used to simultaneously tear down both the protagonist's and the player's current perception as to "who and what he can trust" within both the context and reality of the game.
Other Story Events from the Metal Gear franchise also break the 4th wall. Metal Gear 2: Solid Snake makes reference to a package the player needs to obtain in order to find a relevant frequency, but it turns out the game is referring to the actual game's packaging, a puzzle that was later repeated by its sequel Metal Gear Solid. In Metal Gear Solid, when the villainous Psycho Mantis is confronted, he "reads the player's mind" (actually their memory card) and comments on other Konami games they've played, most famously saying "You like Castlevania, don't you?" if they have a save for Castlevania: Symphony of the Night. He also "psycho-kinetically moves the player's controller" by activating the vibration feature. During his boss battle he appears invincible, but Solid Snake's support team then informs him (via codec) that Mantis can be beaten if Snake "unplugs the controller from controller port 1, and then plugs it back into controller port 2." Also, within Metal Gear Solid 3: Snake Eater, if a player attempts to use an auto fire enabled controller during a particular torture scene, a character within the game exclaims "Don't even think of using auto-fire or he'll know!" (a reference to players cheating during a previous Metal Gear Solid game in order to easily pass a similar torture scene.)
Metal Gear Solid 4: Guns of the Patriots continues when Otacon instructs Old Snake to swap discs, only to realize that the game is on a high capacity Blu-Ray disc, and doesn’t need to be swapped out. Guns of the Patriots also acknowledges the famous scene from MGS1 through the spirit of Psycho Mantis; he complains that he cannot read the player's saves because he doesn't understand the PS3 hard drive, and complains about not being able to psycho-kinetically move the controller because the Sixaxis controller does not have a vibration device. (Unless the game is being played with a DualShock 3, of course, in which case he replicates the scene and triumphantly shouts "Rumble's back!")
Commenting on player actions
Another common way to break the wall is to have something or someone within the game's reality to comment on the action of a player. Clicking multiple times on units within many Blizzard games will cause the unit to respond with several unusual, often comical, audible comments.
Characters who break the 4th wall through character-awareness comment or react to the fact that they know that they are in a game. In Jak 3 a monk yells at Jak and Daxter "This isn't a game!", causing them to look at the player with confused expressions. In Paper Mario: The Thousand Year Door, a disguised Lord Crump addresses the player as "you in front of the TV!” and asks you not to tell Mario who he is. In The Simpsons Game, the Simpsons discover they are in a video game, that is being played by God, who appears as a character on Ralph's computer. Ralph then appears to notice the player.
Referencing the player
This consists of the game treating the person playing the game as an actual character or directly asking the player to do something. Perhaps the most well known examples come from Earthbound games: in the SNES game, one of the characters asks the player to enter their name. Later, at the end of the game, when everyone is praying for the defeat of Giygas, the player is the last person to pray for Ness and friends. Mother 3 also asks the player to give their name, but this time, the player is referenced in the ending as one of the few people to survive the events at the end of the game.
In a small number of games, the player is conflated with another character who observes and controls the action; in LifeLine, the player must give audio instructions to the character on-screen as someone observing from a distant security room. In Baten Kaitos, the player is actually a spirit watching over the in-game party; characters refer to the player as a separate entity from the character they control. Whether this qualifies as breaking the fourth wall is debatable, as these are references to a character that represents the player, rather than the player themselves.
In Panzer Dragoon Saga, the player is asked to input their name at the beginning of the game. This may seem odd, as the protagonist's name is Edge, regardless of what name is entered. However, the ending references the player and directly commands them to "press the button." In fact, the ending won't continue until the player presses any button on the Saturn controller.