This way madness lies....

The red gem Bayonetta carries is The Left Eye of the World.  Cereza isn't a "symbol" for Bayonetta, she is Bayonetta as a child brought into the future.  I *think* the "strange clocks" are just Umbra jewelry or some such that all witches wear.  Baldur wanted to awaken Jubileus and create a new world.  Jeanne was brainwashed (or something) by the Lumen Sages to fight against her old friend and rival.  The Lumen Sages killed Luka's father because he getting close to uncovering their plans, he's the guy who wrote all those journals you find throughout the game.

"The red gem Bayonetta carries is The Left Eye of the World.  Cereza isn't a "symbol" for Bayonetta, she is Bayonetta as a child brought into the future.  I *think* the "strange clocks" are just Umbra jewelry or some such that all witches wear.  Baldur wanted to awaken Jubileus and create a new world.  Jeanne was brainwashed (or something) by the Lumen Sages to fight against her old friend and rival.  The Lumen Sages killed Luka's father because he getting close to uncovering their plans, he's the guy who wrote all those journals you find throughout the game. "

Bayonetta herself is the left eye.  

• Cereza was there to awaken Bayonetta, the left eye's, true potential. To lure her to him among other  things I guess.  
• Why wouldn't Baldur wanna create a new world? It's tradition apparently! And he's mad, so does he need a logical reason?  
• Yes, Bayonetta was expelled. But all the other Umbra Witches died so what difference does it make? And the didnät want Jeanne to kill Bayonetta. Once again that had something to do wtih the awakening of the left eye, to get Bayonetta to realize her true potential so they could ressurect Jubileus. 
•   Again, Bayonetta herself is the left eye.  The gem is just some awesome, magical artifact? Jeanne used it to shield her self from that blast if you remember?

"Don't you wanna know that stuff? "

The story is that Bayonetta is Canadian. That is all.

I like the story....and I understand it well, I think.  I watch too much anime...

As best as I can guess...
 
Why is Cereza brought into the future?
To awaken the Left Eye which is Bayonetta/Cereza. According to Balder, she had to see herself as a requirement of awakening the Left Eye in addition to all of the other experiences she had since emerging from her sleep 20 years earlier. She didn't know it was her, but perhaps her adventures in protecting her younger self helped to awaken more of her memories.

Why does Baldur want to create a new world?
Not him, personally, he was more of a tool (he was the Right Eye) willing to do anything in order to resurrect Jubileus whose return would unify the Trinity of Realities. He had tried to resurrect Jubileus 500 years earlier and had failed when it broke out into open warfare.

Isn't Bayonetta an expelled Umbra-witch? (If so, why do they want Jeanne to kill her?) 
Bayonetta was the daughter of an Umbra Witch but while her mother was an exile because of Balder, Bayonetta was raised in their order (even though she was considered an outcast). Jeanne had mental conditioning applied to her by Balder's power, turning her against Bayonetta in order to awaken her memories. After their last battle, the conditioning apparently broke allowing Jeanne to come to her senses.

What is this Left Eye of the World Gem for? I mean Baldur doesn't need it for Jubileus.
The gems are just rocks meant to mislead. The entry on the "Eyes of the World" under Antonio's Notes has a picture of a gemstone 'claimed' to be part of a set named the Eyes of the World, but he goes on to say that no one really knows what they look like. The Cardinal Virtue, Sapientia, had said as much, calling Bayonetta's gem a "worthless rock", hinting only at the real Left Eye.

" @TsuHai said:

"Don't you wanna know that stuff? "

No. When I'm headbutting skyscrapers and quadruple-wielding guns that shoot lipsticks, I really couldn't care less about the story. "

The gemstones themselves mean nothing more than a 'badge of office' so to speak. Father Balder had the power to open the right eye because he was single-minded, determined and devoted. He possessed the right amount of zeal and confidence that his conviction of wanting a new beginning would be recognised by the Creator. Bayonetta, in her 'original' persona was not strong enough, due to her doubts, her overall muddled memory, and her touch of weakness that was portrayed in Cereza when she was first met. Balder needed to let Cereza see and experience Bayonetta first-hand so that Bayonetta, as a matronly figure to Cereza, would instill the same kind of unwavering confidence and devotion in Cereza, which is what Balder needed Bayonetta to have to awaken her potential as someone who could, possibly, have the confident conviction of beginning a new history necessary to awaken the Creator. When Cereza was sent back in time, this changed the persona of the Bayonetta in the future, and the 'left eye' was opened.
 
Balder explains that a little brainwashing was required to get Jeanne to play her part in strengthening the resolve of Bayonetta.