One option is to finish with math and then do graduate school. You could try getting a minor (is that an option?).
I attended college because I wanted to and I knew what I wanted to do. It hasn't really worked out for me, but I don't regret going to college, I only regret the fact that it counts for nothing. It took me a year to find even a part time job, and all the jobs I do apply for tell me I'm "overqualified" and refuse to hire me.