While I'm still pretty baffled by the world of cryptocurrency, this piece by the New York Times really laid it out in layman's terms. Hope it helps you out too!
Basic Run Down:
1. Digital money that is bought and sold online
2. Doesn't go through traditional institutions like banks
3. Is not linked to any concrete substance like gold
Alice wants to buy a bike from Dan using Bitcoin. Alice logs into her Bitcoin wallet with a private Key (pin, series of numbers unique to her) Now usually, the money she's sending would first go to a bank who would record the money being moved and transfered, but with Bitcoin, there is no bank or middle man.
Because so many people are mining, this is meant to ensure that no one computer will monopolize the market, like banks have done in the past.
As for the math problem, the more computers that join the race to solve it, the harder the math problem is. This is why people are fighting for the fastest computers and huge servers to help them become first. Apparently, Bitcoin is set to be available for mining until there are 21,000,000 Bitcoin in circulation. Right now, that's set to happen in 2140...