Here's a quick summary of just how thoroughly everything we ever do will be wiped out (along with some time frames for each).
1. Weather/earthquakes (hundreds to thousands of years) - after a few thousand years, even the Pyramids are being eaten away. Even the worms tend to eat away the soil under structures, slowly sinking them into the ground over time.
2. Ice Ages (tens of thousands of years) - For the last several hundred thousand years or more, this part of the Northeast has been going through cycles of Ice Ages, each lasting tens of thousands of years. Nothing survives a 3000 foot thick glacier sliding over it.
3. Super Volcanoes (hundreds of thousands of years) - Turns out Yellowstone park sits on a giant caldera, a kind of "super volcano" that has erupted massively every 600,000 years or so. The last super volcano eruption in Toba, Indonesia is thought to have nearly wiped out the human race (though we had only just mastered fire way back then).
4. Celestial Impacts by Comets/Meteors (millions of years) - These come in ranges. Take the timescale out long enough and you get something big enough that will hit so hard it won't matter where on the Earth it lands. If you want, you can see just how baked you'll get with this fun site.
5. The Sun (2-5 billion years) - Our Sun will not last forever. And when it runs out of fuel, it's going to become a "Red Giant." It will expand in size to completely engulf Mercury and Venus and the current orbit of the Earth. There is some uncertainty whether the Earth will be buffeted outward during the process and escape total annihilation, or bake much sooner than that even, but it won't be pretty either way.
6. Expanding Universe (tens or hundreds of billions of years) - Current observations suggest the universe is expanding, and doing so ever faster. Even if we were able to engineer the planet out of the Sun's harm, there's no escaping this one - it's a fundamental aspect of space-time itself.
I'm sure there are more. Just go to places like this to immerse in it if you want - or watch Discovery channel!