Salem

Seems like every time I head out exploring I get lost and have to drop all my items, Is there an easy way to not get lost or do I just need to be more careful?

just zoom out on your in-game map and walk the way back.

Try exploring in the cardinal directions. For example, try to explore more or less straight East, so that you know you need to go straight West to get back.

Another tip is to try to find a nice big landmark by your base that you can use to track back home. For example, follow along a river, so you just need to follow the river back to get home, or follow along a mountain so you can easily track you way back.

If you aren't already, try using an alternative client (like Enders) that will give you a bigger map that you can move around, and zoom out of.
Enders also has an alternative to the standard client's drunken camera, which also helps with navigation.

Thanks

Ender's will also show ridges on the minimap, which can be helpful on occasion.