Whilst not absolutely necceary for finding caches, A compass comes in handy to check your bearing to the cache from time to time.
If you set you GPS display to show distance and bearing on the same screen as the pointer thingy then you can cross check with your compass that you are heading in the correct direction. This may sound obvious "Well DHa! i have a GPS!" but that pointer thing can swing you around especially as you apporoach ground zero and you step within the accuracy limits of the current sattelites.
Below are some links to lessons on using a Compass.