This pattern appears all over the place in Drupal -- including important functions like node_load().
Calling node_load() for a particular node ID requires database hits the first time, but the resulting information is kept in a static variable for the duration of the page load.
If you don't need your cached data to be perfectly up-to-the-second, but you want to keep it reasonably fresh, you can also pass in an expiration date to the cache_set() function.
For example: The final parameter is a unix timestamp value representing the 'expiration date' of the cache data.
If your data is updated sporadically, you might consider simply calling cache_clear_all('my_module_data', 'cache') each time you save the changes to it.
If you're caching quite a few pieces of data (perhaps versions of a particular block for each role on the site), there's a third 'wildcard' parameter: This clears out all the cache values whose keys start with 'my_module'.
Combined with the static variable, future calls during this page request won't even need to call cache_get()!
That means that our function can check if the variable is already populated, and return it immediately without doing any more work.
Drupal's APIs provide three key functions you'll need to be familiar with: cache_get(), cache_set(), and cache_clear_all(). After the initial check of the static variable, this function looks in Drupal's cache for data stored with a particular key.
If it finds it, $my_data is set to $cache-data and we're done.
When modules need absolutely fresh data, they can call drupal_static_reset() to clear out any temporarily cached information.
You might notice that the static variable technique only stores data for the duration of a single page load.