We formulate and study a multi-user multi-armed bandit (MAB) problem that exploits the temporal-spatial reuse of primary user (PU) channels so that secondary users (SUs) who do not interfere with each other can make use of the same PU channel. We first propose a centralized channel allocation policy that has logarithmic regret, but requires a central processor to solve a NP-complete optimization problem at exponentially increasing time intervals. To avoid the high computation complexity at the central processor and the need for SU synchronization, we propose a heuristic distributed policy that incorporates channel access rank learning in a local procedure at each SU at the cost of a higher regret. We compare the performance of our proposed policies with other distributed policies recently proposed for opportunistic spectrum access. Simulations suggest that our proposed policies significantly outperform the benchmark algorithms when spectrum temporal-spatial reuse is allowed.