From colonial times to the early 20th century horses were the primary means of transporting the mail, whether by a single rider on horseback or by wagon or stage coach pulled by a team. Certainly mail was also transported by railroad as as the network of “iron horses” developed, as well as by river- and ocean-going vessels.
The image below is a detail from a poor quality photo postcard from circa 1916. There are two men on horseback. The man on the right is identified as W. H. Bennett of McKee, Kentucky, whose age (in 1959) was 87 years. His horse has two mailbags, one on each side to distribute the burden. The other man is not identified. There are trees and other vegetation behind them but no buildings are visible.

This postcard was privately printed. It is in the U.S. National Archives by an accident of history. On 11 August 1959, Postmaster D. N. Thomas of McKeesville forwarded the postcard along with two leather saddlebags to the Post Office in Cincinnati, Ohio, as “Items for a Postal Museum.” The mailbags were “not desired” and thus were “left in Cincinnati.” The postcard, however, ended up in a collection of “Exhibit Materials Relating to Postal History, 1905-1958” (National Archives Identifier 17027514) assembled by the Post Office Department Library that was subsequently accessioned into the National Archives.