The home was built in 1882 for Capt. Woodbury, and his wife, Martha. The home was the headquarters for a ranch that included a two-story bunkhouse for the ranch hands. Citrus groves and vineyards were planted on the land. Capt. Woodbury planted the deodar cedars that line the contemporary Santa Rosa Ave., also known as Christmas Tree Lane, another National Register site.
In 1892, Capt. Woodbury sold the home to a man named Walker, who in turn sold it to Hampton L. Story in 1894. During the decades following Story's death, the home became a coffee shop, sheriff's station, a fire engine house, office building and a tea house.