An Underground Railroad route superintendent in Girard, Drury was easily moved and morally outraged by the sight of cruelly treated runaway slaves. He hid them in his barn and often in a nearby ravine near his property along Elk Creek where they awaited secret conveyance to Canada. Family lore holds that one of the graves on the Drury family cemetery is that of a savagely beaten fugitive who died on the property.
According to a 1938 interview with Effie Shipman, daughter of Underground Railroad conductor and Unitarian Universalist Church minister Charles Shipman, Elijah Drury was, “a man whose heart was filled with the milk of human kindness.”