Grace Lutheran Church was officially formed in 1979 due to the efforts of Elaine Holmes who contacted the Southeast Synod office of the Lutheran Church of America in late 1977 and inquired about starting a Lutheran church in Carrollton. A year later the LCA formally approved a mission church for Carrollton and soon after appointed Pastor Gregg Kaufman as the mission development pastor.

After six months of going throughout Carrollton in search for families who wanted to join the church, Kaufman found approximately twenty families who agreed to meet at St. Margaret's Episcopal Church in November 1978 for a pot-luck dinner and to become acquainted. Pastor Kaufman continued to arrange similar meetings at other local churches during the following weeks while other sites were considered as a permanent area to hold worship services. In 1979, it was decided that services would be held at Oak Mountain Academy and the first worship service there was set for May 1979. Grace Lutheran Church continued to meet at Oak Mountain Academy for seventeen months before moving their service to a storefront at the corner of Stewart and Brown Streets in 1981.

During the church's time at Oak Mountain Academy and Stewart Street, the Southeast Synod office of the LCA bought a tract of land for a permanent church to be built near Highway 166 and the Carrollton Bypass. Although it held multiple outside services at this location, it was not until 1982 that Grace completed all the requirements of a new church and the Synod gave its blessing for a building to be built there. Establishing a building committee that year, the church finally began to be built in June 1983 and the first service was held there in January 1984. All church services since then have been held at this location and Grace Lutheran Church continues to operate today.