All Saints', as a Parish theoretically began in 1862 but not until 1867 in practice. It's creation was led to by the influx of refugees from Taranaki who were escaping the Maori Wars (over 1000 women, children and old folk arrived in the early 1860's).
To begin with, the services were conducted at the Old Fellow's Hall in Rutherford Street.
In November 1867 the Parish of All Saints was put in the charge of the Rev. R. J. Thorpe and 12 months later the church was erected.
It was built by John Scott at a cost of 931pounds and 7 shillimgs.
All Saints was consecrated on 11th November, 1868.
It was soon to be enlarged in 1884 and then gain the addition of a tower in 1889.