In 1856 New South Wales was a granted responsible government. This important step in self-government brought with it several new portfolios requiring new office space as well as a greater need for the departmental head and his staff to be in approximately the same space.
During the later years of the nineteenth century, while some former responsibilities were removed from the authority of the Colonial Secretary, it remained a pre-eminently prestigious and important political position.
By 1869 sufficient finance had been raised to construct a new and worthy building for the office of the Chief Secretary of the colony as well as providing offices for the Works Department. James Barnet, the Colonial Architect, designed an impressive multi-storied building to occupy the six lots in an "L" shaped portion of the block fronting bridge Street in the period. In 1880 it was reported that work on the Colonial Secretary's building was completed.
However, by the end of the 1880s space within the building was at a premium. Several of the occupants complained to the effect that, despite the construction of the building, they were in little better situation than had been the case prior to its existence. The Commissioner for Roads complained that he was inconvenience having so many officers so far away. The Engineer-in-Chief for the Harbours and River complained that he had insufficient office accommodation and the Acting Engineer-in-Chief for Railways complained that he lost time because his staff were so widely distributed in various offices.
The only option was to extend the ten-year-old building. Many expansions and renovation have taken place since.