The first building on this site was a lodge that housed the slaves who worked in the Company’s Garden. It was built around 1679 on land that originally formed part of the garden.

By 1807, new premises from which to administer the Cape colony were needed, and the Slave Lodge suited most requirements. Many slave inhabitants of the lodge were sold, while others were moved to the west wing of the building. The vacated area was turned into offices. In 1811, the west wing was also converted.

The people responsible for the conversion were the builder Herman Schutte, the sculptor Anton Anreith and the architect Louis Michel Thibault. As well as government offices, the lodge also housed the Supreme Court, the post office and the public library.

The history of slavery at the Cape is illustrated with three-dimensional and audio-visual displays along with text, images and maps. A section that focuses on life at the lodge is currently being developed, based on archaeological and archival sources, as well as the memories of people who trace their roots to the time of slavery in the Cape.