The town castle was built gradually. The parish church was built as the first structure in the 13th century and fortifications were added to it in the 15th century. Earth ramparts and palisades were later replaced by tall stone walls fortified by bastions and a water dike. In the 16th century, the Turkish threat called for further fortifications. Only a quarter of the original town walls and three bastions - Farská (Parish), Banícka (Mining), and Pisárska (Scriveners) - of the original four have survived.
The castle's surrounding area includes not only a parish church and fortifications, but also the Church of the Holy Cross - Slovak Church, which was built in 1452, as well as a barbican with a tower. It used to be the entry gate to the castle. The barbican acquired its present Baroque facade after fire in 1761. Between 2005-2006 the barbican was restored again.