Under the Council of Lorca, a town called “The Alum Houses of Almazarrón” began to grow, where the Vélez laid the foundation of their Castle and the Marquis of Villena built his house. Both noble families moved into the area in the last decade of the 15th century, when Enrique the IV granted the privilege of exploitation of the alum mines to both marquisates.

The Castle has an irregular shape, occupying an area of over 3.000 m2 which can be divided into two zones: the fort and the parade ground.The walls are a meter thick and are directly supported by the mountain’s rock. On the original internal structure of the Castle, the written testimonies of 'González Simancas’ visit to the Castle in 1903 describe that it held remains of several periods: paved areas, doors with worked limestone lintels, decorations created using red stucco, decorative elements, stairs, underground constructions, and other details which have disappeared or become hidden under an old mine excavated inside the fortified place.

The Marquis of Vélez used to stay at the Castle occasionally during his sporadic visits, which was an everyday home for the governor of the villa.

The building also had a small garrison in charge of defending the coast from the Barbary pirates’ attacks. During these threats the Castle also protected the population of Mazarrón, mainly composed of miners.

The Vélez Castle has been declared a Site of Cultural Interest.