The painting was in 1650 in Antwerp, Van Verle collection. It was purchased in Flanders for Don Luis Méndez de Haro y Guzmán, nephew of the Count-Duke of Olivares and who was 6th Marquis of Carpio and 4th Count-Duke of Olivares. Upon his death, he passed into the possession of his son, the 7th Marquis of Carpio, thus entering the Casa de Alba. It is one of the 32 paintings that passed into the hands of the 7th Duke of Berwick and 14th Duke of Alba, Don Carlos Miguel Fitz-James Stuart, when the 13th Duchess of Alba died.
Ángel M. Barcia in his Catalogue of the Collection of Paintings of His Excellency Mr. Duke of Berwick and Alba, 1911, after collecting various opinions, set the table as Raphael's, following the attribution of a list from 1931. Más already includes the judgments of Passavant (who cites him with number 310 in his catalog of works by Raphael, as a work of probable Venetian school), and by Berenson, who already considers it, without hesitation, as of Palma the Elder. Professor Lafuente Ferrari wrote the following note on this painting: “Spahn's book on Palma Vecchio talks about paintings from the Liria Palace...”. Saphn's words lose some force when he confesses that he hasn't seen the paintings, but they have value from the point of view that he knows the bibliography very well and all the opinions about the paintings that have been given in well-known works. The most interesting thing is that it fixes a fact about the origin of Palma's portrait of a man, which is provided by an engraving of this same painting made by the famous engraver Wenzel Hollar. The paragraph that Spahn dedicates to the portrait of Palma goes like this: “Towards the beginning of the third decade, that is, around 1520, a portrait of a man can be placed (within Palma's work), about which I must make the reservation that I have not seen him. Palma's characteristic features are the penetration and sharpness of the eye. In the strong neck and in the clear and sober conception of the portrait, the portrait of Madrid is similar to that of Petersburg; in the posture and in the arrangement of the dress it seems like the development of a similar idea. In the robust way of filling the space, in the heavy type, in the wide magnificence of the costume, the painting of Madrid is related to the Conversazione Colonna (rep.47) and to the three sisters (Dresden Museum, rep.55). The contour of the neck is reminiscent of the sister who appears in profile and the way in which the right sleeve of London is treated (National Gallery, rep.60). His right hand is comparable to the hand of Saint Joseph with the staff of Serina (Church of the Anunziata) and is even closer between the relationship of the hand, also thick and delicate, of the sister seated on the left pointing to the center wing, in the Dresden painting. It can only be seen that the glove was added later, as can be seen from the comparison with the beautiful engraving by Wenzel Hollar (rep.49) made according to the painting that was then in the Van Verle collection in Antwerp.” Professor Lafuente notes that, in addition to Berenson, they confirm the attribution to Palma, Borenius, Cook and Venturi. Wenzel Hollar's engraving bears the following inscription: “Arcolano Armaphrodite dato da Coregio la Istessa Divinitá. F. vanden Wyngaerde excudit. W. Hollar Fecit. A. 1650. Former Joannes & Jacobi van Verle collection.” Dr. Don Arturo Perera identifies the character with Francisco Guarini or another individual from the same family.