The Renaissance palace, whose design, for many years was attributed to Raphael, was almost certainly not his direct hand, but a design from one of his pupils, perhaps Lorenzo Lotti, called "il Lorenzetto". Built between 1515 and 1536 for Bernardino Caffarelli. In the 18th century, the building became property of cardinal Vidoni, who enlarged the building. In the early twentieth century it housed the German embassy, now it houses the Ministry of Public Functions.
The building design harkens to a classic Renaissance design likely developed by Bramante in his design for the now destroyed Palazzo Caprini, the home of Raphael. The home has successive layers where the degree of facade decoration matches the function of the interiors. The ground floor is rustic stone, almost fortress-like. The second floor, the piano nobile, is decorated with doric columns.
The Palazzo Vidoni-Caffarelli conserves 16th-century frescoes in its Charles V hall that depict events in the life of the emperor. The author is unknown, but described as belonging to the Mannerist school of Perin del Vaga. The palace also contains some 18th-century frescoes attributed to Nicola Lapiccola, Bernardino Nocchi, Tommaso Conca, Ludovico Mazzanti. The interior has some Roman statues, including a statue of emperor Lucio Aurelio Vero, and a fountain derived from a Roman sarcofagus. It also contains a marble portal decoration of the Venetian lion of St Mark from a Croatian city.