Villa Imperiale is one of the most interesting examples of Mannerist architecture in Italy. The foundation stone was laid in 1468 by the Emperor Frederick III of Habsburg while he was a guest of Alessandro Sforza in Pesaro; it took two centuries to complete the villa.

In 1521, after regaining the Duchy usurped by the Medici, Francesco Maria I Della Rovere commissioned the architect Gerolomo Genga to add another building, thereby transforming the whole into a holiday residence (1523-’38). The Della Rovere wing was conceived as a walled organism also enclosing the three gardens on the hill, where aromatic herbs typical of Classical and Renaissance cultures were grown. Genga was also responsible for coordinating the decoration of the interior, on which, among others, worked Dosso and Battista Dossi, Raffaellino del Colle, Bronzino and Francesco Menzocchi. The themes represented in the frescoes are nature, antiquity, love and the feats of Francesco Maria I.
In 1635, on the marriage of Ferdinando II, the last Della Rovere heir, to Vittoria, the villa became the property of the Medici and then passed to the House of Lorraine. It was sold to the Apostolic Chamber and granted in perpetual emphyteusis by Pius VI to the Albani family, who are the present owners.