The White House has been called many names. Dolley Madison, wife of President James Madison, called it the “President’s Castle.” The White House was also called the “President’s Palace,” the “President’s House,” and the “Executive Mansion.” The name “White House” didn’t become official until 1901, when President Theodore Roosevelt officially adopted it.