White House a 'ghost town' as infectious president returns