The Churchill War Rooms & Museum is underground.  The entrance is to the 
right of the stairs, under these massive buildings.