An Isometric View of the Temple Mount and the Vale of Kidron, Jerusalem
(From "The New Jerusalem" by Arthur Kutchner, 1973, The M.I.T. Press)
There is an opinion that the Calif's Palace was directly on the back of the Aqsa Mosque.
It could make easy for the Calif to go and out of the mosque from his house.