Some Jews believe the Tomb of King David is in a lower room in the same building. Jewish protesters gathered at the venue several times this month to denounce what they said were Israeli plans to hand over parts of the site to the Vatican.
Israeli authorities deny any such intention. The hall on Mount Zion, just outside the walls of the Old City of Jerusalem, was constructed by the Crusaders and renovated by the Franciscans during the Middle Ages.
Police spokesman Micky Rosenfeld said 150 people took part in Sunday's demonstration and that some protesters threw rocks at police forces, who took 26 people into custody.
Francis, on his first pilgrimage to the Holy Land as Pope, flew on Sunday from Jordan to Bethlehem, the traditional birthplace of Jesus, in the occupied West Bank. He is scheduled to travel on to Israel later in the day.
Last week, Israeli police and the Shin Bet internal security service handed restraining orders to some right-wing Jewish activists, whom they said intended to cause disruptions during the pope's brief visit.
Earlier this month "Death to Arabs and Christians and all those who hate Israel" was daubed in Hebrew on an outer column of the Office of the Assembly of Bishops at the Notre Dame Center in East Jerusalem.