He spoke of both. If you want to insist that He would refer to the church as "these great buildings", then I guess I can't stop you from believing nonsense like that which comes about by way of your supposed spiritual discernment.
Now, if He would have said "this great building" instead of "these great buildings" then I would consider whether or not He could have been referring to His body or the corporate body of Christ that Paul described as "the building fitly framed together groweth unto an holy temple in the Lord: In whom ye also are builded together for an habitation of God through the Spirit" (Ephesians 2:21-22).
The Lord spoke of His People, His builders that the "great buildings" of the city and temple represents. They are Christ's congregation. Stones=People. The Lord did not talk about physical destruction of physical stones and building here.
Mar 13:1-2
(1) And as he went out of the temple, one of his disciples saith unto him, Master, see what manner of stones and what buildings are here!
(2) And Jesus answering said unto him, Seest thou these great buildings? there shall not be left one stone upon another, that shall not be thrown down.
Now read carefully what Mark wrote:
Mar 12:1-11
(1) And he began to speak unto them by
parables. A certain man planted a vineyard, and set an hedge about it, and digged a place for the winefat, and built a tower, and let it out to husbandmen, and went into a far country.
(2) And at the season he sent to the husbandmen a servant, that he might receive from the husbandmen of the fruit of the vineyard.
(3) And
they caught him, and beat him, and sent him away empty.
(4) And again he sent unto them
another servant; and
at him they cast stones, and
wounded him in the head, and
sent him away shamefully handled.
(5) And
again he sent another; and him they killed, and many others; beating some, and killing some.
(6) Having yet therefore
one son, his wellbeloved,
he sent him also last unto them, saying, They will reverence my son.
(7) But
those husbandmen said among themselves, This is the heir; come, let us kill him, and the inheritance shall be ours.
(8) And they took him, and killed him, and cast him out of the vineyard.
(9) What shall therefore the lord of the vineyard do?
he will come and destroy the husbandmen, and will give the vineyard unto others.
(10) And have ye not read this scripture;
The stone which the builders rejected is become the head of the corner:
(11) This was the Lord's doing, and it is marvellous in our eyes?
The certain man is God.
His servants were His Prophets.
His son was of course Christ (God Himself)
His husbandmen were the Jews.
The vineyard was the congregation (kingdom representation).
The Jews rejected Son. God did destroy them and took the vineyard (kingdom representation) from them and gave to the church. Again we read in Matthew 21...
Mat 21:39-43
(39) And
they caught him, and cast him out of the vineyard, and slew him.
(40) When the lord therefore of the vineyard cometh,
what will he do unto those husbandmen?
(41) They say unto him,
He will miserably destroy those wicked men and
will let out his vineyard unto other husbandmen, which shall render him the fruits in their seasons.
(42) Jesus saith unto them, Did ye never read in the scriptures,
The stone which the builders rejected, the same is become the head of the corner: this is the Lord's doing, and it is marvellous in our eyes?
(43) Therefore say I unto you,
The kingdom of God shall be taken from you, and given to a nation bringing forth the fruits thereof.
The former husbandmen were the Jews of the Old Testament Congregation who have rejected Son and slew Him. In judgment, the Lord took the kingdom representation (vineyard) from them and gave to another husbandmen which is CHRISTIANS. The very people you have mentioned in Ephesians 2:21-22. We, as Christians are the builder of God's New Testament Congreation where Christ is now a chief corner stone of that building! Did The Lord talk about physical stones and buildings here? No? Same goes with former husbandmen, Jews that God saw as stones of the Old Testament buildings that was falling! In other words, they no longer represents God's Kingdom. The physical temple does not have to be destoryed to confirm this. It already took place at the Cross where Christ prophesied that they (the Jews) will come and destroy the temple and the sanctuary by put Christ to the Cross! In other words, they have de stored their kingdom representation. So they are the stones falling that Christ talked about. Spiritual discernment!
But no... just like the Jews of old, you are blind to believe that the Lord was talking about physical temple and stones falling to build doctrine upon.