It is on Earth. I don't need to assume it. I can find it on a map. We understand that passage in light of the epistle as a whole. Paul's contrast is not a comparison between the heavenly Zion and the earthly Zion; he compares Mt. Zion with Mt. Sinai. Hebrews 12:18ff describes the day Moses came down from Mt. Sinai and the fact that Israel begged Moses out of fear to keep God away.
Paul isn't suggesting that another Zion exists in heaven. He speaks about a time when Mt. Zion, which is the city of David, will become the city of God and Jerusalem will be heavenly rather than earthly through Jesus Christ.
Another perspective emerges. The idea of converts being transported into heaven is not seen here. Instead, we see 144k Hebrews, marked by God for protection, presumably because of their fear of the Lord and their avoidance of arrogance and evil deeds, as per Malakai chapter 4. They gather to Mt. Zion to meet with Jesus, as per Zechariah 14. There, they are taught the new song, eternal salvation in Jesus Christ.