As you say, it was certainly no secret that all the nations would be blessed in Abraham, but there is something that was kept secret until God revealed it to Paul.
Rom 16:25,
Now to him that is of power to stablish you according to my gospel, and the preaching of Jesus Christ, according to the revelation of the mystery, which was kept secret since the world began,
First of all, the word "mystery" in the KJV and other English translations is the Greek work "musterion" which Strong's Concordance defines as: "a secret of 'mystery'"
Abbott-Smith's Manual Greek Lexicon of the New Testament says: "that which is known to the μύστης (initiated), a mystery or secret doctrine"
I think secret better communicates. A mystery may never be known, but a secret can be known once revealed. Romans 16:25 says that it was kept secret since the world began and wasn't known until it was revealed by Paul (there are other verses that say so also, which I'll get to).
The phrase, "my (Paul) Gospel" is interesting. Usually we think of Matthew, Mark, Luke, and John's gospel, but here we see that Paul had a gospel, or some good news. Paul's gospel was very different than the other four. For one thing, the four gospels all deal with Israel. Jesus said as much a few times. In Matthew 15 there is an account of a Canaanite woman who wanted Jesus to cast devil spirits out of her daughter. Jesus said, "I am not sent but unto the lost sheep of the house of Israel." In Matthew 10 Jesus told his apostles to go to the lost sheep of the tribe of Israel.
Paul described the state of Gentiles during the entire OT including the four gospels.
Eph 2:11-12,
11 Wherefore remember, that ye [being] in time past Gentiles in the flesh, who are called Uncircumcision by that which is called the Circumcision in the flesh made by hands;
12 That at that time ye were without Christ, being aliens from the commonwealth of Israel, and strangers from the covenants of promise, having no hope, and without God in the world:
That must of hurt! But things did get better:
Eph 3:3-6,
3 How that by revelation he made known unto me the mystery; (as I wrote afore in few words,
4 Whereby, when ye read, ye may understand my knowledge in the mystery of Christ)
5 Which in other ages was not made known unto the sons of men, as it is now revealed unto his holy apostles and prophets by the Spirit;
6 That the Gentiles should be fellowheirs, and of the same body, and partakers of his promise in Christ by the gospel:
Abraham did not know that the Gentiles should be fellowheirs, of the same body, and partakers of the promise. He knew they'd be blessed in Israel because they could ride on their coat tails so to speak, but that everyone in the whole world could be God's chosen people was not known. The devil had a hard enough time keeping a few people (Israel) from getting God's promises, but now he had to deal with the whole world.
Along those lines, there is another interesting verse about the devil and the secret.
1 Cor 2:7-8,
7 But we speak the wisdom of God in a mystery, [even] the hidden [wisdom], which God ordained before the world unto our glory:
8 Which none of the princes of this world knew: for had they known [it], they would not have crucified the Lord of glory.
Now I know their is some debate on who are the princes of the world are. I don't want to get into that unless you really want me to. For now, suffice it to say the there is little doubt the Jew of the first century would have understood it to mean a spiritual being, in this case, the devil. While the upshot of the mystery would have totally neutered the power of the devil, it wouldn't have meant much to earthly rulers. They probably would have crucified him anyway, whereas the devil most definitely would not have killed him. You'll see why I say that in a minute.
Col 1:25-27,
25 Whereof I am made a minister, according to the dispensation of God which is given to me for you, to fulfil the word of God;
26 [Even] the mystery which hath been hid from ages and from generations, but now is made manifest to his saints:
27 To whom God would make known what [is] the riches of the glory of this mystery among the Gentiles; which is Christ in you, the hope of glory:
Again, from verse 26, you can see that the mysery was kept secret until it was revealed to Paul. The "now" is when Paul was alive. Prior to the "now" of Paul's time the nobody knew the mystery. But reall kicker, and the reason the devil would not have killed Jesus had he know the mystery is in verse 27, namely that it is Christ in each and every born again believer. The devil had a hard enough time dealing with Jesus. But Jesus could only be in one place at a time doing one part of God's will. Now wherever their is a Christian, their is Jesus. In the Gospels Jesus said he'd be wherever there was two people. But that was before Pentecost. Now it only take one born again believer, filled with all the fullness of God (Eph 3:9) to destroy the works of the devil! Ha, take that Satan, you old bird!
Of course, if the devil could keep Christians from knowing the secret, it's minimize the effects and he'd have more power. That's why the mystery is so important for all Christians to know.