It happens when they receive Christ. Of course they are forgiven then. And received into fellowship. But not until then.
Fellowship is not really something we DO so much, but rather it's more something that born again believers HAVE with one another....if we have Christ in us by His Spirit, because His Spirit is the source of fellowship. We are told not to have fellowship with unbelievers. That would be a false fellowship and lead to compromise and dilution and defiling of the gospel. Participation in church life per se is not for the world, it is for believers. And even among believers there are circumstances when we are not to fellowship with someone.
2Co 6:14-18
Be ye not unequally yoked together with unbelievers: for what fellowship hath righteousness with unrighteousness? and what communion hath light with darkness?
And what concord hath Christ with Belial? or what part hath he that believeth with an infidel?
And what agreement hath the temple of God with idols? for ye are the temple of the living God; as God hath said, I will dwell in them, and walk in them; and I will be their God, and they shall be my people.
Wherefore come out from among them, and be ye separate, saith the Lord, and touch not the unclean thing; and I will receive you,
And will be a Father unto you, and ye shall be my sons and daughters, saith the Lord Almighty.
2Jo 1:10-11
If there come any unto you, and bring not this doctrine, receive him not into your house, neither bid him God speed:
For he that biddeth him God speed is partaker of his evil deeds.
1Co 5:1-2
It is reported commonly that there is fornication among you, and such fornication as is not so much as named among the Gentiles, that one should have his father's wife.
And ye are puffed up, and have not rather mourned, that he that hath done this deed might be taken away from among you.
2Ti 3:5-8
Having a form of godliness, but denying the power thereof: from such turn away.
For of this sort are they which creep into houses, and lead captive silly women laden with sins, led away with divers lusts,
Ever learning, and never able to come to the knowledge of the truth.
Now as Jannes and Jambres withstood Moses, so do these also resist the truth: men of corrupt minds, reprobate concerning the faith.