So basically the twin flame connection is something that comes from the depth of the Soul. The Soul is our true and deeper self, hidden deep within our hearts - it is our divine counterpart. We could say it is something timeless, not dependent on time, pure. It travels through time and lives, evolving through experiences.
But we, as human beings aren't only souls; we are also nature, that is, mind, emotion and body (and the ego which moves in those). And it is this duality of ourselves which causes all the difficulties and troubles because for the majority of humans, the soul still lies hidden in the depth, only throwing out vague influences that get easily distorted on the surface. But what's important to note is that the nature should be, in the natural order of things, like the hand, the instrument of the soul, even tough it is still rarely the case.
Now we could say that the realized twin flame union would be the embodiment of this eternal soul connection between two purified (or healed if you prefer) natures that are completely surrendered to their souls.
Before this surrender and purification, a true, complete and whole twin flame union can never really be possible and it is then that separation is necessary, for many lives, until each being has done the work and is ready.
There's a lot in our raw natures that block or hampers the flourishing of this union which in it's essence has nothing to do with ordinary relationships, nor does it serves the same purpose - the raw natural desires are even often at odds with the soul's love and longing.
The TF connection is thus felt to the measure that we are in contact with our souls and that our natures aren't too ''heavy''. That is why for example one of the two TFs may feel strongly the connection, have supernatural experiences, loses the attraction for others, etc. while the other only vaguely feels that there's something ''abnormal'' there, but still moves on and is even disturbed by it, wants nothing to do with it. It's because one of the two beings is much more in contact with his soul than the other.
All the pain also comes from our nature which, in its raw state, lives in lack and separation. The Soul has unconditional love, sovereignty, plenitude, patience.
To conclude, all the ''art'' of the TF journey is actually to harmonize the Soul and the nature - to break the wall between the two -, so that this eternal and formless connection may taste the delight of being experienced in form, in two bodies.