Having more than one husband or partner is destined in your horoscope charts, but in matters of children you will be fine. His affection or are with your child is more of the childs destiny to get love versus your ability to be with the right partner.
I would advise you to be with him for atleast next year second half before you take a call on whether you should be with him or not. February 2023 would bring some important changes in your life. Wait untill then and then decide way forward in matters of your relationship.
Absence of dedicated partner in your life is an important theme, but your child won't feel the absence. Hence, focus on your realtionship with partner rather than to think about him in context with your child.
Hope this clarifies your query.
Rahu - Ketu axis in 1-7 dicatates an absent partner, Mercury in house 7 gives multiplicity in relationship matters. Sade sati phase in last charan after Feb 2023 and hence clarity will be obtained in matters of house 2 which is family.
Jup-Ven Dasha is very sound in matters of child and issues connected to that.