I'm a young single parent of a soon to be 8 year old, so I'm going to regail my experience onto you:
I started dating someone way too soon after my divorce when my daughter was only 2 1/2. He turned out to be horrible to me in every sense of the word, but was really great to my daughter (but it got a little rocky at the end). I put up with him for 4 hellish years before I finally snapped out of it and stopped making excuses for him and his trepidation of committment (and I'm glad I did so because I would have been a fool to keep him)! I told him it was over in December of 2010 and never looked back. It pained me for a while as my daughter had grown a great deal during our relationship together, but I knew deep down he wasn't right for our family. By the sound of it, you seem to be just as concerned for your daughters interest above your own (as you should be), and worry about your boyfriend's involvement. With the right guy, there won't be any confusion. He will make it absolutely clear and certain he is in it for the long haul. His 'you're not my dad' reasoning sounds like a copout that is the beginning of a slow exit. Dating single parents is no joke, and those who have never dated one and try it out, bolt more often than not from being overwhelmed with the responsibility.
Continue concerning yourself with your own wants, and your daughter. If he's scared, then let him go figure himself out. Don't push, beg, or plead for his attention. Don't try and fix it if he's making it loud and clear he doesn't want fixing. Should you break up with him? If your gut is telling you so, then yes. If he has you more upset/sad/always wondering if things will work out than being happy, then yes, break up. Life is too short to waste any time on guys with 'possible' potential. Good luck :]
ANSWER THIS QUESTION