I suspect that if you keep at it you will see imporvement. Try incorprating more lifting motion in your fingers and less heavy pressure. Play with how little pressure you can use to pluck the pull offs. You might be surprised at how little pressure you need to pluck a clear note. It takes a quick release.

Also look at the fingers your are moving to with the pull offs. Those new notes need to be ready before you make your move. I found that I got hammer ons easier than pull offs since it took less prep and technique to make clear notes.