I use a dowel rod and have not had a problem poking holes in fabric (after literally hundreds of puppets). Perhaps you are pushing to hard?
I use the paper towel tube method, also use a small tube for fingers.