I was also having a big problem with this initially. At the time I was trying to stay low on the silks because I was just learning the sequence. Later I found that once I began climbing higher and inverting closer to the rigging point, the pole stayed closer to me and stopped swiveling away. I also found that putting more tension on the tails before the first wrap around my free leg helped to anchor the pole in place.
