If the square shaft goes deeply enough into the cap it should not be loose. Tightening it up with some epoxy is OK, just do something so you do not glue the cap onto the shaft so you can disassemble things without breaking them in the future if necessary. I don't like using tape on the square shaft , it is kind of squishy, especially if it gets hot and gets messy. Turning the inner core by hand is not a great test of how tight or loose it is. When it is spinning fast it can get harder to spin. It can't hurt to lubricate the cable. I had a car that was eating angle drives and dust caps. It turned out to be some parts came loose in the speedometer head and were jamming it making it hard to turn.
David Teitelbaum