When I did the QM2 tandem eastbound crossing with our lady on her final transatlantic, my cab driver had no idea where the QM2 terminal was and had to ask for guidance twice
Sounds like you had a genuine lost cabbie, I have heard that a lot of the NY cabbies claim they don't know where the terminal is as they know they won't get a return fair from Redhook. Although I've not had any personal experience I've also been told that the shuttle bus service provided also leaves a lot to be desired.
jdl