Some inquisitive users on the internet, specifically a developer on the Gearbox forums helped to find a Hidden Basement in New Haven. There are some specific requirements though:
- You need to have One Eyed Jack's eye.
- You need to have the quest to kill the Rakk Hive - The mission tag is Another piece of the puzzle
- The door leading to the basement is in the blue shack directly in front of Erik Franks.
- Do NOT turn in the Eye or kill the Rakk Hive. Doing either of these will close the basement.
The basement holds two red chests, one containing the special sniper rifle Rider.
 Video Guide
Neoseeker Member Shadow of Death on his Youtube Account KageNoAku posted a high quality video with notes, available in Youtube HD (720p):