The threat was received last week, and local and federal officers started investigating.
"We received some threatening communication that was a little different than the usual stuff that we get from people that don't like us," Director of the center Will Batts said.
Police said someone left a handwritten letter with a swastika on the envelope, anti-homosexual slurs and threats of violence if the center "keeps pushing your sordid agenda."
Batts said he was used to getting angry calls and emails, but nothing this extreme.
"It's a little unsettling, I think, for people," he admitted.
The FBI was also investigating.
Batts said they have security measures in place at the center, but they may add more.
"We're reviewing all of our policies and procedures and our security measures that we have now," he said. "With those things that we think we need enhancement or improvements or additions to, we are taking those steps."
Batts wanted members to feel safe at the center, but he also wanted to send a message that threats, even threats of violence, won't stop them from serving the Memphis LGBT community.
"We always talk about this being a safe place and a lot of times, that's been more psychological safety. But we need it to be physically safe for our community too," Batts said.