The restaurant’s name is derived from the nearby Fanny Goldmine Hill. According to The Alabama Review in January 1957, "This section of land belonged to the widow of an American Indian, and although the name implies that it was a mining property, it was not. The woman was told that to register her land in the proper manner, she would have to have an English name. From the settlers she had heard the name ‘Fanny,’ and having seen and liked the gold and mines at Blue Hill, she proceeded to register her land under the name of 'Fanny Goldmine.'"