At the tip of Michigan’s Thumb, near Port Austin, a low rock ledge now called “graffiti rock” juts out into Lake Huron. Based on postcards and historic photos, that formation once looked much like today’s Turnip Rock farther up the shore, only larger and split into two towering pillars. It must have been a sight.

Old newspaper stories from Michigan’s Upper Thumb tell of a recluse named Hugh Loosemore, who owned land on what locals once called Loosemore Point. He kept to himself, collecting driftwood from wrecked ships along the shore and stacking it into a strange structure that people began calling “Loosemore’s Castle.” The story has been told in local histories, poems and barroom talk for nearly a century. Yet hard evidence for some of its most colorful details remains thin.

The Legend of Loosemore Castle

Local accounts say Loosemore’s driftwood structure grew until it looked like a rough, multi-story building. Some compared it to a small hotel and claimed it even had a “ballroom.” Over time, the story expanded. The same shoreline became the setting for a tale about a pot of gold, supposedly buried there long before Loosemore arrived.

Hugh Loosemore kept to himself. So the locals attributed all kinds of thoughts of what his story was. A fugitive? a pirate? To add to the mystery, it was widely thought that Loosemore’s property was a burial site of ancient Indian Inca gold. The legend of his creation along with a Michigan Indian tale of gold highlighted in a 1932 book with a poem called Loosemore’s Castle.

Sadly, local lore indicates Hugh Loosemore was found frozen to death in his large driftwood castle. However, we are unable to find any official records of Hugh Loosemore, despite being widely mentioned in antidotal histories by published researchers.

At Loosemore’s Point, we are told,
Indians buried a pot of gold,
Pure gold by the Incas mined.
For some lucky person to find.
(Loosemore’s Point, End of the Rainbow)

A Lonely End to Loosemore Rock

Today, reaching Hat Point is a short walk along the Lake Huron shore east of Oak Beach. The sand narrows as you approach a blocky ledge of layered rock that sticks out from the beach. On calm days, waves slap gently at its base. During storms, they slam into it.

The four historic images of the site help tell a simple story of erosion.

An early 20th-century postcard titled “Rock Formation near Oak Beach, Huron County, Michigan” shows a tall arch carved in bedrock. People stand under the opening, dwarfed by the overhead span. Vegetation hangs over the rim.

Early 1900s postcard showing tall rock arch near Oak Beach at Loosemore Pointe in Huron County, Michigan, with people standing under the opening. — Rock arch at Loosemore Pointe near Oak Beach, Huron County, in the early 1900s. A natural opening spans the base of the bluff while pine trees still grow on top.

By the mid-1900s, photos label the same feature “Loosemore Pointe, Port Austin, Mich.” but the arch has failed and the rock mass is starting to slump toward the lake.

Loosemore Point inthe 1930s - Black-and-white postcard view of Loosemore Pointe rock formation near Port Austin, Michigan, showing collapsed arch, beach fencing and a Keep Off sign. — Mid-20th-century postcard labeled “Loosemore Pointe, Port Austin, Mich.” The arch has failed, leaving a shortened rock block with fencing and a “Keep Off” sign on the beach.

A 1959 view shows a lower, more rounded block with less overhang.

Loosemore Point Black and White Shot — Loosemore Point around 1959. The formation is lower, with more of the upper bluff gone, and the shoreline already narrowed by erosion.| Huron County Centennial

By 2019, only a squat slab remains, surrounded by water and covered in spray-painted names. The arch is gone, most of the upper bluff has collapsed and the feature is now often called “Graffiti Rock” or “The Big Rock.”

Loosemore/ Hat Point in 2019 — The same rock formation in 2019, often called “Graffiti Rock” or “The Big Rock,” now reduced to a flat slab in the water and covered with spray-painted names.

Geologists note that this stretch of shoreline sits in soft Marshall Formation sandstone, which is easily undercut by Great Lakes wave action, ice shove and changing lake levels. That process creates arches, caves and stacks like Turnip Rock and other formations along Pointe Aux Barques, then eventually breaks them down.

Doubtful of the Loosemore Legend?

We have not found a historical map that labels “Loosemore Point” anywhere in Huron County. The name appears in the Huron Times but without a precise location, which makes it hard to verify.

Historian Florence McKinnon Gwinn places the point “three and a half miles west of Port Crescent,” which matches the general area now known as Hat Point. The Huron County Centennial volume also repeats the story of a buried pot of gold. Taken together, those clues suggest the modern Hat Point area is where residents once used the name Loosemore Point.

We are thinking that it’s today called Hat Point. Furthermore, it’s interesting to note that Loosemore Road offers a southern border of sorts for the area near Hat Point.

Hat-Point - Michigan Indian gold — Area Near Hat Point from Google Maps

But Hey, We Like a Good Legend

Despite the doubts, we think there is a hint of truth to the saga. So, the next time you stop at Walt’s Meats, or Kreb’s Cottages or at Duggan’s Family Campground you may be only steps away from the site of a legendary driftwood castle and a fortune of buried Inca gold.

We took a walk to the legionary spot. It’s quite unique. There is a limestone rock outcropping that stops at the water’s edge. (Which is quite high this year). There are overhangs that create a bit of a shelter. We didn’t go any further as we didn’t want to get wet. Here is a bit of our adventure.

Until a land record or contemporary account turns up, Loosemore’s Castle sits somewhere between fact and folklore, perched on a piece of shoreline where history and imagination still blur.

The Legend Remains at Hat Point

Based on the surviving references, Loosemore Point was likely a local name for the stretch of shore now called Hat Point. A recluse may have built an oversized driftwood structure there, but there is no firm record of Hugh Loosemore himself, or of any buried Inca-mined gold. What is clear is that the story has lasted almost a century in Huron County histories and poems. That endurance, more than any pot of coins, is what keeps people walking that shoreline and wondering what really stood there.

If you grew up hearing stories about Loosemore’s Castle or have old family photos from Hat Point, share them in the comments below. We may feature them in a future update.