After Harold Holt waded into the surf and vanished forever in 1967, the jaw-dropping theory that he rendezvoused with a Chinese submarine and was whisked away to a new life in Beijing convinced a few and amused many.

But not the late prime minister’s grandson, Robert Holt, who would love to see the far-fetched conspiracy theory put to rest once and for all.

The story is “madness”, he said on the 50th anniversary of the Liberal PM’s final swim in the sea he loved.

As Robert Holt notes, his grandfather didn’t even like Chinese food.

Harold Holt left his Portsea holiday home on Victoria’s Mornington Peninsula for a swim at turbulent Cheviot Beach on the hot and humid afternoon of December 17, 1967.

His mistress, Marjorie Gillespie, who accompanied him to the rugged beach, watched him stride into the waves and swim out through the breakers crashing on a rocky reef.

He was there one minute and then, after his bobbing head was obscured by an incoming wave, he was gone.

There is no doubt Holt drowned, but the 59-year-old’s body was never recovered and conspiracy theories swirled, including suicide and the wild allegations of spying for Chairman Mao.

Harold Holt’s great-granddaughter Zara Holt and son Sam Holt lay a wreath during his memorial ceremony at Point Nepean in Victoria. Photo: AAP

But as grandson Robert points out, even if the espionage theory boasted a shred of credibity Cheviot Beach is too shallow and the currents far too treacherous for such a vessel to get close to shore.

And, on top of that, the notion that Holt was brought in from the cold by Chinese spymasters as his reward for loyal service runs into a huge stumbling block: Holt was no fan of Chinese cuisine.

“Every year there are people who drown in Australian waters and we never find the body,” Robert told Channel Nine, speaking on the windswept beach where his grandfather was last seen.

“If you look at the beach here and how shallow it is, the idea that a submarine could have got in here is absolute madness.

“My grandmother said he didn’t even like Chinese food. So it’s unlikely [he was] taken away by a Chinese submarine.”

Robert hopes his grandfather, the successor and political heir to Sir Robert Menzies, Australia’s longest-serving leader, would be remembered for his achievements rather than the way he vanished.

Holt oversaw the dismantling of the White Australia policy and drove the historic referendum to ensure Aboriginal people were included in the census.

“He was very progressive in his thinking,” Robert said.

“So we are very keen that he is remembered for his achievements, not just the last few minutes of his passing.”

A memorial service for Holt at the former Army barracks near Point Nepean on Sunday was organised to mark the half-century anniversary of his disappearance, with guests including Health Minister Greg Hunt and federal Speaker Tony Smith, each representing nearby electorates.

Malcolm Turnbull earlier this month led tributes in Canberra’s lower house, calling Holt’s disappearance one of the most “confounding” events in the country’s history.

“He ushered in many of the reforms that we now consider so crucial, such watersheds in our evolution to the modern nation that we are today,” Mr Turnbull said.

Holt saw the change in 1966 from pounds to dollar currency, the introduction of child endowment paid directly to mothers for the first time, and the creation of the Reserve Bank of Australia.

He also ramped up Australia’s commitment to the Vietnam War and some conspiracy theories about Holt’s disappearance have been linked to his war leadership.

-with AAP