Simon Wilkes is keen to continue wild swimming but needs a dedicated chaperone to assist him.
The local council is looking to cover inflationary costs.
Sheringham swim organisers say they are "devastated" but there is better news in Aldeburgh.
Video shows one of the few Christmas Day swims to go ahead, as more Boxing Day dips are called off.
The festive swims are called off because of worries about high tides and strong winds.
The sea crashes on to the rocks and pushes up the concrete slope, as Storm Benjamin hits Norfolk.
A campaigner says young inquisitive seals end up entangled in plastic flying rings.
MP Steffan Aquarone says the charges are "alarming" and that people need "clarity and reassurance".
A parish council says there is "considerable indignation" about the battle over the fishing site.
