If you don’t want to pay the permit fee to stay in Lower Glenelg National Park, pitch your tent at Dry Creek. Just before you cross the SA border, there is a road that heads north called Border Road. This road will wind into South Australia and the last stop before the road winds back into Victoria is Dry Creek.
This rest area only has a few spots available, and while it might not be completely isolated or private, you have direct access to the Glenelg River, flushing toilets, running water and a shelter with a fireplace.
We did a stint of fishing, and while the campers next to us caught a 30cm snapper, we weren’t so lucky due to our shitty bait and lack of experience. Dinner was leftover beef mince stew in a wrap with some cheese and a bottle of Pepperjack Shiraz by the sheltered fireplace, and we packed up at sundown so we could have an early start in the morning.