Prison opponents vow to keep fighting despite Labor's election defear