What do you do if you have one stop way farther than the main grouping of your stops?

It wasn’t so bad that I thought it was an error and the staff would take it off for me, but it ended up taking 30 minutes to drive out from the main area of my deliveries to this one stop out on the country and back. Could I have left it in the return bin before heading out and taken the “package not delivered” hit? My standing is “fantastic” currently. Is there any other way to handle outlier stops the system really shouldn’t assign to a block?