Tens of millions pounds from the fund meant to help revive high streets in deprived towns is instead being pumped into the Kent port to tackle Brexit-fuelled delays.