A seafood organisation has warned the Brexit red tape seen earlier this year could be the demise of small fishing businesses.