The fragmentation of Europe's capital market due to Brexit has increased operating costs for banks to the point where some struggle to make their UK and EU hubs viable, a report showed on Tuesday.