The Commission on Unclaimed Assets, which proposed in July that the funds should be channelled into a social investment bank, is opposed to such a move, fearing it will not receive the minimum £200m it needs to get started.
Stephen Bubb, chief executive of Acevo, said: "We are concerned that the social investment bank might not happen if unclaimed assets are diverted into the BLF to cover up the money taken to pay for the Olympics.
"The ideal solution would be that the unclaimed assets are given to the social investment bank and part of that is used by the BLF for youth and community projects."
The Treasury denied any link between Olympic funding and the proposed use of unclaimed assets. The BLF refused to comment.