i am sorry to hear so. there are several potential solutions to this, i will try to list a few. mix and match as you feel.
1 - if you are in Europe or US, and can document him doing this to you (for instance, email feedback from potential employer suggesting your former managers recommendations etc.) you should immediately report to the employment tribunal or a similar body.
2 - the potential new employer has no right to demand specific references, you should be able to decide, so pick 2 or 3 people that are more reliable (for instance another manager, a peer, a client, a former professor etc)
3 - you are also entitled to receive a letter of service / reference from HR, which would detail basic information as hire date, tenure, assignments etc. request it so that you don't have to rely on a former manager to confirm employment.
4 - speak to a lawyer, see if a potential harassment case can be filed or at least a letter to the manager with copy to HR can be sent.
all the very best mate.