“Would have” is a modal verb which has two common structures. The first is with 'but'. I would have A, but I had to B. This structure is used to show that you wanted to do something in the past, but you could not. So, here, in the given context, 'would have' will be used to make the sentence grammatically correct.