Suppose a5b7 occurs in the (r + 1)th term of the expansion (a−2b)12
We know that the (r + 1)th term in the expansion of (x+a)n is given by
Tr+1 =

n

‌

Crxn−rar
⇒ Tr+1 =

12

‌

Cr(a)12−r(−2b)r =

12

‌

Cra12−rbr(−2)r ... (i)
On comparing power of a as well as b in a5b7 and Tr+1 , we get
12 – r = 5 ⇒ 12 – 5 = r ⇒ r = 7
Putting r = 7 in (i), we obtain the coefficient of a5b7 in the expansion of
(a−2b)12 =

12

‌

C7(−2)7
=

12!

7!5!

× (- 128) =

12×11×10×9×8

5×4×3×2×1

× (- 128) = - 101376