Let a rectangle ABCD be inscribed in a circle with radius r.

In ∠DBC = θ
In right ΔBCD ;

BC

BD

= cos θ
⇒ BC = BD cos θ = 2r cos θ

CD

BD

= sin θ
⇒ CD = BF sin θ = 2r sin θ
Let A be the area of rectangle ABCD.
∴ A = BC × CD
⇒ A = 2r cos θ 2r sin θ = 4r2 sin θ cos θ
⇒ A = 2r2 sin 2θ , sin 2θ = 2 sin θ cos θ
∴

dA

dθ

= 2 . 2r2 cos 2θ = 4r2 cos 2θ
Now ,

dA

dθ

= 0
⇒ 4r2 cos 2θ = 0 ⇒ cos 2θ = 0
⇒ cos 2θ = cos

π

2

⇒ θ =

π

4

d2A

dθ2

= - 2 . 4r2 sin 2θ = - 8r2 sin 2θ
∴ (

d2A

dθ2

)(θ=

π

4

) = −8r2sin(2,

π

4

) = - 8r2 . 1 = −8r2 < 0
Therefore, by the second derivative test, θ =

π

4

is the point of local maxima of A.
So, the area of rectangle ABCD is the maximum at θ =

π

4

Now, θ =

π

4

⇒

CD

BC

= tan

π

4

⇒

CD

BC

= 1 ⇒ CD = BC
⇒ Rec tangle ABCD is a square
Hence, the rectangle of the maximum area that can be inscribed in a circle is a square.
OR
Let a cylinder be inscribed in a cone of radius R and height h.
Let the radius of the cylinder be r and its height be h1 .

It can be easily seen that Δ AGI and Δ ABD are similar.
∴

AI

AD

=

GI

BD

⇒

h−h1

h

=

r

R

⇒ r =

R

h

(h - h1)
Volume (V) of the cylinder = πr2h1
⇒ V = π

R2

h2

h−h12h1
⇒ V = π

R2

h2

h2+h12−2hh1h1
⇒

dV

dh1

=
⇒

dV

dh1

= π

R2

h2

h2+3h12−4hh1
Now,

dV

dh1

= 0
⇒

πR2

h2

h2+3h12−4hh1 = 0
⇒ 3h12−4hh1+h2 = 0
⇒ 3h12−3hh1−hh1+h2 = 0
⇒ 3h1h1−h−hh1−h = 0
⇒ (h1−h)(3h_1-h) = 0
⇒ h1 = h , h1 =

h

3

It can be noted that if h1 = h, then the cylinder cannot be inscribed in the cone.
∴ h1 =

h

3

Now,

d2V

dh12

=

πR2

h2

(0 + 6h1 - 4h) =

πR2

h2

(6h1−4h)
∴

d2V

dh12(h1= h 3 )

=

πR2

h2

[

6h

3

−4h] =

−2πR2

h

< 0
Therefore, by the second derivative test, h1 =

h

3

is the point of local maxima of V.
So, the volume of the cylinder is the maximum when h1 =

h

3

Hence, the height of the cylinder of the maximum volume that can be inscribed in a cone of height h is

1

3

h