The $ symbols are "locks". Remove the "locks" on the part of the reference you want to adjust itself when copied.

$A$1 = locked column, locked row
$A1 = locked column, adjustable row
A$1 = adjustable column, locked row
A1 = adjustable column, adjustable row