Can a person have aphasia without a physical impairment?

Yes. Some affected individuals may only have aphasia, but most people with aphasia also have various physical disabilities, including, but not limited to, limb weakness, hemiplegia (paralysis of the right leg and right arm), loss of sensation (from the right leg and right arm) and visual impairment.