A Special Permit is required generally if your structure is non-conforming and you wish to make an alteration or extension to it. The Board of Appeals approval or special exceptions shall not be granted unless the applicant demonstrates that "no undue nuisance, hazard, or congestion will be created and that there will be no substantial harm to the established or future character of the neighborhood nor of the town".
A variance is required if you want to do something with your property which is generally prohibited by the Zoning Bylaw. The applicant must show a hardship imposed by the Bylaw which is caused by a unique condition of the lot, or structure, and the hardship is owing to circumstances relating to the soil conditions, shape or topography of the land or structure and especially affecting the land or structures, but not affecting generally the zoning district in which it is located. Relief may be granted without substantial detriment to the public good, and without nullifying or substantially derogating from the intent or purpose of such ordinance or bylaw. The criteria for a Variance is very strict, you might want to look at all of your options before applying.