A limit on people, but not PACs?

Those who favor limiting First Amendment rights often denounce PACs, which they call tools of special interests. If the aggregate limit is so important, why does it only apply to people and not to PACs? PACs can give to as many candidates as they want, and they can give a lot more to each candidate, up to $10,000 if the PAC donates for both the primary and general election campaigns.