I said it will slow down your program overall, because there is overhead in checking every object’s count every time you access it.
The only potential downside to GC is the notorious “pause” for collection. However, this can be mitigated in many ways. Other garbage-collected languages such as Java, C#, and Go have no problem with this; they are very efficient.
If ARC was so superior to GC, it would be much more common in other programming languages.