Hi there!
Maybe it's just me, but I keep seeing someone's avatar, check their profile and I realize that I've checked them already at least two times (that very day), and then maybe I already either followed them (because I thought they are great folks) or decided we wouldn't click (but they are still great folks, so I don't want to ignore them).
Would that be a huge problem to add an overlay tags to avatars, seen only by a particular user; say I would mark someone "interesting", "not a match" (or "followed" created by following someone) and then the next time I would see them in a list of users I wouldn't scare them with yet another visit to their profile?