Well, keeping in mind that all handle materials can be damaged, Almite does tend to scratch more easily than stainless steel. On top of that, because it is a colored coating, scratches and small dings cannot be buffed out of Almite the way they can on stainless, most natural materials, and most plastics. So the only realistic choices are to either avoid damage, or accept it and consider it adding character to the knife.
We do tend to give our knives a tough set of expectations to live up to. On the one hand we want to be able to use them as tools, and on the other, we want them to look like jewelry. I try my best to avoid the problem. While I've carried darn near every Spyderco I own, I always carry any knife in a pocket by itself, or at worst, in the same pocket with my wallet. Either way, this eliminates the chance of the knife getting scratched up by other objects, such as keys, lighter, roll-box, or change. My lighters fare much worse. I also carry most of my knives loose in my pocket rather than clipped and the ones I do carry clipped are clipped to the inside breast pocket on a sports coat. That way the clip cannot snag on something and either be damaged, cause damage, or cause the knife to come out and land on the ground and get dinged up, or even worse, fall unnoticed and be lost. Almite, however, does present a special problem even with that kind of care. Eventually the coloring will be worn off the "corners" of the handle just from the minute amounts of abrasive material bound to be embedded in the fabric of the pocket.
All that being said, I'm sure there are folks who use a Toad for EDC and probably considerably more who carry an Almite Navigator or other Almite handled knife.