Silicone grease/lube has its purposes and works well for them. But its lubrication properties do not come close to a good grease in high wear areas.
For stationary o-rings silicon is fine and appropriate. For high wear areas, bolt actions type areas, a better alternative is recommended.
All lubes/grease attract contamination, a quality grease can suspend/encapsulate it better to help avoid wear. Key here is to not over do it, a very very little good grease goes a long way. Just use the thinnest of coatings, sparsely. You can even lightly wipe it off and enough will still be there to do the job.
jmo