Ok the kinder gentler me.
I have the same wildcat. The Gen 1.
If you cock the gun and ease a pellet into the chamber, you will feel that pellet passing into the breech seal. After many shots this wears the seal down. Typically you will get anywhere from 500 shots to 2000 shots and anywhere in between. You just never know when a breech seal will be compromised. I usually remove the barrel and change mine about every 1500 if they make it that long. At 3 thousand shots, if the seal makes it that long, you should just go ahead and change it anyway. It's time.
Any damage to the seal at all , such as small nicks from pellets rubbing against it will cause erratic changes in poi just exactly as the OP describes.
It is also not unusual to look into the chamber and see no breech seal at all. That is because it wore down, broke and blew out of the barrel on one of your shots. The Wildcat is not the only gun that does this. It's more common than one might think.
Changing a breech seal in the Wildcat is pretty easy. Remove the barrel, get yourself a number 5 worm hook, typically used for bass fishing. Bend the hook straight, then put a crook at the point and stick it in there and pull the seal out. Replacing with a new seal is a little harder. It has to seat in the grove. I use what is known as a pecan pick. It has a curve that allows me to seat the seal perfectly in the grove without damage.
Reinstall the barrel: Put it back in, find your index mark, and screw the grub screw precisely down on that mark. If the barrel is set back too far, you won't be able to get your magazine in, or it will go in, but will not cycle, Do not force it to cycle if it seems harder than usual. Loosen the grub screw and slightly pull the barrel out until it is flush with the breech.