Lots of ways to look at this.
You can get by with a cheap PCP, as mentioned, less than $300 and a simple Ebay hand pump for $40.
I've owned Hill, FX, Logun pumps and will say that I've had less trouble out of my $40 Ebay pump...actually NO trouble in 2 years, than I did on the premium models. If/when it breaks, I'll simply buy another. Granted, for right now, it's a backup as I have a tank and compressor.
So under $350 will get your feet wet.
BUT...
Also as mentioned, you will soon tire of pumping. Next step up is to get a SCUBA tank and if you have a dive shop, can get it filled there. Next step from that, would be a small compressor... and finally a large compressor.
So like with much else in life, you pay for convenience. Do you need a $2000 compressor, $500 tank, etc... nope. But if you start getting into PCP and especially into the big bores, you'll sure wish you had them.
But to try it out... $350 will do it.