If you want to get a Neo Geo, you should probably get the MVS (arcade) model instead of the AES (home) model. The games are much cheaper and 200-in-1 type cartridges are readily available. If you have an MVS board, you’ll need a way to power it and a way to connect controls and the screen.
As the Neo Geo MVS uses the JAMMA arcade connector standard, there are many ready-made solutions that will give you standard SCART RGB output, some controller connectors and a way to connect a power supply (either ATX or special arcade ones). I got one from http://www.retroelectronik.com. You can reuse this for non Neo Geo arcade boards, too.
Here’s the system how it was when I got it.
I cleaned everything with some alcohol (didn’t put any in me) and fitted the Jamma adapter. After that it looked like this.
As you can see, you can just connect an ATX power supply to this and it will work fine. One or two moves ago, my big ATX power supply collection must have found its way to the trash, so instead I ordered an arcade power supply from china for 15€ or so. You just screw everything directly to the terminals. Try not to die! I used a switchable outlet so I can actually power it on or off. Measuring the voltage, it was slightly off, but you can adjust it. The -5V lane didn’t work at all, but the Neo Geo doesn’t use it, so no problem for me.
After connecting it to the PVM and attaching a Neo Geo controller it worked right away. Here’s a pic of Neo Turf Masters, started in Japanese mode.
King of Fighters 97 works well, too.
You can usually get a board for around 70€, the jamma adapter for 50€ and the power supply for 15€. Add a multi cart for 60€ and you can play hundreds of Neo Geo games on the original hardware for under 200€. Neo Geo AES consoles go for over ~350€ usually and the games are much more expensive. So if you want one, get the MVS.