How are you situated for gear? You'll need a helmet and goggles, and may want to buy some dedicated gloves, pants, jacket, and such. I still ride in a ski jacket though, so depending on your situation, don't be afraid to try stuff you have and see how it goes. If you want to go off-trail much, you'll definitely need a shovel, and a beacon and probes are highly recommended. If you're not too picky on buying used stuff and already have some winter gear, you won't need to spend a ton of money, but I'd suggest setting around $500 aside. Then there's the question of transporting the sled, but if you have a full-size truck, you can get by with just a ramp; if you're going to tow it, basic snowmobile trailers can be found for <$500. It's not uncommon to find a sled and trailer together, sometimes people get out of riding and want to unload the whole setup.
As for the sled itself, there are some decent options for <$2000. It's kind of luck of the draw, but spring is a good time to buy. My biggest piece of advice is don't get caught up in just buying the newest sled your money will bring. Assuming you're thinking of mountain sleds, you can probably do better than the first generation or two (late 90's into early 00s). They can be modded to do quite a bit, but as delivered, the handling and ergonomics make them difficult to ride on anything more than gentle terrain. In that price range, I'd say the furthest forward you could reach is a late-00s 600 RMK (iQ chassis); specifically, the carbureted version. You may find several iQ RMKs for your price range (including Dragons and 900s), but most of those motors are a real roll of the dice. Lots of potential for something that looks good to turn bad there. To my mind, the best sled that's also likely to be a safe buy in that range would be a 02-05 600 or 700. Those motors are great when taken care of (still, at least do a compression check), they usually have enough track to get around, and they're a big step up from the earlier RMKs, though eclipsed by the newer stuff. Just don't get in a hurry: watch CL and such and try to get a sense of what people are asking for what, and if you walk away from a few sleds before you buy, you're probably doing it right.