All cavers should have a few basic items to safely and comfortably enjoy going underground.
Like with most hobbies, there are as many gear opinions as there are cavers. Try out the basics and see what you like before making niche/expensive purchases.
Some popular items are recommended below:
Going on your first-ever beginner trip to a horizontal cave? Pick these up at any sports store:
Helmet: Unlike bicycle helmets, caving/climbing helmets are built to take impacts (e.g. rockfall) from above. The most popular helmets are the Petzl Boreo and the Black Diamond Half Dome.
Headlamp: You need TWO light sources mounted on your helmet, with a third in your pack. Your phone doesn't count!
Cheap camping headlamps (e.g. Black Diamond) are fine for beginner trips/smaller caves.
Kneepads: Expect dry, maybe gravelly crawls in your first California cave. Generic volleyball or construction kneepads are fine. Avoid skater-style kneepads - hard plastic caps may get caught on rocks.
Elbow Pads (optional): Like with kneepads, basic gear is fine. Will protect your elbows, will also get snagged on rocks and make squeezes harder. Many cavers don't bother with elbow pads.
Gloves: Keep your hands clean and safe! Gardening gloves or rubber kitchen gloves are fine.
Clothes: Hopefully a given. Beginner caves are typically in the mid-50's, so regular pants and a shirt are plenty (you'll be exercising). Bring another top layer just in case.
Bear in mind that anything you wear caving will eventually be torn into dirty shreds.
Once you've done a few caves you may want to make some upgrades:
Headlamp: Want to actually see the other side of The Big Room? Check out Fenix, Nitecore, and Zebralight.
Look for L-shaped lights that use rechargable 18650 batteries for maximum compatibility with other cavers' systems.
Kneepads: Specifically made for caving, Dirty Dave's, PMI Crawlers, and OR1 Howitzers are more likely to stay in place while crawling.
Gloves: If you're doing any amount of ropework, get some cut-resistent or leather climbing gloves.
Bag: Carry critical supplies (like lunch) into the cave. The REI Flash 22 is lightweight, holds everything you need for a daytrip, and is cheap enough that you won't be sad when it eventually disintegrates.
Hardcore cavers frequently use either a Swaygo or Petzl Speleo.
Cave Suit: If you're reading this, you probably don't need a cave suit yet. You'll know when you need one, and by then should know a few cavers who can offer advice.
That said, they'll probably recommend the Adventure Vertical Titan Man, Landjoff Cordura Plus, or <some third thing>