At only one point during my four days on Catalina did the whole thing seem like a bad idea. At the time I was technically just offshore of the California island, scuba diving through the famed kelp forests that ring its shores. The tall stalks stand like underwater redwoods, the leaves undulating in the current like big flat noodles with mesmerizing calm. But get a little tangled in them and I can assure you, they basically morph into sentient tentacles.

Terrifying, yes. But it only lasted a moment, and my dive partner quickly freed me from the kelpy embrace. We slowly surfaced near our dive boat, and I appreciated that Catalina may be Los Angeles's own offshore vacation playground, but the nature here doesn't mess around. My second thought was that I was gonna need a lot of ice cream to recover.

Ice cream Catalina has in spades, in at least four or five shops spread on a waterfront downtown that only stretches a few blocks. It tracks, given that much of the island's current state is thanks to chewing gum. The 76-square-mile Catalina was already partially settled in the early twentieth century when William Wrigley Jr., he of the gum fortune, purchased a controlling interest in the company that owned all the real estate. Much of what came after was all Wrigley; he even used it as the spring training site for his Chicago Cubs for 30 years.

Today, Avalon—that town of numerous confectionaries—remains the only major settlement on Catalina. Passenger ferries arrive every day from docks near Los Angeles and San Diego, and cruise ships dock just offshore. People say it feels like Europe, the dense, small town cascading down toward a marina crammed with yachts.

But in Avalon there are even more fish-fry restaurants than there are scoop shops, and twice as many T-shirt stands as all those put together. For four days we were shoulder to shoulder with fellow tourists in Avalon's slightly shrunken downtown; with few cars on the island, golf carts are the main form of transportation.

Avalon may have been named for King Arthur's mythical island, but it turns out that the town is the least exciting part of Catalina Island. Not to knock walking the beachfront with an ice cream cone—I partook daily—but the cruise crowds could be overwhelming. Restaurants were fine, but the views beat the cuisine every time. The island really blows you away in its outer reaches.

There's those kelp forests, which feel like hiking through the Olympic rainforest, only orange and underwater. They're home to giant sea bass, who earn their name by growing over seven feet long. Calm but big as a manatee, they're a shocking sight on what's a fairly shallow dive, and another reminder of just how wild California can be.

Some of the best diving and snorkeling is right under the Catalina Casino, which is technically in Avalon but on its farthest reaches. The round historic building was never a gambling hall; it's a community center, movie theater, and old-fashioned ballroom, a piece of stunning architecture that wouldn't seem out of place in Italy or the French Riviera (another reason the place gives international vibes).

But it's the truly distant parts of Catalina that feel worth the trek—that trek being a flight to Los Angeles, an Uber to the ferry, and the boat ride into the Pacific. Catalina Island Conservancy, a land trust that oversees most of the island's land, almost entirely undeveloped, runs tours in open-sided all-terrain vehicles into the miles of rolling hills.

Eco guides know to find the island's herd of bison for visitors, even if they're a strange kind of "eco" sighting; the 150 or so individuals are in no way native, having been brought to Catalina as a backdrop for a movie in 1924. According to our guide, the bison immediately wandered out of frame and started repopulating.

There's gruesome history too; one viewpoint on the island's western side, looking out at kelpy shore and the uninterrupted Pacific, also happens to point at the cove where actress Natalie Wood died in 1981. Hollywood's history feels woven into every part of Catalina.

(On the goofier side of that coin, there's the Catalina Wine Mixer. If you saw the 2008 Will Ferrell movie Step Brothers you know it, and if you don't you'll be puzzled as to why it's referenced everywhere in Avalon—by T-shirts and wall signs and tour guides. The movie made it up, but of course now it's a real event that takes place in spring.)

Back in Avalon and itching to escape the crowds, we borrowed bicycles and pedaled down one of the island's few paved roads to an industrial stretch of waterfront, hidden from downtown. Among the island's water treatment facilities and dock for the supplies barges, we settled into lunch at the Buffalo Nickel.

The place only took cash and our meal was interrupted a few times by the landings at the helipad outside. But our ribs fell off the bone and we enjoyed shade and space for a long stretch of the afternoon. I'll never knock eating an ice cream cone in a sunny beach town, but barbecue in Catalina's quiet corner beats even that.