When I was a child, my brother and I, along with some cousins, would fly down to Florida to visit my grandparents every year. My uncle and my mother were present, but only peripherally. There was a world we children created with our grandparents, one that started in the car ride from the airport when we’d sing all sorts of sleepy folk songs, my grandfather at the helm.

Then we’d toss off our shoes and run to the golf course just beyond their backyard. The eucalyptus tree shuffled like paper in the wind. On the other side of the sliding glass doors, my grandfather reclined in his La-Z-Boy and flipped on the TV. His yellow golf sweater stretched across his belly. Dark-red welts on his legs had erupted through his tanned skin; Band-Aids covered a few.

One morning, while everybody else ate breakfast, I rifled through pictures in my grandmother’s office. There was one of my grandfather in a swimming contest, his number plastered across the chest of his one-piece bathing suit. His body was lean and trim; the lines of his long torso, rigid. I wandered into the kitchen and asked why he stopped swimming. A Winston hung from his mouth when he said that I jabbered too much, just like my grandmother, and that if I wanted to go fishing with him, I must be silent—otherwise, I’d scare the fish. I kept the promise, and when we got home, I studied our catch as it flopped in the shallow water of the slop sink. He cradled a fillet knife; soon he’d gut the fish. “I clean it, your grandmother cooks it,” he said. I wondered if that rainbow trout could have lived forever in the slop sink, but I didn’t dare ask.

We built a house of cards that night—a skyscraper. The trick, my grandfather said, is to bend them.

“Jules, tell them about the time you dealt cards in the circus when you were twelve,” my grandmother said. But he didn’t want to talk about it. I asked him to teach me how to hit and stick at blackjack, like the way he did on the cruises they’d take us on, but he just showed me a bridge shuffle instead.

The next day after that, my grandfather took my brother and my cousin to the dog track. He wouldn’t take me—I asked too many questions, and I wouldn’t sit still. When my brother came home, I asked if they won, but my brother was sworn to secrecy. “Grandpa told us not to tell because if he wins, he has to share half the earnings with Grandma.”

That night at dinner I set the table with my grandmother’s good dishes and then leafed through Phyllis Diller’s autobiography. “Yes, we know a lot of famous people,” my grandmother said. “Phyllis Diller. Monty Hall. Your grandfather even met Salvador Dali.” My grandfather was on a cruise with his family. 1934. There was a masquerade party and my grandfather dressed as the man in the flying trapeze. Salvador Dali wanted to paint him, but my grandfather said that he was too busy to sit for a painting.

“If you change your mind,” Dali said, “let me know.” He handed my grandfather his card.

My grandfather lit a Winston. “Many years later,” my grandmother said, “we’re at a dinner party and we see him, Dali, right there. Twenty feet away.” She doled out bowls of beef barley stew. “So I say, ‘Jules, there’s Dali. There’s your chance.’” My grandfather smiled and coughed—it was a rattling cough. He flicked his cigarette into an ashtray resting to the left of his place-mat on the kitchen table.

“Your grandfather goes over to him, and he says, ‘Mr. Dali, when I was a young man, you saw me at a costume party. I was dressed as the man in the flying trapeze and you wanted me to sit for a painting. You told me to contact you when I was ready. Well, sir, I’m ready now.’ Dali shook his finger at your grandfather and said, ‘Young man, you would have made a lot of money.’”

My grandfather snubbed most of my questions, and his thoughts, of course, were impenetrable. One exception was the time after his son’s—my uncle’s—funeral. “I’ll never get over this,” my grandfather said. And he too died soon after.

My memory of my grandfather is a lot like an abstract painting; gaps and stories weave without logic. Sometimes I’ll even dream of him cut up into landscapes and angular shapes on the wall of the Museum of Modern Art. In one of those dreams I think I even pointed and said, There, right there, is my elusive grandfather—but he’s still so far away.