I Was Making A Meme Of My Wife
I wish I had some wonderfully dramatic story about how this began. I don't.
I wasn't conducting security research. I wasn't testing Google's privacy model. I wasn't attempting prompt injection or trying to trick an artificial intelligence system into telling me something it shouldn't. I was making a meme of my wife.
That's it.
She is beautiful, ridiculously funny and, after everything she's survived in her life, extraordinarily protective of her privacy. Years ago, she created an alias to protect herself. Not because she had something to hide, but because she deserved the same thing every human being deserves: the ability to decide what parts of themselves belong to the world and what parts don't. I won't be publishing that alias here. It isn't mine to publish. So I uploaded an innocent photograph and asked Gemini to make me something funny. Then I sat staring at my phone wondering if I was losing my mind. Because it knew something I had never told it.
Before anyone becomes too excited, allow me to disappoint both the conspiracy theorists and Google's harshest critics equally. I don't know what happened. That's true in the first paragraph of this article and I expect it will still be true in the last. What I do know is considerably narrower and considerably easier to defend. I uploaded a photograph. Gemini returned personal information that I did not provide anywhere in that conversation and confidently explained how it believed it had arrived there. That happened. Everything else remains unanswered.
My first thought wasn't that Google had done something sinister. It was that I had done something stupid. Surely there was a setting I'd forgotten about or a permission I'd granted years ago and never revoked. I've worked with technology long enough to know that the idiot sitting between the keyboard and the chair is usually responsible for most of life's problems. So I went looking. Everything was off.
Search history was off. Timeline was off. Web & App Activity was off. YouTube History was off. Play History was off. Every privacy control I would expect a reasonably technical and privacy-conscious user to check had been disabled. Then I read Google's own words sitting directly above those controls.
"Regardless of these settings, Google can collect and use data that is not associated with your account." I read that sentence twice. Not because it was difficult to understand, but because it quietly changes the meaning of the word "off".
Then I discovered I had been looking at the wrong door entirely.
Gemini has its own controls. Memory was enabled. Connected applications were enabled. Google Photos was connected and explicitly described as allowing Gemini to understand people, places and moments from my photographs. Somewhere during this process I stopped asking: Where did this information come from?
and started asking: Which switch is the privacy switch? Because I don't think most people know the answer anymore.
There is another problem here that I find equally interesting and entirely independent of where the information originated. Gemini did not simply provide an unexpected output. It confidently provided multiple mutually exclusive explanations for that output. First, it came from previous conversations.
Then it didn't. Then it wasn't derived from any information I had provided at all and was simply an unintended output generated by the model itself.
Then, after I informed Gemini that the information was real and private, it confidently explained that it had originated from connected account activity and information visible within the image itself. Four materially different explanations for the same event.
Even if tomorrow we establish beyond all doubt that the information was present somewhere within the image I provided, I am left asking precisely the same question: Why was the model so confident whilst repeatedly changing its account of what happened? There remains one significant unresolved question that may change the scope of this discussion entirely.
Was the information actually visible anywhere within the image provided to Gemini? That question matters enormously because it is the hinge upon which two very different explanations turn. If it was visible, then modern multimodal systems reading it is entirely mundane. If it wasn't, then the information was surfaced from somewhere else and every explanation subsequently provided becomes unreliable, including the reassuring ones.
Before reaching any conclusion, that question deserves an answer. Finally, I want to say something that has very little to do with Google, Gemini or artificial intelligence.
I have deliberately removed and anonymised almost everything that made me investigate this in the first place. The person involved is represented by a cartoon stand-in. Their information is withheld deliberately and will remain that way.
Not because it weakens my argument, but because she's my wife.
I don't particularly care whether this article ever becomes an interesting piece of security research. I care considerably more about the reason it was written.
There is a strange habit amongst people who work with technology. We slowly stop talking about people and start talking about systems. Privacy becomes architecture. Trust becomes design decisions. Human beings become datasets and edge cases.
Somewhere along the way we forget that every interesting privacy discussion begins with somebody.
This one began with my wife. A reasonable person sees the word "Off" and believes it means exactly what it says. Increasingly, I am no longer convinced that is a reasonable assumption to make.
The original question remains unanswered and I am perfectly comfortable leaving it that way until evidence changes that.
I don't know what happened. I know what I saw. I know what I was told. And I know that I was making a meme of my wife when I accidentally discovered that my understanding of Google's privacy model was incomplete. That's enough for today. The rest of the story can wait for evidence.
About the Author
Clayton Bax is an independent security researcher, technologist and writer with a particular interest in privacy, artificial intelligence, human-computer interaction and the unintended consequences of emerging technologies. His work sits at the intersection of technical analysis and human stories, with an emphasis on documenting where the evidence begins and, perhaps more importantly, where it ends.
"The more I learn, the more I realize how much I don't know." - Michel de Montaigne
Contact:
If this article resonates with you, or if you've observed similar behaviour in modern AI systems, I'd be delighted to hear from you. The fastest way to reach me is through my published work and associated contact channels.