This concern was first raised when Gmail started, 20 years ago now; at the time people reeled at the idea of "google reads your emails to give you ads", but at the same time the 1 GB inbox and fresh UI was a compelling argument.

I think they learned from it, and google drive and co were less "scary" or less overt with scanning the stuff you have in it, also because they wanted to get that sweet corporate money.

Some other engines don't do this, and the difference is remarkabe. Try a full-content search in Windows 7, you'll be staring at the dialog for two minutes while it tries to find a file that's in the same directory as you started the search in.

The poster clearly thought about search in terms of the existing Google search functionality which is near instantaneous.

Usability matters to the average end user and a delayed search is not usable for most people.

Apple now encrypts most data in transit and in place also, and they document which data is protected. I am up in the air on whether a future Apple will want to use my data for training public models. Apple’s design of pre trained core LLMs, with local training of pluggable fine tuning layers would seem to be fine, privacy wise, but I don’t really know.

I tend to trust the privacy of Google Drive less because I have authorized access to drive from Colab Pro, and a few third parties. That said, if this article is true, then less trust.

Your analogy with early Gmail is good. I got access to Gmail three years before it became public (Peter Norvig gave me an early private invite) and I liked, at the time, very relevant ads next to my Gmail. I also, gave Google AI plus (or whatever they called their $20/month service) full access to all my Google properties because I wanted to experiment with the usefulness of LLMs integrated into a Workplace type environment.

So, I have on my own volition surrendered privacy if Google properties.

Yeah, you are, actually.

Well, while some of our fellow humans are far too quick to jump on concluding that everything and the rest comes from some conspiracy, it shouldn't void the existence of any conspiracy as an extreme opposite.

In that case, whether these actors do it or not is almost irrelevant: they have the means and incentives to do so. What safeguard civil society is putting in place to avoid it to happen is a far more interesting matter.

With NDA being all over the place, it does strike me as doable.

NDAs should have a time limit.

Additionally, no-one in their right mind will be a whistleblower nowadays.

Sludge in a hole in your shop's backyard? Not worth it.

High level of chemicals in the air which may cause stillborns? Not worth it.

Scanning private files for an AI training? Not worth it.

Genocide? Not worth it.

There is absolutely nothing nowadays worth being banished from your livelihood.

Snowden and Assange are heroes. And insane. Threw away $$$ for their morals. Stanislav Petrov threw his career away instead of passing it up and let it be somebody's else problem.

Of course. Google (and Apple, Microsoft not so much - but it is for your own good) will deny that they store your encryption keys.

This should be mandatory, enforced, and come with strict fines for companies that do not comply.

It's literally just running an algorithm over your data and spitting out the results for you. Fundamentally it's no different from spellcheck, or automatically creating a table of contents from header styles.

As long as the results stay private to you (which in this case, they are), I don't see what the concern is. The fact that the algorithm is LLM-based has zero relevance regarding privacy or security.

I don't want any results from AI. I don't even want to see them. And there is too much of a grey area. What if they use how I use the results to improve their AI. I hate AI also and want nothing to do with its automations.

If I want a document summarized, I will read it myself. I still want to be human and do things AT A REASONABLE LEVEL with my own two hands.

OK, sure. But then just don't use it.

The problem is that you're calling for a legal policy against it to be "mandatory, enforced, and come with strict fines".

Have your own personal preferences, that's great. But I don't want you imposing your preferences on the products I use. I want companies and the market to decide.

An auto-summary feature that is enabled by default is not something we should be asking for government regulation over, any more than we should be asking the government to prohibit wavy red lines unless they're explicitly opted into.

Early search engines had a problem, which was that when they crawled willy nilly, people would block their IP addresses. Inventing this concept of `robots.txt` worked because search engines wanted something: to avoid IP blocks, which they couldn't easily get around. And site hosts generally wanted to be indexed.

Today it's WAY harder to block relevant IP addresses, so site hosts generally can't easily block a crawler that wants its data: there is no compromise to be found here, and the imbalance of power is much stronger. And many site hosts generally don't want to be crawled for free for AI purposes at all. Pretty much anyone who sets up an `ai.txt` uses it to just reject all crawling, so there is no reason for any crawler to respect it.

That's a game of whack-a-mole that always lets a few miscreants through. I used to find that an acceptable amount of error until I learned that crawlers were gathering data to be used to train LLMs. That's a situation where even a single bot getting through is very problematic.

I still haven't found a solution to that aside from no longer allowing access to my sites without an account.

IMO the bigger concern is that this data is not just used to train models. It is stored, completely verbatim, in the training set data. They aren’t pulling from PDFs in realtime during training runs, they’re aggregating all of that text and storing it somewhere. And that somewhere is prone to employees viewing, leakage to the internet, etc.

Isn't this like encryption, though?

I'm fairly sure that the cryptography community basically says: if someone has a copy of your encrypted data for a long time, the likelihood over time for them to be able to read it approaches 100%, regardless of the current security standard you're using.

Who could possibly guarantee that whatever LLM is safe now will be safe at all times over the next 5-10-20 years? And if they're guaranteeing, they're lying.

To which the reply was "they'll just make the LLM able to better defend itself".

And my point was "the attackers will learn to build better prompts, too".

It is not surprising that Gemini will summarize a document if you ask it to. "Scanning" is doing heavy lifting here; The headline implies Google is training Gemini on private documents, when the real issue is Gemini was run with a private document as input to do a summary when the user thought they had explicitly switched that off.

That having been said, it's a meaningful bug in Google's infrastructure that the setting is not being respected and the kind of thing that should make a person check their exit strategy if they are completely against using The new generation of AI in general.

No, but it is surprising that Gemini will summarize every single PDF you have on your Drive if you ask it to summarize a single PDF one time.

Honestly it sounds like he was toggling permissions off and on and actually has no idea why it summarized that particular document despite him requesting it summarize other documents. Google should make the settings more clear, but "I had the options off, except when I didn't, and I set some other options in a different place that I didn't think would override the others, and also I toggled a bunch of the options back and forth" is hardly the condemnation that everyone is making it out to be.

One would think, maybe even expect, a single setting in a single place would control this. And that their docs would be correct.

Glue pizza incident illustrated they're just yolo'ing this

your hacked SMS messages from AT&T are probably next, and everyone will be just as surprised when keystrokes from your phones get hit, or there is a collection agent for model training (privacy enhanced for your pleasure, surely) added as an OS update to commercial platforms.

Make an example of the product managers and engineers behind this, or see it done worse and at a larger scale next time.

They’re trying to suggest that exposing an LLM to a document in any way is equivalent to including that document in the LLM’s training set. That’s the hook in the article and the original Tweet, but the Tweet thread eventually acknowledges the differences and pivots to being angry about the existence of the AI feature at all.

There isn’t anything of substance to this story other than a Twitter user writing a rage-bait thread about being angry about an AI popup, while trying to spin it as something much more sinister.

We really need to get to the point that all data remotely stored needs to be encrypted and unable to be decrypted by the servers, only our devices. Otherwise we just allow the companies to mine the data as much as they want and we have zero insight into what they are doing.

Yes this requires the trust that they in fact cannot decrypt it. I don't have a good solution to that.

Any AI access to personal data needs to be done on device, or if it requires server processing (which is hopefully only a short term issue) a clear prompt about data being sent out of your device.

It doesn't matter if this isnt specifically being used to train the model at this point in time, it is not unreasonable to think that any data sent through Gemini (or any remote server) could be logged and later used for additional training, sitting plaintext in a log, or just viewable by testers.

Yes, this is where it all breaks down. In the end, it all boils down to the company saying "trust us", and it's very clear that companies simply cannot be trusted with these sorts of things.

Ultimately you have to make a decision based on the companies actions and your own personal risk threshold with your own data.

In this particular case, we know that at the very least Google's track record on this is... basically non existent.

Ever since ‘cloud’ privacy took a nosedive

I think it was a reasonable analogy. You can't see inside it; you don't know how it works, and you don't need to. Note that at this time, 'the internet' wasn't the only way of joining heterogenous networks; there was also the OSI stack.

So I was annoyed when some bunch of kids who had never seen such whiteboard diagrams decided to re-purpose the term to refer to whatever piece of the internet they had decided to appropriate, fence-in and then rent out.

I assume anything stored in such a system will be data mined for many purposes. That includes all of Gmail and Google Docs.

Yeah, that should be obvious for many here, but even software engineers believe that AI are sentient things that will remember everything that they see. And that is a problem, because public is afraid of the tech due to a wrong understanding of how it works. Eventually they will demand laws protecting them from stuff that have never existed.

Yes, there are social issues with AI. But the article just shows a big tech illiteracy gap.

Honestly the general public doesn't seem to care, the people freaking out are the tech adjacent people who make money driving clicks to their own content. Regular Joes aren't upset that google shows them a summary of their documents and many of them actively appreciate it.

How long til gmail attachments get uploaded into drive by default through some obscure update that toggles everything to 'yes'?

This already is the case for attachments that exceed 25 megabytes.

I’m not sure how they can claim they have informed consent for this from their customers

I think the more interesting point is how little people seem to care for the auto-summarization feature. Like, why would anyone want to see their archived tax docs summarized by a chatbot? I think whether an "AI" did that or not is almost a red herring.

> "[it] only happens after pressing the Gemini button on at least one document"

I agree the AI aspect is largely a red herring. But I don't think running an algo like a spellchecker within an open document is so awful. If people hate it or it's not useful or accurate, then it should be binned, ofc. And if we're ignoring the AI aspect, then it's just a meh/crappy feature. Not especially newsworthy IMHO.

The autosummarization of unopened documents is closer to the image search functionality I mentioned above than it is to a spell checker running on an open doc. Both autosummarization and image search are content retrieval mechanisms. The difference is only in how its presented. Does it just point you to your file, or does it process it further for you? The privacy aspects are equivalent IMO. The only difference is in whether the feature is useful and well received.

^the privacy settings used to inform Gemini should be openly available, but they aren't, which means the AI is either "hallucinating (lying)" or some internal systems on Google's servers are outright malfunctioning*

Many AI systems do use user interactions as part of training data. So at most you might guess those documents aren’t directly being used for training AND they will never include conversations in training data but you don’t know.

I'm not sure how that is implied.

>Many AI systems do use user interactions as part of training data.

There is no evidence of that being the case here, and none of the mainstream AIs do that yet. They'd be much more useful if they did.

>So at most you might guess those documents aren’t directly being used for training

Or we can actually know that, because that's the case.

>AND they will never include conversations in training data but you don’t know.

Conversations aren't part of this discussion at all, so I'm not sure what you're trying to imply, but it's wrong.

People only know about it because information from these documents is showing up in conversations.

It’s unclear which systems have access and why, but at a minimum Google is showing the data. If things are “misconfigured” or even intentionally set up like this then any assumptions about what’s private goes out the window.

we should just ignore physical constraints of assets which do not have them, like any and all digital data

which do you prefer? everybody can access all digital data of everybody (read only mode), or what we have now which is trending towards having so many microtransactions that every keystroke gets reflected in my bank account

Also, see:

"What's more, Bankston did eventually find the settings toggle in question... only to find that Gemini summaries in Gmail, Drive, and Docs were already disabled"

The author deliberately asked for at least one document to be scanned. He goes on to talk about all the other things that might be overriding the setting, other, potentially more specific settings that would override this.

I agree, there appear to be interactions that aren't immediately obvious, and what takes priority isn't clear. However, the setting was off, and the author did deliberately ask for at least one document to be scanned. Further, there author talks about Labs being on, and that could easily have priority over default settings. After all, that's sort of what Labs is about. Experimenting with stuff and giving approval to do these sorts of things.

Somebody took the time to talk down my comment about this being a strategy to give their AI more training data. I continue believing that if they have your data they will use it.

I think there will be a real shift back on prem with software delivered traditionally due to increased in shit like this (and also due to cost)

Untying photos from my google account is even better!

Pixels have first class support

You can also disable Network access to any app

(It's a buggy ride though and requires reading a lot of docs and forum posts)

I've been using GrapheneOS for years (Pixel 3 through 7), with only open source add-on apps and no Google Play Store, and it's been pretty solid. (Other than my carrier seeming to hate the 6a hardware or model specifically.)

It was referring to the overall experience to which I was referring, not the OS specifically ("it's a buggy ride", it = the ride, not GrapheneOS)

I imagine a lot of the issues are because of the apps not testing on GrapheneOS.

But I've had lots of little issues:

- Nova Launcher on a daily basis stopped working when pressing the right button (the 'overview' button). I had to kill the stock Launcher app to fix it, interestingly. Had to revert to the stock launcher

- 1Password frequently doesn't trigger auto-fill in Vivaldi

- Occasionally on boot the SIM unlock doesn't trigger

- Camera crashing often (yes, "could be hardware"...I read the forums/GitHub issues)

More that I can't remember. It's a bit frustrating.

But don't get me wrong, I appreciate the project. I'm not going to go back to stock

Because that's an insane interpretation of what's happening.

The author first refers to his source as Kevin Bankston in the article's subtitle. This is also the name shown in the embedded tweet. But the following two references call him Kevin _Bankster_ (which seems like an amusing portmanteau of banker and gangster I guess).

Is the author not proofreading his own copy? Are there no editors? If the author can't even keep the name of his source straight and represent that consistently in the article, is there reason to think other details are being relayed correctly?

Hopefully along with ten hallucinated life stories for the AI author, to pad the blog spam recipe page for SEO.

I am unaware of "these corporations" -- which ones exactly? my answer doesn't hinge on you making that clear but you still should -- throwing their opponents into reeducation camps, or outright killing them.