The investments are absurdly large but if you believe LLMs will be fundamentally transformative then I can understand the logic.

The big context window is so nice, I can dump in huge files or conversation exports and it can handle them without problems

Am I the same only one who uses the web interface?

I keep thing simple (KISS) because I teach for a living, and only things my student can pick up in a class are useful to me.

I have a slide in a presentation on the topic that's an inverted pyramid, as pretty much the entire LLM field rests on only a few companies, who aren't necessarily even doing everything correctly.

The fact that they even have a seat at such a small table with so much in the pot already and much more forecasted means they command a large buy in regardless of their tech. They don't ever need to be in the lead, they just need to maintain their seat at the table and they'll still have money being thrown their way.

The threat of missing the next wave or letting a competitor gain exclusive access is too high at this point.

Of course, FOMO driving investments is also the very well known pattern of a bubble, and we may have a bit of a generative AI bubble among the top firms where large sums of money are going to go down the drain on investing into overvalued promises because the cost of missing a promise that will actually come to fruition is considered too high.

Ironically the real payoff is probably in focusing on integration layers at this point, particularly given the gains in performance over the past year in research by developing improved interfacing with SotA models.

LLMs at a foundational pretrained layer are in a race towards parity. Having access to model A isn't going to be much more interesting than having access to model B. But if you have a plug and play intermediate product that can hook into either model A or B and deliver improved results to direct access to either - that's where the money is going to be for cloud providers in the next 18 months.

The big context window is pretty magical for some use cases. There are lots of things RAG with limited context can't do.

Having a large context window is pointless unless the model is able to attend to attention on the document submitted. As RAG is basically search, this helps set attention regardless of the model's context window size.

Stuffing random thing into a prompt doesn't improve things, so RAG is always required, even if it's just loading the history of the conversation into the window.

Assuming a new unheard of play called "Romeo and Juliet " RAG can answer "Who is Rosaline?".

But a large context window can answer "How does Shakespeare use foreshadowing to build tension and anticipation throughout the play?" "How are gender roles and societal expectations depicted in the play" or "What does the play suggest about the power of love to overcome adversity?".

In other words, RAG doesn't help you answer questions that pertain to the whole document and aren't keyword driven retrieval queries. It's a pretty big limitation if you aren't just looking for a specific fact.

RAG limits you to answers where the information can be contained in your chunk size.

I dunno I kind of get the perception Microsoft's locked up OpenAI's funding so Google couldn't throw money at them even if they wanted to?

Claude is more creative and that comes with a higher rate of hallucinations. I hope I'm not misremembering but GPT-4 was also initially more prone to hallucinations but also more capable in some ways.

https://docs.anthropic.com/claude/docs/give-claude-room-to-t...

You can see how the AI arrived at the response.

Anecdotally, I now use ChatGPT for at least 25-50% of the queries that I previously would have had no other channel for other than a search engine.

If I was in charge of Alphabet I'd be starting to worry. This move makes them look a bit desperate.

It's not even "trust but verify" but "Oh yup, I didn't thought of that. However it may be a lie because ChatGPT is a pathological liar, and as I cannot possibly trust a lying tool, I'll now verify" (I know, I know, there's a discussion regarding nomenclature: "lying" / "hallucinating" or whatever. But anyone who's actually using ChatGPT knows what I mean).

Basically the output of ChatGPT, for me, goes directly into Google / Wikipedia / etc.

The one case where I can use the output of ChatGPT directly is when I translate from, say, english to french or vice-versa and I know both languages well enough to be able to tell if the translation is okay or not.

Those believing they can use the output of ChatGPT without verifying it are basically these lawyers who referenced hallucinated cases to a judge.

As another person as commented: it didn't even make a dent in Google's search requests and that is no surprise.

For example, all my tech support searches are now GPT4. Those are painful on Google. There's no need to verify with a Google search, since you can just try out what GPT4 says.

Concrete example: I use it all the time to help me with Excel. What it suggests is nearly always correct. It has turned me into an Excel power user within a few weeks.

You need to develop a sense for what it's likely to be correct on, but once you do it's insanely useful. Simple rule of thumb: if you think you'd find a direct response to your question by wading through pages of ad infested Google results, it'll definitely work great on GPT4.

The way I recall it, it took multiple years for Google to go from a secret power user thing to displacing Yahoo and Altavista for the broad user base. And that was at a time where being online in itself was sort of an early adopter thing.

Anyway, I guess my point is, I would be worried if I was Google, and ignore this tech at your own risk...

If for this text output, ChatGPT also linked me directly to the gardens’ sites, scraped the info from the live site, and summarized this - that would actually save a ton of time. Google could have a leg up bc it has a knowledge graph, but so does Microsoft. This requires a lot more than training an LLM - this requires it to be an actual product, not a tech demo like it is. A chat with an agent that occasionally lies is a terrible UI.

I think there’s great scope for UI innovation here. But such an experience might be pretty expensive in terms of compute - lots of LLM queries and extra lookup systems. Someone who does this hard integration work and is willing to spend a lot of resources per query will deliver a delightful, time-saving user experience, and can probably charge for it. And that may be a great value-prop for local AI - you can give it tons of resources to solve your particular problem. As I see it, mass market LLMs that are provided for free will never do this extra work for you. ChatGPT might be in a good position bc it already has a ton of paying customers that it can continue to draw a wall around. Their early Nov announcement might be something along these lines.

Write a polite email to X saying Y -> doesn't need verification.

Rewrite this in formal English for a grant application -> doesn't need verification.

How to tar gz a folder in Linux? -> doesn't need verification. When I get the answer, I will probably think "oh, sure, it was czvf". And even if I didn't know what the arguments were, I would know enough to know that a tar command isn't going to delete my files or anything like that, so I would just try and see if it worked.

Write Python code to show a box plot with such and such data -> doesn't need verification, I'm too lazy to write the code myself (or look into how seaborn worked) but once I see the code, I can quickly understand it and check that it does what it should. Or actually run it and see if it worked.

Brainstorming (give me a bunch of titles for a paper about X/some ideas about how I could do Y) -> doesn't need verification, I can see the ideas and decide which I like and which I don't.

I get that if you're a journalist, a lawyer or something like that, probably the majority of your ChatGPT queries will need verification, but that's definitely not my experience... probably because I don't often ask ChatGPT for things that I don't know, most of my use is either for things that I could do myself but require a time investment and I'd rather ChatGPT does them in a few seconds, or for brainstorming. Neither of those require a Google search at all.

Otherwise I'm regularly coding with it as an assistant while it's not always 100% correct, it's often decimating the time especially when I can request things like a website skeleton layout and it just seamlessly doing it all using CSS flow layouts and whatnot. I ask it to implement a Javascript library for a chat-style interface with bubbles and just bam it's there, even adapted for existing code.

I'll still have to debug and I _will_ still find issues occasionally but the win is still often enormous for me.

ChatGPT 3.5 though. That's mostly just a toy, yes.

It doesn’t always get it perfect, but it gets a lot right. Having enough experience in the field makes it easy to know when it’s right or not. Then you either ask again with more context and information, or do it yourself.

Overall, it’s definitely increased my productivity.

And this is besides the other stuff that search ones can’t do directly but it does quite well. It has definitely cut web searches for me.

If you want to basic information about a physical exercise, a recipe, a travel destination or such... All the content in Google seems to be content farmed. Written by people who don't know, quickly, by copying similar articles. It's just a buggy, manual version of the proverbial gpt anyway. At this point, I may as well just go direct.

If I want to know about doing weighted crunches instead of sit ups, Google results already represent all the downsides of using gpt. The got UI is nicer. You can also narrow in on your questions, push back and generally get the same level of content in a better package.

Maybe the old chestnut of "computerized recipe book" will finally be solved.

^Ra Ra!

you are just forgetting to add word "reddit" at the end of your query..

The good information is there, often, but finding it is hit and miss. Until relatively recently, unofficially archived gives web forums were still available. Reddit search is a downgrade, imo.

In any case gpt can do reddit pretty well.

I am not sure I trust gpt, hallucinations are still very common.

If chat apps take over, I think they will first taker over desktop computers because on mobile it is easier to type a few keywords in the web search bar and get 5 or 6 relevant web results than it is to chat with a chat bot for like 5 minutes.

Informational quires are most likely to be more useful in chat apps than in the classical web search but navigational and transactional quires are here to stay on Google or whatever search engine comes about.

However, while Claude.ai certainly showcases its strengths in specific areas, it doesn't quite measure up to OpenAI's ChatGPT in terms of adaptability and precision. ChatGPT stands out for its capability to understand intricate queries and produce nuanced, tailored responses.

Both platforms have distinct strengths; I firmly believe they'll evolve to dominate different niches in the AI ecosystem.

ChatGPT on GPT-3.5? I can buy this. GPT-4? No fucking way. If Claude got anywhere close, this would be front page news in every tech and tech-adjacent outlet.

This has to be said again and again: "ChatGPT" is meaningless without specifying whether you mean GPT-3.5 or GPT-4; statements about "ChatGPT" (or LLMs in general) capability limits are invalid unless tested on GPT-4.

You're correct, I should have specified. I rarely if ever use GPT-3.5. I have 7 paid / premium accounts, and I primarily use GPT-4.

Have you compared both? For my use case of generating & modifying simple code & Q&A based off of docs claude is in a similar range to GPT-4 & you can pay as you go instead of 20 bucks a month.

Quoting:

"""The Trust is an independent body of five financially disinterested members with an authority to select and remove a portion of our Board that will grow over time (ultimately, a majority of our Board). Paired with our Public Benefit Corporation status, the LTBT helps to align our corporate governance with our mission of developing and maintaining advanced AI for the long-term benefit of humanity."""

and how large is that portion?

N=1

They've already announced a replacement / better model coming but considering how long Google has been doing the AI thing, it's amazing they dropped the ball this badly.

This investment is Claude sounds like their backup plan.

I do expect it to get much smarter by the end of next year.

The fact that 1 or 2 years later the people will get their money back doesn’t mean there was any hope of surviving this a year ago.

The question here is whether the bank can get back some or all of the stolen money, i.e. can FTX customers get back the money & cryptocurrency they gave to FTX for safe keeping, maybe as shares in Anthropic?

Google investing in Anthropic shows that their priorities are on the capture over the usefulness. Because that's all that Claude really is.

[1] https://www.reuters.com/technology/openai-track-generate-mor...

ChatGPT with Bing search is just very good, intuitive and convenient that I highly recommend it if you have not tried it. Imagine a version of ChatGPT that has tight integration with Google Search, Google Scholar, Google Patent, Google Books, Google Deepmind, etc.

Early last year or so I've read that people were lamenting on how Google at the time was struggling to monetize Deepmind products and suddenly OpenAI ChatGPT 3 came and changed the game forever, while the game changing algorithm is right under their nose, so to speak. It's quite telling that none of the original authors of the Attention paper is still with Google for now.

To showcase the powerful nature of ChatGPT 4 with Bing search I've queried where the original authors of the Attention paper are working now and the answers are as follows:

"As of 2023, here are the current affiliations or recent activities of the original authors of the "Attention is All You Need" paper:

Ashish Vaswani: Co-Founder and Chief Scientist at Adept AI

Noam Shazeer: Co-founder and CEO of Character.AI

Niki Parmar: While there isn't specific information about Niki Parmar's current affiliation, it's known that both Vaswani and Parmar were authors at Google when the paper was published in 2017

Jakob Uszkoreit: CEO and co-founder of Inceptive Nucleics, Inc

Llion Jones: Left Google Japan in July 2023 to launch a startup, Sakana AI, alongside David Ha, the former head of Google's AI research arm in Japan

Aidan N. Gomez: CEO & Co-founder of Cohere, a company focused on Natural Language Processing (NLP)

Lukasz Kaiser: The search for Lukasz Kaiser's current affiliation did not yield relevant results. Further research may be required to find his current affiliation.

It's also noted that many of the co-authors have left Google to start their own ventures or joined other organizations since the publication of the paper."

The Lukasz Kaiser who's been at OpenAI since 2021?

https://www.linkedin.com/in/lukaszkaiser

I know it's popular on HN to give Google crap about having bad search results these days and to use ChatGPT instead, but this was literally the first Google search result for me.

IMO it's especially embarrassing that Bing (owned by Microsoft) was unable to find Lukasz Kaiser via LinkedIn (also owned by Microsoft).

That said: native Bing Search gets it right:

    Lukasz Kaiser | LinkedIn

    Connections: 500+
    Followers: 5.3K
    Works For: OpenAI

I guess my point is that it's really hard to tell which pieces of information are correct vs outdated vs outright hallucinated, without doing the legwork yourself.

[0] https://www.linkedin.com/in/nikiparmar

[1] https://www.theinformation.com/briefings/two-co-founders-of-...

To be fair. It says research yourself and provided data for most of the others saving the user enormous amount of time.

The genie is out of the bag.

Google wants Google Bard to become that and use all the Google's services.

The problem was that all of the ML/AI products would demonetize Google's flagship moneymaker.

Everybody is trying to use ML/AI to undo all the damage the Google monopoly on search has done to the web ecosystem. Microsoft is fine with this for now because Google has the dominant position.

It is pretty clear, however, that this is not a sustainable situation. At some point, people are somehow going to want to turn ML/AI into cash extraction from end users. How to do that isn't obvious.

You're there (presumably) because you want to make an impact. You gave them a massive lead with a novel approach... and Google failed to capitalise on that so spectacularly that they're now investing billions into their competitor who's miles ahead of Google, using Google's approach, all to try and head off another competitor who's miles ahead of everybody also using Google's approach

I hope that (counter to stories) they pay the researchers very well, because the best result I can see is Google shutting down what they can publish to try and stop this happening again.

Hopefully they work at the same company, but if they don't - it's not your problem to solve, but VPs and above.

But yes, it can be frustrating if you're junior and take your work personally and attribute its success to its (and even worse, your own) value. After a few years of career, you learn to look at it from a distance.

Google publishes amazing stuff, people are paid great salaries, they have a ton of fun doing it, work life balance is way better than at startups - what's not to like?

Compared to AI startups, neither of these is true.

Food for thought: https://www.aichat.blog/google-exodus-where-are-the-authors-...

Each of the co-authors who spoke to the FT said they wanted to discover what the toolbox they had created was capable of. “The years after the transformer were some of the most fertile years in research. It became apparent . . . the models would get smarter with more feedback,” Vaswani says. “It was too compelling not to pursue this.” [2]

[2] - https://www.ft.com/content/37bb01af-ee46-4483-982f-ef3921436...

Most researchers don't care about products. Many don't even care about any practicality (just the scientific pursuit). How do I know? Well, I happen to work as Research Scientist. I worked at Google 5y. :) For the majority of my colleagues, "having to" support their work after it was published was a nuisance and they wanted to work on new cool stuff instead.

I can. None of this tech will maintain Google as the chokepoint of internet and make billions for them. You can download and run a LLaMA or Mistral but can't download a Google. On-topic information has been commoditised.

Even OpenAI is in a bad spot. They owned the whole LLM mountain in 2020, now they only own the peaks. Almost the whole mountain has been conquered by open models. Their area of supremacy is shrinking by the day.

The two factors at play here are 1. ability to run LLMs locally and 2. ability to improve local LLMs with data exfiltrated from SOTA LLMs.

I foresee an era of increased privacy as a consequence of this. We will be filtering our browsers with LLMs to remove junk and override Google, Twitter and FB. Bad news for ad providers, they got to move into some other profitable line of business now. Nobody got time to read garbage and ads.

OpenAI's models are likely to remain the best, but I see open models catching up and becoming "good enough." Why pay for GPT-4 when I can run a model locally for free? (Barring the initial capital cost of a GPU; and not even this if you're, say, using a Macbook)

https://huggingface.co/codellama/CodeLlama-34b-hf

[1] https://ai.meta.com/llama/

So grandparent's argument makes even less sense as search should also be fungible but it isn't when Google still provides the best search product and has since it started. If OpenAI can maintain that lead and continue to ship improvements that continue to push their peak higher, most people will not pay for anything less even when that price is free.

I would like LLaMa to be more local, responsive and private rather than more intelligent; it is good enough in that regard.

Could I get my employer to pay for it if all free options disappeared? Probably, but I don't have to while they exist.

For assistance, chatgpt is the best but llama is more than adequate.

For code generation they all suck, chatgpt 4 before they nerfed it was good.

I use codellama phind locally

Exactly, there is a paradox that is getting more extreme by the day - that social media (and the broader web) is a wellspring of knowledge, yet also a vortex of addiction, filter bubbles & lost productivity. I want one without the other.

This pushed me to start building open source at OpenLocus, contributions & feedback are welcome - details in other comment.

I'd then put gray rectangles on what where ads and only show visually that, with the grey rectangles, after having covered the ads with gray rectangles.

I just did it as some quick proof-of-concept: there are plenty different ways to do this but I liked that one. It's not dissimilar to services that renders a webpage on x different devices, without you needing to open that webpage on all these devices.

But the issue is that while it's relatively easy to get rid of ads, it's near impossible to get rid of submarine articles/blogs and it's getting harder and harder by the day to get rid of all the pointless webpages generated by ChatGPT or other LLMs that are flooding the web.

Meanwhile sticking to the sites I know (Wikipedia / HN / used car sales websites / a few forums I frequent etc.) and running a DNS locally that blocks hundreds of thousands of domains (and entire countries) is quite effective (I run unbound, which I like a lot for it's got many features and can block domains/subdomains using wildcards).

I'm pretty sure detecting and covering ads before displaying a webpage can be done but I'd say the bigger problem is submarines and overall terribly poor quality LLM generated webpages.

So basically: is it even worth it to detect ads while the web has now got a much bigger problem than ads?

https://doi.org/10.1016/j.procs.2019.08.210

https://doi.org/10.48550/arXiv.1811.04115

https://doi.org/10.3390/s20164587

It pretty much means you have to detect the ad locally. And by then you've already lost and transferred it down at least.

A weak analogy: as you might take a hybrid cloud/local compute strategy, I think it makes sense to be very flexible and use LLMs from different sources for diversity and not getting locked in. I mostly use OpenAI, but I am constantly experimenting with options. Local options are most exciting to me right now.

I usually use APIs, but OpenAI’s app that supports multi modal input images and voice conversation is impressive and points to a future mode of human computer interactions.

Exciting times!

I cannot wait for the day where I can create a list of blocked terms "Musk, Trump, American politics, ..." and have an intelligent LLM filter so that it doesn't matter what site I visit, whether The Verge, or The Guardian, or Reddit, all articles or posts related to these terms will be gone.

I think this has a chance to have a far higher effect than just ad tech. It's going to effect all media publishers. But I also expect them, led by Google and Facebook, to fight this tooth and nail with every dirty trick they can.

As a suggestion, personally I'd rather the offending items just be removed entirely rather than showing "redacted".

The problem is that it requires a bit of self discipline. Google and Meta are probably safe then...

I do use a feed reader already but it's not good for discovering new things. The problem is when I venture out to discover what's going on in the world I'm deluged with Musk spam. A couple of years ago it was Trump spam.

In the more medium term we are collaborating on improving information overload, filter bubbles & misinformation with labs at AllenAI, CMU, UPenn & Utah.

Contributions and feedback are welcome! Feel free to hit me up for an early access - email in the profile.

Like I know the state of journalism is less than stellar, but patching it after the fact for each reader seems like the wrong direction. The implicit conception of "the news" in this desire reifies it into a weird kind of commodity for your personal entertainment/edification; which is precisely the conception operating today which makes it so bad!

Like, maybe, if you have psychological considerations where certain triggers are very damaging, I can kinda understand this. But if that is really the case, then just why read the news anyway? Of course you gotta read some sometimes, but in general you can read other things. There is a lifetime and a half of fiction and nonfiction to read, no GPU required!

It's not about being triggered by anything or trying to hide from anything. I also don't need (or even want) 100% efficacy. It's about cleaning up noise. For example, at this point I'm fully aware that Musk has turned Twitter into even more of a cesspool. I don't need any more information about that. And yet I get it, all the latest "juicy Musk gossip" any time I go near any tech sites. And it's just noise to me at this point.

Same with American politics. I'm not from the US so I'd be happy with a short monthly synopsis on what's happening there. But on the English speaking web, American politics is everywhere. It's exhausting. I want to filter it, reclaim the attention it steals from me while still being engaged with online society to a degree that I choose. And I believe that reclaiming this attention, energy, and time would allow me to engage more with subjects I do care about.

I am contemplating feeding Firefox extensions code to AI to detect possible malicious behaviors quickly.

Like, go back and read old mailing lists. See how everyone was so assured that computers themselves, then cryptography, then the modern internet, would create radical changes in the way the economy works, and enable avenues for greater self-determination and happiness. They all seem so naive in retrospect, knowing what we know and how it all played out.

I love that so many people are these days inspired by the llm tech to imagine a better world with them, and its important in itself to inspired like this. But precedence does not favor putting all your hopes in solely the technology itself for a better world. Hope I am wrong though!

This will have social implications, it will speed up the learning process of any new technology. For example by upgrading Copilot, Github can upgrade coding practices everywhere (where it's used) to new coding standards instead of taking years, that's how fast it can go.

On another line of thinking, an image-language-action model can be used to control robots, and robot hardware is getting accessible. There is a chance for inventing self reliance automation for people, a possible solution to job loss.

It doesn't matter what it can do when there is a truly scary amount of power and interest directed towards, e.g., making sure job loss (or unemployment in general) doesn't change too much. They'll make these things illegal before they'd let it get even close to messing something like that up!

There are already so many innovations and pieces of technology that could be helping instead of hurting. It should seem clear that the sheer capability of something could never be enough to change the state of affairs alone.

Sure the printing press was critical, but there was also a whole lot of blood shed and tumultuous times before the enlightened subject of the printing press could enjoy their new class and literacy.

In terms of FLOPS, Google is rumoured to be by far the biggest, since they have lots of their own silicon, while most other people are limited by what NVidia can make or smallish amounts of custom silicon.

If Google can't turn that into a commercial product, the researchers don't really care.

What models do they have to show after 15 years of AI research? AlphaGo, AlphaFold, and that's about it. Search is mediocre, translation the same, not even their OCR APIs are very good.

On pure research they invented the transformer in 2017 (Vaswani) and word embeddings in 2012 (Mikolov). But Microsoft invented residual connections (Kaiming He 2015) which underline transformers and modern CNNs. CNNs and LSTMs were invented longe before, by Yann LeCun (1989) and Jürgen Schmidhuber (1997).

And yes, OpenAI also invented something of their own - the Adam optimiser (Kingma 2015), that trains 99% of the models today.

Hinton invented Dropout (2014) and backpropagation (1986).

OpenAI has made 1 major contribution - the auto regressive decoder. You could argue also their productisation of RL for LLMs has been highly influential even though they strictly didn’t invent it.

OpenAI is ahead on the RL data side. To me that’s the likely biggest advantage they have in GPT-4.

I've already introduced methods of instruct formatting within our org, because as the architectures change, what we learn from performing the work to organize and account for our internal knowledge will continuously pay off.

Palm-2 was never intended to match 4 (Gemini is). Llama 2 was never intended to match 4 either. Same with Claude 2. Nobody has matched GPT-4's training compute and failed to reach it.

> Diederik P. Kingma* University of Amsterdam, OpenAI [email protected]

Transformers were invented in the early 90s: https://people.idsia.ch/~juergen/fast-weight-programmer-1991... Google were just the first with the compute to scale them up.

They're bad at turning the research into a product.

Specifically, most AI based products need to be trained on user data, and Google is too scared to make any real use of much user data in AI models for fear of getting sued into oblivion if, for example, gmail can be tricked into writing something in an email which reveals some other gmail users private data.

Tbh, Google Lens is one of the best computer vision products in the market. Generally speaking, it's hard to crate a product around something that is completely new.

What? between what languages?

I think that's core to what I'm getting at - I don't see how Google continues to let them publish core research at this point given how much of a lead they've given their competitors (especially combined with their own inability to execute)

From what I hear from Google DeepMind researchers, the current policies are making it very difficult to publish NLP research. The barrier to disclosing research advances to the wider community appears much higher than it used to be.

Have you heard of money?

But seriously, Google have talent because they pay insane salaries like all the other big tech companies

From everything I've read/watched, the original researchers didn't know at that point their work was going to be so impactful. In hindsight, yeah. But otherwise no.

I can't speak to AI researchers specifically but I can speak to research more generally. Ideally you want to focus on somewhere you can make and impact, pursue the general line of research you find interesting/promising, AND have stability to at least live (or ideally enough pay to not care).

I've known enough highly talented researchers who almost always have to sacrifice the "making a difference" and "pursuing their pathway" to large degree just to survive. Ultimately what people are paying for to be researched is what matters, be it some business R&D division's head, federal agency biased directions, of whatever philanthropic connections some organizations are able to extract decide you should be doing. It's not just medicore researchers, it's what one might consider "world class" top of their field researchers. Everyone at some point makes sacrifices to pay the bills but there's this neverending gaslighting that occurs about what researchers want to do and what areas they choose to impact or some nonsense like that. That happens in very few idealized cases. E.g., tenured professorship at highly endowed institutions where they've already invested enough time doing the stuff they don't want to game the system to get to a point they can finally pursue their own paths and so on.

If you want to focus on an impact and perform research you need to be able to self-fund that and hope the area you work isn't capital intensive (AI specifically DNN, LLM, and highly data/compute intensive approaches is for the most part pretty capital intensive, hence one reason big tech pursued them to reduce competition--you can pursue paths theoretically but ultimately unless you find approaches that aren't as capital intensive it'll be awhile before you can experimentally test and iterate approaches).

Researchers just hope to get remotely close to the area they want to work in but by in large have to chase the money and lines of research behind it if they hope to remain a researcher. Maybe Google is different but I doubt it. This happens in other industries as well with people in R&D near the tops of their field working at market leaders still caving.

If by some miracle you happen to be near the top of your field and happen to guess or have some natural deep insight to the path forward where you're fortunate enough to make a huge breakthrough, you can often just spin it out yourself or take on some investment risk. Why work for Google if you have the practical path to say AGI in your hands? Try and do it on your own or find someone who doesn't own all your IP at the end of it. You work at big corp because there's ideally some stability balance with some semi-interesting research paths to you, probably not exactly what you want but as close as you can get.

Google truly has grown complacent and whoever has a good idea feel like they can't develop it there. But if you have an idea on how Google can lose money and reputation by retiring another product they're all ears

At this point Google is offering nothing which comes close to ChatGPT, Bard is a laughing stock compared to it. But I don't doubt that Google will catch up and then offer a better experience.

The only thing OpenAI has from me is that (valuable) chat history (and the monthly subscription money), so I wouldn't have any problems with moving away from them once they are no longer in the unique position they're currently in. This is different with Google, which has several orders of magnitude more data about me and which they also do manage for me.

I'm concerned that it's only a matter of time until OpenAi gets hacked and some of the user account's data gets leaked.

OpenAI says it was an issue with the user devices not their service, though.

OpenAI gave Microsoft a competitive edge and this could just be a hedge to prevent Amazon from getting even further ahead.