I'm a C programmer, so I can with some effort gradually work my way through random Linux kernel things. But what I can do now instead is take a random function, ask GPT4 what it does and what subsystem it belongs to, and then ask GPT4 to write me a dummy C program that exercises that subsystem (I've taken to asking it to rewrite kernel code in Python, just because it's more concise and easy to read).

I don't worry at all about GPT4 hallucinating stuff (I'm sure it's doing that all the time!), because I'm just using its output as Cliff's Notes for the actual kernel code; GPT4 isn't the "source of truth" in this situation.

I don't need the answer to be correct because I'm going to do that part myself. What they do is make it an order of magnitude faster to get anything on the board. They're the ultimate prep cook.

There are things to dislike and yes there is over-hype but "making learning less tedious" is huge!

I feel more interested in approaching challenging problems in fact because I know I can get over those frustrating phases much more easily and quickly.

Together with increasingly powerful speech to text I find myself talking to the computer more and more.

There are flaws, there are weaknesses, and a bubble, but any dev that can't find any benefit in LLMs is just not looking.

Especially for debugging issues that could previously take days of searching documentation, Stack overflow, and obscure tech forums. I can now ask an LLM, and maybe 75% of the time I get the right answer. The other 25% of the time it still cuts down on debugging time by helping me try various fixes, or it at least points me in the right direction.

It has also made making small pure vanilla html/js based tools fun. It gives me a good enough prototype which I can mold to my needs. I have wrote a few very useful scripts/tools past few months which otherwise I would never even have started because of all the required first steps and basic learnings.

(never thought I would see your comment as a user)

For example, I was looking at a differential equation recently and saw some unfamiliar notation[1] (Newton's dot notation). So I asked claude for why people use Newton's notation vs Lagrange's notation. It gave me an excellent explanation with tons of detail, which was really helpful. Except in every place it gave me an example of "Lagrange" notation it was actually in Leibniz notation.

So it was super helpful and it didn't matter that it made this specific error because I knew what it was getting at and I was treating it as a "smart friend" who was able to explain something specific to me. I would have a problem if I was using it somewhere where the absolute accuracy was critical because it made such a huge mistake throughout its explanation.

[1] https://en.wikipedia.org/wiki/Notation_for_differentiation#N...

As I use it more and more often the mistakes are born of ambiguity. As I supply more information to the LLM it's answer(s) gets better. I'm finding more and more ways to supply it with robust and extensive information.

- "things trivial to verify", so it doesn't matter if the answer is not correct - I can iterate/retry if needed and fallback to writing things myself, or

- "ideas generator", on the brainstorming level - maybe it's not correct, but I just want a kickstart with some directions for actual research/learning

Expecting perfect/correct results is going to lead to failure at this point, but it doesn't prevent usefulness.

The lid lock came a few days later, I put it in, and I'm able to wash laundry again.

Sobering thought: centaur tournaments at the top level are no more. That's because the computers got so good that the human half of the beast no longer added any meaningful value.

https://en.wikipedia.org/wiki/Advanced_chess

Beautiful story, and thanks for sharing :)

When I'm diving into a wholly new subject for the first time, in a field totally unrelated to my field (similar to the author, C programming and security) for example biochemistry or philosophy or any field where I don't have even a basic grounding, I still worry about having subtly-wrong ideas about fundamentals being planted early-on in my learning.

As a programmer I can immediately spot "is this code doing what I asked it to do" but there's no equivalent way to ask "is this introductory framing of an entire field / problem space the way an actual expert would frame it for a beginner" etc.

At the end of the day we've just made the reddit hivemind more eloquent. There's clearly tons of value there but IMHO we still need to be cognizant of the places where bad info can be subtly damaging.

High school science is a great example: once you get to university you have to un-learn all sorts of things that you learned earlier because they were simplifications that no longer apply.

Terry Pratchett has a great quote about this: https://simonwillison.net/2024/Jul/1/terry-pratchett/

For fields that I'm completely new to, the thing I need most is a grounding in the rough shape and jargon of the field. LLMs are fantastic at that - it's then up to me to take that grounding and those jargon terms and start building my own accurate-as-possible mental model of how that field actually works.

If you treat LLMs as just one unreliable source of information (like your well-read friend who's great at explaining things in terms that you understand but may not actually be a world expert on a subject) you can avoid many of the pitfalls. Where things go wrong is if you assume LLMs are a source of irrefutable knowledge.

I guess part of my problem with using them this way is that I am that well-read friend.

I know how the sausage is made, how easy it is to bluff a response to any given question, and for myself I tend to prefer reading original sources to ensure that the understanding that I'm conveying is as accurate as I can make it and not a third-hand account whose ultimate source is a dubious Reddit thread.

> High school science is a great example: once you get to university you have to un-learn all sorts of things that you learned earlier because they were simplifications that no longer apply.

The difference between this and a bad mental model generated by an LLM is that the high school science models were designed to be good didactic tools and to be useful abstractions in their own right. An LLM output may be neither of those.

That's a great point about high school models being deliberately designed as didactic tools.

LLMs will tend to spit those out too, purely because the high school version of anything has been represented heavily enough in the training data that it's more likely than not to fall out of the huge matrix of numbers!

That assumes that the high school version of the subject exists, which is unlikely because I already have the high school version of most subjects that have a high school version.

The subjects that I would want to dig into at that level would be something along the lines of chemical engineering, civil engineering, or economics—subjects that I don't yet know very much about but have interest or utility for me. These subjects don't have a widely-taught high school version crafted by humans, and I don't trust that they would have enough training data to produce useful results from an LLM.

If I were a lawyer using these things for work, I'd be insane to trust one at this stage, but the reality is I'm not using my digging into things I don't know about for anything load bearing, but even if I were, I'd still use an LLM to get started. Eg the post didn't state how the author learned anything the name for the dropped letter O, but I can describe a thing and have the LLM give me the name of it. there's an emphasis on getting things totally 100% right does erode trust, but you get a sense for what's could be a hallucination and then check background resources if you get enough experience with the tool.

And writing shell scripts that "mostly" work is what it does.

I don't expect it to work. Just like I don't expect my own code to ever work.

My stuff mostly works too. In either case I will be shaving yaks to sort out where it doesn't work.

At a certain level of complexity, the whole house of cards does break down where LLMs get stuck in a loop.

Then I will try using a different LLM to get it unstuck from the loop, which works well.

You will have cases where both LLMs get stuck in a loop, and you're screwed. Okay.. well, now you're however far ahead you were at that stage.

Essentially, some of us have spent more of our life fixing code, than we have writing it from scratch.

At that level, it's much easier for me to fix code, than write it from scratch. That's the skill you're implementing with LLMs.

1. Search engine - Words like "teach" or "learn" used to be slapped on Google once upon a time. One real great thing about LLMs here is that they do save time. The internet these days is unbelievably crappy and choppy. It often takes more time to click through the first item in the Google result and read it than to simply ask an LLM and wait for its slowish answer.

2. Pattern matching and analysis - LLMs are probably the most advanced technology for recognizing well-known facts and patterns from text, but they do make quite some errors especially with numbers. I believe that a properly fine-tuned small LLMs would easily beat gigantic models for this purpose.

3. Interleaving knowledge - this is the biggest punch that LLMs have, and also the main source of all the over-hype (which does still exist). It can produce something valuable by synthesizing multiple facts, like writing complex answers and programs. But this is where hallucination happens most frequently, so it's critical that you review the output carefully.

The problem is that AI is being sold to multiple industries as the cure for their data woes.

I work in education, and every piece of software now has AI insights added. Multiple companies are selling their version as hallucination free.

The problem is the data sets they evaluate are so large and complicated for a college that there is literally no way for humans to verify the insights.

It's actually kind of scary. Choices are being made about the future of human people based on trust in New Software.

> I believe that a properly fine-tuned small LLMs would easily beat gigantic models for this purpose.

I've long felt that vertical search engines should be able to beat the pants off Google. I even built one (years ago) to search for manufacturing suppliers that was, IMO, superior to Google's. But the only way I could get traffic or monetize was as middleware to clean up google, in a sense.

I was a bit excited at something being able to do that, but this apparently means simplifying a single file, based on their example.

I suspect they're having an unusually positive experience with these tools due to working on a lot of new, short, programs.

That's academia for you :)

It also helps that he specialises deep learning models and LLMs and knows a thing or two about the inner workings, how to prompt (he authored papers about adversial attacks on LLMs) and what to expect.

It is strange, I can read okay - by formations of words is challenging.

LLMs are far from flawless of course and I often get stuck with non working code. Or it is taking annoying shortcuts in giving a detailed answer, or it just wastes a lot of time repeating the same things over and over again. But that's often still useful. And you can sometimes trick them into doing better. Once it goes down the wrong track, it's usually best to just start a new conversation.

There are a few neat tricks that I've learned over the last year that others might like:

- you can ask chat gpt to generate some files and make them available as a zip file. This is super useful. Don't wait for it to painfully slowly fill some text block with data or code. Just ask it for a file and wait for the link to become available. Doesn't always seem to work but when it does it is nice. Great for starting new projects.

- chat gpt has a huge context window so you can copy paste large source files in it. But why stop there? I wrote a little script (with a little help of course) that dumps the source tree of a git repository into a single text file which I can then copy into the context. Works great for small repositories. Then you can ask questions like "add a function to this class that does X", "write some unit tests for foo", "analyze the code and point out things I've overlooked", etc.

- LLMs are great for the boring stuff. Like writing exhaustive unit tests that you can't be bothered with or generating test data. And if you are doing test data, you might as well have some fun and ask it to inject some movie quotes, litter it with hitchhiker's guide to the galaxy stuff, etc.

The recent context window increase to 128K with chat gpt 4o and other models was a game changer. I'm looking forward to that getting even larger. The first few publicly available LLMs had the memory of a gold fish. Not any more. Makes them much more useful already. Right now most small projects easily fit into its context already.

It also helps to remind it that its role is a "senior developer" and that it should write code that likens it to a senior developer. It will be happy to act like a junior dev if you don't explicitly tell it.

Also, always remember to say please, thank you, hello, and that you'll tip it money - these HAVE made differences over time in my tests.

1. Nicholas really does know how badly machine learning models can be made to screw up. Like, he really does. [0]

2. This is how Nicholas -- an academic researcher in the field of security of machine learning -- uses LLMs to be more efficient.

I don't know whether Nicolas works on globally scaled production systems with have specific security/data/whatever controls that need to be adhered to, or whether he even touches any proprietary code. But seeing as he heavily emphasised the "i'm a researcher doing research things" in the article -- I'd take a heavy bet that he does not. And academic / research / proof-of-concept coding has different limitations/context/needs than other areas.

I think this is a really great write up, even as someone on the anti-LLM side of the argument. I really appreciate the attempt to do a "middle of the road" post which is absolutely what the conversation needs right now (pay close attention to how this was written LLM hypers).

I don't share his experience, I still value and take enjoyment from the "digging for information" process -- it is how I learn new things. Having something give me the answer doesn't help me learn, and writing new software is a learning process for me.

I did take a pause and digested the food for thought here. I still won't be using an LLM tomorrow. I am looking forward to his next post, which sounds very interesting.

[0]: https://nicholas.carlini.com/papers

> academic / research / proof-of-concept coding has different limitations/context/needs than other areas.

I’m personally drawn to threads contentious enough to be flagged, but that have been vouched for by folks who have the vouch capability (mods and participants who haven’t had vouch capability suspended). Good signal imho.

But as the AI gets increasingly competent at showing us how to do things, knowing what task is worth doing and what not is still a task for the one who asks, not the AI.

Edit: One type of question I've found interesting is to make it speculate, that is asking questions that it doesn't know the answer to, but still is able to speculate because they involve combining things it does know about in novel (though not necessarily valid) ways.

- CLI commands and switches I don't care to or easily remember

- taking an idea and exploring it in various ways

- making Slack messages that are more engaging

Using GPTs has a cost of breaking my concentration/flow, so it's not part of my core workflows.

I really need to start weaving it into the programming aspects of my workday.

It’s not perfect, but AI for working with code has been an absolute game changer for me.

Typically there is fixing needed (e.g. it shall fuck up things as trivial as parentheses placement) but it's still useful.

Lots of french/english translation too: it's actually good at that.

Together with voice to text through whisper for example we broke the UI barrier.

It takes a little bit of time to rebuild our ecosystem but llms are game changer already.

I'm waiting for a finetuned none fact knowing small LLM which knows everything it needs to know for this specific task.

And I'm waiting until everything critical is rewritten so I can use one ai agent to control my bank, calendar, emails and stuff.

Perhaps through banking read only account permissions or whatnot.

LLMs just use the commons, and should only be able to be owned by everyone in society.

The problem comes in with unaccountable private totalitarian institutions. But that doesn't mean the technology is an ethical social issue, it's the corporations who try to own common things like the means of production that is the problem.

Yes, there's the pragmatic view of the society we live in, and the issues that it contains, but that's the ethical issue that we need to address.

Not that we can as a society create LLMs based on the work of society.

For example, if you use a LLM to summarize this whole debate, you would get a decent balanced report, incorporating many points of view. Many times the article generated from the chat thread is better than the original one. Certainly better grounded in the community of readers, debunks claims, represents many perspectives. (https://pastebin.com/raw/karBY0zD)

That's funny, because I was using forum debates as LLM reference precisely in order to reduce errors. People usually debunk stupid articles, the discussion is often centered on fact checking. A LLM referencing a HN/reddit thread is more informed than one reading the source material.

There is a fundamental conflict of interest in press. It costs money to run a newspaper, and then you just give it away for free? No, you use it to push propaganda and generally to manipulate. Forums have become the last defense for actual readers. We trust other people will be more aligned with our interests than who wrote the article.

I trust my forum mates more than press, and LLM gives a nice veneer to the text. No wonder people attach "reddit" to searches, they want the same thing. The actual press is feeding us the real slop. LLMs are doing a service to turn threads into a nice reading format. Might become the only "press" we trust in the future.

If it’s ok I’d like to both share how I’m navigating my skepticism and also being mindful of the need to keep in perspective other people’s skepticism if it doesn’t offer anything to compare.

Why? I have friends who can border on veiled cynicism without outlining what might be in the consideration of skepticism. The only things being looked at are why something is not possible, not a combination. Both can result in a similar outcome.

Not having enough time to look into intent enough, it just invalidates the persons skepticism until they look into it more themselves. Otherwise used as a mechanism to try and trigger the world to expend mental labour for free on your behalf.

It’s important to ask one’s self if there may be partially relevant facts to determine what kind of skepticism may apply:

- Generally, is there a provenance of efficiency improvement both in the world of large scale software and algorithmic optimizations?

- Have LLMs become more optimized in the past year or two? (Can someone M1 Max Studio run more and more models that are smaller and better to do the same)

- Generally and historically is there provenance in compute hardware optimizations, for LLm type or LLM calculations outright?

- Are LLMs using a great deal more resources on average than new technologies preceding it?

- Are LLMs using a massive amount of resources in the start similar to servers that used to take up entire rooms compared to today?

Otherwise Claude 3.5 is really good and gpt-4o is ok with apps like Plandex and Aider. You need to get a feel for which one is better for what task though. Plain questions to Claude 3.5 API through the Chatbox app.

Research questions often go to perplexity.ai because it points to the source material.

I just wish they were better at recognizing when their help is not wanted because I would often disable it and forget to turn it back on for a while. Maybe a "mute for an hour" would fix that.

It's just me asking questions/pasting code into a ChatGPT browser window.

Allows you to open chats directly in a neovim window. Also, allows you to select some text and then run it with certain prompts (like "implement" or "explain this code"). Depending on the prompt you can make the result appear directly inside the buffer you're currently working on. The request to the ChatGPT API also is enriched with the file-type.

I hated AI before I discovered this approach. Now I'm an AI fanboy.

I mean, sure, you do, but there's less off-putting ways to display your credentials...

Not to mention that the author is not really learning the underlying tech in a useful way. They may learn how to prompt to correct the mistakes the LLM makes, but if it was a nightmare to go through this process once, then dealing with repeating the same laborious walkthrough each time you want to do something with Docker or build a trivial REST API sounds like living in hell to me.

Glad this works for some folks. But this is not the way I want to interact with computers and build software.

Probably not, there's a very long tail to this sort of stuff, and there's plenty of programming to go around.

I'll chime in with your enthusiasm though. Like the author of the post, I've been using LLMs productively for quite a while now and in a similar style (and similarly skeptical about previous hype cycles).

LLMs are so useful, and it's fascinating to see how far people swing the opposite way on them. Such variable experiences, we're really at the absolute beginning of this whole thing (and the last time I said that to a group of friends there was a range of agreement/disagreement on that too!)

Very exciting.

At this point i believe most of them can not be convinced that LLMs are valuable or useful by any sort of evidence, but if anything could do it, this article could. Well done.

Ops engineers [0] are the ones who have to spend weekends fixing production systems when the development team has snuck in "new fangled tools X, Y and Z" into a "bugfix" release.

We have been burned by "new fangled" too many times. We prefer "old reliable" until "new fangled" becomes "fine, yes, we probably should".

[0]: DevOps has now become a corporate marketing term with no actual relevance to the original DevOps methodology

They've been great for helping me with work-related tasks. It's like having a knowledgeable co-worker with infinite patience, and nothing better to do. Neither the people nor the LLM give back perfect answers every time, but it's usually more than enough to get me to the next step.

That said, having good domain knowledge helps a lot. You make fewer mistakes, and you ask better questions.

When I use LLMs for tasks I don't know much about, it takes me a lot longer than someone who does know. I think a lot of people - not just infrastructure people - are missing out by not learning how to use LLMs effectively.

I always just let it write my first draft of docker, k8s and terraform configs.

“Open” AI - customer noncompete Copilot - customer noncompete Anthropic - customer noncompete Gemini - customer noncompete (api only, wow)

Just imagine millions of people know about the imitation game and still pay someone to fuck them over like that.

Surely, our descendants will thank us for such great contributions to the memory banks of the monopolies of the boring dystopia ..

For instance -

The Hough transform detects patterns with certain structure in images, e.g. circles or lines.

So I'm looking for academic research papers which apply the Hough transform to audio spectra, to recognize the harmonic structure of tonal audio and thus determine the fundamental pitch. (i.e. the Hough space would be a 1D space over fundamental frequency).

I've spent probably 90 minutes, over the several times I've read an optimistic post like this, asking various LLMs (mostly GPT-4o, though my early tests predate GPT-4o, and I've also tried Gemini and Claude), prompts along the lines of

> The Hough transform detects patterns with certain structure in images, e.g. circles or lines. > I'm looking for academic research papers (please link them or provide a DOI.org link at least) which apply the Hough transform to audio spectra, to identify the harmonic structure of audio and thus determine the fundamental pitch. > Make sure to provide only papers that actually exist. If you can't find anything particularly relevant, say so as a disclaimer & just provide the most relevant papers you can.

This is a reliable "fake paper generator", unfortunately - it'll just make up plausible garbage like

> Here are some academic papers related to applying the Hough transform to audio spectra for identifying harmonic structures and fundamental pitch:

> "An Audio Pitch Detection Algorithm Based on the Hough Transform" > Authors: Mark W. and John D. > Published In: IEEE Transactions on Audio, Speech, and Language Processing > DOI: 10.1109/TASL.2008.2000773 > Abstract: This paper proposes an audio pitch detection algorithm that utilizes the Hough transform to analyze the harmonic structure of audio spectra and determine the fundamental pitch.

This paper does not exist. Complete waste of my time. And again, this behavior persists over the >1 year period I've been trying this query.

And it's not just search-like tasks. I've tried asking for code and gotten stuff that's outright dangerous (try asking for code to do safe overflow-checked addition on int64_t in C- you have about an 80% chance of getting code that triggers UB in one way or another). I've asked for floating-point calling conventions on RISC-V for 32-bit vs 64-bit (would have been faster than going through the extension docs), and been told that RV64 has 64 floating-point registers (hey, it's got a 64 in the name!). I've asked if Satya Nadella ever had COVID-19 and been told- after GPT-4o "searched the web"- that he got it in March of 2023.

As far as I can tell, LLMs might conceivably be useful when all of the following conditions are true:

1. You don't really need the output to be good or correct, and 2. You don't have confidentiality concerns (sending data off to a cloud service), and, 3. You don't, yourself, want to learn anything or get hands-on - you want it done for you, and 4. You don't need the output to be in "your voice" (this is mostly for prose writing, for code this doesn't really matter); you're okay with the "LLM dialect" (it's crucial to delve!), and 5. The concerns about environmental impact and the ethics of the training set aren't a blocker for you.

For me, pretty much everything I do professionally fails condition number 1 and 2, and anything I do for fun fails number 3. And so, despite a fair bit of effort on my part trying to make these tools work for me, they just haven't found a place in my toolset- before I even get to 4 or 5. Local LLMs, if you're able to get a beefy enough GPU to run them at usable speed, solve 2 but make 1 even worse...

2 researchgate papers (Overlapping sound event recognition using local spectrogram features with the Generalised Hough Transform July 2013 Pattern Recognition Letters)

and one ieee publication (Generalized Hough Transform for Speech Pattern Classification, in IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 23, no. 11, pp. 1963-1972, Nov. 2015)

When I am looking for real web results chatgpt is not very good, but perplexity very often shines for me

and for python programming have a look at withpretzel.com which does the job for me.

just my 2 ct

I've not yet tried asking LLMs Kotlin-based questions, so don't know how good they are. I'm still exploring how to fit LLMs and other AI models into my workflow.

Regardless, yeah- I can definitely believe your point about corpus size. If I was doing, say, frontend dev with a stack that's been around a few years, or Linux kernel hacking as tptacek mentioned, I could plausibly imagine getting some value.

One thing I do do fairly often is binary reverse engineering work- there's definitely things an LLM could probably help with here (for things like decompilation, though, I wonder whether a more graph-based network could perform better than a token-to-token transformer - but you'd have to account for the massive data & pretrain advantage of an existing LLM).

So I've looked at things like Binary Ninja's Sidekick, but haven't found an opportunity to use them yet - confidentiality concerns rule out professional use, and when I reverse engineer stuff for fun ... I like doing it, I like solving the puzzle and slowly comprehending the logic of a mysterious binary! I'm not interested in using Sidekick off the clock for the same reason I like writing music and not just using Suno.

One opportunity that might come up for Sidekick, at least for me, is CTFs- no confidentiality concerns, time pressure and maybe prizes on the line. We'll see.

> 2. You don't have confidentiality concerns (sending data off to a cloud service)

At $PREVIOUS_COMPANY LLMs were straight up blanket banned for these reasons too. Confidentiality related to both the code and data for the customers.

The possibility that "it might get some things right, some of the time" was nowhere near a good enough trade-off to override the confidentiality concerns.

And we definitely did not have staff/resources to do things local only.

Asking a pure (ie no internet/search access) LLM for papers on a niche subject is doubling down on their weaknesses. That requires LLMs to have very high resolution specific knowledge, which they do not have. They have more coarse/abstract understanding from their training data, so things like paper titles, DOIs, etc are very unlikely to persist through training for niche papers.

There are some LLMs that allow searching the internet; that would likely be your best bet for finding actual papers.

As an experiment I tried your exact prompt in ChatGPT, which has the ability to search, and it did a search and surfaced real papers! Maybe your experiment was from before it had search access. https://chatgpt.com/share/a1ed8530-e46b-4122-8830-7f6b1e2b1c...

I also tried approaching this problem with a different prompting technique that generally tends to yield better results for me: https://chatgpt.com/share/9ef7c2ff-7e2a-4f95-85b6-658bbb4e04...

I can't really vouch how well these papers match what you're looking for since I'm not an expert on Hugh transforms (would love to know if they are better!). But my technique was: first ask it about Hugh transforms. This lets me (1) verify that we're on the same page, and (2) loads a bunch of useful terms into the context for the LLM. I then expand to the example of using Hugh transforms for audio, and again can verify that we're on the same page, and load even more terms. Now when I ask it to find papers, it had way more stuff loaded in context to help it come up with good search terms and hopefully find better papers.

With regards to your criteria:

1. The code from an LLM should never be considered final but a starting point. So the correctness of the LLM's output isn't super relevant since you are going to be editing it to make it fully correct. It's only useful if this cleanup/correction is faster than writing everything from scratch, which depends on what you're doing. The article has great concrete examples of when it makes sense to use an LLM.

2. Yep , although asking questions/generating generic code would still be fine without confidentiality concerns. Local LLMs though do exist, but I personally haven't seen a good enough flow to adopt one.

3. Strong disagree on this one. I find LLMs especially useful when I am learning. They can teach me eg a new framework/library incredibly quickly, since I get to learn from my specific context. But I also tend to learn most quickly by example, so this matches my learning style really well. Or they can help me find the right terms/words to then Google.

4. +1 I'm not a huge fan of having an LLM write for me. I like it more as a thinking tool. Writing is my expression. It's a useful editor/brainstormer though.

5. +1

"make me a docker container that does foo." "now have it do bar."

though the author uses emacs, so maybe they get the same level of not-having-to-copy-and-paste.

- LLM-assistance helps solve 80% of programming tasks, so 80% of programmers lose their jobs

- LLM-assistance provides that exact same productivity boost, and as a result individual programmers become FAR more valuable to companies - for the same salary you get a lot more useful work out of them. Companies that never considered hiring programmers - because they would need a team of 5 over a 6 month period to deliver a solution to their specific problem - now start hiring programmers. The market for custom software expands like never before.

I expect what will actually happen will be somewhere between those two extremes, but my current hope is that it will still work out as an overall increase in demand for software talent.

We should know for sure in 2-3 years time!

ML supercharges all disparity, business owners or superstars who made a nice career and name will earn more by commanding fleets of cheap (except energy) llms while their previous employees/reports get laid off by tens of thousands (ironically they do it to themseves by wecoming llms and thinking that the next guy will be the unlucky one, same reason unions don't work there I guess...)

And to small businesses who never hired programmers before, companies like ClosedAI monetize our work for their bosses to get full products out of chatbots (for now buggy but give it a year). Those businesses will grow but when they hire they will get cheap minimal wage assistants who talk to llms. That's at best where most programmers are headed. The main winners will be whoever gets to provide ML that monetize stolen work (unless we stop them by collective outrage and copyright defense), so Microsoft

As for "their bosses to get full products out of chatbots": my current thinking on that is that an experienced software engineer will be able to work faster with and get much higher quality results from working with LLMS than someone without any software experience. As such, it makes more sense economically for a company to employ a software engineer rather than try to get the same thing done worse and slower with cheaper existing staff.

I hope I'm right about this!

> than someone without any software experience

- So you are betting against ML becoming good enough soon enough. I wouldn't be so sure considering the large amount of money and computing energy being thrown into it and small amount of resistance from programmers.

- Actually someone doesn't have to be zero experience. But if someone is mostly an llm whisperer to save boss some yacht time, instead of engineer, someone is paid according minimal wage.

When that’s not true any more we will have built AGI/ASI. Then we are into science fiction Star Trek utopia / Matrix dystopia world and all bets are off.

    1 Fire 80% programmers
    2 Spread the 20% non-bullshit parts among 20%
    3 Use llm for 80% bullshit parts

You and your good useful programmer coworkers do 80% llmable bullshit, 20% good stuff. So among you, if your boss is smart he will fire 80% of you and spread 20% non-llmable work across remaining people. You hope your coworker gets fired, your coworker hopes it's you, and you both help make it happen

If all of this is just a hobby for you, then it doesn't matter. But it matters a lot when you are serving other people; it matters when you must account for your work.

You could make the case that all testing is a waste of time, because "I can do this, and this, and this. See? It appears to work. Why bother testing it?" We test things because it's irresponsible not to. Because things fail fairly often.

I'm looking through the author's examples. It appears that he knows a lot about technology in general, so that he can be specific about what he wants. He also appears to be able to adjust and evaluate the results he gets. What if someone is bad at that? The LLM can't prompt itself or supervise itself.

I come to everything with the mindset of a tester. LLMs have most definitely been overhyped. That doesn't mean they are useless, but any article about what they are able to do which doesn't also cover how they fail and how you be ready for them to fail is a disservice to the industry.

> For instance, the emission footprint of a US resident is approximately 15 metric tons CO2e per year [22], which translates to roughly 1.7 kg CO2e per hour. Assuming that a person’s emissions while writing are consistent with their overall annual impact, we estimate that the carbon footprint for a US resident producing a page of text (250 words) is approximately 1400 g CO2e.

Averaging this makes no sense. I would imagine driving a car is going to cause more emissions than typing on a laptop. And if we are comparing "emissions from AI writing text" to "emissions from humans writing text" we cannot be mixing the the latter with a much more emissions causing activity and still have a fair comparison.

But that's besides the point, since it seems that the number being used by the authors isn't even personal emissions -- looking at the source [22], the 15 metric tons CO2e per year is labeled as "Per capita CO₂ emissions; Carbon dioxide (CO₂) emissions from fossil fuels and industry. Land-use change is not included."

This isn't personal emissions! This is emissions from the entire industrial sector of the USA divided by population. No wonder why AI is supposedly "100-1000x" more efficient. Counting this against the human makes no sense since these emissions are completely unrelated to the writing task the person is doing, its simply the fact they are a person living in the world.

That's the whole point! If a task requires some time from a human, then you have to include the appropriate fraction of the (huge!) CO2 cost of "being a human" - the heating/cooling of their house, the land that was cleared for their lawn, and the jet fuel they burn to get to their overseas trip, etc, because all of those are unalienable parts of having a human to do some job.

If the same task is done by a machine, then the fraction of the fixed costs of manufacturing the machine and the marginal costs of running (and cooling) it are all there is.

1) Pretend I want something written, and I want to minimize emissions. I can ask my AI or a freelancer. The total CO2 emissions of the entire industrial sector has nearly no relation to the emissions increase by asking the freelancer or not. Ergo, I should not count it against the freelancer in my decision making.

2) In the above scenario, there is always a person involved - me. In general, an AI producing writing must be producing it for someone, else it truly is a complete waste of energy. Why do the emissions from a person passively existing count when they are doing the writing, but not when querying?

3) If you do think this should be counted anyways, we are then missing emissions for the AI as the paper neglects to account for the emissions of the entire semiconductor industry/technology sector supporting these AI tools; it only computes training and inference emissions. The production of the GPUs I run my AI on are certainly an unalieanable part of having an AI do some job.

I'm not sure the scale of 2024 models and usage was influential in that paper at all.