I’m Worried That They Put Co-Pilot in Excel
Key topics
The article discusses concerns about Microsoft's Copilot AI being integrated into Excel, potentially introducing errors and risks into financial spreadsheets, sparking debate among commenters about the reliability and impact of AI on financial processes.
Snapshot generated from the HN discussion
Discussion Activity
Very active discussionFirst comment
2h
Peak period
87
3-6h
Avg / period
16
Based on 160 loaded comments
Key moments
- 01Story posted
Nov 5, 2025 at 3:54 AM EST
2 months ago
Step 01 - 02First comment
Nov 5, 2025 at 5:55 AM EST
2h after posting
Step 02 - 03Peak activity
87 comments in 3-6h
Hottest window of the conversation
Step 03 - 04Latest activity
Nov 6, 2025 at 5:46 PM EST
2 months ago
Step 04
Generating AI Summary...
Analyzing up to 500 comments to identify key contributors and discussion patterns
Want the full context?
Jump to the original sources
Read the primary article or dive into the live Hacker News thread when you're ready.
This is true if you blame a bad vendor, or something you don’t even control like the weather. Your job is to deliver. If bad weather is the new norm, you better figure out how to build circus tents so you can do construction in the rain. If your AI call center is failing, you better hire 20 people to answer phones.
I'm not saying that this can't happen and it's not bad. Take a look at nudge theory - the UK government created an entire department and spent enormous amounts of time and money on what they thought was a free lunch - that they could just "nudge" people into doing the things they wanted. So rather than actually solving difficult problems the uk government embarked on decades of pseudo-intellectual self agrandizement. The entire basis of that decades long debacle was based on bullshit data and fake studies. We didn't need AI to fuck it up, we managed it perfectly well by ourselves.
It was taken up by the UK government at that time because the government was, unusually, a coalition of two quite different parties, and thus found it hard to agree to actually use the normal levers of power.
This NY Times opinion piece by Loewenstein and Ubel makes some good arguments along these lines: https://web.archive.org/web/20250906130827/https://www.nytim...
(I pulled the quote by using yt-dlp to grab the MP4 and then running that through MacWhisper to generate a transcript.)
I actually transcribed the whole TikTok which was about 50% longer than what I quoted, then edited it down to the best illustrative quote.
Especially since typing a transcript still involves checking that you got it right!
So in summary I think it was just part of automated process (maybe) or it will become one in the future.
Is MacWhisper a $60 GUI for a Python script that just runs the model?
Yes, a large genre of MacOS apps are "Native GUI wrappers around OSS scripts"
Which is incredibly value. The OSS script has zero value to someone who doesn't know it exists or doesn't understand how to run it.
The script ran in a machine located at the corner of a cubicle and only one employee had the admin password. Nobody but a handful of people knew of the machine's existence, certainly not anyone in middle management and above. The script could only be updated by an admin.
Copilot may be good, but sure as hell doesn't know that admin password.
At one of my jobs we had a server rack with UPS, etc, all the usual business. On the floor next to it was a dell desktop with a piece of paper on it that said “do not turn off”. It had our source control server in it, and the power button didn’t work. We did eventually move it to something more sensible but we had that for a long time
But we didn't (and nobody was hit by a bus)
Yes, most situations are terrible compared to what would be if an expert was present to perfect it.
Except if there isn’t an expert, and there’s a normal person, how do they know the output is right ?
The mess already existed for a reason. There’s a certain amount of expertise in the average firm.
If they could afford an expert, they wouldn’t be the same firm.
If they do get an AI expert - how do they check the output given the level of ability they have?
This is an ironclad argument against fully replacing employees with AI.
Every single organization on Earth requires the people who were part of creating the current mess to be involved in keeping the organization functioning.
Yes you can improve the current mess. But it's still just a slightly better mess and you still need some of the people around who have been part of creating the new mess.
Just run a thought experiment: every employee in a corporation mysteriously disappear from the face of the Earth. If you bring in an equal number of equally talented people the next day to run it, but with no experience with the current processes of the corporation, how long will it take to get to the same capability of the previous employees?
Looking at the web interface, I can tell it’s still running, doing its thing. I’m sure its still running Linux from 2008.
The problem is that people ignore them.
For a more serious example, consider the Paperclip Problem[0] for a very smart system that destroys the world due to very dumb behaviour.
[0]: https://cepr.org/voxeu/columns/ai-and-paperclip-problem
But let's consider real life intelligence:
- Our super geniuses do not take over the world. It is the generationally wealthy who do.
- Super geniuses also have a tendency to be terribly neurotic, if not downright mentally ill. They can have trouble functioning in society.
- There is no thought here about different kinds of intelligence and the roles they play. It is assumed there is only one kind, and AI will have it in the extreme.
None of us knows what an actual, artificial intelligence really looks like. I find it hard to draw conclusions from observing human super geniuses, when their minds may have next to nothing in common with the AI. Entirely different constraints might apply to them—or none at all.
Having said all that, I'm pretty sceptical of an AI takeover doomsday scenario, especially if we're talking about LLMs. They may turn out to be good text generators, but not the road to AGI. But it's very hard to make accurate predictions in either direction.
I'm pretty sure there are already humans who do this. Perhaps there are even entire conferences where the majority of people do this.
Reading it, what I understood was: ah, so Brenda is safe - co-pilot will screw up, she will point it out and management will learn not to trust the bot.
And what I believe OP intended is "Brenda will lose her job!"
What I think I mean is: yes, both are valid, but conflating both in a single anecdote sounds bad.
Are you a fanatic that thinks anyone saying that there are any limitations to current models = nay-sayer?
Like if someone says they wouldnt wanna get a heart transplant operation done purely by GPT5, are they a nay-sayer or is that just reflecting reality?
I don't have the slightest idea who "Simon" is and I'm taking the post at face value: it contradicts itself, and that's a bad argument.
Just think about it... In this scenario, management screws up a formula through AI... Which, at least st this point, will surface at some point - not all of us are math ignorant - so what happens is Brenda gets her position back and upper management loses trust in AI. That's the likely outcome; the Brenda's of the world will suffer until upper management realizes their mistake, but end of the day, they have _not_ lost their value, given the post says that AI screws up.
But this is clearly not the conclusion the author ("Simon") intends - they believe AI will erode Brenda value.
That's what I'm saying - AI can't be both incapable and a job menace. For it to threaten jobs like Brenda's, it need to be very capable.
And sorry but "heart transplant" by a transformer model is laughable. Writing formulas, on the other hand, isn't.
I've been quite entertained at how quoting a (very funny) joke on TikTok has resulted in a 400+ comment thread on Hacker News.
Another issue is that my org disallows AI transcription bots. It’s a legit security risk if you have some random process recording confidential info because the person was too busy to attend the meeting and take notes themselves. Or possibly they just shirk off the meetings and have AI sit in.
Is it because of a globally trained model (as opposed to trained[tweaked on] on context specific data) or because of using different classes of models.
It could be they simply use a mediocre transcription model. Wispr is amazing but would hurt their pride to use a competitor.
But i feel it's more likley the experience is; GPT didn't actually improve on the raw transcription, just made it worse. Especially as any miss-transcipted words may trip it up and make it misunderstand while making the summary.
if i can choose between a potentially confused and misunderstood summary, and a badly spellchecked (flipped words) raw transcription, i would trust the latter.
I am all good for nice completion on VS, or help decypher compiler errors, but lets do this AI push with some contention.
Also what I really deslike is the prompt interface, AI integrations have to feel natural transparent part of the workflow, not trying to put everything into a tiny chat window.
And while we're at it, can we please improve voice reckognition?
this has been the microsoft business model for 40 years
“There are two Brendas - their job is to make spreadsheets in the Finance department. Well, not quite - they add the months and categories to empty spreadsheets, then they ask the other departments to fill in their sales numbers every month so it can be presented to management.
“The two Brendas don’t seem to talk, otherwise they would realize that they’re both asking everyone for the same information, twice. And they’re so focused on their little spreadsheet worlds that neither sees enough of the bigger picture to say, ‘Wait… couldn’t we just automate this so we don’t need to do this song and dance every month? Then we wouldn’t need two people in different parts of the company compiling the same data manually.’
“But that’s not what Brenda was hired for. She’s a spreadsheet person, not a process fixer. She just makes the spreadsheets.”
We need fewer Brendas, and more people who can automate away the need for them.
What would this be replaced by? Some kind of large SAP like system that costs millions of dollars and requires a dozen IT staff to maintain?
So one good BI developer who knows Tableau and Salesforce and Excel and SQL can replace those pure collection points with a better process, but they can also generate insight into the data because they have some business understanding from being close to the teams, which is what my hypothetical Brenda can’t do.
In my example, Brenda would be asking sales leaders to enter in their data instead of going into Salesforce herself because she doesn’t know that tool / side of the company well enough.
I was making the point that, contrary to the article, the Brendas I know aren’t touched by the Excel angels, they’re just maintaining spreadsheets that we probably shouldn’t have anyway.
A hill I will die on is that business analytics need "view source" or they aren't worth the pixels they are rendered with.
At my last large employer I genuinely lost count of the number of times I saw a BI report which pulled numbers from our data warehouse... and then found out it had misinterpreted a key detail because the engineering team had changed some table design six months ago and the data analysis team hadn't been told about the change.
Then you end up with a report that goes out automatically every month to leadership pulled directly from the Salesforce data, along with a real time dashboard anyone in the org can look at, broken down by team, vertical, and sales volume.
Why are people so attached to manual process?
Are you suggesting that Brenda should stay in her box?
She should replaced with someone who says, “this box doesn’t need to be here… there is a better way of doing things.”
NOT to be confused with the junior engineer who comes into a project and says it’s garbage and suggests we rewrite it from scratch in ${hotLanguage} because they saw it on a blog somewhere.
At large companies in particular, there are far too many people who simply turn their widgets - this was the entire point of the tech revolution.
Think about how many bookkeepers were needed before Excel. Someone could have made your exact same argument (but it’s just the latest gimmick!) about Excel 30 years ago. And yet, technology will make businesses more efficient whether people stand in its way or not.
Even at a small company of one or two, QuickBooks will reduce the amount of bookkeepers and accountants needed. TurboTax will further reduce that.
We will need fewer people in the future maintaining their Excel spreadsheets, and more people building the automation for those processes.
The change averse will always find reasons not to adapt - they will create their own obsolescence.
(inb4 but it’s way more expensive to pay developers to automate!)
Currently I'd put it worse than tearing things up for ${hotLanguage} because at least ${hotLanguage} is deterministic and debuggable.
Honestly, I'm not sure why you're going to the mat for AI in spreadsheets, or why you think it's a good use case, or why you seem to think "automation" doesn't come with overhead of its own. Current iterations of AI are recommendation engines. Even then you better have version control.
The article is about this kind of Brenda.
If you do your job, you get paid periodically. If you automate your job, you get paid once for automating it and then nothing, despite your automation constantly producing value for the company.
To fix this, we need to pay people continually for their past work as long as it keeps producing value.
If you don’t automate it:
1a) your company keeps you hanging on forever maintaining the same widget until the end of time
OR
1b) more likely, someone realizes your job should be automated and lays you off at some point down the road
If you do automate it
2a) your company thanks you then fires you
OR
2b) you are now assigned to automate more stuff as you’ve proven that you are more valuable to the company than just maintaining your widget
————
2b is really the safest long term position for any employee, I think. It’s not always foolproof, as 2a can happen.
But I’d rather be in box 2 than box 1 any day of the week if we’re talking long term employment potential.
When automation produces value for the company, the people automating it should capture a chunk of that value _as a matter of course_.
Even if you argue that you can then negotiate better compensation:
1) That is uncertain and delayed reward - and only if other people feel like it, it's not automatic.
2) The reward stops if you get fired or leave, despite the automation still producing value - you are also basically incentivized to build stuff that requires constant maintenance. Imagine you spend a man-month building the automation and then leave, it then requires a man-month of maintenance over the next 5 years. At the end of the 5 years, you should still be getting 50% of the reward.
What would that look like in practice?
That being said, it's clear that in the current system, rich people can get richer faster than poor people.
We have a two class system a) workers who get paid per unit of work b) owners who capture any surplus income, who decide hiring/firing/salaries, who can sell the company and whose wealth keeps increasing (assuming the company does well) whether they do any work themselves.
Note: I see very few things which have inherent value - natural resources (plus land?) and human time. Everything else (with monetary value) is built from natural resources using human time.
---
If a company starts with 1 guy in a shed, he does 100% of the work, owns 100% of the company and ... it gets muddy here ... gets 100% of the income / decides where 100% of the revenue goes - if it's a grocery shop he can just pocket any surplus, if he's making stuff, he'll probably reinvest into better tooling or to hire more workers.
A year later, he hires 9 workers. Now he does only 10% but still owns 100% of the company.[0]
There's a couple issues here:
- He owns 100% of the future value of the company despite being created only 10% by him. Well, not exactly, he was creating 100% for the first year and 10% from then on.
- He still gets to decide who gets paid what. He has more information when negotiating.
- He can sell the company to whoever and the workers have no say in it. He can pass it on to his children (who performed 0 work there) when he dies.
The solution I'd like to see tested is ownership being automatically and periodically (each month) redistributed according to the amount and skill level of work performed.[1]
So at the end of year 2, the original founder has done 2 man-years of work, while the other 9 people have done 1 man-year of work each. This means the founder owns 2/11ths of the company while everyone else owns 1/11th. This could further be skewed by skill levels. I am sure starting and running a company for a year takes more skill than doing only some tasks. OTOH there are specialized tasks which only very few people can perform and the founder is not one of them.
The skill level involved would be part of the negotiations about compensation.
---
This is complex. I am sure somebody is prone to rejecting it based solely on that. But open a wiki page about e.g. bonds[2] and see how many blue words just the initial sentence has and ask yourself whether you could explain all of them (and then transitively all the linked concepts on their wiki pages).
Slavery is very simple but very unfair. Employment is more complex and less unfair. I have a theory that the more fair a system is, the more complex it is because it needs to capture more nuances of the real world.
---
[0]: Some people think this is right because owners take all the risk and employees take 0 risk. That is misrepresenting what really happens - sane investors/owners don't risk losing so much they would go homeless/starve if they lose it all. They can also optimize their risk by spreading it across many companies. Meanwhile workers get 100% of their income from one company and drop down to no income if the company goes bankrupt. They can also be fired at any time.
This was argued here: https://news.ycombinator.com/item?id=45731811 in the comment by kristov and the reply by me. I also have other comments there with relevant ideas.
[1]: What happens to monetary compensation? I don't know, I see multiple options:
a) Everybody gets paid monetary wages like today, plus (newly) a part of their reward is the growing share of the company they own. If we allow selling it to anyone, it has high monetary value but then ownership gets diluted to outside investors. If we allow selling it only back to the company, it has value only relative to the decision-making power it gave. If we don't allow selling it, its monetary value only comes from the ability to vote on dividends.
b) Everybody gets paid a portion of the income divided according to their share. This sounds simple but likely wouldn't give enough money to newly joined workers to survive. There could be a floor. (Or, because hard cutoffs suck, a smooth mathematical function from owned percentage to monthly compensation which would have a floor at minimum wage.)
[2]: https://en.wikipedia.org/wiki/Bond_(finance)
> - He owns 100% of the future value of the company despite being created only 10% by him. Well, not exactly, he was creating 100% for the first year and 10% from then on.
1) If you believe this, then you have a massively simplistic view of employee value. The distribution of actual value provided by employees is probably log normal, and certainly not normal (gaussian).
2) This is basically the labor theory of value. That is an economic theory that was discarded as wrong about 150 years ago. If it was true, the value of a newly discovered gold mine would be 0.
That's why I talk about skill levels later, but briefly because this is a comment, not a book.
There's also a difference between how much value is provided can be attributed to a particular person vs a particular position. Some positions allow a much wider range of possible outcomes. How much extra wealth does a 90th percentile carpenter produce over and average carpenter? What about programmer, fashion designer, manager, salesman, doctor?
Does this mean the value of life of different people is based on how productive they are?
Because each person has roughly the same amount of time available to them and if they are spending an equal amount of it building a company, does one deserve to own more of it? Should this distribution be the same or different from the (monthly) monetary compensation?
These are rhetorical questions (mostly) but they are questions society should be discussing IMO.
Tangent: a carpenter who has no salesman and is so shit at selling his furniture that he gives it away for free is still producing value for society, even if he goes broke doing it. OTOH a salesman who has no carpenter and is so shit at making furniture that nobody wants it even for free is not producing any value at all.
> This is basically the labor theory of value
Ok, I need to read up on LVT. Seems like I am finally getting somewhere because I can't believe I am the only one saying things like this but at the same time I have not found anybody else with similar opinions. At best, I've seen people try to pattern match my opinions onto something similar they were familiar with but actually different.
> If it was true, the value of a newly discovered gold mine would be 0.
I don't know how this results from LVT yet but it seems what I am proposing must then be fundamentally different depending on how you meant it:
a) You meant gold as a natural resource with inherent value. It is necessary for making e.g. some electronics. The only question remains how to distribute the reward for discovering and mining it.
b) You meant gold as a substitute for money, ignoring its value as a natural resource. In that case, yes, money is a medium of exchange, you can't eat it or make anything out of it (maybe a fire?). Having more money in circulation does not bring any extra value for society, it just multiplies all monetary values by a number slightly larger than 1. (OTOH for the person discovering and mining it, it would be beneficial but only because he now has more relative to others. The same way as if he printed more money.)
- i used to work on small jobs younger, as a nerd, i could use software better than legacy employees, during the 3 months, i found their tools were scriptable so I did just that. I made 10x more with 2x less mental effort (I just "copilot" my script before it commits actual changes) all that for min wage. and i was happy like a puppy, being free to race as far as i want it to be, designing the script to fit exactly the needs of an operator.
- later i became a legit software engineer, i'm now paid a lot all things considered, to talk to the manager of legacy employees like the above, to produce some mediocre web app that will never match employees need because of all the middle layers and cost-pressure, which also means i'm tired because i'm not free to improve things and i have to obey the customer ...so for 6x more money you get a lot less (if you deliver, sometimes projects get canned before shipping)
It's not about how much I get paid. It's about realizing how much of the value I produce goes to me and how much goes to the owner class.
At least I never worked in a big corporation and I always had the ability to do work that directly benefited people using my code. But I still saw too much of the "I built this company" self-congratulatory BS from people who just shuffled money while doing 0 actual work.
I don't think ownership is theft, I just think it's distributed wrongly - to people who have money instead of to people who do work. See my other comment here: https://news.ycombinator.com/item?id=45826823
there's a blend of "i'm my own man": i get the money and handle the responsibility on my own and it's thrilling feeling
i don't dimiss the layers of HR managing legal and financial duties in a company and thus taking a cut, but there's a kind of pleasure to also do your own business for a while
I don't wanna dismiss them either but (along with management):
- It's not positive-sum work. It doesn't produce positive value for society, it's just necessary work which needs to be done as a side effect of actual positive-sum work being done.
- The pyramid should be inverted. Managers, layers, accountants, etc. should be assistants. The people doing the actual work should (collectively) decide to hire them when they think it would make them more productive or be otherwise beneficial to them. Not the other way around.
It is always in my self interest to automate my job as much as possible. Nothing looks better for moving up than this. Even more so, nothing makes me happier than automating a business process.
There are always so many various road blocks to automation it is hard to count.
It is like there is a type of entropy that increases over time that people are largely getting paid to keep at bay with simple business processes that can be easily adapted as things change. So often automation works great for a short time until this entropy breaks the automation. It doesn't take that many times for management to figure out the investment in automation gives poor returns.
That said, every finance function is different and it may be unknown to them that you’re being asked for some data multiple times. If you’re enduring this process, I’m of the opinion you’re equally at fault. Suggest a solution that will be easier on you. As it’s possible they don’t even know it’s happening. In the case provided, email to all relevant finance people “Here’s a link to a shared workbook. I’ll drop the numbers here monthly, please save the link and get the data directly from that file. Thanks!” Problem solved. Until you don’t follow through which is what causes most finance people to be constantly asking for data/things. So be kind and also set yourself a monthly recurring reminder on your calendar and actually follow through.
But Usually finance is always preferring on demand access so the communication feedback loop of asking for stuff is not well liked so I’m sure they appreciate this middle step too.
There are many cases where there’s no easy way to give access to the data and a human in the loop is required. In that case, do the shared workbook thing I mentioned as a starting point at least. It may evolve from there.
Only different companies were all sold different enterprise finance products, but they need to communicate with each other (or themselves after mergers), so it all gets manually copied into Excel and emailed around each month.
And now if one of the Brendas wants to change their process slightly, add some more info, they can't just do it anymore. They have to have a three way discussion with the other Brenda, the automation guy and maybe a few managers. It will take months. So then its likely better for Brenda to just go back to using her spreadsheet again, and then you've got an automated process that no longer meets peoples needs and will be a faff to update.
Do you have anything to say other than, “I don’t need to hear what you have to say”?
> op (as legacy business): BAU
> you (as tech): disrupt! disrupt! disrupt!
> me: no thank you; that's not necessary
> you (as tech): stop being mean!
Not wanting your "disruption" is not being un-nice. Your disruption was not asked for in the first place. Forcing it (Uber, Doge, et. al.) on marketplaces, often illegally, and vacuuming it up the income ladder to the already-wealthy IS the "not nice" thing.
You just see me as a target to displace that onto. I'm the representative for what you believe is wrong with tech.
I see your hot take as emblematic of those issues. Why would you think any internet comment is about you?
We need more Brendas (those who excel goddesses come and kiss on the forehead) and need less people who are disrespectful of Brendas. The example in this post is someone giving more respect to AI than Brenda.
At least half of the work in my senior Finance team involves meeting people in operations to find out what they are planning to do and to analyse the effects, and present them to decision makers to help them understand the consequences of decisions. For an AI to help, someone would have to trigger those conversations in the first place and ask the right questions.
The rest of the work involves tidying up all the exceptions that the automation failed on.
Meanwhile copilot in Excel can't even edit the sheet you are working on. If you say to it, 'give me a template for an expense claim' it will give you a sheet to download... probably with #REF written in where the answers should be.
True... I have an on-staff data engineer for the purpose. But not all companies (especially in the SMB space) have that luxury.
I suppose the person that wrote that have not ideia Excel is just an app builder where you embed data together with code.
You know that we have excel because computers didn’t understand column names in databases and so data extraction needed to be made by humans. Humans then design those little apps in excel to massage the data.
Well, now an agent can read the boss saying gimme the sales from last month and the agent don’t need excel for that, because it can query the database itself, massage the data itself using python and present the data itself with html or PNGs.
So, we are in the process of automating Brenda AND excel away.
Also, finance departments are a very small part of excel users. Just think everywhere were people need small programs, excel is there.
If we have to compare LLM’s against people who are bad at their jobs in order to highlight their utility we’re going the wrong direction.
Excel - whatever its origin story - is the actual Swiss Army knife of the tech world.
There’s easily a few billion people who use excel. There is a reason it survives.
Financial statements are correct because of auditors who check the numbers.
If you have a good audit process then errors get detected even if AI helped introduce them. If you aren't doing a good audit then I suspect nobody cares whether your financial statement is correct (anyone who did would insist on an audit).
Volume matters. The single largest problem I run into: AI can generate slop faster than anyone can evaluate it.
1. We advocate automation because people like Brenda are error-prone and machines are perfect.
2. We disavow AI because people like Brenda are perfect and the machine is error-prone.
These aren't contradictions because we only advocate for automation in limited contexts: when the task is understandable, the execution is reliable, the process is observable, and the endeavour tedious. The complexity of the task isn't a factor - it's complex to generate correct machine code, but we trust compilers to do it all the time.
In a nutshell, we seem to be fine with automation if we can have a mental model of what it does and how it does it in a way that saves humans effort.
So, then - why don't people embrace AI with thinking mode as an acceptable form of automation? Can't the C-suite in this case follow its thought process and step in when it messes up?
I think people still find AI repugnant in that case. There's still a sense of "I don't know why you did this and it scares me", despite the debuggability, and it comes from the autonomy without guardrails. People want to be able to stop bad things before they happen, but with AI you often only seem to do so after the fact.
Narrow AI, AI with guardrails, AI with multiple safety redundancies - these don't elicit the same reaction. They seem to be valid, acceptable forms of automation. Perhaps that's what the ecosystem will eventually tend to, hopefully.
No, no. We disavow AI because our great leaders inexplicably trust it more than Brenda.
Then I think managers would be fine hiring that worker for that rate as well.
That is precisely why we have humans in the loop for so many AI applications.
If [AI + human reviewer to correct it] is some multiple more efficient than [human alone], there is still plenty of value.
I disagree. If something can't be as accurate as a (good) human, then it's useless to me. I'll just ask the human instead, because I know that the human is going to be worth listening to.
Good in most conditions. Not as good as a human. Which is why we still have skilled pilots flying planes, assisted by autopilot.
We don’t say “it’s not as good as a human, so stuff it.”
We say, “it’s great in most conditions. And humans are trained how to leverage it effectively and trained to fly when it cannot be used.”
This can still be problematic! If sensors are feeding the autopilot bad data, the autopilot may do the wrong thing for a situation. Likewise, if the pilot(s) do not understand the autopilot's behaviors, they may misuse the autopilot, or take actions that interfere with the autopilot's operation.
Generative AI has unpredictable results. You cannot make confident statements like "if inputs X, Y, and Z are at these values, the system will always produce this set of outputs".
In the very short timeline of reacting to a critical mid-flight situation, confidence in the behavior of the systems is critical. A lot of plane crashes have "the pilot didn't understand what the automation was doing" as a significant contributing factor. We get enough of that from lack of training, differences between aircraft manufacturers, and plain old human fallibility. We don't need to introduce a randomized source of opportunities for the pilots to not understand what the automation is doing.
It started out as, "AI can make more errors than a human. Therefore, it is not useful to humans." Which I disagreed with.
But now it seems like the argument is, "AI is not useful to humans because its output is non-deterministic?" Is that an accurate representation of what you're saying?
Remember "garbage in, garbage out"? We expect technology systems to generate expected outputs in response to inputs. With generative AI, you can get a garbage output regardless of the input quality.
Aviation autopilot systems are the complete opposite. They are arguably the most reliable computer-based systems ever created. While they cannot fly a plane alone, pilots can trust them blindly to do specific, known tasks consistently well in over 99.99999% of cases, and provide clear diagnostics in case they cannot.
If gen AI agents were this consistently good at anything, this discussion would not be happening.
(More seriously, she also has 20+ years of institutional knowledge about how the company works, none of which has ever been captured anywhere else.)
You don't have a human to manage. The relationship is completely one-sided, you can query a generative AI at 3 in the morning on new years eve. This entity has no emotions to manage and no own interests.
There's cost.
There's an implicit promise of improvement over time.
There's an the domain of expertise being inhumanly wide. You can ask about cookies right now, then about XII century France, then about biochemistry.
The fact that an average worker would be fired if they perform the same way is what the human actually competes with. They have responsibility, which is not something AI can offer. If it was the case that, say, Anthropic, actually signed contracts stating that they are liable for any mistakes, then humans would be absolutely toast.
So now you don't have to pay people to do their actual work, you assign the work to ML ("AI") and then pay the people to check what it generated. That's a very different task, menial and boring, but if it produces more value for the same amount of input money, then it's economical to do so.
And since checking the output is often a lower skilled job, you can even pay the people less, pocketing more as an owner.
The CEO has been itching to fire this person and nuke her department forever. She hasn't gotten the hint with the low pay or long hours, but now Copilot creates exactly the opening the CEO has been looking for.
174 more comments available on Hacker News