Microsoft's Lack of Quality Control Is Out of Control

There's an opportunity to automate some of the QA traditionally done manually. I tried this last week on our main app (not some toy thing):

(turn on agent mode in chat gpt)

  "Hey put on your QA hat and test <url> with <user> and password. Give me detailed feedback on UX, bugs, etc."

I got a pretty detailed report back and most of it was valid/constructive. I'm planning to do more of this. It beats developers doing QA (they are too biased usually and seem reluctant to do it) and we don't have any dedicated QA people. Manual QA can be very expensive. I don't think the need for that totally goes away, but you should probably focus that on the most valuable/hardest things to test.

In any case, it's pretty cool to watch Chat GPT explore a UI and attempting to use features. It's very thorough and it seems to figure out workarounds for UX issues as well when things don't work as expected. This is exactly the kind of stuff that developers testing their own UIs are blind to. They know where to click and don't even think about it.

A related issue is actually updating documentation and marketing material with up-to date screen captures and screenshots. Annoyingly, Chat GPT doesn't allow me to save the videos it takes of the AI using the browser. But that stuff could actually be documentation gold. Doing this manually is very tedious.

Why quoting Robin Williams is relevant here?

He seems some kind of actor. Are you quoting his fantasy how Bill Gates thinks?

In the same way that their incompetence has been very slow to move the needle, once they lose the market it’s going to be almost impossible to get it back.

I happen to use Windows on both personal and work laptop. Some of the bugs I see exist across Home and Enterprise version. Sleep remains a nightmare on Windows, and yes across laptops made by different manufacturers. I have created tickets and this, and IT doesn't have a solution.

I have decided that my next personal laptop definitely won't run Windows, and if I am allowed to ask for a Mac machine at work in the future, I'll jump at that opportunity.

That would mean two fewer Windows licenses and less usage of related products (good riddance, Edge!). And I am sure I am not the only one who is thinking about all this.

But of course I have no idea if that matters in the grand scheme of things -- after all, many people tolerate these bugs just like they tolerate all the ads by Microsoft, Google, Meta etc.

But I think it's the worst aspect of the subscription model. In the past, people just wouldn't buy the new version if it sucked

But it is a beta version at best for years. There are two version of editors that frequently break. You cannot overtax the system or all running instances morph into endless running tasks. Flows regularly break if you want to update something and the flow was created in a version of the editor no longer available.

You can also see their attempts to monetize this test version that needs a powerhouse of a machine to run its editor in a browser.

I like it for what it is if you are already (trapped?) in a Microsoft environment. There is some potential here and few consultants will tell you that it basically isn't production ready. But the product manager should be quite ashamed.

Also I heavily doubt it will stay free, so perhaps plan infrastructure accordingly.

Remember Steve Balmer chanting "Developers, developers, developers" (in about 2000)? That's why.

I'm not saying I totally agree (although I think I do at least a bit), just that this is hardly new.

My instinct is that there is some general principle that relates “friction” and “quality”, although I’m not sure I have the vocabulary to describe it.

I.e. where there is a barrier to entry, quality of results tends to improve.

I also see this in ease of publishing to social media, bias of “old music is better” (time has sorted wheat from chaff) and so on.

Perhaps there’s a well known description of this phenomenon somewhere already…

I think that depended (and still depends) a lot on the organization and the nature of the product.

I distinctly remember doing backend and some frontend development, requirements specification, database design, customer interfacing and even a bit of ops, all on the same job and with the same title in the 00's. That was in a small-to-medium company and my clients were on the small side so the projects might not have even had half a dozen people to begin with.

Larger organizations and more enterprisey projects would have had more specialized and limited roles: customer/specs people, possibly frontend and backend devs, DBAs, testing people, and those in charge of ops and environments. In my experience, that's still more or less true in enterprisey development today.

I think a part of the problem is that while new technologies have emerged and reduced the need to manually work with some older or underlying technologies, they haven't replaced previous skills.

Containers have reduced the amount of work needed to deal with deployments and environments but they haven't removed the need to know servers or operating systems. Cluster management can reduce the amount of manual work on setting up containers but it doesn't remove the need to know the underlying container engine. So now you need to know Linux servers and containers and k8s and whatnot just in order to manage a local backend development setup. At the same time, frameworks have made a lot of frontend work more manageable but they haven't made JavaScript or other underlying stuff disappear.

Thus the scope of what being a fully-versed full-stack developer entails has grown.

I don’t know a single specialized camera operator anymore. Literally every shooter I know is also a competent editor, which I do think is neat and makes us better Cam Ops, but it also means people expect everyone to shoot and edit. Also capture excellent sound. Don’t forget the set has to look good. Make sure you’ve got a good Rolodex of locations ready to go as well.

We don’t need to keep every individual role just because it’s traditionally been there, but in a lot of industries we’ve clearly gone the wrong direction. And with something like QA/QC I could see that being a huge problem because the payoff is not obvious so upper management is going to want you to get something out the door no matter what state it is in.

The software dev supply market is absolutely saturated.

Only biggest companies were able to have that. If you have a single application to run there is no work for DBA as FTE, in big company where you have multiple projects you can most likely have DBA as a department that handles dozens of databases and running infra. Same with Ops, you can have SRE or OPS doing your infra if you have dozens of applications to run.

Problem is having separate QA/DBA/Dev/Ops departments was breaking because people would "do their stuff" and throw problems over the fence. So everything would go to shit and we have seen it in big companies.

Other thing is - I have read about multiple companies trying "to be professional" burning money on exactly having separate roles, but in reality you cannot simply afford FTE or having full department of DBA or QA or OPS or just Dev - unless you basically are swimming in money.

Are we there yet?

And their vacuum cleaner. /s

Furthermore, with modern MBAs at the helm, the concern is only pumping figures for the next quarter - five years be damned.

It's also completely possible to do agile with QA. One place I worked had 1 QA per 3 devs, and they were able to maintain a single (2 week) sprint behind cadence.

Scrum was always how Agile was sold to behemoths. It necessarily makes several compromises that deeply undermine it.

also, qa is simply not optional, and agile actually stresses that pretty strongly.

The issue with “quality” is that it's really subjective. As someone remembering the switch from Windows 98 (DOS-based) to Windows 2000 (NT-based), the boost in subjective quality was immense. But to someone who's already been on Linux for years, it would have looked like playing catch-up.

I think they went too Moneyball and figured telemetry and metrics could solve everything. McNamara fallacy and all that

Maybe they should read bug reports posted by the end users, and not have half-baked solutions posted by Very Ignorant Persons.

This is true since at least Win 95. One usually needed to wait until SP2 to get a resemblance of quality from Microsoft.

Now, since Vista, they got rid of ServicePacks. This says a lot about their quality culture.

Google absolutely sucks ass at this: You can't set your default language without having a Google account which I refuse to sign in except on YouTube, and there's no easy way to change the language on some of their pages.