A swarm of iOS 13 bugs lead Apple to change the way it’s testing iOS 14. Will it work?

Topic

New Reply

I bet it doesn’t. Paul Thurrot has a succinct tweet: https://twitter.com/thurrott/status/1197261094596820993 If anything, the iOS 13 buggy patch saga
[See the full post at: A swarm of iOS 13 bugs lead Apple to change the way it’s testing iOS 14. Will it work?]

Reply | Quote

Replies

Let’s face it, a progressive mindset rules the industry and conservative thinking has no place anymore. Poorly developed software is the new normal because todays developers (a.k.a script kiddies) did not grew up with the software and systems they have to work on/with. It is no surprise that Microsoft is having issues (most of their developers never used a computer running Windows and code on Apple computers), but Apple? Apparently, Apple is having diffculties to find skilled developers.

Reply | Quote

This is a pet topic of mine and has been for a long time (my doctoral work was in Software Engineering and static analysis of programs, although I claim zero practical experience).

If you’re interested, take a look at this explainer published a couple of years ago. It basically analyzes the old build-test-manage triumvirate and explains how it didn’t work at Microsoft. Then it shows how MS moved to combine the functions of development and testing:

A core principle of the Combined Engineering is elimination of tasks passing from one team to another. Handoff introduce delays and dilutes accountability. There are no specialized central teams that do certain tasks. Each feature team has end to end accountability of delivering features.

Which is certainly a noble experiment, but I would argue that it’s failing  miserably.

I don’t pretend to have a solution. All I know is that, demonstrably, the current technique (which is different from the one rolled out two years ago) isn’t working.

2 users thanked author for this post.

AlexEiffel, Noel Carboni

Reply | Quote

From the linked article:

Enter Cloud Cadence

Things changed dramatically with the arrival of the cloud cadence around mid-2000s.

Well, there’s your problem.  Here’s the solution: Exit cloud cadence.

Is this “cloud cadence” something the actual users are asking for, or is it something they simply have to tolerate?  Who asked for this?

People have been asking Microsoft about that for the whole of the WaaS era, but Microsoft doesn’t own up to making the decision either.  The “cloud cadence,” as the citation states, just arrived… it wasn’t anyone’s decision, but was just an irresistible and unquestionable force of reality, like continental drift, and Microsoft was just along for the ride, just like the Windows users themselves.

What’s so different about the “cloud cadence” era? People today want software that is stable, bug-free, and that does the job.  Prior to the “cloud cadence,” people wanted software that is stable, bug-free, and that does the job.  They wanted it right now, and they wanted it for a reasonable price.

Today, in the cloud era, people still want the same things.  No one likes to wait… that has always been the case.  Stop blaming the cloud and recognize that every time you release something, it’s your decision and your decision alone, and the quality of that release (or the lack thereof) is wholly a function of your decisions.  If you release a bad update to the world, it’s you that did it, not the cloud (that applies to Apple as well), and the difficulties of dealing with the “cloud cadence” are no excuse at all, since it was your decision to adopt the “cloud cadence” in the first place.

Dell XPS 13/9310, i5-1135G7/16GB, KDE Neon 6.2
XPG Xenia 15, i7-9750H/32GB & GTX1660ti, Kubuntu 24.04
Acer Swift Go 14, i5-1335U/16GB, Kubuntu 24.04 (and Win 11)

1 user thanked author for this post.

AlexEiffel

Reply | Quote

[Elrod]

I looked at the explainer.   A couple of points surface that are interesting.

There is a link in the article to another article about eliminating flaky tests.  These are tests that pass sometimes and fail sometimes.  Their decision was to not stop running those tests, but to remove their results from the overall reliability score.  Supposedly there are bugs automatically filed and engineers are assigned to look into them, but there seems to be doubt as to whether the problem is with the code or with the test.  So they ignore the results of those tests in their overall score.

Okay, but what if the problem really is with the code, and the problem only manifests under special conditions?  “Green” means nothing if it doesn’t reflect the true state of the system.  And if the engineers who are assigned to these problems know that, no matter what they do, the test that verifies their work is being ignored anyway, then what motivation do they have to diagnose and fix the problem?

Secondly, the end of the main article discusses the shift-left, shift-right portion of the test strategy.  This is where they made the decision to eliminate a “testing lab” kind of test strategy, and put the responsibility for end-to-end quality on the same group that designs and develops the code.  The problem with that approach is that someone who writes the code is intimately familiar with how it is “supposed” to work, i.e. exactly how the code itself is intended to be used.  That doesn’t necessarily reflect how actual people will use the code in the real world.  This is why, generally speaking, you don’t want the same people who wrote the code to test it – you want someone who is NOT intimately familiar with the code itself to design the tests according to the original specification for the system.

I understand that this was done in order to support ever-decreasing development times, but you will eventually reach a point where you are pushing so much bad software out the door so quickly that no one cares how fast you can deliver, because all of your stuff is always broken.

There needs to be a happy medium somewhere.

• This reply was modified 5 years, 6 months ago by Elrod.

2 users thanked author for this post.

AlexEiffel, Noel Carboni

Reply | Quote

[Alex5723]

What iOS 13 bugs ?
I am running iOS/iPAD beta versions for years and never had a crash, apps not responding, apps bugs…

Since when Paul Thurrot has any knowledge of Apple’s OSs ?

• This reply was modified 5 years, 6 months ago by Alex5723.

1 user thanked author for this post.

Myst

Reply | Quote

Would love to have your confidence and experience, and I’m not being sarcastic with this statement. I do believe in all your beta testing it is exactly what you say. My sister has an 8 Plus and has not had any problems with her iOS 13 updates that a hard reset couldn’t fix. What I have read when it comes to some of these reported bugs is it affects the newer iPhones – iPhone X and beyond, and not in all cases. Holding off until I feel good about going to iOS 13 from where I am right now at iOS 12.4.1 on my 8 Plus. The apps used by those reporting bugs, some major, are apps I don’t have or want, I’m pretty basic and always relieved to hear you say everything works fine with your testing. Nathan is also very much in tune and aims for the positive but lately has reported some problems with his iPhone X apps under various iOS app related subjects here at AskWoody. To be safe I’ve chosen to hold off for a bit longer and may decide to go forward after an iTunes backup in the event there’s a need to restore. @alex5723 I appreciate your reports on these updates as I do with the expertise Nathan Parker shares.

MacOS iPadOS and sometimes SOS

Reply | Quote

Alex5723 wrote:

Since when Paul Thurrot has any knowledge of Apple’s OSs ?

Since 2008:

https://www.techtalkz.com/threads/paul-thurrott-you-need-an-iphone.504496/

https://en.wikiquote.org/wiki/Paul_Thurrott

1 user thanked author for this post.

Myst

Reply | Quote

I guess we Apple folk will have to migrate to Android and Windows, and, presto, no more Apple problems — just Android and Windows problems, for which this web site has been suggesting ridiculous workarounds for years with no end in sight.

Reply | Quote

My guess is that Apple will fix whatever caused this.  We know what caused Microsoft’s issues, and they quite evidently are not concerned enough to fix the problem.  They have monopoly power, so people have to take what they get.  Apple’s iOS has a minority of the mobile market, and that percentage they do have is there because of the perception that Apple stuff “just works.”  Apple has much to lose if they let this continue, and I think they know that.

Dell XPS 13/9310, i5-1135G7/16GB, KDE Neon 6.2
XPG Xenia 15, i7-9750H/32GB & GTX1660ti, Kubuntu 24.04
Acer Swift Go 14, i5-1335U/16GB, Kubuntu 24.04 (and Win 11)

2 users thanked author for this post.

AlexEiffel, Myst

Reply | Quote

“What ever happened to dedicated testing, by a well-funded team of professionals?”

I think the answer lies in the word ‘well-funded’ and the fact that it often goes hand-in-hand with being in the ‘professional’ category. They dont want to spend the money on it.

Reply | Quote

It’s all about saving money… aka DevOps.  Layoff all the dedicated Q/A (UAT and SIT) testing teams.  And layoff all the dedicated support folks and make the developers suck up all the extra work.  All in the name of cost cutting and saving boat loads of money.  Nobody cares about quality these days because quality software products just cost too d*** much.  And new offshore developers (actually called SENIOR developers with 3 months experience) are so very cheap.

Reply | Quote

Relevancy vs. Reliability has always been an issue in preparing and auditing financial information.

Reply | Quote

[Alex5723]

Myst wrote:

What I have read when it comes to some of these reported bugs is it affects the newer iPhones – iPhone X and beyond

I use “newer” iPhone XS Max and 10.5″ iPad Pro.
I am now on iOS/iPadOS 13.3. beta 3.
Don’t remember the last year I did a hard reset on an iPhone/iPad.

• This reply was modified 5 years, 6 months ago by Alex5723.

Reply | Quote

Alex5723 wrote:

Don’t remember the last year I did a hard reset on an iPhone/iPad.

Then I guess you’re one of the more fortunate iPhone owners/beta testers. Like I said earlier, wish I had your luck.

MacOS iPadOS and sometimes SOS

Reply | Quote

Alex5723 wrote:

What iOS 13 bugs ? I am running iOS/iPAD beta versions for years and never had a crash

A sample size of one does not a study make.

Apple would not be releasing all those new versions if there was not something that needed fixing.

cheers, Paul

2 users thanked author for this post.

Myst

Reply | Quote

[Mele20]

My iPhone 10R is my first smart phone and my first Apple product. iOS 12 was fine with the exception of one problem that I didn’t think was caused by a flaky OS. Instead, I blamed my 2 bars, sometimes 1, while indoors at home (concrete walls didn’t help) and thought the weak speaker, even when turned up full force when making or receiving calls, was because of being indoors and the level was only slightly better outdoors. I had to strain to hear yet on my old Samsung non-smart phone the speaker volume was very strong even at medium level.

I lived unhappily with the problem and used my landline, on speaker, instead in my home and used iPhone for other things than phone calls. Then came iOS 13 which has been a real pain in the butt in MANY ways and each new version brought new headaches…UNTIL 13.2.3. Imagine my shock when I answered the iphone after the update and the sound on speaker BLARED SO LOUD that I almost dropped the phone! Speaker level was on maximum as it has been since I got the phone a year ago yet suddenly it was way too loud. I have no idea what 13.2.3 did but I sure am grateful that it fixed my speaker problem. I just hope that the upcoming version does not mess with this wonderful improvement!

I remain a bit shocked though that Apple appears to be following in Microsoft’s footsteps as far as testing new versions and it disappoints me as I used to think Apple was superior in every way to Microsoft (grass is greener on the other side of the fence syndrome?). However, I am seriously considering buying an Apple 5 smart watch on Black Friday as I don’t currently have a smart watch.

• This reply was modified 5 years, 6 months ago by Mele20.

Reply | Quote

Elrod up above has it right: Continuous integration and rapid fire release are at odds with “To Work”.

Not only do engineers just not get the time to get things right, but who actually _needs_ an operating system that’s a moving target? How do application developers design for an OS that keeps changing out from under them?

Don’t get me wrong – a well oiled organization firing on all cylinders can get _much better_ at continuous integration, but nothing can replace everyone getting in lock step over a period of, say, 3 YEARS _designing_, _developing_, _testing_ and _stabilizing_, THEN _releasing_.

We need all the modern development practices that are good (automated testing is essential, for example, given modern complexities) but tied to the schedules and business needs of old times (e.g., 3 year cycles). It’ll never happen though.

Nowhere in this message am I identifying any particular company or operating system. This is a conceptual issue. The high tech development world is actively steering toward things we don’t really need. I just hope we don’t forget how to feed ourselves in the process.

-Noel

2 users thanked author for this post.

AlexEiffel, Myst

Reply | Quote

There is a “why, because they can aspect” to this breakneck software updating that is unnecessarily consuming the limited productive time of users. My wife frequently complained about the constant updating of apps on her Android smartphone and I would casually reply that they were likely known bug fixes. She would get annoyed with that and in disgust say, they should wait and release well-tested versions of the apps.

Well, eventually I started to notice that the updating pace was accelerating and started to research whether I actually understood why this was the case. I was surprised to learn that some of the update cadence may be due to aggressive marketing of the apps within the app stores. The thinking goes that the update cadence keeps bringing the app to the top of the users mind like a commercial in drag. This promotes usage or trials of preloaded apps that may be the product of joint marketing agreements. At any rate, I assume that others, like myself, are noticing that more of their valuable time is being consumed by software maintenance for little discernible benefit.

1 user thanked author for this post.

Myst

Reply | Quote

Updates are off on my Android devices. If I need to update an app I will do it when I’ve got time and spare data.

cheers, Paul

Reply | Quote

[Mele20]

anonymous wrote:

At any rate, I assume that others, like myself, are noticing that more of their valuable time is being consumed by software maintenance for little discernible benefit.

Not sure I agree. iOS 13 was promised by Apple to streamline and bring much shorter app updates. It has delivered on that promise. App updates are MUCH shorter now than with iOS 12.

I don’t allow them to update automatically. I deliberately went to App Store on my iPhone 10 R yesterday and six updates were waiting (the last update I did had been two days earlier). So, I updated those. Then today, I went to the App Store again and found four new updates waiting.

So, what Apple did was basically a trade off…shorter updates but much more frequent. In retrospect, I’d rather have the old way of longer updates but not so often. Something you don’t want is to be complaining to the company that owns an app that is missing (I have that frequently…apps just disappear for no reason) or that is not up to date at the time of your complaint. For me, that means I have to check daily now for updates so I know when I complain that I have the app fully updated yet it still didn’t work correctly.

What throws me for a loop is an app that silently disappears or maybe just part of it disappears and that problem is getting worse on iOS 13.

• This reply was modified 5 years, 6 months ago by Mele20.

Reply | Quote