WTF UIAutomation? Y u no respect object equality?

August 24, 2012 | Jason Kozemczak

I’ve got good news and bad news. Mostly bad news, at least as far as I’m concerned.

An issue was recently submitted on Github for my UIAutomation framework, mechanic.js. Investigating the issue, I mostly discovered that the resolution was mainly clarifying which functions do what. Fortunately, the documentation for mechanic.js is fairly decent (though it could certainly use more concrete code samples), so a simple link pretty much cleared that up.

I also double-checked the implementation of a number of mechanic.js core functions to ensure they were behaving as designed (I even added a few new specs, hurray!). Everything appears A-okay on that front. That’s the good news.

Now, the bad news: While exploring the Github Issue, I came across what appears to be a fairly recent issue introduced into UIAutomation. From what I can tell from my poking and prodding, Apple seems to have broken element equality when comparing UIAElements. For example, if were to loop over an element’s subelements (say, in a UIATableView) and compare an element to itself, UIAutomation tells me they are not equal (a major FAIL in my mind).

To repro this, you can create a simple Single-View App in XCode, drop some elements onto the NIB, assign accessibility labels to those elements (otherwise UIAutomation won’t pick them up), and then run the following script in Instruments, you’ll see the behavior I’m talking about:

What the hell, Apple? This behavior pretty much flies in the face of how the programmers writing automation test scripts think; to that end, equality did work as expected prior to at least XCode 4.3.3. If somebody can confirm that it is also broken in XCode 4.4, I’d appreciate it (my instinct is that it is).

What this ultimately means for mechanic.js is that it’s pretty broken at this point. Most of the functions related to filtering (`filter()`, `is()`, `not()`, etc.) flat out don’t work. Additionally, quite a few of my former team’s existing automation tests are likely broken as result, mechanic.js aside. I’m going to investigate if there’s a workaround I can implement to get mechanic.js back up to working order, but that’s likely not a one-night sort of fix. I utilize referential integrity pretty significantly (like any decent programmer would) in mechanic.js.

I’ve submitted a Radar ticket to Apple for this. Additionally, I’ve opened an OpenRadar ticket that can be duped if anybody reading this post is so inclined to put some weight behind this issue.

I went ahead and created a sample project on Github that you can run to confirm the issue exists. Give it a run and tell me if I’m crazy or not!

Follow me on Twitter to get the latest updates. Tweet or DM me if you’ve got info that might be helpful; thanks!