E-mail List Archives
Thread: Measuring colours accurately in mobile apps
Number of posts in this thread: 12 (In chronological order)
From: Steve Green
Date: Tue, Jun 22 2021 11:02PM
Subject: Measuring colours accurately in mobile apps
No previous message | Next message →
Does anyone have a method for accurately measuring the text and background colours in mobile apps in order to make colour contrast measurements? For context, we are independent testers, so we never have access to the development environment. Sorry this is a bit long, but it's perhaps the most challenging test process issue we currently have.
Aside from an accessibility scanner app for Android, I am not aware of any tools that can make the measurements on a mobile device. All the methods we have tried involve making the measurements on a desktop machine, which means transferring an image of the device screen.
We have done extensive testing and found that every method we can think of is wildly inaccurate. Such methods include:
* Taking screenshots and emailing them as attachments.
* Taking screenshots and emailing them as embedded images.
* Taking screenshots and transferring them through the file system.
* Screen mirroring using Mobizen and Lonely Screen.
* Setting up a Zoom call between the mobile device and the desktop machine.
The key findings are:
* For any given colour in the app, each method of measurement gives a different result. The differences can be massive. None are close enough to be useful.
* For any given method of measurement, some colours might be measured fairly accurately, but the measurement of other colours is hopelessly inaccurate.
The accessibility scanner app for Android is useful, but only up to a point. It reports the colour values in combinations that do not have sufficient contrast, which is good. However, it does not report the colour values in combinations that pass. That won't bother a lot of people, but our process requires all combinations to be measured and recorded as proof of what the colours were on that day and proof that all combinations were tested.
Also, we don't have any solution for iOS and there is the issue of measuring non-text contrast.
As far as I can tell, the errors are due to compression during the mirroring or streaming process and the different colour spaces on different devices. It is also possible that the screenshot application on mobile devices introduces errors. Images may also be degraded when embedded in emails sent from mobile devices. As such, making the measurement on the mobile device would seem to be best if that's possible.
Are we the only people who don't know how to do this? What does everyone else do?
Is it possible to download an app from the App Store or Play Store and somehow pull it into XCode or another development environment in order to see all the colours? Or are there any decompilers, like we used to use to pull apart Flash movies?
Regards,
Steve Green
Managing Director
Test Partners Ltd
020 3002 4176 (direct)
0800 612 2780 (switchboard)
07957 246 276 (mobile)
020 7692 5517 (fax)
Skype: testpartners
= EMAIL ADDRESS REMOVED =
www.testpartners.co.uk
Â
Connect to me on LinkedIn - http://uk.linkedin.com/in/stevegreen2
From: Glen Walker
Date: Wed, Jun 23 2021 12:39AM
Subject: Re: Measuring colours accurately in mobile apps
← Previous message | Next message →
A while back I had tried the "Color Contrast" app but wasn't overly impressed.
https://apps.apple.com/us/app/color-contrast/id1095478187
It still relies on a screenshot or loading a url (which doesn't help with a native app). I haven't tried running two apps on a newer iPad to see if that helps.
Although tedious, I haven't had too much trouble with a screenshot and emailing the image. I don't see any color degradation and even if the color isn't exactly right, it's usually close enough. Unless I'm near 4.49:1, having a color that is close to the original will still either fail when the exact color fails or pass when the exact color passes.
Sent from my iPhone
> On Jun 22, 2021, at 11:02 PM, Steve Green < = EMAIL ADDRESS REMOVED = > wrote:
>
> Does anyone have a method for accurately measuring the text and background colours in mobile apps in order to make colour contrast measurements? For context, we are independent testers, so we never have access to the development environment. Sorry this is a bit long, but it's perhaps the most challenging test process issue we currently have.
>
> Aside from an accessibility scanner app for Android, I am not aware of any tools that can make the measurements on a mobile device. All the methods we have tried involve making the measurements on a desktop machine, which means transferring an image of the device screen.
>
> We have done extensive testing and found that every method we can think of is wildly inaccurate. Such methods include:
>
> * Taking screenshots and emailing them as attachments.
> * Taking screenshots and emailing them as embedded images.
> * Taking screenshots and transferring them through the file system.
> * Screen mirroring using Mobizen and Lonely Screen.
> * Setting up a Zoom call between the mobile device and the desktop machine.
>
> The key findings are:
> * For any given colour in the app, each method of measurement gives a different result. The differences can be massive. None are close enough to be useful.
> * For any given method of measurement, some colours might be measured fairly accurately, but the measurement of other colours is hopelessly inaccurate.
>
> The accessibility scanner app for Android is useful, but only up to a point. It reports the colour values in combinations that do not have sufficient contrast, which is good. However, it does not report the colour values in combinations that pass. That won't bother a lot of people, but our process requires all combinations to be measured and recorded as proof of what the colours were on that day and proof that all combinations were tested.
>
> Also, we don't have any solution for iOS and there is the issue of measuring non-text contrast.
>
> As far as I can tell, the errors are due to compression during the mirroring or streaming process and the different colour spaces on different devices. It is also possible that the screenshot application on mobile devices introduces errors. Images may also be degraded when embedded in emails sent from mobile devices. As such, making the measurement on the mobile device would seem to be best if that's possible.
>
> Are we the only people who don't know how to do this? What does everyone else do?
>
> Is it possible to download an app from the App Store or Play Store and somehow pull it into XCode or another development environment in order to see all the colours? Or are there any decompilers, like we used to use to pull apart Flash movies?
>
> Regards,
> Steve Green
> Managing Director
> Test Partners Ltd
> 020 3002 4176 (direct)
> 0800 612 2780 (switchboard)
> 07957 246 276 (mobile)
> 020 7692 5517 (fax)
> Skype: testpartners
> = EMAIL ADDRESS REMOVED =
> www.testpartners.co.uk
>
> Connect to me on LinkedIn - http://uk.linkedin.com/in/stevegreen2
>
> > > >
From: Abi James
Date: Wed, Jun 23 2021 1:01AM
Subject: Re: Measuring colours accurately in mobile apps
← Previous message | Next message →
On iOS there is an app called Color Contrast (https://apps.apple.com/gb/app/color-contrast/id1095478187) that allows you to identify colours at a pixel level and calculate contrast either from screenshots (for apps) or websites directly. Used be free but now has small monthly charge.
I generally test with that and if colours are close to failing then I ask the developer and design to share HEX codes to confirm (in part to make sure they agree as sometimes the wrong colour has been used).
Abi James
Sent from my iPhone
On 23 Jun 2021, at 06:03, Steve Green < = EMAIL ADDRESS REMOVED = > wrote:
CAUTION: This e-mail originated outside the University of Southampton.
Does anyone have a method for accurately measuring the text and background colours in mobile apps in order to make colour contrast measurements? For context, we are independent testers, so we never have access to the development environment. Sorry this is a bit long, but it's perhaps the most challenging test process issue we currently have.
Aside from an accessibility scanner app for Android, I am not aware of any tools that can make the measurements on a mobile device. All the methods we have tried involve making the measurements on a desktop machine, which means transferring an image of the device screen.
We have done extensive testing and found that every method we can think of is wildly inaccurate. Such methods include:
* Taking screenshots and emailing them as attachments.
* Taking screenshots and emailing them as embedded images.
* Taking screenshots and transferring them through the file system.
* Screen mirroring using Mobizen and Lonely Screen.
* Setting up a Zoom call between the mobile device and the desktop machine.
The key findings are:
* For any given colour in the app, each method of measurement gives a different result. The differences can be massive. None are close enough to be useful.
* For any given method of measurement, some colours might be measured fairly accurately, but the measurement of other colours is hopelessly inaccurate.
The accessibility scanner app for Android is useful, but only up to a point. It reports the colour values in combinations that do not have sufficient contrast, which is good. However, it does not report the colour values in combinations that pass. That won't bother a lot of people, but our process requires all combinations to be measured and recorded as proof of what the colours were on that day and proof that all combinations were tested.
Also, we don't have any solution for iOS and there is the issue of measuring non-text contrast.
As far as I can tell, the errors are due to compression during the mirroring or streaming process and the different colour spaces on different devices. It is also possible that the screenshot application on mobile devices introduces errors. Images may also be degraded when embedded in emails sent from mobile devices. As such, making the measurement on the mobile device would seem to be best if that's possible.
Are we the only people who don't know how to do this? What does everyone else do?
Is it possible to download an app from the App Store or Play Store and somehow pull it into XCode or another development environment in order to see all the colours? Or are there any decompilers, like we used to use to pull apart Flash movies?
Regards,
Steve Green
Managing Director
Test Partners Ltd
020 3002 4176 (direct)
0800 612 2780 (switchboard)
07957 246 276 (mobile)
020 7692 5517 (fax)
Skype: testpartners
= EMAIL ADDRESS REMOVED =
https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.testpartners.co.uk%2F&data%7C01%7Ca.james%40soton.ac.uk%7Cdbce6dd780144daf0fe708d936042f78%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C637600213932011148%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Sr1PWHkIVEPeOpTa%2F62hCYeStGcUDIz%2BwPss7fQZa2c%3D&reserved=0
Connect to me on LinkedIn - https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fuk.linkedin.com%2Fin%2Fstevegreen2&data%7C01%7Ca.james%40soton.ac.uk%7Cdbce6dd780144daf0fe708d936042f78%7C4a5378f929f44d3ebe89669d03ada9d8%7C0%7C0%7C637600213932011148%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=QMw7lndYFjX%2BtIUFsRTMAR4wRpB0wvhLXYlFcqcTmnQ%3D&reserved=0
From: Patrick H. Lauke
Date: Wed, Jun 23 2021 1:03AM
Subject: Re: Measuring colours accurately in mobile apps
← Previous message | Next message →
We generally go down the "take a screenshot on device, then transfer it
to a desktop/laptop device for traditional testing with CCA or similar"
path, despite its potential flaws (for instance, even taking an
on-device screenshot can lead to specific colour profiles/adaptations on
the device being applied, making the colour drift slightly). And you're
exactly right, other methods like Zoom, Reflector, or similar will
compress the video leading to changes in colour, and should always be
avoided.
When emailing, if given the option, go for "original" rather than any
email client resizing. That should guarantee that it takes the raw
screenshot that was saved, without any further recompression.
There's no (legal/legit) way to grab an app from the app store/google
play store and then decompiling it (especially since these apps are
digitially signed etc when packaged).
When reporting on colour contrast failures (and luckily, we only do
failures, and don't list all colour combos including passes) we also
include wording that explains the values are approximate, based on
having to test what was available to us. If a client wants more
precision/certainty, we ask for actual accurate colour values they used,
or request that they give us access to a development build/XCode
project/similar to open in an actual development tool.
P
--
Patrick H. Lauke
https://www.splintered.co.uk/ | https://github.com/patrickhlauke
https://flickr.com/photos/redux/ | https://www.deviantart.com/redux
twitter: @patrick_h_lauke | skype: patrick_h_lauke
From: Steve Green
Date: Wed, Jun 23 2021 1:12AM
Subject: Re: Measuring colours accurately in mobile apps
← Previous message | Next message →
Thanks for the suggestion of the app - I'll try that.
I am surprised you don't see the colour degradation in the screenshots, although it is much more pronounced with some colour schemes than others. We created some test pages so we knew the exact colours and contrast ratios, and the error levels were astonishingly high when emailing screenshots.
Here are some examples. The true colours were green text (#006600) on white (#FFFFFF), which has a contrast ratio of 7.24:1. The values we measured were:
Measurement Method
Foreground Colour
Background
Colour
Contrast Ratio
Error
Actual Values
#006600
#FFFFFF
7.24
0%
Safari (iPhone) > Zoom desktop share
#066208
#FFFCFE
7.47
3.2%
Chrome (Android Pixel 4a) > Zoom desktop share
#005400
#FEFEFD
9.17
26.7%
Screenshot of image attached to an email
#005500
#FDFDFB
8.97
23.9%
Screenshot of image embedded to an email
#015700
#FFFFFF
8.88
22.6%
Steve
From: Steve Green
Date: Wed, Jun 23 2021 1:17AM
Subject: Re: Measuring colours accurately in mobile apps
← Previous message | Next message →
Doh! All that data was in a table, but it's been linearised. Hopefully it still makes sense. The main point is that in most cases the contrast ratio measurement was wrong by more than 20%.
Steve
From: Steve Green
Date: Wed, Jun 23 2021 1:22AM
Subject: Re: Measuring colours accurately in mobile apps
← Previous message | Next message →
Thanks, it's good to know we're not missing anything obvious.
Steve
From: Joe Humbert (A11y)
Date: Wed, Jun 23 2021 5:58AM
Subject: Re: Measuring colours accurately in mobile apps
← Previous message | Next message →
Accessibility Inspector that comes with Xcode on MacOS will allow you to get accurate colors if they fail certain criteria. You can use this with any Application (your own developed, client source code, or Apps from the app store). You will need to connect you iOS/iPadOS device to a Mac with Xcode installed. I think you need to register the device with Xcode, but then it will work.
https://developer.apple.com/videos/play/wwdc2019/257/
I can probably do a screen recording tomorrow and send a link.
For Android, Accessibility Scanner will get you the colors, but not the font sizes or weights. I don't remember off the top of my head if Accessibility Inspector's Audit gives the font size/ weight.
Thankx,
Joe Humbert, CPWA
Accessibility Champion
Android & iOS Accessibility Novice
Twitter: @joehumbert
From: Steve Green
Date: Tue, Jun 29 2021 1:25AM
Subject: Re: Measuring colours accurately in mobile apps
← Previous message | Next message →
Thanks, Joe. Are you sure you can do a colour contrast test on apps from the App Store? I don't have a Mac at the moment, but one of my team says that he can do a colour contrast test on an app he has developed, but he can't do it on an app from the App Store. Having seen the video you linked to, I am sceptical of what he is telling me, but I can't check it. Is he missing something?
Steve
From: joe
Date: Tue, Jun 29 2021 7:57AM
Subject: Re: Measuring colours accurately in mobile apps
← Previous message | Next message →
Hi Steve,
Having worked for a consulting firm for 4 years where 90% of the time we did not get source code access to the app, but instead downloaded directly from the App store, I think he might be missing something.
Video link (300MB ::sigh:: QT non-compression): https://www.dropbox.com/s/fyx4blvh3pisf0m/Using%20Accessbility%20Inspector%20on%20App%20from%20App%20Store.mov?dl=0
Link will be deleted in 2 days!
Note: The video has no audio.
Accessible Alternative for video: It shows me downloaded the Hulu App from the App store and opening it, Opening Accessibility Inspector, Choosing my iPhone as the target devices, running an audit on the Hulu app and then showing a (possibly not valid) color contrast issue with provided hex values for foreground and background colors.
I have no professional affiliation with Disney, Hulu or any related companies at this time.
Thankx,
Joe Humbert, CPWA
Accessibility Champion
Android & iOS Accessibility Novice
Twitter: @joehumbert
From: Steve Green
Date: Tue, Jun 29 2021 8:26AM
Subject: Re: Measuring colours accurately in mobile apps
← Previous message | Next message →
Hi Joe,
Thanks ever so much for the video. That looks like what we have been doing, so I suspect that my colleague has been testing apps that didn't have any colour contrast failures, in which case I will chastise him appropriately for wasting our time.
Nevertheless, it's interesting that the colour measurements in your video are completely wrong, as you noted. I will investigate that when I get my Mac back.
Steve
From: joe
Date: Tue, Jun 29 2021 8:39AM
Subject: Re: Measuring colours accurately in mobile apps
← Previous message | No next message
Hi Steve,
I just grabbed the first app could find that I was ok downloading. The colors for the login screen were (possibly) not correct because they were colors over an image-based background which is inaccurate with most (probably all) tools except manually picking or getting the hex values from the designer. I didn't actually test the colors so my comment was a guess.
The Accessibility Inspector is not perfect so it can miss contrast issues, but it can help. Don't chastise, this may help others on the list.
Thankx,
Joe Humbert, CPWA
Accessibility Champion
Android & iOS Accessibility Novice
Twitter: @joehumbert