By leveraging speech to text and automated captioning technologies, content producers and platform owners can ensure that their videos are accessible for people with visual and hearing disabilities.

A person using navigational keyboard
Atom Accessibility Friday, May 22, 2020 - 14:23

By Shriram Parthasarathy

Around six years ago, when the second season of House of Cards aired on television, I happen to see some part of the episode in which Kevin Spacey’s character, Frank Underwood, becomes the president for the first time. I wanted to start over from the first season, but never got to doing it. But come March 2020, I took about three weeks to watch all the 73 episodes across the six seasons.

Yes, I did have more time on my hands over the weekends and even some part of the weekday to binge watch due to the COVID-19 lockdown but more importantly, the addition of built-in audio descriptions meant I could engage more with the content now than I could six years ago.

An audio description is the audio narration of the elements on the screen when the characters are not speaking. These would include expressions, silent actions, surroundings, and scenery to call out a few. Some might argue that ‘audio description’ is good to have, but for people with visual disabilities like myself, it is a deal-breaker for consuming video content where emotions and silent situations also convey information.

Democratizing access to information

It is estimated that a billion people or nearly 15% of the world’s population have some kind of a disability. They are part of today’s education system, professional and vocational workforce, entrepreneurs, and specialists amidst others. In today’s time, when in-person activities have become virtual due to COVID-19, platforms and content not being accessible is a severe tax on one’s right to information.

At the start of March 2020, the average number of messages across my chat groups would range between 100 to 150 messages per day. As we reached the end of March, when social interactions became virtual, it increased to 700 messages per day. This included a range of formats across photos, stickers, gifs, silent videos, and videos with monologues without captions, most of which weren’t accessible for people with hearing and visual disabilities.

As I traverse different social media platforms in and out of work, there are a lot of videos gracing our news feeds that are consumed with sound off. This is even more a reality today when different people under the same roof are consuming different types of content at the same time. This is where captions play a critical role. Primarily being associated as a hack for people with hearing impairment, captioning is just a good inclusive design practice. Similarly, for photos, providing alt-text can help people with visual disabilities to be able to make sense of the image.

I could interpret the photos by having them recognized by Seeing AI to describe the image. But there was no way for me to engage with stickers, GIFs and silent videos, unless I asked someone to describe them to me. While I could ignore most of the non-accessible content that came my way in chat groups, basking in the knowledge that it was not critical, in the COVID-19 world we need to take a closer look at whether information, especially from official sources, is accessible or not.

Unlike a decade ago when making content accessible could be a time consuming and often an expensive proposition, today artificial intelligence (AI) plays a critical role as it brings a multi-sense opportunity with a range of cognitive services, which can be integrated in any application to make it accessible.

By leveraging speech to text and automated captioning technologies, which have become mainstream thanks to AI, content producers and platform owners can ensure that their videos are accessible for persons with visual and hearing disabilities. Similarly, the computer vision API can help generate automatic alt-text for images.

I believe it should be mandatory for all official channels that disseminate news to have close captioning in videos and alt-text in photos as the content is essential piece of information.

The same goes for apps providing essential services—if some of your users can’t access it, you are losing out on business. A couple of weeks ago, I was looking to order in some dinner and turned to one of the leading food delivery apps. The interface had a slight makeover, but all the images and banners within the application had no labels, and the screen reader on my phone achieved a hung state toggling between ‘banner’ and ‘image’. I exited the app and some Italian outlet out there missed slicing some cash for bread and a little more.

Accessibility is a business imperative. There are tools like Accessibility Insights that can help report any accessibility-related gaps in your apps across operating systems.

Making virtual interactions accessible

The fundamental shift from physical interactions to virtual connects has become the new normal for everyone. Be it one-on-one conversations, team meetings, or company townhalls, everything has moved to the digital realm. Considering that you are not in the same room as everyone else anymore, it is even more critical to ensure your interactions are accessible. Thankfully, AI is here to help too.

If you are creating a presentation, check out the inclusive presentation checklist to ensure your presentation is accessible for people across the board. PowerPoint also enables live captioning, which will ensure what you speak is also accessible for people with hearing disabilities.

If your meeting is happening on Microsoft Teams, participants can also turn on live captions for any meeting, even one-on-one voice calls.

Making everyday communications accessible

“What image is that?” This used to be a regular question I’d ask my colleagues when they sent me emails with screenshots and sometimes images as glamorous as a group of people rowing a boat.

Cut to May 2020, a colleague of mine was planning a game night for our team and we got talking about how games could be made more accessible. We spoke about adding alt text to images, which has become an intelligent feature over the years with AI bringing in the recommendation engine to pre-populate suggestions.

As AI continues to empower human ingenuity, there are certain human-first actions that each one of us can put into practice. These practices that were good-to-have once upon a time, are more of a must-have today. Here are a few of them:

  • Do not assume your audience – in the way they consume your content. They range across people using screen readers, captions, pictures, simple language, lipreading to consume content optimally. With Accessibility Checker, which is available across the Microsoft 365 suite of apps like Outlook, Word, Excel, PowerPoint among others, you can ensure that your content is accessible to everyone before you share it with others. It is as simple as that!          
  • Leverage the option to add captions as you post pictures on social media. Remember, a screen reader can read the text, but not interpret the image. When using hashtags, ensure you capitalize the first alphabet of every word for screen readers to be able to relay them properly.
  • Think accessibility at the time of building a website or an app, rather than it being an afterthought. As you select the platform for digital events, webinars, meetings, and interactions, check for accessibility ranging from live captioning to keyboard only navigation.

The practices we put in place today would continue to ensure an inclusive environment for all times, even when the physical spaces would be back to make their presence felt. The digital-first economy will continue to mushroom diverse experiences as we progress through 2020 and beyond. It is up to us whether these experiences are accessible to all or not.

Shriram Parthasarathy is the Lead for Social Media Marketing at Microsoft India; accessibility evangelist; and recipient of the NCPEDP Mindtree Hellen Keller Award for 2019 for being a role model for persons with disabilities.

 

Views expressed are author's own