At work and in my personal life I’ve had a chance to explore Microsoft’s new integration using Chat GPT where Copilot brings AI into the M365 Microsoft Office Suite. At work, we are exploring it as a possible tool to help or support a wide variety of people with different types of disabilities. In my personal life, I’ve installed it to help me with a number of the activities involving documents or research. As a person who is blind, I am planning on writing about my own experience and the things we learn at work. Let me start by saying that my early experience has shown this to be a game changer for people who are blind.
What is Copilot?
Copilot is a new technology that uses artificial intelligence to help you write better and faster. It can assist you with various tasks, such as writing emails, reports, articles, code, and more. Copilot can also answer your questions, generate summaries, and provide suggestions based on your context and goals.
Why is Copilot Useful for Blind Users?
For people who are blind or have low vision, screen readers have limitations, such as not being able to interpret complex graphics, charts, or diagrams, or not being able to capture the tone, mood, or style of a text. Screen readers also work in a very linear fashion which often means a multi-step activity for items where a sighted person would “glance” or jump to items. Screen readers and Accessible documents and material provides mechanisms to jump around but visual activities are still faster with sight. Accessible documents are machine readable documents which means accessible document creation improves your results with copilot or other AI systems. Regardless, Copilot can level the playing field for blind users by providing them with more information, feedback, and visual support. Copilot can also help blind users save time and effort by automating some of the tasks that are tedious or difficult for screen reader users. Before we get into the nitty gritty of the topic, I want to preface the below material with a clear statement that you still need to reason through material that is generated. Additionally, like any tool that searches or generates text, you still need to verify material. You might use Copilot to draft something but you still need to edit it and it won’t generate perfect solutions. However; it can save huge amounts of time and get you started.
What are Some Examples of How Copilot Can Help Blind Users?
Here are some common activities that are extremely time consuming or difficult for many blind users, where copilot has proven exceptionally useful for me and my work:
- Note taking during meetings: For someone who is using a screen reader, participating in a meeting, listening to a screen reader, and taking notes is a challenge. You have to try and pay attention to two different streams of audio (an analogy to this would be a person trying to follow two TV shows at once). Enabling transcripts and Copilot allows you to do note taking after the meeting with questions like:
- What were the key items raised by Bob?
- Summarize the action items I agreed to.
- Summarize the meeting.
- Working with spreadsheets: For someone who is using a screen reader, excel spreadsheets are a visual way of presenting data, which many people who are blind find complex or difficult to work with. Additionally, reading and working through a large spreadsheet of data and information can be a long task that is time consuming when you have to hear the screen reader read out information auditorily. . Copilot allows you to work through the spreadsheet without having to interact with the visual layout directly, with questions like:
- What are the trends?
- How much did Cisco go up over the last three years?
- Compare the results from Cisco and Apple.
- Interpreting images and graphics: For someone who is using a screen reader, images, graphics, and even social elements are often only provided visually. Copilot allows you to get visual information about images and graphical information, with questions like:
- Describe the image.
- What is the bar chart showing?
- What is the color of the desk in the photo?
- For someone who is using a screen reader, visual activities like turning a word document into a Powerpoint can be hugely time consuming. Copilot allows you to work in MS Word, create and craft the content and than use copilot to create the Powerpoint from the document and adjust the “look” of the Powerpoint.
- Searching for content and information on the web can be time consuming. You can go to copilot.microsoft.com and ask it to do the searching for you. You can also ask it follow up questions. I find that many times it’s quicker to do searching this way rather than manually with Google or other search engines. It gives you the links as a reference so you can check the accuracy or read more about the material it is quoting. Here are some recent examples:
- My family was trying to figure out what a hot sauce from a large hot sauce sampler tasted like. We asked copilot what does Raging Cajun Hot Sauce taste like? Similarly, we asked it what west coast wings from Wings Up! (taste like A local restaurant). In both cases it gave me a very useful answer.
- When shopping for a headset, I asked it what the pros and cons of the EPOS 5011 headset were and it gave me an excellent bullet point list with links. Similarly, when asked for similar comparable headsets or if the headset would work with Microsoft Teams it answered both.
I could do the same with a google search but the time I saved with these and other research items is noteable.
What hasn’t worked
I had hoped to make heavy use of Copilot with Microsoft Outlook. I had planned to use it to summarise long chains of emails or extract items from a chain of emails. However; Microsoft has recently released a different version of MS Outlook. This version is more of a web interface wrapped in a desktop app’s container. Microsoft has renamed the previous version that many are familiar with as Classic Outlook. The new outlook lacks many of the common keyboard shortcuts that the classic outlook uses and in my testing using the latest versions of 3 popular screen readers, the experience is not good. Since your only options are to use the web interface or the new outlook, copilot is not going to save you time if you are a screen reader user.
What’s Next
That is what I’ve learned so far. I am planning n writing a series of articles as I learn more. I’d be interested in other blind people’s experience with copilot. Please feel free to use the comment section below and let’s learn together.
