Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Read Outlook's auto-complete suggestions in the To, CC & BCC fields of new emails #8502

Open
Qchristensen opened this issue Jul 13, 2018 · 74 comments · Fixed by #8919
Open
Labels
app/microsoft-office bug/regression component/UIA-ms-office Specific to UIA used in MS office applications. p4 https://github.com/nvaccess/nvda/blob/master/projectDocs/issues/triage.md#priority triaged Has been triaged, issue is waiting for implementation.

Comments

@Qchristensen
Copy link
Member

Steps to reproduce:

In Outlook, when you start typing a contact's name, Outlook displays a list of recently used email addresses and contacts which match what has been typed so far. So, if "Mar" is typed, Outlook will list any Mary's or Martin's in your contacts, or anyone with "mar" in their email address.

To navigate through the list, use the arrow keys and to use the selected contact or address, press enter or tab.

Actual behavior:

When using the arrow keys to move through the list of contacts, NVDA reads "Blank". The only way to have NVDA read the contacts is to hover the mouse over, then the information is read.

Expected behavior:

A user would expect NVDA to read the contacts as the focus moves to each.

System configuration:

Windows 10 fast insider build 17713.1000
Microsoft Office 365 / Outlook 2016 Version 1806 (Build 10228.20104 Click to run) - updated just now.

NVDA Installed/portable/running from source:

NVDA 2018.2.1 portable
NVDA Alpha 15527,9c3f5bc8 installed
NVDA 2016.4 temporary

(Same results with all three versions)

Other information about your system:

In the current build of Windows 10, Narrator does announce "Suggestions available" when the suggestions appear then proceeds to read the first one, and the current suggestion when moving through the list.

Other questions:

Does the issue still occur after restarting your PC?

Have you tried any other versions of NVDA?

I thought this used to work. Reading issue #8055 which talks about the autocomplete list (but encounters an error with Braille), it seems to indicate it was working then (if it wasn't, that surely would have been worth mentioning in that ticket), so it's possibly a regression in a recent build of Outlook.

@josephsl
Copy link
Collaborator

josephsl commented Jul 13, 2018 via email

@Qchristensen
Copy link
Member Author

Qchristensen commented Jul 13, 2018 via email

@josephsl
Copy link
Collaborator

josephsl commented Jul 13, 2018 via email

@Qchristensen
Copy link
Member Author

Qchristensen commented Jul 13, 2018 via email

@josephsl
Copy link
Collaborator

Hi,

This feature worked as advertised in earlier Office 365 releases (including Office 2013 I believe). I'm thinking the control has changed and we didn't keep up with that change.

CC @derekriemer

@Adriani90
Copy link
Collaborator

Adriani90 commented Jul 13, 2018

@Novalis7747, are you planning to raise a new pull request with your code?

@Brian1Gaff
Copy link

Brian1Gaff commented Jul 13, 2018 via email

@Ednunp
Copy link

Ednunp commented Oct 14, 2018

Guys, I'm confused about the state of this issue. It is being reported as closed, but I for one am still seeing the issue with latest NVDA and office 365 update. IN the issue that it is linked to in the merge, it doesn't seem to clearly say whether the speech side of it has been resolved. Can someone comment either way please and if not resolved, are we any closer to resolving this with speech? not having the list of names complete is extremely annoying as it can take significantly longer to enter addresses in a large organisation.

@josephsl
Copy link
Collaborator

josephsl commented Oct 14, 2018 via email

@Ednunp
Copy link

Ednunp commented Nov 21, 2018

Guys, is this ever going to get resolved? What is the current status? is MS looking into it, are NVDA looking into it, is anyone looking at it? it was opened in July and is by all accounts dead in the water. I want to stress again this is a significant issue when working in a large org with a lot of names and addresses. Currently the only workaround such as it is, is to enter a few letters of someones name, then hit enter, see if the name is correct, delete it if not, enter the same few letters of the name again, scroll down once and hit enter to try it all over again, repeat until you get the name right. This is truely ridiculous if you're entering 30 addresses in one of these fields. Can we please have a resolution plan or an indication of what's going on? Thanks, Ed.

@josephsl
Copy link
Collaborator

Hi,

Since we haven't heard from the individual who did a pull request on this, I'll take this on.

Because of its high impact, I propose raising the priority of this one to P2 at least. If I do come up with a PR, it'll be based on beta unless @michaelDCurran says otherwise.

Thanks.

@Ednunp
Copy link

Ednunp commented Nov 21, 2018

Thanks Joseph apreciate this. I know of at least 3 people who wil be extremely happy once this is fixed.

@josephsl
Copy link
Collaborator

Hi,

Further investigation says: yes, my initial hypothesis was right: I think we've been so complacent to a point where we cost someone his or her productivity.

Technical: the "To:" field and friends have way better UIA implementation. This is evidenced by the fact that:

  1. Narrator announces suggestions.
  2. Controller for event is raised.

Several solutions come to my mind:

  1. Tell NVDA that message address fields are good UIA controls (outlook.AppModule.isGoodUIAWindow). This then allows UIA to take over. Results announcement can be taken care of through NVDAObjects.UIA.SearchField.
  2. Add an IAccessible class for auto-complete fields and ask if it has a UIA implementation (UIA.kwargsFromSuper(dict)). For older Outlook versions, it should fail, but since 2016/Office 365 (which includes Outlook 2019), these fields should present a UIA version of themselves. This guarantees backward compatibility and is an application of UIA grid row strategy used a few years back to deal with Outlook message list.
  3. Same as second solution, but this time add suggestion announcement capability (NvDAObjects.behaviors.EditableTextWithSuggestions). Then while handling name change events, see if the underlying UIA implementation says anything about controller for array, and if it changes for some reason, raise suggestionsOpen/Closed events as needed. If chosen, this solution becomes an application of search field handler outside of Windows 10, also taking care of another issue about hiding suggestion sounds checkbox in object presentation panel outside of Windows 10.

I imagine all of these (and more possibilities) have downsides, especially when it comes to supporting Outlook 2007 and earlier:

  1. If first solution is chosen, we need to tell NVDA to use UIA for auto-complete fields on Outlook 2013 and above unless someone using Outlook 2010 says it is working as well.
  2. The second solution needs to catch possible exceptions.
  3. The third one is perhaps the hardest to implement but is a bit more flexible.

CC @michaelDCurran, @jcsteh, and @LeonarddeR in case you have something to say.

Thanks.

@josephsl
Copy link
Collaborator

Hi,

Technical: although name change event may seem promising, it won't cover certain situations. Also, the root culprit is the fact that state change event for auto-complete list item is not raised in some cases, but when using UIA, it works.

I'll compile a try build for you all to try (the code is based on latest master snapshot).

@josephsl
Copy link
Collaborator

Hi,

Try build:

https://ci.appveyor.com/api/buildjobs/3967ub6tudlp64xa/artifacts/output%2Fnvda_snapshot_try-outlookAutoCompleteUIA-16299%2Ccd98032c.exe

Don't worry about error tones when entering address into "To:" field in Outlook.

Please test this build under the following combinations:

  1. Windows 10 and all of the following Outlook versions if possible: 2007, 2010, 2013, 2016, 2019, Office 365
  2. Windows 8.x and the following Outlook versions: 2007, 2010, 2013, 2016, Office 365 (Office 2019 does not support releases other than Windows 10).
  3. Windows 7 and Outlook 2007, 2010, 2013, 2016, Office 365

Notes:

  1. Windows releases earlier than 7 SP1 won't be supported.
  2. Outlook 2003 is end of life, therefore I recommend not supporting it.
  3. Support for Outlook 2007 is contingent on community feedback (especially from enterprises).
  4. Outlook 2010 support may or may not make the cut depending on whether UIA implementation used in this try build is even usable.
  5. Full support is expected for Outlook 2013 and later (including recent iterations of Office 365/2019).

Scenarios while entering address into "To:" field and friends:

  1. Type the name of a known contact and see if NVDA announces the top suggestion.
  2. As you type addresses, see if NVDA announces refined search results.
  3. While doing all these, see if NVDA plays search suggestion sound (to configure this, go to NVDA Menu/Settings/Object presentation/Play a sound when auto-suggestions appear checkbox must be checked).
  4. If you have a braille display, see if suggestions appear on the display.

For the first two scenarios, if one of them fails, let us know how to reproduce the problem (note that my computer has Office 365/2016 at the moment), along with Windows version and Outlook version in use. The third item shouldn't be a problem unless you've got Windows 7 or 8.x, and the fourth item is applicable only for those using a braille display.

Thanks.

@Adriani90
Copy link
Collaborator

Hey @josephsl, I've tested with MS Outlook 2016 and 365 with speech. Suggestion sound plays properly and NVDA announces the suggestions as expected. It works in to, cc and bcc.
Using Windows 10 1809 update.

I have noticed that the address in the field is recognized as button. This is also announced when navigating with left arrow to the address. It says "button" and "button end". This is not the case in NVDA 2018.4 Beta 1.

@josephsl
Copy link
Collaborator

josephsl commented Nov 21, 2018 via email

@govsta1981
Copy link

govsta1981 commented Nov 21, 2018 via email

@Novalis7747
Copy link

running Office 365 under Win 1809. the suggestion in the TO field works fine when typing. When arrowing down the suggested email addresses they are spoken correctly, but when up arrowing in the list it only says blank and arrowing doen again only brings me the value I typed in the address field.

@Novalis7747
Copy link

Novalis7747 commented Nov 21, 2018 via email

@josephsl
Copy link
Collaborator

josephsl commented Nov 21, 2018 via email

@Novalis7747
Copy link

Novalis7747 commented Nov 21, 2018 via email

@josephsl
Copy link
Collaborator

josephsl commented Nov 21, 2018 via email

@Novalis7747
Copy link

Novalis7747 commented Nov 21, 2018 via email

@Novalis7747
Copy link

Novalis7747 commented Nov 22, 2018 via email

@josephsl
Copy link
Collaborator

josephsl commented Nov 22, 2018 via email

@Novalis7747
Copy link

Novalis7747 commented Nov 22, 2018 via email

@josephsl
Copy link
Collaborator

josephsl commented Nov 22, 2018 via email

@LeonarddeR
Copy link
Collaborator

I'm still suffering from this issue on Outlook 2016 16.0.11727.20230. Even when up and down arrowing through the suggestions, NVDA stays silent.

@Novalis7747
Copy link

Novalis7747 commented Jul 13, 2019 via email

@Ednunp
Copy link

Ednunp commented Jul 14, 2019 via email

@Novalis7747
Copy link

Novalis7747 commented Jul 14, 2019 via email

@Qchristensen
Copy link
Member Author

And since I originally reported this, just confirming that yes it is working now in Office 365. If it's not working in Office 2016 as per @Novalis7747 Comment, can anyone please confirm whether that is fixable in NVDA itself? If it's going to require an external fix from Microsoft, we can probably close this now.

@Qchristensen
Copy link
Member Author

Another update. This isn't completely fixed in Office 365:

  • The first time the auto-complete suggestions are shown for any given message, nothing is read. EG if I press M to write Mary, Martin and Mary might be suggested, but NVDA won't read anything until I arrow through the list. This is problematic if there is only one name in the list, and the focus cannot move so nothing is read (and reading the current object just reads what you have typed, not any suggestions).

  • The second and later times the auto-complete suggestions are shown, NVDA instantly reads the first one.

@Novalis7747
Copy link

Novalis7747 commented Aug 13, 2019 via email

@lukaszgo1
Copy link
Contributor

If it regressed with #8919 it looks like Office 2016 has no usable IAccessible implementation for autocomplete list whereas 365 does. The problem is that Microsoft in their infinite wisdom made build numbers for Office 365 and 2016 to overlap, so we cannot use version numbers to use IAccessible /UIA in these cases as needed.
According to https://stackoverflow.com/questions/52892882/how-to-detect-office-2019-programmatically/52949687#52949687
it should be possible to query the registry and check if we are dealing with 365 or 2016. Note that it is still asked on user list from time to time - Office 2016 seems to be still very popular.

@Ednunp
Copy link

Ednunp commented Aug 13, 2019 via email

@Qchristensen
Copy link
Member Author

Yes, @Ednunp - my workaround for now in Office 365 is, say you want to enter someone's name which begins with a, in the to field type a, backspace, a, then it reads the suggestions.

@chojiro
Copy link
Contributor

chojiro commented Feb 29, 2020

Also happens in Office 2019 using NVDA 2019.3.1 and Office 2019, unfortunately. Workaround not applicable as described for Office 365.

@zahra21
Copy link

zahra21 commented Jul 18, 2020

chojiro,
you told in office 2019 and office 2019 that i did not understand!
do you mean only office 2019 or 2016 and 2019 both of them?

@chojiro
Copy link
Contributor

chojiro commented Jul 18, 2020 via email

@Qchristensen
Copy link
Member Author

With this issue still in Office 2016 and 2019 apparently but not in Office 365, does fixing it in those versions of Office require a fix from NVDA's end or from Microsoft?

@josephsl
Copy link
Collaborator

Hi,

To address hte latest question:

I think it will involve looking at build range and taking appropriate action, as Office versions from 2016 onwards (including 2019/2021/365) are tagged as version 16.0. Note that Office 2016 is in extended support, so I guess 2019 is the earliest we might support (2019 enters shortened extended support soon, 2021 won't pick up pace for a while, and folks are moving toward 365).

Thanks.

@Qchristensen Qchristensen added the component/UIA-ms-office Specific to UIA used in MS office applications. label Oct 25, 2021
@XLTechie
Copy link
Contributor

XLTechie commented May 3, 2022

This appears to be working correctly in 2019 now, as of NVDA alpha-25324,0f08ccb7 (and probably before, I don't use Outlook much so haven't noticed).

@fernando-jose-silva
Copy link

I'm sure this doesn't work in the office 2016 version used at my place of work, which is even a problem for a very basic computer user who works with us.
Now I can't consult the office version but tomorrow I can put the still problematic version in this call.
In the office version that I have on my personal computer OUTLOOK.EXE, Microsoft Outlook 16.0.15128.20178
everything works fine.

@CyrilleB79
Copy link
Collaborator

It's not working either on Office 2016 on my workplace (version '16.0.5197.1000').
It is working on Office 2016 at home (version 16.0.15128.xyz... if I remember well).

@fernando-jose-silva
Copy link

the version of outlook I work from that this problem still exists is
OUTLOOK.EXE, Microsoft Outlook 16.0.5197.1000

@fernando-jose-silva
Copy link

I just noticed that the problematic version of my office is the same as the
@CyrilleB79

@josephsl
Copy link
Collaborator

josephsl commented May 4, 2022 via email

@seanbudd
Copy link
Member

@fernando-jose-silva, @CyrilleB79 is this still in an issue after updating Outlook?

@seanbudd seanbudd added triaged Has been triaged, issue is waiting for implementation. p4 https://github.com/nvaccess/nvda/blob/master/projectDocs/issues/triage.md#priority labels Jun 16, 2022
@CyrilleB79
Copy link
Collaborator

@seanbudd, the version has remained the same, i.e. '16.0.5197.1000' as indicated in the file version in this update, which is the last one as indicated on this page.
As expected, the behaviour is unchanged.

@fernando-jose-silva
Copy link

according
@CyrilleB79
reports, I still have this problem in outlook
'16.0.5197.1000'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
app/microsoft-office bug/regression component/UIA-ms-office Specific to UIA used in MS office applications. p4 https://github.com/nvaccess/nvda/blob/master/projectDocs/issues/triage.md#priority triaged Has been triaged, issue is waiting for implementation.
Projects
None yet
Development

Successfully merging a pull request may close this issue.