Bug 100086 - Accessible events for selection of multiple cells on sheet are not correctly exposed to screen reader
Summary: Accessible events for selection of multiple cells on sheet are not correctly ...
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
5.1.3.2 release
Hardware: All All
: medium normal
Assignee: Michael Weghorn
URL:
Whiteboard: target:7.3.0
Keywords: accessibility
Depends on:
Blocks: a11y, Accessibility
  Show dependency treegraph
 
Reported: 2016-05-27 11:06 UTC by minakonono3519
Modified: 2021-10-14 06:02 UTC (History)
10 users (show)

See Also:
Crash report or crash signature:


Attachments
NVDA demo patch to use IAccessibleTable2 and IAccessibleTableCell interfaces in LO (4.17 KB, patch)
2021-09-08 14:38 UTC, Michael Weghorn
Details

Note You need to log in before you can comment on or make changes to this bug.
Description minakonono3519 2016-05-27 11:06:41 UTC
This occurs in EDITING when using a screen reader NVDA 2016.1jp.

Procedures
1)Enter 'apple' in cell A1, 'orange' in A2, 'pair' in A3.
2)Focus the A1 cell by cursor.
3)Press shift+down key.
4)Press shift+down key.

Readouts by NVDA at the procedure above
2)apple A1
3)(no sound)
4)(no sound)

We would like LibreOffice Calc to read out them like in Excel, if not, at least in OpenOfficeCalc 4.1.1.

Readouts when Excel is used
3)from A1 apple to A2 orange selected
4)from A1 apple to A3 pear selected

Readouts when OpenOfficeCalc 4.1.1 is used
3)orange selected A2
4)pear selected A3
Comment 1 V Stuart Foote 2016-05-27 18:12:31 UTC
On Windows 10 Pro 64-bit en-US withVersion: 5.1.3.2 (x64)
Build ID: 644e4637d1d8544fd9f56425bd6cec110e49301b
CPU Threads: 8; OS Version: Windows 6.19; UI Render: GL; 
Locale: en-US (en_US)

With NVDA 2016.1 installed and functioning, accprode (w/x86 JRE) monitoring events.

Confirming. Seems like we get additional accessible events beyond doing the cell selection.

Could someone check what Orca exposes.
Comment 2 am_dxer 2016-11-29 14:25:23 UTC
I tested with Orca on FreeBSD. Nothing is spoken when pressing Shift in conjunction with the arrow keys. It seems as though the cells are selected, however, nothing is spoken by the screen readers to indicate the extent of the selection.
Comment 3 Yousuf Philips (jay) (retired) 2016-11-29 16:16:25 UTC
Tested Orca on Ubuntu Mate and step 3 and 4 resulted in no sound.
Comment 4 am_dxer 2016-12-21 18:41:43 UTC
Should the importance of this bug get upgraded? This is really probably a show stopper for assistive technology users trying to use the spreadsheet application. When selecting a large group of cells, it is extremely difficult to count the cells with no audio feedback.
Comment 5 V Stuart Foote 2016-12-21 19:27:30 UTC
(In reply to am_dxer from comment #4)
> Should the importance of this bug get upgraded?

No the priority and impact are correctly set. The "a11y" meta and "accessibility" keyword keep topic in focus for Calc developers.
Comment 6 Yousuf Philips (jay) (retired) 2016-12-22 10:36:13 UTC
Kohei just finished fixing bug 71409, so hopefully he could look at this a11y calc bug as well.
Comment 7 am_dxer 2017-06-26 17:26:16 UTC
Repro in 4.4.0.2. Must be older.
Comment 8 am_dxer 2017-06-26 19:35:59 UTC
Repro in 3.5.7.2. Must be older.
Comment 9 Alex ARNAUD 2017-07-13 15:04:35 UTC
Dear all,

Joanie: Do we have the correct events emitted on GNU/Linux ?

Since the resolution of the bug 93825 we have "object:state-changed:selected" events emitted but I'm not sure it is sufficient for Orca.

Best regards.
Comment 10 QA Administrators 2018-07-14 02:47:55 UTC Comment hidden (obsolete)
Comment 11 Peter Vágner 2018-08-03 12:18:11 UTC
Isn't this essentially a duplicate of 93825?

Still I think there are more issues with this implementation we just slowly need to document it.

=== Steps to reproduce ===
* Grab the accessible listener from 93825
* Launch libreoffice calc spread sheet
* Press shift+space to select entire row
* Now launch the accessible event listener in a terminal
* Focus back to the calc window
* Press and hold down shift key and repeatedly press left arrow key several times. 5 or 10 times is fine to see the issue in action.
* Examine the event listener output in the terminal window

=== Expected result ===
Corresponding with switching back into the calc window single object:active-descendant-changed is expected.
Then number of object:selection-changed events is expected for each selection contraction. So if you have pressed shift+left arrow keys 5 times you should have at least 6 events printed to the terminal window.

=== Actual result ===
Corresponding accessibility event is only being sent for the first selection change resulting from pressing shift+left arrow key. Second, third and all other shift+left key presses don't generate appropriate accessibility events.


=== Notes ===
I think windows screen readers such as NVDA don't handle this at all so on windows work shal be done on both ends of the stack.
Also I think there might be more issues like this we haven't yet clearly documented them.
Comment 12 Alex ARNAUD 2018-08-03 13:42:40 UTC
I've added Tamas in CC to answer to Peter comment #11 because he has resolved the issue #93825

Best regards,
Alex.
Comment 13 QA Administrators 2019-08-04 05:49:18 UTC Comment hidden (obsolete)
Comment 14 Adriani90 2020-05-24 14:35:09 UTC
Dear Libre office team,

Testing with Libre Office 7.4.4.2 and Open Office 4.1.7, in both cases the sellected cell range is not reported. The expected behavior when pressing shift+arrow keys is to report the sellected cell range (i.e. a1 through a3 when pressing shift+down arrow key two times).

I am testing with NVDA 2020.1.
Comment 15 Adriani90 2020-05-24 14:36:45 UTC
Sorry I meant Libre Office 6.4.4.2 of course.
Comment 16 norman 2020-10-15 09:09:49 UTC
Following up on this, What needs to happen to get some time put towards fixing this? If i'm not mistaken selection does work on libreoffice with orca. But nvda and windows still have troubles.
Comment 17 Michael Weghorn 2021-07-24 16:44:13 UTC
Corresponding issue in NVDA's issue GitHub issue tracker: https://github.com/nvaccess/nvda/issues/6897

FYI: I have a local work-in progress branch for implementing IAccessibleTable2 and IAccessibleTableCell support for winaccessibility in LO and hope to find some time to continue with this (presumably in September). This might help here, but more will have to be done (on LO and/or NVDA side).
Comment 18 Michael Weghorn 2021-09-08 14:38:05 UTC
Created attachment 174894 [details]
NVDA demo patch to use IAccessibleTable2 and IAccessibleTableCell interfaces in LO

Pending Gerrit changes to add support for the IAccessibleTable2 and IAccessibleTableCell interfaces to LibreOffice:

https://gerrit.libreoffice.org/c/core/+/121820
https://gerrit.libreoffice.org/c/core/+/121821

As mentioned in my previous comment, this isn't enough.

For demo/testing, I used the attached demo patch to NDVA (on top of master as of 8979880164cae91209436e57b414b063625e92e0) to access and print information about the currently selected cells using those interfaces. (It's not fit for integration in NVDA as is, just a test/demo usage of the interfaces.)
Comment 19 Commit Notification 2021-09-08 18:06:41 UTC
Michael Weghorn committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/839dbf9ecf9f8fbec7de983d1a2e16d7de6f868c

tdf#100086 tdf#124832 wina11y: Implement IAccessibleTable2

It will be available in 7.3.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 20 Commit Notification 2021-09-08 18:07:02 UTC
Michael Weghorn committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/97a88e30e2e084ab860635ff4e0a03442d8a12af

tdf#100086 tdf#124832 wina11y: Implement IAccessibleTableCell

It will be available in 7.3.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 21 Commit Notification 2021-09-16 07:26:11 UTC
Michael Weghorn committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/836a226205df9e76e77d26af80f402de7b876d61

tdf#100086 wina11y: Don't delete a11y object for removed cell right away

It will be available in 7.3.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 22 Michael Weghorn 2021-09-16 07:53:26 UTC
Some further discussion currently going on in this NVDA issue: https://github.com/nvaccess/nvda/issues/9310

It works with LO master and a current draft NVDA pull request by Leonard de Ruijter:  https://github.com/nvaccess/nvda/pull/12849
Comment 23 Commit Notification 2021-09-18 07:07:24 UTC
Michael Weghorn committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/1b94deea2c7648255e3efac08fd352f80c2bda06

tdf#100086 wina11y: Return "fresh" IEnumVARIANT for get_accSelection

It will be available in 7.3.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 24 Michael Weghorn 2021-10-14 06:02:44 UTC
(In reply to Michael Weghorn from comment #22)
> Some further discussion currently going on in this NVDA issue:
> https://github.com/nvaccess/nvda/issues/9310
> 
> It works with LO master and a current draft NVDA pull request by Leonard de
> Ruijter:  https://github.com/nvaccess/nvda/pull/12849

The NVDA PR is now scheduled to be merged for NVDA 2022.1.

I'm closing this bug, since the LO side has already been merged and will be available from LO 7.3 on.

In case you still encounter any issues with LO >= 7.3 and NVDA >= 2022.1, please file new bug reports with details on what exactly is not working as expected.