1.3.1 Example: Motor imagery BCI.

Motor imagery (MI) EEG is a widely-used paradigm for non-invasive non-evoked BCI [14–17]. It is a prime exemplar of the marginal reliability input device. In this paradigm, users imagine moving different parts of the body and the corresponding event-related rhythm changes in the motor cortex are detected in the electrical signals measured at the scalp. The lateralisation of motor function in the brain leads to a spatial separation of imagined motions which can be classified [18, 19]. Even with modern techniques for feature selection and classification, a motor imagery BCI can typically produce binary decisions at the order of one per second, with accuracies of around 60-90% [2] being typical. One minute of input might produce 60 binary decisions, 10 of which would be flipped.

Ahn [2] reviews motor imagery BCI systems and and finds error rates reported in the range 35% and above; Ahn [20] illustrates the very high variation in inter-subject error rates with the same classifier from ∼50% to less than ∼5%. Padfield et al. [21] give example error levels from the literature of 9% for visually evoked potentials; 13–31% for event-related potentials; 16% for motor imagery BCI. Lotte [22] reviews per-decision accuracies in the literature for a broad range of non-evoked BCIs; the key results of which are summarised in Fig 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Non-invasive BCI accuracy summary.

Summary of per-classification accuracy from many binary non-invasive BCI EEG studies, plotted from the Tables A1-A3 of Lotte et al. [22], including movement intention and mental task imagination BCIs. This figure summarises the accuracy of classifiers used in a large number of brain-computer interfaces, indicating that accuracies vary widely from around 65% to 95% (i.e. there are error rates of 5% to 35%). Where ranges or bounds were given the stated numerical value is used.

https://doi.org/10.1371/journal.pone.0233603.g002

There are many otherwise promising technologies which have very high error rates; for example EMG systems with error rates of ∼30% [23]; or 8.1-5.6% [24]. Such high levels of error, combined with frustratingly slow response times make conventional “undo” functionality insufficient (see 4.1.1 for numerical simulation results and the theoretical analysis in [5]). The binary motor imagery channel is an excellent exemplar of the class of niche interaction methods we are interested in, for two reasons: it is a well-known input mechanism for which improved interfaces could offer immediate benefits; and it is an instructive example of designing for extremely challenging input devices. We do not always have the luxury of improving the qualities of channels to be harnessed for input, and it seems likely that current non-invasive EEG-based techniques will have a substantial subset of users for whom two-class motor imagery classification accuracies will be less than 95%. Beyond brain-computer interfaces, marginal reliability channels can be found across systems where input is impaired either physically or situationally and the control akin to a noisy two-state button is the only functionality available.

2.1.1 Classification and origin of error.

Avizienenis et al. [28] outline a detailed taxonomy of errors (a deviation from intended state) and faults (the proximate cause of an error) in the context of safety-critical systems. In this work, we are focused on errors that arise because of natural phenomena, human action, or hardware deficiencies. We do not consider robustness to malicious, deliberate or adversarial actions, or robustness to enduring design and implementation deficiencies in the software itself. We also exclude from consideration enduring cognitive or perceptual issues, such as inability to identify targets or inability to form short-term memories.

In particular, we consider:

• cognitive errors such as slips [33, 34], defined by Norman [33] as“a form of human error defined to be the performance of an action that was not what was intended”. These are errors in cognition, such as forgotten actions, mis-ordered sequences of action, mode identification errors or incorrectly repeated actions.
• performance errors [6]: random variation internal to a human user during the production of motor action, such as poor coordination, or muscle tremor that leads motor action to deviate from intent;
• environmental disturbances: unrelated variations external to both a user and a system, such as power fluctuations, lighting variations, or external movement (e.g. vibration inside a vehicle) that pollute control signals; and
• measurement noise: distortions originating within an observation system caused by sensing or processing inadequacies, such as the effect of electrical noise, occlusion, quantisation, insufficient classifier training, mis-calibration, etc.

All of these error sources are distinct in nature, but from the perspective of human-machine control similarly lead to deviations between user intention and system state during an interaction. We focus on implementing robustness to transient errors caused by essentially random deviations, usually though not necessarily fully independent in time. In particular, we may encounter errors correlated in time in measurement noise (e.g. a sensor getting stuck due to loss of contact) or cognitive errors such as slips which introduce error over several interaction steps (e.g. a user starting a sequence of actions to perform one task, before realising the task was incorrectly chosen).

2.1.2 Mitigation strategies.

In the presence of input error, mitigation strategies can be categorised [28]:

• Rollback or backward error correction [31]: the system reverts to a previous state; this is the undo or backspace operation and requires either automatic error detection or explicit correction actuation. Sometimes this includes larger scale correction strategies [32, 35] such as cancel/abort to revert a higher-level task or stop to cease execution of a higher-level task.
• Rollforward or forward error correction [31]: errors are ignored and state changes anyway. This is appropriate when the cost of an incorrect choice is smaller than the cost of correction. For example, accidentally hitting the volume up control on a music player might change the volume slightly but be of little consequence to the user.
• Compensation: the system has sufficient redundancy that errors in input, up to some level of tolerance, do not lead to deviations in internal state. This is the domain of error correcting codes.

Various forms of undo have been extensively studied in human-computer interaction [35–37, 37–40] as a widely implementable way of establishing error tolerance. This typically involves choices about the granularity of undo [38, 40], the structure of undo (linear/branching) [41] and the controls for actuating undo. Other approaches have looked at structuring the finite state machines (FSMs) that define interface behaviour such that they that simply admit fewer errors or are at least harder to drive into erroneous states [42, 43].

Our focus is on the compensation strategy via error-correcting codes that introduce exactly enough tolerance to random deviations (for a given input channel) that the internal state remains consistent with user intention. We also examine how to integrate this model with undo-style rollback approaches.

2.1.3 Typical error rates in standard interfaces.

We will use the term standard interface to collect together common, widely used interfaces: mouse pointing on a desktop GUI; typing on a physical keyboard; tapping on a touchscreen GUI; and typing on a virtual keyboard.

Targeting errors in mouse pointing in controlled tasks has been found to be relatively constant around 3% for visual targets from 0.83 to 183mm [44]; studies of mouse pointing in realistic desktop GUI situations found error rates of 3% [45] and between 2-20% [46], with the higher rates for an elderly population. Pointing tasks that can be modelled by Fitts’ law are often assumed to result in a speed-accuracy trade-off that maintains a 4% error rate [47]; a strong predictive model of error rates in pointing tasks is given by Wobbrock et al. [48, 49]. Large scale text entry studies on physical keyboards [1] have suggested fairly stable correction (i.e. backspace) rates of 6% with 1% uncorrected errors remaining, and 1.17 keystrokes/character (from 136M measured keystrokes). A N = 37000 user study on mobile devices with virtual keyboards [50] found uncorrected errors from were around 2% with 1.18 keystrokes/character, suggesting similar level of mis-keying error. Large scale studies of error rates (mis-targeting error) in touch screen tapping have found to be between 10% (15mm targets) to 30%(9mm targets) from 100M touch events on mobile devices [51].

2.1.4 Target error rates.

Shannon’s noisy channel theorem [52, 53] indicates that any arbitrary level of transmission error can be achieved over a channel subject to noise, with some bounded cost in pre-encoding the data. Viewing the human input problem as a noisy channel, we can therefore theoretically mitigate any level of noise to achieve any desired level of reliability. Perfect reliability will induce some penalty in communication, and one consideration is the tolerable error rate for a user interface.

The studies on typing, pointing and tapping discussed above have error rates typically around 3-10%. This suggests reducing selection error rates to around the 4% error assumed in Fitts’ law-like pointing tasks [47] will give comparable performance to standard interactions. This assumes a similar distribution of options and similar utility/importance per option as in a comparable standard interaction. For reliability-critical systems, we can target much lower error rates and accept slower interaction; for time-sensitive systems like real-time control of a wheelchair, we can target higher error rates and trade-off increased responsiveness for occasional deviations.

2.2.1 Universality.

A widely applicable approach should be able to couple a wide range of noise levels to activities with a spectrum of desired error rates. For example, transforming a noisy pressure sensor into a selection device with error rates comparable with standard mouse GUI interaction (∼4% error rate); or interfacing a BCI with low classification accuracy to a safety-critical function such as neuroprosthesis elbow extension control [54] when pouring boiling water, where error rates of 1 × 10⁻⁶ may be required.

2.2.2 Predictability.

Evaluating designs with users is expensive. We would prefer to know, or at least bound, the performance of a design in advance of building it. This motivates interfaces for which there are strong, parameterised predictive models. Such models would predict performance characteristics, like error level or entry rate, from basic estimates of the properties of an input device. We show that there are rigorous theoretical approaches to achieve this, and devote significant portion of this work to developing theoretical and numerical predictions validated against human behaviour.

2.2.3 Graceful degradation.

An interaction method suitable for a variety of input device error rates should not have cliff-edge performance failures. There should be smooth, parameterisable adjustments to the interaction which can cope with increasing error at a proportional performance cost.

2.2.4 Adaptability.

Similarly, the interaction should be adaptable to changes in performance, both at design time (i.e. a parameterised and well-understood performance envelope) and online, during interactions, to cope with changing input conditions. Many user interface contexts, especially those encountered in wet-electrode brain-computer interaction, have input properties that vary strongly with time [55, 56] and an interaction model that can cope with these changes by online adaptation will be more widely applicable.

2.2.5 Simplicity and uniformity.

Finally, we would like to have an interaction that is conceptually simple for both users (requires little working memory and minimal mental computation) and designers (straightforward, predictable parameterisation of the interaction). To maximise transferability of skills, interfaces developed using interactions should be uniform in their appearance and behaviour, across input devices (e.g. from BCI to pressure sensor), interaction contexts (e.g. from a media player to text entry) and across levels of reliability (e.g. no special handling for high error inputs).

2.3.1 Coding in user interfaces.

In the case of a human-computer interface, it is the user who must perform the compression, coding and modulation for the channel; the system performs demodulation, decoding and decompression to recover user intention. User interfaces mediate this process by offering feedback which supports the user in these tasks. They can, for example, make the modulation explicit via feedback (as a mouse pointer and a set of targets, for example). They can make the compression explicit (perhaps by offering autocompletions of a partially-entered phrase). Or they can make the error coding process explicit (perhaps by requiring confirmation stages in a sequence of dialogs). Fig 4 illustrates these nested layers of coding for input in asymmetric interfaces; note that the encoding is serial, but the feedback at each level is displayed in parallel via a high-bandwidth noise-free display. We propose to explicitly consider these steps, and their corresponding feedback loops, when designing an interface. Thoughts originating in a users mind must be compressed, by the user, to a small range of state changes available to them. These must be encoded such that they can robustly pass through the noisy processes of the body and the unreliable sensing of the system. They then have to be realised by modulating the physical state of the world; moving a hand, twitching an eyebrow or imagining a foot tapping. The system interpreting those signals must demodulate the sensed physical action, trying to reconstruct the intention the user attempted to signal. This must then be decoded to reliably infer which action was intended; and this decoded action must then be used to optimally select among the many options available according to some probability distribution (decompression).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Entropy, channel and line coding in the user interface.

The input problem in an asymmetric user interface, viewing the interface communication channel. The diagram shows how the feedback loop allows users to drive the internal state of a system towards their intention, via a nested series of entropy, channel and line coding steps. Sophisticated transport of information across an interface can be implemented by pushing the complexity of the encoding process from the user into the system and relying on feedback to mediate the process.

https://doi.org/10.1371/journal.pone.0233603.g004

Designing for human communication is quite different from the issues encountered in traditional communication theory. The physiological and cognitive abilities of humans are very different from those encountered in computer to computer communications and creating user interfaces requires creative engineering to exploit the quirks of human memory and perception. The configuration of joints and muscles, for example, imposes complex ergonomic constraints on the modulation process; even simple spatial targeting has hundreds of variants to optimise the information capacity in different contexts (see Section 2.5).

The design of a user interface implicitly embodies compression, encoding and modulation. This is often obscured by the subtle interweaving of these three processes and the layers of metaphor by which interaction designers make interfaces usable, aesthetic and practical to implement, but it is the underlying purpose of a user interface to facilitate communication. The class of interfaces which we are interested in are highly asymmetric: rich, high-information-capacity, zero-noise feedback display is available but the input channel is severely restricted.

4.1.1 Simulating backspace.

We consider the problem of entering a sequence of n symbols, using an alphabet S of size 2^k, where S = s₁, s₂, …, s_←. One symbol is backspace s_←, and 2^k−1 are unique terminal symbols. Decoding the backspace symbol “undoes” the previous symbol decoded. The performance of this backspace channel can be characterised as the number of bits required to perfectly enter a string of n symbols for a given alphabet size 2^k. The number of binary decisions required from the user to produce one correct bit with backspace coding on a binary symmetric channel is R_b(k, f) with bit flip probability f and residual error e_b(f, k) = 0. Empirical results for simulated backspace-entry from N = 10000 random trials for various values of f and k are shown in Fig 8. The decisions per correct bit for the backspace channel R_b(k, f) is approximated by a Gamma function (Fig 8): (3) where , the cost of assigning one of the symbols to backspace and p_k = (1 − f)^k, the probability of correctly entering one k bit symbol correctly given a bit error rate of f. In regimes where the backspace decoder does not function this formula gives negative values which we treat as infinite. From numerical simulation, the mean absolute error of this approximation is approximately 3% for f ∈ [0.0, 0.5], k ∈ [2, 16]. It is quite clear that although introducing a backspace character makes entry on an unreliable channel possible, the performance is very far from optimal, and works very poorly indeed for f > 0.1. Even if we settle with entering from four symbol alphabet S = {A, B, C, ←}, channels with error rates f > 0.25 have effectively zero capacity. This is on top of the mental effort the user must apply to remap those three symbols onto the desired input. Unfortunately, this is often the only error correction available in many assistive technology systems, either as a literal backspace in text entry, or an undo functionality in a more general user interface context. Our problem is to create an input metaphor with a rate R(k, f)>R_b(k, f) for f > 0.2, and thus make usable interfaces for channels with reliabilities in the 60%–80% range. As well as the strictly limited range of f for which a coding with backspace is useful there are several other issues which make backspace a restrictive error correction technique:

• There is no obvious way to deal with Z channels, with f₀ ≠ f₁.
• Users must alternate between entering symbols and editing to correct errors. These are distinct tasks which require separate mental attention and can become frustrating as errors increase.
• For low-reliability channels (f < 0.1), the only effective control has three symbols plus backspace. In most cases users have to concatenate two codes: first a mapping to three symbol+backspace and then onto some higher level symbols such as characters.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Capacity of the binary symmetric channel with backspace R_b(k, f).

Capacity of backspace for alphabets with k = 2, 3, 4, 6 for N = 10000 simulated entries of a n = 32 symbol sequence. Throughput is shown as the number of input bits per correct bit R; solid lines shows the mean throughput, and the shaded region shows the standard deviation. The hatched region is the Shannon bound for the binary symmetric channel. Even with k = 2 bit symbols, capacity goes to zero as the reliability drops below 75%. Dashed lines show the fit of Eq 3.

https://doi.org/10.1371/journal.pone.0233603.g008

4.1.2 Predicting error-free rates from non-zero e_k.

If we devise a new channel code with some residual uncorrected error rate e_k, we can always augment it by adding backspace to reduce the error to zero, if e_k is low enough. We concatenate the inner code with the backspace code. We can predict the number of bits/error-free symbol for this concatenated decoder using Eq 3: (4)

4.2.1 Optimal noisy bisection.

From the point of view of a user, Horstein’s code is a generalisation of bisection to noisy inputs. Bisection is the optimal way to identify a point (within some tolerance) on a bounded interval with noise-free binary input. Options—target symbols that a user might select—are laid out on the interval [0, 1], and there is a “cursor” which divides the interval into two, initially placed at 0.5. Input is sequential, where the user indicates via the input device if the symbol they wish to input is left or right of the cursor. The same approach is used in Horstein’s algorithm, but by accumulating inputs over a whole sequence, the process will reliably converge to an intended target in the fewest possible inputs even when the input is corrupted by random flipping, if the algorithm is configured with knowledge of the true error rate.

4.2.2 Algorithm.

A Horstein decoder maintains a continuous probability density over the unit interval [0, 1]. For each step i we define:

• p_i(x) = P(x = θ) The probability distribution for possible values of the unknown target θ;
• f_i(x) the probability density function (PDF) and F_i(x) the cumulative distribution function (CDF) that define p_i(x);
• the inverse cumulative distribution function.

F_i is stored as a piecewise linear function, and so the probability density is a mixture of uniforms.

Typically we begin the process with a uniform prior p₀(x)∼U(0, 1) but any other prior could be used instead. Algorithm 1 shows the complete algorithm.

Algorithm 1 Horstein’s algorithm.

1: function horstein(k, β, f₀, f₁)

2:  p ← (1 − f₀)/((1 − f₀) + f₁)

3:  q ← (1 − f₁)/((1 − f₁) + f₀)

4:  CDF F₀(x) ← line segment [(0, 0), (1, 1)]

5:  while H(f_i(x)) < (k + β) do

6:    (median from inverse CDF)

7:   Display m_i

8:   Receive b_i from input device

9:   if b_i = 0 then

10:    F_i+1[0: m_i] ← pF_i[0: m_i]

11:    F_i+1[m_i:] ← (1 − p)F_i[m_i:]

12:   else

13:    F_i+1[0: m_i] ← (1 − q)F_i[0: m_i]

14:    F_i+1[m_i:] ← qF_i[m_i:]

15:   end if

16:  end while

17:  return m_i

18: end function

4.2.3 Horstein’s algorithm as a Bayesian update.

Horstein’s algorithm process is simply the recursive Bayesian updates of a probability distribution p_i+1(x|b_i) given an noisy input b_i that indicates whether the target θ < m_i. The median m_i, is defined such that . Then we use the distribution at the previous step p_i(x) as a prior, and the posterior is given by: (5) (Alg. 1 9-15), where (6) (Alg. 1 2-3).

Because we always divide at the median m_i, we can assume that there is an equal probability of θ < m_i at any step i; then the prior P(θ < m_i) = P(θ ≥ m_i) = 0.5. (7) and by symmetry: (8)

4.2.4 Horstein decoding in the human-machine loop.

The algorithm elicits a “left” (b_i = 0) or “right” (b_i = 1) decision from the user for each input step, by sending the targets and the current median m_i of the cumulative density function (CDF). The user inputs a “left” (b_i = 0) if the desired target is less than the median, and right (b_i = 1) otherwise. F_i(x) is then distorted according to how reliable the input is regarded as being. These distortion steps gradually steepen the cumulative density function F_i(x), or equivalently, concentrate the probability density. Fig 9 illustrates the key update step of the algorithm.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9. The key step of the Horstein algorithm: Distorting the CDF.

The CDF transition is shown for the case where the initial transition is b₀ = 0. The CDF F_i(x) is initially a line segment with gradient 1. It is partitioned at the point m_i where F_i(x) = 0.5 and the left and right gradients are scaled by factors p and 1 − p. In the case b₁ = 1, the respective factors would be 1 − q and q.

https://doi.org/10.1371/journal.pone.0233603.g009

4.2.5 Block coding.

We present a slight modification of Horstein’s original stream code, using fixed length symbols (though in practice we can relax this to variable length codes to accommodate arbitrary priors over targets). To use this code, we choose a symbol length k, and an adjustable confidence level β (measured in bits). We then map each of the 2^k symbols onto an interval in [0, 1] of length 2^−k, i.e. each codeword onto a subsection the unit interval. We can of course introduce a non-uniform prior over outcomes (e.g. as in arithmetic coded interfaces), such that the codewords are then assigned to non-equal sub-divisions of the unit interval (Fig 10).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 10. Uniform and nonuniform targets.

Subdivision of the unit interval into discrete targets for selection can be performed uniformly (top, corresponding to a flat prior over targets), or according to some known prior distribution π(s_i) (bottom).

https://doi.org/10.1371/journal.pone.0233603.g010

Because of this change, our termination condition differs slightly from the original given by Horstein, which terminates when a region around the median becomes sufficiently dense. We instead continue until the entropy of the distribution over the interval drops by a set level: (9)

At the termination, we now have a new distribution over the unit interval, and consequently over the symbol set. This transformed into a symbol by choosing the symbol whose interval which contains the median m_i at termination. Under a uniform subdivision, given a target symbol s_i of length k, we compute s_i = [2^k m_i]. Fig 11 shows an illustration of evolution of the probability density function (PDF) and cumulative density function under the Horstein algorithm with k = 5, β = 0 for the noise-free and noisy cases. An illustration of how the updating inverse PDF at each step of the Horstein algorithm can be used to remap the unit interval to “stretch out” areas of higher density is shown in the sequence of steps in Fig 13.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 11. Ridge-plots of the PDFs from the Horstein algorithm.

PDFs are plotted following each input bi applying Horstein algorithm to select the the 5 bit symbol 01010 (mapped to the interval around θ = 0.3125, highlighted in red), for the noise-free case (left, f = 0) and with simulated noise (right, f = 0.15), with k = 5, β = 2. Sharpening of the PDF is gentler in the case with noise.

https://doi.org/10.1371/journal.pone.0233603.g011

The Horstein decoder is only optimal for discrete memoryless channels where the channel statistics are known. Section 6.1 presents empirical results which show how the decoder throughput varies against the mismatch between the expected and actual channel statistics.

4.2.6 Biased channels.

If the channel under consideration is not symmetric but is biased (i.e. f₀ = P(0 → 1) ≠ f₁ = P(1 → 0)) the performance will clearly be affected. The bias can be represented as a term δ, −1 ≤ δ ≤ 1, so that P(0 → 1) = f₀ = f + f_δ, P(1 → 0) = f₁ = f − f_δ, which maintains a constant average error rate f. The Horstein algorithm can deal optimally with biased channels. Fig 12 shows simulations illustrating the evolution of the density function for biased and unbiased errors. A general Horstein decoder is fully parameterised by the tuple (k, β, f₀, f₁).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 12. Example simulation of the Horstein algorithm with noisy inputs.

k = 8, β = 0 selecting a target θ = 0.71875, with symmetric (left) and biased (right) noise. The pulse traces show the inputs for b_i = 0 and b_i = 1 respectively; highlighted sections indicate erroneous inputs. The centre plot shows the log PDF logf_i(x) at each step.

https://doi.org/10.1371/journal.pone.0233603.g012

4.2.7 Headroom.

Since we have an imperfect knowledge of the true channel statistics f₀ and f₁, and there is a steep penalty for under-estimating the error rate (see Section 6.3) it is prudent to add some tolerance to the expected channel statistics when setting the decoder’s configured rates and . This headroom f_h introduces a penalty in reduced communication rate but in return offers protection against uncorrectable error cascades when the uncorrected error rate e_k slips above the rate that a concatenated backspace decoder can recover from.

4.2.8 Trisection and q-ary inputs.

In some use cases it is easier to imagine an interface splitting a set into an inner and outer part, rather than bisecting on a central point (for example, consider an interface requiring a motion towards or away from a screen). This can be implemented with the Horstein decoder by trisecting the CDF F_i(x) at the 25% and 75% percentiles instead of the median, and using the input b_i to either scale the first and fourth quartile or the second and third quartile.

The Horstein decoder extends naturally to q-ary channels. Instead of splitting at the median m_i at each step, the splitting is perfomed at each quantile m₀…m_q, dividing the CDF F_i(x) into q units. Given a new q-ary symbol b_i, the slopes of each quantile segment F_i[j], j ≠ b_i are multiplied by 1 − f_b and the slope of F_i[b_i] is multiplied by f_b, where f_b is the expected probability of error for input symbol f_b.

4.2.9 Entropy coding.

It is straightforward to combine the Horstein algorithm with entropy coded data using arithmetic coding. In this case, we have a non-uniform prior over targets, which is represented as distribution over the unit interval π(x). We simply continue with the Horstein code in k length chunks then output any completed symbols pending.

4.2.10 Decision quality metrics.

It has been assumed that only a fixed, unvarying estimate of the channel statistics is available, for example from calibration. Some input devices can report reliability on a per-decision basis (e.g. from a probabilistic classifier), instead of a discrete binary value. The reliability measure can be used to dynamically estimate f₀ and f₁ at each step. In the simplest case, the classifier emits probabilities directly which are used as f₀ and f₁. Other methods (e.g. support vector machine-based classification) may report distance measures from which an (approximate) probability can be derived.

4.2.11 Adaptation.

In the simplest case, a calibration procedure with known targets can be used to estimate and ; however, this requires the user to spend time performing this calibration task, or a strong prior model of the channel to be known. In cases where the channel statistics may be unknown, or may change over time, it is possible to adapt the decoder online. This can be done by counting the number of inputs n actually required to reduce the entropy to k + β for each symbol, and compare with the expected inputs for the configured channel statistics using Eq 2, . This leads to the adaptive update rule where (10) for some threshold ϵ_n, and a small fixed quantity δ_n. This provides a simple way to adapt the decoder online for symmetric channels. See Section 6.4.2 for an online adaptation algorithm suitable for biased channels.

5.1.1 Multiple dimensions.

The decoder can be extended to N dimensions by maintaining N independent Horstein decoders C⁰…C^N, each representing a marginal probability distribution at decision i. As these are independent, we can compute the joint distribution on the N dimensional space simply as Simply cycling through decoders in round robin order is inefficient, because the random distribution of errors may have one decoder almost certain, while others are far from convergence. We propose a more intelligent selection of decoder at each decision, as the feedback channel gives us the freedom to elicit information from dimensions on an arbitrary schedule by changing the display. To select the next decoder to update, we compute the entropies H^j(x) for each decoder C^j, and request input for the dimension with the largest entropy. This entropy-based scheduling can also be extended to multiple input modalities (e.g. hybrid BCI).

5.1.2 Mapping controls: Diagonal split interfaces.

This approach provides a straightforward decoding process for 2D grids (although it is perfectly possible to perform the selection in 3 or more dimensions, a 2D grid is the most practical to display). Multiple options can be laid out on a 2^k × 2^k grid, and the system can request input from the appropriate decoder dimension by drawing a median line on appropriate axis. From a user’s point of view, a binary input device provides two options, normally with a strongly associated directional component (e.g. left hand versus right hand). Scheduling dimensions to distinct decoders according to entropy is theoretically efficient, but it requires changing the mapping from the input device to the display every time the decoder switches (switching left vs. right to up vs. down, for example). It is quite challenging to deal with an input device whose interpretation changes regularly, especially for input channels like motor imagery, where the input classes might be left hand imagination and right hand imagination; rapidly switching between a left hand being interpreted as “select the left side” and meaning “select the upper side” makes the input task much more difficult, as we observed in initial early prototypes of our interface. A simple solution in 2D is to rotate the displayed grid 45°, so that every decision is always between left and right, even as the axes alternate (Fig 14). This allows packing options onto a 2D grid, but requiring only left/right decisions which map precisely to the visual display.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 14. Diagonal split interfaces.

Rotating the plane 45 degrees allows partition on both axes with only left-right decisions.

https://doi.org/10.1371/journal.pone.0233603.g014

5.2.1 Implementation.

We implemented a number of variants of the 2D Horstein decoder, including linear and non-linear zooming, point targets, rectangular targets, circle-packed targets, space-filling curve models, diagonal split interfaces and trisection interfaces. Images of these implementations are shown in Fig 16.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 16. Screenshots of prototype Horstein decoder interfaces.

Left-to-right: (a) Simple 2D point based non-linear zooming (b) Nonlinear zooming with rectangular area targets (c) Linear zooming with a randomised circle-packing (d) Diagonal-split interface which only requires left-right decisions (e) An interface using Jigsaw space-filling curves [111] to layout blocks of ordered targets.

https://doi.org/10.1371/journal.pone.0233603.g016

5.3.1 Space filling curves.

Space filling curves, like the classical Hilbert or Peano curves, or modern compact curves like the Jigsaw curve [111] or the Balanced GP curve [112] provide a natural way to wind a 1D sequence onto a 2D unit square. This provides a straightforward mapping for ordered data into a 2D Horstein decoder. In particular, curves like the Balanced GP curve which optimise for bounding-box optimality result in subdivisions that are reasonable to select with this interface design. Space-filling curve approaches are suitable for 2D selection of ordered data types.

5.3.2 Packings and tilings.

In cases where ordering is not the primary organisational cue, some form of packing may be used to allocated targets to the 2D space, maintaining a probability to area relationship. Packing of rectangular or circular targets via randomised algorithms gives a straightforward way to construct an interface. This will necessarily leave gaps in the unit square, which is suboptimal from a coding point of view. This “dead space” between packed targets, however, can be reclaimed as a natural way to include a backspace control; selection of the gap area actuates backspace. Packing structures make most sense for unordered data types or for data types where a 2D layout is approximately known. For example, an image collection in a photo browser application might be laid out by some form of dimensional reduction to establish approximate 2D locations; a prior probability over images could be defined to determine target areas; and a packing algorithm used to place appropriately sized targets.

5.3.3 Hierarchies.

Hierarchies of symbols are easily accommodated using the strategy applied by Dasher [83] which nests sub-symbols, leading to a spatial representation of the arithmetic coding of the symbol sequence. There are three competing factors in a hierarchical 2D layout: good aspect ratio for subdivisions to maintain visibility so that labels remain clear; visual representation of the hierarchy; accurate representation of the underlying probability of each subdivision as its visual area. In 2D the treemap/squarified treemap approach [113, 114] gives a suitable algorithm for the case where the subdivisions are orthogonal to the axes. Alternative variants can subdivide the plane non-orthogonally and retain better aspect ratios [115]. This style of layout is natural for many interaction problems like navigating hierarchical file systems or hierarchical menus. Some hierarchical layout algorithms can become irregular as symbols nest deeply. Sub-optimal layouts like circle or square packings provide a simpler navigation experience, at the cost of null space.

6.1.1 Concatenating with the backspace code.

A real interface would apply a backspace correction to the output of the Horstein decoder, concatenating these two error correcting codes. This will reduce the uncorrected error rate to zero at a cost given by Eq 3, where e(k, f₀, f₁) is the uncorrected error rate of the Horstein decoder. Each set of simulation results presented shows three plots: input bits/uncorrected output bit R; the uncorrected error rate of k bit symbols e_k; the predicted input bits/error free output bit R′ using the backspace decoder approximation of Equation 4.1.2.

6.2.1 Effect of β and k.

As k increases, the performance of the decoder approaches the Shannon bound (Fig 19). Shorter codes have poorer performance, as expected. As the confirmation factor β increases the probability of uncorrected errors decreases, at a corresponding increase in the number bits per correct symbol (Fig 17). Fig 18 shows that, for a fixed β, there is a very strong linear relationship between f and the uncorrected error level e_k that does not depend on k. The gradient and offset of this line reducing approximately exponentially with increasing β. An empirical log-linear fit to the error rate for a given bit flip probability e_k(f) gives the approximation (shown in dashed lines in Fig 17): (11)

This simple closed-form approximation for e_k(f) means that a desired residual error level can easily be optimised for when implementing a decoder for an input device; for example, targeting the 4% error rate discussed in Section 2.1.

The total number of bits for one correct entry with a perfectly matched decoder j is: (12) for some constant m. The probability of uncorrected error depends directly on β (see Fig 19), so larger k is more efficient both because of a better approximation to the Shannon bound and because of a reduced influence of β on the total overhead. For larger k (e.g. k = 16), the Horstein code approaches the Shannon bound very closely. Increasing β > 0 typically degrades performance when combined with the backspace decoder, because it is more efficient to a small number of residual correct errors via backspace than to eliminate all errors in the Horstein stage; however, this becomes less straightforward when the channel statistics are imprecisely known. In some cases, it is important that symbols be selected accurately the first time (e.g. if they have real-world consequences that cannot be corrected after the fact, or to minimise user interface complexity). In this case, increasing β gives a way of reducing the error level to any arbitrary level without requiring a separate undo stage. Additionally, it should be noted that β can be fractional, allowing any level of confirmation to be achieved.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 17. Decoder performance for varying confirmation β.

k = 8 in all trials. (Left) input bits per output bit R, (Centre) uncorrected k bit symbol errors e_k (Right) backspace corrected rate R′(k; f). As β increases e_k → 0.

https://doi.org/10.1371/journal.pone.0233603.g017

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 18. Relation of β, f and e_k.

Simulated uncorrected error rates ek from N = 10000 repetitions, for k ∈ 4, 6, 8, 10 and various different β. There is a very strong linear relation between the uncorrected error rate and β, which does not depend on k. The linear fit is shown as a dashed line. The gradient and offset of the line reduce exponentially with increasing β. The dashed line shows the approximated error rate e_k(f) using Eq 11.

https://doi.org/10.1371/journal.pone.0233603.g018

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 19. Decoder performance for varying symbol size k.

β = 2. Larger k has improved capacity.

https://doi.org/10.1371/journal.pone.0233603.g019

6.2.2 Effect of bias f_δ.

The Horstein decoder is efficient with biased channels and can take advantage of known biases, as shown from the simulation results of Fig 20. The bias of a channel has no strong effect on the uncorrected error rate.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 20. Decoder performance for varying relative bias b.

k = 12, β = 2, f_δ = bf where f₀ = f + bf, f₁ = f − bf. Dashed lines show the theoretical bound for the given bias, and the solid lines show the mean simulated results from the Horstein decoder. The decoder approaches the bound for any level of bias.

https://doi.org/10.1371/journal.pone.0233603.g020

6.3.1 Bursty channels and non-stationarity.

The Horstein decoder (in the binary case) assumes corruption by memoryless iid Bernoulli noise. However, many real assistive technology channels do not have independent white noise distributions. There are often strong slowly-varying time varying components to the noise introduced, for example from classifier drift in BCI [123], electrode drying in EMG [124] or illumination changes on vision-based systems.

A simple but versatile model of non-iid noise in a binary channel is the Gilbert-Elliot bursty channel model [125, 126], which is widely used in modelling bursty packet loss on networking systems (e.g. packet-based network channels subject to varying congestion [127]). The two state Gilbert model is constructed around a binary Markov chain switching between a good state G with error probability f_G and a bad state B with error probability f_B (Fig 23). The Markov chain has transition matrix: (13) and stationary distribution: (14)

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 23. The Gilbert-Elliot Markov chain for a bursty channel.

The Markov chain randomly transitions between a good state G and bad state B.

https://doi.org/10.1371/journal.pone.0233603.g023

We can apply this model to simulate the effect of non-stationarity on the Horstein decoder. Assuming that the good state G is perfect with no error f_G = 0, and the bad state B is always flipped f_B = 1, we can parameterise a Gilbert-Elliot model in terms of expected flip probability (average error rate) f and a “burstiness” t, t > 1. We can set: (15) Fig 24 illustrates the effect of increasing burstiness on the Horstein decoder. As the channel becomes less iid the uncorrected error rate goes up, but the number of decisions per bit decreases because the errors become more predictable. We conclude that non-iid noise—perhaps surprisingly given that the code is only optimal for memoryless channels—does not significantly affect the performance of the Horstein decoder if we consider the backspace-corrected rate R′. Increasing burstiness t decreases the raw decisions/bit R while uncorrected error rate e_k increases; these effects nearly perfectly cancel out.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 24. Effect of burstiness on decoding performance.

The plots show the effects of varying burstiness t and average error rate f on theHorstein decoder with k = 8, β = 8 from N = 10000 random simulations. Increasing burstiness leads to increased uncorrected error rates but a decrease in the decisions/bit. Consequently, the backspace-corrected entry rate is largely unaffected by burstiness.

https://doi.org/10.1371/journal.pone.0233603.g024

The Gilbert-Elliot Markov model can be generalised to good/bad states with other error probabilities and to multi-state non-stationary biased channel models where the one state may be “burstier” than the other and/or bias varies in good and bad states. We do not consider these extensions here.

6.3.2 Change of heart analysis.

The decoder is modelled with the assumption that the user has a specific, fixed intention for a target symbol s and then consistently produces inputs to drive the decoder towards that state until termination according to Equation 4.2.5. However, sometimes a user may start down a path of selecting some target s_a, but decide they really wanted to select another target s_b. This could be the result of an initial mistake in identifying targets, or a change in circumstances during the selection process. If the decoding process is long, completing a selection of the “wrong target”, undoing, then selecting the correct target will be frustrating. This can be seen as an extreme form of non-stationarity in error distribution.

The Horstein decoder is not designed for switches of intention, but sufficient level of tolerance β allows for some level of initially incorrect selection to be accommodated. We conducted a change of heart analysis with our simulator to quantify this performance, where we simulate users switching from intending s_a to s_b partway through selection, without completing the selection of s_a.

To analyse the effect of this we run simulations where we control:

• x_Δ the target separation, x_Δ = |d(s_a)−d(s_b)|, where d(s) is the function that maps target symbol centres to the unit interval [0, 1]. Larger x_Δ indicates the decoder must make a more radical change in the probability density to select s_b.
• λ The switchpointλ, 0 ≤ λ ≤ 1 controls when the change of heart is initiated. The simulator switches targets when decoder entropy H(X)<H_λ, where H_λ = −λ(k + β). For example, when λ = 0.5 the switch happens when the decoder is halfway to completion in terms of information accumulated.

Fig 25 illustrates the Monte Carlo simulations of the entropy decoder for a change of heart for H_λ = 0.5, x_Δ = 0.25. Following the change of heart, the decoder’s uncertainty gradually increases as inputs indicate a changed intention, then decreases as the new target becomes more certain. It is clear that the decoder can cope with changing targets, as long as β is sufficient. Fig 26 shows results of Monte Carlo simulations for a wide range of noise/decoder configurations. The ratio of inputs/bit from the “no change of heart” case R*/R, is shown, along with the absolute difference in error and the backspace-corrected ratio . The bit error rate f and the headroom f_h have no noticeable effect on the ability to recover from a change of heart, but λ, β and x_Δ affect the recovery. In the left panel, there is a clear decrease in the uncorrected error rate e_k as β increases (brighter colours), and this is sufficiently large that even with the additional overhead increased β implies, the backspace-corrected ratio improves for larger β, particularly when very late corrections are made (larger λ). There is a smaller effect for x_Δ (right panel)—smaller deviations are more easily tolerated because they reduce R*/R but have almost no effect on the uncorrected error rate e_k. As might be expected, small changes made early are easier to cope with, and a larger β can absorb more errors.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 25. Example entropy time series with a change of heart.

Change of heart occurs at λ = 0.5; x_Δ = 0.25(H_λ = −5). The decoder is configured with k = 8, β = 2; f = 0.15; f_h = 0.1, N = 250 trials, mean curve shown in blue. The simulation changes target when H_λ ≤ −5 to a target with separation x_Δ = 0.25. 99.2% of trials acquired the changed target sb correctly. There is a marked v shape to the curve as the decoder entropy increases after input starts to become incompatible with the original target s_a, and then decreases as sb is approached.

https://doi.org/10.1371/journal.pone.0233603.g025

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 26. Change of heart analysis.

This plot shows the effect of a sudden change of intention during selection. Coloured by β (left) and by x_δ (right). Simulations run with k = 8, f ∈ [0.0, 0.3], β ∈ {0, 1, 2, 4, 8}, f_h ∈ [0.0, 0.2], f′ = f + f_h, x_Δ ∈ [0.0, 1.0], λ ∈ [0.0, 1.0]. Each point represents the mean of N = 500 trials. λ represents the proportion of the selection at which the target intention changes (in terms of decoder entropy). x_Δ is the distance between the originally intended and final targets in the unit interval. Artificial jitter added to x values to separate points.

https://doi.org/10.1371/journal.pone.0233603.g026

6.4.1 Online adaptation for symmetric channels.

Section 4.2.11 introduced an adaptive algorithm to adapt channel statistics online. We ran numerical simulations, adapting f′ to match an unknown (randomly selected) true error rate f. The simulations used ϵ_n = 0.01 and δ_n = 0.005. Fig 27 summarises the results, showing sequential runs of k = 8, β = 8 decoding with f₀ = f₁ = f with random starting f′ ∈ [0.01, 0.4] and fixed random target f in each panel. Adaptation is relatively slow, taking around 100 symbols to converge for these parameters, but this would often be sufficient for slowly-varying channels.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 27. Online adaptation of the channel statistics.

A decoder with k = 8, β = 8 is used for selection. In each panel, the simulated error level f is held fixed, and the configured expected error level f′ is adapted online. Each panel shows 50 replications with random initial starting values for f′ (shown as circles at left), showing the decoder will converge regardless of the initialisation.

https://doi.org/10.1371/journal.pone.0233603.g027

6.4.2 Online adaptation for biased channels.

Adapting to biased channels is slightly trickier. We need to adjust the and based on the count of input bits b_i = 0 and b_i = 1, but these obviously depend on the specific target being acquired and there is not a convenient closed-form formula. However, we found a simple heuristic that can adapt to biased channels online. During selection, we record the count of b_i = 0, n₀ and the count of b_i = 1, n₁ received during the selection. Then, we use the numerical simulator to simulate selection of the same target—but with the current configured parameters as the simulated noise level—and count the number of 0s and 1s in this simulation, . This replicate simulation can be run multiple times and the results averaged to reduce variance. Then we update and as follows: (16) where ϵ_n is a small constant. Fig 28 shows examples of online adaptation to a step change in channel statistics, using ϵ_n = 0.0005, with a k = 8, β = 8 decoder. This simulated change is an extreme example of channel condition variations and a real user interface would typically adapt to more slowly varying components.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 28. Online adaptation of the channel statistics for a biased channel.

A decoder with k = 8, β = 8 is used for selection. In each panel, the decoder is initially run with and (both randomly chosen in the range [0.0, 0.35]). At symbol 100, f₀ and f₁ are changed to random values, and and are automatically adapted using Eq 16.

https://doi.org/10.1371/journal.pone.0233603.g028

6.5.1 Scenario A: Wheelchair controls.

A system is being developed for a wheelchair with four directional controls |S| = 4 = 2²; therefore k = 2. Errors obviously cannot be directly corrected after movement has happened, so the uncorrected error rate is taken to be 1 in 100; e_k < 0.01. The input is a BCI which is known to be heavily biased and expected to have f₀ = 0.01 and f₁ = 0.3. This estimate of accuracy is expected to be within a tolerance of f_h = 0.05, so a decoder is configured with and . Each binary classification takes 300ms, t_i = 0.3.

• Simulated performance (a) β is selected via bisection to achieve e_k = 0.01 with β = 2.3. R = 7.64 decisions/bit. Each wheelchair command will take T_k = 7.64 × 2 × 0.3 = 4.61s.
• Simulated performance (b) If we imagine that subsequent experiments are performed which suggest that the accuracy was mis-estimated, and the real channel is f₀ = 0.1, f₁ = 0.4; 10% of “left” inputs are flipped and 40% of “right” inputs are flipped. With the same decoder, we get R = 9.48, T_k = 5.69s but e_k = 0.08 (8% error rate).

6.5.2 Scenario B: Word selection.

A communication support system is being built which allows users to select one word at a time from a set of N = 1000 common requests, so k = 10 ≈ log₂(1000). Errors can be corrected by undoing the last word. The input is a eyebrow-switch which has f = 0.2, accurately estimated from extensive calibration. Each classification takes 500ms, t_i = 0.5.

• Simulated performance (a) With β = 0, R = 4.07, e_k = 0.08. The backspace-corrected rate is R′ = 4.99 decisions/correct bit. Each correct word will take T_k = 24.96s.
• Simulated performance (b) After a (hypothetical) trial, our imagined users indicate that they find backspace frustrating. We can use the simulator to model a reduced reliance on backspace by setting β = 7. This reduces the error rate to e_k = 0.003 at a cost of increasing R to 6.63. Each correct word will take T_k = 33.45s, and backspace will be required less than 0.3% of the time.

6.5.3 Scenario C: Classifier choice.

An interface to let users select a personal contact to phone from a list of |S| = 200 is being created. k = 8 ≈ log₂(200) and we assume a tolerable error level of e_k = 0.01 (one misdial every 100 calls); undo is not meaningful. Three classifiers for an assistive technology sensor have been developed. In all cases, a headroom of f_h = 0.03 is assumed to account for mis-calibration of classifier accuracy.

• (a) Fast, unreliable, biased f₀ = 0.15, f₁ = 0.4, t_i = 0.15
• (b) Moderate, some error f₀ = 0.1, f₁ = 0.1, t_i = 0.3
• (c) Slow, reliable f₀ = 0.01, f₁ = 0.06, t_i = 1.5

Simulated performance

• (a) β = 3, e_k = 0.007, R = 10.60. T_k = 12.72s/dial.
• (b) β = 2, e_k = 0.007, R = 2.93. T_k = 7.03s/dial.
• (c) β = 1, e_k = 0.002, R = 1.58, T_k = 18.95s/dial.

Classifier (b) will be significantly faster for this application.

7.2.1 Participants.

N = 20 participants (10 male, 10 female) were recruited locally. The participants were healthy adult volunteers without any relevant impairments. Each participant completed the study in a single session, while seated in laboratory conditions, controlling a laptop via keyboard commands. This research was approved by the University of Glasgow College of Science and Engineering ethics board. The ethics board approval number is CSE-01125. Written informed consent was obtained from all participants. Participants were offered a £10 reward for their participation. Each experimental trial took approximately one hour.

7.2.2 Visual display.

The visual interface appeared as the diagonal subdivision interface described in Section 5.1.2, similar to the prototype (d) in Fig 16; the exact interface layout is shown in illustrated in Fig 29. Linear zooming was used for the display, showing the smallest subsection of square such that at least of the 50% HDPI was spanned in each axis.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 29. The visual display used in the experiment.

A single target is shown in red, appearing at an initial random location. The target has sides of length 2⁻⁶. A diagonal split is used to elicit user responses and linear zooming is used. A progress bar shows the entropy drop remaining.

https://doi.org/10.1371/journal.pone.0233603.g029

A single targets were represented as visually as a red square in the 2D space. We did not test the effect of searching for labelled targets. The size of the targets was fixed in terms of the information required to identify them, which corresponds to a fixed visual area in the zooming interface. The interface was configured to simulate selecting from a twelve bit alphabet of symbols (1 from 4096). A “twelve bit” target is represented as a square of sides 2⁻⁶ × 2⁻⁶ inside a unit square and requires twelve bits to reliably identify, as 2¹² such targets will fit in a 1×1 unit square. Showing the true size of the targets in a linear zooming interface with twelve bit targets makes them very small (a few pixels across) at the initial fully zoomed out state of the interface. To make the target visible at all zoom levels, the target square was displayed as a fixed size when its true visual area would be too small to see reliably. As the display zoomed in, the target took on its true area. Fig 29 shows images of the experimental software. A progress bar showing the remaining entropy before termination was shown at the bottom of the screen.

7.2.3 Trial procedure.

Participants were asked to select the red square representing the target by selecting the left or right side of the visible dividing line. Participants pressed the [LEFT SHIFT] or [RIGHT SHIFT] keys to indicate a leftward or rightward movement, which would expand the space on the side of the diagonal line specified. Participants were instructed to press the key corresponding to the side of the divider the target was on; they were not given further instructions on the selection task. This process completed until the entropy of the decoder dropped by 12 bits, at which point the selection was determined to be correct if the decoder medians m_xi, m_yi both lay within the target square (i.e. the correct symbol was decoded on both axes) and incorrect otherwise. There is no explicit actuation of selection in this interface; that is, there is no equivalent of a mouse click that indicates a selection happens at a particular moment. Selection is implicitly performed once the decoder is sufficiently certain. The input was user-paced (asynchronous) and participants could wait as long as desired before pressing a key, and once a key was pressed the transition happened following a 300ms delay. The transition could not be interrupted or reversed once actuated, and the screen was grayed-out during this period. An interpolated zoom was used to transition between zoom states.

7.2.4 Tasks.

In each condition, the participants had to select six twelve-bit targets, each target having six bits of information in the x and y axes (total of twelve bits per target), for a total of 72 bits communicated per condition. User keyboard input was randomly flipped according to the channel configuration for each condition. For example, if condition specified f₀ = 0.05, f₁ = 0.25, the system would actually move to the right of the dividing line 5% of the time when the user pressed [LEFT SHIFT] and move left 25% of the time when [RIGHT SHIFT] was pressed. Once the termination criteria for each target was reached, participants were invited to take a short break.

7.2.5 Conditions.

Before beginning the experiment, every participant performed a training condition. In the training condition, no errors were introduced and the participant was allowed to discuss what was happening with the experimenter. An onscreen label was shown to indicate where the target was, and which key to press during the training session.

Following the training session, five different conditions were presented, each with different channel properties. The full set of conditions tested are shown in Table 1. These span a range of reliabilities, including moderate error rates (C-15-15); error rates that would be very challenging for most user interfaces (C-25-25); and extremely biased inputs (C-5-45) where one control is nearly non-functional. The presentation order of the conditions following the training session was randomised for each participant to mitigate learning effects.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. The experimental conditions for the human-in-the-loop experiment.

https://doi.org/10.1371/journal.pone.0233603.t001

7.2.6 Decoder.

The decoder was configured as a pair of Horstein decoders each with k = 6, β = 0 and , where f₀, f₁ are the simulated error levels. We set the headroom f_h = 0.02. For example, for C-5-45, we introduced errors to the keyboard input with f₀ = 0.05 and f₁ = 0.45, and configured the decoder to expect error rates of , .

7.2.7 Exclusions.

One participant failed to select any targets at all in the training condition, apparently due to a misunderstanding of the instructions, and was excluded from the study. The remaining 19 participants completed all tasks and their data is included in the analysis.

7.3.1 Terminology.

When we report results comparing experimental results to simulations or theoretical predictions, we report the comparison of the experimentally measured decisions/bit and uncorrected error rate against three theoretical models. We compare against , the maximum possible performance at the actual observed channel statistics, empirically measured from the user responses, which is the most meaningful prediction; , the bound using the configured statistics (the most pessimistic model, including the headroom f_h), and , the bound using the simulated statistics (i.e. the bit flip rate used to inject noise into the simulator), the most optimistic model. We use the following terms to distinguish user inputs and decoded selections: Target: One decoded 12 bit symbol s_i; selecting a target is a one from 4096 choice. Decision: A single binary input provided by the user, corresponding to one keypress. Bit: One bit of the entropy used to select a target. “Decisions per bit” means the number of keypresses required to select 1/12th of a target.

7.5.1 Can users approach the Shannon bound?

The most salient overall metric is the number of input bits required to produce one error-free output bit. Our simulator did not include a backspace function, but we can directly estimate the correction penalty required to get error-free output using Equation 4.1.2. This gives a directly comparable measure to the theoretical channel bounds. The key results are Table 5, which compares the backspace-corrected observed entry rates against the numerical simulation using the actual channel statistics and the Shannon bound for the actual statistics . The percentage of the Shannon bound achieved is also given, showing that the interface achieves approximately 50-75% of the theoretical maximum. The backspace-corrected decision/bit rates against all three of the theoretical models are summarised in Table 6, which compares the decisions per/bit across conditions, along with the theoretical minimum from Eq 2 at each of the actual, expected and configured models. Fig 31 shows a regression of the observed against the theoretical bounds for the actual and configured models. Performance is nearly linear across the full range of error rates, and is on average 54% of the theoretical upper bound for the configured statistics.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Backspace-corrected experimental rates against simulated backspace-corrected rate and Shannon bound, and fraction of simulator performance/Shannon bound achieved.

https://doi.org/10.1371/journal.pone.0233603.t005

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Backspace-corrected rate against theoretical Shannon upper bounds.

https://doi.org/10.1371/journal.pone.0233603.t006

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 31. Measured performance against Shannon upper bound.

is plotted against , for each condition. A line at α = 0:5 showing 50% of the Shannon bound is shown.

https://doi.org/10.1371/journal.pone.0233603.g031

Table 7 summarises the timing of the inputs including duration to select each 12 bit target, the number of keypresses in the whole condition (for six targets), the duration of each condition in seconds, and the average number of keypresses per second. There is no strong variation across conditions in terms of input timing. Fig 32 summarises the effort required to make each selection, including the number of binary inputs per bit successfully communicated and the mean time taken for each binary decision. Users showed little variation in generating inputs, suggesting they did not spend long pondering the correct decision to move towards the target.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 7. Timing of inputs.

All numbers in seconds. is duration of one decision (from prompt to keypress); T_min is the 300ms minimum delay enforced; is the time taken to enter one bit of information, on average.

https://doi.org/10.1371/journal.pone.0233603.t007

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 32. Decisions per bit and timing of decisions.

(Left) The number of decisions R required for each bit (Right) The time taken for each decision , which remained approximately constant across conditions. Red line shows the minimum fixed delay.

https://doi.org/10.1371/journal.pone.0233603.g032

7.5.2 Illustrations of entry process.

As a user interacts with the linear zooming interface, there is a visual expansion of the view corresponding to the concentration of probability density. Fig 33 illustrates the viewports displayed in one example trial after each keypress. The marginal probability densities p_x(x), p_y(x) for the X and Y axes after each keypress are shown. The gradual contraction of probability density around the target is clearly visible. Fig 34 shows how entropy of the PDFs decreases as each input is received during selection of a target, averaged across all users for each condition. As would be expected, the PDF decreases by twelve bits across the target selection process. The information rate is almost exactly the linear drop that would be predicted.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 33. PDF evolution form a random run from C-15-15.

(above) Colormap showing PDF after each input; darker indicates greater density (below) Ridge-plot of the same density sequence, where the height of the line is proportional to the log PDF; each line corresponds to a single decision. The maintenance of multiple hypotheses is visible.

https://doi.org/10.1371/journal.pone.0233603.g033

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 34. Mean entropy time series.

Each plot shows the mean (solid line), one standard deviation (shaded area) and the theoretical prediction (dashed line) of the entropy of the decoder’s PDF against decision number, for each of the experimental conditions.

https://doi.org/10.1371/journal.pone.0233603.g034

8.4.1 Monte Carlo simulations.

Our numerical simulations demonstrate that this decoder is near-optimal across a range of real-world conditions outside of the theoretical predictions. While it is relatively sensitive to calibration with true channel characteristics, it is possible to bound the channel with sufficient headroom to cope with minor fluctuations in reliability at a cost of some loss of input rate. The Horstein decoder can reduce the error to a level that a backspace decoder can “mop up” any remaining error and still retain control very close to the theoretical optimum. Our simulations show the approach works across a full spectrum of biased channels without modification and functions effectively even in the presence of non-stationary noise. In simulation, the algorithm can adapt online to changing signal conditions, including changes in bias. Our results with heavily biased channels are particularly promising as these are frequently encountered in marginal reliability input devices and adoption of this style of interface could render many otherwise frustrating inputs usable. The ability to integrate probabilistic classifiers, and hybrid input devices with infrequent reversal channels (such as EMG-triggered undo) make this an attractive fundamental component to build reliable assistive technology interfaces.

8.4.2 Human-in-the-loop user trials.

The user trials indicate that the numerical simulations generalise to the human-in-the-loop case, and that the interaction design based on nonlinear zooming is sufficiently transparent that non-expert users can immediately control systems with extreme input corruptions. User performance is very close to that predicted by numerical simulations and suggests that the interface is transparent to users.