Remote drivers as AI-coordinated groups

Automated vehicles promise a host of societal benefits, including dramatically improved safety, increased accessibility, greater productivity, and higher quality of life. In order to deliver on these promises, the vehicles must be able to operate and reason over a nearly infinite number of known and unknown potential conflict situations. By adopting lessons and experiences from the air traffic control system, we argue that using humans to supervise driverless vehicles is 1) technically feasible, 2) necessary to achieve safety goals, and 3) an available source of employment [10].

Fully autonomous (Level 5) vehicle control is a relatively distant goal, as it requires AI to both understand scenarios involving people and other objects in the environment and know how to respond. Current autonomous vehicles (AVs) can drive quite well in typical (frequently encountered) settings but fail in exceptional cases. Worse, these exceptional cases are often the most dangerous and may arise suddenly, leaving human drivers with only a couple of seconds, at most, to react—precisely a setting in which people can be expected to perform worst. Compared to precautionary takeovers, these sudden scenarios already comprise most of the disengagements that GM Cruise reported in 2018, accounting for 53.5% of autonomous driving interruptions [11]. As self-driving capabilities improve, non-time-critical disengagements should become increasingly rare. For uncommon situations that are not time-critical (e.g., when the vehicle is stopped), remote human drivers can navigate and return the AV to a setting from which it can resume autonomous control. However, in critical settings where rapid response is needed (e.g., sudden events that could result in collision), asking a remote human driver to step in is not viable due to the latency period required for a driver to perceive/understand the context of a scenario and react, as well as the network latency involved in transmitting video and responses between safety drivers and vehicles.

Our team’s current work explores how AI-coordinated groups of remote drivers might best attain superhuman collective performance, overcoming previously insurmountable barriers of human and network latency. By leveraging AVs’ “understanding” of the world (e.g., state space representation, transition model), human effort/insight can be guided toward reachable future states of the world. This allows us to simulate potential situations mere seconds or even fractions of a second before they occur, and cache responses indicating to the AV how a human would respond locally in such a situation—the result being an ability to leverage human responses in milliseconds rather than seconds, opening a whole new frontier for possible applications to critical, life-saving scenarios. See [9] for more details on the foundational techniques underlying our approach.

Human-assisted AI

With the rise of artificial intelligence as a service, human-backed algorithms at scale have become the norm rather than the exception for intelligent systems. Google, Facebook, Apple (Siri), Samsung, Bloomberg, and countless other organizations use large groups of human annotators and checkers to ensure their intelligent services’ quality and reliability. However, while reliability and accuracy are important in all these settings, none of the prior methods has leveraged low-latency, real-time systems to provide input faster than any one person alone could.

Here’s how it can work—within five seconds or less:

• Software in the autonomous vehicle would analyze real-time vehicle data and electronically estimate the likelihood of “disengagement”—due to a situation in which the car’s automated systems might need human help—10–30 seconds in the future.
• If the likelihood exceeds a pre-set threshold, the system contacts a remotely located control center, sending data from the car. One or more remote drivers are assigned to resolve the pending disengagement.
• The control center’s system analyzes the car’s data, generates several possible scenarios, and provides them to several human supervisors situated in driving simulators.
• The remote drivers respond to the simulations and their responses are sent back to the vehicle.
• The vehicle now has a library of human-generated responses that it can choose from instantaneously, based on information from on-board sensors.

Previous work establishes the feasibility of this approach in low-latency environments. Responses in < 0.3 s (roughly equivalent to the human reaction time, but more stable and less subject to distraction) by combining reinforcement learning with crowd feedback [9]. More work is needed to formally model collective input mediation strategies that can optimize for either input reliability or low latency [13].

The proposed system asks groups of remote drivers to help concurrently with a given monitored or control task in as little as ∼350 ms of a need’s arising. With video-based remote control latencies as low as 100 ms, total latency for control can be under 0.5 s. Thus, in any environment where we can predict possible outcomes 0.5 s in the future, “instant” responses become possible. This scope of settings is far larger than those we can observe prior to deployment. Prior work learned how to effectively interleave and combine groups’ input over short time spans [9]. More work is needed to modify these approaches to workloads with short, sudden bursts of requests.

To improve response speed, methods are needed to directly leverage the AV’s ability to understand possibilities that may arise in real settings (even when the system does not know how to respond to a possible setting) to pre-fetch possible configurations of the world. Using these future states, remote drivers can (in parallel) provide feedback before a system needs to know what action to take. What makes this possible is the speed of existing real-time staffing approaches. While 0.5 s may be a relatively slow response time for an engaged driver to respond to an event (usually accomplished within 200-300 ms), an ability to respond this quickly means that we need only pre-fetch future states of the world.

Recent work has shown that just-in-time (JIT) training can result in an average response time below 3.5 ms, reducing latency by three orders of magnitude [9]. Further, the collective response is more likely to be correct than a single person’s (i.e., a local driver’s) response [14]. The remaining challenge is to scale up from laboratory settings (simple, fully controlled problems) to real-world settings with massive state spaces. To improve scalability, research is needed on workload, arousal levels, and task routing optimization to utilize remote drivers’ time and attention effectively and efficiently. This work must cost-effectively optimize the next selected set of states, as well as worker availability and response speed to reduce the horizon needed to train a system for an event. Improved machine learning algorithms are also required to integrate task- and scenario-specific knowledge, to form teams of remote drivers who have performed well in similar settings (e.g., navigating a front-wheel drive car in the rain with minor driver distraction). More generally, this building block is concerned with group efficiency, collective human performance on critical tasks, and ad hoc team formation to do highly skilled work.

The human element

Air traffic controllers are well known to have one of the most stressful jobs in the world [15]. The job is both cognitively and physically challenging, as they are responsible for maintaining and managing all incoming and outgoing air traffic, which requires highly sustained concentration and decision making. We expect the role of remote driver will be similarly demanding and challenging to deal with all routine and unexpected situations. In general, under stress, human cognitive and perceptual motor performance are both impaired [16]. Human memory, especially the encoding and maintenance processes, are very sensitive to stress effects, due to reduced resource capacity [16]. Negative effects of stress on perceptual and psycho-motor tasks have also been reported consistently. For example, Scerbo [17] examined human sustainable attentional processes and suggested that vigilance under stress can lead to decremental motor performance accuracy and increased response time. Although they constitute distinct processes and outcomes, both human judgment and decision making under generally stressful conditions tend to become less flexible, with fewer alternatives considered [18, 19].

While much research has been done on air traffic controllers, more will be needed to understand how remote drivers perform on monitoring and operating tasks. Performance measures should include evaluating accuracy, service time, cognitive load, and fatigue resulting from processing service requests. Understanding interactions of the above factors within a single person or team is critical to the safety performance of the teleoperations system. The findings of the human factors research should inform a licensing standard for remote drivers. The state of California statute requires the testing permit holder to document and certify that the remote drivers have adequate training and education. Such an approach creates standards of safety and certification needed to create a professional job category for remote drivers. Further research is needed to estimate the number of available workers with requisite cognitive and emotional skills, and judgement to become remote drivers.

System operations and industrial organization

This teleoperations system might have several possible operating models. In one, private companies who own or operate driverless vehicles would also own and operate remote assistance centers. This model is similar to the current GM OnStar system, in which only GM-equipped vehicles can access OnStar. This approach would allow the industry to compete on safety. The teleoperations system would be a feature that users or fleet purchasers can choose much like adaptive cruise control (ACC). However, this approach would likely lead to balkanized teleoperations systems that do not talk to each other. Standards groups like the Society of Automotive Engineers (SAE) or the International Organization for Standardization (ISO) would need to set standards to promote interoperability and communication between teleoperations systems. Additionally, this approach would require employing more drivers, due to a scale smaller than a centralized system.

Another model resembles the air traffic control system operated by the Federal Aviation Administration (FAA). Vehicle support tasks are split between workers at various local, regional, and national centers. As a vehicle moves between various locations, oversight is handed off between centers. A private firm under contract with the federal government could operate this system. If an example is wanted, a private nonprofit runs the nationwide air traffic control system in Canada [20]. This type of arrangement, under which oversight is handed off between different regions, can be modeled in a way that is similar to wireless communication networks [21, 22]. In this type of model, one separates the state space into two parts: One dimension consists of those drivers on the road who need no assistance; the other dimension consists of drivers who do assistance. This type of model has also been used in the context of healthcare, to model emergency-room patients in critical and stable conditions [23].

Blocking model perspective

We model the number of remote drivers needed as an Erlang-loss queueing system. We assume, for the sake of simplicity, that requests for service to our dynamic queueing system is driven by a stationary Poisson process with rate λ. This allows us to perform staffing calculations for peak hour demand. In the subsequent subsection we discuss how to extend these computations to time-varying staffing settings, but for now let us consider the peak hour. (1) In this formula, the offered load q is defined as the arrival rate λ of requests for assistance divided by the rate of service μ. Moreover, the blocking probability can be calculated as the conditional probability of an infinite server queue’s being in state C given that the queue length never exceeds C. Using this formula, one observes that in order to satisfy 1 − ϵ fraction of all service requests immediately, a minimum number of drivers (C) is needed such that B_C(q) ≤ ϵ. Finding such a C normally would be difficult numerically because of the factorial in the blocking probability expression. A method exists, however, to solve for the number of drivers recursively. Known as the Erlang-B recursion, an expression for this method is shown below: (2) In addition to the recursion, one can derive simple, accurate approximations for the number of drivers needed to satisfy 1 − ϵ of all service requests immediately. Also of note: the Erlang-B formula is valid for the M/G/C/C queue, where the service time distribution is general, since the Erlang-B formula is beautifully insensitive to the distribution of this random variable. However, insensitivity is not true for the vehicle arrival process. Thus, for the G/G/C/C queue, the Erlang-B formula needs not hold, so approximations are needed to compute the number of drivers necessary to immediately satisfy percent of all service requests. When inter-arrival and service time distributions are exponential, an approximation of the minimum number of required drivers [25, 26] equals (3) in which x satisfies the following inequality (4) This formula relies on the probability density function (PDF), ϕ(⋅), and the cumulative distribution function (CDF), Φ(⋅), of a standard normal random variable, which can be quickly calculated. To derive this formula, we start with the Erlang-B formula in terms of the infinite server queue. Note that when the arrival rate is large or demand for servers is sufficient, the infinite server queue can be approximated by a Gaussian distribution. The distribution of the infinite server queue in steady state is Poisson, satisfying the condition that all of its cumulant moment equal the offered load [27]. In particular, the mean equals the variance, which implies that Q_∞ ≈ N(q, q). Using this Gaussian approximation, we can derive an approximate formula for the number of drivers needed to satisfy the blocking probability requirements: (5) Now, by inverting the function φ(x), we see that the number of drivers needed to satisfy our blocking probability is approximately equal to: (6) When the mean does not equal the variance, Q_∞ ≈ N(q, v), which yields a slightly different formula for the approximate number of drivers needed to satisfy the blocking probabilities. Following the same approach as above, we find: (7) By inverting this new function φ(x), we find that the number of drivers needed to satisfy our blocking probability is now approximately equal to: (8) This expression is equivalent to the Hayward approximation [26, 28, 29, 30]. Why does the Erlang-B formula need to be so modified? For one thing, inter-arrival times and service times are not expected to be independent and identically distributed. Imagine a fallen tree blocking the road: The first vehicle whose driver observes the downed tree might disengage or need assistance. However, vehicles close behind might also require assistance, for the same reason. Thus, arrivals might tend to cluster during events and require similar service times since they reference the same type of disengagement.

Extension to time-varying calculations

In this section, we will discuss the generalization to non-stationary arrival rates. In doing so, we derive closed form formulas for mean queue length of the M_t/G/∞ queueing model. These results are not new as they were derived in [31, 32] for the time varying infinite server queue. Eick et al. use the properties of the Poisson arrival process and use Poisson random measure arguments to show that the M_t/G/∞ queue Q_∞(t), has a Poisson distribution with time varying mean q(t) that is known [32]. The infinite server queue is an important model to study despite it having an infinite number of servers since it represents the queueing process as if there were an unlimited amount of resources to satisfy the demand process. In fact, [32] show that q(t) has the following probabilistic representation (9) where λ(u) is the time varying arrival rate at time u, S represents a service time with distribution G, , and S_e is a random variable with distribution that follows the stationary excess of residual-lifetime CDF G_e, defined by (10) When the service time distribution is exponential, we know that the mean queue length, q(t), solves the ordinary differential equation (11) In fact, since the differential equation is linear non-homogeneous ordinary differential equation, the solution is given by (12) Moreover, from the standard theory of infinite server queues, the distribution of the queue length process is Poisson with mean q(t) when initialized with a Poisson number customers or initialized at zero.

Recent work by [33] and [34] uses the infinite server queue to develop staffing algorithms for multi-server queues. Like in the stationary case, the number of servers needed to achieve a blocking probability of ϵ is given by the time varying function (13) From [35] and [34], it has been shown that Eq 13 can be used to stabilize the blocking and delay probabilities regardless of the arrival rate and in some cases the arrival and service rate distributions. We find it also important to make the comment that it clearly suffices to analyze a stationary model for staffing purposes if the arrival rate is given by λ = sup_0≤s≤T λ(s). One can observe the similarity between Eqs 13 and 6. Using this arrival rate will certainly produce an overestimate of the number of servers needed, however, it will most definitely satisfy the probabilistic constraints. Thus, this both justifies our peak hour analysis and demonstrates how it can be extended to time-varying settings.

Delay model perspective

In addition to the Erlang-B model, we can also use the Erlang-C model for situations in which vehicles wait for an agent to provide service. In this section, we assume the inter-arrival and service time distributions are exponential, with rates λ and μ, respectively. Under these assumptions, the probability of delay is given by the following expression: (14) We can exploit our knowledge of the M/M/C queue by using its conditional waiting time distribution, which we know to be exponential. Thus, we can also know that: (15) Solving for the number of drivers to satisfy the excessive wait probability yields: (16) Similarly, we can use the expected delay formula for the M/M/C queue to derive the number of drivers that would satisfy a bound on the expected delay for a driver experiencing disengagement: (17) Solving for the number of drivers to satisfy the expected conditional delay yields: (18)

Calculating staffing levels from data

Table 2 shows both the annual passenger miles driven and the estimated number of remote drivers needed per hour to handle disengagements under different staffing models. As discussed at the beginning of this Materials and Methods section, the data in this table were sourced from the 2017 National Household Travel Survey (NHTS). The rate of one disengagement per 11,000 miles is based on 2018 California disengagement reports. Table 2 differs from Table 1 in presenting the number of remote drivers for some of the largest cities in the nation for a greater variety of staffing models. From left to right, we have the city name, annual number of miles driven, percent of miles driven nationwide, remote drivers given by the Erlang-B model (with blocking probability 0.001), remote drivers given by the probability of delay in the Erlang-C model (with delay probability 0.001), remote drivers given by the conditional mean wait of the Erlang-C model (with a target wait of 20 s), remote drivers given by the conditional excessive wait of the Erlang-C model (with only 0.1% of waiting times exceeding 60 s), and conditional Gaussian approximation using peakedness (variance to mean ratio) parameters {0.25, 0.5, 1, 2, 4}. Because the ratio of the variance to the mean serves as a measure of under- or over-dispersion, cases in which the peakedness parameter is greater than one approximate models with bursts of arrivals [36–39]. Table 2 indicates that most of the queueing models suggest staffing a similar number of remote drivers. The largest differences in number of remote servers occur in our Gaussian approximation for different peakedness parameters. Thus, we see that if the variance of disengagement interarrival times is high, then a larger number of remote drivers is needed.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Estimated number of remote drivers needed in select cities nationwide during peak travel time, under five staffing models.

https://doi.org/10.1371/journal.pone.0232837.t002