Application of Haptic Feedback Technology in Automotive Information Systems

doi:10.12141/j.issn.1000-565X.220548

Abstract

Abstract:

In recent years, cars have been becoming more and more intelligently connected, and modern vehicles have been transformed into mobile computer platforms with information systems such as navigation systems, entertainment systems, climate control systems, and vehicle performance systems. The amount of information that drivers can access and interact with in the vehicle has increased significantly. However, operating a wide range of automotive information systems while driving can distract the driver’s visual and auditory attention, causing distracted driving and increasing the likelihood of traffic accidents. The use of haptic modalities while driving can complement audiovisual modalities, reduce the driver’s audiovisual burden, and improve driving safety, interaction reliability and comfort. On the basis of summarizing the development history of haptic feedback technology in automotive applications, this paper classified the specific types of in-vehicle interaction activities and haptic modalities that can be displayed in the vehicle, and analyzed the research progress of haptic feedback technology in automotive information systems from three directions, namely, in-vehicle haptic warning system, in-vehicle haptic navigation system, and in-vehicle haptic operation assistance system. It also summarized the main sources of haptic excitation in in-vehicle warning systems, navigation systems and operation assistance systems, respectively. And by analyzing the relevant literature, it expatiated the experimental environment, evaluation indexes, evaluation methods and results of the research on the influence of automotive haptic information systems on driving performance. Based on the shortcomings of existing technologies and research methods, the paper discussed and prospected the future research of automotive haptic information system from the perspectives of the location of haptic feedback excitation signals, the selection of modes and parameters, the study of behavioral adaptation problems, the improvement of the experimental environment, the consideration of different factors affecting haptic perception, and the integration of multiple automotive haptic information systems.

Key words: haptic feedback technology, automotive haptic information system, performance evaluation method, status of research, automotive driving safety

CLC Number:

TP399

SUN Xiaoying, ZHAO Yinan. Application of Haptic Feedback Technology in Automotive Information Systems[J]. Journal of South China University of Technology(Natural Science Edition), 2023, 51(8): 98-109.

References 88

1	宋爱国，田磊，倪得晶，等．多模态力触觉交互技术及应用［J］．中国科学F辑，2017，47（9）：1183-1197.
	SONG Aiguo， TIAN Lei， NI Dejing，et al ．Multi-mode haptic interaction technique and its application ［J］．Chinese Science F Series，2017，47（9）：1183-1197.
2	GAFFARY Y， LÉCUYER A ．The use of haptic and tactile information in the car to improve driving safety：a review of current technologies ［J］．Frontiers in ICT，2018，5（12）：1-11.
3	JANSSEN W J， NILSSON L ．An experimental evaluation of in-vehicle collision avoidance systems［J］．Mmw Fortschritte Der Medizin，2013，143（9）：47-48.
4	Volvo，Pilot Assist and Lane Keeping Aid ［EB/OL］．［2022-06-20］．.
5	KERN D， PFLEGING B ．Supporting interaction through haptic feedback in automotive user interfaces ［J］．Interactions，2013，20（2）：16-21.
6	BOLL S， ASIF A， HEUTEN W ．Feel your route：a tactile display for car navigation［J］．IEEE Pervasive Computing，2011，10（3）：35-42.
7	STRAYER D L， DREWS F A ．Cell-phone induced driver distraction［J］．Current Directions in Psychological Science，2010，16（3）：128-131.
8	BENEDETTO A， CALVI A， D’AMICO F ．Effects of mobile telephone tasks on driving performance：a driving simulator study ［J］．Advances in Transportation Studies，2011，26（26）：29-44.
9	WICKENS C D ．Multiple resources and performance prediction ［J］．Theoretical Issues in Ergonomics Science，2002，3（2）：159-177.
10	VAN ERP J B F， VAN VEEN H ．Vibro-tactile information presentation in automobiles［C］∥ Procee-dings of Eurohaptics．Paris：Eurohaptics Society，2001：99-104.
11	PETERMEIJER S M， ABBINK D A， MULDER M，et al ．The effect of haptic support systems on driver performance：a literature survey ［J］．IEEE Transactions on Haptics，2015，8（4）：467-479.
12	LAO Y T， ZHANG G H， WANG Y H，et al ．Generalized nonlinear models for rear-end crash risk analysis ［J］．Accident Analysis & Prevention，2014，62（1）：9-16.
13	TAN H Z， GRAY R， YOUNG J J ．A haptic back display for attentional and directional cueing ［J］．Haptics E，2003，3（1）：1-20.
14	HO C， TAN H Z， SPENCE C ．Using spatial vibrotactile cues to direct visual attention in driving scenes ［J］．Transportation Research Part F Psychology & Behaviour，2005，8（6）：397-412.
15	MORRELL J， WASILEWSKI K ．Design and evaluation of a vibrotactile seat to improve spatial awareness while driving ［C］∥ Proceedings of the Haptics Symposium．Waltham：IEEE，2010：281-288.
16	GRAH T，EPP F， MESCHTSCHERJAKOV A，et al ．Dorsal haptic sensory augmentation：fostering drivers awareness of their surroundings with a haptic car seat ［C］∥ Proceedings of the International Conference on Persuasive Technology．Salzburg：IEEE，2016：59-62.
17	LÖCKEN A， BUHL H， HEUTEN W，et al ．TactiCar：towards supporting drivers during lane change using vibro-tactile patterns ［C］∥ Proceedings of the Adjunct the 7th International Conference．Nottingham：ACM，2015：32-37.
18	OCHIAI Y， TOYOSHIMA K ．Invisible feet under the vehicle ［C］∥ Proceedings of the 3rd Augmented Human International Conference．Megève：ACM，2012：1-2.
19	FITCH G M， HANKEY J M， KLEINER B M，et al ．Driver comprehension of multiple haptic seat alerts intended for use in an integrated collision avoidance system ［J］．Transportation Research Part F：Traffic Psychology and Behaviour，2011，14（4）：278-290.
20	ADELL E， VARHELYI A， HJAELMDAHL M ．Auditory and haptic systems for in-car speed management-a comparative real life study ［J］．Transportation Research Part F：Traffic Psychology and Behaviour，2008，11（6）：445-458.
21	ADELL E， VÁRHELYI ANDRÁS， FONTANA M D，et al ．Test of HMI alternatives for driver support to keep safe speed and safe distance-a simulator study ［J］．The Open Transportation Journal，2008，2（1）：53-64.
22	JENSEN M J， TOLBERT A M， WAGNER J R，et al ．A customizable automotive steering system with a haptic feedback control strategy for obstacle avoidance notification ［J］．IEEE Transactions on Vehicular Technology，2011，60（9）：4208-4216.
23	ITOH M， INAGAKI T， TANAKA H ．Haptic steering direction guidance for pedestrian-vehicle collision avoidance ［C］∥ Proceedings of the 2012 IEEE International Conference on SMC．Seoul：IEEE，2012：3327-3332.
24	STRAUGHN S M， GRAY R， TAN H Z ．To go or not to go：stimulus-response compatibility for tactile and auditory pedestrian collision warnings ［J］．IEEE Transactions on Haptics，2009，2（2）：111-117.
25	RACINE D P， CRAMER N B， ZADEH M H ．Active blind spot crash avoidance system：a haptic solution to blind spot collisions ［C］∥ Proceedings of the IEEE International Symposium on Haptic Audio-visual Environments & Games．Phoenix：IEEE，2010：1-5.
26	JAEMIN C A， GUNHYUK P B， JONGMAN S B，et al ．Efficacy of haptic blind spot warnings applied through a steering wheel or a seatbelt-sciencedirect ［J］．Transportation Research Part F：Traffic Psychology and Behaviour，2013，21（4）：231-241.
27	MATTES S ．The lane change task as a tool for driver distraction evaluation［J］．Quality of Work and Pro-ducts in Enterprises of the Future，2003，57（60）：1-30.
28	中国汽车工程学会．车道偏离预警系统［EB/OL］．（2015-04-02）［2022-08-01］．.
29	SUZUKI K， JANSSON H ．An analysis of driver’s steering behaviour during auditory or haptic warnings for the designing of lane departure warning system［J］．Jsae Review，2003，24（1）：65-70.
30	ONIMARU S， KITAZAKI M ．Visual and tactile information to improve drivers’ performance［C］∥ Proceedings of the IEEE Virtual Reality Conference．Massachusetts：IEEE，2010：295-296.
31	STANLEY L M ．Haptic and auditory cues for lane departure warnings ［J］．Human Factors & Ergonomics Society Annual Meeting Proceedings，2006，50（22）：2405-2408.
32	KURIHARA Y， HACHISU T， SATO M，et al ．Periodic tactile feedback for accelerator pedal control ［C］∥ Proceedings of the World Haptics Conference．Daejeon：IEEE，2013：187-192.
33	中国汽车工程学会．超速行驶［EB/OL］．（2015-05-11）［2022-08-01］．.
34	ADELL E， VARHELYI A， HJAELMDAHL M ．Auditory and haptic systems for in-car speed management-a comparative real life study ［J］．Transportation Research Part F：Traffic Psychology and Behaviour，2008，11（6）：445-458.
35	VLASSENROOT S， BROEKX S， MOL J D，et al ．Driving with intelligent speed adaptation：final results of the Belgian ISA-trial ［J］．Transportation Research Part A：Policy and Practice，2007，41（3）：267-279.
36	JAMSON A H， HIBBERD D L， MERAT N ．Interface design considerations for an in-vehicle eco-driving assistance system［J］．Transportation Research Part C Emerging Technologies，2015，58（1）：642-656.
37	JAMSON A H， HIBBERD D L， MERAT N ．The design of haptic gas pedal feedback to support eco-driving ［C］∥ Proceedings of the 7th International Driving Symposium on Human Factors in Driver Assessment，Trai-ning，and Vehicle Design． Leeds：University of Lowa，2013：264-270.
38	HIBBERD D L， JAMSON A H， JAMSON S L ．The design of an in-vehicle assistance system to support eco-driving［J］．Transportation Research Part C：Emerging Technologies，2015，58（Part D）：732-748.
39	BIRRELL S A， YOUNG M S， WELDON A M ．Vibrotactile pedals：provision of haptic feedback to support economical driving ［J］．Ergonomics，2013，56（2）：282-292.
40	VAN ERP J B F， VAN VEEN H A H C ．Vibrotactile in-vehicle navigation system ［J］．Transportation Research Part F：Traffic Psychology and Behaviour，2004，7（4/5）：247-256.
41	HOGEMA J H， VRIES S，ERP J，et al ．A tactile seat for direction coding in car driving：field evaluation ［J］．IEEE Transactions on Haptics，2010，2（4）：181-188.
42	HWANG J， CHUNG K， HYUN J，et al ．Development and evaluation of an in-vehicle haptic navigation system ［J］．Lecture Notes in Electrical Engineering，2012，180（1）：47-53.
43	KERN D， MARSHALL P， HORNECKER E，et al ．Enhancing navigation information with tactile output embedded into the steering wheel ［C］∥ Proceedings of the Springer Berlin Heidelberg．Berlin：Pervasive Computing，2009：42-58.
44	MEDEIROS-WARD N， COOPER J M， DOXON A J，et al ．Bypassing the bottleneck：the advantage of fingertip shear feedback for navigational cues ［J］．Human Factors & Ergonomics Society Annual Meeting Proceedings，2010，54（24）：2042-2047.
45	HWANG S， RYU J H ．The haptic steering wheel：vibro-tactile based navigation for the driving environment ［C］∥ Proceedings of the IEEE International Conference on Pervasive Computing & Communications Workshops．Mannheim：IEEE，2010：660-665.
46	ASIF A， BOLL S ．Where to turn my car？：comparison of a tactile display and a conventional car navigation system under high load condition ［J］．ACM，2010，50（3）：64-71.
47	ASIF A， HEUTEN W， BOLL S ．Exploring distance encodings with a tactile display to convey turn by turn information in automobiles ［C］∥ Proceedings of the Nordic Conference on Human-computer Interaction．New York：ACM，2010：32-41.
48	BREITSCHAFT S J， PASTUKHOV A， CARBON C C ．Where’s my button？evaluating the user experience of surface haptics in featureless automotive user interfaces ［J］．IEEE Transactions on Haptics，2021，15（2）：292-303.
49	CASE A ．The hidden cost of touchscreens ［EB/OL］．（2020-10-23）［2022-08-01］．.
50	RAMNATH R， KINNEAR N， CHOWDHURY S，et al ．Interacting with android auto and apple interacting with android auto and apple car play when driving：the effect on driver performance［EB/OL］．（2021-02-19）［2022-08-01］．.
51	LI J. Why touchscreens in cars don’t work ［EB/OL］．（2020-10-22）［2022-08-01］．.
52	PITTS M J， BURNETT G， SKRYPCHUK L，et al ．Visual-haptic feedback interaction in automotive touchscreens ［J］．Displays，2012，33（1）：7-16.
53	RICHTER H， ECKER R， DEISLER C，et al ．HapTouch and the 2+1 state model：potentials of haptic feedback on touch based in-vehicle information systems ［C］∥ Proceedings of the International Conference on Automotive User Interfaces & Interactive Vehicular Applications．Pennsylvania：ACM，2010：72-79.
54	MULLENBACH J， BLOMMER M， COLGATE J E，et al ．Reducing driver distraction with touchpad physics ［J］．Driving Simulators，2013，1（1）：3-12.
55	SPIES R， BLATTNER A， LANGE C，et al ．Measurement of driver's distraction for an early prove of concepts in automotive industry at the example of the development of a haptic touchpad ［J］．International Conference on Human-computer Interaction：Interaction Techniques & Environments，2011，2（1）：125-132.
56	AUDI A G，Die neue form der haptik ［EB/OL］．［2023-06-16］．.
57	KESSELS C ．An in-depth look at the UX of the porsche taycan［EB/OL］．（2019-11-07）［2022-08-01］．.
58	SCHÄFER P ．Neue mercedes-benz S-Klasse nutzt zweite generation des MBUX ［EB/OL］．（2020-07-13）［2022-08-01］．.
59	POUPYREV I， MARUYAMA S， REKIMOTO J，et al.Haptic technology ［EB/OL］．（2021-02-14）［2022-07-01］．.
60	LOPREWTI P， COLGATE E， PESHKIN M，et al.TanvasTouch for automotive：feel more than the road ［EB/OL］．（2021-02-14）［2022-07-01］．.
61	KOO J， KWAC J， JU W，et al ．Why did my car just do that？explaining semi-autonomous driving actions to improve driver understanding，trust，and performance ［J］．International Journal on Interactive Design and Manufacturing （IJIDeM），2015，9：269-275.
62	HIROKAWA M， UESUGI N， FURUGORI S，et al ．Effect of haptic assistance on learning vehicle reverse parking skills ［J］．IEEE Transactions on Haptics，2014，7（3）：334-344.
63	PROFUMO L， POLLINI L， ABBINK D A ．Direct and indirect haptic aiding for curve negotiation ［C］∥ Proceedings of the IEEE International Conference on Systems．Manchester：IEEE，2013：1846-1852.
64	WINTER J， HAPPEE R， MARTENS M H，et al ．Effects of adaptive cruise control and highly automated driving on workload and situation awareness：a review of the empirical evidence ［J］．Transportation Research Part F：Traffic Psychology and Behaviour，2014，27（B）：196-217.
65	BLASCHKE C， BREYER F， FAERBER B，et al ．Driver distraction based lane-keeping assistance ［J］．Transportation Research Part F：Psychology and Beha-viour，2009，12（4）：288-299.
66	MOHEBBI R， GRAY R， TAN H Z ．Driver reaction time to tactile and auditory rear-end collision warnings while talking on a cell phone ［J］．Human Factors，2009，51（1）：102-110.
67	NAVARROA J， MARSA F， HOCA J M，et al ．Evaluation of human-machine cooperation applied to lateral control in car driving ［C］∥ Proceedings of the World Congress of the International Ergonomics Association．Maastricht：ResearchGate，2006：2-7.
68	HOC J M， MARS F， MILLEVILLE-PENNEL I，et al ．Human-machine cooperation in car driving for lateral safety：delegation and mutual control［J］．Le Travail Humain，2006，69（2）：153-182.
69	EGE E S， CETIN F， BASDOGAN C ．Vibrotactile feedback in steering wheel reduces navigation errors during GPS-guided car driving ［C］∥ Proceedings of the World Haptics Conference．Turkey：IEEE，2011：345-348.
70	ASIF A， BOLL S， HEUTEN W ．Right or left：tactile display for route guidance of drivers ［J］．Information Technology，2012，54（4）：188-198.
71	NAVARRO J， MARS F， HOC J M ．Lateral control support for car drivers：a human-machine cooperation approach ［C］∥ Proceedings of the European Confe- rence on Cognitive Ergonomics：Invent！Explore．London：ACM，2007：28-31.
72	ROB G， CRISTY H， CHARLES S，et al ．A comparison of different informative vibrotactile forward collision warnings：does the warning need to be linked to the collision event？［J］．PLoS ONE，2014，9（1）：e87070.
73	SCOTT J J， GRAY R ．A comparison of tactile，visual，and auditory warnings for rear-end collision prevention in simulated driving ［J］．Human Factors the Journal of the Human Factors and Ergonomics Society，2008，50（2）：264-275.
74	LEE J D， HOFFMAN J D， HAYES E ．Collision warning design to mitigate driver distraction ［C］∥ Proceedings of the Conference on Human Factors in Computing Systems．Vienna：DBLP，2004：65-72.
75	HO C， SPENCE C， TAN H Z ．Warning signals go multisensory［C］∥ Proceedings of the HCI International. Las Vegas：ACM，2005：1-10.
76	PITTS M J， WILLIAMS M A， WELLINGS T，et al ．Assessing subjective response to haptic feedback in automotive touchscreens ［C］∥ Proceedings of the 1st International Conference on Automotive User Interfaces and Interactive Vehicular Applications．Essen：ACM，2009：11-18.
77	PITTS M J， BURNETT G E， WILLIAMS M A，et al ．Does haptic feedback change the way we view touchscreens in cars？［C］∥ Proceedings of the International Conference on Multimodal Interfaces & the Workshop on Machine Learning for Multimodal Interaction．Beijing：ACM，2010：1-4.
78	RYDSTRÖM A， GRANE C， BENGTSSON P ．Driver behaviour during haptic and visual secondary tasks ［C］∥ Proceedings of the 1st International Conference on Automotive User Interfaces and Interactive Vehicular Applications．Essen：ACM，2009：121-127.
79	GRANE C， BENGTSSON P ．Haptic addition to a visual menu selection interface controlled by an in-vehicle rotary device ［J］．Advances in Human-Computer Interaction，2012，2012（1）：1-12.
80	TUNCA E， FLEISCHER R， SCHMIDT L，et al ．Advantages of active haptics on touch surfaces ［C］∥ Proceedings of the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications．New York：Association for Computing Machinery，2016：137-144.
81	TUNCA E， ZOLLER I， LOTZ P ．An investigation into glance-free operation of a touchscreen with and without haptic support in the driving simulator ［C］∥ Proceedings of the 10th International Conference．Toronto：ACM，2018：332-340.
82	BERUSCHA F， KRAUTTER W， LAHMER A，et al ．An evaluation of the influence of haptic feedback on gaze behavior during in-car interaction with touch screens ［C］∥ Proceedings of the World Haptics Conference．Munich：IEEE，2017：201-206.
83	WEDDLE A B， YU H，Confirmation haptics for automotive interfaces：cadillac cue user study ［EB/OL］．（2019-01-09）［2022-07-01］．.
84	MENG F， SPENCE C ．Tactile warning signals for in-vehicle systems ［J］．Accident Analysis & Prevention，2015，75（1）：333-346.
85	ENRIQUEZ M， AFONIN O， YAGER B，et al ．A pneumatic tactile alerting system for the driving environment［C］∥ Proceedings of the 2001 Workshop on Perceptive User Interfaces．Florida：ACM，2001：1-7.
86	PARASURAMAN R， RILEY V ．Humans and automation：use，misuse，disuse，abuse ［J］．Human Factors，1997，39（2）：230-253.
87	WINTER J， DODOU D ．Preparing drivers for dangerous situations：a critical reflection on continuous shared control ［C］∥ Proceedings of the IEEE International Conference on Systems. Anchorage：IEEE，2011：1050-1056.
88	KATZOURAKIS D I， VELENIS E， HOLWEG E，et al ．Haptic steering support for driving near the vehicle’s handling limits；skid-pad case ［J］．International Journal of Automotive Technology，2014，15（1）：151-163.