Architecture & Civil Engineering

Research on Audio-Visual Integration Based on Architectural Acoustic Simulation

Expand
  • State Key Laboratory of Subtropical Building Science/School of Architecture,South China University of Technology,Guangzhou 510640,Guangdong,China
孙海涛(1975-),男,博士,教授级高级工程师,主要从事建筑声学与环境声学研究。

Received date: 2022-04-17

  Online published: 2022-09-29

Supported by

the National Natural Science Foundation of China(51878281)

Abstract

In the article, the multi-channel recording signal of Mozart's Symphony No. 40 was selected as the source dry signal, and the Tianjin Cultural Center concert hall was used as an example to simulate the sound field using the architectural acoustics simulation software ODEON. The symphony orchestra was divided into 49 point sound sources according to musical instrument, corresponding to the directivity of each musical instrument respectively. Five typical receiving point positions were set in the audience seat area, and the binaural impulse response from each sound source point to the receiving point was calculated respectively, then use binaural impulse response to convolve with the instrument signal corresponding to each sound source point, and finally the 49 resulting audio signals were synchronized and played together to synthesize the multi-source audible signal of the orchestra. At the same time, according to the traditional audibility method, set a point sound source in the center of the symphony orchestra, calculate the binaural impulse response from the point sound source to the receiving point, and then use binaural impulse response to convolve with the signal of the symphony composed of 49 dry signals and mix it into a single sound source audible signal. In terms of vision, firstly, the 3D model of the concert hall was established in sketch up and the real materials were given to each surface. The model of sketch up was output to the VR virtual reality simulation software SIMLAB composer, in which the parameters such as light, material and environment were adjusted. The virtual model was rendered and baked with adjusted information parameters to obtain a better sense of immersion. HTC VIVE Cosmos glasses were used to output VR virtual scenes and to simulate the three-dimensional vision of the hall. During the architectural acoustics audio-visual experiment, the subjective evaluation differences of Reverberance and ASW (Apparent Source Width) of audible signals convoluted by multiple sound sources and single sound source were compared under the condition of with or without VR visual signal. The experimental results show that the audible signal convoluted by multiple sound sources can significantly improve the Reverberance and ASW, and the improvement degree of different receiving points is different. Due to the significant difference in sound quality between the single-source and multi-source audible signals, the listening test participants in the architectural acoustics and audiovisual experiment focused more on the improvement of spatial sound quality. The addition of VR visual signals did not significantly improve the subjective evaluation results of the audible signals.

Cite this article

SUN Haitao, YANG Yu . Research on Audio-Visual Integration Based on Architectural Acoustic Simulation[J]. Journal of South China University of Technology(Natural Science), 2023 , 51(4) : 71 -79 . DOI: 10.12141/j.issn.1000-565X.220209

References

1 吴硕贤,赵越喆 .建筑环境声学的前沿领域[J].华南理工大学学报(自然科学版)201240(10):28-31.
  WU Shuo-xian, ZHAO Yue-zhe .Frontiers of architectural and environmental acoustics [J].Journal of South China University of Technology (Natural Science Edition)201240(10): 28-31.
2 OTONDO F, RINDEL J H .A new method for the radiation representation of musical instruments in auralizations[J].Acta Acust200591(5):902-906.
3 VIGEANT M C, WANG L M, RINDEL J H. Investigations of orchestra auralizations using the multi-channel multi-source auralization technique[J].Acta Acustica united with Acustica200894(6):866-882.
4 VIGEANT M C, WANG L M, RINDEL J H .Objective and subjective evaluations of the multi-channel auralization technique as applied to solo instruments[J].Appl Acoust201172(6):311-323.
5 RINDEL J H, CHRISTENSEN C L. Auralisation of concert halls using multisource representation of a symphony orchestra[C]∥Proceedings of the International Conference on Auditorium Acoustics 2008.Oslo:Institute of Acoustics,2008:333-338.
6 王鑫,郝欣言,王元中 .民族乐器的多通道可听化研究[J].中国传媒大学学报(自然科学版)202027(3):29-35,40.
  WANG Xin, HAO Xin-yan, WANG Yuan-zhong .The research on multi-channel auralization for chinese traditional instruments[J].Journal of Communication University of China (Science and Technology)202027(3):29-35,40.
7 SATO S I, WANG Shuo, ZHAO Yue-zhe, et al .Effects of acoustic and visual stimuli on subjective preferences for different seating positions in an Italian style theater[J].Acta Acustica and Acustica201298(5):749-759.
8 WANG Shuo, ZHAO Yuezhe, WU Shuoxian .Seat selection system for theatres and concert halls based on audio-visual integration technology[C]∥Proceedings of the 22nd International Conference on Computer-Aided Architectural Design Research in Asia:Protocols,Flows and Glitches. Suzhou:The Association for Computer-Aided Architectural Design Research in Asia (CAADRIA),2017:157-165.
9 王朔. 虚拟厅堂环境视听一体化[D].广州:华南理工大学,2014.
10 CASTRO D, BALLESTERO E, PLATT S .Walk-through auralization framework for virtual reality environments powered by game engine architectures[C]∥Proceedings of the Institute of Acoustics.Bristol:Institute of Acoustics,2018:151-160.
11 CASTRO Daniel, VERSTAPPEN André, PLATT Samuel .Walk-through auralization framework for virtual reality environments powered by game engine architectures,Part Ⅱ[C]∥Proceedings of the Annual Conference of the Australian Acoustical Society. Cape Schanck:Australian Acoustical Society,2020:495-505.
12 LEE Y,RYU J .A study on the auralization system using flexible rendering for virtual reality-based acoustic simulation[J].Journal of Advanced Mechanical Design Systems and Manufacturing201913(5):1-15.
13 YAGUNOVA Yulia, POLETTI Mark A, TEAL Paul D .Ambisonics and sonic simulation in virtual reality[C]∥Proceedings of the IEEE Region 10 Annual International Conference.Auckland:Institute of Electrical and Electronics Engineers Inc,2021:369-374.
14 THERY D, POIRIER-QUINOT D, POSTMA B N .J,et al .Impact of the visual rendering system on subjective auralization assessment in VR[C]∥Proceedings of the 14 th International Conference on Virtual Reality and Augmented Reality.Laval:Springer,2017:105-118.
15 孙海涛,赵越喆,吴硕贤,等 .建筑声学缩尺模型实验技术研究——以天津文化中心音乐厅为例[J].建筑学报2015(S1):228-231.
  SUN Hai-tao, ZHAO Yue-zhe, WU Shuo-xian,et al. A technique developed for room acoustic scaled model test in auditorium—case study the Concert Hall in Tianjin Cultural Center[J].Architectural Journal2015(S1):228-231.
16 李青梅,吴硕贤 .关于厅堂音质响度指标的评价[J].建筑学报2000(2):53-55,67-68.
  LI Qing-Mei, WU Shuo-xian. Comment on the sonority norm of acoustic fidelity in the great hall[J]. Architectural Journal2000(2):53-55,67-68.
Outlines

/