Fine-grained distributed averaging for large-scale radio interferometric measurement sets

作     者：Shou-Lin Wei Kai-Da Luo Feng Wang Hui Deng Ying Mei Shou-Lin Wei;Kai-Da Luo;Feng Wang;Hui Deng;Ying Mei

作者机构：Faculty of Information Engineering and AutomationComputer Technology Application Key Lab of Yunnan ProvinceKunming University of Science and TechnologyKunming 650500China Center for AstrophysicsGuangzhou UniversityGuangzhou 510006China 

出 版 物：《Research in Astronomy and Astrophysics》 (天文和天体物理学研究（英文版）)

年 卷 期：2021年第21卷第4期

页      面：93-100页

核心收录：

学科分类：07[理学] 070401[理学-天体物理] 0825[工学-航空宇航科学与技术] 0704[理学-天文学] 

基　　金：the National Key Research and Development Program of China(2020SKA0110300) the Joint Research Fund in Astronomy(U1831204,U1931141)under cooperative agreement between the National Natural Science Foundation of China(NSFC) the Chinese Academy of Sciences(CAS) the NSFC(No.11903009) the Funds for International Cooperation and Exchange of the NSFC(11961141001) Yunnan Key Research and Development Program(2018IA054) 

主　　题：techniques:interferometric methods:data analysis methods:numerical instrumentation:interferometers 

摘      要：The Square Kilometre Array(SKA)would be the world’s largest radio telescope with eventually over a square kilometre of collecting ***,there are enormous challenges in its data *** use of modern distributed computing techniques to solve the problem of massive data processing in the SKA is one of the most important *** this study,basing on the Dask distribution computational framework,and taking the visibility function integral processing as an example,we adopt a multi-level parallelism method to implement distributed averaging over time and *** Array was used to implement super large matrix or arrays with supported *** maximize the usage of memory,we further exploit the data parallelism provided by Dask that intelligently distributes the computational load across a network of computer agents and has a built-in fault tolerance *** validity of the proposed pattern was also verified by using the Common Astronomy Software Application(CASA),wherein we analyze the smearing effects on images reconstructed from different resolution visibilities.