Enhanced Userspace and In-Kernel Trace Filtering for Production Systems

作     者：Suchakrapani Datt Sharma Michel Dagenais 

作者机构：Department of Computer and Software Engineering Ecole Polytechnique de Montrdal Montreal H3T 1J4 Canada 

出 版 物：《Journal of Computer Science & Technology》 (计算机科学技术学报（英文版）)

年 卷 期：2016年第31卷第6期

页      面：1161-1178页

核心收录：

学科分类：081901[工学-采矿工程] 0819[工学-矿业工程] 0808[工学-电气工程] 08[工学] 0835[工学-软件工程] 0701[理学-数学] 0811[工学-控制科学与工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 081202[工学-计算机软件与理论] 

基　　金：加拿大自然科学与工程研究理事会（NSERC）项目 the research grants from Ericsson, EfficiOS and PROMPT Quebec 

主　　题：tracing debugging filtering operating system interpreter 

摘      要：Trace tools like LTTng have a very low impact on the traced software as compared with traditional debuggers. However, for long runs, in resource constrained and high throughput environments, such as embedded network switching nodes and production servers, the collective tracing impact on the target software adds up considerably. The overhead is not just in terms of execution time but also in terms of the huge amount of data to be stored, processed and analyzed offiine. This paper presents a novel way of dealing with such huge trace data generation by introducing a Just-In-Time （JIT） filter based tracing system, for sieving through the flood of high frequency events, and recording only those that are relevant, when a specific condition is met. With a tiny filtering cost, the user can filter out most events and focus only on the events of interest. We show that in certain scenarios, the JIT compiled filters prove to be three times more effective than similar interpreted filters. We also show that with the increasing number of filter predicates and context variables, the benefits of JIT compilation increase with some JIT compiled filters being even three times faster than their interpreted counterparts. We further present a new architecture, using our filtering system, which can enable co-operative tracing between kernel and process tracing VMs （virtual machines） that share data efficiently. We demonstrate its use through a tracing scenario where the user can dynamically specify syscall latency through the userspace tracing VM whose effect is reflected in tracing decisions made by the kernel tracing VM. We compare the data access performance on our shared memory system and show an almost 100 times improvement over traditional data sharing for co-operative tracing.