• Integration and Background: Last year, Codiga joined Datadog and integrated its static analyzer with Datadog's infrastructure to release Datadog Code Analysis. The static analyzer was written in Java and supported Python, JavaScript, and TypeScript, initially using ANTLR for generating abstract syntax trees.
• Acquisition Challenges: After the acquisition, expanding support to additional languages faced two major issues - partial support for some languages and slower parsing, especially for large repositories.
• Migration to Tree-sitter: Migrating from ANTLR to Tree-sitter addressed these shortcomings. Tree-sitter has a vibrant open source community and is faster due to its C implementation. However, its Java support was limited at the time.
• Performance vs. Control Tradeoff: Datadog's approach gives users more control over static analysis but can be a problem in resource-restricted CI environments. Codiga had more control over analysis on its servers. Continuing with Java presented drawbacks, leading to the decision to optimize for performance.
• Analyzer Architecture: The static analyzer is architected in layers with components like parsing and execution. It relies on third-party dependencies like Tree-sitter for AST building and GraalVM for JavaScript execution.
• Migration from Java to Rust: Rust made the most sense given the dependency on Tree-sitter. Migrating the parser was easy as Rust is a first-class citizen in the Tree-sitter ecosystem. Other components like the JavaScript runtime were migrated to deno.
• Learning Rust: Learning Rust was a large undertaking but was faster than expected. Understanding concepts like the Copy and Clone traits, the borrow checker, and parallelization was crucial. Adopting strict coding standards helped maintain code quality.
• Enhanced IDE Performance: After migration, analysis time was reduced by a factor of three, and the analyzer could be embedded directly into the IDE, reducing setup time and tightening the feedback loop.
• Next Steps: For those considering migrating to Rust, start with small components. The Rust ecosystem is growing, and specialized Java tools may not be readily available. In the next post, they will discuss experiences in the Tree-sitter ecosystem.
• Resources: Initial Java version of the static analyzer is on GitHub at DataDog/rosie. The current Rust version is on GitHub at DataDog/datadog-static-analyzer.