The main idea behind Sonargraph has always been to provide a tool that eases the creation and maintenance of high-quality software. Creating high-quality software is difficult: You need to know where the pain-points are and how to solve them.

For any serious project that must live longer than a couple of months, it is actually cheaper to spend part of your resources to keep your software constantly at a good level of quality than using all your time to create new features. Martin Fowler explains this very well in his article "Is High Quality Worth the Cost?"[1]. The bottom line is, that apart from the very early development stages, high-quality software is actually cheaper to develop, because it allows adding new features at almost constant speed, whereas it becomes more and more time consuming to add new features into a code base with low quality.

We at hello2morrow believe that a consistent architecture is a fundamental part of software quality. When we use the term "architecture", we think of it in terms of the IEEE 1471 standard:

This chapter describes why architectural design as an activity is needed, why conformance checks need to be done automatically by a tool and how Sonargraph supports you as a developer and architect during these activities.

The Need for Architecture

Martin Fowler wrote an excellent article "Is Design Dead?"[2] back in 2004. He argues that for anything serious, you cannot just code along and hope for the best, but need “planned design”:

He is not arguing to design everything up front, but rather making the design activity part of the agile development process. As a consequence, the architecture evolves together with the code base and the knowledge of the team.

If you take the definition for architecture mentioned at the start of the chapter, then the top-level architecture should contain the main components and their dependencies. As with construction architectures for large buildings, no single diagram exists that contains all information for a large-scale software system. There is an upper limit of elements that our brain can process, especially if there are also interconnections of different types between elements. Thus, if the system grows beyond a certain size, abstractions are needed which can be thought of as maps at different scale. Simon Brown gives an example of this with the C4 model[3]. Where details are needed, additional diagrams can be created.

Of course, several viewpoints for software architecture exists as described by "The 4+1 ViewModel"[4]. This "static" architecture that describes the decomposition of a system in its parts is the foundation for the "dynamic" aspects like information flow: If there is no direct dependency between two components or between them and any of their commonly shared components, there cannot be any information flow between them.

Consistency of the diagrams now becomes a challenge: Higher-level abstractions must not be violated at the lower level: There must not be a secret tunnel at the detailed level where the higher level puts a clear barrier between components.

The last decades of agile software development have shown that it is impossible and impractical to do a big design upfront. Not only the requirements from the outside (read 'business') may change, but most certainly the developers' understanding of the domain improves over time and thus the architecture likely also needs to be adapted as a consequence. The effort needed to change a functionality and the effects this change causes for the rest of the system very much depend on the number of usages of this functionality.

Design for changeability therefore means to minimize coupling (i.e. the number of dependencies) between elements, because elements with low coupling can be more easily re-arranged. Especially bad for coupling are cyclic dependencies that may cause mental challenges in form of hen-and-egg problems and also make it harder to understand a system's structure if a large number of elements are involved. You want to avoid an entangled mess as shown in the following picture (where green arcs represent dependencies between Java packages and the direction of dependencies is counter-clockwise) that is often described as "big ball of mud":

As a consequence, we think that spending effort on a clean and consistent software architecture and controlling dependencies is essential to code quality as well as regularly cleaning up other code smells.

Our experiences match those of well-known experts. Here is an incomplete list of resources that we found affirmative to our thinking. They haven't lost any importance despite dating back a couple of years.

Automating Checks

Manually tracking the evolution of the internal structure of a software system is not efficiently manageable. With thousands of classes and millions of dependencies this is impossible for any large system. You need a tool that automatically reports deviations in the implementation from your envisioned architecture. We think that the lack of proper tool support is the reason why so many projects suffer from software rot. There are many static analysis tools and linters that report errors and problems at source-file level, but most are missing out on the big picture: Tracking dependencies across source files and validating if the structure matches the envisioned design.

As a result, most systems contain a large number of cycle groups, elements are tightly coupled, no clear structure exists and if documentation exists, it is not up-to-date. New developers have a hard time to know how they should structure new functionality, where to place the new code and how it should interact with the existing code. Once you have lost control, the structural quality typically spirals downwards quickly and the software ends up in a "big ball of mud". Regular quality initiatives feel like Sisyphean tasks, because problems are created faster than they can be fixed.

Therefore, quality checks must be executed automatically by the Continuous Integration (CI) build, whenever new code is committed. Even better, quality checks should be executed while programming, so that those problems never get into the version control system.

Setting the Focus

Starting a new project with automated architecture checks in place is very practicable to maintain quality at a high level. It is way harder to sustainably improve quality for existing projects. Resources are scarce, and new features need to be implemented, so it is not feasible to simply stop all work and clean up everything first.

For any quality improvements, it is important to spend efforts where it supports the current work: This makes the positive effects visible, and the enthusiasm for code quality will stay and won't fade away quickly.

For this reason, Sonargraph offers to compare the current state of the system against a previously run analysis. This 'System Diff' identifies where quality was improved and worsened, making it ideal to support code and sprint reviews. Additionally, quality gates can be defined that ensure that quality trends in the right direction. A ranking algorithm highlights those issues where fixes provide high benefit, i.e. that are urgent and important: Issues that were added recently, that have a high impact on the system and where involved files have been changed recently.

The Sonargraph Product Family

Architects and developers are supported by Sonargraph to maintain and improve the quality of their software. Its focus is on architecture and dependencies, but it offers also a large number of metrics, duplicate code checks and additional rules that can be activated as needed, e.g. to detect unused code. Sonargraph is built upon the experiences that hello2morrow gained during the development and support of the predecessor products Sonargraph 7, SotoGraph and SotoArc. Sonargraph is lightweight and integrates smoothly with different IDEs, build and quality infrastructures (e.g. Eclipse, IntelliJ, SonarQube, Jenkins, Ant, Maven, ...).

Sonargraph consists of several products that help to ensure quality throughout the software development as shown in the following image:

Use Cases and Key Functionality

The following describes key functionality of Sonargraph and typical uses cases. This is just a summary, the rest of the user manual provides more details.

Architecture Definition

Sonargraph uses a Domain Specific Language (DSL) approach to describe the architecture. A system's architecture can consist of multiple architecture aspects which are checked in parallel. Alternatively, the architecture can be defined interactively. Architecture diagrams can be generated allowing to investigate connections between architecture artifacts.

Simulate Refactorings

Sonargraph allows the simulation of refactorings. For this, you can create multiple so-called virtual models. A virtual model is a space where the model from the parser(s) can be modified by refactorings and detected issues can be transformed into tasks or ignored (called resolutions). This allows the simulation of different approaches to change an existing structure. A virtual model can be based on another virtual model making it possible to reuse common refactorings and resolutions.

The 'cycle-breakup' analyzer proposes refactorings to find an efficient way to eliminate a cycle. It takes into account defined architectures and allows to interactively fine-tune the solution.

NOTE: A virtual model might affect metric values since the structure of the system can be changed with refactorings and issues can be transformed into tasks or ignored.

Hotspot Visualization

Sonargraph analyzes information from Source-Control Management (SCM), currently Git. The combination of issues and code changes and the visualization as software maps (a.ka. "Code Cities") allows the visual identification of hotspots.

Tracking Changes in Quality

If Sonargraph is used in existing projects there might be an overwhelming number of reported issues. The 'System Diff' analyzer allows focussing on changes, making it the ideal companion during reviews. Quality gates can be defined on the current system state or in comparison to a baseline, making it easy to follow the 'Boyscout Rule' and gradually improving the system's quality. The 'Issue Ranking' view recommends issues that are both urgent and important to fix.

Great Parser Model Detail, Little Memory Consumption

Dependencies are tracked down to method and field level offering more detailed exploration. Sonargraph has little memory consumption, as only the model coming from the different parsers is held in memory and all 'derived' structural elements (e.g. a layer) and their dependencies are calculated on demand.

Snapshots

The complete model of a system is stored in a compact binary format. This enables fast startup times (the last snapshot is used if available) without having to perform a full re-parse. Furthermore complete systems might be compared and historically analyzed - even passed around to enable reviews based on them - by directly loading the snapshot.

Fast Execution

Analyzers calculate metrics and analyze dependency structures (e.g. cycles) and content of source files (e.g. duplicated code). These analyzers run in parallel in a multi-threaded environment providing more speed while not blocking user interaction. Once an analyzer has finished, it`s results are available to the user.

Extensible Analysis

The user can extend Sonargraph's functionality by writing Groovy scripts accessing the model created by Sonargraph. These scripts can either simply act as custom queries finding artifacts with specific characteristics and/or to create issues pointing to potential problems in the system or create additional metrics.

Sonargraph also offers a plugin API to integrate external analyzers and to extend the parser model by custom elements. The currently existing plugins are 'Spotbugs' and 'PMD' for further file-local issues and 'Swagger' and 'Spring Microservices' to reveal web service dependencies.

Multiple Language Support

Sonargraph supports different languages depending only on the license without the need to have different installations. There is a unified approach (i.e. one user interface) to explore and monitor systems implemented in different languages. Systems have a module structure where each module can have a different language. A generic component approach is used for all supported languages - currently Java/Kotlin, C#, C/C++, Python.

Flexible Exploration of Dependency Structures

You are free to decide how to explore dependencies. Sonargraph offers a tree-like explorer, a graph viewer and a simple table-based viewer.

Automated Updates and Flexible User Interface

Automated updates and a flexible user interface (layout and customization) are provided as Sonargraph is built upon the Eclipse Rich Client Platform (RCP). Sonargraph-Build plugins for Maven and Gradle can also be configured to update automatically.

Exchangeable Quality Artifacts

The software system analysis comes with a multiple file approach. The software system is comprised of a main software system file, analyzer configurations, user defined scripts, different architecture aspects and so forth. The approach makes it easy to share valuable aspects of the analysis between software systems as well as to centralize common aspects in bigger companies.