ClamAV 0.96 adds executable virus signatures and more

Version 0.96 of the open source virus scanner Clam AntiVirus (ClamAV) was released in April, bringing with it support for new file formats, better signatures, and several major new features — such as the first official support for Windows. It also includes an entirely new method for virus signature authors to write the detection schemes at the heart of ClamAV, using a C-like language run in a bytecode interpreter. Finally, the project issued an update to the official virus database that disabled outdated and incompatible versions of the software.

ClamAV is one of the most popular anti-virus products running on Linux, in large part due to its easy integration with Linux server software. ClamAV runs as a daemon, and accepts local and TCP connections to scan files against its virus database. As such, it is a popular choice for Linux email and file servers. Tools also exist for desktop Linux machines, and the daemon has long run on other Unix-like operating systems. Apple has even included it in OS X since version 10.4.

New features

ClamAV 0.96 adds support for scanning several important new file formats, such as InstallShield, Cpio, and 7-Zip archive files, and 64-bit ELF, UPX 3.0, and OS X Mach-O universal binary executables. The scanner can now also detect another common deception technique: packaging Windows viruses with phony Portable Executable (PE) headers and icons. The new release also includes improved wildcard-matching in virus signatures, and supports DazukoFS, which is a "stackable" filesystem designed to facilitate virus scanning. It sits on top of an existing filesystem and implements file access control in user space by allowing a process to permit or block access to particular files based on their contents.

0.96 also introduces a "Personal Stats" feature, which allows ClamAV users to remotely track their specific installation's malware detection statistics. The project already keeps anonymous global statistics of ClamAV detections, which uploads the names of recently-found malware when checking for database updates. The personal stats option requires the user to actively create a host ID on the ClamAV server, which is then copied to the ClamAV configuration file and included in subsequent upstream reports.

ClamAV's freshclam service allows installations to check for updates to the official virus database over the Internet, several times per hour, and to download incremental updates. That functionality was at the root of the need to disable very old ClamAV instances with the release of 0.96.

Version 0.94 and older contained a bug in freshclam which failed to build the updated virus database if an incremental update contained a virus signature longer than 980 bytes. It was still possible for clients to download the full database, but the project was concerned that the traffic generated would tax the ClamAV servers excessively. The bug was fixed for 0.95, and users were warned six months in advance that on April 15, 2010, the database would be updated with a special signature that disabled installations still running 0.94 or older code.

More importantly than the bandwidth hit of clients attempting full-database retrievals — though there were no virus signatures longer than 980 bytes prior to 0.96's release — that limit prevented the creation of the new "logical signatures" at the core of ClamAV 0.96's other major enhancement, the bytecode interpreter.

Byte codes

0.96's bytecode engine is the new release's most fundamental change, and has sparked its share of controversy. In previous releases, the creators of the virus signatures stored in ClamAV's database were limited to pattern-matching techniques to recognize malware. With the bytecode engine, signature creators can now develop "logical" signatures that involve heuristics, complex routines, and even unpacking file contents for examination. It also theoretically allows signature creators to examine new file formats without waiting for the main ClamAV program to support them explicitly.

ClamAV can run bytecode-engine signatures through a built-in interpreter or through a Just-In-Time (JIT) compiler built with LLVM. The syntax of the signature definition language is described as "C-like", and although it has not been formally described in the project documentation, it is partially described in the ClamAV code itself inside the bytecode_api.h header file.

Understandably, when the feature was first announced during the 0.96 development cycle, several in the ClamAV community were uneasy about the ability to incorporate executable code in malware-detection signatures, and even attempted to deactivate the feature.

The developers responded with an explanation of the security measures taken to protect hosts from malicious or problematic routines in bytecode signatures. First, all bytecode distributed by the project will come with embedded source code that can be examined by the user with the clambc utility. Second, all bytecodes in the virus database will be cryptographically signed by the project to verify their integrity. Third, bytecodes themselves have access only to the limited ClamAV API, cannot access system calls or memory, and can only read from the currently-scanned file. Finally, bounds-checking and other security measures are inserted by the compiler and by LibClamAV itself. In addition, the entire feature can be deactivated with a simple line in the freshclam.conf configuration file.

Windows

With 0.96, ClamAV builds on Windows using Visual Studio for the first time. This means that the daemon and server-side tools should work on Windows machines just as they do on all Unix-based operating systems. By itself, the basic ClamAV package allows on-demand scanning with a command-line tool, but does not implement an on-access scanning service (i.e., automatically scanning files whenever they are read or written). On Unix systems, implementing this functionality has always been the domain of the third-party mail or file server code that connects to the ClamAV daemon.

In addition to building the server utilities on Windows, however, the project also announced the availability of an official graphical Windows client-side product. The appropriately-named ClamAV for Windows implements on-access scanning, but, intriguingly, it does not run on the Windows client computer itself. Rather, it connects to a cloud-based ClamAV service run by security company Immunet.

The client sends an SHA hash and file heuristics for each accessed file to the Immunet cloud, where it is scanned against the ClamAV database, and against other detection resources run by Immunet. A ClamAV for Windows FAQ page addresses several security concerns vital to this technique, assuring users that heuristics are only sent to Immunet for executable files, not documents, and points to Immunet's privacy policy.

ClamAV for Windows is a free service, although the source code to the Windows front-end and to Immunet's cloud backend are not open source. ClamAV assures users that in spite of this, the project has no intention of deviating from the GPL for releases of ClamAV itself.

There have been other, unofficial Windows clients for ClamAV in the past. At present, the most popular is ClamWin, which does not itself provide on-access scanning, though that feature can be added through the use of Clam Sentinel.

Moving forward

Bytecode-based virus signatures are provided in their own database, bytecode.cvd, and thus far it is quite small: only three as of May 11th. But it is clearly the way forward for the project. The old system's pattern-matching approach was very limited, and is at least in part responsible for ClamAV's lower performance than the well-funded proprietary virus scanners.

Nevertheless, judging by the response on the mailing list, the added feature may not be an immediate hit with ClamAV users, especially considering how security-conscious they are as a group. Similar wariness is probably to be expected about the cloud-based ClamAV for Windows product, though over privacy rather than security concerns alone.

ClamAV has very little active competition in the open source anti-virus marketplace. Perhaps that is due to the "scratch-your-own-itch" mentality in the Linux and open source communities, which have never seen the level of virus and malware problems still found in Windows. Consequently, it may be that the most important new bullet point of ClamAV's 0.96 release is the project's ability to build on Windows itself. That will attract more developers who will build the kinds of add-ons for client and server software that the project needs to grow and evolve further.