Programming

Removing comments from source code needs a lexer

Removing comments is a lexical problem.

A code card with muted comment rows flowing through a state-machine diagram into a clean card

Removing comments is a lexical problem. Searching for // or /* alone will damage valid code when those characters occur inside a string.

cpp
const char* url = "https://example.com";
const char* marker = "/* this is text, not a comment */";

Keep track of where the scanner is#

A scanner for a limited C-like grammar needs separate states for ordinary code, quoted strings, character literals, line comments and block comments. Escaped quotes must not end a string. Line endings inside removed comments may need to remain so that diagnostics still refer to useful line numbers.

Real C and C++ add complications such as preprocessing, continued lines and raw string literals. Other languages have different rules again. There is no universal comment remover for “all programming languages” based on the old loop.

What was wrong with the original example?#

The legacy example was labeled C while using old C++ headers. It read command-line arguments without validating their count, mishandled comment endings and did not protect quoted text. Those defects make it unsuitable as a general source transformation.

For a production transformation, use a lexer for the target language and test strings, escapes, empty files and unterminated comments. For C/C++, Clang's LibTooling is one compiler-based starting point. Preserve the original file and write to a separate output while testing.