Looking for an open source text editor component with syntax highlighting

I’m trying to give a little back to an open source project, namely Irony that I blogged about. My idea was this: We’re getting used to intellisense features in mainstream .NET languages such as syntax coloring, error highlighting, code completion, signature tool tips and more. When writing a DSL to be used and understood by business experts and developers, those things are not readily available.

The benefit of writing an add-in for Visual Studio that allows intellisense for DSLs parsed with Irony is big. Writing a managed language service for Visual Studio is no simple task. I want to avoid the complexity of the Visual Studio domain model and keep it simple using a stand alone text editor component. So I went searching…

There had to be lots of open source syntax highlighting text editors out there, I thought. And there is. I found a couple of well written candidates like SharpDevelop IDE TextEditor, the xacc  IDE editor and SyntaxBox from the Puzzle Framework. Finally, there are quite a few commercial offerings, but introducing a dependency to a commercial product in an open source project is not the road to popularity.

Both SyntaxBox and SharpDevelop let you extend the built-in syntax schemes with your own. This is done  by writing an XML based grammar. It’s possible to do and I have done it a couple of times. But having already written my grammar once using Irony, it just doesn’t feel right having to do it over again — this time using a different syntax.

Running an Irony-generated parser gives as output not only an AST, but also a list of tokens. If I could feed that list of tokens to the editor and bypass the built in lexing mechanism, I’d be laughing now. But it’s not that easy.

The problems with the three components that I tried using, fell into three categories:

• The editor is entangled in references to other parts of the project, making it impossible to reuse it in other projects like Irony.
• The built in lexer cannot by turned off, so lexing is being done twice, and two sets of formats for each token have to be merged — resulting in awfully bad performance.
• The extension points are leaky abstractions that requires you to know of the inner workings of the text rendering.

Not unsolvable issues, but issues that result in poor maintainability and hacks en masse. Plus it takes a lot more (spare)time than I have.

Do you know of any open source text editor component that supports syntax highlighting AND lets me replace the tokenizer/lexer with something else?

2 Responses to 'Looking for an open source text editor component with syntax highlighting'

Subscribe to comments with RSS or TrackBack to 'Looking for an open source text editor component with syntax highlighting'.

1. Daniel Grunwald said,

   ON APRIL 18TH, 2008 AT 11:38 AM

   You can supply our own implementation of IHighlightingStrategy for SharpDevelop’s text editor and consume your own tokens in that. The interface isn’t as clean as it should be (at least in SharpDevelop 2.x), but it’s certainly possible. I would like to hear what problems you had exactly with SharpDevelop’s editor.

2. Soren said,

   ON APRIL 19TH, 2008 AT 8:41 PM

   Daniel,

   the #dev editor is a great editor, and I really hope you will help me get this scenario working.

   Having failed several attempts to make a custom IHighlightingStrategy work, Finally, I tried cutting to the bone by implementing the simplest possible scenario: Not tampering LineSegment.Words at all within MarkTokens(). I expected the text to be all black and otherwise work normally. But the editor starts behaving weird and it’s not possible to enter or edit text, because it gets cut off at a fixed column.

   Try downloading http://skarpt.dk/blog/SDTextEditorTest.zip. It references ICSharpCode.TextEditor.dll version 3.0.0.2970.

   Looking at the default implementaion I get a feeling that MarkTokens needs to be doing something else, something that is not obvious from the interface definition?