Sphene Community Tools

TinyMCE is a HTML WYSIWYG editor, meaning you store HTML directly into your database, this is OK for authors of a website but not suitable in a forum because it allows too much flexibility, and a user will be able to write some bad HTML (intentionally or not) to ruin your whole page.

That's why forums typically use a much more restricted BBcode system, but TinyMCE's support for BBcode is crappy. And you can see no major forum software uses TinyMCE, they either have their own solution or don't support WYSIWYG at all. The main strength of my editor is the ability to convert BBcode to HTML for display and editing and then convert HTML to BBcode for storage, thus eliminating the risk. Actually my code for this functionality is derived from and much improved over TinyMCE's BBcode plugin so I know it's more sophisticated, feature rich, and has better cross-browser support.

since there seems quite some interest into a bbcode wysiwyg editor, but no editor (yet) - what do you think about converting this sphboard addon into it's own standalone project similar to tinymce and make it more configurable?

i would love to use it in more places .. (e.g. for content pages) where i would need additional functionality and configuration settings (e.g. without smileys, etc.) - would you be interested in improving this bbcode wysiwyg editor?

Ummm... but why not just use tinymce for content pages? Make it more configurable, yes, that would be nice, and not hard to do; but what do you mean by "additional functionality"? The functionality of this editor is restricted by the power of BBcode anyway, or is there a widely accepted extension to BBcode for content page editing?

Maybe we can separate the backend from the frontend though, so that we can have a "wiki backend", which converts HTML to the Wikipedia-style markup, etc. But this can also be done as tinymce plugins, and this way we don't have to rewrite the frontend code.

i want the content to be editable from ordinary users.. so i need validation .. and XHTML validation is a PITA - and i also need a WYSIWYG editor.. so wiki style isn't what i'm looking for..

there is no widely accepted extension for bbcode.. but i want additional functionality for SCT.. like including thumbnails of attached images (or for news a '[ readmore ]' separator).. that's why i would need a WYSIWYG editor which is simple to extend.. (and.. i want to have to code this inclusion of thumbnails in a WYSIWYG editor only once.. ie. i want to use the same for content as for forum posts.. why would i waste time to do it twice.. once for the BBCode WYSIWYG editor, and once for tinymce?)

do you understand my reasoning? :)

Let's see whether I have got it. You want to see this editor be like a javascript library, you can extend it, say, add a button, by calling a function, providing maybe 2 callbacks with the related frontend and backend code, respectively, and maybe an image, and a positioning param. Am I right?

Javascript is indeed particularly powerful in doing things like this, but I can imagine there are complexities too, especially in the backend. Are you talking about just adding new interface functionalities which will be mapped to the same bbcode as before or you want to modify the underlying bbcode implementation to include more tags?

yep, i think you got it :)
but i guess it could be as easy as registering a callback method to add additional UI .. and another callback method which is called during bbcode -> html transformation and html -> bbcode transformation .. although it would be cool to make it a bit more powerful :) but for a start this should work.. my current only use case is including an attached image by ID instead of by it's full URL .. (and maybe add a width/height attribute to it)


(btw. have you found out a way to include an image into a wysiwyg editor but don't allow the user to resize the image?)

OK, the main complexity I see with this approach is that the bbcode<->html conversions may not work in such an "incremental" way. They may conflict with each other and/or with the base one. But I think it should still be doable, we just need to make sure these transformers all obey certain rules. I guess the first step is trying to completely "decouple" the current implementation and make every button an independent "plugin", I'll see if it's possible at all.

For your second question, I don't know whether it's possible, and I don't have time to investigate right now. I'll be busy in the next 2-3 weeks, after that maybe I'll have some time to play with this new idea.

hi lootgoal,

fyi - you might also be interested in the new thread about wiki/forum markup and wysiwyg functionality. would be interesting what you think about that

Well, I don't have much experience with different wiki markups yet.. But as to the wysiwyg editor, I do have several new thoughts.

I'm planning to completely rewrite it, and dump the ugly "regex conversion" approach, in favor of a *simple* DOM parser. Looks like it's not as difficult as I originally thought to do it this way.

The basic goals include:
1. It should be as extensible as it can get. The core is only responsible for DOM parsing, event dispatching, plugin management, etc, and every user-level functionality is implemented through a plugin. Plugins may correspond to a toolbar widget, a right-click menu entry, both, or neither. The only requirement is to provide some event handlers.

2. For each plugin, the appearance (how its widgets are displayed), the frontend (how it modifies the html content) and the backend (what markup syntax is used and how to do the conversions) should be completely decoupled, meaning you can change each of them independently of others to suit your needs. Several versions of each can be provided out of the box for the user to choose from. For example, you may have a backend for BBcode and another backend for some wiki markup and they are interchangable.

3. Some kind of theming support.

Worth mentioning is the backend. Plugins register the HTML tags, attributes or styles they are responsible for, and provide corresponding replace strings or functions to do the conversion. When a registered element is encountered in the html during a conversion, either the replace string is injected, or the function is called with necessary parameters to allow custom processing.

The replace string supports a simple syntax for variables and necessary flow control. I'm currently thinking of the following special characters: $, @, (, and ). $ indicates variables, and @ means "for each".

For example, to implement the "bold" button, the plugin would register the tags <b> and <strong>, and the style font-weight with value "bold". The corresponding replace string would be something like "[ b]$tag.html[ /b]" for bbcode and "'''$tag.html'''" for wiki markup, which indicates that the markups are placed around the html content of the tag.

A more complex example is to deal with tables. The plugin would register the tag <table>, and the replace string might look like:
"{|( width=$table.width)( border=$table.border)( cellpadding=$table.cellpadding)( cellspacing=$table.cellspacing)
@tr(|-
@td(|$td.html)
)
|}"
The braces in the first line indicate that if the variable doesn't exist (no such attribute), don't include the entire content in the braces (it's like an exception handler, if none of these is encountered and a "variable not existed" exception propagates to the top level, an error occurs). And those in the second and third lines are used to mark the boundaries of the loops.

Each tag can only be registered by one plugin.

If what you need to do can't be achieved with replace strings you can use a custom function, which is passed an object containing the entire DOM tree as well as your position on it. So you have the complete info you might need at hand. The drawback is that you have to also provide another function to convert the markup back to html. While for the replace string case, it is not needed.

Plugins are ordered and called in this order. This would lead to more consistent code. For example, "[ b][ i]blah blah[ /i][ /b]" won't become "[ i][ b]blah blah[ /b][ /i]" next time it's submitted.

I'm also thinking of supporting various wiki macros. The problem here is that some macros need global info that is not available on the client side. So the plugin might want to contact the server using Ajax and inject the result later.


I'm kind of using this post to record my thoughts on the implementation, so sorry if it contains too many details. On the other hand, I'm sure there are many complications and the details would probably change a lot later, but I think the big picture is here. Sounds interesting to you?

I more or less had these thoughts more than a month ago but haven't been able to write any code, I guess I'll try another strategy. So instead of trying to squeeze out a whole day I'll try to spend an hour everyday to work on it. Let's see whether it works out...

sounds very interesting. but i'm wondering if the effort is really necessary.
is there really any benefit in converting between HTML and some custom markup, instead of directly using HTML? once you add WYSIWYG to your page.. is anyone still caring about the markup anyway?

my current thought is to use TinyMCE and add functionality so it does what i need to integrate it with SCT (e.g. wiki links, links to attachments, inserting images from attachments, etc.) - seems to be quite straight forward.. and than use lxml to verify the generated HTML and parse/render whatever is necessary.

what do you think about that? why use a custom markup language if we already have (X)HTML?