Ash - Automatic Syntax Highlight (Attempt 2)
[Please note:. I have my Delphi Editor colors set-to the [Ocean] colour speed settings for testing This setting works well on the default RichEdit white background, and most TokenTypes are in different colors from each other purposes]
Okay now to do some real work. Most of the function have been written thereabouts. As a basis for writing this ASH I'm going to use Project1.dpr which comes out of mpas2rtf.zip in the YourPasEdit zip file yrpasedit.zip. This IS Because It Much Smaller Than YourPasedit, And Thus Quicker To Compile.
I suggest you put the contents of the mpas2rtf.zip into a separate directory Also copy mwPas2Rtf.pas to testinput.pas using the Explorer shell -. We'll be using this file as a sample pascal file for benchmarking.
Open Project1.dpr in Delphi, compile Project1, run it, and open the file testinput.pas by pressing [Button 1] and selecting it in the [OpenFile Dialog]. Do it a number of times, and record the time taken for each ONCE The File Is Stabilied in The System Cache. on My System It Averages About 0.47 - 0.41 Seconds Once ITS in The Cache (P133 - 16M - Win95B)
Preparing Project1's Unit1.PAS
Now Replace the contents of mPas2rtf.Pas with That code in jhdpas2rtf.pas. Recompile. Now open up the test.pas sample file again by using [Button 1]. As you see - We get color - But it take Takes a "LOT" LONGER: 1.20-1.25 seconds.
Try and speed it up if you like. You can start by commenting out the pascal-code that codes in the different Font and FontSizes in TPasConversion.SetRtf. Recompile and run again. This time it improves a bit to 1.10-1.15. Now try commenting out the code for different Colors Wow -. the speed decreases down to 0.49 -.. 0.44.Hmm This font and color stuff really packs a punch We may need to look at this later in more detail if things end up too slow for. The Moment We'll Leave The Code Back In Full Working Condition (SO You'll Need To Go Back and UNComment The Code).
Now Put The Following Base Code InTo The Richedit1 in Unit1.Pas of Project1. Most of this code is just based on what we have already covewhere.
Procedure tform1.richedit1change (sender: TOBJECT); VAR
Wasselstart, WasRow, Row, BeginselStart, EndselStart: Integer; Myre: TrichEdit; MypBuff: Array [0..255] of char;
Begin
MyRe: = TRichEdit (Sender); WasSelStart: = MyRE.SelStart; WasRow: = MyRE.Perform (EM_LINEFROMCHAR, MyRE.SelStart, 0); BeginSelStart: = MyRe.Perform (EM_LINEINDEX, Row, 0); EndSelStart: = BeginSelStart Length (myre.lines.strings [rot); row: = WASROW;
END;
WERE Going to Use the gettoken () function to do all the hard work. We'll Need Some Extra Variables to pass to the gettoken function: SO Add to the VAR Section:
MYTOKENSTR: STRING; MyTokenState: TtokenState; Myrun: Pchar; MyselStart: Integer;
THESE ARE SIMILAR TO The VARIABLES WE Used In The ConvertReadstream - In Fact We Want to Do "Exactly" The Same Thing, Just One Single Line At A Time. Add this code before the last end;
StrPcopy (mypbuff, myre.lines.strings [row]); mypBuff [length (myre.lines.strings [rower)]: = # 0; myselstart: = beginselstart; myrun: = mypbuff; while (myrun ^ <> # 0) Do Begin
Myrun: = Pascon.getToken (MyRun, MyTokenState, MyToKenstr);
// // scanforrtf; // setRTF; // WriteBuffer (prefix tokenstr postfix); //
END;
END;
NB: AS We Will Be Using Pascon You'll Have to Move It From Being a Local Variable of TFORM1.BUTTON1CLICK TO BE A GLOBAL VARIABLE. THIS WILL Mean You'll Have To Move All The Initialising:
Pascon: = TPASCONVERSION.CREATE; Pascon.usedelphihighlighting (3);
To a tform1.show, and the pascon.free to TFORM1.CLOSE Procedure. it will site work if you only move the variable definition - But not for long ... :-)
I've left the code from the old ConvertReadStream in the example above to show what we "logically" still need to implement in the current context - that is manipulating the RichEdit Control directly What we have now is the ability to cut up the current. Line in to Different tokens, And Know What Type The The Right Attributes (Fonts, Colors, Bold ETC), and KNow What Typent Tokens.
But wait They are already on the line -. Well the text is anyway, but maybe not in the correct format (Color, Bold etc) So what actually could do is to select each token in its corresponding positon in the RichEdit control and just. Apply the appropriate attributes to them.
WE DID THIS BACK IN THE BEGINNING REMEBER? WHEN WHEN WHEN WHEN WHIS INTS LOOW WE Do THIS NOW? Lets Look at What We Have In The Variables At Hand WHEN We Hit "// setRTF" the First Time:
(These example uses Uni1.pas as the input file as its more interesting) VARIABLES 01234567901234567890Lines.Strings [R0] unit Unit1; MyPBuffunit Unit1; MyTokenStatetsIdentifierMyTokenStrunitMyRun Unit1;
So what we need to do is select the word 'unit' in the RichEdit control, and set its attributes. We do this by setting SelStart to the position of 'unit' in the RichEdit control, and SelLength to the length of the word ' '. And since' unit unit 'is at the beginning of the current line - thats position is BeginSelStart (which I conveninently have stored in MySelStart - you'll see why) Lets replace the "pseudo" comment code with the following.:
Myre.selstart: = myselstart; myre.sellength: = length (myToKenstr); myre.selattributes.assign (pascon.fparsefont [myTokenState]);
END;
But Remember We Are in A Loop - When We Go Around Again We'll Have The next token in the line, and the variables will look like this:
Variables 01234567901234567890101234567890Lines.strings [r0] unit unit1; mypbuffunit unit1; myTokenStatetsSpacemyToKenstr (space character) MyrunUnit1;
But (space character) is not at BeginSelStart (# 0) in the RichEdit control. Its further along (at position # 4). Which just happens to be BeginSelStart Length ( 'unit'). We need to update MySelStart after we Process the Preceeding token, But Before We go around the loop again:
MyselStart: = myselstart length (MyTOKENSTR);
END;
Okay - this is where we are standing at the moment:
Procedure tform1.richedit1change (sender: TOBJECT); VAR
WasSelStart, WasRow, Row, BeginSelStart, EndSelStart: Integer; MyRe: TRichEdit; MyPBuff: array [0..255] of char; MyTokenStr: string; MyTokenState: TTokenState; MyRun: PChar; MySelStart: Integer; begin
Myre: = trichedit (sender);
WasSelStart: = MyRE.SelStart; WasRow: = MyRE.Perform (EM_LINEFROMCHAR, MyRE.SelStart, 0); Row: = WasRow; BeginSelStart: = MyRe.Perform (EM_LINEINDEX, Row, 0); EndSelStart: = BeginSelStart Length (MyRE .Lines.strings [rot]);
Strpcopy (mypbuff, myre.lines.strings [rot]); mypbuff [length (myre.lines.strings [rower)]: = # 0;
MyselStart: = BeginselStart; Myrun: = MypBuff;
While (myrun ^ <> # 0) do begin
Myrun: = pascon.gettoken (MyRun, MyTokenState, MyToKenstr); Myre.Selstart: = myselstart; myre.sellength: = length (MyToKenstr);
Myre.selattributes.assign (pascon.fparsefont [myTokenState]);
MyselStart: = myselstart length (MyTOKENSTR);
END;
Myre.selstart: = WasselStart; myre.sellength: = 0;
END;
Now: Put the debugging code on, do [build all] and then r s t s s on The debugger stops in the onchange event, Press
Somehow in our event we are triggering off another [OnChange] event. This call to the [OnChange] event code is stored in the message queue. When the event were currently in is finished, a new one is just waiting on the Event queue, Which Executes and Creates More Events ... a re-entrant loop.
This behaviour is not surprising - after all we are actually changing the control in the process of our code, so no wonder another [OnChange] event is being triggered.The way to fix such things is to ensure our actions do not trigger of the Event . We can do this by "temporarily" storing the RichEdits.OnChange property (which contains a reference to call our procedure TForm1.RichEdit1Change) in our own internal variable, and then setting the OnChange property to nil.
WE THEN DO - IT HAPPENT - TRIGGER AN [ONCHANGE] EVENT - There Ining to call as onchange is nil, and so the event doesn't go office queeue .hen we're finished HOWEVER WE MUST RETURN The Onchange Property To It Original Value, Otherwise The Reprocessing Want Happen Next Time Around.
IF WE LOOK AT The Delphi Helpfile We See That The Onchange Property Is of a Certain Type, The Same Type We Have To make Our SaveOnchangein Variable:
VAR
SaveonChangein: TNOTIFYEVENT;
~~~~~ Rest of Code
Begin
Myre: = trichedit (sender); SaveOnchangein: = myre.onchange; myre.onchange: = nil;
~~~~~ Rest of Code
Myre.onchange: = saveonchangein;
END;
Try it out !!!
Compile and run open unit1.pas in the "editor" we have written click in the richedit in the center of the first lines, in the middle of "unit". Press the [space bar] press the [backspace key] arrow to The end of the line press [backspace] [backspace] Away the entire line re-type the entire line. - TYPE THE Entire Line
Result: "functionally" the Control should look that same as it did before we clicked in it The line "unit Unit1;" should highlighted properly as per your Delphi 3.0 Editor (save the background colour) However its slow and flickers a great.. DEAL. TRY OPENING UP A New Line and Just Type A Long Phrase - EG "IF (RicheDit = Santa);" And You'll Agree With Me That: Good - IT IS Highlighting Properly Bad - There Is Flickering Bad - The Longer The Line Gets, The Longer It Takes To Do The Re-Highlighting Bad - You Get The "Someone Is ChaSING Me Effect"
The Flickering Is Due to a Number of Components. We'll Have to DEAL with Each Seperately.
The most obvious is the "selecting" of each Token Visually the control is just repeating what we were able to do manually -. When a piece of text is selected it becomes highlighted by the black stripe We need to stop this from happening Back.. To the helpfile (s) again. Have a Search Around, And Come Back After A Snack Break with some ideas ... i'm hungry :-)
Death to the Black Stripe
Marks: 5/10
Most of you would have found the HideSelection property of the RichEdit control. When it is set to TRUE and the RichEdit looses the focus (the user clicks onto another control) the selection bar (the black stripe) is hidden. In fact if you try it out by selecting some text in the RichEdit1 then clicking in the Edit1 control at the top of the "editor" you'll see the selection disappears [Tab] back into the RichEdit control and it reappears Lets do this programmatically!.:
Begin
Edit1.setfocus;
~~~~~
Myre.setfocus;
END;
Take My Word for IT, But if you look closely, The Black Strip is gone. Pity WE GOT STUCK WIONE. PITY INE EDIT1 Control :-( if Your Programmed in Delphi You May Know A Little Trick:
Delphi rule # 4: You can't setfocus on a disabled control.
The Converse However is Also True:
Delphi Rule # 4b: a Disabled Control Is Not "Focused"
SO Try INSTEAD We can Just Disable (THEN Enable) The Richedit Control Like this:
Begin
Myre: = trichedit (sender); myre.enabled: = false;
~~~~~~
Myre.enabled: = true;
END;
OOPS. I shop has known. After all i said it: a disabled control is not "focused" - Barely Ten Lines Ago! When the Richedit is enabled Again, We Also Have to setfocus back to it. Shees .. :-)
Begin
Myre: = trichedit (sender);
Myre.enabled: = false;
~~~~~~
Myre.enabled: = true;
Myre.setfocus;
END;
Try it again. This time things are working better, and we're leaving poor old Edit1 Control alone. Thats good practice, as it may have had an [OnFocus] event that does wierder things than what we're trying to do. Maybe Not Now, But It Could in The Future!
Marks: 10/10
On the other kind, some of you may have found instead the EM_HIDESELECTION message in the Win32.HLP. If you had delved in, you would have found something very interesting. The Delphi HideSelection property only implements half the capabilities of this message. You can Also, by Calling It Direct, Tell It To Temporalily Hide The Black Stripe Even The Control Has The Focus. so instead you could Use the Following Lines of Code:
Begin
Myre: = trichedit (sender);
Myre.Perform (em_hideselection, 1, 0);
~~~~~~
Myre.perform (em_hideselection, 0,0);
end
Yummy. Nice Clean Coding:
Death to the Flicker
The next major problem is this bloody flicker. You should pop back into Delphi for a second, and types some lines in its editor, to see if it flickers at all. It does. But only when it is changing colors when it recognizes a change HAS Occured. OtherWise It Doesn't Bother. Now Look At What's Happening in Our "Editor". Do you see?
The problem is that we are not "conserving" what we are doing. If something is still the same TokenType it does not need to be re-highlighted because it already correct on the screen. We need to check if the TokenType of each token HAS Changed Since Last Time We repainted this line, and only the repaint to.
In fact we do not need to do even that -. We can just check whether the SelAttributes (which represents the current selection's attributes) is any different from what we want to change it to ie FParseFont [MyTokenType] This way if even the TokenType Had Changd, But The New and Old Tokenype Shared The Same Display Attributes, We Would Still Conserve Our Drawing.
Actually the problems is that the RichEdit is not doing the conserving. In the old text based system I used to use, if you printed something to the screen, and it was the same as something already on the screen, in the same position, then the program would not rewrite it to the screen It would "conserve" the amount of writing it did, as in the old days 1200 baud screens were SLOW, and printing the same characters was a waste of time Huh -.. and people said WE Have Come So Dar with Windows. Sloppy, Sloppy, Sloppy I Say! :-)
So lets replace:
Myre.selattributes.assign (pascon.fparsefont [myTokenState]);
With:
If MyRe.SelAttributes.Name <> PasCon.FParseFont [MyTokenState] .Name then MyRe.SelAttributes.Name: = PasCon.FParseFont [MyTokenState] .Name; If MyRe.SelAttributes.Color <> PasCon.FParseFont [MyTokenState] .Color then Myre.selattributes.color: = pascon.fparsefont [MyTokenState] .color;
IF myre.selattributes.style <> pascon.fparsefont [myTokenState] .Style Then myre.selattributes.style: = pascon.fparsefont [myTokenState] .style;
And off you go and try it out ... (ps. Yes the last bit of code is big programming ...)
