Some scanners (such as those which support "include" files) require reading from several input streams. As flex scanners do a large amount of buffering, one can not control where the next input will be read from by simply writing a YY_INPUT which is sensitive to the scanning context. YY_INPUT is only called when the scanner reaches the end of its buffer, which may be a long time after scanning a statement such as an "include" which requires switching the input source.
To Negotiate There Sorts of Problems, Flex Provides A Mechanism for Creating and Switching Between Multiple Input Buffers. An Input Buffer IS CREATED BYUSING:
YY_BUFFER_STATE YY_CREATE_BUFFER (File * file, int size)
which takes a FILE pointer and a size and creates a buffer associated with the given file and large enough to hold size characters (when in doubt, use YY_BUF_SIZE for the size). It returns a YY_BUFFER_STATE handle, which may then be passed to other routines (see below). The YY_BUFFER_STATE type is a pointer to an opaque struct yy_buffer_state structure, so you may safely initialize YY_BUFFER_STATE variables to ((YY_BUFFER_STATE) 0) if you wish, and also refer to the opaque structure in order to correctly declare input buffers . in source files other than that of your scanner Note that the FILE pointer in the call to yy_create_buffer is only used as the value of yyin seen by YY_INPUT; if you redefine YY_INPUT so it no longer uses yyin, then you can safely pass a nil File Pointer to YY_CREATE_BUFFER. You SELECT A Particular Buffer To Scan from Using:
Void yy_switch_to_buffer (YY_Buffer_State new_buffer)
switches the scanner's input buffer so subsequent tokens will come from new_buffer. Note that yy_switch_to_buffer () may be used by yywrap () to set things up for continued scanning, instead of opening a new file and pointing yyin at it. Note also that switching input Sources via either yy_switch_to_buffer () or yywrap () does not change the start condition.void yy_delete_buffer (YY_Buffer_State Buffer)
Is Used to Reclaim The Storage Associated with a buffer. You Can Also Clear The Current Contents of A Buffer Using:
Void yy_flush_buffer (YY_Buffer_State Buffer)
This Function Discards The Buffer's Contents, So The Next Time The Scanner Attempts To Match A Token From The Buffer, IT Will First Fill The Buffer Anew Using Yy_Input.
YY_NEW_BUFFER () is an alias for yy_create_buffer (), provided for Compatibility with the C use of new and delete for create and destroying Dynamic Objects.
Finally, The YY_CURRENT_BUFFER Macro Returns a yy_buffer_state handle to the current buffer.
Here is an esample of using these features for Writing a scanner Which Expands include files (the << EOF >> Feature IS Discussed Below):
/ * The "incl" state is buy for picking up the name
* of an include file
* /
% x INCL
% {
#define max_include_depth 10
YY_BUFFER_STATE INCLUDE_STACK [MAX_INCLUDE_DEPTH];
INT INCLUDE_STACK_PTR = 0;
%}
%%
INCLUDE BEGIN (INCL);
[A-Z] ECHO;
[^ a-z / n] * / n? echo;
IF (include_stack_ptr> = max_include_depth) { FPRINTF (stderr, "incrudes nested to deeply); Exit (1); } INCLUDE_STACK [INCLUDE_STACK_PTR ] = YY_CURRENT_BUFFER; YYIN = FOPEN (YYTEXT, "R"); IF (! yyin) Error (...); YY_SWITCH_TO_BUFFER YY_CREATE_BUFFER (YYIN, YY_BUF_SIZE)); Begin (Initial); } << EOF >> { IF (--NClude_stack_ptr <0) { Yyterminate (); } Else { YY_DELETE_BUFFER (YY_CURRENT_BUFFER); YY_SWITCH_TO_BUFFER INCLUDE_STACK [INCLUDE_STACK_PTR]); } } Three routines are available for setting up input buffers for scanning in-memory strings instead of files. All of them create a new input buffer for scanning the string, and return a corresponding YY_BUFFER_STATE handle (which you should delete with yy_delete_buffer () when done with IT). The next call to yylex () Will Start Scanning The String. YY_SCAN_STRING (Const Char * STR) Scans a Nul-Terminated String. YY_SCAN_BYTES (Const Char * Bytes, INT LEN) SCANS LEN BYTES (Including Possibly Nul ') Starting At Location Bytes. Note That Both of these Functions Create and scan a copy of the string or bytes (Since YyleX () Modifies The Contents of the Buffer IT IS Scanning.) You can avoid the copy by using: YY_SCAN_BUFFER (Char * Base, YY_SIZE_T SIZE) which scans in place the buffer starting at base, consisting of size bytes, the last two bytes of which must be YY_END_OF_BUFFER_CHAR (ASCII NUL) These last two bytes are not scanned;. thus, scanning consists of base [0] through base [size -2], Inclusive. If You Fail to set up base in this manner (i.e., forget the final two yy_end_of_buffer_char bytes), The YY_SCAN_Buffer () Returns A NIL Pointer INSTEAD OF CREATING A New Input Buffer.
