PEP: 255 Title: Simple GENERATOR version: $ Revision: 1.18 $ Author: Neil Schemenauer
Completion time: 18-May-2001python version number: 2.2post-history: 14-JUN-2001, 23-JUN-2001
Summary This PET introduces the concept of Generator in the Python language, as well as some related statements, "yield" statements in them. Purpose When a process function is to handle a fairly difficult task, it requires maintenance of the process value, most programming languages add a callback function to the parameter list of process functions without more efficient and more appropriate solutions. Call each process value. For example: Tokenize.py within a standard library adopts the following method: The caller must pass a "tokeneater" to tokenize (), that is, whenever () can find the next token. This makes the tokenize () function can be encoded in a natural way. However, the program that calls tokenize is very troublesome, because it needs to remember which token is finally seen when callback. The Tokeneater function in tabnanny.py is a good example. It maintains a status device in the global variable, let it remember when you have seen it, which is what you want to see. This is difficult for it to run correctly and is difficult to understand. Unfortunately, this is the typical place of this method. For Tokenize, another option is to immediately generate a Python program fully explained in a large list. Thus, the Tokenize client will be written in a natural manner, using local variables and local control flows (such as loops and nested IF statements) to track their status. But this is also unrealistic: the program will be very large, the memory required to achieve the entire interpretation will be unable to limit, but the tokenize client just wants to see if the special thing will happen, these events may only be in the program. Former part (such as: future statement, idle call, first nested statement), so first trace the overall process of waste time. Another way is to use tokenize as an iterator [1], whether it is .NEXT () method, it passes the token. This is a happy thing for the caller, which is the same for the big list of the upcoming results, and it does not have memory and "which is what I want to take it out?" Problem shortcomings. Even, he also reduced the burden on the status of the * ITS * that shouldkenize remembers the .next () function call. And the reader can realize that it is a terrible thing that it is a terrible cumbersome thing as long as the reader is mentioned. Or describe a recursive algorithm to generate an ordinary tree structure encoding: put it into an iterator frame requires manual unloading of recursive and manually maintaining this round-trip movement. The fourth way is to run the producer and consumers on a separate thread. This can keep them in a natural way, so this is happy for both. In fact, Demo / Threads / Generator.py in the Python resource distribution provides a synchronous-communication class that can be used, which can be held in a universal manner. This method is unable to work on the platform that does not support threads, even if it works, it is very slow (compared to methods that do not use threads). The last method is to use the non-stack [2] [3], different Python implementations, which supports lightweight collaborative procedures. This has the same effect as threads and is more effective. However, there is no stack method is a controversial idea of Python core, and for Jython, it is impossible to achieve the same semantic. However, PEP is not a place to debate this problem, so it can be sufficiently said here that the Generator provides a useful stack of bonji with a useless stack function in some way. It is easy to suit the current CPYTHON implementation process, and I believe it is quite easy to understand for other Python. All alternative methods are listed above.
Some other high-level languages provide a happy solution and outstanding iterator in Sather [4]. SATHER is inspired by the iterator from the CLU; they also provide Generator in Icon [5], a new language that represents "is a recognor". These are different in performance, but the basic concept is the same: to provide a function that can return a middle result ("next value") to its caller; but hold the local state of the function, so it can be in it Renained place to leave. Here is a simple example: DEF FIB (): a, b = 0, 1 while 1: Yield B A, B = B, A B
When the FIB () function is called, he sets a = 0, b = 1, so Yield B returns to the caller, the caller saw 1, when the FIB continues, from its point of view, The Yield statement is actually the same as the print statement: Yield execution, all local variables have not changed, and FIB continues. A and B will become 1 and 1, and the FIB loop back to Yield call, which gives 1 calling program. Push it in this class. From the view of FIB, it is just the order of the results, as if it is the same as the callback procedure. However, from the caller's point of view, the FIB call is an over-start freely object. In the thread mode, this allows both sides to be encoded in natural way; however, it is different from the thread that it can be more efficient and operate on all platforms. In fact, restart a Generator should call us a similar resource. The same manner is applied to the multi-producer / consumer function. For example: tokenize.py can assign a value to the next token, rather than calling a callback function as a parameter, and the Tokenize client function can iterate the token with a natural way; a Python Generator is a Python iterator [1] And quite powerful. Note: Yield is introduced in this way: Yield_Stmt: "Yield" expression_list "yield" statement is a new keyword, all Future statements [8] need to experience the following phases: In initial release, you need to use Generator The module must include underscore: from __future__ import generators and placed in the beginning of the beginning (see PEP 236 [8])). The module that uses the flag "Yield" without a FUTURE statement will cause a warning. In the next release, "Yield" will become a language keyword, and the FUTURE statement does not need. The Yield statement can only be used inside the operation function. The operation function including a Yield statement is called the generator operation function. The Generator operation function is a normal operation function in all aspects, but there is a new CO_generator tag setting in the CO_FLAGS member of the code object. When a Generator function is called, the actual parametric corresponds to the shape of the local operation function, but does not perform code in the body of the operation function. It no longer returns a Generator-iterator object, but abides by the iterator protocol [6], so in special circumstances can be applied in a natural manner in the FOR cycle statement. Remember: When the context meaning clearly, the absolute name "generator" either refers to a Generator operation function, or refers to the generator-iterator. Every time you call a .next () function of the generator-iterator, the encoding in the body of the Generator operation function will continue until a Yield statement or return statement (see above), is the end of the body. If you touch the Yield statement, the status of the operation function is frozen and returns the value of the parameter list to the call at .next (). Frozen means all states are maintained, including the current bundle of local variables, instruction pointers, and internal computing stacks: Save enough information for the next time to call .next (), the operation function can be executed accurately, It seems that Yield statement is just a call.
Limit: YIELD statement does not allow an TRY clause in a try / finally structure. The problem is that it cannot guarantee that when Generator can restart, but it is impossible to ensure that the finally block is executed; it is too much for Finally purposes. Limit: Do not start again at Generator: >>> def g (): ... i = me.next () ... Yield I >>> ME = g () >>> me.next () TRACEBACK (MOST Recent Call Last): ... file "
For example, if you do not understand language but you can explain this view): >>> def f (): ... return 1/0 >>> DEF g (): ... Yield f () # The Zero Division Exception Propagates ... yield 42 # and we'll never get here >>> k = g () >>> k.next () Traceback (MOST Recent Call Last): file "
Note: TRY / EXCEPT / FINALLY is like the above, the Yield statement does not allow in a TRY clause in a try / finally structure. The result is that Generator should be very careful to allocate key resources. There is no restriction in the FinalLe clause, Except clause, or the TRY clause of the TRY / Except structure in the Try / Except structure: >>> def f (): ... Try: ... yield 1 ... : ... yield 2 ... 1/0 ... Yield 3 # never get here ... Except ZerodivisionError: ... Yield 4 ... Yield 5 ... raise ... ExcePt: ... Yield 6 ... Yield 7 # The "Raise" Above Stops this ... Except: ... Yield 8 ... Yield 9 ... Try: ... x = 12 ... Finally: ... Yield 10 ... Yield 11 >>> Print List (f ()) [1, 2, 4, 5, 8, 9, 10, 11] >>> Example # a binary tree class. Class Tree: Def __init __ (self, Label, left = none, right = none): self.label = label self.LEFT = left self.right = Rightdef __repr __ (self, level = 0, indent = "): s = level * indent ` Self.Label `If self.Left: S = s "/ n" self.Left .__ REPR __ (Level 1, Indent) if self.right: s = s "/ n" self.right .__ REPR __ (Level 1, Indent) Return S
DEF __ITER __ (Self): Return Inorder (Self)
# Create a tree from a list. Def tree (list): n = len (list) if n == 0: return [] i = N / 2 return Tree (list [i], tree (list [: i]) , Tree (List [i 1:])))
# A recursive generator That Generes Tree Labels in in-Order. Def inorder (t): IF T: for x in inorder (t.left): yield x yield T.Label for x in inorder (t.right): yield x # SHOW IT OFF: CREATE A TREE. T = Tree ("AbcdefghijklmnopqrStuvwxyz") # Print The Nodes of The Tree in-Order. For x in t: print x, print
# A Non-recursive generator. Def inorder (node): stack = [] while node: while node.right: stack.Append (node) node = node.left yield node.label While NOT NODE.right: try: node = Stack.pop () Except Indexerror: Return Yield Node.Label Node = Node.right
# EXERCISE The Non-Recursive Generator. For x in t: Print X, Print
The output block of the two is displayed as: A B C D e f G H i J K L M N o P Q R S T U V W x YZ Questions & Answers: Why don't you use a new keyword instead of reuse "DEF"? Solution: Please see the BDFL section below. Question: Why use a new key to indicate "yield"? Why don't you replace it already existing functions? Answer: The control process is much better through the keyword expression in the Python language, and Yield is a control structure. We also believe that using Jython language is effective in practical process requires compilers to determine potential breakpoints in compile time, and a new keyword makes this process easier. CPYTHON Reference The implementation process also has to develop it, in order to determine which operation function is the generator operation function (although use a new keyword instead of "DEF" to solve this problem for CPYTHON - but people will ask "Why use a new Keywords ", it doesn't need any new keywords). Question: So why do some other special syntax don't have to have a new keyword? For example, one of them can replace "Yield 3": return 3 and continuRn and continue 3 return generating 3 Continue Return 3 Return >>, 3 from Generator Return 3 Return >> 3 Return << 3 >> 3 << 3 * 3 Answer: I missed a
At the beginning, the lack of "Return EXPR" is forcibly used. For the transfer of the value, only "Yield" is simple and clear. BDFL Declaration Protection: Introduction to another new keyword ("gen" or "generator") replacing "DEF", or other alternative syntax, distinguishing the generator operation function and the non-Generator operation function. Opposition: In fact (no matter how you think), Generator belongs to the operation function, but they have changed slightly, they can be restarted. How do they start mechanisms are relatively small technical puzzles, and introducing a new keyword does not help fully emphasize how Generator is started with a mechanism (a key in the life of Generator). Expert: In fact (no matter how you think), the Generator operation function is a real library function. Its magic produces the generator-iterator. In this regard, they have the essence of the nonenerator operation function, which is more like a structure, not just an operation function, so repeatedly uses "DEF" is the best choice. "Yield" hidden in the body is not enough to warn: this part is so different. BDFL: "DEF" it retains, any party is not completely convincing, so I consulted my language designer. It tells us that the syntax assumed in the PEP is quite correct - neither hot, not too cold. However, like the prediction of Greek mythology, it will not tell us why, so I will not argue in PEP. What I can can ask (away from the defensive ... has been done) is "ful". Once a day is part of the language, I am very suspected whether it plagiarized the "Python" article "of Andrew Kuchling". Reference implementation process current implementation, in the initial stage (no document, but testing is good and reliable). It is part of Python CVS Development Tree [9]. It requires you to create a Python language from the resource. This article is derived from an early patch of Neil Schemenauer [7].
