Everyone knows that PB's data pipe is a very useful data transfer tool that supports data transfer between multiple databases, and has a complete transmission error control mechanism.
I have seen some netizens implemented data pipelines, and there are several kinds:
1. Data pipeline object, pre-established, then in the program, call these pipe objects, implement data transmission, this can be transmitted in compiled programs'
2. There are also dynamically modified pipe SQL syntax, but only modify the search criteria.
This approach, if data is transmitted, or the table or the necessary data pipes must be established in the development environment.
The data pipes I have now considering implement the following features:
1. Can be transferred between the two databases arbitrarily;
2. You can transfer any set of data between two tables, so-called arbitrary is that no need to establish any data pipeline object in the development environment, the pipeline object is generated by the program;
3. A batch of data pipes can be generated according to the user selected, and data is transmitted at once.
4. Support data between multiple databases (including the same and different DBMS databases).
If you can implement the above 3 points, just connect the upper source database, destination database, select the number of tables that need to be passed, and write the range of data sets (SQL conditions), click "Transfer", you can sit down and drink cup of coffee. (Because the pipeline transmission efficiency is too low, wait.
However, to achieve this function, talk about what is easy, in fact, one of the priorities should be how to generate different pipe syntax for a variety of databases, I don't know how to use PB's related DLL function, otherwise it can be separated into data pipe syntax The function is really a big contribution to the PB forum.
Gossip less, let's talk about low-level processing methods, the following is my profile design and analysis: 1. First determine those necessary elements that build dynamic data pipes, these elements take it from there?
After analysis, you need to know that the source table has those columns, these columns, whether to allow empty, whether it is the primary key, the score of the column, the default value, etc., these tables need to be taken from the system table, and different databases, systems Table is often different, the table structure is different.
You also need to build the SQL statement of the pipeline. For transport table data, you only need to specify where the WHERE condition can be used, which can be directly input to SQL directly.
2. How to build a pipeline grammar in the PB code?
Considering that a variety of databases must be supported, different database data type types may vary, while saving table columns and with supported databases continue to increase, control is more complicated, how to make the program more light, clear, stable and easy to expand? I Considering that the first thing is that the database is supporting standard SQL, then the basic SQL element required to establish a destination table can create a set of intermediate views (or tables), through these views (or tables), PB The program can be easily manipulated using DataStore, and generates a pipe syntax.
to be continued...
