|
| 1 | +The following introduction will take this SQL statement as an example: |
| 2 | +```SQL |
| 3 | +select a+b*c, c from test where a>10; |
| 4 | +``` |
| 5 | + |
| 6 | +Firstly, according to the link you provided, we can see that the main calculation code starts in BatchPrimitiveProcess::execute() from here: |
| 7 | +```c++ |
| 8 | + fe2Output.resetRowGroup(baseRid); |
| 9 | + fe2Output.getRow(0, &fe2Out); |
| 10 | + fe2Input->getRow(0, &fe2In); |
| 11 | + |
| 12 | + for (j = 0; j < outputRG.getRowCount(); j++, fe2In.nextRow()) |
| 13 | + { |
| 14 | + if (fe2->evaluate(&fe2In)) |
| 15 | + { |
| 16 | + applyMapping(fe2Mapping, fe2In, &fe2Out); |
| 17 | + fe2Out.setRid(fe2In.getRelRid()); |
| 18 | + fe2Output.incRowCount(); |
| 19 | + fe2Out.nextRow(); |
| 20 | + } |
| 21 | + } |
| 22 | +``` |
| 23 | +
|
| 24 | +By calling the `evaluate` method of `fe2`, with a reference of `row` type `fe2In` as the parameter. The execution of the code then enters the `FuncExpWrapper`, which contains the current calculation expression. It mainly performs two steps, which are: |
| 25 | +
|
| 26 | +1. Filtering the conditions in the `where` statement through the functions defined in the `filters` variable. If the `where` condition is met, the calculation proceeds to the next step; otherwise, it will return directly. |
| 27 | +2. Calculating the data in `row` through `expression` and obtaining the final result. |
| 28 | +
|
| 29 | +```c++ |
| 30 | +bool FuncExpWrapper::evaluate(Row* r) |
| 31 | +{ |
| 32 | + uint32_t i; |
| 33 | +
|
| 34 | + for (i = 0; i < filters.size(); i++) |
| 35 | + // filter the conditions in the where statement |
| 36 | + if (!fe->evaluate(*r, filters[i].get())) |
| 37 | + return false; |
| 38 | + // calculate the row that meets the where condition |
| 39 | + fe->evaluate(*r, rcs); |
| 40 | +
|
| 41 | + return true; |
| 42 | +} |
| 43 | +``` |
| 44 | + |
| 45 | +That is to say, the function that actually performs the calculation is the `evaluate` method pointed to by `fe`. Its parameters are a `row` and a `ReturnedColumn` type array `expression`. |
| 46 | +```c++ |
| 47 | +void FuncExp::evaluate(rowgroup::Row& row, std::vector<execplan::SRCP>& expression) |
| 48 | +{ |
| 49 | + bool isNull; |
| 50 | + |
| 51 | + for (uint32_t i = 0; i < expression.size(); i++) |
| 52 | + { |
| 53 | + isNull = false; |
| 54 | + |
| 55 | + switch (expression[i]->resultType().colDataType) |
| 56 | + { |
| 57 | + case CalpontSystemCatalog::DATE: |
| 58 | + { |
| 59 | + int64_t val = expression[i]->getIntVal(row, isNull); |
| 60 | + |
| 61 | + // @bug6061, workaround date_add always return datetime for both date and datetime |
| 62 | + if (val & 0xFFFFFFFF00000000) |
| 63 | + val = (((val >> 32) & 0xFFFFFFC0) | 0x3E); |
| 64 | + |
| 65 | + if (isNull) |
| 66 | + row.setUintField<4>(DATENULL, expression[i]->outputIndex()); |
| 67 | + else |
| 68 | + row.setUintField<4>(val, expression[i]->outputIndex()); |
| 69 | + |
| 70 | + break; |
| 71 | + } |
| 72 | + |
| 73 | + .... |
| 74 | +} |
| 75 | +``` |
| 76 | +
|
| 77 | +In this function, each expression will be looped through, and the calculation results will be obtained by calling the `getValue` method. Taking the SQL statement mentioned at the beginning as an example, there will be two expressions, representing the columns `a + b * c` and `c`. Next, we will use the `a + b * c` expression as an example to introduce its data type and calculation process. |
| 78 | +
|
| 79 | +`a + b * c` is an arithmetic operation column, so its type is `ArithmeticColumn` inherited from `ReturnedColumn`. It contains a member variable of type `ParseTree* fExpression`, as well as the implementation of the `getIntValue` method: |
| 80 | +```c++ |
| 81 | +virtual int64_t getIntVal(rowgroup::Row& row, bool& isNull) |
| 82 | +{ |
| 83 | + return fExpression->getIntVal(row, isNull); |
| 84 | +} |
| 85 | +``` |
| 86 | + Here is AST graphical representation of a the given math expression. |
| 87 | +After obtaining the structure of `fExpression`, we continue to look at the calculation process and enter the `getIntVal` method pointed to by `fExpression`: |
| 88 | + |
| 89 | +```c++ |
| 90 | +//fExpression->getIntVal(row, isNull); |
| 91 | + inline int64_t getIntVal(rowgroup::Row& row, bool& isNull) |
| 92 | + { |
| 93 | + if (fLeft && fRight) |
| 94 | + return (reinterpret_cast<Operator*>(fData))->getIntVal(row, isNull, fLeft, fRight); |
| 95 | + else |
| 96 | + return fData->getIntVal(row, isNull); |
| 97 | + } |
| 98 | +``` |
| 99 | +From the code, we can see that the entire calculation process is actually a recursive process. When both the left and right subtrees of the current node are not empty, the current node is converted into an `Operator` type and continues to recursively call; otherwise, the value of the leaf node is returned. Taking the binary tree in the image as an example, when both left and right subtrees exist, the current node's `TreeNode` will be converted to an `operator` type (in this case, the `ArithmeticOperator` type), and the `getIntVal` method defined in it will be recursively called for calculation. When it is a leaf node, the node data is extracted. |
| 100 | +
|
| 101 | +Each row of data will go through this recursive process, which will affect the calculation performance to some extent. Therefore, here is my implementation plan for the GSOC project: |
| 102 | +
|
| 103 | +Since we can obtain the `ParseTree` before the calculation, we can dynamically generate LLVM calculation code based on the `ParseTree` using JIT technology. When executing `void FuncExp::evaluate(rowgroup::Row& row, std::vector<execplan::SRCP>& expression);`, it is only necessary to call the generated LLVM code, which can avoid the recursive calculation operation for each row of data. In this way, there will be only one recursive call (when dynamically generating code using JIT). |
| 104 | +
|
| 105 | +The above are some thoughts I had after reading this part of the code. I hope you can give me some suggestions. Thank you very much. |
0 commit comments