transform helper (#56)

* adding helper function to utils.py

The helper function parses the run properties and results data so that the user doesn't need to include any parsing script in their transform script. The helper function only accepts a user-defined function that accepts an array of results data as its input and returns an array of results data matching the assay design it is used with. Also, the helper function requires that users explicitly define the run properties in their transform script, otherwise the labkey transform script parser does not recognize ${runInfo}.

* Create example transform script that uses labkey.utils.transform_helper

* Delete assay_transform_function_example.py

moving filter to /docs folder

* documentation for transform_helper function

* Update transform_helper.md

* Update transform_helper.md

* updating transform_helper to be more robust and use snake case

switched from camel case to snake case for variable naming. additionally, changed script to get file_path_in from the row beginning with "runDataUploadedFile" in runProperties file. This is to better handle results data with numerical header names, such as when the results data is a grid representing well plates.

* Update labkey/utils.py

Co-authored-by: Alan Vezina <[email protected]>

* Update utils.py

using with statements when opening files

---------

Co-authored-by: Alan Vezina <[email protected]>

Author: jryurkanin

docs/transform_helper.md

@@ -0,0 +1,45 @@
+# Transform Helper Function Support
+
+This helper function provides a convenient way for users to write transform scripts for assays without requiring the user to read the results and run data before transforming the data, and it also takes care of writing the transformed data back to labkey.
+
+### Importing and preparing the helper function
+
+First, your script must begin with these two lines to properly use the transform helper function:
+
+```python
+from labkey.utils import transform_helper
+
+filepath = '${runInfo}'
+```
+
+The substitution token `${runInfo}` is replaced with the runProperties file path when the transform script is run within LabKey Server.
+
+### Write your user-defined transform function
+
+This is an example function that could be used as an argument when using the transform_helper function:
+
+```python
+def transform(grid):
+    isHeaderChecked = False    
+    # iterate through the rows of your results data, checking for the header
+    for row in grid:
+        if isHeaderChecked == False:
+            row.append('averageResult')
+            isHeaderChecked = True
+        else:
+            # In this example, we are taking the average of the 3rd-5th column values by row 
+            # and appending that average value in a 6th column called "averageResult"
+            newValTemp = sum([float(val) for val in row[2:]])/len(row[2:])
+            row.append(round(newValTemp, 2))
+    return grid
+```
+
+As you can see in the example function, the transform function must accept only one argument, the tabular data (including headers) that needs to be transformed in an list of lists format. The user-defined transform function must also return tabular data the same list of lists format with a header row that matches the header of the results data as defined by the targeted assay design.
+
+### Using the helper function
+
+The last thing to do is call the transform_helper function, supplying the user-defined transfrom function as the first argument, and the file path (as defined by ${runInfo}) as the second argument.
+
+```python
+transform_helper(transform, filepath)
+``` 

labkey/utils.py

@@ -31,3 +31,42 @@ def default(self, o):
 def json_dumps(*args, **kwargs):
     kwargs.setdefault("cls", DateTimeEncoder)
     return json.dumps(*args, **kwargs)
+
+
+def transform_helper(user_transform_func, file_path_run_properties):
+    #file_path_run_properties must be explicitly defined as ${runInfo} within the user's transform script
+    #parse run properties to for results data in and out filepaths
+    file_path_in = ''
+    file_path_out = ''
+    with open(file_path_run_properties) as file_run_properties:
+        for l in file_run_properties:
+            row = l.strip().split('\t')
+            if row[0] == 'runDataFile':
+                file_path_out = row[3]
+            if row[0] == 'runDataUploadedFile':
+                file_path_in = row[1]
+    
+    #parse results data into array, confirming supported file type is used
+    with open(file_path_in) as file_in:
+        data_grid = []
+
+        for l in file_in:
+            if '\t' in l:
+                row = l.replace('\n', '').split('\t')
+            elif ',' in l:
+                row = l.replace('\n', '').split(',')
+            else:
+                raise ValueError('Unsupported file type or delimiter used. Header used: \n' + str(l))
+            data_grid.append(row)    
+    
+    #run user transform on parsed results data array
+    transformed_grid = user_transform_func(data_grid)
+    
+    #write transformed results data array to LabKey assay results data grid
+    #transformed array must be a python list object, not a numpy array or pandas dataframe
+    with open(file_path_out, mode='w') as file_out:
+        for row in transformed_grid:
+            row = [str(el).strip() for el in row]
+            row = '\t'.join(row)        
+            file_out.write(row + '\n')
+