Step Function registered flow: MetaflowCard named html not found

[michaelhagel]

Issue

Local runs attached to an S3 metadata store integrate seamlessly with the HTML card. Utilizing the Netflix Metaflow extensions, we also install metaflow-card-html within the Flow-level @pip_base decorator. Take this example of running a monitoring job:

@card(type="html", options={"attribute":"html_report"})
@step
def run_reports(self):
    report = Report(
        metrics=[*self.monitor._reports]
    )

    report.run(
        reference_data=self.monitor.datasets.reference,
        current_data=self.monitor.datasets.current,
        column_mapping=self.col_mapping
    )

    self.html_report = str(get_evidently_html(report))

    self.next(self.run_test_suites)

We will see the card for the corresponding run in the Metaflow UI.

However, once we register this in a flow as a step-function w/ Batch tasks, we receive the following error:
MetaflowCard named html not found. Check the type attribute in @card
Note this error references the attribute name html regardless of what option is set in the decorator, which to me implies that the underlying Batch runner doesn't know about this card extension.

Question

For Step Function flow deployments, do we need to have the HTML card module installed anywhere else within our Metaflow infra? Is there a recommended way of debugging this card rendering fail, regardless of the answer to the aforementioned question?

[michaelhagel]

My best guess is that I need to update the Batch runner's metaflow_extensions dir directly

[elutins]

@michaelhagel did you end up finding a solution? I am running into the same issue when running a flow on AWS batch.