description	keywords
dlt source for morphais.com	Morphais API morphais.com

dlt-source-morphais

DLT source for Morphais.

Currently loads the following data:

Table	Contains
persons	Items of the Person model with all properties
persons_*	Multi-value properties derived of the Person model
highlights	Unique highlights
audiences	Unique audiences (B2B, B2C, etc.)
legal_forms	Unique legal forms (GmbH, Ltd., etc.)
funding_stages	Unique funding stages (Pre-Seed, Seed, etc.)
industries	Unique industries
solutions	Unique solutions
startups	Items of the Startup model with all properties
startups_*	Multi-value properties derived of the Startup model

Usage

Create a .dlt/secrets.toml with your API key and email:

morphais_email="<YOUR-EMAIL>"
morphais_api_key="<YOUR_API_KEY>"

and then run the default source with optional list references:

from dlt_source_morphais import source as morphais_source

pipeline = dlt.pipeline(
   pipeline_name="morphais_pipeline",
   destination="duckdb",
   dev_mode=True,
)
morphais_data = morphais_source()
pipeline.run(morphais_data)

Development

This project is using devenv.

Commands:

Command	What does it do?
generate-model	generates the morphais Pydantic model from the current spec file, applies patches, etc.
update-spec	Pulls in the latest main#HEAD of planet-a-ventures/morphais-openapi-spec
refresh-model	Both commands above plus adds it to git and commits the changes.
format	Formats & lints all code
sample-pipeline-run	Runs the sample pipeline. By default dev_mode=True which fetches resources with a limit of 1 (page)
sample-pipeline-show	Starts the streamlit-based dlt hub

Run the sample

MORPHAIS_EMAIL=[...] \
   MORPHAIS_API_KEY=[...] \
   sample-pipeline-run

alternatively you can also create a .dlt/secrets.toml (excluded from git) with the following content:

morphais_api_key="..."
morphais_email="[email protected]"