Skip to content

Conversation

@pbrady
Copy link

@pbrady pbrady commented Jan 31, 2025

The default option of "function_calling" for Ollama was leading to errors. Defaulting to "json_schema" (as ChatOpenAI already does) results in success for Ollama. Expose the method argument to the user for finer-grained control if needed.

Fixes #38.

I'm not sure if this is the right approach but figured it would be a good starting point for a discussion. The script below used to trigger the parsing error mentioned in the issue. With the fix here, Ollama is able to produce Nodes and Relationships:

from langchain_ollama import ChatOllama
from langchain_experimental.graph_transformers import LLMGraphTransformer
from langchain_core.documents import Document

llm = ChatOllama(model="llama3.1", num_ctx=4096)
doc = Document(
    "The lazy dog watched as the red ball rolled.  Bob the clown scared the missles."
)

allowed_nodes = ["Animal", "Person", "Object"]

doc_transformer = LLMGraphTransformer(
    llm=llm, allowed_nodes=allowed_nodes
)

graph_doc = doc_transformer.convert_to_graph_documents([doc])
print(graph_doc)

Output:

GraphDocument(nodes=[Node(id='Lazy Dog', type='Person', properties={}), Node(id='Red Ball', type='Object', properties={}), Node(id='Bob The Clown', type='Person', properties={})], relationships=[Relationship(source=Node(id='Lazy Dog', type='Person', properties={}), target=Node(id='Red Ball', type='Object', properties={}), type='WATCHED', properties={}), Relationship(source=Node(id='Bob The Clown', type='Person', properties={}), target=Node(id='Missles', type='Object', properties={}), type='SCARED', properties={})], source=Document(metadata={}, page_content='The lazy dog watched as the red ball rolled.  Bob the clown scared the missles.'))]

The default option of "function_calling" for Ollama was leading to
errors.  Defaulting to "json_schema" (as ChatOpenAI already does)
results in success for Ollama.  Expose the `method` argument to the
user for finer-grained control if needed.
Copy link
Contributor

@efriis efriis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(ignore initial review - thought this was modifying ChatOllama not this)

@efriis efriis self-assigned this Feb 8, 2025
)
structured_llm = llm.with_structured_output(schema, include_raw=True)
structured_llm = llm.with_structured_output(
schema, method=method, include_raw=True
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I misunderstood - let's pass this in as a kwarg when it's defined, but let's remove the default of json_schema above.

Also open to something more general like structured_output_kwargs: Optional[dict] where you'd achieve this with structured_output_kwargs={"method": "json_schema"}

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review. I went with structured_output_kwargs in the latest commit.

@pbrady pbrady requested a review from efriis February 24, 2025 16:34
@pbrady
Copy link
Author

pbrady commented Mar 29, 2025

This should be ready to be reviewed again

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

No open projects
Status: In review

Development

Successfully merging this pull request may close these issues.

Leaving method unspecified in LLMGraphTransformer triggers Pydantic parsing errors

2 participants