Refactor output directory structure and add TEI-to-JSON conversion #28
Conversation
- Renamed output directory from `output_xmls` to `output_teis` for clarity and consistency. - Updated `grobid_service.py` to save processed PDFs to the new output directory. - Added a new module `tei_to_json.py` for converting TEI XML documents to JSON format. This module includes functions for recursive XML parsing and saving the JSON output. - Expanded development dependencies in `dev.txt` to include `lxml`.
There was a problem hiding this comment.
Why write your own Grobid class when there is a client library that already exists? https://github.com/kermitt2/grobid_client_python
You can install it from git by putting git+https://github.com/kermitt2/grobid_client_python in your requirements file and pip installing it or you can run pip install git+https://github.com/kermitt2/grobid_client_python
There was a problem hiding this comment.
I did try this library and it was a complete pain and seemed like it was in need of being updated, so i just went with my own... I'm kind of blanking on what problems I ran into so maybe worth looking into again. But my memory is this library was not great.
| return child_dict | ||
|
|
||
|
|
||
| def tei_to_json(input_tei_path: str, output_json_path: str) -> None: |
There was a problem hiding this comment.
https://github.com/allenai/s2orc-doc2json
I think this library does exactly what you are trying to do here. Can you reuse their work?
output_xmlstooutput_teisfor clarity and consistency.grobid_service.pyto save processed PDFs to the new output directory.tei_to_json.pyfor converting TEI XML documents to JSON format. This module includes functions for recursive XML parsing and saving the JSON output.dev.txtto includelxml.