classification model versus extraction model

[mophilly]

For classification I use a very simple model that has a few elements. For extracting the bulk of a document I have a complex model, with employs several classes. In working the splitter into the work flow, running into errors from deep in ExtractThinker, e.g. "list index out of range" when invoking the routine for classification.

The routine for the complex document:

            document_loader = DocumentLoaderPyPdf()

            # create the extractor
            extractor = Extractor()
            extractor.load_document_loader(document_loader)
            extractor.load_llm("gpt-4o")  # or gpt-4o-mini

            # Extract data from your document
            result = extractor.extract(source_file_path, RunStatementContract, vision=True)

The routine for the classification:

        try:
            tesseract_path = os.getenv("TESSERACT_PATH")
            process = Process()
            process.load_document_loader(DocumentLoaderTesseract(tesseract_path))
        except Exception as e:
            raise SplittingError(f"Splitting error 1: {str(e)}")

        # Initialize text splitter with model
        try:
            process.load_splitter(TextSplitter("claude-3-5-sonnet-20241022"))
        except Exception as e:
            raise SplittingError(f"Splitting error 2: {str(e)}")

        try:
            # Split document
            result = process.load_file(source_path)\
                .split(classifications, strategy=SplittingStrategy.EAGER)\
                .extract(vision=True) 

I assume the extract() method on the splitter process is strictly for the splitting and classification work.

My question is putting the two together. I may be conflating process.extract and the Extractor() instance for the complex data extraction.

First run the source document to get the classification response, then run the source document into the appropriate complex model extractor as two separate steps?
Or is there a method to inform the Process() object of the complex model for post-classification extraction?

[enoch3712]

Ok, checking this one!

[mophilly]

Don’t worry about the index error. I am not sure I can replicate it at this point.
The question for this thread is the use of Process and Extractor. I see Process used for classification in the docs, and I have it working in my project. Likewise, I have Extractor working in a separate routine that does not involve Process.

I would like to better understand the purpose and application of Process. Is it limited to classification?