Paper Processor

Go package for automated evaluation of academic papers using LLM-based criteria

Features

• Process multiple papers with configurable API settings
• Structured evaluation results (accepted/rejected)
• Rate limiting with request delay configuration
• File-based processing (JSON input/output)
• Customizable evaluation criteria
• Robust error handling with failure tracking
• Automatic dump file for failed analyses

Installation

go get gitea.r8z.us/stwhite/paperprocessor

API Reference

ProcessFile

func ProcessFile(inputPath, outputPath, criteriaPath string, config Config) error

Processes papers from input JSON file and writes results to output JSON file

Parameters:

• inputPath: Path to input JSON file containing papers array
• outputPath: Path to write processing results JSON
• criteriaPath: Path to text file with evaluation criteria
• config: Configuration settings for API and processing

Returns:

• error: Processing error or nil if successful

Usage

Basic Configuration

import "gitea.r8z.us/stwhite/paperprocessor"

config := paperprocessor.Config{
    APIEndpoint:  "https://api.llm-provider.com/v1/chat/completions",
    APIKey:       "your-api-key",
    Model:        "llm-model-name",
    RequestDelay: time.Second * 2,
}

File Processing

err := paperprocessor.ProcessFile(
    "input/papers.json",
    "output/results.json",
    "criteria.txt",
    config,
)
if err != nil {
    log.Fatal("Processing failed:", err)
}

Input Formats

Papers JSON

[
    {
        "title": "Paper Title",
        "abstract": "Paper abstract text...",
        "arxiv_id": "1234.56789"
    }
]

Criteria File

Evaluation criteria:
- Relevance to quantum computing
- Novelty of approach
- Technical rigor

Output Example

{
    "accepted": [
        {
            "paper": {
                "title": "Advanced Quantum Computing Methods",
                "abstract": "...",
                "arxiv_id": "2301.12345"
            },
            "decision": "ACCEPT",
            "explanation": "Fulfills all criteria..."
        }
    ],
    "rejected": [
        {
            "paper": {
                "title": "Basic Classical Algorithms",
                "abstract": "...",
                "arxiv_id": "2301.67890"
            },
            "decision": "REJECT",
            "explanation": "Doesn't meet novelty requirements..."
        }
    ],
    "failed": [
        {
            "paper": {
                "title": "Problematic Paper",
                "abstract": "...",
                "arxiv_id": "2301.11111"
            },
            "error": "invalid decision format",
            "output": ""
        }
    ]
}

When papers fail processing, they are added to the "failed" list in the output JSON and also written to a dump.json file for detailed review.

Configuration Options

Parameter	Description	Default
APIEndpoint	LLM API endpoint URL	Required
APIKey	API authentication key	Required
Model	LLM model name	Required
RequestDelay	Delay between API requests	1 second

License

MIT