Skip to content

PuneetKumar1790/EY

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

15 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

Tender Eligibility Analyzer API

A Node.js backend API that automates tender eligibility analysis using Google Gemini 2.5 Flash AI. The system processes multiple tender PDFs, analyzes company eligibility, performs SKU matching, calculates pricing, and sends automated email notifications with detailed reports.

Features

  • πŸ“„ Extract text from PDF and DOCX documents
  • πŸ€– AI-powered summarization using Google Gemini 2.5 Flash
  • πŸ“Š Comprehensive eligibility analysis and table generation
  • πŸ”„ Advanced 9-step workflow for Tender 2 processing
  • 🎯 SKU matching with intelligent scoring algorithm
  • πŸ’° Automated pricing calculations with real value extraction
  • βœ‰οΈ Automated email notifications with DOCX/CSV reports
  • πŸš€ RESTful API with 6 main endpoints
  • πŸ”‘ Multi-key API rotation (supports up to 4 Gemini API keys)
  • πŸ“¦ Postman collection for easy testing

Prerequisites

  • Node.js (v14 or higher)
  • npm or yarn
  • Google Gemini API keys (1-4 keys recommended for better rate limit handling)
  • SendGrid account with API key (free tier: 100 emails/day)

Installation

  1. Clone or download this project

  2. Install dependencies

    npm install
  3. Set up environment variables

    Create a .env file in the root directory (or copy from .env.example):

    PORT=3000
    GEMINI_API_KEY_1=your_first_gemini_api_key
    GEMINI_API_KEY_2=your_second_gemini_api_key
    GEMINI_API_KEY_3=your_third_gemini_api_key
    GEMINI_API_KEY_4=your_fourth_gemini_api_key
    SENDGRID_API_KEY=your_sendgrid_api_key
    SENDGRID_FROM_EMAIL=your_verified_sender@example.com
    EMAIL_TO=recipient@example.com
    

    API Key Rotation (Aggressive Strategy):

    • The system supports up to 4 Gemini API keys for optimal performance
    • Aggressive rotation: Switches to next key on EVERY error (not just rate limits)
    • Maximum retries: 3x the number of configured keys
    • Automatic exponential backoff with jitter for single-key setups
    • You can use 1-4 keys; more keys = better reliability and throughput

    Important:

    • Get your Gemini API keys from Google AI Studio
    • Configure 2-4 API keys for optimal performance and rate limit handling
    • For SendGrid:
      • Sign up at SendGrid (free tier: 100 emails/day)
      • Create an API key from Settings β†’ API Keys
      • Verify a sender email address from Settings β†’ Sender Authentication
      • Use the API key for SENDGRID_API_KEY and verified email for SENDGRID_FROM_EMAIL
  4. Start the server

    npm start

    Or for development with auto-reload:

    npm run dev

The server will start on http://localhost:3000

API Endpoints

1. Upload Tender PDFs

POST /api/upload-tender/:tenderId

Upload up to 5 tender PDFs. The system will:

  • Extract text from each PDF
  • Generate detailed summaries using AI
  • Save summaries in summaries/tenderId/

Example:

curl -X POST http://localhost:3000/api/upload-tender/tender1 \
  -F "pdfs=@document1.pdf" \
  -F "pdfs=@document2.pdf"

Parameters:

  • tenderId: tender1 or tender2
  • pdfs: Array of PDF files (multipart/form-data)

2. Upload Company Information

POST /api/upload-company

Upload company information document (PDF or DOCX).

Example:

curl -X POST http://localhost:3000/api/upload-company \
  -F "document=@company_info.docx"

Parameters:

  • document: PDF or DOCX file (multipart/form-data)

3. Analyze Tender Eligibility

POST /api/analyze-tender/:tenderId

Analyzes tender eligibility by:

  1. Concatenating all 5 PDF summaries
  2. Cross-referencing with company info
  3. Generating eligibility analysis table

Example:

curl -X POST http://localhost:3000/api/analyze-tender/tender1

Response:

{
  "success": true,
  "message": "Eligibility analysis completed for tender1",
  "tenderId": "tender1",
  "tablePath": "analysis/tender1/table.txt",
  "summary": {
    "totalSummaries": 5,
    "comprehensiveSummaryLength": 15234,
    "companyInfoLength": 8921,
    "eligibilityTableLength": 3456
  }
}

4. Check Eligibility

POST /api/check-eligibility/:tenderId

Checks final eligibility (YES/NO) and:

  • If NO: Generates DOCX report and sends email
  • If YES: Returns eligibility status

Example:

curl -X POST http://localhost:3000/api/check-eligibility/tender1

Response:

{
  "success": true,
  "message": "Company is NOT eligible for tender1",
  "tenderId": "tender1",
  "eligible": false,
  "decision": "NO",
  "emailSent": true,
  "docxPath": "analysis/tender1/eligibility_report.docx"
}

5. Process All Tenders

POST /api/process-all

Automates the entire workflow for both tenders sequentially.

Example:

curl -X POST http://localhost:3000/api/process-all

Response:

{
  "success": true,
  "message": "All tenders processed",
  "summary": {
    "eligible": 0,
    "notEligible": 2,
    "total": 2
  },
  "results": {
    "tender1": {
      "success": true,
      "eligible": false,
      "decision": "NO",
      "emailSent": true
    },
    "tender2": {
      "success": true,
      "eligible": false,
      "decision": "NO",
      "emailSent": true
    }
  }
}

6. Process Tender 2 Workflow (Advanced 9-Step Pipeline)

POST /api/process-tender2-workflow

Complete automated workflow for Tender 2 with advanced features:

  1. Process 5 PDFs β†’ Generate individual summaries
  2. Concatenate summaries β†’ Combine all summaries
  3. Generate comprehensive summary β†’ Create unified analysis
  4. Eligibility check β†’ YES/NO decision
  5. Build Table B1 β†’ Matching operations table (if eligible)
  6. Match SKUs β†’ Intelligent SKU matching with scoring
  7. Calculate pricing β†’ Real value extraction and pricing
  8. Generate holistic table β†’ Combined summary table
  9. Email result β†’ Send appropriate report

Example:

curl -X POST http://localhost:3000/api/process-tender2-workflow

Response (Eligible):

{
  "success": true,
  "message": "Tender 2 workflow completed successfully",
  "eligibility": "YES",
  "emailSent": true,
  "workflowDir": "workflows/tender2",
  "outputs": {
    "summary1": "workflows/tender2/summary_1.txt",
    "comprehensiveSummary": "workflows/tender2/comprehensive_summary.txt",
    "eligibilityResult": "workflows/tender2/eligibility_result.txt",
    "tableB1": "workflows/tender2/table_b1.csv",
    "matchedSKUs": "workflows/tender2/matched_skus.txt",
    "pricingTable": "workflows/tender2/pricing_table.csv",
    "holisticTable": "workflows/tender2/holistic_summary_table.csv"
  }
}

Response (Not Eligible):

{
  "success": true,
  "message": "Tender 2 workflow completed - Company NOT eligible",
  "eligibility": "NO",
  "emailSent": true,
  "docxPath": "analysis/tender2/eligibility_report.docx",
  "workflowDir": "workflows/tender2"
}

Required Input Files:

  • 5 PDFs in Tendor-2/ directory
  • Tender_SKU_Matching_Descriptions.txt (SKU master file)
  • company_profile.json (company information)

Key Features:

  • Conditional execution: Steps 5-9 only run if eligible
  • Real value extraction from documents (no placeholders)
  • CSV table outputs for structured data
  • Intelligent SKU matching with weighted scoring
  • Automated pricing calculations

Testing with Postman

  1. Import postman_collection.json into Postman
  2. Configure environment variables if needed
  3. Start testing the endpoints

Project Structure

project-root/
β”œβ”€β”€ src/
β”‚   β”œβ”€β”€ controllers/
β”‚   β”‚   β”œβ”€β”€ tenderController.js              # Tender PDF upload handling
β”‚   β”‚   β”œβ”€β”€ companyController.js             # Company info upload handling
β”‚   β”‚   β”œβ”€β”€ analysisController.js            # Eligibility analysis logic
β”‚   β”‚   β”œβ”€β”€ eligibilityController.js         # Final eligibility check & email
β”‚   β”‚   └── tender2WorkflowController.js     # 9-step Tender 2 workflow
β”‚   β”œβ”€β”€ services/
β”‚   β”‚   β”œβ”€β”€ pdfService.js                    # PDF text extraction
β”‚   β”‚   β”œβ”€β”€ docxService.js                   # DOCX text extraction
β”‚   β”‚   β”œβ”€β”€ groqService.js                   # Gemini AI integration (all prompts)
β”‚   β”‚   β”œβ”€β”€ emailService.js                  # Email sending with attachments
β”‚   β”‚   └── docService.js                    # DOCX report generation
β”‚   β”œβ”€β”€ utils/
β”‚   β”‚   β”œβ”€β”€ csvUtils.js                      # CSV conversion utilities
β”‚   β”‚   β”œβ”€β”€ distanceUtils.js                 # Distance calculation utilities
β”‚   β”‚   └── docxTableUtils.js                # DOCX table formatting
β”‚   β”œβ”€β”€ routes/
β”‚   β”‚   └── apiRoutes.js                     # API endpoint definitions
β”‚   └── config/
β”‚       └── groqClient.js                    # Gemini client with key rotation
β”œβ”€β”€ uploads/                                  # Uploaded files (auto-generated)
β”‚   β”œβ”€β”€ tender1/
β”‚   β”œβ”€β”€ tender2/
β”‚   └── company/
β”œβ”€β”€ summaries/                                # Generated summaries (auto-generated)
β”‚   β”œβ”€β”€ tender1/
β”‚   └── tender2/
β”œβ”€β”€ analysis/                                 # Analysis results (auto-generated)
β”‚   β”œβ”€β”€ tender1/
β”‚   └── tender2/
β”œβ”€β”€ workflows/                                # Tender 2 workflow outputs (auto-generated)
β”‚   └── tender2/
β”œβ”€β”€ Tendor-1/                                 # Tender 1 input PDFs
β”œβ”€β”€ Tendor-2/                                 # Tender 2 input PDFs
β”œβ”€β”€ server.js                                 # Express server entry point
β”œβ”€β”€ package.json                              # Dependencies and scripts
β”œβ”€β”€ .env                                      # Environment variables (create from template)
β”œβ”€β”€ env.template                              # Environment variables template
β”œβ”€β”€ .gitignore                                # Git ignore rules
β”œβ”€β”€ README.md                                 # Main documentation (this file)
β”œβ”€β”€ TENDER2_WORKFLOW_README.md                # Detailed Tender 2 workflow docs
β”œβ”€β”€ QUICK_START.md                            # Quick start guide
β”œβ”€β”€ INSTALL.md                                # Installation guide
β”œβ”€β”€ POSTMAN_SETUP_GUIDE.md                    # Postman testing guide
β”œβ”€β”€ postman_collection.json                   # Postman API collection
└── Tender_SKU_Matching_Descriptions.txt      # SKU master file for matching

Workflows

Standard Workflow (Tender 1 & 2)

  1. Upload Tender PDFs β†’ System extracts text and generates summaries
  2. Upload Company Info β†’ System extracts and saves text
  3. Analyze Tender β†’ System creates eligibility analysis table
  4. Check Eligibility β†’ System decides YES/NO and emails if NO
  5. Process All β†’ Automated workflow for both tenders

Advanced Tender 2 Workflow (9-Step Pipeline)

  1. Process 5 PDFs β†’ Individual summaries generated
  2. Concatenate Summaries β†’ Combined into single document
  3. Comprehensive Summary β†’ Unified analysis created
  4. Eligibility Check β†’ YES/NO decision made
  5. Build Table B1 β†’ Matching operations table (if YES)
  6. Match SKUs β†’ Intelligent matching with scoring (if YES)
  7. Calculate Pricing β†’ Real value extraction and pricing (if YES)
  8. Generate Holistic Table β†’ Combined summary table (if YES)
  9. Email Result β†’ Appropriate report sent based on eligibility

AI Model

The system uses Google's Gemini 2.5 Flash model with:

  • Ultra-large context window (2M tokens support)
  • Fast and efficient inference
  • Cost-effective for large documents
  • Excellent for summarization, analysis, and complex reasoning tasks
  • Enhanced capabilities over previous versions

Current Model: gemini-2.5-flash (configured in src/config/groqClient.js)

Model Configuration:

  • Default max output tokens: 65,536
  • Adaptive timeout: 7 minutes for all requests
  • Temperature: 0.3-0.7 (depending on task)
  • Safety settings: Configured to BLOCK_NONE for all categories

Note: You can change the model in src/config/groqClient.js line 140.

Email Configuration

Not Eligible (Standard Workflow)

  • Subject: "Tender Eligibility Analysis Result - NOT ELIGIBLE - {TENDER_ID}"
  • Body: "Please find attached the detailed eligibility analysis report."
  • Attachment: DOCX file with eligibility table

Eligible (Tender 2 Workflow)

  • Subject: "Tender 2 - Holistic Summary Table - ELIGIBLE"
  • Body: "Company is eligible for Tender 2. Please find attached the holistic summary table."
  • Attachment: CSV file with holistic summary table

Not Eligible (Tender 2 Workflow)

  • Subject: "Tender 2 Eligibility Analysis Result - NOT ELIGIBLE"
  • Body: "Please find attached the detailed eligibility analysis report."
  • Attachment: DOCX file with eligibility analysis

API Key Rotation System

The system implements an aggressive rotation strategy for optimal reliability:

Key Features

  • Multi-key support: Configure 1-4 Gemini API keys
  • Aggressive rotation: Switches to next key on EVERY error (not just rate limits)
  • Smart retry logic: Maximum retries = 3x number of configured keys
  • Exponential backoff: For single-key setups with jitter
  • Automatic recovery: Seamlessly handles quota exhaustion and rate limits

How It Works

  1. System starts with API key #1
  2. On any error (rate limit, quota, timeout, etc.), immediately rotates to next key
  3. Tracks which keys have been attempted to avoid infinite loops
  4. After trying all keys, throws error with detailed message
  5. Short delay (1 second) after rotation to avoid cascading rate limits

Configuration

GEMINI_API_KEY_1=your_first_key    # Required
GEMINI_API_KEY_2=your_second_key   # Optional
GEMINI_API_KEY_3=your_third_key    # Optional
GEMINI_API_KEY_4=your_fourth_key   # Optional

Recommendation: Use all 4 keys for maximum reliability and throughput.

Error Handling

All endpoints include comprehensive error handling and logging:

  • console.log for successful operations
  • console.error for errors with detailed stack traces
  • Structured JSON error responses
  • Automatic API key rotation on failures
  • Graceful degradation with fallback mechanisms

Dependencies

  • express: Web framework
  • multer: File upload handling
  • @google/generative-ai: Google Gemini AI API integration
  • nodemailer: Email functionality with attachment support
  • pdf-parse: PDF text extraction
  • mammoth: DOCX text extraction
  • docx: Generate DOCX reports with tables
  • fs-extra: Enhanced file system operations
  • dotenv: Environment variables management

Dev Dependencies

  • nodemon: Auto-reload during development

Additional Documentation

For more detailed information, refer to these guides:

Key Features Summary

πŸ”„ Advanced Workflows

  • Standard workflow: Upload β†’ Analyze β†’ Check β†’ Email
  • Tender 2 workflow: 9-step pipeline with SKU matching and pricing

πŸ€– AI-Powered Analysis

  • Comprehensive tender summarization
  • Eligibility criteria extraction and matching
  • Intelligent SKU matching with weighted scoring
  • Real value extraction from documents

πŸ”‘ Robust API Key Management

  • Support for up to 4 Gemini API keys
  • Aggressive rotation on every error
  • Automatic retry with exponential backoff
  • Maximum reliability and throughput

πŸ“Š Data Processing

  • PDF and DOCX text extraction
  • CSV table generation
  • DOCX report creation with formatted tables
  • Real-time distance and pricing calculations

βœ‰οΈ Automated Notifications

  • Email reports for eligible and non-eligible cases
  • DOCX and CSV attachments
  • Customizable email templates

Troubleshooting

Common Issues

API Key Errors:

  • Ensure all API keys are valid and active
  • Check quota limits on Google AI Studio
  • Verify .env file is properly configured

File Upload Issues:

  • Check file size limits (default: no limit set)
  • Ensure directories have write permissions
  • Verify file formats (PDF, DOCX supported)

Email Not Sending:

  • Verify SendGrid API key is correct and active
  • Check sender email is verified in SendGrid dashboard
  • Ensure SENDGRID_API_KEY and SENDGRID_FROM_EMAIL are set correctly
  • Check SendGrid dashboard for delivery logs and errors

Workflow Failures:

  • Check all required input files exist
  • Review console logs for detailed error messages
  • Verify API keys have sufficient quota

Performance Optimization

With 4 API Keys

  • Throughput: ~4x higher than single key
  • Reliability: Automatic failover on errors
  • Rate Limits: Distributed across multiple keys
  • Recommended for: Production environments

With 1 API Key

  • Throughput: Standard rate limits apply
  • Reliability: Exponential backoff on errors
  • Rate Limits: Single key quota
  • Recommended for: Development/testing

Support

For issues or questions:

  • Check console logs for detailed error messages
  • Review output files in respective directories
  • Verify API response messages
  • Check the additional documentation files

Quick Reference

Environment Variables (4 API Keys)

PORT=3000
GEMINI_API_KEY_1=your_first_gemini_api_key
GEMINI_API_KEY_2=your_second_gemini_api_key
GEMINI_API_KEY_3=your_third_gemini_api_key
GEMINI_API_KEY_4=your_fourth_gemini_api_key
SENDGRID_API_KEY=your_sendgrid_api_key
SENDGRID_FROM_EMAIL=your_verified_sender@example.com
EMAIL_TO=recipient@example.com

Quick Start Commands

# Install dependencies
npm install

# Start server
npm start

# Development mode (auto-reload)
npm run dev

# Test Gemini API
curl http://localhost:3000/api/test-gemini

# Health check
curl http://localhost:3000/health

API Endpoints Summary

Endpoint Method Description
/api/upload-tender/:tenderId POST Upload tender PDFs
/api/upload-company POST Upload company info
/api/analyze-tender/:tenderId POST Analyze eligibility
/api/check-eligibility/:tenderId POST Check & email result
/api/process-all POST Process all tenders
/api/process-tender2-workflow POST Run Tender 2 pipeline
/api/test-gemini GET Test Gemini API
/health GET Health check

Directory Structure Quick Reference

uploads/        β†’ Uploaded PDFs and documents
summaries/      β†’ AI-generated summaries
analysis/       β†’ Eligibility analysis results
workflows/      β†’ Tender 2 workflow outputs
Tendor-1/       β†’ Tender 1 input PDFs
Tendor-2/       β†’ Tender 2 input PDFs

License

ISC

About

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors