Skip to content

feat: implement document embedding CLI command with comprehensive chunking and OpenAI integration #138

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
priyankeshh wants to merge 18 commits into
base: develop
Choose a base branch
from
Open

feat: implement document embedding CLI command with comprehensive chunking and OpenAI integration #138

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
0a77a9d
feat: implement documents embed CLI command
priyankeshh Aug 24, 2025
400b067
demo: add comprehensive demonstration of embed command functionality
priyankeshh Aug 24, 2025
93c7d02
refactor: organize test files and create integration testing suite
priyankeshh Aug 24, 2025
2f63419
docs: add comprehensive testing documentation and instructions
priyankeshh Aug 24, 2025
b5b1d33
test: complete end-to-end validation of embed functionality with real…
priyankeshh Aug 24, 2025
b06f6df
feat: implement configurable embedding system with random vectors
priyankeshh Aug 24, 2025
8fc7d08
feat: enhance embedding functionality and improve document processing
JonnyTran Aug 24, 2025
b1ffa82
Apply ruff formatting
priyankeshh Aug 27, 2025
95dd534
Add PyMuPDF integration with document metadata in extralit-server
priyankeshh Aug 27, 2025
dcf53e0
fix: address review feedback - proper dataset creation, remove test f…
priyankeshh Aug 27, 2025
8138efd
refactor: minimize code and remove llama-index dependency
priyankeshh Aug 29, 2025
7a765d2
style: apply pre-commit formatting
priyankeshh Aug 29, 2025
d875219
fix: simplify RQ job to use HF space service for margin lookup
priyankeshh Aug 29, 2025
eb415ee
fix: remove redundant PyMuPDF job definition from extralit-server
priyankeshh Sep 1, 2025
1b39f1f
fix: simplify table context to follow existing workflow patterns
priyankeshh Sep 1, 2025
7321381
fix: fetch analysis metadata from document for table extraction
priyankeshh Sep 1, 2025
fd1b9e9
Merge branch 'develop' into feat/document-embedding-cli
priyankeshh Sep 1, 2025
9faff49
chore: remove redundant ocr/tables.py (functionality already in workf…
priyankeshh Sep 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -161,4 +161,5 @@ extralit/site
**/*.db
**/*.pdf
.claude/
output/
output/

14 changes: 0 additions & 14 deletions extralit-server/src/extralit_server/contexts/ocr/tables.py
View file
Open in desktop

This file was deleted.

2 changes: 2 additions & 0 deletions extralit/src/extralit/cli/documents/__main__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@

from extralit.cli.documents.add import add_document
from extralit.cli.documents.delete import delete_document
from extralit.cli.documents.embed import embed_documents
from extralit.cli.documents.import_bib import import_bib
from extralit.cli.documents.import_history import list_import_histories
from extralit.cli.documents.list import list_documents
Expand All @@ -28,6 +29,7 @@
app.command(name="add")(add_document)
app.command(name="import")(import_bib)
app.command(name="delete")(delete_document)
app.command(name="embed")(embed_documents)

# Import history commands - new structure
app.command(name="history")(list_import_histories)
Expand Down
Loading
Loading