-
-
Notifications
You must be signed in to change notification settings - Fork 3.1k
feat(fetcher): implement SearchBasedFetcher for ScienceDirect using scopus search API #14551
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
…PI v2 Migrate ScienceDirect from FulltextFetcher-only to also support PagedSearchBasedFetcher interface using Elsevier's API v2 with PUT method. Changes: - Implement PagedSearchBasedFetcher interface for search functionality - Use HTTP PUT method with JSON request body (API v2 native approach) - Add performSearchPaged() for paginated search results - Add JSON request/response parsing for API v2 format - Support pagination via offset/show parameters (max offset: 6000) - Add proper error handling for 401/400 responses - Parse authors, DOI, journal, volume, issue, pages from JSON response The ScienceDirect Search API v1 was deprecated in December 2018. This implementation follows Elsevier's recommended API v2 approach. API docs: https://dev.elsevier.com/
|
Hey @Siva-Sai22! 👋 Thank you for contributing to JabRef! We have automated checks in place, based on which you will soon get feedback if any of them are failing. After all automated checks pass, a maintainer will also review your contribution. Once that happens, you can go through their comments in the "Files changed" tab and act on them, or reply to the conversation if you have further inputs. Please re-check our AI Usage Policy to ensure that your pull request is in line with it. It also contains links to our contribution guide in case of any other doubts related to our contribution workflow. |
koppor
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
According to https://dev.elsevier.com/technical_documentation.html Scopus is not ScienceDirect.
I see two options:
- To avoid confusion,
ScienceDirectshould be renamed toElsevier. But then, we have a "big ball of mud", because the methods in that class are not coupled, but only semnatically correlated - Add new functionality to
Scopus- this should make maintainence easier. This fetcher should be added to the list of fetchers in the "Web Search" prefernces
Please do option 2 to have the fetcher class small.
- API key should renamed from "Scopus" to "Esevier", since it covers both.
Future work: Split up API key configuration and fetcher enablement.
| String response = new BufferedReader(new InputStreamReader(inputStream)) | ||
| .lines() | ||
| .collect(Collectors.joining(OS.NEWLINE)); | ||
|
|
||
| JSONObject jsonObject = new JSONObject(response); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There must be a method to convert an inputstream to a JSON object directly. But maybe, not mit dem JSONObject.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
** JSONObject response = JsonReader.toJsonObject(inputStream);
**
@koppor I think it would be better to take a scopus api key also. |

Closes #14421
Implemented web search capability for ScienceDirect by integrating the Scopus Search API, since the ScienceDirect Search API requires institutional access that most users don't have. Added a new
ScopusQueryTransformerto convert JabRef search queries to Scopus query syntax, and rewrote theScienceDirectclass to implementPagedSearchBasedParserFetcherinterface, enabling paginated search results with proper JSON response parsing, this allows users to search academic literature via the "Web search" feature in JabRef, which was previously not available for ScienceDirect.Steps to test
Mandatory checks
CHANGELOG.mdin a way that is understandable for the average user (if change is visible to the user)